2024-09-17T21:04:37.330 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T21:04:37.348 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T21:04:37.434 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909340 branch: main description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} email: ceph-qa@ceph.com first_in_suite: false job_id: '7909340' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi no_nested_subset: false openstack: - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: main ceph: conf: global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: stupid bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: 08cec6974d86f81dc304360b31113f30b4085bf8 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: extra_packages: - rbd-nbd flavor: default sha1: 08cec6974d86f81dc304360b31113f30b4085bf8 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 5ecc7400dbbb63ce990b663d2a92e87dbc17f30d owner: scheduled_teuthology@teuthology priority: 950 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - osd.0 - osd.1 - osd.2 - - mon.b - mgr.x - osd.3 - osd.4 - osd.5 - - client.0 seed: 4954 sha1: 08cec6974d86f81dc304360b31113f30b4085bf8 sleep_before_teardown: 0 subset: 16/128 suite: rbd suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_5ecc7400dbbb63ce990b663d2a92e87dbc17f30d/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 5ecc7400dbbb63ce990b663d2a92e87dbc17f30d targets: smithi083.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB5Xm7xQCeN0sApjqCR1qm7iDSFBNjCLYO4AuVsxlcrDVdXswXNvgVbPkwaBLIqD4z2jfYlGJVt/zbRsBWzsTic= smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOFo8SkZ//19bJHgQ9jO43IlqeomE47IvSKIiTDoE2SiswX/j/3FUm+vCqaQZxJjMVzQeno3Emldk/bNykvBk+U= smithi184.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG4xlYdCBWyzeSoZ5Bz9fy3xaDFieROqBTYGCsHgO56/QNM1u1CYU/xUQf2xs8O3RQU8GMG18uiHSpbrzmFtNck= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 holebdy: 512 nbd: true ops: 6000 punch_holes: true readbdy: 512 truncbdy: 512 writebdy: 512 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_20:16:19 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T21:04:37.434 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_5ecc7400dbbb63ce990b663d2a92e87dbc17f30d/qa; will attempt to use it 2024-09-17T21:04:37.435 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_5ecc7400dbbb63ce990b663d2a92e87dbc17f30d/qa/tasks 2024-09-17T21:04:37.435 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T21:04:37.437 INFO:teuthology.task.internal:Checking packages... 2024-09-17T21:04:37.457 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '08cec6974d86f81dc304360b31113f30b4085bf8' 2024-09-17T21:04:37.457 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T21:04:37.457 INFO:teuthology.packaging:ref: None 2024-09-17T21:04:37.457 INFO:teuthology.packaging:tag: None 2024-09-17T21:04:37.457 INFO:teuthology.packaging:branch: main 2024-09-17T21:04:37.457 INFO:teuthology.packaging:sha1: 08cec6974d86f81dc304360b31113f30b4085bf8 2024-09-17T21:04:37.457 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=main 2024-09-17T21:04:37.664 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4937-g74984932-1jammy 2024-09-17T21:04:37.665 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T21:04:37.672 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T21:04:37.672 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T21:04:37.677 INFO:teuthology.task.internal:Saving configuration 2024-09-17T21:04:37.691 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T21:04:37.697 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T21:04:37.721 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909340', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 20:59:07.083114', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIa7cUmYTBU52mtiwPjQXqHeOPMFSCC4nDQWMkMu3ItU'} 2024-09-17T21:04:37.744 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909340', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 20:59:07.081504', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIq0Fuf39Ybs5Q0gsErur4DB4IibI2bz+j5pEXQpaXaO'} 2024-09-17T21:04:37.768 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909340', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 20:59:07.079607', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKuRuHMiqBi8uMA+JbM2xPesmfGRRZezx3GlpLJb85jy'} 2024-09-17T21:04:37.768 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T21:04:37.774 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'] 2024-09-17T21:04:37.774 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'] 2024-09-17T21:04:37.774 INFO:teuthology.task.internal:roles: ubuntu@smithi184.front.sepia.ceph.com - ['client.0'] 2024-09-17T21:04:37.774 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T21:04:37.857 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb3e2c65c60>, signals=[15]) 2024-09-17T21:04:37.858 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T21:04:37.864 INFO:teuthology.task.internal:Opening connections... 2024-09-17T21:04:37.864 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi083.front.sepia.ceph.com 2024-09-17T21:04:37.865 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:04:37.945 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2024-09-17T21:04:37.946 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:04:38.023 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T21:04:38.024 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:04:38.101 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T21:04:38.108 DEBUG:teuthology.orchestra.run.smithi083:> uname -m 2024-09-17T21:04:38.113 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2024-09-17T21:04:38.114 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2024-09-17T21:04:38.161 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-17T21:04:38.161 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="Ubuntu" 2024-09-17T21:04:38.161 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="22.04" 2024-09-17T21:04:38.161 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-17T21:04:38.161 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_CODENAME=jammy 2024-09-17T21:04:38.161 INFO:teuthology.orchestra.run.smithi083.stdout:ID=ubuntu 2024-09-17T21:04:38.161 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE=debian 2024-09-17T21:04:38.161 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-17T21:04:38.161 INFO:teuthology.orchestra.run.smithi083.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-17T21:04:38.161 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-17T21:04:38.161 INFO:teuthology.orchestra.run.smithi083.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-17T21:04:38.161 INFO:teuthology.orchestra.run.smithi083.stdout:UBUNTU_CODENAME=jammy 2024-09-17T21:04:38.162 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2024-09-17T21:04:38.194 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2024-09-17T21:04:38.200 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2024-09-17T21:04:38.200 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2024-09-17T21:04:38.246 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-17T21:04:38.246 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="Ubuntu" 2024-09-17T21:04:38.246 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="22.04" 2024-09-17T21:04:38.246 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-17T21:04:38.246 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_CODENAME=jammy 2024-09-17T21:04:38.246 INFO:teuthology.orchestra.run.smithi097.stdout:ID=ubuntu 2024-09-17T21:04:38.246 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE=debian 2024-09-17T21:04:38.247 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-17T21:04:38.247 INFO:teuthology.orchestra.run.smithi097.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-17T21:04:38.247 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-17T21:04:38.247 INFO:teuthology.orchestra.run.smithi097.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-17T21:04:38.247 INFO:teuthology.orchestra.run.smithi097.stdout:UBUNTU_CODENAME=jammy 2024-09-17T21:04:38.247 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2024-09-17T21:04:38.269 DEBUG:teuthology.orchestra.run.smithi184:> uname -m 2024-09-17T21:04:38.274 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64 2024-09-17T21:04:38.274 DEBUG:teuthology.orchestra.run.smithi184:> cat /etc/os-release 2024-09-17T21:04:38.323 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-17T21:04:38.323 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="Ubuntu" 2024-09-17T21:04:38.323 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="22.04" 2024-09-17T21:04:38.323 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-17T21:04:38.323 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_CODENAME=jammy 2024-09-17T21:04:38.323 INFO:teuthology.orchestra.run.smithi184.stdout:ID=ubuntu 2024-09-17T21:04:38.323 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE=debian 2024-09-17T21:04:38.323 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-17T21:04:38.323 INFO:teuthology.orchestra.run.smithi184.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-17T21:04:38.324 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-17T21:04:38.324 INFO:teuthology.orchestra.run.smithi184.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-17T21:04:38.324 INFO:teuthology.orchestra.run.smithi184.stdout:UBUNTU_CODENAME=jammy 2024-09-17T21:04:38.324 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2024-09-17T21:04:38.352 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T21:04:38.362 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T21:04:38.367 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T21:04:38.367 DEBUG:teuthology.orchestra.run.smithi083:> test '!' -e /home/ubuntu/cephtest 2024-09-17T21:04:38.370 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2024-09-17T21:04:38.372 DEBUG:teuthology.orchestra.run.smithi184:> test '!' -e /home/ubuntu/cephtest 2024-09-17T21:04:38.378 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T21:04:38.384 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T21:04:38.384 DEBUG:teuthology.orchestra.run.smithi083:> test -z $(ls -A /var/lib/ceph) 2024-09-17T21:04:38.417 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2024-09-17T21:04:38.420 DEBUG:teuthology.orchestra.run.smithi184:> test -z $(ls -A /var/lib/ceph) 2024-09-17T21:04:38.437 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T21:04:38.542 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T21:04:38.557 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-17T21:04:38.557 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'client.0': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-17T21:04:38.557 DEBUG:teuthology.orchestra.run.smithi083:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T21:04:38.558 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T21:04:38.558 DEBUG:teuthology.orchestra.run.smithi184:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T21:04:38.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:04:38.563 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-09-17T21:04:38.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:04:38.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:04:38.565 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2024-09-17T21:04:38.565 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-09-17T21:04:38.610 INFO:teuthology.orchestra.run.smithi097.stdout:5.15.0-56-generic 2024-09-17T21:04:38.610 INFO:teuthology.task.kernel:Running kernel on smithi097: 5.15.0-56-generic 2024-09-17T21:04:38.610 DEBUG:teuthology.orchestra.run.smithi097:> sudo apt-get clean 2024-09-17T21:04:38.612 INFO:teuthology.orchestra.run.smithi184.stdout:5.15.0-56-generic 2024-09-17T21:04:38.612 INFO:teuthology.task.kernel:Running kernel on smithi184: 5.15.0-56-generic 2024-09-17T21:04:38.612 DEBUG:teuthology.orchestra.run.smithi184:> sudo apt-get clean 2024-09-17T21:04:38.613 INFO:teuthology.orchestra.run.smithi083.stdout:5.15.0-56-generic 2024-09-17T21:04:38.613 INFO:teuthology.task.kernel:Running kernel on smithi083: 5.15.0-56-generic 2024-09-17T21:04:38.613 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-get clean 2024-09-17T21:04:38.746 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-get update 2024-09-17T21:04:38.755 DEBUG:teuthology.orchestra.run.smithi097:> sudo apt-get update 2024-09-17T21:04:38.769 DEBUG:teuthology.orchestra.run.smithi184:> sudo apt-get update 2024-09-17T21:04:38.880 INFO:teuthology.orchestra.run.smithi083.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-17T21:04:38.900 INFO:teuthology.orchestra.run.smithi184.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T21:04:38.923 INFO:teuthology.orchestra.run.smithi184.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-17T21:04:38.985 INFO:teuthology.orchestra.run.smithi184.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-17T21:04:38.996 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T21:04:39.016 INFO:teuthology.orchestra.run.smithi184.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-17T21:04:39.036 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T21:04:39.037 INFO:teuthology.orchestra.run.smithi097.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-17T21:04:39.079 INFO:teuthology.orchestra.run.smithi083.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-17T21:04:39.091 INFO:teuthology.orchestra.run.smithi097.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-17T21:04:39.097 INFO:teuthology.orchestra.run.smithi083.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-17T21:04:39.126 INFO:teuthology.orchestra.run.smithi184.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [701 kB] 2024-09-17T21:04:39.169 INFO:teuthology.orchestra.run.smithi083.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-17T21:04:39.208 INFO:teuthology.orchestra.run.smithi083.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-17T21:04:39.209 INFO:teuthology.orchestra.run.smithi184.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,058 kB] 2024-09-17T21:04:39.210 INFO:teuthology.orchestra.run.smithi083.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-17T21:04:39.210 INFO:teuthology.orchestra.run.smithi083.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-17T21:04:39.231 INFO:teuthology.orchestra.run.smithi083.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-17T21:04:39.231 INFO:teuthology.orchestra.run.smithi083.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-17T21:04:39.236 INFO:teuthology.orchestra.run.smithi083.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-17T21:04:39.236 INFO:teuthology.orchestra.run.smithi083.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [628 kB] 2024-09-17T21:04:39.242 INFO:teuthology.orchestra.run.smithi083.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [903 kB] 2024-09-17T21:04:39.245 INFO:teuthology.orchestra.run.smithi184.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [355 kB] 2024-09-17T21:04:39.247 INFO:teuthology.orchestra.run.smithi184.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-17T21:04:39.248 INFO:teuthology.orchestra.run.smithi184.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,495 kB] 2024-09-17T21:04:39.250 INFO:teuthology.orchestra.run.smithi083.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-17T21:04:39.251 INFO:teuthology.orchestra.run.smithi083.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-17T21:04:39.252 INFO:teuthology.orchestra.run.smithi083.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-17T21:04:39.252 INFO:teuthology.orchestra.run.smithi083.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-17T21:04:39.263 INFO:teuthology.orchestra.run.smithi083.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-17T21:04:39.264 INFO:teuthology.orchestra.run.smithi184.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-17T21:04:39.265 INFO:teuthology.orchestra.run.smithi184.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [429 kB] 2024-09-17T21:04:39.268 INFO:teuthology.orchestra.run.smithi184.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-17T21:04:39.268 INFO:teuthology.orchestra.run.smithi184.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-17T21:04:39.273 INFO:teuthology.orchestra.run.smithi184.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-17T21:04:39.280 INFO:teuthology.orchestra.run.smithi184.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-17T21:04:39.282 INFO:teuthology.orchestra.run.smithi184.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-17T21:04:39.282 INFO:teuthology.orchestra.run.smithi184.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-17T21:04:39.283 INFO:teuthology.orchestra.run.smithi184.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-17T21:04:39.290 INFO:teuthology.orchestra.run.smithi083.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-17T21:04:39.298 INFO:teuthology.orchestra.run.smithi184.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-17T21:04:39.325 INFO:teuthology.orchestra.run.smithi184.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-17T21:04:39.326 INFO:teuthology.orchestra.run.smithi184.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-17T21:04:39.326 INFO:teuthology.orchestra.run.smithi184.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-17T21:04:39.327 INFO:teuthology.orchestra.run.smithi184.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-17T21:04:39.327 INFO:teuthology.orchestra.run.smithi184.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-17T21:04:39.327 INFO:teuthology.orchestra.run.smithi184.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-17T21:04:39.328 INFO:teuthology.orchestra.run.smithi184.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-17T21:04:39.328 INFO:teuthology.orchestra.run.smithi184.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-17T21:04:39.328 INFO:teuthology.orchestra.run.smithi184.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-17T21:04:39.435 INFO:teuthology.orchestra.run.smithi083.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-17T21:04:39.445 INFO:teuthology.orchestra.run.smithi097.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-17T21:04:39.471 INFO:teuthology.orchestra.run.smithi184.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-17T21:04:39.473 INFO:teuthology.orchestra.run.smithi097.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-17T21:04:39.572 INFO:teuthology.orchestra.run.smithi083.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [701 kB] 2024-09-17T21:04:39.583 INFO:teuthology.orchestra.run.smithi097.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,058 kB] 2024-09-17T21:04:39.767 INFO:teuthology.orchestra.run.smithi083.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,058 kB] 2024-09-17T21:04:39.894 INFO:teuthology.orchestra.run.smithi184.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-17T21:04:39.903 INFO:teuthology.orchestra.run.smithi097.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [701 kB] 2024-09-17T21:04:39.910 INFO:teuthology.orchestra.run.smithi184.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-17T21:04:39.915 INFO:teuthology.orchestra.run.smithi097.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-17T21:04:39.920 INFO:teuthology.orchestra.run.smithi184.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-17T21:04:39.920 INFO:teuthology.orchestra.run.smithi184.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-17T21:04:39.921 INFO:teuthology.orchestra.run.smithi184.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-17T21:04:39.926 INFO:teuthology.orchestra.run.smithi083.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [355 kB] 2024-09-17T21:04:39.936 INFO:teuthology.orchestra.run.smithi097.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-17T21:04:39.938 INFO:teuthology.orchestra.run.smithi097.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [355 kB] 2024-09-17T21:04:39.941 INFO:teuthology.orchestra.run.smithi083.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-17T21:04:39.941 INFO:teuthology.orchestra.run.smithi083.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,495 kB] 2024-09-17T21:04:39.946 INFO:teuthology.orchestra.run.smithi097.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-17T21:04:39.947 INFO:teuthology.orchestra.run.smithi097.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-17T21:04:39.949 INFO:teuthology.orchestra.run.smithi097.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-17T21:04:39.952 INFO:teuthology.orchestra.run.smithi097.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-17T21:04:39.953 INFO:teuthology.orchestra.run.smithi097.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,495 kB] 2024-09-17T21:04:40.011 INFO:teuthology.orchestra.run.smithi083.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-17T21:04:40.012 INFO:teuthology.orchestra.run.smithi083.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [429 kB] 2024-09-17T21:04:40.017 INFO:teuthology.orchestra.run.smithi184.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-17T21:04:40.023 INFO:teuthology.orchestra.run.smithi097.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-17T21:04:40.027 INFO:teuthology.orchestra.run.smithi097.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [429 kB] 2024-09-17T21:04:40.028 INFO:teuthology.orchestra.run.smithi083.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-17T21:04:40.029 INFO:teuthology.orchestra.run.smithi083.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-17T21:04:40.029 INFO:teuthology.orchestra.run.smithi184.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-17T21:04:40.029 INFO:teuthology.orchestra.run.smithi184.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [628 kB] 2024-09-17T21:04:40.041 INFO:teuthology.orchestra.run.smithi097.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-17T21:04:40.044 INFO:teuthology.orchestra.run.smithi184.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [903 kB] 2024-09-17T21:04:40.051 INFO:teuthology.orchestra.run.smithi083.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-17T21:04:40.062 INFO:teuthology.orchestra.run.smithi097.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-17T21:04:40.063 INFO:teuthology.orchestra.run.smithi097.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-17T21:04:40.063 INFO:teuthology.orchestra.run.smithi097.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [903 kB] 2024-09-17T21:04:40.063 INFO:teuthology.orchestra.run.smithi097.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-17T21:04:40.073 INFO:teuthology.orchestra.run.smithi184.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-17T21:04:40.076 INFO:teuthology.orchestra.run.smithi097.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-17T21:04:40.078 INFO:teuthology.orchestra.run.smithi097.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [628 kB] 2024-09-17T21:04:40.079 INFO:teuthology.orchestra.run.smithi083.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-17T21:04:40.079 INFO:teuthology.orchestra.run.smithi184.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-17T21:04:40.080 INFO:teuthology.orchestra.run.smithi184.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-17T21:04:40.080 INFO:teuthology.orchestra.run.smithi184.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-17T21:04:40.081 INFO:teuthology.orchestra.run.smithi184.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-17T21:04:40.088 INFO:teuthology.orchestra.run.smithi083.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-17T21:04:40.091 INFO:teuthology.orchestra.run.smithi083.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-17T21:04:40.091 INFO:teuthology.orchestra.run.smithi083.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-17T21:04:40.095 INFO:teuthology.orchestra.run.smithi097.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-17T21:04:40.099 INFO:teuthology.orchestra.run.smithi097.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-17T21:04:40.099 INFO:teuthology.orchestra.run.smithi097.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-17T21:04:40.099 INFO:teuthology.orchestra.run.smithi097.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-17T21:04:40.102 INFO:teuthology.orchestra.run.smithi097.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-17T21:04:40.102 INFO:teuthology.orchestra.run.smithi097.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-17T21:04:40.103 INFO:teuthology.orchestra.run.smithi097.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-17T21:04:40.103 INFO:teuthology.orchestra.run.smithi097.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-17T21:04:40.104 INFO:teuthology.orchestra.run.smithi097.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-17T21:04:40.149 INFO:teuthology.orchestra.run.smithi097.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-17T21:04:40.149 INFO:teuthology.orchestra.run.smithi097.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-17T21:04:40.150 INFO:teuthology.orchestra.run.smithi097.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-17T21:04:40.151 INFO:teuthology.orchestra.run.smithi097.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-17T21:04:40.155 INFO:teuthology.orchestra.run.smithi184.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-17T21:04:40.161 INFO:teuthology.orchestra.run.smithi083.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-17T21:04:40.162 INFO:teuthology.orchestra.run.smithi083.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-17T21:04:40.162 INFO:teuthology.orchestra.run.smithi083.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-17T21:04:40.163 INFO:teuthology.orchestra.run.smithi083.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-17T21:04:40.164 INFO:teuthology.orchestra.run.smithi083.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-17T21:04:40.172 INFO:teuthology.orchestra.run.smithi083.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-17T21:04:40.173 INFO:teuthology.orchestra.run.smithi083.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-17T21:04:40.173 INFO:teuthology.orchestra.run.smithi083.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-17T21:04:40.182 INFO:teuthology.orchestra.run.smithi097.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-17T21:04:40.184 INFO:teuthology.orchestra.run.smithi097.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-17T21:04:40.185 INFO:teuthology.orchestra.run.smithi097.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-17T21:04:40.185 INFO:teuthology.orchestra.run.smithi097.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-17T21:04:40.186 INFO:teuthology.orchestra.run.smithi097.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-17T21:04:40.186 INFO:teuthology.orchestra.run.smithi097.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-17T21:04:40.230 INFO:teuthology.orchestra.run.smithi097.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-17T21:04:40.253 INFO:teuthology.orchestra.run.smithi083.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-17T21:04:40.254 INFO:teuthology.orchestra.run.smithi083.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-17T21:04:45.631 INFO:teuthology.orchestra.run.smithi184.stdout:Fetched 16.2 MB in 2s (7,561 kB/s) 2024-09-17T21:04:45.639 INFO:teuthology.orchestra.run.smithi083.stdout:Fetched 16.2 MB in 2s (7,523 kB/s) 2024-09-17T21:04:46.119 INFO:teuthology.orchestra.run.smithi097.stdout:Fetched 16.2 MB in 3s (5,757 kB/s) 2024-09-17T21:04:47.091 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-09-17T21:04:47.095 INFO:teuthology.orchestra.run.smithi184.stdout:Reading package lists... 2024-09-17T21:04:47.110 DEBUG:teuthology.orchestra.run.smithi083:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-17T21:04:47.113 DEBUG:teuthology.orchestra.run.smithi184:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-17T21:04:47.173 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-09-17T21:04:47.175 INFO:teuthology.orchestra.run.smithi184.stdout:Reading package lists... 2024-09-17T21:04:47.382 INFO:teuthology.orchestra.run.smithi083.stdout:Building dependency tree... 2024-09-17T21:04:47.382 INFO:teuthology.orchestra.run.smithi083.stdout:Reading state information... 2024-09-17T21:04:47.387 INFO:teuthology.orchestra.run.smithi184.stdout:Building dependency tree... 2024-09-17T21:04:47.388 INFO:teuthology.orchestra.run.smithi184.stdout:Reading state information... 2024-09-17T21:04:47.548 INFO:teuthology.orchestra.run.smithi083.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T21:04:47.549 INFO:teuthology.orchestra.run.smithi083.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T21:04:47.549 INFO:teuthology.orchestra.run.smithi083.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T21:04:47.549 INFO:teuthology.orchestra.run.smithi083.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T21:04:47.549 INFO:teuthology.orchestra.run.smithi083.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-17T21:04:47.549 INFO:teuthology.orchestra.run.smithi083.stdout: python2.7-minimal 2024-09-17T21:04:47.549 INFO:teuthology.orchestra.run.smithi083.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T21:04:47.550 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2024-09-17T21:04:47.550 INFO:teuthology.orchestra.run.smithi083.stdout:The following additional packages will be installed: 2024-09-17T21:04:47.550 INFO:teuthology.orchestra.run.smithi083.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-17T21:04:47.551 INFO:teuthology.orchestra.run.smithi083.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-17T21:04:47.551 INFO:teuthology.orchestra.run.smithi083.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-17T21:04:47.552 INFO:teuthology.orchestra.run.smithi083.stdout:Suggested packages: 2024-09-17T21:04:47.552 INFO:teuthology.orchestra.run.smithi083.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-17T21:04:47.552 INFO:teuthology.orchestra.run.smithi083.stdout:Recommended packages: 2024-09-17T21:04:47.552 INFO:teuthology.orchestra.run.smithi083.stdout: thermald 2024-09-17T21:04:47.568 DEBUG:teuthology.orchestra.run.smithi097:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-17T21:04:47.581 INFO:teuthology.orchestra.run.smithi184.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T21:04:47.581 INFO:teuthology.orchestra.run.smithi184.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T21:04:47.581 INFO:teuthology.orchestra.run.smithi184.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T21:04:47.582 INFO:teuthology.orchestra.run.smithi184.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T21:04:47.582 INFO:teuthology.orchestra.run.smithi184.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-17T21:04:47.582 INFO:teuthology.orchestra.run.smithi184.stdout: python2.7-minimal 2024-09-17T21:04:47.582 INFO:teuthology.orchestra.run.smithi184.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T21:04:47.583 INFO:teuthology.orchestra.run.smithi184.stdout:The following additional packages will be installed: 2024-09-17T21:04:47.584 INFO:teuthology.orchestra.run.smithi184.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-17T21:04:47.584 INFO:teuthology.orchestra.run.smithi184.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-17T21:04:47.584 INFO:teuthology.orchestra.run.smithi184.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-17T21:04:47.585 INFO:teuthology.orchestra.run.smithi184.stdout:Suggested packages: 2024-09-17T21:04:47.585 INFO:teuthology.orchestra.run.smithi184.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-17T21:04:47.586 INFO:teuthology.orchestra.run.smithi184.stdout:Recommended packages: 2024-09-17T21:04:47.586 INFO:teuthology.orchestra.run.smithi184.stdout: thermald 2024-09-17T21:04:47.597 INFO:teuthology.orchestra.run.smithi083.stdout:The following NEW packages will be installed: 2024-09-17T21:04:47.598 INFO:teuthology.orchestra.run.smithi083.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-17T21:04:47.598 INFO:teuthology.orchestra.run.smithi083.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-17T21:04:47.598 INFO:teuthology.orchestra.run.smithi083.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-17T21:04:47.599 INFO:teuthology.orchestra.run.smithi083.stdout:The following packages will be upgraded: 2024-09-17T21:04:47.600 INFO:teuthology.orchestra.run.smithi083.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-17T21:04:47.622 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2024-09-17T21:04:47.632 INFO:teuthology.orchestra.run.smithi184.stdout:The following NEW packages will be installed: 2024-09-17T21:04:47.632 INFO:teuthology.orchestra.run.smithi184.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-17T21:04:47.632 INFO:teuthology.orchestra.run.smithi184.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-17T21:04:47.632 INFO:teuthology.orchestra.run.smithi184.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-17T21:04:47.634 INFO:teuthology.orchestra.run.smithi184.stdout:The following packages will be upgraded: 2024-09-17T21:04:47.634 INFO:teuthology.orchestra.run.smithi184.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-17T21:04:47.651 INFO:teuthology.orchestra.run.smithi083.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-17T21:04:47.651 INFO:teuthology.orchestra.run.smithi083.stdout:Need to get 113 MB of archives. 2024-09-17T21:04:47.651 INFO:teuthology.orchestra.run.smithi083.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-17T21:04:47.651 INFO:teuthology.orchestra.run.smithi083.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-17T21:04:47.810 INFO:teuthology.orchestra.run.smithi097.stdout:Building dependency tree... 2024-09-17T21:04:47.810 INFO:teuthology.orchestra.run.smithi097.stdout:Reading state information... 2024-09-17T21:04:47.815 INFO:teuthology.orchestra.run.smithi184.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-17T21:04:47.815 INFO:teuthology.orchestra.run.smithi184.stdout:Need to get 113 MB of archives. 2024-09-17T21:04:47.815 INFO:teuthology.orchestra.run.smithi184.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-17T21:04:47.815 INFO:teuthology.orchestra.run.smithi184.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-17T21:04:47.977 INFO:teuthology.orchestra.run.smithi097.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T21:04:47.977 INFO:teuthology.orchestra.run.smithi097.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T21:04:47.978 INFO:teuthology.orchestra.run.smithi097.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T21:04:47.978 INFO:teuthology.orchestra.run.smithi097.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T21:04:47.978 INFO:teuthology.orchestra.run.smithi097.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-17T21:04:47.978 INFO:teuthology.orchestra.run.smithi097.stdout: python2.7-minimal 2024-09-17T21:04:47.978 INFO:teuthology.orchestra.run.smithi097.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T21:04:47.979 INFO:teuthology.orchestra.run.smithi097.stdout:The following additional packages will be installed: 2024-09-17T21:04:47.980 INFO:teuthology.orchestra.run.smithi097.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-17T21:04:47.980 INFO:teuthology.orchestra.run.smithi097.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-17T21:04:47.980 INFO:teuthology.orchestra.run.smithi097.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-17T21:04:47.981 INFO:teuthology.orchestra.run.smithi097.stdout:Suggested packages: 2024-09-17T21:04:47.981 INFO:teuthology.orchestra.run.smithi097.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-17T21:04:47.981 INFO:teuthology.orchestra.run.smithi097.stdout:Recommended packages: 2024-09-17T21:04:47.981 INFO:teuthology.orchestra.run.smithi097.stdout: thermald 2024-09-17T21:04:48.018 INFO:teuthology.orchestra.run.smithi097.stdout:The following NEW packages will be installed: 2024-09-17T21:04:48.018 INFO:teuthology.orchestra.run.smithi097.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-17T21:04:48.019 INFO:teuthology.orchestra.run.smithi097.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-17T21:04:48.019 INFO:teuthology.orchestra.run.smithi097.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-17T21:04:48.020 INFO:teuthology.orchestra.run.smithi097.stdout:The following packages will be upgraded: 2024-09-17T21:04:48.021 INFO:teuthology.orchestra.run.smithi097.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-17T21:04:48.095 INFO:teuthology.orchestra.run.smithi083.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-17T21:04:48.193 INFO:teuthology.orchestra.run.smithi083.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-17T21:04:48.200 INFO:teuthology.orchestra.run.smithi097.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-17T21:04:48.200 INFO:teuthology.orchestra.run.smithi097.stdout:Need to get 113 MB of archives. 2024-09-17T21:04:48.200 INFO:teuthology.orchestra.run.smithi097.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-17T21:04:48.200 INFO:teuthology.orchestra.run.smithi097.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-17T21:04:49.021 INFO:teuthology.orchestra.run.smithi083.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-17T21:04:49.021 INFO:teuthology.orchestra.run.smithi083.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-17T21:04:49.021 INFO:teuthology.orchestra.run.smithi083.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-17T21:04:49.145 INFO:teuthology.orchestra.run.smithi083.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-17T21:04:49.166 INFO:teuthology.orchestra.run.smithi083.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-17T21:04:49.217 INFO:teuthology.orchestra.run.smithi184.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-17T21:04:49.548 INFO:teuthology.orchestra.run.smithi083.stdout:Fetched 113 MB in 2s (72.7 MB/s) 2024-09-17T21:04:49.562 INFO:teuthology.orchestra.run.smithi184.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-17T21:04:49.707 INFO:teuthology.orchestra.run.smithi097.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-17T21:04:49.722 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-17T21:04:50.058 INFO:teuthology.orchestra.run.smithi097.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-17T21:04:51.708 INFO:teuthology.orchestra.run.smithi184.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-17T21:04:51.708 INFO:teuthology.orchestra.run.smithi184.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-17T21:04:51.816 INFO:teuthology.orchestra.run.smithi184.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-17T21:04:52.108 INFO:teuthology.orchestra.run.smithi083.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-17T21:04:52.112 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T21:04:52.284 INFO:teuthology.orchestra.run.smithi097.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-17T21:04:52.284 INFO:teuthology.orchestra.run.smithi097.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-17T21:04:52.285 INFO:teuthology.orchestra.run.smithi097.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-17T21:04:52.354 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:04:52.630 INFO:teuthology.orchestra.run.smithi097.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-17T21:04:52.711 INFO:teuthology.orchestra.run.smithi097.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-17T21:04:52.891 INFO:teuthology.orchestra.run.smithi184.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-17T21:04:53.033 INFO:teuthology.orchestra.run.smithi184.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-17T21:04:53.115 INFO:teuthology.orchestra.run.smithi097.stdout:Fetched 113 MB in 5s (24.2 MB/s) 2024-09-17T21:04:53.296 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-17T21:04:53.450 INFO:teuthology.orchestra.run.smithi184.stdout:Fetched 113 MB in 5s (21.0 MB/s) 2024-09-17T21:04:53.663 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-17T21:04:54.363 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-17T21:04:54.384 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T21:04:54.481 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:04:54.949 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-17T21:04:54.959 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T21:04:54.999 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:04:55.248 INFO:teuthology.orchestra.run.smithi097.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-17T21:04:55.252 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T21:04:55.319 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:04:55.693 INFO:teuthology.orchestra.run.smithi184.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-17T21:04:55.698 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T21:04:55.814 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:04:57.504 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-17T21:04:57.525 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T21:04:57.616 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:04:58.074 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-17T21:04:58.094 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T21:04:58.132 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:04:58.145 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-17T21:04:58.167 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T21:04:58.249 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:04:58.724 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-17T21:04:58.745 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T21:04:58.782 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:05:01.759 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-17T21:05:01.896 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-17T21:05:02.211 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-17T21:05:02.332 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-17T21:05:02.644 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-17T21:05:02.666 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-17T21:05:02.710 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-17T21:05:04.683 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-17T21:05:04.786 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-17T21:05:05.127 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-17T21:05:05.248 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-17T21:05:05.513 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-17T21:05:05.542 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-17T21:05:05.553 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-17T21:05:05.592 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-17T21:05:05.653 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-17T21:05:05.952 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-17T21:05:06.098 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-17T21:05:06.392 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-17T21:05:06.415 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-17T21:05:06.476 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-17T21:05:07.972 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-17T21:05:08.010 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T21:05:08.055 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:05:10.755 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-17T21:05:10.879 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-17T21:05:11.203 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-17T21:05:11.316 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:05:11.425 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-17T21:05:11.483 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-17T21:05:11.521 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T21:05:11.546 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:05:11.566 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:05:12.801 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-17T21:05:12.838 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T21:05:12.865 INFO:teuthology.orchestra.run.smithi083.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-17T21:05:12.865 INFO:teuthology.orchestra.run.smithi083.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-17T21:05:12.865 INFO:teuthology.orchestra.run.smithi083.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-17T21:05:12.865 INFO:teuthology.orchestra.run.smithi083.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-17T21:05:12.875 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:05:13.024 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:05:14.157 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-17T21:05:14.270 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-17T21:05:15.150 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-17T21:05:15.359 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-17T21:05:15.460 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:05:15.580 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-17T21:05:15.676 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:05:15.802 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-17T21:05:15.931 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:05:16.588 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-17T21:05:16.710 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-17T21:05:16.829 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:05:16.891 INFO:teuthology.orchestra.run.smithi083.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-17T21:05:16.891 INFO:teuthology.orchestra.run.smithi083.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-17T21:05:17.077 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-17T21:05:17.198 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:05:17.247 INFO:teuthology.orchestra.run.smithi097.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-17T21:05:17.247 INFO:teuthology.orchestra.run.smithi097.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-17T21:05:17.247 INFO:teuthology.orchestra.run.smithi097.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-17T21:05:17.247 INFO:teuthology.orchestra.run.smithi097.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-17T21:05:17.341 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-17T21:05:17.407 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:05:17.478 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:05:18.792 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-17T21:05:18.830 INFO:teuthology.orchestra.run.smithi184.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-17T21:05:18.830 INFO:teuthology.orchestra.run.smithi184.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-17T21:05:18.830 INFO:teuthology.orchestra.run.smithi184.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-17T21:05:18.831 INFO:teuthology.orchestra.run.smithi184.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-17T21:05:18.902 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-17T21:05:19.028 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:05:19.037 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:05:20.405 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:05:21.024 INFO:teuthology.orchestra.run.smithi097.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-17T21:05:21.024 INFO:teuthology.orchestra.run.smithi097.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-17T21:05:21.480 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-17T21:05:21.606 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-17T21:05:21.732 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:05:23.176 INFO:teuthology.orchestra.run.smithi184.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T21:05:23.236 INFO:teuthology.orchestra.run.smithi184.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-17T21:05:23.236 INFO:teuthology.orchestra.run.smithi184.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-17T21:05:31.426 INFO:teuthology.orchestra.run.smithi083.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-17T21:05:31.426 INFO:teuthology.orchestra.run.smithi083.stdout:Sourcing file `/etc/default/grub' 2024-09-17T21:05:31.437 INFO:teuthology.orchestra.run.smithi083.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T21:05:31.467 INFO:teuthology.orchestra.run.smithi083.stdout:Generating grub configuration file ... 2024-09-17T21:05:31.719 INFO:teuthology.orchestra.run.smithi083.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-17T21:05:31.753 INFO:teuthology.orchestra.run.smithi083.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-17T21:05:32.030 INFO:teuthology.orchestra.run.smithi083.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T21:05:32.035 INFO:teuthology.orchestra.run.smithi083.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T21:05:32.107 INFO:teuthology.orchestra.run.smithi083.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T21:05:32.113 INFO:teuthology.orchestra.run.smithi083.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T21:05:32.274 INFO:teuthology.orchestra.run.smithi083.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T21:05:32.275 INFO:teuthology.orchestra.run.smithi083.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T21:05:32.275 INFO:teuthology.orchestra.run.smithi083.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T21:05:32.339 INFO:teuthology.orchestra.run.smithi083.stdout:done 2024-09-17T21:05:32.868 DEBUG:teuthology.orchestra.run.smithi083:> dpkg -s linux-image-generic 2024-09-17T21:05:32.885 INFO:teuthology.orchestra.run.smithi083.stdout:Package: linux-image-generic 2024-09-17T21:05:32.886 INFO:teuthology.orchestra.run.smithi083.stdout:Status: install ok installed 2024-09-17T21:05:32.886 INFO:teuthology.orchestra.run.smithi083.stdout:Priority: optional 2024-09-17T21:05:32.886 INFO:teuthology.orchestra.run.smithi083.stdout:Section: kernel 2024-09-17T21:05:32.886 INFO:teuthology.orchestra.run.smithi083.stdout:Installed-Size: 21 2024-09-17T21:05:32.886 INFO:teuthology.orchestra.run.smithi083.stdout:Maintainer: Ubuntu Kernel Team 2024-09-17T21:05:32.886 INFO:teuthology.orchestra.run.smithi083.stdout:Architecture: amd64 2024-09-17T21:05:32.886 INFO:teuthology.orchestra.run.smithi083.stdout:Source: linux-meta 2024-09-17T21:05:32.886 INFO:teuthology.orchestra.run.smithi083.stdout:Version: 5.15.0.122.122 2024-09-17T21:05:32.886 INFO:teuthology.orchestra.run.smithi083.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-17T21:05:32.886 INFO:teuthology.orchestra.run.smithi083.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-17T21:05:32.886 INFO:teuthology.orchestra.run.smithi083.stdout:Recommends: thermald 2024-09-17T21:05:32.887 INFO:teuthology.orchestra.run.smithi083.stdout:Description: Generic Linux kernel image 2024-09-17T21:05:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: This package will always depend on the latest generic kernel image 2024-09-17T21:05:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: available. 2024-09-17T21:05:32.887 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-17T21:05:32.887 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T21:05:32.887 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T21:05:32.887 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-17T21:05:32.888 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi083.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T21:05:32.888 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-get clean 2024-09-17T21:05:33.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-get update 2024-09-17T21:05:33.206 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-17T21:05:33.311 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T21:05:33.412 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-17T21:05:33.473 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-17T21:05:34.660 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-09-17T21:05:34.678 DEBUG:teuthology.orchestra.run.smithi083:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-17T21:05:34.740 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-09-17T21:05:34.950 INFO:teuthology.orchestra.run.smithi083.stdout:Building dependency tree... 2024-09-17T21:05:34.951 INFO:teuthology.orchestra.run.smithi083.stdout:Reading state information... 2024-09-17T21:05:34.965 INFO:teuthology.orchestra.run.smithi097.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-17T21:05:34.965 INFO:teuthology.orchestra.run.smithi097.stdout:Sourcing file `/etc/default/grub' 2024-09-17T21:05:34.975 INFO:teuthology.orchestra.run.smithi097.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T21:05:35.003 INFO:teuthology.orchestra.run.smithi097.stdout:Generating grub configuration file ... 2024-09-17T21:05:35.143 INFO:teuthology.orchestra.run.smithi083.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-17T21:05:35.143 INFO:teuthology.orchestra.run.smithi083.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T21:05:35.144 INFO:teuthology.orchestra.run.smithi083.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T21:05:35.144 INFO:teuthology.orchestra.run.smithi083.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T21:05:35.144 INFO:teuthology.orchestra.run.smithi083.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T21:05:35.144 INFO:teuthology.orchestra.run.smithi083.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T21:05:35.144 INFO:teuthology.orchestra.run.smithi083.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T21:05:35.145 INFO:teuthology.orchestra.run.smithi083.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T21:05:35.186 INFO:teuthology.orchestra.run.smithi083.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-17T21:05:35.188 DEBUG:teuthology.orchestra.run.smithi083:> dpkg -s linux-image-generic 2024-09-17T21:05:35.203 INFO:teuthology.orchestra.run.smithi083.stdout:Package: linux-image-generic 2024-09-17T21:05:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:Status: install ok installed 2024-09-17T21:05:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:Priority: optional 2024-09-17T21:05:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:Section: kernel 2024-09-17T21:05:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:Installed-Size: 21 2024-09-17T21:05:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:Maintainer: Ubuntu Kernel Team 2024-09-17T21:05:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:Architecture: amd64 2024-09-17T21:05:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:Source: linux-meta 2024-09-17T21:05:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:Version: 5.15.0.122.122 2024-09-17T21:05:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-17T21:05:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-17T21:05:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:Recommends: thermald 2024-09-17T21:05:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:Description: Generic Linux kernel image 2024-09-17T21:05:35.204 INFO:teuthology.orchestra.run.smithi083.stdout: This package will always depend on the latest generic kernel image 2024-09-17T21:05:35.204 INFO:teuthology.orchestra.run.smithi083.stdout: available. 2024-09-17T21:05:35.205 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2024-09-17T21:05:35.251 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.pfLaozHIw3 2024-09-17T21:05:35.252 DEBUG:teuthology.orchestra.run.smithi083:> sudo cp /boot/grub/grub.cfg /tmp/tmp.pfLaozHIw3 2024-09-17T21:05:35.277 INFO:teuthology.orchestra.run.smithi097.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-17T21:05:35.292 INFO:teuthology.orchestra.run.smithi097.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-17T21:05:35.306 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.pfLaozHIw3 2024-09-17T21:05:35.422 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.pfLaozHIw3 is 10KB 2024-09-17T21:05:35.434 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.pfLaozHIw3 2024-09-17T21:05:35.439 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-17T21:05:35.491 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T21:05:35.491 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-17T21:05:35.491 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-17T21:05:35.554 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-17T21:05:35.555 DEBUG:teuthology.orchestra.run.smithi083:> sudo update-grub 2024-09-17T21:05:35.575 INFO:teuthology.orchestra.run.smithi097.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T21:05:35.580 INFO:teuthology.orchestra.run.smithi097.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T21:05:35.651 INFO:teuthology.orchestra.run.smithi097.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T21:05:35.657 INFO:teuthology.orchestra.run.smithi097.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T21:05:35.833 INFO:teuthology.orchestra.run.smithi097.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T21:05:35.833 INFO:teuthology.orchestra.run.smithi097.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T21:05:35.833 INFO:teuthology.orchestra.run.smithi097.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T21:05:35.868 INFO:teuthology.orchestra.run.smithi097.stdout:done 2024-09-17T21:05:36.351 DEBUG:teuthology.orchestra.run.smithi097:> dpkg -s linux-image-generic 2024-09-17T21:05:36.373 INFO:teuthology.orchestra.run.smithi097.stdout:Package: linux-image-generic 2024-09-17T21:05:36.374 INFO:teuthology.orchestra.run.smithi097.stdout:Status: install ok installed 2024-09-17T21:05:36.374 INFO:teuthology.orchestra.run.smithi097.stdout:Priority: optional 2024-09-17T21:05:36.374 INFO:teuthology.orchestra.run.smithi097.stdout:Section: kernel 2024-09-17T21:05:36.374 INFO:teuthology.orchestra.run.smithi097.stdout:Installed-Size: 21 2024-09-17T21:05:36.374 INFO:teuthology.orchestra.run.smithi097.stdout:Maintainer: Ubuntu Kernel Team 2024-09-17T21:05:36.374 INFO:teuthology.orchestra.run.smithi097.stdout:Architecture: amd64 2024-09-17T21:05:36.374 INFO:teuthology.orchestra.run.smithi097.stdout:Source: linux-meta 2024-09-17T21:05:36.374 INFO:teuthology.orchestra.run.smithi097.stdout:Version: 5.15.0.122.122 2024-09-17T21:05:36.374 INFO:teuthology.orchestra.run.smithi097.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-17T21:05:36.374 INFO:teuthology.orchestra.run.smithi097.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-17T21:05:36.375 INFO:teuthology.orchestra.run.smithi097.stdout:Recommends: thermald 2024-09-17T21:05:36.375 INFO:teuthology.orchestra.run.smithi097.stdout:Description: Generic Linux kernel image 2024-09-17T21:05:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: This package will always depend on the latest generic kernel image 2024-09-17T21:05:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: available. 2024-09-17T21:05:36.375 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-17T21:05:36.375 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T21:05:36.375 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T21:05:36.375 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-17T21:05:36.375 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi097.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T21:05:36.376 DEBUG:teuthology.orchestra.run.smithi097:> sudo apt-get clean 2024-09-17T21:05:36.571 DEBUG:teuthology.orchestra.run.smithi097:> sudo apt-get update 2024-09-17T21:05:36.773 INFO:teuthology.orchestra.run.smithi083.stderr:Sourcing file `/etc/default/grub' 2024-09-17T21:05:36.775 INFO:teuthology.orchestra.run.smithi083.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T21:05:36.776 INFO:teuthology.orchestra.run.smithi083.stderr:Generating grub configuration file ... 2024-09-17T21:05:36.825 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T21:05:36.825 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-17T21:05:36.890 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-17T21:05:36.959 INFO:teuthology.orchestra.run.smithi083.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-17T21:05:36.965 INFO:teuthology.orchestra.run.smithi083.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-17T21:05:36.972 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-17T21:05:37.204 INFO:teuthology.orchestra.run.smithi083.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T21:05:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T21:05:37.252 INFO:teuthology.orchestra.run.smithi083.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T21:05:37.258 INFO:teuthology.orchestra.run.smithi083.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T21:05:37.386 INFO:teuthology.orchestra.run.smithi083.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T21:05:37.386 INFO:teuthology.orchestra.run.smithi083.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T21:05:37.386 INFO:teuthology.orchestra.run.smithi083.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T21:05:37.405 INFO:teuthology.orchestra.run.smithi083.stderr:done 2024-09-17T21:05:37.416 DEBUG:teuthology.orchestra.run.smithi083:> sudo shutdown -r now 2024-09-17T21:05:37.423 INFO:teuthology.orchestra.run.smithi184.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-17T21:05:37.423 INFO:teuthology.orchestra.run.smithi184.stdout:Sourcing file `/etc/default/grub' 2024-09-17T21:05:37.454 INFO:teuthology.orchestra.run.smithi184.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T21:05:37.475 INFO:teuthology.orchestra.run.smithi184.stdout:Generating grub configuration file ... 2024-09-17T21:05:37.771 INFO:teuthology.orchestra.run.smithi184.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-17T21:05:37.790 INFO:teuthology.orchestra.run.smithi184.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-17T21:05:38.126 INFO:teuthology.orchestra.run.smithi184.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T21:05:38.131 INFO:teuthology.orchestra.run.smithi184.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T21:05:38.193 INFO:teuthology.orchestra.run.smithi184.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T21:05:38.198 INFO:teuthology.orchestra.run.smithi184.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T21:05:38.274 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2024-09-17T21:05:38.293 DEBUG:teuthology.orchestra.run.smithi097:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-17T21:05:38.354 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2024-09-17T21:05:38.372 INFO:teuthology.orchestra.run.smithi184.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T21:05:38.372 INFO:teuthology.orchestra.run.smithi184.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T21:05:38.372 INFO:teuthology.orchestra.run.smithi184.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T21:05:38.415 INFO:teuthology.orchestra.run.smithi184.stdout:done 2024-09-17T21:05:38.566 INFO:teuthology.orchestra.run.smithi097.stdout:Building dependency tree... 2024-09-17T21:05:38.566 INFO:teuthology.orchestra.run.smithi097.stdout:Reading state information... 2024-09-17T21:05:38.746 INFO:teuthology.orchestra.run.smithi097.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-17T21:05:38.746 INFO:teuthology.orchestra.run.smithi097.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T21:05:38.746 INFO:teuthology.orchestra.run.smithi097.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T21:05:38.746 INFO:teuthology.orchestra.run.smithi097.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T21:05:38.746 INFO:teuthology.orchestra.run.smithi097.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T21:05:38.746 INFO:teuthology.orchestra.run.smithi097.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T21:05:38.747 INFO:teuthology.orchestra.run.smithi097.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T21:05:38.747 INFO:teuthology.orchestra.run.smithi097.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T21:05:38.787 INFO:teuthology.orchestra.run.smithi097.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-17T21:05:38.789 DEBUG:teuthology.orchestra.run.smithi097:> dpkg -s linux-image-generic 2024-09-17T21:05:38.804 INFO:teuthology.orchestra.run.smithi097.stdout:Package: linux-image-generic 2024-09-17T21:05:38.805 INFO:teuthology.orchestra.run.smithi097.stdout:Status: install ok installed 2024-09-17T21:05:38.805 INFO:teuthology.orchestra.run.smithi097.stdout:Priority: optional 2024-09-17T21:05:38.805 INFO:teuthology.orchestra.run.smithi097.stdout:Section: kernel 2024-09-17T21:05:38.805 INFO:teuthology.orchestra.run.smithi097.stdout:Installed-Size: 21 2024-09-17T21:05:38.805 INFO:teuthology.orchestra.run.smithi097.stdout:Maintainer: Ubuntu Kernel Team 2024-09-17T21:05:38.805 INFO:teuthology.orchestra.run.smithi097.stdout:Architecture: amd64 2024-09-17T21:05:38.805 INFO:teuthology.orchestra.run.smithi097.stdout:Source: linux-meta 2024-09-17T21:05:38.805 INFO:teuthology.orchestra.run.smithi097.stdout:Version: 5.15.0.122.122 2024-09-17T21:05:38.805 INFO:teuthology.orchestra.run.smithi097.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-17T21:05:38.805 INFO:teuthology.orchestra.run.smithi097.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-17T21:05:38.805 INFO:teuthology.orchestra.run.smithi097.stdout:Recommends: thermald 2024-09-17T21:05:38.805 INFO:teuthology.orchestra.run.smithi097.stdout:Description: Generic Linux kernel image 2024-09-17T21:05:38.806 INFO:teuthology.orchestra.run.smithi097.stdout: This package will always depend on the latest generic kernel image 2024-09-17T21:05:38.806 INFO:teuthology.orchestra.run.smithi097.stdout: available. 2024-09-17T21:05:38.809 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-09-17T21:05:38.853 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.yG5H7yrWVQ 2024-09-17T21:05:38.853 DEBUG:teuthology.orchestra.run.smithi097:> sudo cp /boot/grub/grub.cfg /tmp/tmp.yG5H7yrWVQ 2024-09-17T21:05:38.903 DEBUG:teuthology.orchestra.run.smithi184:> dpkg -s linux-image-generic 2024-09-17T21:05:38.907 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.yG5H7yrWVQ 2024-09-17T21:05:38.930 INFO:teuthology.orchestra.run.smithi184.stdout:Package: linux-image-generic 2024-09-17T21:05:38.930 INFO:teuthology.orchestra.run.smithi184.stdout:Status: install ok installed 2024-09-17T21:05:38.930 INFO:teuthology.orchestra.run.smithi184.stdout:Priority: optional 2024-09-17T21:05:38.931 INFO:teuthology.orchestra.run.smithi184.stdout:Section: kernel 2024-09-17T21:05:38.931 INFO:teuthology.orchestra.run.smithi184.stdout:Installed-Size: 21 2024-09-17T21:05:38.931 INFO:teuthology.orchestra.run.smithi184.stdout:Maintainer: Ubuntu Kernel Team 2024-09-17T21:05:38.931 INFO:teuthology.orchestra.run.smithi184.stdout:Architecture: amd64 2024-09-17T21:05:38.931 INFO:teuthology.orchestra.run.smithi184.stdout:Source: linux-meta 2024-09-17T21:05:38.931 INFO:teuthology.orchestra.run.smithi184.stdout:Version: 5.15.0.122.122 2024-09-17T21:05:38.931 INFO:teuthology.orchestra.run.smithi184.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-17T21:05:38.931 INFO:teuthology.orchestra.run.smithi184.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-17T21:05:38.931 INFO:teuthology.orchestra.run.smithi184.stdout:Recommends: thermald 2024-09-17T21:05:38.931 INFO:teuthology.orchestra.run.smithi184.stdout:Description: Generic Linux kernel image 2024-09-17T21:05:38.931 INFO:teuthology.orchestra.run.smithi184.stdout: This package will always depend on the latest generic kernel image 2024-09-17T21:05:38.931 INFO:teuthology.orchestra.run.smithi184.stdout: available. 2024-09-17T21:05:38.932 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-17T21:05:38.932 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T21:05:38.932 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T21:05:38.932 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-17T21:05:38.933 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi184.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T21:05:38.933 DEBUG:teuthology.orchestra.run.smithi184:> sudo apt-get clean 2024-09-17T21:05:39.035 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.yG5H7yrWVQ is 10KB 2024-09-17T21:05:39.048 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.yG5H7yrWVQ 2024-09-17T21:05:39.054 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-17T21:05:39.107 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-17T21:05:39.107 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-17T21:05:39.107 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-17T21:05:39.132 DEBUG:teuthology.orchestra.run.smithi184:> sudo apt-get update 2024-09-17T21:05:39.180 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-17T21:05:39.180 DEBUG:teuthology.orchestra.run.smithi097:> sudo update-grub 2024-09-17T21:05:39.378 INFO:teuthology.orchestra.run.smithi184.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T21:05:39.378 INFO:teuthology.orchestra.run.smithi184.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-17T21:05:39.450 INFO:teuthology.orchestra.run.smithi184.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-17T21:05:39.532 INFO:teuthology.orchestra.run.smithi184.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-17T21:05:40.427 INFO:teuthology.orchestra.run.smithi097.stderr:Sourcing file `/etc/default/grub' 2024-09-17T21:05:40.428 INFO:teuthology.orchestra.run.smithi097.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T21:05:40.430 INFO:teuthology.orchestra.run.smithi097.stderr:Generating grub configuration file ... 2024-09-17T21:05:40.601 INFO:teuthology.orchestra.run.smithi097.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-17T21:05:40.607 INFO:teuthology.orchestra.run.smithi097.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-17T21:05:40.832 INFO:teuthology.orchestra.run.smithi184.stdout:Reading package lists... 2024-09-17T21:05:40.845 INFO:teuthology.orchestra.run.smithi097.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T21:05:40.851 DEBUG:teuthology.orchestra.run.smithi184:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-17T21:05:40.852 INFO:teuthology.orchestra.run.smithi097.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T21:05:40.892 INFO:teuthology.orchestra.run.smithi097.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T21:05:40.899 INFO:teuthology.orchestra.run.smithi097.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T21:05:40.906 INFO:teuthology.orchestra.run.smithi184.stdout:Reading package lists... 2024-09-17T21:05:41.044 INFO:teuthology.orchestra.run.smithi097.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T21:05:41.044 INFO:teuthology.orchestra.run.smithi097.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T21:05:41.044 INFO:teuthology.orchestra.run.smithi097.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T21:05:41.056 INFO:teuthology.orchestra.run.smithi097.stderr:done 2024-09-17T21:05:41.073 DEBUG:teuthology.orchestra.run.smithi097:> sudo shutdown -r now 2024-09-17T21:05:41.106 INFO:teuthology.orchestra.run.smithi184.stdout:Building dependency tree... 2024-09-17T21:05:41.106 INFO:teuthology.orchestra.run.smithi184.stdout:Reading state information... 2024-09-17T21:05:41.281 INFO:teuthology.orchestra.run.smithi184.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-17T21:05:41.281 INFO:teuthology.orchestra.run.smithi184.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T21:05:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T21:05:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T21:05:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T21:05:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T21:05:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T21:05:41.282 INFO:teuthology.orchestra.run.smithi184.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T21:05:41.320 INFO:teuthology.orchestra.run.smithi184.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-17T21:05:41.322 DEBUG:teuthology.orchestra.run.smithi184:> dpkg -s linux-image-generic 2024-09-17T21:05:41.380 INFO:teuthology.orchestra.run.smithi184.stdout:Package: linux-image-generic 2024-09-17T21:05:41.380 INFO:teuthology.orchestra.run.smithi184.stdout:Status: install ok installed 2024-09-17T21:05:41.381 INFO:teuthology.orchestra.run.smithi184.stdout:Priority: optional 2024-09-17T21:05:41.381 INFO:teuthology.orchestra.run.smithi184.stdout:Section: kernel 2024-09-17T21:05:41.381 INFO:teuthology.orchestra.run.smithi184.stdout:Installed-Size: 21 2024-09-17T21:05:41.381 INFO:teuthology.orchestra.run.smithi184.stdout:Maintainer: Ubuntu Kernel Team 2024-09-17T21:05:41.381 INFO:teuthology.orchestra.run.smithi184.stdout:Architecture: amd64 2024-09-17T21:05:41.381 INFO:teuthology.orchestra.run.smithi184.stdout:Source: linux-meta 2024-09-17T21:05:41.381 INFO:teuthology.orchestra.run.smithi184.stdout:Version: 5.15.0.122.122 2024-09-17T21:05:41.381 INFO:teuthology.orchestra.run.smithi184.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-17T21:05:41.381 INFO:teuthology.orchestra.run.smithi184.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-17T21:05:41.381 INFO:teuthology.orchestra.run.smithi184.stdout:Recommends: thermald 2024-09-17T21:05:41.381 INFO:teuthology.orchestra.run.smithi184.stdout:Description: Generic Linux kernel image 2024-09-17T21:05:41.381 INFO:teuthology.orchestra.run.smithi184.stdout: This package will always depend on the latest generic kernel image 2024-09-17T21:05:41.382 INFO:teuthology.orchestra.run.smithi184.stdout: available. 2024-09-17T21:05:41.382 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2024-09-17T21:05:41.425 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.7FXAzZgEGw 2024-09-17T21:05:41.426 DEBUG:teuthology.orchestra.run.smithi184:> sudo cp /boot/grub/grub.cfg /tmp/tmp.7FXAzZgEGw 2024-09-17T21:05:41.491 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.7FXAzZgEGw 2024-09-17T21:05:41.619 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.7FXAzZgEGw is 10KB 2024-09-17T21:05:41.632 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.7FXAzZgEGw 2024-09-17T21:05:41.638 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-17T21:05:41.694 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T21:05:41.695 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-17T21:05:41.695 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-17T21:05:41.793 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-17T21:05:41.794 DEBUG:teuthology.orchestra.run.smithi184:> sudo update-grub 2024-09-17T21:05:43.129 INFO:teuthology.orchestra.run.smithi184.stderr:Sourcing file `/etc/default/grub' 2024-09-17T21:05:43.130 INFO:teuthology.orchestra.run.smithi184.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T21:05:43.139 INFO:teuthology.orchestra.run.smithi184.stderr:Generating grub configuration file ... 2024-09-17T21:05:43.341 INFO:teuthology.orchestra.run.smithi184.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-17T21:05:43.347 INFO:teuthology.orchestra.run.smithi184.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-17T21:05:43.612 INFO:teuthology.orchestra.run.smithi184.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T21:05:43.618 INFO:teuthology.orchestra.run.smithi184.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T21:05:43.661 INFO:teuthology.orchestra.run.smithi184.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T21:05:43.667 INFO:teuthology.orchestra.run.smithi184.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T21:05:43.815 INFO:teuthology.orchestra.run.smithi184.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T21:05:43.815 INFO:teuthology.orchestra.run.smithi184.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T21:05:43.815 INFO:teuthology.orchestra.run.smithi184.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T21:05:43.826 INFO:teuthology.orchestra.run.smithi184.stderr:done 2024-09-17T21:05:43.828 DEBUG:teuthology.orchestra.run.smithi184:> sudo shutdown -r now 2024-09-17T21:06:07.441 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-17T21:06:07.441 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-17T21:06:07.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:06:11.077 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-17T21:06:11.077 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-17T21:06:11.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:06:13.833 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-17T21:06:13.833 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-17T21:06:13.834 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:06:25.981 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-09-17T21:06:29.565 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-09-17T21:06:34.985 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-17T21:06:34.986 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:06:38.049 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-09-17T21:06:38.567 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-17T21:06:38.567 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:06:41.629 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-09-17T21:06:50.057 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-17T21:06:50.057 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:06:53.633 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-17T21:06:53.633 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:07:08.481 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-09-17T21:07:13.841 DEBUG:teuthology.orchestra.remote:timed out 2024-09-17T21:07:22.843 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-17T21:07:22.844 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:07:22.846 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.184 2024-09-17T21:07:23.483 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-17T21:07:23.484 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:07:23.701 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-09-17T21:07:24.328 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-17T21:07:24.328 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-122-generic"... 2024-09-17T21:07:24.328 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-09-17T21:07:24.374 INFO:teuthology.orchestra.run.smithi083.stdout:5.15.0-122-generic 2024-09-17T21:07:24.374 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-17T21:07:24.374 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T21:07:24.374 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-17T21:07:25.375 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-17T21:07:25.376 DEBUG:teuthology.orchestra.run.smithi083:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T21:07:25.436 INFO:teuthology.orchestra.run.smithi083.stdout:ttyS1 2024-09-17T21:07:25.479 DEBUG:teuthology.parallel:result is None 2024-09-17T21:07:25.610 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-09-17T21:07:26.171 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-17T21:07:26.171 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-122-generic"... 2024-09-17T21:07:26.171 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-09-17T21:07:26.219 INFO:teuthology.orchestra.run.smithi097.stdout:5.15.0-122-generic 2024-09-17T21:07:26.219 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-17T21:07:26.219 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T21:07:26.219 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-17T21:07:27.220 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-17T21:07:27.221 DEBUG:teuthology.orchestra.run.smithi097:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T21:07:27.557 INFO:teuthology.orchestra.run.smithi097.stdout:ttyS1 2024-09-17T21:07:27.572 DEBUG:teuthology.parallel:result is None 2024-09-17T21:07:34.847 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-17T21:07:34.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:07:35.071 DEBUG:teuthology.orchestra.run.smithi184:> true 2024-09-17T21:07:35.631 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-17T21:07:35.631 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.15.0-122-generic"... 2024-09-17T21:07:35.631 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2024-09-17T21:07:35.678 INFO:teuthology.orchestra.run.smithi184.stdout:5.15.0-122-generic 2024-09-17T21:07:35.678 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-17T21:07:35.678 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T21:07:35.678 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-17T21:07:36.679 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2024-09-17T21:07:36.679 DEBUG:teuthology.orchestra.run.smithi184:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T21:07:37.028 INFO:teuthology.orchestra.run.smithi184.stdout:ttyS1 2024-09-17T21:07:37.060 DEBUG:teuthology.parallel:result is None 2024-09-17T21:07:37.060 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T21:07:37.066 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T21:07:37.067 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T21:07:37.069 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T21:07:37.071 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T21:07:37.077 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T21:07:37.083 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T21:07:37.090 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T21:07:37.090 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T21:07:37.115 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T21:07:37.120 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T21:07:37.178 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T21:07:37.185 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T21:07:37.186 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T21:07:37.188 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T21:07:37.190 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T21:07:37.203 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T21:07:37.206 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T21:07:37.207 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T21:07:37.211 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T21:07:37.214 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T21:07:37.215 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T21:07:37.217 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T21:07:37.223 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T21:07:37.223 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T21:07:37.259 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T21:07:37.263 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T21:07:37.291 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T21:07:37.300 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T21:07:37.300 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T21:07:37.335 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T21:07:37.337 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T21:07:37.343 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T21:07:37.388 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T21:07:37.434 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T21:07:37.435 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T21:07:37.525 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T21:07:37.531 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T21:07:37.579 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-17T21:07:37.579 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T21:07:37.644 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T21:07:37.650 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T21:07:37.698 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T21:07:37.698 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T21:07:37.766 DEBUG:teuthology.orchestra.run.smithi083:> sudo service rsyslog restart 2024-09-17T21:07:37.768 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2024-09-17T21:07:37.771 DEBUG:teuthology.orchestra.run.smithi184:> sudo service rsyslog restart 2024-09-17T21:07:37.856 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T21:07:37.863 INFO:teuthology.task.internal:Starting timer... 2024-09-17T21:07:37.864 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T21:07:37.873 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T21:07:37.881 DEBUG:teuthology.task.selinux:Excluding smithi083: OS 'ubuntu' does not support SELinux 2024-09-17T21:07:37.881 DEBUG:teuthology.task.selinux:Excluding smithi097: OS 'ubuntu' does not support SELinux 2024-09-17T21:07:37.881 DEBUG:teuthology.task.selinux:Excluding smithi184: OS 'ubuntu' does not support SELinux 2024-09-17T21:07:37.881 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T21:07:37.881 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-17T21:07:37.881 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T21:07:37.881 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T21:07:37.890 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-17T21:07:37.893 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-17T21:07:38.017 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T21:07:38.027 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T21:07:38.028 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi083.front.sepia.ceph.com,smithi097.front.sepia.ceph.com,smithi184.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T21:15:08.497 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi083.front.sepia.ceph.com'), Remote(name='ubuntu@smithi097.front.sepia.ceph.com'), Remote(name='ubuntu@smithi184.front.sepia.ceph.com')] 2024-09-17T21:15:08.499 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-17T21:15:08.500 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:15:08.581 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-09-17T21:15:08.653 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-17T21:15:08.653 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-17T21:15:08.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:15:08.735 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-09-17T21:15:08.812 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-17T21:15:08.813 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-17T21:15:08.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:15:08.893 DEBUG:teuthology.orchestra.run.smithi184:> true 2024-09-17T21:15:08.972 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-17T21:15:08.973 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T21:15:08.983 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T21:15:08.984 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T21:15:08.984 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T21:15:08.986 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T21:15:08.987 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T21:15:08.989 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T21:15:08.989 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T21:15:09.016 INFO:teuthology.orchestra.run.smithi083.stdout:17 Sep 21:15:09 ntpd[18485]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-17T21:15:09.016 INFO:teuthology.orchestra.run.smithi083.stdout:17 Sep 21:15:09 ntpd[18485]: Command line: ntpd -gq 2024-09-17T21:15:09.017 INFO:teuthology.orchestra.run.smithi083.stdout:17 Sep 21:15:09 ntpd[18485]: ---------------------------------------------------- 2024-09-17T21:15:09.017 INFO:teuthology.orchestra.run.smithi083.stdout:17 Sep 21:15:09 ntpd[18485]: ntp-4 is maintained by Network Time Foundation, 2024-09-17T21:15:09.017 INFO:teuthology.orchestra.run.smithi083.stdout:17 Sep 21:15:09 ntpd[18485]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-17T21:15:09.017 INFO:teuthology.orchestra.run.smithi083.stdout:17 Sep 21:15:09 ntpd[18485]: corporation. Support and training for ntp-4 are 2024-09-17T21:15:09.017 INFO:teuthology.orchestra.run.smithi083.stdout:17 Sep 21:15:09 ntpd[18485]: available at https://www.nwtime.org/support 2024-09-17T21:15:09.017 INFO:teuthology.orchestra.run.smithi083.stdout:17 Sep 21:15:09 ntpd[18485]: ---------------------------------------------------- 2024-09-17T21:15:09.017 INFO:teuthology.orchestra.run.smithi097.stdout:17 Sep 21:15:09 ntpd[18491]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-17T21:15:09.018 INFO:teuthology.orchestra.run.smithi097.stdout:17 Sep 21:15:09 ntpd[18491]: Command line: ntpd -gq 2024-09-17T21:15:09.018 INFO:teuthology.orchestra.run.smithi097.stdout:17 Sep 21:15:09 ntpd[18491]: ---------------------------------------------------- 2024-09-17T21:15:09.018 INFO:teuthology.orchestra.run.smithi097.stdout:17 Sep 21:15:09 ntpd[18491]: ntp-4 is maintained by Network Time Foundation, 2024-09-17T21:15:09.018 INFO:teuthology.orchestra.run.smithi097.stdout:17 Sep 21:15:09 ntpd[18491]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-17T21:15:09.018 INFO:teuthology.orchestra.run.smithi097.stdout:17 Sep 21:15:09 ntpd[18491]: corporation. Support and training for ntp-4 are 2024-09-17T21:15:09.018 INFO:teuthology.orchestra.run.smithi097.stdout:17 Sep 21:15:09 ntpd[18491]: available at https://www.nwtime.org/support 2024-09-17T21:15:09.018 INFO:teuthology.orchestra.run.smithi097.stdout:17 Sep 21:15:09 ntpd[18491]: ---------------------------------------------------- 2024-09-17T21:15:09.018 INFO:teuthology.orchestra.run.smithi083.stdout:17 Sep 21:15:09 ntpd[18485]: proto: precision = 0.058 usec (-24) 2024-09-17T21:15:09.019 INFO:teuthology.orchestra.run.smithi083.stdout:17 Sep 21:15:09 ntpd[18485]: basedate set to 2022-02-04 2024-09-17T21:15:09.019 INFO:teuthology.orchestra.run.smithi083.stdout:17 Sep 21:15:09 ntpd[18485]: gps base set to 2022-02-06 (week 2196) 2024-09-17T21:15:09.019 INFO:teuthology.orchestra.run.smithi083.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T21:15:09.019 INFO:teuthology.orchestra.run.smithi083.stdout:17 Sep 21:15:09 ntpd[18485]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T21:15:09.019 INFO:teuthology.orchestra.run.smithi083.stdout:17 Sep 21:15:09 ntpd[18485]: restrict ::: KOD does nothing without LIMITED. 2024-09-17T21:15:09.020 INFO:teuthology.orchestra.run.smithi083.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-17T21:15:09.020 INFO:teuthology.orchestra.run.smithi083.stdout:17 Sep 21:15:09 ntpd[18485]: Listen and drop on 0 v6wildcard [::]:123 2024-09-17T21:15:09.020 INFO:teuthology.orchestra.run.smithi083.stdout:17 Sep 21:15:09 ntpd[18485]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-17T21:15:09.021 INFO:teuthology.orchestra.run.smithi083.stdout:17 Sep 21:15:09 ntpd[18485]: Listen normally on 2 lo 127.0.0.1:123 2024-09-17T21:15:09.021 INFO:teuthology.orchestra.run.smithi083.stdout:17 Sep 21:15:09 ntpd[18485]: Listen normally on 3 enp3s0f1 172.21.15.83:123 2024-09-17T21:15:09.021 INFO:teuthology.orchestra.run.smithi083.stdout:17 Sep 21:15:09 ntpd[18485]: Listen normally on 4 lo [::1]:123 2024-09-17T21:15:09.021 INFO:teuthology.orchestra.run.smithi083.stdout:17 Sep 21:15:09 ntpd[18485]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:ce91%5]:123 2024-09-17T21:15:09.021 INFO:teuthology.orchestra.run.smithi083.stdout:17 Sep 21:15:09 ntpd[18485]: Listening on routing socket on fd #22 for interface updates 2024-09-17T21:15:09.022 INFO:teuthology.orchestra.run.smithi097.stdout:17 Sep 21:15:09 ntpd[18491]: proto: precision = 0.060 usec (-24) 2024-09-17T21:15:09.022 INFO:teuthology.orchestra.run.smithi097.stdout:17 Sep 21:15:09 ntpd[18491]: basedate set to 2022-02-04 2024-09-17T21:15:09.022 INFO:teuthology.orchestra.run.smithi097.stdout:17 Sep 21:15:09 ntpd[18491]: gps base set to 2022-02-06 (week 2196) 2024-09-17T21:15:09.022 INFO:teuthology.orchestra.run.smithi097.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T21:15:09.022 INFO:teuthology.orchestra.run.smithi097.stdout:17 Sep 21:15:09 ntpd[18491]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T21:15:09.023 INFO:teuthology.orchestra.run.smithi097.stdout:17 Sep 21:15:09 ntpd[18491]: restrict ::: KOD does nothing without LIMITED. 2024-09-17T21:15:09.023 INFO:teuthology.orchestra.run.smithi097.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-17T21:15:09.024 INFO:teuthology.orchestra.run.smithi097.stdout:17 Sep 21:15:09 ntpd[18491]: Listen and drop on 0 v6wildcard [::]:123 2024-09-17T21:15:09.024 INFO:teuthology.orchestra.run.smithi097.stdout:17 Sep 21:15:09 ntpd[18491]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-17T21:15:09.024 INFO:teuthology.orchestra.run.smithi097.stdout:17 Sep 21:15:09 ntpd[18491]: Listen normally on 2 lo 127.0.0.1:123 2024-09-17T21:15:09.024 INFO:teuthology.orchestra.run.smithi097.stdout:17 Sep 21:15:09 ntpd[18491]: Listen normally on 3 enp3s0f1 172.21.15.97:123 2024-09-17T21:15:09.024 INFO:teuthology.orchestra.run.smithi097.stdout:17 Sep 21:15:09 ntpd[18491]: Listen normally on 4 lo [::1]:123 2024-09-17T21:15:09.024 INFO:teuthology.orchestra.run.smithi097.stdout:17 Sep 21:15:09 ntpd[18491]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d805%5]:123 2024-09-17T21:15:09.024 INFO:teuthology.orchestra.run.smithi097.stdout:17 Sep 21:15:09 ntpd[18491]: Listening on routing socket on fd #22 for interface updates 2024-09-17T21:15:09.039 INFO:teuthology.orchestra.run.smithi184.stdout:17 Sep 21:15:09 ntpd[18515]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-17T21:15:09.040 INFO:teuthology.orchestra.run.smithi184.stdout:17 Sep 21:15:09 ntpd[18515]: Command line: ntpd -gq 2024-09-17T21:15:09.040 INFO:teuthology.orchestra.run.smithi184.stdout:17 Sep 21:15:09 ntpd[18515]: ---------------------------------------------------- 2024-09-17T21:15:09.040 INFO:teuthology.orchestra.run.smithi184.stdout:17 Sep 21:15:09 ntpd[18515]: ntp-4 is maintained by Network Time Foundation, 2024-09-17T21:15:09.040 INFO:teuthology.orchestra.run.smithi184.stdout:17 Sep 21:15:09 ntpd[18515]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-17T21:15:09.040 INFO:teuthology.orchestra.run.smithi184.stdout:17 Sep 21:15:09 ntpd[18515]: corporation. Support and training for ntp-4 are 2024-09-17T21:15:09.040 INFO:teuthology.orchestra.run.smithi184.stdout:17 Sep 21:15:09 ntpd[18515]: available at https://www.nwtime.org/support 2024-09-17T21:15:09.040 INFO:teuthology.orchestra.run.smithi184.stdout:17 Sep 21:15:09 ntpd[18515]: ---------------------------------------------------- 2024-09-17T21:15:09.041 INFO:teuthology.orchestra.run.smithi184.stdout:17 Sep 21:15:09 ntpd[18515]: proto: precision = 0.059 usec (-24) 2024-09-17T21:15:09.041 INFO:teuthology.orchestra.run.smithi184.stdout:17 Sep 21:15:09 ntpd[18515]: basedate set to 2022-02-04 2024-09-17T21:15:09.041 INFO:teuthology.orchestra.run.smithi184.stdout:17 Sep 21:15:09 ntpd[18515]: gps base set to 2022-02-06 (week 2196) 2024-09-17T21:15:09.041 INFO:teuthology.orchestra.run.smithi184.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T21:15:09.041 INFO:teuthology.orchestra.run.smithi184.stdout:17 Sep 21:15:09 ntpd[18515]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T21:15:09.042 INFO:teuthology.orchestra.run.smithi184.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-17T21:15:09.043 INFO:teuthology.orchestra.run.smithi184.stdout:17 Sep 21:15:09 ntpd[18515]: restrict ::: KOD does nothing without LIMITED. 2024-09-17T21:15:09.043 INFO:teuthology.orchestra.run.smithi184.stdout:17 Sep 21:15:09 ntpd[18515]: Listen and drop on 0 v6wildcard [::]:123 2024-09-17T21:15:09.043 INFO:teuthology.orchestra.run.smithi184.stdout:17 Sep 21:15:09 ntpd[18515]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-17T21:15:09.043 INFO:teuthology.orchestra.run.smithi184.stdout:17 Sep 21:15:09 ntpd[18515]: Listen normally on 2 lo 127.0.0.1:123 2024-09-17T21:15:09.043 INFO:teuthology.orchestra.run.smithi184.stdout:17 Sep 21:15:09 ntpd[18515]: Listen normally on 3 enp3s0f1 172.21.15.184:123 2024-09-17T21:15:09.043 INFO:teuthology.orchestra.run.smithi184.stdout:17 Sep 21:15:09 ntpd[18515]: Listen normally on 4 lo [::1]:123 2024-09-17T21:15:09.043 INFO:teuthology.orchestra.run.smithi184.stdout:17 Sep 21:15:09 ntpd[18515]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7b8f%5]:123 2024-09-17T21:15:09.043 INFO:teuthology.orchestra.run.smithi184.stdout:17 Sep 21:15:09 ntpd[18515]: Listening on routing socket on fd #22 for interface updates 2024-09-17T21:15:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:10 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:10 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:10 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:10 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:10 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:10 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:10 ntpd[18485]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:10 ntpd[18485]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:10 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:10 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:10 ntpd[18485]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:10.019 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:10 ntpd[18485]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:10 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:10 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:10 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:10.020 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:10 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:10.020 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:10 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:10.020 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:10 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:10.040 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:10 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:10.040 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:10 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:10.040 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:10 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:10.040 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:10 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:10.040 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:10 ntpd[18515]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:10.040 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:10 ntpd[18515]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:11.017 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:11 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:11.017 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:11 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:11.018 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:11 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:11.018 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:11 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:11.018 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:11 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:11.018 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:11 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:11.018 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:11 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:11.018 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:11 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:12.020 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:12 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:12.020 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:12 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:12.020 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:12 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:12.021 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:12 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:12.021 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:12 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:12.021 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:12 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:12.021 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:12 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:12.021 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:12 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:12.021 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:12 ntpd[18485]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:12.021 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:12 ntpd[18485]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:12.022 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:12 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:12.022 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:12 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:12.022 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:12 ntpd[18485]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:12.022 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:12 ntpd[18485]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:12.022 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:12 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:12.022 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:12 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:12.023 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:12 ntpd[18485]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:12.023 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:12 ntpd[18485]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:12.024 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:12 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:12.024 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:12 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:12.024 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:12 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:12.024 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:12 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:12.024 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:12 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:12.024 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:12 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:12.040 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:12 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:12.040 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:12 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:12.040 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:12 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:12.040 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:12 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:12.040 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:12 ntpd[18515]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:12.041 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:12 ntpd[18515]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:13.018 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:13 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:13.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:13 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:13.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:13 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:13.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:13 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:13.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:13 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:13.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:13 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:13.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:13 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:13.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:13 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:13.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:13 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:13.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:13 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:13.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:13 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:13.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:13 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:13.040 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:13 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:13.040 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:13 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:13.040 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:13 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:13.041 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:13 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:13.041 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:13 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:13.041 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:13 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:13.041 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:13 ntpd[18515]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:13.041 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:13 ntpd[18515]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:14.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:14 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:14.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:14 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:14.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:14 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:14.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:14 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:14.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:14 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:14 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:14 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:14 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:14 ntpd[18485]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:14 ntpd[18485]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:14 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:14 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:14 ntpd[18485]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:14 ntpd[18485]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:14 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:14 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:14 ntpd[18485]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:14 ntpd[18485]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:14.021 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:14 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:14.021 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:14 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:14.021 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:14 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:14.021 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:14 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:14.021 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:14 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:14.021 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:14 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:14.021 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:14 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:14.021 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:14 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:14.041 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:14 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:14.041 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:14 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:14.041 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:14 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:14.041 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:14 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:14.041 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:14 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:14.042 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:14 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:14.042 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:14 ntpd[18515]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:14.042 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:14 ntpd[18515]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:14.042 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:14 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:14.042 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:14 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:14.042 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:14 ntpd[18515]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:14.042 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:14 ntpd[18515]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:15.018 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:15 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:15.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:15 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:15.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:15 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:15.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:15 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:15.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:15 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:15.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:15 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:15.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:15 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:15.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:15 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:15.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:15 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:15.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:15 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:15.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:15 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:15.019 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:15 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:15.040 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:15 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:15.040 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:15 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:15.040 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:15 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:15.040 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:15 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:15.041 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:15 ntpd[18515]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:15.041 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:15 ntpd[18515]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:16.017 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:16 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:16.017 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:16 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:16.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:16 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:16.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:16 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:16.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:16 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:16.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:16 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:16.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:16 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:16.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:16 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:16.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:16 ntpd[18485]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:16.018 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:16 ntpd[18485]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:16.018 INFO:teuthology.orchestra.run.smithi083.stdout:17 Sep 21:15:16 ntpd[18485]: ntpd: time slew -0.002743 s 2024-09-17T21:15:16.018 INFO:teuthology.orchestra.run.smithi083.stdout:ntpd: time slew -0.002743s 2024-09-17T21:15:16.019 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:16 ntpd[18485]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-17T21:15:16.019 INFO:teuthology.orchestra.run.smithi083.stderr:17 Sep 21:15:16 ntpd[18485]: can't open /var/log/ntpstats/loopstats.20240917: Permission denied 2024-09-17T21:15:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:16 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:16 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:16 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:16 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:16 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:16 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:16.020 INFO:teuthology.orchestra.run.smithi097.stdout:17 Sep 21:15:16 ntpd[18491]: ntpd: time slew -0.000691 s 2024-09-17T21:15:16.020 INFO:teuthology.orchestra.run.smithi097.stdout:ntpd: time slew -0.000691s 2024-09-17T21:15:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:16 ntpd[18491]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-17T21:15:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:17 Sep 21:15:16 ntpd[18491]: can't open /var/log/ntpstats/loopstats.20240917: Permission denied 2024-09-17T21:15:16.040 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:16 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:16.040 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:16 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:16.041 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:16 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:16.041 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:16 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:16.041 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:16 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:16.041 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:16 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:16.041 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:16 ntpd[18515]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:16.041 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:16 ntpd[18515]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:16.041 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:16 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T21:15:16.041 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:16 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T21:15:16.041 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:16 ntpd[18515]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T21:15:16.041 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:16 ntpd[18515]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T21:15:16.041 INFO:teuthology.orchestra.run.smithi184.stdout:17 Sep 21:15:16 ntpd[18515]: ntpd: time slew -0.001729 s 2024-09-17T21:15:16.042 INFO:teuthology.orchestra.run.smithi184.stdout:ntpd: time slew -0.001729s 2024-09-17T21:15:16.042 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:16 ntpd[18515]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-17T21:15:16.042 INFO:teuthology.orchestra.run.smithi184.stderr:17 Sep 21:15:16 ntpd[18515]: can't open /var/log/ntpstats/loopstats.20240917: Permission denied 2024-09-17T21:15:16.070 INFO:teuthology.orchestra.run.smithi083.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T21:15:16.070 INFO:teuthology.orchestra.run.smithi083.stdout:============================================================================== 2024-09-17T21:15:16.070 INFO:teuthology.orchestra.run.smithi083.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T21:15:16.070 INFO:teuthology.orchestra.run.smithi083.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T21:15:16.070 INFO:teuthology.orchestra.run.smithi083.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T21:15:16.070 INFO:teuthology.orchestra.run.smithi083.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T21:15:16.074 INFO:teuthology.orchestra.run.smithi097.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T21:15:16.074 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================== 2024-09-17T21:15:16.075 INFO:teuthology.orchestra.run.smithi097.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T21:15:16.075 INFO:teuthology.orchestra.run.smithi097.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T21:15:16.075 INFO:teuthology.orchestra.run.smithi097.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T21:15:16.075 INFO:teuthology.orchestra.run.smithi097.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T21:15:16.102 INFO:teuthology.orchestra.run.smithi184.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T21:15:16.103 INFO:teuthology.orchestra.run.smithi184.stdout:============================================================================== 2024-09-17T21:15:16.103 INFO:teuthology.orchestra.run.smithi184.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T21:15:16.103 INFO:teuthology.orchestra.run.smithi184.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T21:15:16.103 INFO:teuthology.orchestra.run.smithi184.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T21:15:16.103 INFO:teuthology.orchestra.run.smithi184.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T21:15:16.103 INFO:teuthology.run_tasks:Running task install... 2024-09-17T21:15:16.111 DEBUG:teuthology.task.install:project ceph 2024-09-17T21:15:16.112 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '08cec6974d86f81dc304360b31113f30b4085bf8'}} 2024-09-17T21:15:16.112 DEBUG:teuthology.task.install:config {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '08cec6974d86f81dc304360b31113f30b4085bf8'} 2024-09-17T21:15:16.112 INFO:teuthology.task.install:Using flavor: default 2024-09-17T21:15:16.118 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-17T21:15:16.118 INFO:teuthology.task.install:extra packages: ['rbd-nbd'] 2024-09-17T21:15:16.118 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-key list | grep Ceph 2024-09-17T21:15:16.119 DEBUG:teuthology.orchestra.run.smithi097:> sudo apt-key list | grep Ceph 2024-09-17T21:15:16.119 DEBUG:teuthology.orchestra.run.smithi184:> sudo apt-key list | grep Ceph 2024-09-17T21:15:16.174 INFO:teuthology.orchestra.run.smithi083.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-17T21:15:16.176 INFO:teuthology.orchestra.run.smithi097.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-17T21:15:16.200 INFO:teuthology.orchestra.run.smithi083.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-17T21:15:16.201 INFO:teuthology.orchestra.run.smithi083.stdout:uid [ unknown] Ceph.com (release key) 2024-09-17T21:15:16.201 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-nbd on remote deb x86_64 2024-09-17T21:15:16.201 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=08cec6974d86f81dc304360b31113f30b4085bf8 2024-09-17T21:15:16.205 INFO:teuthology.orchestra.run.smithi184.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-17T21:15:16.206 INFO:teuthology.orchestra.run.smithi097.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-17T21:15:16.206 INFO:teuthology.orchestra.run.smithi097.stdout:uid [ unknown] Ceph.com (release key) 2024-09-17T21:15:16.206 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-nbd on remote deb x86_64 2024-09-17T21:15:16.207 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=08cec6974d86f81dc304360b31113f30b4085bf8 2024-09-17T21:15:16.244 INFO:teuthology.orchestra.run.smithi184.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-17T21:15:16.244 INFO:teuthology.orchestra.run.smithi184.stdout:uid [ unknown] Ceph.com (release key) 2024-09-17T21:15:16.244 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-nbd on remote deb x86_64 2024-09-17T21:15:16.244 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=08cec6974d86f81dc304360b31113f30b4085bf8 2024-09-17T21:15:16.343 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default/ 2024-09-17T21:15:16.343 INFO:teuthology.task.install.deb:Package version is 19.3.0-4975-g08cec697-1jammy 2024-09-17T21:15:16.351 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default/ 2024-09-17T21:15:16.351 INFO:teuthology.task.install.deb:Package version is 19.3.0-4975-g08cec697-1jammy 2024-09-17T21:15:16.383 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default/ 2024-09-17T21:15:16.383 INFO:teuthology.task.install.deb:Package version is 19.3.0-4975-g08cec697-1jammy 2024-09-17T21:15:16.457 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T21:15:16.457 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-17T21:15:16.471 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-17T21:15:16.471 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-17T21:15:16.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-get update 2024-09-17T21:15:16.486 DEBUG:teuthology.orchestra.run.smithi097:> sudo apt-get update 2024-09-17T21:15:16.491 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T21:15:16.491 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-17T21:15:16.505 DEBUG:teuthology.orchestra.run.smithi184:> sudo apt-get update 2024-09-17T21:15:16.641 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-17T21:15:16.641 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T21:15:16.652 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-17T21:15:16.655 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-17T21:15:16.670 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-17T21:15:16.675 INFO:teuthology.orchestra.run.smithi184.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T21:15:16.685 INFO:teuthology.orchestra.run.smithi184.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-17T21:15:16.693 INFO:teuthology.orchestra.run.smithi083.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy InRelease 2024-09-17T21:15:16.703 INFO:teuthology.orchestra.run.smithi184.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-17T21:15:16.708 INFO:teuthology.orchestra.run.smithi097.stdout:Ign:2 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy InRelease 2024-09-17T21:15:16.714 INFO:teuthology.orchestra.run.smithi083.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy Release [7,005 B] 2024-09-17T21:15:16.729 INFO:teuthology.orchestra.run.smithi184.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy InRelease 2024-09-17T21:15:16.730 INFO:teuthology.orchestra.run.smithi097.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy Release [7,005 B] 2024-09-17T21:15:16.736 INFO:teuthology.orchestra.run.smithi083.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-17T21:15:16.751 INFO:teuthology.orchestra.run.smithi097.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-17T21:15:16.752 INFO:teuthology.orchestra.run.smithi184.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy Release [7,005 B] 2024-09-17T21:15:16.757 INFO:teuthology.orchestra.run.smithi083.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,423 B] 2024-09-17T21:15:16.772 INFO:teuthology.orchestra.run.smithi097.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,423 B] 2024-09-17T21:15:16.774 INFO:teuthology.orchestra.run.smithi184.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-17T21:15:16.777 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T21:15:16.783 INFO:teuthology.orchestra.run.smithi083.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-17T21:15:16.795 INFO:teuthology.orchestra.run.smithi184.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,423 B] 2024-09-17T21:15:16.799 INFO:teuthology.orchestra.run.smithi097.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-17T21:15:16.800 INFO:teuthology.orchestra.run.smithi184.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-17T21:15:16.822 INFO:teuthology.orchestra.run.smithi184.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-17T21:15:16.856 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-17T21:15:16.938 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-17T21:15:16.987 INFO:teuthology.orchestra.run.smithi083.stdout:Fetched 28.6 kB in 0s (86.3 kB/s) 2024-09-17T21:15:17.029 INFO:teuthology.orchestra.run.smithi184.stdout:Fetched 28.6 kB in 0s (84.5 kB/s) 2024-09-17T21:15:17.082 INFO:teuthology.orchestra.run.smithi097.stdout:Fetched 28.6 kB in 0s (69.8 kB/s) 2024-09-17T21:15:17.950 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-09-17T21:15:17.966 DEBUG:teuthology.orchestra.run.smithi083:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4975-g08cec697-1jammy cephadm=19.3.0-4975-g08cec697-1jammy ceph-mds=19.3.0-4975-g08cec697-1jammy ceph-mgr=19.3.0-4975-g08cec697-1jammy ceph-common=19.3.0-4975-g08cec697-1jammy ceph-fuse=19.3.0-4975-g08cec697-1jammy ceph-test=19.3.0-4975-g08cec697-1jammy ceph-volume=19.3.0-4975-g08cec697-1jammy radosgw=19.3.0-4975-g08cec697-1jammy python3-rados=19.3.0-4975-g08cec697-1jammy python3-rgw=19.3.0-4975-g08cec697-1jammy python3-cephfs=19.3.0-4975-g08cec697-1jammy python3-rbd=19.3.0-4975-g08cec697-1jammy libcephfs2=19.3.0-4975-g08cec697-1jammy libcephfs-dev=19.3.0-4975-g08cec697-1jammy librados2=19.3.0-4975-g08cec697-1jammy librbd1=19.3.0-4975-g08cec697-1jammy rbd-fuse=19.3.0-4975-g08cec697-1jammy rbd-nbd=19.3.0-4975-g08cec697-1jammy 2024-09-17T21:15:18.031 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-09-17T21:15:18.075 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2024-09-17T21:15:18.078 INFO:teuthology.orchestra.run.smithi184.stdout:Reading package lists... 2024-09-17T21:15:18.092 DEBUG:teuthology.orchestra.run.smithi097:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4975-g08cec697-1jammy cephadm=19.3.0-4975-g08cec697-1jammy ceph-mds=19.3.0-4975-g08cec697-1jammy ceph-mgr=19.3.0-4975-g08cec697-1jammy ceph-common=19.3.0-4975-g08cec697-1jammy ceph-fuse=19.3.0-4975-g08cec697-1jammy ceph-test=19.3.0-4975-g08cec697-1jammy ceph-volume=19.3.0-4975-g08cec697-1jammy radosgw=19.3.0-4975-g08cec697-1jammy python3-rados=19.3.0-4975-g08cec697-1jammy python3-rgw=19.3.0-4975-g08cec697-1jammy python3-cephfs=19.3.0-4975-g08cec697-1jammy python3-rbd=19.3.0-4975-g08cec697-1jammy libcephfs2=19.3.0-4975-g08cec697-1jammy libcephfs-dev=19.3.0-4975-g08cec697-1jammy librados2=19.3.0-4975-g08cec697-1jammy librbd1=19.3.0-4975-g08cec697-1jammy rbd-fuse=19.3.0-4975-g08cec697-1jammy rbd-nbd=19.3.0-4975-g08cec697-1jammy 2024-09-17T21:15:18.094 DEBUG:teuthology.orchestra.run.smithi184:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4975-g08cec697-1jammy cephadm=19.3.0-4975-g08cec697-1jammy ceph-mds=19.3.0-4975-g08cec697-1jammy ceph-mgr=19.3.0-4975-g08cec697-1jammy ceph-common=19.3.0-4975-g08cec697-1jammy ceph-fuse=19.3.0-4975-g08cec697-1jammy ceph-test=19.3.0-4975-g08cec697-1jammy ceph-volume=19.3.0-4975-g08cec697-1jammy radosgw=19.3.0-4975-g08cec697-1jammy python3-rados=19.3.0-4975-g08cec697-1jammy python3-rgw=19.3.0-4975-g08cec697-1jammy python3-cephfs=19.3.0-4975-g08cec697-1jammy python3-rbd=19.3.0-4975-g08cec697-1jammy libcephfs2=19.3.0-4975-g08cec697-1jammy libcephfs-dev=19.3.0-4975-g08cec697-1jammy librados2=19.3.0-4975-g08cec697-1jammy librbd1=19.3.0-4975-g08cec697-1jammy rbd-fuse=19.3.0-4975-g08cec697-1jammy rbd-nbd=19.3.0-4975-g08cec697-1jammy 2024-09-17T21:15:18.148 INFO:teuthology.orchestra.run.smithi184.stdout:Reading package lists... 2024-09-17T21:15:18.151 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2024-09-17T21:15:18.252 INFO:teuthology.orchestra.run.smithi083.stdout:Building dependency tree... 2024-09-17T21:15:18.253 INFO:teuthology.orchestra.run.smithi083.stdout:Reading state information... 2024-09-17T21:15:18.334 INFO:teuthology.orchestra.run.smithi184.stdout:Building dependency tree... 2024-09-17T21:15:18.334 INFO:teuthology.orchestra.run.smithi184.stdout:Reading state information... 2024-09-17T21:15:18.355 INFO:teuthology.orchestra.run.smithi097.stdout:Building dependency tree... 2024-09-17T21:15:18.356 INFO:teuthology.orchestra.run.smithi097.stdout:Reading state information... 2024-09-17T21:15:18.456 INFO:teuthology.orchestra.run.smithi083.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T21:15:18.456 INFO:teuthology.orchestra.run.smithi083.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T21:15:18.456 INFO:teuthology.orchestra.run.smithi083.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T21:15:18.456 INFO:teuthology.orchestra.run.smithi083.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T21:15:18.457 INFO:teuthology.orchestra.run.smithi083.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T21:15:18.457 INFO:teuthology.orchestra.run.smithi083.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T21:15:18.457 INFO:teuthology.orchestra.run.smithi083.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T21:15:18.457 INFO:teuthology.orchestra.run.smithi083.stdout:The following additional packages will be installed: 2024-09-17T21:15:18.457 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-17T21:15:18.458 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-17T21:15:18.458 INFO:teuthology.orchestra.run.smithi083.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T21:15:18.458 INFO:teuthology.orchestra.run.smithi083.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-17T21:15:18.458 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-17T21:15:18.459 INFO:teuthology.orchestra.run.smithi083.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-17T21:15:18.459 INFO:teuthology.orchestra.run.smithi083.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T21:15:18.459 INFO:teuthology.orchestra.run.smithi083.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T21:15:18.459 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-17T21:15:18.459 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-17T21:15:18.459 INFO:teuthology.orchestra.run.smithi083.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-17T21:15:18.459 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-17T21:15:18.460 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-17T21:15:18.460 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-17T21:15:18.460 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-17T21:15:18.460 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-17T21:15:18.460 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-17T21:15:18.460 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-17T21:15:18.460 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-17T21:15:18.460 INFO:teuthology.orchestra.run.smithi083.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-17T21:15:18.460 INFO:teuthology.orchestra.run.smithi083.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-17T21:15:18.460 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-17T21:15:18.460 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-17T21:15:18.460 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet zip 2024-09-17T21:15:18.461 INFO:teuthology.orchestra.run.smithi083.stdout:Suggested packages: 2024-09-17T21:15:18.461 INFO:teuthology.orchestra.run.smithi083.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-17T21:15:18.461 INFO:teuthology.orchestra.run.smithi083.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-17T21:15:18.461 INFO:teuthology.orchestra.run.smithi083.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-17T21:15:18.461 INFO:teuthology.orchestra.run.smithi083.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-17T21:15:18.461 INFO:teuthology.orchestra.run.smithi083.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-17T21:15:18.461 INFO:teuthology.orchestra.run.smithi083.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-17T21:15:18.461 INFO:teuthology.orchestra.run.smithi083.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-17T21:15:18.461 INFO:teuthology.orchestra.run.smithi083.stdout:Recommended packages: 2024-09-17T21:15:18.461 INFO:teuthology.orchestra.run.smithi083.stdout: btrfs-tools 2024-09-17T21:15:18.521 INFO:teuthology.orchestra.run.smithi184.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T21:15:18.521 INFO:teuthology.orchestra.run.smithi184.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T21:15:18.521 INFO:teuthology.orchestra.run.smithi184.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T21:15:18.522 INFO:teuthology.orchestra.run.smithi184.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T21:15:18.522 INFO:teuthology.orchestra.run.smithi184.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T21:15:18.522 INFO:teuthology.orchestra.run.smithi184.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T21:15:18.522 INFO:teuthology.orchestra.run.smithi184.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T21:15:18.523 INFO:teuthology.orchestra.run.smithi184.stdout:The following additional packages will be installed: 2024-09-17T21:15:18.523 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-17T21:15:18.523 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-17T21:15:18.523 INFO:teuthology.orchestra.run.smithi184.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T21:15:18.523 INFO:teuthology.orchestra.run.smithi184.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-17T21:15:18.523 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-17T21:15:18.524 INFO:teuthology.orchestra.run.smithi184.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-17T21:15:18.525 INFO:teuthology.orchestra.run.smithi184.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T21:15:18.525 INFO:teuthology.orchestra.run.smithi184.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T21:15:18.525 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-17T21:15:18.525 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-17T21:15:18.525 INFO:teuthology.orchestra.run.smithi184.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-17T21:15:18.525 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-17T21:15:18.525 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-17T21:15:18.525 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-17T21:15:18.525 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-17T21:15:18.525 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-17T21:15:18.525 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-17T21:15:18.525 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-17T21:15:18.525 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-17T21:15:18.525 INFO:teuthology.orchestra.run.smithi184.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-17T21:15:18.526 INFO:teuthology.orchestra.run.smithi184.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-17T21:15:18.526 INFO:teuthology.orchestra.run.smithi184.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-17T21:15:18.526 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-17T21:15:18.526 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet zip 2024-09-17T21:15:18.526 INFO:teuthology.orchestra.run.smithi184.stdout:Suggested packages: 2024-09-17T21:15:18.526 INFO:teuthology.orchestra.run.smithi184.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-17T21:15:18.526 INFO:teuthology.orchestra.run.smithi184.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-17T21:15:18.526 INFO:teuthology.orchestra.run.smithi184.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-17T21:15:18.526 INFO:teuthology.orchestra.run.smithi184.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-17T21:15:18.527 INFO:teuthology.orchestra.run.smithi184.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-17T21:15:18.527 INFO:teuthology.orchestra.run.smithi184.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-17T21:15:18.527 INFO:teuthology.orchestra.run.smithi184.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-17T21:15:18.527 INFO:teuthology.orchestra.run.smithi184.stdout:Recommended packages: 2024-09-17T21:15:18.527 INFO:teuthology.orchestra.run.smithi184.stdout: btrfs-tools 2024-09-17T21:15:18.551 INFO:teuthology.orchestra.run.smithi097.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T21:15:18.551 INFO:teuthology.orchestra.run.smithi097.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T21:15:18.551 INFO:teuthology.orchestra.run.smithi097.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T21:15:18.551 INFO:teuthology.orchestra.run.smithi097.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T21:15:18.552 INFO:teuthology.orchestra.run.smithi097.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T21:15:18.552 INFO:teuthology.orchestra.run.smithi097.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T21:15:18.552 INFO:teuthology.orchestra.run.smithi097.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T21:15:18.553 INFO:teuthology.orchestra.run.smithi097.stdout:The following additional packages will be installed: 2024-09-17T21:15:18.553 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-17T21:15:18.553 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-17T21:15:18.553 INFO:teuthology.orchestra.run.smithi097.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T21:15:18.554 INFO:teuthology.orchestra.run.smithi097.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-17T21:15:18.554 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-17T21:15:18.555 INFO:teuthology.orchestra.run.smithi097.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-17T21:15:18.555 INFO:teuthology.orchestra.run.smithi097.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T21:15:18.555 INFO:teuthology.orchestra.run.smithi097.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T21:15:18.555 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-17T21:15:18.555 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-17T21:15:18.555 INFO:teuthology.orchestra.run.smithi097.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-17T21:15:18.555 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-17T21:15:18.555 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-17T21:15:18.555 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-17T21:15:18.555 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-17T21:15:18.555 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-17T21:15:18.556 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-17T21:15:18.556 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-17T21:15:18.556 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-17T21:15:18.556 INFO:teuthology.orchestra.run.smithi097.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-17T21:15:18.556 INFO:teuthology.orchestra.run.smithi097.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-17T21:15:18.556 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-17T21:15:18.556 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-17T21:15:18.556 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet zip 2024-09-17T21:15:18.556 INFO:teuthology.orchestra.run.smithi097.stdout:Suggested packages: 2024-09-17T21:15:18.557 INFO:teuthology.orchestra.run.smithi097.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-17T21:15:18.557 INFO:teuthology.orchestra.run.smithi097.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-17T21:15:18.557 INFO:teuthology.orchestra.run.smithi097.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-17T21:15:18.557 INFO:teuthology.orchestra.run.smithi097.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-17T21:15:18.557 INFO:teuthology.orchestra.run.smithi097.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-17T21:15:18.557 INFO:teuthology.orchestra.run.smithi097.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-17T21:15:18.557 INFO:teuthology.orchestra.run.smithi097.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-17T21:15:18.557 INFO:teuthology.orchestra.run.smithi097.stdout:Recommended packages: 2024-09-17T21:15:18.557 INFO:teuthology.orchestra.run.smithi097.stdout: btrfs-tools 2024-09-17T21:15:18.564 INFO:teuthology.orchestra.run.smithi083.stdout:The following NEW packages will be installed: 2024-09-17T21:15:18.564 INFO:teuthology.orchestra.run.smithi083.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-17T21:15:18.564 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-17T21:15:18.564 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-17T21:15:18.566 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-17T21:15:18.566 INFO:teuthology.orchestra.run.smithi083.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-17T21:15:18.566 INFO:teuthology.orchestra.run.smithi083.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-17T21:15:18.566 INFO:teuthology.orchestra.run.smithi083.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-17T21:15:18.566 INFO:teuthology.orchestra.run.smithi083.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-17T21:15:18.566 INFO:teuthology.orchestra.run.smithi083.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-17T21:15:18.566 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-17T21:15:18.566 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-17T21:15:18.566 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-17T21:15:18.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-17T21:15:18.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-17T21:15:18.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-17T21:15:18.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-17T21:15:18.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-17T21:15:18.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-17T21:15:18.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-17T21:15:18.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-17T21:15:18.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-17T21:15:18.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-17T21:15:18.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-17T21:15:18.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-17T21:15:18.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-17T21:15:18.567 INFO:teuthology.orchestra.run.smithi083.stdout: radosgw rbd-fuse rbd-nbd socat unzip xmlstarlet zip 2024-09-17T21:15:18.626 INFO:teuthology.orchestra.run.smithi083.stdout:0 upgraded, 120 newly installed, 0 to remove and 331 not upgraded. 2024-09-17T21:15:18.626 INFO:teuthology.orchestra.run.smithi083.stdout:Need to get 244 MB of archives. 2024-09-17T21:15:18.626 INFO:teuthology.orchestra.run.smithi083.stdout:After this operation, 909 MB of additional disk space will be used. 2024-09-17T21:15:18.626 INFO:teuthology.orchestra.run.smithi083.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-17T21:15:18.629 INFO:teuthology.orchestra.run.smithi184.stdout:The following NEW packages will be installed: 2024-09-17T21:15:18.629 INFO:teuthology.orchestra.run.smithi184.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-17T21:15:18.630 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-17T21:15:18.630 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-17T21:15:18.631 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-17T21:15:18.631 INFO:teuthology.orchestra.run.smithi184.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-17T21:15:18.631 INFO:teuthology.orchestra.run.smithi184.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-17T21:15:18.631 INFO:teuthology.orchestra.run.smithi184.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-17T21:15:18.631 INFO:teuthology.orchestra.run.smithi184.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-17T21:15:18.631 INFO:teuthology.orchestra.run.smithi184.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-17T21:15:18.631 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-17T21:15:18.631 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-17T21:15:18.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-17T21:15:18.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-17T21:15:18.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-17T21:15:18.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-17T21:15:18.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-17T21:15:18.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-17T21:15:18.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-17T21:15:18.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-17T21:15:18.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-17T21:15:18.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-17T21:15:18.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-17T21:15:18.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-17T21:15:18.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-17T21:15:18.633 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-17T21:15:18.633 INFO:teuthology.orchestra.run.smithi184.stdout: radosgw rbd-fuse rbd-nbd socat unzip xmlstarlet zip 2024-09-17T21:15:18.660 INFO:teuthology.orchestra.run.smithi097.stdout:The following NEW packages will be installed: 2024-09-17T21:15:18.660 INFO:teuthology.orchestra.run.smithi097.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-17T21:15:18.660 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-17T21:15:18.660 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-17T21:15:18.661 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-17T21:15:18.661 INFO:teuthology.orchestra.run.smithi097.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-17T21:15:18.661 INFO:teuthology.orchestra.run.smithi097.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-17T21:15:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-17T21:15:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-17T21:15:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-17T21:15:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-17T21:15:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-17T21:15:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-17T21:15:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-17T21:15:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-17T21:15:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-17T21:15:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-17T21:15:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-17T21:15:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-17T21:15:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-17T21:15:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-17T21:15:18.663 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-17T21:15:18.663 INFO:teuthology.orchestra.run.smithi097.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-17T21:15:18.663 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-17T21:15:18.663 INFO:teuthology.orchestra.run.smithi097.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-17T21:15:18.663 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-17T21:15:18.663 INFO:teuthology.orchestra.run.smithi097.stdout: radosgw rbd-fuse rbd-nbd socat unzip xmlstarlet zip 2024-09-17T21:15:18.709 INFO:teuthology.orchestra.run.smithi083.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4975-g08cec697-1jammy [3,634 kB] 2024-09-17T21:15:18.715 INFO:teuthology.orchestra.run.smithi083.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-17T21:15:18.717 INFO:teuthology.orchestra.run.smithi083.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-17T21:15:18.735 INFO:teuthology.orchestra.run.smithi083.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-17T21:15:18.772 INFO:teuthology.orchestra.run.smithi184.stdout:0 upgraded, 120 newly installed, 0 to remove and 331 not upgraded. 2024-09-17T21:15:18.772 INFO:teuthology.orchestra.run.smithi184.stdout:Need to get 244 MB of archives. 2024-09-17T21:15:18.772 INFO:teuthology.orchestra.run.smithi184.stdout:After this operation, 909 MB of additional disk space will be used. 2024-09-17T21:15:18.772 INFO:teuthology.orchestra.run.smithi184.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4975-g08cec697-1jammy [3,634 kB] 2024-09-17T21:15:18.793 INFO:teuthology.orchestra.run.smithi083.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-17T21:15:18.798 INFO:teuthology.orchestra.run.smithi083.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-17T21:15:18.803 INFO:teuthology.orchestra.run.smithi097.stdout:0 upgraded, 120 newly installed, 0 to remove and 331 not upgraded. 2024-09-17T21:15:18.804 INFO:teuthology.orchestra.run.smithi097.stdout:Need to get 244 MB of archives. 2024-09-17T21:15:18.804 INFO:teuthology.orchestra.run.smithi097.stdout:After this operation, 909 MB of additional disk space will be used. 2024-09-17T21:15:18.804 INFO:teuthology.orchestra.run.smithi097.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4975-g08cec697-1jammy [3,634 kB] 2024-09-17T21:15:18.810 INFO:teuthology.orchestra.run.smithi083.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-17T21:15:18.814 INFO:teuthology.orchestra.run.smithi083.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-17T21:15:18.816 INFO:teuthology.orchestra.run.smithi083.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-17T21:15:18.816 INFO:teuthology.orchestra.run.smithi083.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-17T21:15:18.817 INFO:teuthology.orchestra.run.smithi083.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-17T21:15:18.818 INFO:teuthology.orchestra.run.smithi184.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-17T21:15:18.825 INFO:teuthology.orchestra.run.smithi083.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-17T21:15:18.828 INFO:teuthology.orchestra.run.smithi083.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-17T21:15:18.830 INFO:teuthology.orchestra.run.smithi083.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-17T21:15:18.832 INFO:teuthology.orchestra.run.smithi083.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-17T21:15:18.833 INFO:teuthology.orchestra.run.smithi083.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-17T21:15:18.849 INFO:teuthology.orchestra.run.smithi097.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-17T21:15:18.850 INFO:teuthology.orchestra.run.smithi083.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-17T21:15:18.852 INFO:teuthology.orchestra.run.smithi083.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-17T21:15:18.854 INFO:teuthology.orchestra.run.smithi083.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-17T21:15:18.854 INFO:teuthology.orchestra.run.smithi083.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-17T21:15:18.855 INFO:teuthology.orchestra.run.smithi083.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-17T21:15:18.856 INFO:teuthology.orchestra.run.smithi083.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-17T21:15:18.856 INFO:teuthology.orchestra.run.smithi083.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-17T21:15:18.857 INFO:teuthology.orchestra.run.smithi083.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-17T21:15:18.866 INFO:teuthology.orchestra.run.smithi083.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-17T21:15:18.867 INFO:teuthology.orchestra.run.smithi083.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-17T21:15:18.884 INFO:teuthology.orchestra.run.smithi083.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-17T21:15:18.884 INFO:teuthology.orchestra.run.smithi083.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-17T21:15:18.887 INFO:teuthology.orchestra.run.smithi083.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-17T21:15:18.887 INFO:teuthology.orchestra.run.smithi083.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-17T21:15:18.888 INFO:teuthology.orchestra.run.smithi083.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-17T21:15:18.888 INFO:teuthology.orchestra.run.smithi083.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-17T21:15:18.889 INFO:teuthology.orchestra.run.smithi083.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-17T21:15:18.889 INFO:teuthology.orchestra.run.smithi083.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-17T21:15:18.901 INFO:teuthology.orchestra.run.smithi083.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-17T21:15:18.902 INFO:teuthology.orchestra.run.smithi083.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-17T21:15:18.912 INFO:teuthology.orchestra.run.smithi083.stdout:Get:38 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4975-g08cec697-1jammy [3,334 kB] 2024-09-17T21:15:18.918 INFO:teuthology.orchestra.run.smithi083.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-17T21:15:18.919 INFO:teuthology.orchestra.run.smithi083.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-17T21:15:18.924 INFO:teuthology.orchestra.run.smithi083.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-17T21:15:18.924 INFO:teuthology.orchestra.run.smithi083.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-17T21:15:18.925 INFO:teuthology.orchestra.run.smithi083.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-17T21:15:18.926 INFO:teuthology.orchestra.run.smithi083.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-17T21:15:18.927 INFO:teuthology.orchestra.run.smithi083.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-17T21:15:18.929 INFO:teuthology.orchestra.run.smithi083.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-17T21:15:18.935 INFO:teuthology.orchestra.run.smithi083.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-17T21:15:18.937 INFO:teuthology.orchestra.run.smithi083.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-17T21:15:18.947 INFO:teuthology.orchestra.run.smithi083.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-17T21:15:18.949 INFO:teuthology.orchestra.run.smithi083.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-17T21:15:18.950 INFO:teuthology.orchestra.run.smithi083.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-17T21:15:18.951 INFO:teuthology.orchestra.run.smithi083.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-17T21:15:18.953 INFO:teuthology.orchestra.run.smithi083.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-17T21:15:19.006 INFO:teuthology.orchestra.run.smithi083.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-17T21:15:19.006 INFO:teuthology.orchestra.run.smithi083.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-17T21:15:19.008 INFO:teuthology.orchestra.run.smithi083.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-17T21:15:19.008 INFO:teuthology.orchestra.run.smithi083.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-17T21:15:19.009 INFO:teuthology.orchestra.run.smithi083.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-17T21:15:19.026 INFO:teuthology.orchestra.run.smithi083.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-17T21:15:19.026 INFO:teuthology.orchestra.run.smithi083.stdout:Get:60 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4975-g08cec697-1jammy [722 kB] 2024-09-17T21:15:19.027 INFO:teuthology.orchestra.run.smithi184.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4975-g08cec697-1jammy [3,334 kB] 2024-09-17T21:15:19.028 INFO:teuthology.orchestra.run.smithi083.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-17T21:15:19.028 INFO:teuthology.orchestra.run.smithi083.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-17T21:15:19.043 INFO:teuthology.orchestra.run.smithi083.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-17T21:15:19.043 INFO:teuthology.orchestra.run.smithi083.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-17T21:15:19.043 INFO:teuthology.orchestra.run.smithi083.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-17T21:15:19.043 INFO:teuthology.orchestra.run.smithi083.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-17T21:15:19.044 INFO:teuthology.orchestra.run.smithi083.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-17T21:15:19.044 INFO:teuthology.orchestra.run.smithi083.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-17T21:15:19.059 INFO:teuthology.orchestra.run.smithi083.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-17T21:15:19.059 INFO:teuthology.orchestra.run.smithi083.stdout:Get:70 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4975-g08cec697-1jammy [356 kB] 2024-09-17T21:15:19.059 INFO:teuthology.orchestra.run.smithi083.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-17T21:15:19.062 INFO:teuthology.orchestra.run.smithi083.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-17T21:15:19.076 INFO:teuthology.orchestra.run.smithi083.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-17T21:15:19.078 INFO:teuthology.orchestra.run.smithi083.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-17T21:15:19.078 INFO:teuthology.orchestra.run.smithi083.stdout:Get:75 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4975-g08cec697-1jammy [32.8 kB] 2024-09-17T21:15:19.078 INFO:teuthology.orchestra.run.smithi083.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-17T21:15:19.078 INFO:teuthology.orchestra.run.smithi083.stdout:Get:77 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4975-g08cec697-1jammy [180 kB] 2024-09-17T21:15:19.079 INFO:teuthology.orchestra.run.smithi097.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4975-g08cec697-1jammy [3,334 kB] 2024-09-17T21:15:19.081 INFO:teuthology.orchestra.run.smithi083.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-17T21:15:19.083 INFO:teuthology.orchestra.run.smithi083.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-17T21:15:19.084 INFO:teuthology.orchestra.run.smithi083.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-17T21:15:19.084 INFO:teuthology.orchestra.run.smithi083.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-17T21:15:19.090 INFO:teuthology.orchestra.run.smithi083.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4975-g08cec697-1jammy [71.3 kB] 2024-09-17T21:15:19.093 INFO:teuthology.orchestra.run.smithi083.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-17T21:15:19.094 INFO:teuthology.orchestra.run.smithi083.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-17T21:15:19.095 INFO:teuthology.orchestra.run.smithi083.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4975-g08cec697-1jammy [338 kB] 2024-09-17T21:15:19.096 INFO:teuthology.orchestra.run.smithi083.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-17T21:15:19.102 INFO:teuthology.orchestra.run.smithi083.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4975-g08cec697-1jammy [7,966 kB] 2024-09-17T21:15:19.110 INFO:teuthology.orchestra.run.smithi083.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-17T21:15:19.111 INFO:teuthology.orchestra.run.smithi083.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-17T21:15:19.111 INFO:teuthology.orchestra.run.smithi083.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-17T21:15:19.113 INFO:teuthology.orchestra.run.smithi083.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-17T21:15:19.113 INFO:teuthology.orchestra.run.smithi083.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-17T21:15:19.119 INFO:teuthology.orchestra.run.smithi083.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-17T21:15:19.119 INFO:teuthology.orchestra.run.smithi083.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-17T21:15:19.127 INFO:teuthology.orchestra.run.smithi083.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-17T21:15:19.129 INFO:teuthology.orchestra.run.smithi083.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-17T21:15:19.130 INFO:teuthology.orchestra.run.smithi083.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-17T21:15:19.167 INFO:teuthology.orchestra.run.smithi184.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4975-g08cec697-1jammy [722 kB] 2024-09-17T21:15:19.210 INFO:teuthology.orchestra.run.smithi184.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4975-g08cec697-1jammy [356 kB] 2024-09-17T21:15:19.236 INFO:teuthology.orchestra.run.smithi097.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4975-g08cec697-1jammy [722 kB] 2024-09-17T21:15:19.238 INFO:teuthology.orchestra.run.smithi184.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-17T21:15:19.248 INFO:teuthology.orchestra.run.smithi184.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4975-g08cec697-1jammy [32.8 kB] 2024-09-17T21:15:19.249 INFO:teuthology.orchestra.run.smithi184.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4975-g08cec697-1jammy [180 kB] 2024-09-17T21:15:19.251 INFO:teuthology.orchestra.run.smithi184.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-17T21:15:19.254 INFO:teuthology.orchestra.run.smithi184.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4975-g08cec697-1jammy [71.3 kB] 2024-09-17T21:15:19.256 INFO:teuthology.orchestra.run.smithi184.stdout:Get:11 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4975-g08cec697-1jammy [338 kB] 2024-09-17T21:15:19.262 INFO:teuthology.orchestra.run.smithi184.stdout:Get:12 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4975-g08cec697-1jammy [7,966 kB] 2024-09-17T21:15:19.272 INFO:teuthology.orchestra.run.smithi097.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-17T21:15:19.282 INFO:teuthology.orchestra.run.smithi097.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4975-g08cec697-1jammy [356 kB] 2024-09-17T21:15:19.285 INFO:teuthology.orchestra.run.smithi097.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-17T21:15:19.324 INFO:teuthology.orchestra.run.smithi097.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4975-g08cec697-1jammy [32.8 kB] 2024-09-17T21:15:19.325 INFO:teuthology.orchestra.run.smithi097.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4975-g08cec697-1jammy [180 kB] 2024-09-17T21:15:19.329 INFO:teuthology.orchestra.run.smithi097.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4975-g08cec697-1jammy [71.3 kB] 2024-09-17T21:15:19.330 INFO:teuthology.orchestra.run.smithi097.stdout:Get:11 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4975-g08cec697-1jammy [338 kB] 2024-09-17T21:15:19.332 INFO:teuthology.orchestra.run.smithi184.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-17T21:15:19.338 INFO:teuthology.orchestra.run.smithi097.stdout:Get:12 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4975-g08cec697-1jammy [7,966 kB] 2024-09-17T21:15:19.366 INFO:teuthology.orchestra.run.smithi097.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-17T21:15:19.461 INFO:teuthology.orchestra.run.smithi083.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4975-g08cec697-1jammy [112 kB] 2024-09-17T21:15:19.464 INFO:teuthology.orchestra.run.smithi083.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4975-g08cec697-1jammy [473 kB] 2024-09-17T21:15:19.473 INFO:teuthology.orchestra.run.smithi083.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4975-g08cec697-1jammy [28.3 MB] 2024-09-17T21:15:19.561 INFO:teuthology.orchestra.run.smithi184.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-17T21:15:19.572 INFO:teuthology.orchestra.run.smithi184.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-17T21:15:19.594 INFO:teuthology.orchestra.run.smithi097.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-17T21:15:19.602 INFO:teuthology.orchestra.run.smithi184.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-17T21:15:19.605 INFO:teuthology.orchestra.run.smithi097.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-17T21:15:19.610 INFO:teuthology.orchestra.run.smithi184.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-17T21:15:19.613 INFO:teuthology.orchestra.run.smithi184.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-17T21:15:19.614 INFO:teuthology.orchestra.run.smithi184.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-17T21:15:19.615 INFO:teuthology.orchestra.run.smithi184.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-17T21:15:19.633 INFO:teuthology.orchestra.run.smithi184.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-17T21:15:19.635 INFO:teuthology.orchestra.run.smithi097.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-17T21:15:19.637 INFO:teuthology.orchestra.run.smithi184.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-17T21:15:19.643 INFO:teuthology.orchestra.run.smithi097.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-17T21:15:19.644 INFO:teuthology.orchestra.run.smithi184.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-17T21:15:19.646 INFO:teuthology.orchestra.run.smithi097.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-17T21:15:19.647 INFO:teuthology.orchestra.run.smithi097.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-17T21:15:19.648 INFO:teuthology.orchestra.run.smithi097.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-17T21:15:19.665 INFO:teuthology.orchestra.run.smithi097.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-17T21:15:19.670 INFO:teuthology.orchestra.run.smithi097.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-17T21:15:19.676 INFO:teuthology.orchestra.run.smithi097.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-17T21:15:19.684 INFO:teuthology.orchestra.run.smithi184.stdout:Get:24 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4975-g08cec697-1jammy [112 kB] 2024-09-17T21:15:19.687 INFO:teuthology.orchestra.run.smithi184.stdout:Get:25 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4975-g08cec697-1jammy [473 kB] 2024-09-17T21:15:19.696 INFO:teuthology.orchestra.run.smithi184.stdout:Get:26 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4975-g08cec697-1jammy [28.3 MB] 2024-09-17T21:15:19.723 INFO:teuthology.orchestra.run.smithi184.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-17T21:15:19.724 INFO:teuthology.orchestra.run.smithi184.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-17T21:15:19.728 INFO:teuthology.orchestra.run.smithi184.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-17T21:15:19.732 INFO:teuthology.orchestra.run.smithi184.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-17T21:15:19.735 INFO:teuthology.orchestra.run.smithi184.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-17T21:15:19.737 INFO:teuthology.orchestra.run.smithi184.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-17T21:15:19.737 INFO:teuthology.orchestra.run.smithi184.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-17T21:15:19.739 INFO:teuthology.orchestra.run.smithi184.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-17T21:15:19.740 INFO:teuthology.orchestra.run.smithi184.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-17T21:15:19.756 INFO:teuthology.orchestra.run.smithi097.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-17T21:15:19.756 INFO:teuthology.orchestra.run.smithi097.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-17T21:15:19.761 INFO:teuthology.orchestra.run.smithi097.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-17T21:15:19.765 INFO:teuthology.orchestra.run.smithi097.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-17T21:15:19.769 INFO:teuthology.orchestra.run.smithi097.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-17T21:15:19.770 INFO:teuthology.orchestra.run.smithi097.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-17T21:15:19.771 INFO:teuthology.orchestra.run.smithi097.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-17T21:15:19.773 INFO:teuthology.orchestra.run.smithi097.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-17T21:15:19.773 INFO:teuthology.orchestra.run.smithi097.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-17T21:15:19.790 INFO:teuthology.orchestra.run.smithi097.stdout:Get:33 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4975-g08cec697-1jammy [112 kB] 2024-09-17T21:15:19.792 INFO:teuthology.orchestra.run.smithi097.stdout:Get:34 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4975-g08cec697-1jammy [473 kB] 2024-09-17T21:15:19.802 INFO:teuthology.orchestra.run.smithi097.stdout:Get:35 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4975-g08cec697-1jammy [28.3 MB] 2024-09-17T21:15:19.806 INFO:teuthology.orchestra.run.smithi184.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-17T21:15:19.823 INFO:teuthology.orchestra.run.smithi184.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-17T21:15:19.823 INFO:teuthology.orchestra.run.smithi184.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-17T21:15:19.824 INFO:teuthology.orchestra.run.smithi184.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-17T21:15:19.825 INFO:teuthology.orchestra.run.smithi184.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-17T21:15:19.830 INFO:teuthology.orchestra.run.smithi184.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-17T21:15:19.831 INFO:teuthology.orchestra.run.smithi184.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-17T21:15:19.839 INFO:teuthology.orchestra.run.smithi097.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-17T21:15:19.856 INFO:teuthology.orchestra.run.smithi097.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-17T21:15:19.857 INFO:teuthology.orchestra.run.smithi097.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-17T21:15:19.857 INFO:teuthology.orchestra.run.smithi097.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-17T21:15:19.858 INFO:teuthology.orchestra.run.smithi097.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-17T21:15:19.864 INFO:teuthology.orchestra.run.smithi097.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-17T21:15:19.865 INFO:teuthology.orchestra.run.smithi097.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-17T21:15:19.888 INFO:teuthology.orchestra.run.smithi184.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-17T21:15:19.889 INFO:teuthology.orchestra.run.smithi184.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-17T21:15:19.891 INFO:teuthology.orchestra.run.smithi184.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-17T21:15:19.922 INFO:teuthology.orchestra.run.smithi097.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-17T21:15:19.922 INFO:teuthology.orchestra.run.smithi097.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-17T21:15:19.924 INFO:teuthology.orchestra.run.smithi097.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-17T21:15:19.969 INFO:teuthology.orchestra.run.smithi184.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-17T21:15:19.970 INFO:teuthology.orchestra.run.smithi184.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-17T21:15:19.972 INFO:teuthology.orchestra.run.smithi184.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-17T21:15:19.974 INFO:teuthology.orchestra.run.smithi184.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-17T21:15:19.975 INFO:teuthology.orchestra.run.smithi184.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-17T21:15:19.985 INFO:teuthology.orchestra.run.smithi184.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-17T21:15:20.002 INFO:teuthology.orchestra.run.smithi097.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-17T21:15:20.003 INFO:teuthology.orchestra.run.smithi097.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-17T21:15:20.007 INFO:teuthology.orchestra.run.smithi097.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-17T21:15:20.010 INFO:teuthology.orchestra.run.smithi097.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-17T21:15:20.011 INFO:teuthology.orchestra.run.smithi097.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-17T21:15:20.023 INFO:teuthology.orchestra.run.smithi097.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-17T21:15:20.051 INFO:teuthology.orchestra.run.smithi184.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-17T21:15:20.052 INFO:teuthology.orchestra.run.smithi184.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-17T21:15:20.053 INFO:teuthology.orchestra.run.smithi184.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-17T21:15:20.055 INFO:teuthology.orchestra.run.smithi184.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-17T21:15:20.083 INFO:teuthology.orchestra.run.smithi097.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-17T21:15:20.084 INFO:teuthology.orchestra.run.smithi097.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-17T21:15:20.085 INFO:teuthology.orchestra.run.smithi097.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-17T21:15:20.088 INFO:teuthology.orchestra.run.smithi097.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-17T21:15:20.132 INFO:teuthology.orchestra.run.smithi184.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-17T21:15:20.134 INFO:teuthology.orchestra.run.smithi184.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-17T21:15:20.136 INFO:teuthology.orchestra.run.smithi184.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-17T21:15:20.138 INFO:teuthology.orchestra.run.smithi184.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-17T21:15:20.143 INFO:teuthology.orchestra.run.smithi184.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-17T21:15:20.144 INFO:teuthology.orchestra.run.smithi184.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-17T21:15:20.165 INFO:teuthology.orchestra.run.smithi097.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-17T21:15:20.166 INFO:teuthology.orchestra.run.smithi097.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-17T21:15:20.169 INFO:teuthology.orchestra.run.smithi097.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-17T21:15:20.170 INFO:teuthology.orchestra.run.smithi097.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-17T21:15:20.175 INFO:teuthology.orchestra.run.smithi097.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-17T21:15:20.176 INFO:teuthology.orchestra.run.smithi097.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-17T21:15:20.214 INFO:teuthology.orchestra.run.smithi184.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-17T21:15:20.218 INFO:teuthology.orchestra.run.smithi184.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-17T21:15:20.246 INFO:teuthology.orchestra.run.smithi097.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-17T21:15:20.250 INFO:teuthology.orchestra.run.smithi097.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-17T21:15:20.381 INFO:teuthology.orchestra.run.smithi184.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-17T21:15:20.383 INFO:teuthology.orchestra.run.smithi184.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-17T21:15:20.386 INFO:teuthology.orchestra.run.smithi184.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-17T21:15:20.386 INFO:teuthology.orchestra.run.smithi184.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-17T21:15:20.388 INFO:teuthology.orchestra.run.smithi184.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-17T21:15:20.413 INFO:teuthology.orchestra.run.smithi097.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-17T21:15:20.415 INFO:teuthology.orchestra.run.smithi097.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-17T21:15:20.417 INFO:teuthology.orchestra.run.smithi097.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-17T21:15:20.418 INFO:teuthology.orchestra.run.smithi097.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-17T21:15:20.420 INFO:teuthology.orchestra.run.smithi097.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-17T21:15:20.457 INFO:teuthology.orchestra.run.smithi097.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-17T21:15:20.460 INFO:teuthology.orchestra.run.smithi097.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-17T21:15:20.461 INFO:teuthology.orchestra.run.smithi097.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-17T21:15:20.461 INFO:teuthology.orchestra.run.smithi184.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-17T21:15:20.465 INFO:teuthology.orchestra.run.smithi184.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-17T21:15:20.466 INFO:teuthology.orchestra.run.smithi184.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-17T21:15:20.497 INFO:teuthology.orchestra.run.smithi184.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-17T21:15:20.497 INFO:teuthology.orchestra.run.smithi184.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-17T21:15:20.523 INFO:teuthology.orchestra.run.smithi097.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-17T21:15:20.523 INFO:teuthology.orchestra.run.smithi097.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-17T21:15:20.548 INFO:teuthology.orchestra.run.smithi184.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-17T21:15:20.549 INFO:teuthology.orchestra.run.smithi184.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-17T21:15:20.549 INFO:teuthology.orchestra.run.smithi184.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-17T21:15:20.550 INFO:teuthology.orchestra.run.smithi184.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-17T21:15:20.551 INFO:teuthology.orchestra.run.smithi184.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-17T21:15:20.580 INFO:teuthology.orchestra.run.smithi097.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-17T21:15:20.581 INFO:teuthology.orchestra.run.smithi097.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-17T21:15:20.582 INFO:teuthology.orchestra.run.smithi097.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-17T21:15:20.583 INFO:teuthology.orchestra.run.smithi097.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-17T21:15:20.584 INFO:teuthology.orchestra.run.smithi097.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-17T21:15:20.585 INFO:teuthology.orchestra.run.smithi097.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-17T21:15:20.591 INFO:teuthology.orchestra.run.smithi097.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-17T21:15:20.630 INFO:teuthology.orchestra.run.smithi184.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-17T21:15:20.638 INFO:teuthology.orchestra.run.smithi184.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-17T21:15:20.642 INFO:teuthology.orchestra.run.smithi184.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-17T21:15:20.646 INFO:teuthology.orchestra.run.smithi184.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-17T21:15:20.647 INFO:teuthology.orchestra.run.smithi184.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-17T21:15:20.661 INFO:teuthology.orchestra.run.smithi097.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-17T21:15:20.665 INFO:teuthology.orchestra.run.smithi097.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-17T21:15:20.667 INFO:teuthology.orchestra.run.smithi097.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-17T21:15:20.711 INFO:teuthology.orchestra.run.smithi184.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-17T21:15:20.718 INFO:teuthology.orchestra.run.smithi184.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-17T21:15:20.720 INFO:teuthology.orchestra.run.smithi184.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-17T21:15:20.721 INFO:teuthology.orchestra.run.smithi184.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-17T21:15:20.732 INFO:teuthology.orchestra.run.smithi184.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-17T21:15:20.743 INFO:teuthology.orchestra.run.smithi097.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-17T21:15:20.749 INFO:teuthology.orchestra.run.smithi097.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-17T21:15:20.752 INFO:teuthology.orchestra.run.smithi097.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-17T21:15:20.753 INFO:teuthology.orchestra.run.smithi097.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-17T21:15:20.763 INFO:teuthology.orchestra.run.smithi097.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-17T21:15:20.765 INFO:teuthology.orchestra.run.smithi097.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-17T21:15:20.772 INFO:teuthology.orchestra.run.smithi097.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-17T21:15:20.793 INFO:teuthology.orchestra.run.smithi184.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-17T21:15:20.800 INFO:teuthology.orchestra.run.smithi184.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-17T21:15:20.801 INFO:teuthology.orchestra.run.smithi184.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-17T21:15:20.803 INFO:teuthology.orchestra.run.smithi184.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-17T21:15:20.803 INFO:teuthology.orchestra.run.smithi184.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-17T21:15:20.824 INFO:teuthology.orchestra.run.smithi097.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-17T21:15:20.826 INFO:teuthology.orchestra.run.smithi097.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-17T21:15:20.826 INFO:teuthology.orchestra.run.smithi097.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-17T21:15:20.875 INFO:teuthology.orchestra.run.smithi184.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-17T21:15:20.876 INFO:teuthology.orchestra.run.smithi184.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-17T21:15:20.893 INFO:teuthology.orchestra.run.smithi184.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-17T21:15:20.894 INFO:teuthology.orchestra.run.smithi184.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-17T21:15:20.894 INFO:teuthology.orchestra.run.smithi184.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-17T21:15:20.905 INFO:teuthology.orchestra.run.smithi097.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-17T21:15:20.907 INFO:teuthology.orchestra.run.smithi097.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-17T21:15:20.924 INFO:teuthology.orchestra.run.smithi097.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-17T21:15:20.925 INFO:teuthology.orchestra.run.smithi097.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-17T21:15:20.926 INFO:teuthology.orchestra.run.smithi097.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-17T21:15:20.931 INFO:teuthology.orchestra.run.smithi097.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-17T21:15:20.933 INFO:teuthology.orchestra.run.smithi097.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-17T21:15:20.956 INFO:teuthology.orchestra.run.smithi184.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-17T21:15:20.957 INFO:teuthology.orchestra.run.smithi184.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-17T21:15:21.084 INFO:teuthology.orchestra.run.smithi083.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4975-g08cec697-1jammy [5,207 kB] 2024-09-17T21:15:21.144 INFO:teuthology.orchestra.run.smithi184.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4975-g08cec697-1jammy [5,207 kB] 2024-09-17T21:15:21.211 INFO:teuthology.orchestra.run.smithi083.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4975-g08cec697-1jammy [251 kB] 2024-09-17T21:15:21.229 INFO:teuthology.orchestra.run.smithi083.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4975-g08cec697-1jammy [124 kB] 2024-09-17T21:15:21.232 INFO:teuthology.orchestra.run.smithi083.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4975-g08cec697-1jammy [1,562 kB] 2024-09-17T21:15:21.267 INFO:teuthology.orchestra.run.smithi184.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4975-g08cec697-1jammy [251 kB] 2024-09-17T21:15:21.269 INFO:teuthology.orchestra.run.smithi097.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4975-g08cec697-1jammy [5,207 kB] 2024-09-17T21:15:21.273 INFO:teuthology.orchestra.run.smithi184.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4975-g08cec697-1jammy [124 kB] 2024-09-17T21:15:21.286 INFO:teuthology.orchestra.run.smithi184.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4975-g08cec697-1jammy [1,562 kB] 2024-09-17T21:15:21.315 INFO:teuthology.orchestra.run.smithi083.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4975-g08cec697-1jammy [6,310 kB] 2024-09-17T21:15:21.353 INFO:teuthology.orchestra.run.smithi184.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4975-g08cec697-1jammy [6,310 kB] 2024-09-17T21:15:21.429 INFO:teuthology.orchestra.run.smithi097.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4975-g08cec697-1jammy [251 kB] 2024-09-17T21:15:21.434 INFO:teuthology.orchestra.run.smithi097.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4975-g08cec697-1jammy [124 kB] 2024-09-17T21:15:21.437 INFO:teuthology.orchestra.run.smithi097.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4975-g08cec697-1jammy [1,562 kB] 2024-09-17T21:15:21.502 INFO:teuthology.orchestra.run.smithi097.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4975-g08cec697-1jammy [6,310 kB] 2024-09-17T21:15:21.640 INFO:teuthology.orchestra.run.smithi083.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4975-g08cec697-1jammy [23.1 MB] 2024-09-17T21:15:21.650 INFO:teuthology.orchestra.run.smithi184.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4975-g08cec697-1jammy [23.1 MB] 2024-09-17T21:15:21.820 INFO:teuthology.orchestra.run.smithi097.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4975-g08cec697-1jammy [23.1 MB] 2024-09-17T21:15:22.852 INFO:teuthology.orchestra.run.smithi184.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4975-g08cec697-1jammy [14.0 kB] 2024-09-17T21:15:22.853 INFO:teuthology.orchestra.run.smithi184.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4975-g08cec697-1jammy [880 kB] 2024-09-17T21:15:22.870 INFO:teuthology.orchestra.run.smithi184.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4975-g08cec697-1jammy [2,455 kB] 2024-09-17T21:15:22.911 INFO:teuthology.orchestra.run.smithi184.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4975-g08cec697-1jammy [813 kB] 2024-09-17T21:15:22.915 INFO:teuthology.orchestra.run.smithi083.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4975-g08cec697-1jammy [14.0 kB] 2024-09-17T21:15:22.916 INFO:teuthology.orchestra.run.smithi083.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4975-g08cec697-1jammy [880 kB] 2024-09-17T21:15:22.933 INFO:teuthology.orchestra.run.smithi083.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4975-g08cec697-1jammy [2,455 kB] 2024-09-17T21:15:22.938 INFO:teuthology.orchestra.run.smithi184.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4975-g08cec697-1jammy [160 kB] 2024-09-17T21:15:22.940 INFO:teuthology.orchestra.run.smithi184.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4975-g08cec697-1jammy [61.5 MB] 2024-09-17T21:15:22.989 INFO:teuthology.orchestra.run.smithi083.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4975-g08cec697-1jammy [813 kB] 2024-09-17T21:15:23.025 INFO:teuthology.orchestra.run.smithi083.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4975-g08cec697-1jammy [160 kB] 2024-09-17T21:15:23.028 INFO:teuthology.orchestra.run.smithi083.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4975-g08cec697-1jammy [61.5 MB] 2024-09-17T21:15:23.047 INFO:teuthology.orchestra.run.smithi097.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4975-g08cec697-1jammy [14.0 kB] 2024-09-17T21:15:23.048 INFO:teuthology.orchestra.run.smithi097.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4975-g08cec697-1jammy [880 kB] 2024-09-17T21:15:23.065 INFO:teuthology.orchestra.run.smithi097.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4975-g08cec697-1jammy [2,455 kB] 2024-09-17T21:15:23.125 INFO:teuthology.orchestra.run.smithi097.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4975-g08cec697-1jammy [813 kB] 2024-09-17T21:15:23.179 INFO:teuthology.orchestra.run.smithi097.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4975-g08cec697-1jammy [160 kB] 2024-09-17T21:15:23.189 INFO:teuthology.orchestra.run.smithi097.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4975-g08cec697-1jammy [61.5 MB] 2024-09-17T21:15:26.207 INFO:teuthology.orchestra.run.smithi083.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4975-g08cec697-1jammy [8,625 kB] 2024-09-17T21:15:26.350 INFO:teuthology.orchestra.run.smithi083.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4975-g08cec697-1jammy [14.2 kB] 2024-09-17T21:15:26.351 INFO:teuthology.orchestra.run.smithi083.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4975-g08cec697-1jammy [48.5 MB] 2024-09-17T21:15:26.471 INFO:teuthology.orchestra.run.smithi184.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4975-g08cec697-1jammy [8,625 kB] 2024-09-17T21:15:26.673 INFO:teuthology.orchestra.run.smithi097.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4975-g08cec697-1jammy [8,625 kB] 2024-09-17T21:15:26.685 INFO:teuthology.orchestra.run.smithi184.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4975-g08cec697-1jammy [14.2 kB] 2024-09-17T21:15:26.686 INFO:teuthology.orchestra.run.smithi184.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4975-g08cec697-1jammy [48.5 MB] 2024-09-17T21:15:26.968 INFO:teuthology.orchestra.run.smithi097.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4975-g08cec697-1jammy [14.2 kB] 2024-09-17T21:15:26.968 INFO:teuthology.orchestra.run.smithi097.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4975-g08cec697-1jammy [48.5 MB] 2024-09-17T21:15:28.279 INFO:teuthology.orchestra.run.smithi083.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4975-g08cec697-1jammy [130 kB] 2024-09-17T21:15:28.281 INFO:teuthology.orchestra.run.smithi083.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4975-g08cec697-1jammy [39.0 kB] 2024-09-17T21:15:28.282 INFO:teuthology.orchestra.run.smithi083.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4975-g08cec697-1jammy [15.8 MB] 2024-09-17T21:15:28.915 INFO:teuthology.orchestra.run.smithi083.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4975-g08cec697-1jammy [91.2 kB] 2024-09-17T21:15:28.917 INFO:teuthology.orchestra.run.smithi083.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-nbd amd64 19.3.0-4975-g08cec697-1jammy [178 kB] 2024-09-17T21:15:29.452 INFO:teuthology.orchestra.run.smithi184.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4975-g08cec697-1jammy [130 kB] 2024-09-17T21:15:29.454 INFO:teuthology.orchestra.run.smithi184.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4975-g08cec697-1jammy [39.0 kB] 2024-09-17T21:15:29.456 INFO:teuthology.orchestra.run.smithi184.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4975-g08cec697-1jammy [15.8 MB] 2024-09-17T21:15:29.463 INFO:teuthology.orchestra.run.smithi097.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4975-g08cec697-1jammy [130 kB] 2024-09-17T21:15:29.466 INFO:teuthology.orchestra.run.smithi097.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4975-g08cec697-1jammy [39.0 kB] 2024-09-17T21:15:29.467 INFO:teuthology.orchestra.run.smithi097.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4975-g08cec697-1jammy [15.8 MB] 2024-09-17T21:15:29.495 INFO:teuthology.orchestra.run.smithi083.stdout:Fetched 244 MB in 10s (23.6 MB/s) 2024-09-17T21:15:29.665 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-17T21:15:29.765 INFO:teuthology.orchestra.run.smithi083.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-17T21:15:29.770 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-17T21:15:29.855 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T21:15:29.926 INFO:teuthology.orchestra.run.smithi097.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4975-g08cec697-1jammy [91.2 kB] 2024-09-17T21:15:29.928 INFO:teuthology.orchestra.run.smithi097.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-nbd amd64 19.3.0-4975-g08cec697-1jammy [178 kB] 2024-09-17T21:15:29.939 INFO:teuthology.orchestra.run.smithi184.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4975-g08cec697-1jammy [91.2 kB] 2024-09-17T21:15:29.941 INFO:teuthology.orchestra.run.smithi184.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-nbd amd64 19.3.0-4975-g08cec697-1jammy [178 kB] 2024-09-17T21:15:30.159 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-17T21:15:30.185 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-17T21:15:30.217 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T21:15:30.504 INFO:teuthology.orchestra.run.smithi097.stdout:Fetched 244 MB in 11s (21.7 MB/s) 2024-09-17T21:15:30.522 INFO:teuthology.orchestra.run.smithi184.stdout:Fetched 244 MB in 11s (21.6 MB/s) 2024-09-17T21:15:30.545 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-17T21:15:30.571 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-17T21:15:30.612 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T21:15:30.655 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-17T21:15:30.683 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-17T21:15:30.753 INFO:teuthology.orchestra.run.smithi097.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-17T21:15:30.758 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-17T21:15:30.784 INFO:teuthology.orchestra.run.smithi184.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-17T21:15:30.789 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-17T21:15:30.838 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T21:15:30.865 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T21:15:31.074 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-17T21:15:31.100 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T21:15:31.149 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T21:15:31.169 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-17T21:15:31.174 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-17T21:15:31.195 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-17T21:15:31.200 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-17T21:15:31.235 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T21:15:31.240 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T21:15:31.485 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-17T21:15:31.512 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T21:15:31.552 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T21:15:31.563 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-17T21:15:31.590 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-17T21:15:31.610 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-17T21:15:31.630 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T21:15:31.636 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-17T21:15:31.677 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T21:15:31.889 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-17T21:15:31.915 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T21:15:31.955 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T21:15:32.050 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-17T21:15:32.072 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-17T21:15:32.077 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T21:15:32.098 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T21:15:32.125 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T21:15:32.139 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T21:15:32.308 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-17T21:15:32.335 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-17T21:15:32.375 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T21:15:32.512 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-17T21:15:32.538 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T21:15:32.550 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-17T21:15:32.576 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T21:15:32.578 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T21:15:32.616 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T21:15:32.695 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package librados2. 2024-09-17T21:15:32.721 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../007-librados2_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:32.761 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking librados2 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:32.940 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-17T21:15:32.966 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T21:15:32.969 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-17T21:15:32.996 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T21:15:33.006 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T21:15:33.036 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T21:15:33.206 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libnbd0. 2024-09-17T21:15:33.232 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-17T21:15:33.281 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-17T21:15:33.372 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-17T21:15:33.376 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-17T21:15:33.388 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-17T21:15:33.402 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-17T21:15:33.431 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T21:15:33.434 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T21:15:33.601 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package librbd1. 2024-09-17T21:15:33.627 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../009-librbd1_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:33.667 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking librbd1 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:33.746 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package librados2. 2024-09-17T21:15:33.750 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package librados2. 2024-09-17T21:15:33.772 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../007-librados2_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:33.777 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../007-librados2_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:33.812 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking librados2 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:33.816 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking librados2 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:34.120 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libcephfs2. 2024-09-17T21:15:34.146 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:34.186 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libcephfs2 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:34.282 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package libnbd0. 2024-09-17T21:15:34.294 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libnbd0. 2024-09-17T21:15:34.308 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-17T21:15:34.321 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-17T21:15:34.348 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-17T21:15:34.361 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-17T21:15:34.439 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-rados. 2024-09-17T21:15:34.454 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:34.489 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-rados (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:34.643 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package librbd1. 2024-09-17T21:15:34.670 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../009-librbd1_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:34.672 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package librbd1. 2024-09-17T21:15:34.698 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../009-librbd1_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:34.710 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking librbd1 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:34.739 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking librbd1 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:34.800 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-17T21:15:34.827 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:15:34.867 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-ceph-argparse (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:35.121 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package libcephfs2. 2024-09-17T21:15:35.145 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-cephfs. 2024-09-17T21:15:35.147 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:35.171 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:35.187 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking libcephfs2 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:35.211 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-cephfs (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:35.225 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libcephfs2. 2024-09-17T21:15:35.251 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:35.291 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libcephfs2 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:35.465 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-rados. 2024-09-17T21:15:35.491 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:35.497 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-17T21:15:35.524 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:15:35.531 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-rados (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:35.581 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-ceph-common (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:35.729 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-rados. 2024-09-17T21:15:35.755 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:35.860 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-17T21:15:35.860 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-rados (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:35.888 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:15:35.926 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-ceph-argparse (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:36.076 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-17T21:15:36.102 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-17T21:15:36.142 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T21:15:36.162 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-cephfs. 2024-09-17T21:15:36.172 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-17T21:15:36.177 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:36.187 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:15:36.212 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-cephfs (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:36.222 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-ceph-argparse (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:36.441 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-cephfs. 2024-09-17T21:15:36.462 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-prettytable. 2024-09-17T21:15:36.467 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:36.488 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-17T21:15:36.507 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-cephfs (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:36.523 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-17T21:15:36.528 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-17T21:15:36.550 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:15:36.590 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-ceph-common (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:36.790 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-rbd. 2024-09-17T21:15:36.805 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:36.810 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-17T21:15:36.836 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:15:36.848 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-rbd (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:36.876 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-ceph-common (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:36.901 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-17T21:15:36.927 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-17T21:15:36.968 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T21:15:37.176 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-17T21:15:37.202 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-17T21:15:37.205 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-17T21:15:37.231 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-17T21:15:37.234 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T21:15:37.271 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T21:15:37.279 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-prettytable. 2024-09-17T21:15:37.306 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-17T21:15:37.337 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-17T21:15:37.487 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-17T21:15:37.513 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-17T21:15:37.554 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T21:15:37.599 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-prettytable. 2024-09-17T21:15:37.625 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-17T21:15:37.632 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-rbd. 2024-09-17T21:15:37.654 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:37.665 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-17T21:15:37.700 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-rbd (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:37.848 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-17T21:15:37.875 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-17T21:15:37.909 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-rbd. 2024-09-17T21:15:37.923 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T21:15:37.925 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:37.959 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-rbd (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:38.035 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-17T21:15:38.061 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-17T21:15:38.118 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T21:15:38.252 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package lua5.1. 2024-09-17T21:15:38.278 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-17T21:15:38.296 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-17T21:15:38.318 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-17T21:15:38.322 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-17T21:15:38.354 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T21:15:38.370 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-17T21:15:38.397 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-17T21:15:38.437 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T21:15:38.596 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package lua-any. 2024-09-17T21:15:38.622 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-17T21:15:38.658 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-17T21:15:38.673 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-17T21:15:38.679 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-17T21:15:38.715 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T21:15:38.749 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-17T21:15:38.775 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-17T21:15:38.815 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T21:15:38.941 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package zip. 2024-09-17T21:15:38.956 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-17T21:15:38.999 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking zip (3.0-12build2) ... 2024-09-17T21:15:39.026 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-17T21:15:39.041 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-17T21:15:39.084 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T21:15:39.160 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package lua5.1. 2024-09-17T21:15:39.187 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-17T21:15:39.227 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-17T21:15:39.286 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package unzip. 2024-09-17T21:15:39.312 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-17T21:15:39.352 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-17T21:15:39.421 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package lua5.1. 2024-09-17T21:15:39.447 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-17T21:15:39.487 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-17T21:15:39.530 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package lua-any. 2024-09-17T21:15:39.545 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-17T21:15:39.588 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-17T21:15:39.731 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package luarocks. 2024-09-17T21:15:39.758 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-17T21:15:39.798 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-17T21:15:39.798 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package lua-any. 2024-09-17T21:15:39.824 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package zip. 2024-09-17T21:15:39.825 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-17T21:15:39.839 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-17T21:15:39.865 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-17T21:15:39.874 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking zip (3.0-12build2) ... 2024-09-17T21:15:40.109 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package zip. 2024-09-17T21:15:40.124 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-17T21:15:40.161 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package unzip. 2024-09-17T21:15:40.167 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking zip (3.0-12build2) ... 2024-09-17T21:15:40.188 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-17T21:15:40.189 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package librgw2. 2024-09-17T21:15:40.216 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../026-librgw2_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:40.219 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-17T21:15:40.256 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking librgw2 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:40.487 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package unzip. 2024-09-17T21:15:40.555 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-17T21:15:40.555 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-17T21:15:40.623 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package luarocks. 2024-09-17T21:15:40.650 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-17T21:15:40.690 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-17T21:15:40.833 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-rgw. 2024-09-17T21:15:40.849 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:40.892 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-rgw (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:40.924 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package luarocks. 2024-09-17T21:15:40.951 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-17T21:15:40.991 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-17T21:15:41.064 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package librgw2. 2024-09-17T21:15:41.091 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../026-librgw2_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:41.131 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking librgw2 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:41.349 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package librgw2. 2024-09-17T21:15:41.370 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-17T21:15:41.375 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../026-librgw2_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:41.397 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-17T21:15:41.415 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking librgw2 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:41.436 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T21:15:41.733 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-rgw. 2024-09-17T21:15:41.749 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:41.756 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libradosstriper1. 2024-09-17T21:15:41.783 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:41.783 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-rgw (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:41.822 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libradosstriper1 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:42.017 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-rgw. 2024-09-17T21:15:42.033 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:42.075 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-rgw (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:42.111 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-17T21:15:42.138 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-17T21:15:42.176 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-common. 2024-09-17T21:15:42.177 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T21:15:42.203 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:42.234 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-common (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:42.470 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-17T21:15:42.496 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-17T21:15:42.506 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package libradosstriper1. 2024-09-17T21:15:42.533 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:42.545 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T21:15:42.572 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking libradosstriper1 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:42.873 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libradosstriper1. 2024-09-17T21:15:42.899 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:42.939 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libradosstriper1 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:42.951 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package ceph-common. 2024-09-17T21:15:42.978 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:43.009 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking ceph-common (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:43.368 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-common. 2024-09-17T21:15:43.394 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:43.434 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-common (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:43.740 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-base. 2024-09-17T21:15:43.767 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:43.816 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-base (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:44.436 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-17T21:15:44.462 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-17T21:15:44.502 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-17T21:15:44.556 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package ceph-base. 2024-09-17T21:15:44.582 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:44.633 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking ceph-base (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:44.831 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-cheroot. 2024-09-17T21:15:44.857 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-17T21:15:44.897 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T21:15:44.989 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-base. 2024-09-17T21:15:45.015 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:45.074 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-base (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:45.210 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-17T21:15:45.234 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-17T21:15:45.237 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-17T21:15:45.260 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-17T21:15:45.288 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-17T21:15:45.300 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-17T21:15:45.546 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-cheroot. 2024-09-17T21:15:45.573 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-17T21:15:45.603 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-17T21:15:45.604 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T21:15:45.629 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-17T21:15:45.670 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-17T21:15:45.702 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-17T21:15:45.728 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-17T21:15:45.777 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-17T21:15:45.924 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-17T21:15:45.951 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-17T21:15:45.956 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-17T21:15:45.982 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-17T21:15:45.991 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-17T21:15:46.014 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-17T21:15:46.256 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-cheroot. 2024-09-17T21:15:46.284 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-17T21:15:46.285 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-17T21:15:46.292 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-tempora. 2024-09-17T21:15:46.312 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-17T21:15:46.319 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-17T21:15:46.359 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-17T21:15:46.360 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-17T21:15:46.488 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T21:15:46.662 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-portend. 2024-09-17T21:15:46.663 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-17T21:15:46.689 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-17T21:15:46.690 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-17T21:15:46.728 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-17T21:15:46.730 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-17T21:15:46.807 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-17T21:15:46.834 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-17T21:15:46.891 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-17T21:15:47.023 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-17T21:15:47.033 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-tempora. 2024-09-17T21:15:47.049 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-17T21:15:47.059 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-17T21:15:47.090 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-17T21:15:47.099 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-17T21:15:47.193 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-17T21:15:47.220 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-17T21:15:47.260 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-17T21:15:47.376 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-17T21:15:47.402 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-portend. 2024-09-17T21:15:47.403 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-17T21:15:47.429 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-17T21:15:47.443 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-17T21:15:47.469 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-17T21:15:47.638 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-17T21:15:47.664 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-17T21:15:47.713 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-17T21:15:47.764 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-17T21:15:47.790 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-17T21:15:47.796 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-natsort. 2024-09-17T21:15:47.823 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-17T21:15:47.832 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-17T21:15:47.880 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-17T21:15:48.024 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-tempora. 2024-09-17T21:15:48.050 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-17T21:15:48.090 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-17T21:15:48.119 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-17T21:15:48.145 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-17T21:15:48.150 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-logutils. 2024-09-17T21:15:48.176 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-17T21:15:48.185 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-17T21:15:48.216 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-17T21:15:48.377 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-portend. 2024-09-17T21:15:48.403 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-17T21:15:48.443 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-17T21:15:48.503 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-17T21:15:48.505 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-natsort. 2024-09-17T21:15:48.528 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-17T21:15:48.532 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-17T21:15:48.569 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-17T21:15:48.572 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-17T21:15:48.779 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-17T21:15:48.805 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-17T21:15:48.854 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-17T21:15:48.867 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-logutils. 2024-09-17T21:15:48.872 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-mako. 2024-09-17T21:15:48.893 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-17T21:15:48.898 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-17T21:15:48.933 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-17T21:15:48.938 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T21:15:49.207 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-17T21:15:49.233 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-17T21:15:49.236 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-17T21:15:49.259 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-17T21:15:49.263 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-17T21:15:49.273 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-17T21:15:49.285 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-17T21:15:49.311 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-17T21:15:49.325 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-17T21:15:49.595 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-17T21:15:49.614 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-mako. 2024-09-17T21:15:49.621 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-17T21:15:49.635 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-natsort. 2024-09-17T21:15:49.640 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-17T21:15:49.651 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-17T21:15:49.661 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-17T21:15:49.680 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T21:15:49.685 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-17T21:15:49.955 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-logutils. 2024-09-17T21:15:49.964 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-webob. 2024-09-17T21:15:49.975 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-17T21:15:49.981 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-17T21:15:49.991 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-17T21:15:49.991 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-17T21:15:50.022 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-17T21:15:50.025 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-17T21:15:50.031 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T21:15:50.287 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-17T21:15:50.313 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-17T21:15:50.324 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-17T21:15:50.351 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-17T21:15:50.351 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-waitress. 2024-09-17T21:15:50.353 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-17T21:15:50.378 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-17T21:15:50.391 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-17T21:15:50.421 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T21:15:50.631 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-webob. 2024-09-17T21:15:50.643 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-mako. 2024-09-17T21:15:50.657 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-17T21:15:50.658 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-17T21:15:50.687 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-tempita. 2024-09-17T21:15:50.693 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T21:15:50.714 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T21:15:50.714 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-17T21:15:50.745 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T21:15:50.980 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-17T21:15:51.006 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-17T21:15:51.038 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-17T21:15:51.057 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-paste. 2024-09-17T21:15:51.076 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-waitress. 2024-09-17T21:15:51.083 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-17T21:15:51.102 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-17T21:15:51.123 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T21:15:51.146 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T21:15:51.333 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-17T21:15:51.359 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-17T21:15:51.399 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-17T21:15:51.401 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-17T21:15:51.417 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-17T21:15:51.452 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T21:15:51.462 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-tempita. 2024-09-17T21:15:51.488 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-17T21:15:51.528 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T21:15:51.702 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-webob. 2024-09-17T21:15:51.728 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-17T21:15:51.754 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-17T21:15:51.768 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T21:15:51.781 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-17T21:15:51.781 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-paste. 2024-09-17T21:15:51.796 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-17T21:15:51.821 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-17T21:15:51.839 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T21:15:52.838 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-waitress. 2024-09-17T21:15:52.865 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-17T21:15:52.916 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T21:15:52.925 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-webtest. 2024-09-17T21:15:52.951 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-17T21:15:52.991 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-17T21:15:53.241 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-tempita. 2024-09-17T21:15:53.256 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-17T21:15:53.277 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pecan. 2024-09-17T21:15:53.291 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T21:15:53.293 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-17T21:15:53.327 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T21:15:53.435 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-17T21:15:53.450 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-17T21:15:53.493 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T21:15:53.577 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-paste. 2024-09-17T21:15:53.603 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-17T21:15:53.643 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T21:15:53.666 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-certifi. 2024-09-17T21:15:53.693 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-17T21:15:53.724 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-17T21:15:53.763 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-17T21:15:53.789 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-17T21:15:53.829 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-17T21:15:53.946 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-17T21:15:53.962 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-17T21:15:54.005 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T21:15:54.061 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-idna. 2024-09-17T21:15:54.087 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-17T21:15:54.127 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T21:15:54.141 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-webtest. 2024-09-17T21:15:54.167 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-17T21:15:54.207 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-17T21:15:54.308 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-17T21:15:54.334 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-17T21:15:54.374 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-17T21:15:54.438 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-urllib3. 2024-09-17T21:15:54.465 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-17T21:15:54.505 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T21:15:54.519 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-pecan. 2024-09-17T21:15:54.546 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-17T21:15:54.585 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T21:15:54.677 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-webtest. 2024-09-17T21:15:54.703 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-17T21:15:54.743 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-17T21:15:54.842 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-requests. 2024-09-17T21:15:54.869 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-17T21:15:54.908 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T21:15:54.957 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-certifi. 2024-09-17T21:15:54.983 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-17T21:15:55.029 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pecan. 2024-09-17T21:15:55.040 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-17T21:15:55.056 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-17T21:15:55.096 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T21:15:55.220 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-17T21:15:55.246 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-17T21:15:55.286 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T21:15:55.368 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-idna. 2024-09-17T21:15:55.394 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-17T21:15:55.434 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T21:15:55.459 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-certifi. 2024-09-17T21:15:55.486 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-17T21:15:55.525 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-17T21:15:55.589 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-17T21:15:55.604 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:15:55.647 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:55.749 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-urllib3. 2024-09-17T21:15:55.772 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-17T21:15:55.804 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T21:15:55.837 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-idna. 2024-09-17T21:15:55.863 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-17T21:15:55.903 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T21:15:56.019 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-17T21:15:56.046 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:56.086 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:56.124 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-requests. 2024-09-17T21:15:56.150 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-17T21:15:56.182 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T21:15:56.225 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-urllib3. 2024-09-17T21:15:56.251 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-17T21:15:56.291 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T21:15:56.372 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-17T21:15:56.399 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-17T21:15:56.439 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-17T21:15:56.460 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-17T21:15:56.487 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-17T21:15:56.527 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T21:15:56.611 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-requests. 2024-09-17T21:15:56.637 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-17T21:15:56.677 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T21:15:56.804 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-17T21:15:56.820 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:15:56.854 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:56.950 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-openssl. 2024-09-17T21:15:56.955 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-17T21:15:56.970 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-17T21:15:56.976 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-17T21:15:57.005 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T21:15:57.016 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-17T21:15:57.226 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-17T21:15:57.253 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:57.293 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:57.300 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-17T21:15:57.315 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:15:57.353 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr. 2024-09-17T21:15:57.358 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:57.380 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:57.420 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:57.579 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-17T21:15:57.606 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-17T21:15:57.645 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-17T21:15:57.746 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-17T21:15:57.772 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:57.789 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mon. 2024-09-17T21:15:57.812 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:57.816 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:57.856 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mon (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:57.907 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-openssl. 2024-09-17T21:15:57.933 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-17T21:15:57.973 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-17T21:15:58.107 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-17T21:15:58.133 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-17T21:15:58.173 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-17T21:15:58.318 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package ceph-mgr. 2024-09-17T21:15:58.345 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:58.385 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking ceph-mgr (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:58.468 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-osd. 2024-09-17T21:15:58.476 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-openssl. 2024-09-17T21:15:58.494 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:58.502 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-17T21:15:58.534 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-osd (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:58.543 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-17T21:15:58.788 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package ceph-mon. 2024-09-17T21:15:58.815 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:58.854 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking ceph-mon (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:58.862 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr. 2024-09-17T21:15:58.889 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:58.929 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:59.290 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mon. 2024-09-17T21:15:59.317 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:59.348 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mon (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:59.458 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package ceph-osd. 2024-09-17T21:15:59.484 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:59.524 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking ceph-osd (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:59.546 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph. 2024-09-17T21:15:59.561 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../067-ceph_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:59.604 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:15:59.924 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-fuse. 2024-09-17T21:15:59.935 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-osd. 2024-09-17T21:15:59.950 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:59.962 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:15:59.990 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-fuse (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:00.001 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-osd (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:00.385 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mds. 2024-09-17T21:16:00.411 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:00.452 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mds (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:00.611 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package ceph. 2024-09-17T21:16:00.638 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../067-ceph_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:00.678 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking ceph (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:00.821 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package cephadm. 2024-09-17T21:16:00.836 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../070-cephadm_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:00.871 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking cephadm (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:00.997 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package ceph-fuse. 2024-09-17T21:16:01.024 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:01.046 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph. 2024-09-17T21:16:01.062 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../067-ceph_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:01.063 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking ceph-fuse (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:01.105 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:01.165 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-17T21:16:01.192 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-17T21:16:01.232 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-17T21:16:01.383 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package ceph-mds. 2024-09-17T21:16:01.399 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:01.423 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-fuse. 2024-09-17T21:16:01.433 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking ceph-mds (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:01.450 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:01.490 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-fuse (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:01.545 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-17T21:16:01.572 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T21:16:01.612 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T21:16:01.836 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package cephadm. 2024-09-17T21:16:01.863 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../070-cephadm_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:01.903 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking cephadm (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:02.102 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mds. 2024-09-17T21:16:02.128 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:02.168 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mds (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:02.222 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-17T21:16:02.249 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-17T21:16:02.288 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-17T21:16:02.579 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package cephadm. 2024-09-17T21:16:02.606 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../070-cephadm_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:02.608 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-17T21:16:02.634 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T21:16:02.645 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking cephadm (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:02.683 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T21:16:02.750 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-babel. 2024-09-17T21:16:02.777 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T21:16:02.816 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T21:16:02.940 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-17T21:16:02.955 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-17T21:16:03.006 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-17T21:16:03.145 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jinja2. 2024-09-17T21:16:03.172 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-17T21:16:03.228 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T21:16:03.326 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-17T21:16:03.341 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T21:16:03.384 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T21:16:03.531 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-17T21:16:03.547 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:16:03.581 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:03.727 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-babel. 2024-09-17T21:16:03.754 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T21:16:03.796 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T21:16:03.926 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-17T21:16:03.953 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-17T21:16:03.993 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-17T21:16:04.108 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-jinja2. 2024-09-17T21:16:04.135 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-17T21:16:04.174 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T21:16:04.279 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-routes. 2024-09-17T21:16:04.295 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-17T21:16:04.337 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T21:16:04.414 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-babel. 2024-09-17T21:16:04.441 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T21:16:04.480 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T21:16:04.536 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-17T21:16:04.563 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:16:04.602 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:04.641 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-17T21:16:04.668 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:16:04.699 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:04.825 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jinja2. 2024-09-17T21:16:04.852 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-17T21:16:04.891 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T21:16:04.964 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-17T21:16:04.991 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-17T21:16:05.030 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-17T21:16:05.236 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-17T21:16:05.263 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:16:05.302 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:05.342 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-routes. 2024-09-17T21:16:05.369 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-17T21:16:05.408 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T21:16:05.589 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-17T21:16:05.605 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-17T21:16:05.647 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-17T21:16:05.770 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-17T21:16:05.797 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:16:05.836 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:05.967 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-routes. 2024-09-17T21:16:05.993 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-17T21:16:06.033 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T21:16:06.370 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-17T21:16:06.397 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:16:06.436 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:07.405 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-17T21:16:07.434 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-17T21:16:07.471 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T21:16:07.950 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-joblib. 2024-09-17T21:16:07.977 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-17T21:16:08.016 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T21:16:08.387 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-17T21:16:08.414 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-17T21:16:08.453 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-17T21:16:08.723 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-sklearn. 2024-09-17T21:16:08.747 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-17T21:16:08.858 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T21:16:08.909 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-17T21:16:08.938 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-17T21:16:08.967 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T21:16:09.299 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-17T21:16:09.316 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-17T21:16:09.358 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T21:16:09.403 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-joblib. 2024-09-17T21:16:09.431 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-17T21:16:09.470 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T21:16:09.560 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-17T21:16:09.588 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:16:09.626 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:09.832 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-17T21:16:09.859 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-17T21:16:09.869 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-joblib. 2024-09-17T21:16:09.897 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-17T21:16:09.898 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-17T21:16:09.935 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T21:16:10.193 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-sklearn. 2024-09-17T21:16:10.220 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-17T21:16:10.306 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-17T21:16:10.333 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-17T21:16:10.353 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T21:16:10.372 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-17T21:16:10.621 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-cachetools. 2024-09-17T21:16:10.649 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-17T21:16:10.683 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-sklearn. 2024-09-17T21:16:10.687 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-17T21:16:10.710 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-17T21:16:10.835 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T21:16:10.957 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-17T21:16:10.985 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-17T21:16:11.015 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-17T21:16:11.088 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-17T21:16:11.116 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:16:11.154 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:11.294 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-17T21:16:11.322 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-17T21:16:11.360 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T21:16:11.586 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-17T21:16:11.614 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:16:11.652 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:11.689 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-rsa. 2024-09-17T21:16:11.716 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-17T21:16:11.755 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-17T21:16:12.076 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-google-auth. 2024-09-17T21:16:12.104 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-17T21:16:12.107 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-cachetools. 2024-09-17T21:16:12.123 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-17T21:16:12.142 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-17T21:16:12.157 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-17T21:16:12.452 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-17T21:16:12.479 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-17T21:16:12.479 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-17T21:16:12.507 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-17T21:16:12.518 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-17T21:16:12.545 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T21:16:12.589 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-cachetools. 2024-09-17T21:16:12.604 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-17T21:16:12.638 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-17T21:16:12.847 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-17T21:16:12.874 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-17T21:16:12.913 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T21:16:12.925 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-17T21:16:12.952 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-17T21:16:12.991 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-17T21:16:13.024 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-websocket. 2024-09-17T21:16:13.051 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-17T21:16:13.090 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-17T21:16:13.383 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-rsa. 2024-09-17T21:16:13.403 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-17T21:16:13.411 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-17T21:16:13.431 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-17T21:16:13.466 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-17T21:16:13.469 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T21:16:13.569 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-17T21:16:13.596 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-17T21:16:13.635 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T21:16:13.747 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-rsa. 2024-09-17T21:16:13.775 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-17T21:16:13.779 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-google-auth. 2024-09-17T21:16:13.805 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-17T21:16:13.806 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-17T21:16:13.845 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-17T21:16:14.109 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-google-auth. 2024-09-17T21:16:14.137 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-17T21:16:14.157 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-17T21:16:14.176 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-17T21:16:14.184 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-17T21:16:14.215 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T21:16:14.266 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-17T21:16:14.295 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:16:14.333 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:14.485 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-websocket. 2024-09-17T21:16:14.512 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-17T21:16:14.513 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-17T21:16:14.540 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-17T21:16:14.551 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-17T21:16:14.578 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T21:16:14.610 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-17T21:16:14.639 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-17T21:16:14.677 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T21:16:14.888 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-17T21:16:14.890 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-websocket. 2024-09-17T21:16:14.916 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-17T21:16:14.917 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-17T21:16:14.954 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T21:16:14.956 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-17T21:16:15.006 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-17T21:16:15.034 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T21:16:15.072 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T21:16:15.268 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-17T21:16:15.296 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-17T21:16:15.333 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package jq. 2024-09-17T21:16:15.334 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T21:16:15.361 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T21:16:15.400 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-17T21:16:15.644 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-17T21:16:15.660 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:16:15.661 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package socat. 2024-09-17T21:16:15.677 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-17T21:16:15.702 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:15.711 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-17T21:16:15.998 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-17T21:16:16.014 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package xmlstarlet. 2024-09-17T21:16:16.014 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:16:16.021 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-17T21:16:16.031 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-17T21:16:16.049 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-17T21:16:16.056 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:16.072 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-17T21:16:16.087 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T21:16:16.376 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-17T21:16:16.399 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-17T21:16:16.403 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-17T21:16:16.427 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T21:16:16.435 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-test. 2024-09-17T21:16:16.442 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T21:16:16.464 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:16.465 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T21:16:16.501 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-test (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:16.702 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package jq. 2024-09-17T21:16:16.730 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T21:16:16.762 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-17T21:16:16.790 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T21:16:16.813 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-17T21:16:16.828 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T21:16:17.063 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package socat. 2024-09-17T21:16:17.064 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package jq. 2024-09-17T21:16:17.079 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-17T21:16:17.080 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T21:16:17.114 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-17T21:16:17.121 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-17T21:16:17.441 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package xmlstarlet. 2024-09-17T21:16:17.469 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-17T21:16:17.507 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-17T21:16:17.534 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package socat. 2024-09-17T21:16:17.561 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-17T21:16:17.600 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-17T21:16:17.870 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package ceph-test. 2024-09-17T21:16:17.899 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:17.911 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package xmlstarlet. 2024-09-17T21:16:17.927 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-17T21:16:17.937 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking ceph-test (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:17.969 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-17T21:16:18.332 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-test. 2024-09-17T21:16:18.360 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:18.398 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-test (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:18.690 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-volume. 2024-09-17T21:16:18.718 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:16:18.756 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-volume (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:19.035 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-17T21:16:19.051 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:19.093 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libcephfs-dev (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:19.430 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-17T21:16:19.458 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-17T21:16:19.496 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T21:16:19.835 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-17T21:16:19.862 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-17T21:16:19.901 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-17T21:16:20.241 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package ceph-volume. 2024-09-17T21:16:20.269 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:16:20.272 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package nvme-cli. 2024-09-17T21:16:20.300 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-17T21:16:20.307 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking ceph-volume (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:20.338 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T21:16:20.611 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-17T21:16:20.628 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:20.670 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking libcephfs-dev (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:20.752 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package pkg-config. 2024-09-17T21:16:20.761 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-volume. 2024-09-17T21:16:20.780 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-17T21:16:20.789 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4975-g08cec697-1jammy_all.deb ... 2024-09-17T21:16:20.818 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T21:16:20.827 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-volume (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:21.014 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-17T21:16:21.042 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-17T21:16:21.088 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-17T21:16:21.089 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T21:16:21.104 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-17T21:16:21.131 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-17T21:16:21.147 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-17T21:16:21.159 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:21.198 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libcephfs-dev (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:21.477 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-17T21:16:21.567 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-17T21:16:21.567 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-17T21:16:21.568 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-17T21:16:21.568 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-17T21:16:21.570 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-17T21:16:21.586 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-17T21:16:21.608 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T21:16:21.624 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-17T21:16:21.919 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pastescript. 2024-09-17T21:16:21.947 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-17T21:16:21.955 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-17T21:16:21.983 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-17T21:16:21.985 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-17T21:16:21.998 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package nvme-cli. 2024-09-17T21:16:22.021 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-17T21:16:22.026 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-17T21:16:22.073 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T21:16:22.306 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pluggy. 2024-09-17T21:16:22.334 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-17T21:16:22.372 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-17T21:16:22.375 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package nvme-cli. 2024-09-17T21:16:22.403 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-17T21:16:22.441 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T21:16:22.511 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package pkg-config. 2024-09-17T21:16:22.540 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-17T21:16:22.578 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T21:16:22.684 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-psutil. 2024-09-17T21:16:22.712 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-17T21:16:22.750 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-17T21:16:22.823 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-17T21:16:22.851 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-17T21:16:22.863 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package pkg-config. 2024-09-17T21:16:22.881 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-17T21:16:22.891 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-17T21:16:22.929 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T21:16:23.079 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-py. 2024-09-17T21:16:23.106 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-17T21:16:23.145 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-17T21:16:23.191 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-17T21:16:23.219 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-17T21:16:23.257 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-17T21:16:23.276 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-17T21:16:23.304 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-17T21:16:23.342 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-17T21:16:23.482 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pygments. 2024-09-17T21:16:23.510 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-17T21:16:23.548 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T21:16:23.628 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-pastescript. 2024-09-17T21:16:23.656 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-17T21:16:23.660 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-17T21:16:23.689 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-17T21:16:23.695 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-17T21:16:23.727 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-17T21:16:24.007 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-pluggy. 2024-09-17T21:16:24.029 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pastescript. 2024-09-17T21:16:24.035 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-17T21:16:24.036 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-17T21:16:24.057 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-17T21:16:24.065 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-17T21:16:24.073 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-17T21:16:24.096 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-17T21:16:24.102 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-17T21:16:24.399 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pluggy. 2024-09-17T21:16:24.414 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-toml. 2024-09-17T21:16:24.415 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-17T21:16:24.442 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-17T21:16:24.457 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-17T21:16:24.480 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-17T21:16:24.660 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-psutil. 2024-09-17T21:16:24.688 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-17T21:16:24.726 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-17T21:16:24.752 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-psutil. 2024-09-17T21:16:24.780 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-17T21:16:24.800 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pytest. 2024-09-17T21:16:24.818 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-17T21:16:24.828 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-17T21:16:24.866 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T21:16:25.062 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-py. 2024-09-17T21:16:25.091 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-17T21:16:25.129 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-17T21:16:25.155 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-py. 2024-09-17T21:16:25.183 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-17T21:16:25.204 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-simplejson. 2024-09-17T21:16:25.221 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-17T21:16:25.232 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-17T21:16:25.270 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-17T21:16:25.449 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-pygments. 2024-09-17T21:16:25.477 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-17T21:16:25.508 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pygments. 2024-09-17T21:16:25.524 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T21:16:25.524 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-17T21:16:25.539 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-17T21:16:25.556 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-17T21:16:25.566 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T21:16:25.598 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-17T21:16:25.964 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-17T21:16:25.980 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-17T21:16:26.021 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-17T21:16:26.053 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-17T21:16:26.081 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-17T21:16:26.120 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-17T21:16:26.302 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package radosgw. 2024-09-17T21:16:26.324 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-toml. 2024-09-17T21:16:26.330 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../117-radosgw_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:26.352 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-17T21:16:26.368 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking radosgw (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:26.390 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-17T21:16:26.439 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-toml. 2024-09-17T21:16:26.467 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-17T21:16:26.505 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-17T21:16:26.677 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-pytest. 2024-09-17T21:16:26.705 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-17T21:16:26.743 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T21:16:26.783 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pytest. 2024-09-17T21:16:26.811 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-17T21:16:26.841 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T21:16:27.056 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package python3-simplejson. 2024-09-17T21:16:27.084 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-17T21:16:27.122 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-17T21:16:27.187 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-simplejson. 2024-09-17T21:16:27.215 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-17T21:16:27.238 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package rbd-fuse. 2024-09-17T21:16:27.253 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-17T21:16:27.266 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:27.304 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking rbd-fuse (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:27.383 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-17T21:16:27.411 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-17T21:16:27.449 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-17T21:16:27.514 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-17T21:16:27.531 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-17T21:16:27.532 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package rbd-nbd. 2024-09-17T21:16:27.560 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../119-rbd-nbd_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:27.572 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-17T21:16:27.599 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking rbd-nbd (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:27.929 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-17T21:16:28.179 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T21:16:28.204 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package radosgw. 2024-09-17T21:16:28.232 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../117-radosgw_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package radosgw. 2024-09-17T21:16:28.270 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking radosgw (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:28.279 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../117-radosgw_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:28.280 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T21:16:28.309 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking radosgw (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:28.571 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-17T21:16:28.881 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-17T21:16:29.190 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package rbd-fuse. 2024-09-17T21:16:29.203 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package rbd-fuse. 2024-09-17T21:16:29.206 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:29.220 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:29.248 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking rbd-fuse (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:29.262 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking rbd-fuse (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:29.295 INFO:teuthology.orchestra.run.smithi083.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-17T21:16:29.365 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up cephadm (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:29.492 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package rbd-nbd. 2024-09-17T21:16:29.508 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../119-rbd-nbd_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:29.522 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package rbd-nbd. 2024-09-17T21:16:29.539 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../119-rbd-nbd_19.3.0-4975-g08cec697-1jammy_amd64.deb ... 2024-09-17T21:16:29.550 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking rbd-nbd (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:29.581 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking rbd-nbd (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:29.831 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-17T21:16:29.879 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-17T21:16:29.899 INFO:teuthology.orchestra.run.smithi083.stdout:Adding system user cephadm....done 2024-09-17T21:16:30.056 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T21:16:30.097 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T21:16:30.127 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T21:16:30.215 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T21:16:30.228 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T21:16:30.332 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-17T21:16:30.457 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-17T21:16:30.495 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-17T21:16:30.592 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-17T21:16:30.718 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-17T21:16:30.772 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-17T21:16:30.869 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-17T21:16:30.985 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-17T21:16:31.201 INFO:teuthology.orchestra.run.smithi184.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-17T21:16:31.245 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T21:16:31.280 INFO:teuthology.orchestra.run.smithi097.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-17T21:16:31.284 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up cephadm (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:31.347 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up cephadm (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:31.363 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-17T21:16:31.690 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T21:16:31.821 INFO:teuthology.orchestra.run.smithi097.stdout:Adding system user cephadm....done 2024-09-17T21:16:31.969 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T21:16:32.042 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-17T21:16:32.093 INFO:teuthology.orchestra.run.smithi184.stdout:Adding system user cephadm....done 2024-09-17T21:16:32.216 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T21:16:32.229 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-17T21:16:32.293 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-17T21:16:32.480 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-17T21:16:32.492 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-17T21:16:32.561 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-17T21:16:32.606 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-17T21:16:32.752 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-17T21:16:32.829 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-17T21:16:32.869 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-17T21:16:32.890 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-17T21:16:33.080 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-17T21:16:33.112 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-17T21:16:33.141 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T21:16:33.268 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-17T21:16:33.347 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T21:16:33.473 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-17T21:16:33.498 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-17T21:16:33.586 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T21:16:33.774 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T21:16:33.816 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-ceph-argparse (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:33.946 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-17T21:16:34.059 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T21:16:34.135 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-17T21:16:34.185 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-17T21:16:34.197 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-17T21:16:34.311 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T21:16:34.394 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-17T21:16:34.428 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T21:16:34.448 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-17T21:16:34.538 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-17T21:16:34.654 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-17T21:16:34.700 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-17T21:16:34.781 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-17T21:16:34.930 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-17T21:16:34.959 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-17T21:16:35.173 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-17T21:16:35.182 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-17T21:16:35.211 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-17T21:16:35.259 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T21:16:35.260 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-17T21:16:35.369 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-17T21:16:35.433 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-17T21:16:35.511 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-ceph-argparse (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:35.684 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-ceph-argparse (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:35.736 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T21:16:35.829 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T21:16:35.846 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-17T21:16:35.955 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-17T21:16:36.073 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T21:16:36.097 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T21:16:36.109 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T21:16:36.174 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T21:16:36.219 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-17T21:16:36.275 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-17T21:16:36.337 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T21:16:36.356 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-17T21:16:36.463 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T21:16:36.543 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-17T21:16:36.589 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-17T21:16:36.658 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up zip (3.0-12build2) ... 2024-09-17T21:16:36.759 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T21:16:36.848 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-17T21:16:36.908 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-17T21:16:36.940 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-17T21:16:37.004 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T21:16:37.130 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-17T21:16:37.221 INFO:teuthology.orchestra.run.smithi184.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-17T21:16:37.247 INFO:teuthology.orchestra.run.smithi184.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-17T21:16:37.311 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T21:16:37.411 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-17T21:16:37.436 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T21:16:37.473 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T21:16:37.599 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-17T21:16:37.687 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T21:16:37.762 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T21:16:37.821 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-17T21:16:37.888 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-17T21:16:37.947 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T21:16:38.133 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T21:16:38.156 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T21:16:38.257 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-17T21:16:38.403 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-17T21:16:38.424 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-17T21:16:38.675 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-17T21:16:38.696 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up zip (3.0-12build2) ... 2024-09-17T21:16:38.742 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up zip (3.0-12build2) ... 2024-09-17T21:16:38.839 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T21:16:38.893 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T21:16:38.909 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T21:16:39.294 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T21:16:39.458 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T21:16:39.572 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T21:16:39.579 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T21:16:39.700 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T21:16:39.809 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-17T21:16:39.831 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T21:16:39.910 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T21:16:39.914 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-17T21:16:39.956 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-17T21:16:40.082 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T21:16:40.165 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-17T21:16:40.203 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-17T21:16:40.291 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-ceph-common (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:40.374 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-17T21:16:40.580 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-17T21:16:40.601 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T21:16:40.759 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-17T21:16:40.830 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T21:16:40.910 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T21:16:41.011 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T21:16:41.190 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T21:16:41.371 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T21:16:41.483 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T21:16:41.664 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T21:16:41.818 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-17T21:16:42.007 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-17T21:16:42.066 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-17T21:16:42.066 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T21:16:42.179 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-ceph-common (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:42.258 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-17T21:16:42.268 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T21:16:42.384 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-ceph-common (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:42.402 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-17T21:16:42.480 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T21:16:42.654 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T21:16:42.677 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T21:16:42.755 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-17T21:16:42.789 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T21:16:42.898 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T21:16:42.986 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T21:16:43.199 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-17T21:16:43.467 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-17T21:16:43.693 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T21:16:43.819 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-17T21:16:43.962 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T21:16:44.120 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-17T21:16:44.134 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T21:16:44.205 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T21:16:44.340 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-17T21:16:44.361 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T21:16:44.363 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-17T21:16:44.470 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-17T21:16:44.607 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T21:16:44.631 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-17T21:16:44.725 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-17T21:16:44.730 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T21:16:44.851 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T21:16:44.855 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-17T21:16:44.891 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-17T21:16:44.982 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T21:16:45.125 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-17T21:16:45.145 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-17T21:16:45.275 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-17T21:16:45.384 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T21:16:45.404 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-17T21:16:45.510 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-17T21:16:45.542 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-17T21:16:45.655 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T21:16:45.781 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-17T21:16:45.785 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T21:16:45.795 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T21:16:45.887 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-17T21:16:45.898 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T21:16:46.074 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-17T21:16:46.180 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-17T21:16:46.183 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T21:16:46.308 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-17T21:16:46.431 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-17T21:16:46.500 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-17T21:16:46.559 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-17T21:16:46.690 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-17T21:16:46.751 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T21:16:46.810 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-17T21:16:46.877 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-17T21:16:46.949 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-17T21:16:47.003 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T21:16:47.070 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-17T21:16:47.092 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-17T21:16:47.200 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-17T21:16:47.321 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T21:16:47.371 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T21:16:47.448 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T21:16:47.477 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T21:16:47.497 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-17T21:16:47.578 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-17T21:16:47.781 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T21:16:47.837 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T21:16:47.841 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-17T21:16:47.891 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T21:16:47.963 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T21:16:48.092 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-17T21:16:48.184 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T21:16:48.218 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-17T21:16:48.265 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T21:16:48.476 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-17T21:16:48.486 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T21:16:48.557 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-17T21:16:48.611 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T21:16:48.736 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T21:16:48.817 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T21:16:48.854 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-17T21:16:48.938 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T21:16:48.968 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-17T21:16:48.980 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T21:16:49.068 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-17T21:16:49.086 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T21:16:49.167 INFO:teuthology.orchestra.run.smithi184.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-17T21:16:49.231 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T21:16:49.298 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T21:16:49.399 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T21:16:49.412 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T21:16:49.538 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T21:16:49.540 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-17T21:16:49.666 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-17T21:16:49.784 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-17T21:16:49.923 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-17T21:16:49.934 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-17T21:16:50.026 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-17T21:16:50.152 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-17T21:16:50.183 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-17T21:16:50.272 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T21:16:50.309 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-17T21:16:50.412 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T21:16:50.530 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-17T21:16:50.538 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T21:16:50.560 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T21:16:50.661 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T21:16:50.864 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T21:16:50.907 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-17T21:16:50.971 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T21:16:51.157 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T21:16:51.166 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T21:16:51.255 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T21:16:51.416 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-17T21:16:51.494 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T21:16:51.540 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-17T21:16:51.542 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-17T21:16:51.620 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up librados2 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:51.691 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-17T21:16:51.746 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:51.810 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-17T21:16:51.872 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-17T21:16:51.966 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-17T21:16:52.163 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T21:16:52.175 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libcephfs2 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:52.276 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libradosstriper1 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:52.328 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T21:16:52.414 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-17T21:16:52.427 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-17T21:16:52.563 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-17T21:16:52.695 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up librbd1 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:52.766 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-17T21:16:52.822 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-modules-core (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:52.948 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-fuse (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:52.956 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-17T21:16:53.084 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T21:16:53.157 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T21:16:53.157 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T21:16:53.258 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T21:16:53.445 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T21:16:53.572 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up librados2 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:53.602 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T21:16:53.641 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libcephfs-dev (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:53.689 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:53.703 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up librados2 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:53.783 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-rados (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:53.799 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-17T21:16:53.813 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:53.901 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up librgw2 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:53.939 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-17T21:16:54.069 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T21:16:54.143 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libcephfs2 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:54.269 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libradosstriper1 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:54.275 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up libcephfs2 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:54.395 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-17T21:16:54.401 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up libradosstriper1 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:54.527 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-17T21:16:54.663 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up librbd1 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:54.789 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-modules-core (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:54.796 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up librbd1 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:54.915 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-fuse (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:54.938 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up ceph-mgr-modules-core (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:55.064 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up ceph-fuse (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:55.127 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T21:16:55.127 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T21:16:55.275 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T21:16:55.275 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T21:16:55.315 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-rbd (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:55.466 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up rbd-fuse (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:55.593 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up rbd-nbd (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:55.625 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libcephfs-dev (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:55.728 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up libcephfs-dev (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:55.735 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-rgw (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:55.751 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-rados (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:55.842 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-rados (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:55.853 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-cephfs (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:55.877 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up librgw2 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:55.962 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-common (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:55.984 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up librgw2 (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:56.020 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T21:16:56.094 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T21:16:56.479 INFO:teuthology.orchestra.run.smithi083.stdout:Adding group ceph....done 2024-09-17T21:16:56.798 INFO:teuthology.orchestra.run.smithi083.stdout:Adding system user ceph....done 2024-09-17T21:16:56.908 INFO:teuthology.orchestra.run.smithi083.stdout:Setting system user ceph properties....done 2024-09-17T21:16:56.924 INFO:teuthology.orchestra.run.smithi083.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-17T21:16:57.060 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-17T21:16:57.265 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-rbd (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:57.289 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-rbd (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:57.366 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up rbd-fuse (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:57.405 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-17T21:16:57.415 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up rbd-fuse (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:57.517 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up rbd-nbd (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:57.541 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up rbd-nbd (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:57.643 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-rgw (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:57.667 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-rgw (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:57.769 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-cephfs (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:57.793 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up python3-cephfs (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:57.847 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-test (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:57.903 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up ceph-common (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:57.903 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-common (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:57.972 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up radosgw (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:58.421 INFO:teuthology.orchestra.run.smithi097.stdout:Adding group ceph....done 2024-09-17T21:16:58.421 INFO:teuthology.orchestra.run.smithi184.stdout:Adding group ceph....done 2024-09-17T21:16:58.440 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T21:16:58.440 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T21:16:58.748 INFO:teuthology.orchestra.run.smithi097.stdout:Adding system user ceph....done 2024-09-17T21:16:58.765 INFO:teuthology.orchestra.run.smithi184.stdout:Adding system user ceph....done 2024-09-17T21:16:58.841 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-base (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:58.908 INFO:teuthology.orchestra.run.smithi097.stdout:Setting system user ceph properties....done 2024-09-17T21:16:58.923 INFO:teuthology.orchestra.run.smithi097.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-17T21:16:58.924 INFO:teuthology.orchestra.run.smithi184.stdout:Setting system user ceph properties....done 2024-09-17T21:16:58.941 INFO:teuthology.orchestra.run.smithi184.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-17T21:16:59.072 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-17T21:16:59.083 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-17T21:16:59.158 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-17T21:16:59.367 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-17T21:16:59.384 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-17T21:16:59.593 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mds (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:59.814 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T21:16:59.814 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T21:16:59.854 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-test (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:59.888 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up ceph-test (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:16:59.980 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up radosgw (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:00.014 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up radosgw (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:00.295 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:00.409 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T21:17:00.410 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T21:17:00.510 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T21:17:00.510 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T21:17:00.513 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T21:17:00.513 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T21:17:00.874 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-base (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:00.955 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-osd (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:01.024 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up ceph-base (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:01.216 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-17T21:17:01.231 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T21:17:01.231 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T21:17:01.372 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-17T21:17:01.698 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:01.709 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mds (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:01.822 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up ceph-mds (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:01.824 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:01.936 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T21:17:01.936 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T21:17:01.975 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mon (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:02.028 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T21:17:02.028 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T21:17:02.182 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T21:17:02.183 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T21:17:02.436 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:02.494 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up ceph-mgr (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:02.635 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-cephadm (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:02.653 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T21:17:02.653 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T21:17:02.712 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T21:17:02.712 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T21:17:02.777 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:02.903 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-dashboard (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:03.057 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-volume (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:03.112 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-osd (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:03.221 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up ceph-osd (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:03.371 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T21:17:03.371 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T21:17:03.408 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T21:17:03.508 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T21:17:03.522 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T21:17:03.522 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T21:17:03.652 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T21:17:03.897 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:03.989 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:03.998 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:04.115 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:04.133 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mon (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:04.266 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up ceph-mon (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:04.347 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T21:17:04.347 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T21:17:04.477 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T21:17:04.477 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T21:17:04.834 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-cephadm (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:04.925 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up ceph-mgr-cephadm (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:04.985 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:05.060 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up ceph (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:05.111 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-dashboard (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:05.170 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T21:17:05.203 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up ceph-mgr-dashboard (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:05.262 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-volume (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:05.354 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up ceph-volume (19.3.0-4975-g08cec697-1jammy) ... 2024-09-17T21:17:05.624 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T21:17:05.725 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T21:17:05.732 INFO:teuthology.orchestra.run.smithi184.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T21:17:05.810 INFO:teuthology.orchestra.run.smithi083.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T21:17:05.816 DEBUG:teuthology.parallel:result is None 2024-09-17T21:17:05.833 INFO:teuthology.orchestra.run.smithi184.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T21:17:05.852 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T21:17:06.161 INFO:teuthology.orchestra.run.smithi184.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T21:17:07.036 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T21:17:07.312 INFO:teuthology.orchestra.run.smithi184.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T21:17:07.693 INFO:teuthology.orchestra.run.smithi097.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T21:17:07.698 DEBUG:teuthology.parallel:result is None 2024-09-17T21:17:07.954 INFO:teuthology.orchestra.run.smithi184.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T21:17:07.959 DEBUG:teuthology.parallel:result is None 2024-09-17T21:17:07.960 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=08cec6974d86f81dc304360b31113f30b4085bf8 2024-09-17T21:17:08.102 DEBUG:teuthology.orchestra.run.smithi083:> dpkg-query -W -f '${Version}' ceph 2024-09-17T21:17:08.122 INFO:teuthology.orchestra.run.smithi083.stdout:19.3.0-4975-g08cec697-1jammy 2024-09-17T21:17:08.123 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4975-g08cec697-1jammy 2024-09-17T21:17:08.123 INFO:teuthology.task.install:The correct ceph version 19.3.0-4975-g08cec697-1jammy is installed. 2024-09-17T21:17:08.125 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=08cec6974d86f81dc304360b31113f30b4085bf8 2024-09-17T21:17:08.264 DEBUG:teuthology.orchestra.run.smithi097:> dpkg-query -W -f '${Version}' ceph 2024-09-17T21:17:08.283 INFO:teuthology.orchestra.run.smithi097.stdout:19.3.0-4975-g08cec697-1jammy 2024-09-17T21:17:08.283 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4975-g08cec697-1jammy 2024-09-17T21:17:08.283 INFO:teuthology.task.install:The correct ceph version 19.3.0-4975-g08cec697-1jammy is installed. 2024-09-17T21:17:08.286 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=08cec6974d86f81dc304360b31113f30b4085bf8 2024-09-17T21:17:08.429 DEBUG:teuthology.orchestra.run.smithi184:> dpkg-query -W -f '${Version}' ceph 2024-09-17T21:17:08.448 INFO:teuthology.orchestra.run.smithi184.stdout:19.3.0-4975-g08cec697-1jammy 2024-09-17T21:17:08.448 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4975-g08cec697-1jammy 2024-09-17T21:17:08.448 INFO:teuthology.task.install:The correct ceph version 19.3.0-4975-g08cec697-1jammy is installed. 2024-09-17T21:17:08.450 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-17T21:17:08.450 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T21:17:08.450 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T21:17:08.465 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-17T21:17:08.465 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T21:17:08.479 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T21:17:08.480 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T21:17:08.502 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-17T21:17:08.502 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T21:17:08.502 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T21:17:08.521 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T21:17:08.576 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-17T21:17:08.576 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T21:17:08.590 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T21:17:08.646 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T21:17:08.646 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T21:17:08.660 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T21:17:08.715 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-17T21:17:08.715 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T21:17:08.715 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T21:17:08.729 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T21:17:08.785 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-17T21:17:08.785 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T21:17:08.799 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T21:17:08.853 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T21:17:08.854 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T21:17:08.867 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T21:17:08.922 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-17T21:17:08.922 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T21:17:08.922 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T21:17:08.936 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T21:17:08.989 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-17T21:17:08.989 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T21:17:09.003 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T21:17:09.058 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T21:17:09.059 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T21:17:09.072 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T21:17:09.126 INFO:teuthology.run_tasks:Running task ceph... 2024-09-17T21:17:09.201 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-17T21:17:09.201 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 777 /var/log/ceph 2024-09-17T21:17:09.204 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /var/log/ceph 2024-09-17T21:17:09.206 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 777 /var/log/ceph 2024-09-17T21:17:09.218 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-17T21:17:09.218 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T21:17:09.259 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T21:17:09.261 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T21:17:09.274 INFO:tasks.ceph:Creating extra log directories... 2024-09-17T21:17:09.274 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T21:17:09.315 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T21:17:09.317 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T21:17:09.339 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-17T21:17:09.340 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-17T21:17:09.340 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909340', 'branch': 'main', 'description': 'rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7909340', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '08cec6974d86f81dc304360b31113f30b4085bf8'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '08cec6974d86f81dc304360b31113f30b4085bf8'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '5ecc7400dbbb63ce990b663d2a92e87dbc17f30d'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 950, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'], ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'], ['client.0']], 'seed': 4954, 'sha1': '08cec6974d86f81dc304360b31113f30b4085bf8', 'sleep_before_teardown': 0, 'subset': '16/128', 'suite': 'rbd', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_5ecc7400dbbb63ce990b663d2a92e87dbc17f30d/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '5ecc7400dbbb63ce990b663d2a92e87dbc17f30d', 'targets': {'smithi083.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB5Xm7xQCeN0sApjqCR1qm7iDSFBNjCLYO4AuVsxlcrDVdXswXNvgVbPkwaBLIqD4z2jfYlGJVt/zbRsBWzsTic=', 'smithi097.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOFo8SkZ//19bJHgQ9jO43IlqeomE47IvSKIiTDoE2SiswX/j/3FUm+vCqaQZxJjMVzQeno3Emldk/bNykvBk+U=', 'smithi184.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG4xlYdCBWyzeSoZ5Bz9fy3xaDFieROqBTYGCsHgO56/QNM1u1CYU/xUQf2xs8O3RQU8GMG18uiHSpbrzmFtNck='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'holebdy': 512, 'nbd': True, 'ops': 6000, 'punch_holes': True, 'readbdy': 512, 'truncbdy': 512, 'writebdy': 512}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_20:16:19', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-17T21:17:09.340 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T21:17:09.371 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T21:17:09.373 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T21:17:09.382 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T21:17:09.419 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T21:17:09.424 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T21:17:09.439 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T21:17:09.439 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T21:17:09.478 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T21:17:09.478 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_1 2024-09-17T21:17:09.526 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T21:17:09.527 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:17:09.527 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 878 Links: 1 2024-09-17T21:17:09.527 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:17:09.527 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-09-17 21:11:37.036544466 +0000 2024-09-17T21:17:09.527 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-09-17 21:11:36.804546538 +0000 2024-09-17T21:17:09.527 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-09-17 21:11:36.804546538 +0000 2024-09-17T21:17:09.527 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2024-09-17T21:17:09.527 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T21:17:09.580 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-09-17T21:17:09.580 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-09-17T21:17:09.580 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000331767 s, 1.5 MB/s 2024-09-17T21:17:09.581 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T21:17:09.631 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_2 2024-09-17T21:17:09.677 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T21:17:09.678 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:17:09.678 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 898 Links: 1 2024-09-17T21:17:09.678 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:17:09.678 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-09-17 21:11:37.496540358 +0000 2024-09-17T21:17:09.678 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-09-17 21:11:37.264542431 +0000 2024-09-17T21:17:09.678 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-09-17 21:11:37.264542431 +0000 2024-09-17T21:17:09.678 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2024-09-17T21:17:09.678 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T21:17:09.732 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-09-17T21:17:09.733 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-09-17T21:17:09.733 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000370967 s, 1.4 MB/s 2024-09-17T21:17:09.734 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T21:17:09.783 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_3 2024-09-17T21:17:09.830 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T21:17:09.831 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:17:09.831 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 914 Links: 1 2024-09-17T21:17:09.831 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:17:09.831 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-09-17 21:11:37.972536106 +0000 2024-09-17T21:17:09.831 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-09-17 21:11:37.740538178 +0000 2024-09-17T21:17:09.832 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-09-17 21:11:37.740538178 +0000 2024-09-17T21:17:09.832 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2024-09-17T21:17:09.832 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T21:17:09.884 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-09-17T21:17:09.884 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-09-17T21:17:09.884 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000422261 s, 1.2 MB/s 2024-09-17T21:17:09.885 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T21:17:09.934 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_4 2024-09-17T21:17:09.982 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T21:17:09.982 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:17:09.982 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 931 Links: 1 2024-09-17T21:17:09.982 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:17:09.982 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-09-17 21:11:38.468531676 +0000 2024-09-17T21:17:09.982 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-09-17 21:11:38.220533891 +0000 2024-09-17T21:17:09.982 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-09-17 21:11:38.220533891 +0000 2024-09-17T21:17:09.983 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2024-09-17T21:17:09.983 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T21:17:10.036 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-09-17T21:17:10.036 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-09-17T21:17:10.036 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.00038475 s, 1.3 MB/s 2024-09-17T21:17:10.038 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T21:17:10.087 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-17T21:17:10.087 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-17T21:17:10.087 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T21:17:10.093 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T21:17:10.093 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2024-09-17T21:17:10.139 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T21:17:10.139 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:17:10.139 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 886 Links: 1 2024-09-17T21:17:10.139 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:17:10.140 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-17 21:11:35.970740642 +0000 2024-09-17T21:17:10.140 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-17 21:11:35.730742786 +0000 2024-09-17T21:17:10.140 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-17 21:11:35.730742786 +0000 2024-09-17T21:17:10.140 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2024-09-17T21:17:10.140 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T21:17:10.198 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-17T21:17:10.198 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-17T21:17:10.198 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000448689 s, 1.1 MB/s 2024-09-17T21:17:10.199 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T21:17:10.248 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2024-09-17T21:17:10.295 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T21:17:10.295 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:17:10.295 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 898 Links: 1 2024-09-17T21:17:10.295 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:17:10.295 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-17 21:11:36.478736105 +0000 2024-09-17T21:17:10.296 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-17 21:11:36.214738463 +0000 2024-09-17T21:17:10.296 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-17 21:11:36.214738463 +0000 2024-09-17T21:17:10.296 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2024-09-17T21:17:10.296 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T21:17:10.348 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-17T21:17:10.349 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-17T21:17:10.349 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000400671 s, 1.3 MB/s 2024-09-17T21:17:10.350 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T21:17:10.400 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2024-09-17T21:17:10.451 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T21:17:10.451 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:17:10.451 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 917 Links: 1 2024-09-17T21:17:10.451 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:17:10.451 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-17 21:11:36.722733926 +0000 2024-09-17T21:17:10.451 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-17 21:11:36.718733962 +0000 2024-09-17T21:17:10.451 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-17 21:11:36.718733962 +0000 2024-09-17T21:17:10.451 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2024-09-17T21:17:10.451 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T21:17:10.505 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-17T21:17:10.505 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-17T21:17:10.505 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.0004118 s, 1.2 MB/s 2024-09-17T21:17:10.506 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T21:17:10.556 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2024-09-17T21:17:10.603 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T21:17:10.603 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:17:10.603 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 930 Links: 1 2024-09-17T21:17:10.603 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:17:10.603 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-17 21:11:37.474727208 +0000 2024-09-17T21:17:10.603 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-17 21:11:37.238729316 +0000 2024-09-17T21:17:10.603 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-17 21:11:37.238729316 +0000 2024-09-17T21:17:10.603 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2024-09-17T21:17:10.603 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T21:17:10.656 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-17T21:17:10.657 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-17T21:17:10.657 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000313386 s, 1.6 MB/s 2024-09-17T21:17:10.658 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T21:17:10.712 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-17T21:17:10.712 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi083.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-17T21:17:10.712 INFO:tasks.ceph:Generating config... 2024-09-17T21:17:10.714 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-17T21:17:10.714 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-17T21:17:10.714 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-17T21:17:10.714 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-17T21:17:10.714 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-17T21:17:10.714 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-17T21:17:10.714 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-17T21:17:10.714 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-17T21:17:10.714 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-17T21:17:10.714 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-17T21:17:10.714 INFO:tasks.ceph:[osd] bluestore allocator = stupid 2024-09-17T21:17:10.714 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-17T21:17:10.714 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-17T21:17:10.715 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-17T21:17:10.715 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-17T21:17:10.715 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-17T21:17:10.715 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-17T21:17:10.715 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-17T21:17:10.715 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-17T21:17:10.715 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-17T21:17:10.715 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-17T21:17:10.715 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-17T21:17:10.715 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-17T21:17:10.715 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-17T21:17:10.715 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-17T21:17:10.715 INFO:tasks.ceph:Setting up mon.a... 2024-09-17T21:17:10.716 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-17T21:17:10.751 INFO:teuthology.orchestra.run.smithi083.stdout:creating /etc/ceph/ceph.keyring 2024-09-17T21:17:10.764 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-17T21:17:10.839 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T21:17:10.892 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.83'), ('mon.c', '[v2:172.21.15.83:3301,v1:172.21.15.83:6790]'), ('mon.b', '172.21.15.97')] 2024-09-17T21:17:10.893 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.83,[v2:172.21.15.83:3301,v1:172.21.15.83:6790],172.21.15.97', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-17T21:17:10.894 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T21:17:10.894 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T21:17:10.941 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.83 --addv c '[v2:172.21.15.83:3301,v1:172.21.15.83:6790]' --add b 172.21.15.97 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:17:11.028 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:17:11.028 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool: generated fsid 224a7a29-e112-472e-8a13-f67345d6d123 2024-09-17T21:17:11.028 INFO:teuthology.orchestra.run.smithi083.stdout:setting min_mon_release = quincy 2024-09-17T21:17:11.028 INFO:teuthology.orchestra.run.smithi083.stdout:epoch 0 2024-09-17T21:17:11.028 INFO:teuthology.orchestra.run.smithi083.stdout:fsid 224a7a29-e112-472e-8a13-f67345d6d123 2024-09-17T21:17:11.028 INFO:teuthology.orchestra.run.smithi083.stdout:last_changed 2024-09-17T21:17:11.026988+0000 2024-09-17T21:17:11.028 INFO:teuthology.orchestra.run.smithi083.stdout:created 2024-09-17T21:17:11.026988+0000 2024-09-17T21:17:11.029 INFO:teuthology.orchestra.run.smithi083.stdout:min_mon_release 17 (quincy) 2024-09-17T21:17:11.029 INFO:teuthology.orchestra.run.smithi083.stdout:election_strategy: 1 2024-09-17T21:17:11.029 INFO:teuthology.orchestra.run.smithi083.stdout:0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-09-17T21:17:11.029 INFO:teuthology.orchestra.run.smithi083.stdout:1: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-09-17T21:17:11.029 INFO:teuthology.orchestra.run.smithi083.stdout:2: [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] mon.c 2024-09-17T21:17:11.029 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-17T21:17:11.030 DEBUG:teuthology.orchestra.run.smithi083:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T21:17:11.078 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 224a7a29-e112-472e-8a13-f67345d6d123... 2024-09-17T21:17:11.079 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T21:17:11.123 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T21:17:11.125 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T21:17:11.151 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2024-09-17T21:17:11.151 INFO:teuthology.orchestra.run.smithi083.stdout: chdir = "" 2024-09-17T21:17:11.151 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T21:17:11.151 INFO:teuthology.orchestra.run.smithi083.stdout: auth supported = cephx 2024-09-17T21:17:11.151 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.152 INFO:teuthology.orchestra.run.smithi083.stdout: filestore xattr use omap = true 2024-09-17T21:17:11.152 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.152 INFO:teuthology.orchestra.run.smithi083.stdout: mon clock drift allowed = 1.000 2024-09-17T21:17:11.152 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.152 INFO:teuthology.orchestra.run.smithi083.stdout: osd crush chooseleaf type = 0 2024-09-17T21:17:11.152 INFO:teuthology.orchestra.run.smithi083.stdout: auth debug = true 2024-09-17T21:17:11.152 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.152 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on old message = true 2024-09-17T21:17:11.152 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on bug = true 2024-09-17T21:17:11.152 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.152 INFO:teuthology.orchestra.run.smithi083.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T21:17:11.152 INFO:teuthology.orchestra.run.smithi083.stdout: mon pg warn max object skew = 0 2024-09-17T21:17:11.152 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.153 INFO:teuthology.orchestra.run.smithi083.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T21:17:11.153 INFO:teuthology.orchestra.run.smithi083.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T21:17:11.153 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.153 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default size = 2 2024-09-17T21:17:11.153 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.153 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow primary affinity = true 2024-09-17T21:17:11.153 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow pg remap = true 2024-09-17T21:17:11.153 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on legacy crush tunables = false 2024-09-17T21:17:11.153 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on crush straw calc version zero = false 2024-09-17T21:17:11.153 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on no sortbitwise = false 2024-09-17T21:17:11.153 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on osd down out interval zero = false 2024-09-17T21:17:11.153 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on too few osds = false 2024-09-17T21:17:11.153 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T21:17:11.154 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T21:17:11.154 INFO:teuthology.orchestra.run.smithi083.stdout: mon_allow_pool_size_one = true 2024-09-17T21:17:11.154 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.154 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T21:17:11.154 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.154 INFO:teuthology.orchestra.run.smithi083.stdout: osd default data pool replay window = 5 2024-09-17T21:17:11.154 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.154 INFO:teuthology.orchestra.run.smithi083.stdout: mon allow pool delete = true 2024-09-17T21:17:11.154 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.154 INFO:teuthology.orchestra.run.smithi083.stdout: mon cluster log file level = debug 2024-09-17T21:17:11.154 INFO:teuthology.orchestra.run.smithi083.stdout: debug asserts on shutdown = true 2024-09-17T21:17:11.154 INFO:teuthology.orchestra.run.smithi083.stdout: mon health detail to clog = false 2024-09-17T21:17:11.155 INFO:teuthology.orchestra.run.smithi083.stdout: mon host = "172.21.15.83,[v2:172.21.15.83:3301,v1:172.21.15.83:6790],172.21.15.97" 2024-09-17T21:17:11.155 INFO:teuthology.orchestra.run.smithi083.stdout: mon client directed command retry = 5 2024-09-17T21:17:11.155 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on pool no app = False 2024-09-17T21:17:11.155 INFO:teuthology.orchestra.run.smithi083.stdout: ms inject socket failures = 5000 2024-09-17T21:17:11.155 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = 224a7a29-e112-472e-8a13-f67345d6d123 2024-09-17T21:17:11.155 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.155 INFO:teuthology.orchestra.run.smithi083.stdout:[osd] 2024-09-17T21:17:11.155 INFO:teuthology.orchestra.run.smithi083.stdout: osd journal size = 100 2024-09-17T21:17:11.155 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.155 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub load threshold = 5.0 2024-09-17T21:17:11.155 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub max interval = 600 2024-09-17T21:17:11.155 INFO:teuthology.orchestra.run.smithi083.stdout: osd mclock profile = high_recovery_ops 2024-09-17T21:17:11.155 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.156 INFO:teuthology.orchestra.run.smithi083.stdout: osd recover clone overlap = true 2024-09-17T21:17:11.156 INFO:teuthology.orchestra.run.smithi083.stdout: osd recovery max chunk = 1048576 2024-09-17T21:17:11.156 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.156 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug shutdown = true 2024-09-17T21:17:11.156 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug op order = true 2024-09-17T21:17:11.156 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug verify stray on activate = true 2024-09-17T21:17:11.156 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug trim objects = true 2024-09-17T21:17:11.156 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.156 INFO:teuthology.orchestra.run.smithi083.stdout: osd open classes on start = true 2024-09-17T21:17:11.156 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug pg log writeout = true 2024-09-17T21:17:11.156 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.156 INFO:teuthology.orchestra.run.smithi083.stdout: osd deep scrub update digest min age = 30 2024-09-17T21:17:11.156 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.157 INFO:teuthology.orchestra.run.smithi083.stdout: osd map max advance = 10 2024-09-17T21:17:11.157 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.157 INFO:teuthology.orchestra.run.smithi083.stdout: journal zero on create = true 2024-09-17T21:17:11.157 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.157 INFO:teuthology.orchestra.run.smithi083.stdout: filestore ondisk finisher threads = 3 2024-09-17T21:17:11.157 INFO:teuthology.orchestra.run.smithi083.stdout: filestore apply finisher threads = 3 2024-09-17T21:17:11.157 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.157 INFO:teuthology.orchestra.run.smithi083.stdout: bdev debug aio = true 2024-09-17T21:17:11.157 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug misdirected ops = true 2024-09-17T21:17:11.157 INFO:teuthology.orchestra.run.smithi083.stdout: bdev async discard = True 2024-09-17T21:17:11.157 INFO:teuthology.orchestra.run.smithi083.stdout: bdev enable discard = True 2024-09-17T21:17:11.157 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore allocator = stupid 2024-09-17T21:17:11.157 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore block size = 96636764160 2024-09-17T21:17:11.157 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore fsck on mount = True 2024-09-17T21:17:11.158 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluefs = 1/20 2024-09-17T21:17:11.158 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluestore = 1/20 2024-09-17T21:17:11.158 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-09-17T21:17:11.158 INFO:teuthology.orchestra.run.smithi083.stdout: debug osd = 20 2024-09-17T21:17:11.158 INFO:teuthology.orchestra.run.smithi083.stdout: debug rocksdb = 4/10 2024-09-17T21:17:11.158 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T21:17:11.158 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd full ratio = 0.9 2024-09-17T21:17:11.158 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd nearfull ratio = 0.8 2024-09-17T21:17:11.158 INFO:teuthology.orchestra.run.smithi083.stdout: osd blocked scrub grace period = 3600 2024-09-17T21:17:11.158 INFO:teuthology.orchestra.run.smithi083.stdout: osd failsafe full ratio = 0.95 2024-09-17T21:17:11.158 INFO:teuthology.orchestra.run.smithi083.stdout: osd max markdown count = 1000 2024-09-17T21:17:11.158 INFO:teuthology.orchestra.run.smithi083.stdout: osd objectstore = bluestore 2024-09-17T21:17:11.159 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.159 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr] 2024-09-17T21:17:11.159 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-09-17T21:17:11.159 INFO:teuthology.orchestra.run.smithi083.stdout: debug mgr = 20 2024-09-17T21:17:11.159 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-09-17T21:17:11.159 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-09-17T21:17:11.159 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-09-17T21:17:11.159 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-09-17T21:17:11.159 INFO:teuthology.orchestra.run.smithi083.stdout: mgr/telemetry/nag = false 2024-09-17T21:17:11.159 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.159 INFO:teuthology.orchestra.run.smithi083.stdout:[mon] 2024-09-17T21:17:11.159 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-09-17T21:17:11.159 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-09-17T21:17:11.159 INFO:teuthology.orchestra.run.smithi083.stdout: debug paxos = 20 2024-09-17T21:17:11.160 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-09-17T21:17:11.160 INFO:teuthology.orchestra.run.smithi083.stdout: mon data avail warn = 5 2024-09-17T21:17:11.160 INFO:teuthology.orchestra.run.smithi083.stdout: mon mgr mkfs grace = 240 2024-09-17T21:17:11.160 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-09-17T21:17:11.160 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd reporter subtree level = osd 2024-09-17T21:17:11.160 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd prime pg temp = true 2024-09-17T21:17:11.160 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-09-17T21:17:11.160 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.160 INFO:teuthology.orchestra.run.smithi083.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T21:17:11.160 INFO:teuthology.orchestra.run.smithi083.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T21:17:11.160 INFO:teuthology.orchestra.run.smithi083.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T21:17:11.160 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.160 INFO:teuthology.orchestra.run.smithi083.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T21:17:11.160 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T21:17:11.161 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T21:17:11.161 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.161 INFO:teuthology.orchestra.run.smithi083.stdout: # 1m isn't quite enough 2024-09-17T21:17:11.161 INFO:teuthology.orchestra.run.smithi083.stdout: mon_down_mkfs_grace = 2m 2024-09-17T21:17:11.161 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.161 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_filestore_osds = false 2024-09-17T21:17:11.161 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:11.161 INFO:teuthology.orchestra.run.smithi083.stdout:[client] 2024-09-17T21:17:11.161 INFO:teuthology.orchestra.run.smithi083.stdout: rgw cache enabled = true 2024-09-17T21:17:11.161 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable ops log = true 2024-09-17T21:17:11.161 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable usage log = true 2024-09-17T21:17:11.161 INFO:teuthology.orchestra.run.smithi083.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T21:17:11.161 INFO:teuthology.orchestra.run.smithi083.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T21:17:11.161 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.a] 2024-09-17T21:17:11.162 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.c] 2024-09-17T21:17:11.162 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.b] 2024-09-17T21:17:11.162 INFO:teuthology.orchestra.run.smithi184.stdout:[global] 2024-09-17T21:17:11.162 INFO:teuthology.orchestra.run.smithi184.stdout: chdir = "" 2024-09-17T21:17:11.163 INFO:teuthology.orchestra.run.smithi184.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T21:17:11.163 INFO:teuthology.orchestra.run.smithi184.stdout: auth supported = cephx 2024-09-17T21:17:11.163 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.163 INFO:teuthology.orchestra.run.smithi184.stdout: filestore xattr use omap = true 2024-09-17T21:17:11.163 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.163 INFO:teuthology.orchestra.run.smithi184.stdout: mon clock drift allowed = 1.000 2024-09-17T21:17:11.163 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.163 INFO:teuthology.orchestra.run.smithi184.stdout: osd crush chooseleaf type = 0 2024-09-17T21:17:11.163 INFO:teuthology.orchestra.run.smithi184.stdout: auth debug = true 2024-09-17T21:17:11.163 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.163 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on old message = true 2024-09-17T21:17:11.163 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on bug = true 2024-09-17T21:17:11.163 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.163 INFO:teuthology.orchestra.run.smithi184.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T21:17:11.164 INFO:teuthology.orchestra.run.smithi184.stdout: mon pg warn max object skew = 0 2024-09-17T21:17:11.164 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.164 INFO:teuthology.orchestra.run.smithi184.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T21:17:11.164 INFO:teuthology.orchestra.run.smithi184.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T21:17:11.164 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.164 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default size = 2 2024-09-17T21:17:11.164 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.164 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow primary affinity = true 2024-09-17T21:17:11.164 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow pg remap = true 2024-09-17T21:17:11.164 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on legacy crush tunables = false 2024-09-17T21:17:11.164 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on crush straw calc version zero = false 2024-09-17T21:17:11.164 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on no sortbitwise = false 2024-09-17T21:17:11.164 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on osd down out interval zero = false 2024-09-17T21:17:11.164 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on too few osds = false 2024-09-17T21:17:11.165 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T21:17:11.165 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T21:17:11.165 INFO:teuthology.orchestra.run.smithi184.stdout: mon_allow_pool_size_one = true 2024-09-17T21:17:11.165 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.165 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T21:17:11.165 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.165 INFO:teuthology.orchestra.run.smithi184.stdout: osd default data pool replay window = 5 2024-09-17T21:17:11.165 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.165 INFO:teuthology.orchestra.run.smithi184.stdout: mon allow pool delete = true 2024-09-17T21:17:11.165 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.165 INFO:teuthology.orchestra.run.smithi184.stdout: mon cluster log file level = debug 2024-09-17T21:17:11.165 INFO:teuthology.orchestra.run.smithi184.stdout: debug asserts on shutdown = true 2024-09-17T21:17:11.165 INFO:teuthology.orchestra.run.smithi184.stdout: mon health detail to clog = false 2024-09-17T21:17:11.165 INFO:teuthology.orchestra.run.smithi184.stdout: mon host = "172.21.15.83,[v2:172.21.15.83:3301,v1:172.21.15.83:6790],172.21.15.97" 2024-09-17T21:17:11.166 INFO:teuthology.orchestra.run.smithi184.stdout: mon client directed command retry = 5 2024-09-17T21:17:11.166 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on pool no app = False 2024-09-17T21:17:11.166 INFO:teuthology.orchestra.run.smithi184.stdout: ms inject socket failures = 5000 2024-09-17T21:17:11.166 INFO:teuthology.orchestra.run.smithi184.stdout: fsid = 224a7a29-e112-472e-8a13-f67345d6d123 2024-09-17T21:17:11.166 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.166 INFO:teuthology.orchestra.run.smithi184.stdout:[osd] 2024-09-17T21:17:11.166 INFO:teuthology.orchestra.run.smithi184.stdout: osd journal size = 100 2024-09-17T21:17:11.166 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.166 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub load threshold = 5.0 2024-09-17T21:17:11.166 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub max interval = 600 2024-09-17T21:17:11.166 INFO:teuthology.orchestra.run.smithi184.stdout: osd mclock profile = high_recovery_ops 2024-09-17T21:17:11.166 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.166 INFO:teuthology.orchestra.run.smithi184.stdout: osd recover clone overlap = true 2024-09-17T21:17:11.167 INFO:teuthology.orchestra.run.smithi184.stdout: osd recovery max chunk = 1048576 2024-09-17T21:17:11.167 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.167 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug shutdown = true 2024-09-17T21:17:11.167 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug op order = true 2024-09-17T21:17:11.167 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug verify stray on activate = true 2024-09-17T21:17:11.167 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug trim objects = true 2024-09-17T21:17:11.167 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.167 INFO:teuthology.orchestra.run.smithi184.stdout: osd open classes on start = true 2024-09-17T21:17:11.167 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug pg log writeout = true 2024-09-17T21:17:11.167 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.167 INFO:teuthology.orchestra.run.smithi184.stdout: osd deep scrub update digest min age = 30 2024-09-17T21:17:11.167 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.167 INFO:teuthology.orchestra.run.smithi184.stdout: osd map max advance = 10 2024-09-17T21:17:11.167 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.168 INFO:teuthology.orchestra.run.smithi184.stdout: journal zero on create = true 2024-09-17T21:17:11.168 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.168 INFO:teuthology.orchestra.run.smithi184.stdout: filestore ondisk finisher threads = 3 2024-09-17T21:17:11.168 INFO:teuthology.orchestra.run.smithi184.stdout: filestore apply finisher threads = 3 2024-09-17T21:17:11.168 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.168 INFO:teuthology.orchestra.run.smithi184.stdout: bdev debug aio = true 2024-09-17T21:17:11.168 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug misdirected ops = true 2024-09-17T21:17:11.168 INFO:teuthology.orchestra.run.smithi184.stdout: bdev async discard = True 2024-09-17T21:17:11.168 INFO:teuthology.orchestra.run.smithi184.stdout: bdev enable discard = True 2024-09-17T21:17:11.168 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore allocator = stupid 2024-09-17T21:17:11.168 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore block size = 96636764160 2024-09-17T21:17:11.168 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore fsck on mount = True 2024-09-17T21:17:11.168 INFO:teuthology.orchestra.run.smithi184.stdout: debug bluefs = 1/20 2024-09-17T21:17:11.168 INFO:teuthology.orchestra.run.smithi184.stdout: debug bluestore = 1/20 2024-09-17T21:17:11.169 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-09-17T21:17:11.170 INFO:teuthology.orchestra.run.smithi184.stdout: debug osd = 20 2024-09-17T21:17:11.170 INFO:teuthology.orchestra.run.smithi184.stdout: debug rocksdb = 4/10 2024-09-17T21:17:11.170 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T21:17:11.170 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd full ratio = 0.9 2024-09-17T21:17:11.170 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd nearfull ratio = 0.8 2024-09-17T21:17:11.170 INFO:teuthology.orchestra.run.smithi184.stdout: osd blocked scrub grace period = 3600 2024-09-17T21:17:11.170 INFO:teuthology.orchestra.run.smithi184.stdout: osd failsafe full ratio = 0.95 2024-09-17T21:17:11.170 INFO:teuthology.orchestra.run.smithi184.stdout: osd max markdown count = 1000 2024-09-17T21:17:11.170 INFO:teuthology.orchestra.run.smithi184.stdout: osd objectstore = bluestore 2024-09-17T21:17:11.170 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.170 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr] 2024-09-17T21:17:11.170 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-09-17T21:17:11.171 INFO:teuthology.orchestra.run.smithi184.stdout: debug mgr = 20 2024-09-17T21:17:11.171 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2024-09-17T21:17:11.171 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2024-09-17T21:17:11.171 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2024-09-17T21:17:11.171 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2024-09-17T21:17:11.171 INFO:teuthology.orchestra.run.smithi184.stdout: mgr/telemetry/nag = false 2024-09-17T21:17:11.171 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.171 INFO:teuthology.orchestra.run.smithi184.stdout:[mon] 2024-09-17T21:17:11.171 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-09-17T21:17:11.171 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2024-09-17T21:17:11.171 INFO:teuthology.orchestra.run.smithi184.stdout: debug paxos = 20 2024-09-17T21:17:11.172 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2024-09-17T21:17:11.172 INFO:teuthology.orchestra.run.smithi184.stdout: mon data avail warn = 5 2024-09-17T21:17:11.172 INFO:teuthology.orchestra.run.smithi184.stdout: mon mgr mkfs grace = 240 2024-09-17T21:17:11.172 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2024-09-17T21:17:11.172 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd reporter subtree level = osd 2024-09-17T21:17:11.172 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd prime pg temp = true 2024-09-17T21:17:11.172 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2024-09-17T21:17:11.172 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.172 INFO:teuthology.orchestra.run.smithi184.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T21:17:11.172 INFO:teuthology.orchestra.run.smithi184.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T21:17:11.172 INFO:teuthology.orchestra.run.smithi184.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T21:17:11.172 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.172 INFO:teuthology.orchestra.run.smithi184.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T21:17:11.173 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T21:17:11.173 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T21:17:11.173 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.173 INFO:teuthology.orchestra.run.smithi184.stdout: # 1m isn't quite enough 2024-09-17T21:17:11.173 INFO:teuthology.orchestra.run.smithi184.stdout: mon_down_mkfs_grace = 2m 2024-09-17T21:17:11.173 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.173 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_filestore_osds = false 2024-09-17T21:17:11.173 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T21:17:11.173 INFO:teuthology.orchestra.run.smithi184.stdout:[client] 2024-09-17T21:17:11.173 INFO:teuthology.orchestra.run.smithi184.stdout: rgw cache enabled = true 2024-09-17T21:17:11.173 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable ops log = true 2024-09-17T21:17:11.173 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable usage log = true 2024-09-17T21:17:11.173 INFO:teuthology.orchestra.run.smithi184.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T21:17:11.173 INFO:teuthology.orchestra.run.smithi184.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T21:17:11.174 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.a] 2024-09-17T21:17:11.174 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.c] 2024-09-17T21:17:11.174 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.b] 2024-09-17T21:17:11.174 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2024-09-17T21:17:11.174 INFO:teuthology.orchestra.run.smithi097.stdout: chdir = "" 2024-09-17T21:17:11.174 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T21:17:11.174 INFO:teuthology.orchestra.run.smithi097.stdout: auth supported = cephx 2024-09-17T21:17:11.174 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.174 INFO:teuthology.orchestra.run.smithi097.stdout: filestore xattr use omap = true 2024-09-17T21:17:11.174 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.174 INFO:teuthology.orchestra.run.smithi097.stdout: mon clock drift allowed = 1.000 2024-09-17T21:17:11.174 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.174 INFO:teuthology.orchestra.run.smithi097.stdout: osd crush chooseleaf type = 0 2024-09-17T21:17:11.174 INFO:teuthology.orchestra.run.smithi097.stdout: auth debug = true 2024-09-17T21:17:11.175 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.175 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on old message = true 2024-09-17T21:17:11.175 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on bug = true 2024-09-17T21:17:11.175 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.175 INFO:teuthology.orchestra.run.smithi097.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T21:17:11.175 INFO:teuthology.orchestra.run.smithi097.stdout: mon pg warn max object skew = 0 2024-09-17T21:17:11.175 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.175 INFO:teuthology.orchestra.run.smithi097.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T21:17:11.175 INFO:teuthology.orchestra.run.smithi097.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T21:17:11.175 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.175 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default size = 2 2024-09-17T21:17:11.175 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.175 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow primary affinity = true 2024-09-17T21:17:11.175 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow pg remap = true 2024-09-17T21:17:11.176 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on legacy crush tunables = false 2024-09-17T21:17:11.176 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on crush straw calc version zero = false 2024-09-17T21:17:11.176 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on no sortbitwise = false 2024-09-17T21:17:11.176 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on osd down out interval zero = false 2024-09-17T21:17:11.176 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on too few osds = false 2024-09-17T21:17:11.176 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T21:17:11.176 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T21:17:11.176 INFO:teuthology.orchestra.run.smithi097.stdout: mon_allow_pool_size_one = true 2024-09-17T21:17:11.176 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.176 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T21:17:11.176 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.176 INFO:teuthology.orchestra.run.smithi097.stdout: osd default data pool replay window = 5 2024-09-17T21:17:11.177 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.177 INFO:teuthology.orchestra.run.smithi097.stdout: mon allow pool delete = true 2024-09-17T21:17:11.177 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.177 INFO:teuthology.orchestra.run.smithi097.stdout: mon cluster log file level = debug 2024-09-17T21:17:11.177 INFO:teuthology.orchestra.run.smithi097.stdout: debug asserts on shutdown = true 2024-09-17T21:17:11.177 INFO:teuthology.orchestra.run.smithi097.stdout: mon health detail to clog = false 2024-09-17T21:17:11.177 INFO:teuthology.orchestra.run.smithi097.stdout: mon host = "172.21.15.83,[v2:172.21.15.83:3301,v1:172.21.15.83:6790],172.21.15.97" 2024-09-17T21:17:11.177 INFO:teuthology.orchestra.run.smithi097.stdout: mon client directed command retry = 5 2024-09-17T21:17:11.177 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on pool no app = False 2024-09-17T21:17:11.177 INFO:teuthology.orchestra.run.smithi097.stdout: ms inject socket failures = 5000 2024-09-17T21:17:11.177 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = 224a7a29-e112-472e-8a13-f67345d6d123 2024-09-17T21:17:11.177 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.177 INFO:teuthology.orchestra.run.smithi097.stdout:[osd] 2024-09-17T21:17:11.177 INFO:teuthology.orchestra.run.smithi097.stdout: osd journal size = 100 2024-09-17T21:17:11.178 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.178 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub load threshold = 5.0 2024-09-17T21:17:11.178 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub max interval = 600 2024-09-17T21:17:11.178 INFO:teuthology.orchestra.run.smithi097.stdout: osd mclock profile = high_recovery_ops 2024-09-17T21:17:11.178 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.178 INFO:teuthology.orchestra.run.smithi097.stdout: osd recover clone overlap = true 2024-09-17T21:17:11.178 INFO:teuthology.orchestra.run.smithi097.stdout: osd recovery max chunk = 1048576 2024-09-17T21:17:11.178 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.178 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug shutdown = true 2024-09-17T21:17:11.178 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug op order = true 2024-09-17T21:17:11.178 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug verify stray on activate = true 2024-09-17T21:17:11.178 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug trim objects = true 2024-09-17T21:17:11.178 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.178 INFO:teuthology.orchestra.run.smithi097.stdout: osd open classes on start = true 2024-09-17T21:17:11.178 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug pg log writeout = true 2024-09-17T21:17:11.179 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.179 INFO:teuthology.orchestra.run.smithi097.stdout: osd deep scrub update digest min age = 30 2024-09-17T21:17:11.179 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.179 INFO:teuthology.orchestra.run.smithi097.stdout: osd map max advance = 10 2024-09-17T21:17:11.179 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.179 INFO:teuthology.orchestra.run.smithi097.stdout: journal zero on create = true 2024-09-17T21:17:11.179 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.179 INFO:teuthology.orchestra.run.smithi097.stdout: filestore ondisk finisher threads = 3 2024-09-17T21:17:11.179 INFO:teuthology.orchestra.run.smithi097.stdout: filestore apply finisher threads = 3 2024-09-17T21:17:11.179 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.179 INFO:teuthology.orchestra.run.smithi097.stdout: bdev debug aio = true 2024-09-17T21:17:11.179 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug misdirected ops = true 2024-09-17T21:17:11.179 INFO:teuthology.orchestra.run.smithi097.stdout: bdev async discard = True 2024-09-17T21:17:11.180 INFO:teuthology.orchestra.run.smithi097.stdout: bdev enable discard = True 2024-09-17T21:17:11.180 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore allocator = stupid 2024-09-17T21:17:11.180 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore block size = 96636764160 2024-09-17T21:17:11.180 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore fsck on mount = True 2024-09-17T21:17:11.180 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluefs = 1/20 2024-09-17T21:17:11.180 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluestore = 1/20 2024-09-17T21:17:11.180 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-09-17T21:17:11.180 INFO:teuthology.orchestra.run.smithi097.stdout: debug osd = 20 2024-09-17T21:17:11.180 INFO:teuthology.orchestra.run.smithi097.stdout: debug rocksdb = 4/10 2024-09-17T21:17:11.180 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T21:17:11.180 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd full ratio = 0.9 2024-09-17T21:17:11.180 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd nearfull ratio = 0.8 2024-09-17T21:17:11.180 INFO:teuthology.orchestra.run.smithi097.stdout: osd blocked scrub grace period = 3600 2024-09-17T21:17:11.180 INFO:teuthology.orchestra.run.smithi097.stdout: osd failsafe full ratio = 0.95 2024-09-17T21:17:11.180 INFO:teuthology.orchestra.run.smithi097.stdout: osd max markdown count = 1000 2024-09-17T21:17:11.181 INFO:teuthology.orchestra.run.smithi097.stdout: osd objectstore = bluestore 2024-09-17T21:17:11.181 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.181 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr] 2024-09-17T21:17:11.181 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-09-17T21:17:11.181 INFO:teuthology.orchestra.run.smithi097.stdout: debug mgr = 20 2024-09-17T21:17:11.181 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2024-09-17T21:17:11.181 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2024-09-17T21:17:11.181 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2024-09-17T21:17:11.181 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2024-09-17T21:17:11.181 INFO:teuthology.orchestra.run.smithi097.stdout: mgr/telemetry/nag = false 2024-09-17T21:17:11.181 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.181 INFO:teuthology.orchestra.run.smithi097.stdout:[mon] 2024-09-17T21:17:11.181 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-09-17T21:17:11.182 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2024-09-17T21:17:11.182 INFO:teuthology.orchestra.run.smithi097.stdout: debug paxos = 20 2024-09-17T21:17:11.182 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2024-09-17T21:17:11.182 INFO:teuthology.orchestra.run.smithi097.stdout: mon data avail warn = 5 2024-09-17T21:17:11.182 INFO:teuthology.orchestra.run.smithi097.stdout: mon mgr mkfs grace = 240 2024-09-17T21:17:11.182 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2024-09-17T21:17:11.182 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd reporter subtree level = osd 2024-09-17T21:17:11.182 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd prime pg temp = true 2024-09-17T21:17:11.182 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2024-09-17T21:17:11.182 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.182 INFO:teuthology.orchestra.run.smithi097.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T21:17:11.182 INFO:teuthology.orchestra.run.smithi097.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T21:17:11.182 INFO:teuthology.orchestra.run.smithi097.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T21:17:11.182 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.183 INFO:teuthology.orchestra.run.smithi097.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T21:17:11.183 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T21:17:11.183 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T21:17:11.183 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.183 INFO:teuthology.orchestra.run.smithi097.stdout: # 1m isn't quite enough 2024-09-17T21:17:11.183 INFO:teuthology.orchestra.run.smithi097.stdout: mon_down_mkfs_grace = 2m 2024-09-17T21:17:11.183 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.183 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_filestore_osds = false 2024-09-17T21:17:11.183 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-17T21:17:11.183 INFO:teuthology.orchestra.run.smithi097.stdout:[client] 2024-09-17T21:17:11.184 INFO:teuthology.orchestra.run.smithi097.stdout: rgw cache enabled = true 2024-09-17T21:17:11.184 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable ops log = true 2024-09-17T21:17:11.184 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable usage log = true 2024-09-17T21:17:11.184 INFO:teuthology.orchestra.run.smithi097.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T21:17:11.184 INFO:teuthology.orchestra.run.smithi097.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T21:17:11.184 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.a] 2024-09-17T21:17:11.184 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.c] 2024-09-17T21:17:11.184 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.b] 2024-09-17T21:17:11.186 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-17T21:17:11.186 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-17T21:17:11.278 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-17T21:17:11.278 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T21:17:11.278 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-17T21:17:11.283 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T21:17:11.284 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-17T21:17:11.330 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi083.front.sepia.ceph.com 2024-09-17T21:17:11.330 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T21:17:11.330 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T21:17:11.330 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T21:17:11.395 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T21:17:11.395 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:17:11.445 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi097.front.sepia.ceph.com 2024-09-17T21:17:11.445 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-17T21:17:11.445 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T21:17:11.445 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T21:17:11.469 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-17T21:17:11.469 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:17:11.514 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T21:17:11.515 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T21:17:11.515 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T21:17:11.515 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T21:17:11.539 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T21:17:11.539 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:17:11.587 INFO:tasks.ceph:Setting up mon nodes... 2024-09-17T21:17:11.587 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-17T21:17:11.587 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T21:17:11.632 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T21:17:11.646 INFO:tasks.ceph:Setting up mds nodes... 2024-09-17T21:17:11.646 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-17T21:17:11.646 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-17T21:17:11.684 INFO:teuthology.orchestra.run.smithi184.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-17T21:17:11.710 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-17T21:17:11.711 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi083.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-17T21:17:11.711 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-17T21:17:11.725 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-17T21:17:11.725 INFO:tasks.ceph:role: osd.0 2024-09-17T21:17:11.725 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi083.front.sepia.ceph.com 2024-09-17T21:17:11.725 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T21:17:11.789 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T21:17:11.789 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T21:17:11.789 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T21:17:11.789 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T21:17:11.789 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T21:17:11.789 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-09-17T21:17:11.789 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T21:17:11.790 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T21:17:11.790 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T21:17:11.790 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T21:17:12.271 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-09-17T21:17:12.273 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-09-17T21:17:12.273 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-17T21:17:12.424 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-17T21:17:12.435 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T21:17:12.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:17:12.436 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-17T21:17:12.493 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-17T21:17:12.493 INFO:tasks.ceph:role: osd.1 2024-09-17T21:17:12.493 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi083.front.sepia.ceph.com 2024-09-17T21:17:12.494 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T21:17:12.557 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T21:17:12.557 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T21:17:12.557 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T21:17:12.557 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T21:17:12.557 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T21:17:12.558 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-09-17T21:17:12.558 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T21:17:12.558 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T21:17:12.558 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T21:17:12.558 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T21:17:12.997 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-09-17T21:17:12.998 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-09-17T21:17:12.998 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-17T21:17:13.085 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-17T21:17:13.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:17:13.138 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T21:17:13.139 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-17T21:17:13.191 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-17T21:17:13.191 INFO:tasks.ceph:role: osd.2 2024-09-17T21:17:13.191 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi083.front.sepia.ceph.com 2024-09-17T21:17:13.191 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T21:17:13.255 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T21:17:13.255 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T21:17:13.255 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T21:17:13.255 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T21:17:13.255 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T21:17:13.255 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-09-17T21:17:13.255 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T21:17:13.255 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T21:17:13.255 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T21:17:13.255 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T21:17:13.714 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-09-17T21:17:13.716 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-09-17T21:17:13.717 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-17T21:17:13.840 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-17T21:17:13.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:17:13.851 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T21:17:13.851 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:17:13.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T21:17:13.957+0000 7fd8e912e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-17T21:17:13.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T21:17:13.961+0000 7fd8e912e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-17T21:17:13.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T21:17:13.961+0000 7fd8e912e7c0 -1 bdev(0x55995627d800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-17T21:17:13.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T21:17:13.961+0000 7fd8e912e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-17T21:17:17.113 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-17T21:17:17.127 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:17:17.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T21:17:17.201+0000 7fcfab3ad7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-17T21:17:17.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T21:17:17.205+0000 7fcfab3ad7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-17T21:17:17.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T21:17:17.205+0000 7fcfab3ad7c0 -1 bdev(0x55f747fb1800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-17T21:17:17.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T21:17:17.205+0000 7fcfab3ad7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-17T21:17:20.334 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-17T21:17:20.391 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:17:20.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T21:17:20.465+0000 7f31aa3c07c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-17T21:17:20.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T21:17:20.465+0000 7f31aa3c07c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-17T21:17:20.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T21:17:20.465+0000 7f31aa3c07c0 -1 bdev(0x55a3ad551800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-17T21:17:20.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T21:17:20.465+0000 7f31aa3c07c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-17T21:17:23.616 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-17T21:17:23.678 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-17T21:17:23.692 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-17T21:17:23.693 INFO:tasks.ceph:role: osd.3 2024-09-17T21:17:23.693 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com 2024-09-17T21:17:23.693 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T21:17:23.756 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T21:17:23.756 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T21:17:23.756 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T21:17:23.756 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T21:17:23.756 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T21:17:23.756 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-09-17T21:17:23.756 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T21:17:23.756 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T21:17:23.757 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T21:17:23.757 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T21:17:24.238 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-09-17T21:17:24.240 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-09-17T21:17:24.240 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-17T21:17:24.336 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-17T21:17:24.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:17:24.347 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T21:17:24.348 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-17T21:17:24.403 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-17T21:17:24.403 INFO:tasks.ceph:role: osd.4 2024-09-17T21:17:24.403 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com 2024-09-17T21:17:24.403 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T21:17:24.466 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T21:17:24.466 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T21:17:24.466 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T21:17:24.466 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T21:17:24.466 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T21:17:24.466 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-09-17T21:17:24.467 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T21:17:24.467 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T21:17:24.467 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T21:17:24.467 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T21:17:24.934 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-09-17T21:17:24.937 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-09-17T21:17:24.937 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-17T21:17:25.084 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-17T21:17:25.095 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T21:17:25.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:17:25.096 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-17T21:17:25.150 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-17T21:17:25.150 INFO:tasks.ceph:role: osd.5 2024-09-17T21:17:25.150 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi097.front.sepia.ceph.com 2024-09-17T21:17:25.150 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T21:17:25.214 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T21:17:25.214 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T21:17:25.214 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T21:17:25.215 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T21:17:25.215 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T21:17:25.215 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-09-17T21:17:25.215 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T21:17:25.215 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T21:17:25.215 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T21:17:25.215 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T21:17:25.699 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-09-17T21:17:25.701 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-09-17T21:17:25.701 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-09-17T21:17:25.857 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-17T21:17:25.867 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T21:17:25.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:17:25.868 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:17:25.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-17T21:17:25.974+0000 7f27813f27c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-17T21:17:25.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-17T21:17:25.974+0000 7f27813f27c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-17T21:17:25.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-17T21:17:25.974+0000 7f27813f27c0 -1 bdev(0x5568c57b3800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-17T21:17:25.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-17T21:17:25.974+0000 7f27813f27c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-17T21:17:29.114 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-17T21:17:29.128 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:17:29.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-17T21:17:29.202+0000 7fe62a5a67c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-17T21:17:29.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-17T21:17:29.202+0000 7fe62a5a67c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-17T21:17:29.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-17T21:17:29.202+0000 7fe62a5a67c0 -1 bdev(0x5597f18cb800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-17T21:17:29.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-17T21:17:29.202+0000 7fe62a5a67c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-17T21:17:32.347 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-17T21:17:32.403 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:17:32.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-17T21:17:32.478+0000 7f28cab8c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-17T21:17:32.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-17T21:17:32.478+0000 7f28cab8c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-17T21:17:32.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-17T21:17:32.478+0000 7f28cab8c7c0 -1 bdev(0x55f2788d7800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-17T21:17:32.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-17T21:17:32.478+0000 7f28cab8c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-17T21:17:35.642 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-17T21:17:35.699 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-17T21:17:35.699 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T21:17:35.699 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-17T21:17:35.714 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T21:17:35.714 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-17T21:17:35.768 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T21:17:35.768 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-17T21:17:35.824 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-17T21:17:35.824 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-17T21:17:35.837 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-17T21:17:35.837 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-17T21:17:35.890 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-17T21:17:35.890 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-17T21:17:35.946 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-17T21:17:35.946 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-17T21:17:36.001 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T21:17:36.001 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-17T21:17:36.007 INFO:tasks.ceph:Adding keys to all mons... 2024-09-17T21:17:36.008 DEBUG:teuthology.orchestra.run.smithi083:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T21:17:36.010 DEBUG:teuthology.orchestra.run.smithi097:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T21:17:36.047 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.0] 2024-09-17T21:17:36.047 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQDZ8elm1tVKORAA6qrkyOeIh1LGJHs1mvbJGQ== 2024-09-17T21:17:36.047 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.1] 2024-09-17T21:17:36.047 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQDd8elm9s1iDBAAr9XwvjiBeU/9EWbvM9iZww== 2024-09-17T21:17:36.048 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.2] 2024-09-17T21:17:36.048 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQDg8elmyL4HHBAAA81OukbnbS3ZmayRC+UJKQ== 2024-09-17T21:17:36.048 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr.x] 2024-09-17T21:17:36.048 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQDX8elmLaKlJRAAs/AP+evLoZf7J3yo2XYhsw== 2024-09-17T21:17:36.048 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.3] 2024-09-17T21:17:36.048 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQDl8elmHXpeOhAABllaxgC+ESxg6ikU42P0Bw== 2024-09-17T21:17:36.048 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.4] 2024-09-17T21:17:36.048 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQDp8elm+/hjDBAAcIpFFrffTsJLPQWgaQedhg== 2024-09-17T21:17:36.048 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.5] 2024-09-17T21:17:36.048 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQDs8elmvlHaHBAAOik/7oiSbBk9krofG+zT5Q== 2024-09-17T21:17:36.048 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-09-17T21:17:36.048 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQDX8elmgorKKBAAfqlQKCAFc5UNvSnHn69yYg== 2024-09-17T21:17:36.054 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.0] 2024-09-17T21:17:36.054 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDZ8elm1tVKORAA6qrkyOeIh1LGJHs1mvbJGQ== 2024-09-17T21:17:36.054 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.1] 2024-09-17T21:17:36.055 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDd8elm9s1iDBAAr9XwvjiBeU/9EWbvM9iZww== 2024-09-17T21:17:36.055 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.2] 2024-09-17T21:17:36.055 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDg8elmyL4HHBAAA81OukbnbS3ZmayRC+UJKQ== 2024-09-17T21:17:36.055 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr.x] 2024-09-17T21:17:36.055 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDX8elmLaKlJRAAs/AP+evLoZf7J3yo2XYhsw== 2024-09-17T21:17:36.055 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.3] 2024-09-17T21:17:36.055 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDl8elmHXpeOhAABllaxgC+ESxg6ikU42P0Bw== 2024-09-17T21:17:36.055 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.4] 2024-09-17T21:17:36.055 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDp8elm+/hjDBAAcIpFFrffTsJLPQWgaQedhg== 2024-09-17T21:17:36.055 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.5] 2024-09-17T21:17:36.055 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDs8elmvlHaHBAAOik/7oiSbBk9krofG+zT5Q== 2024-09-17T21:17:36.055 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-09-17T21:17:36.055 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDX8elmgorKKBAAfqlQKCAFc5UNvSnHn69yYg== 2024-09-17T21:17:36.056 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:17:36.094 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:17:36.144 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:17:36.146 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:17:36.188 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:17:36.230 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:17:36.269 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T21:17:36.314 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T21:17:36.359 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:17:36.394 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:17:36.439 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:17:36.482 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:17:36.527 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:17:36.570 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:17:36.615 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T21:17:36.654 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T21:17:36.705 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-17T21:17:36.705 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-17T21:17:36.748 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T21:17:36.836 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-17T21:17:36.847 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-17T21:17:36.898 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T21:17:36.989 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-17T21:17:36.999 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-17T21:17:37.014 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T21:17:37.116 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-17T21:17:37.127 DEBUG:teuthology.orchestra.run.smithi083:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:17:37.130 DEBUG:teuthology.orchestra.run.smithi097:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:17:37.174 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-17T21:17:37.174 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-17T21:17:37.174 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-17T21:17:37.178 INFO:tasks.ceph.mon.a:Started 2024-09-17T21:17:37.178 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-17T21:17:37.178 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-17T21:17:37.181 INFO:tasks.ceph.mon.c:Started 2024-09-17T21:17:37.181 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-17T21:17:37.181 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-17T21:17:37.219 INFO:tasks.ceph.mon.b:Started 2024-09-17T21:17:37.219 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-17T21:17:37.220 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-17T21:17:37.220 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-17T21:17:37.222 INFO:tasks.ceph.mgr.x:Started 2024-09-17T21:17:37.222 DEBUG:tasks.ceph:set 0 configs 2024-09-17T21:17:37.222 INFO:tasks.ceph:Setting crush tunables to default 2024-09-17T21:17:37.222 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd crush tunables default 2024-09-17T21:17:42.713 INFO:teuthology.orchestra.run.smithi083.stderr:adjusted tunables profile to default 2024-09-17T21:17:42.731 INFO:tasks.ceph:check_enable_crimson: False 2024-09-17T21:17:42.731 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-17T21:17:42.732 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T21:17:42.732 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-17T21:17:42.744 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T21:17:42.744 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-17T21:17:42.799 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T21:17:42.800 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-17T21:17:42.856 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-17T21:17:42.856 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-17T21:17:42.869 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-17T21:17:42.869 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-17T21:17:42.921 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-17T21:17:42.921 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-17T21:17:42.973 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new 5bfd41c9-0cac-47c4-9a6e-8dce774c3314 0 2024-09-17T21:17:43.234 INFO:teuthology.orchestra.run.smithi097.stdout:0 2024-09-17T21:17:43.251 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new 78404e0d-d397-4806-8e90-5f32182643bb 1 2024-09-17T21:17:43.451 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-17T21:17:43.468 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new 286ed6e0-c121-4ebb-8cf6-37e506a397aa 2 2024-09-17T21:17:43.668 INFO:teuthology.orchestra.run.smithi097.stdout:2 2024-09-17T21:17:43.684 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new b0af73a1-4bca-4892-a550-2ced87f4c357 3 2024-09-17T21:17:43.891 INFO:teuthology.orchestra.run.smithi097.stdout:3 2024-09-17T21:17:43.908 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new 2188361c-e1a1-45fa-bc01-da5a984ffd9f 4 2024-09-17T21:17:44.101 INFO:teuthology.orchestra.run.smithi097.stdout:4 2024-09-17T21:17:44.117 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new 7feccf4d-d8e4-4d06-ac83-bbc1bc623a5c 5 2024-09-17T21:17:44.309 INFO:teuthology.orchestra.run.smithi097.stdout:5 2024-09-17T21:17:44.325 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-17T21:17:44.325 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-17T21:17:44.327 INFO:tasks.ceph.osd.0:Started 2024-09-17T21:17:44.328 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-17T21:17:44.328 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-17T21:17:44.330 INFO:tasks.ceph.osd.1:Started 2024-09-17T21:17:44.332 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-17T21:17:44.332 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-17T21:17:44.334 INFO:tasks.ceph.osd.2:Started 2024-09-17T21:17:44.334 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-17T21:17:44.335 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-17T21:17:44.337 INFO:tasks.ceph.osd.3:Started 2024-09-17T21:17:44.337 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-17T21:17:44.337 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-17T21:17:44.339 INFO:tasks.ceph.osd.4:Started 2024-09-17T21:17:44.339 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-17T21:17:44.339 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-17T21:17:44.341 INFO:tasks.ceph.osd.5:Started 2024-09-17T21:17:44.341 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T21:17:44.543 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:17:44.544 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":8,"fsid":"224a7a29-e112-472e-8a13-f67345d6d123","created":"2024-09-17T21:17:42.597312+0000","modified":"2024-09-17T21:17:44.301318+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T21:17:44.301318+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5bfd41c9-0cac-47c4-9a6e-8dce774c3314","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"78404e0d-d397-4806-8e90-5f32182643bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"286ed6e0-c121-4ebb-8cf6-37e506a397aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b0af73a1-4bca-4892-a550-2ced87f4c357","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2188361c-e1a1-45fa-bc01-da5a984ffd9f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7feccf4d-d8e4-4d06-ac83-bbc1bc623a5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:17:44.560 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-17T21:17:44.561 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-17T21:17:45.167 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T21:17:45.358 INFO:teuthology.misc.health.smithi083.stdout: 2024-09-17T21:17:45.358 INFO:teuthology.misc.health.smithi083.stdout:{"epoch":8,"fsid":"224a7a29-e112-472e-8a13-f67345d6d123","created":"2024-09-17T21:17:42.597312+0000","modified":"2024-09-17T21:17:44.301318+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T21:17:44.301318+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5bfd41c9-0cac-47c4-9a6e-8dce774c3314","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"78404e0d-d397-4806-8e90-5f32182643bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"286ed6e0-c121-4ebb-8cf6-37e506a397aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b0af73a1-4bca-4892-a550-2ced87f4c357","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2188361c-e1a1-45fa-bc01-da5a984ffd9f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7feccf4d-d8e4-4d06-ac83-bbc1bc623a5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:17:45.374 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-17T21:17:45.490 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:17:45.485+0000 7f36c1f037c0 -1 Falling back to public interface 2024-09-17T21:17:45.506 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:17:45.501+0000 7f128f1cd7c0 -1 Falling back to public interface 2024-09-17T21:17:45.515 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:17:45.510+0000 7f2f6be647c0 -1 Falling back to public interface 2024-09-17T21:17:45.577 INFO:tasks.ceph.mgr.x.smithi097.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-17T21:17:45.577 INFO:tasks.ceph.mgr.x.smithi097.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-17T21:17:45.577 INFO:tasks.ceph.mgr.x.smithi097.stderr: from numpy import show_config as show_numpy_config 2024-09-17T21:17:45.691 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:17:45.686+0000 7f4baec387c0 -1 Falling back to public interface 2024-09-17T21:17:46.004 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:17:45.998+0000 7f2b0962a7c0 -1 Falling back to public interface 2024-09-17T21:17:46.202 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:17:46.197+0000 7f8569fd87c0 -1 Falling back to public interface 2024-09-17T21:17:49.399 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:17:49.393+0000 7f36c1f037c0 -1 osd.2 0 log_to_monitors true 2024-09-17T21:17:49.526 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:17:49.522+0000 7f2f6be647c0 -1 osd.4 0 log_to_monitors true 2024-09-17T21:17:49.553 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:17:49.549+0000 7f128f1cd7c0 -1 osd.1 0 log_to_monitors true 2024-09-17T21:17:49.704 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:17:49.698+0000 7f4baec387c0 -1 osd.3 0 log_to_monitors true 2024-09-17T21:17:50.135 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:17:50.130+0000 7f2b0962a7c0 -1 osd.5 0 log_to_monitors true 2024-09-17T21:17:50.136 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:17:50.129+0000 7f8569fd87c0 -1 osd.0 0 log_to_monitors true 2024-09-17T21:17:50.546 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-09-17T21:17:50.542+0000 7f33c3358640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-17T21:17:50.552 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-09-17T21:17:50.546+0000 7f33c3358640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-17T21:17:51.691 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:17:51.685+0000 7f36be6be640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T21:17:51.771 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:17:51.765+0000 7f36b5cde640 -1 osd.2 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T21:17:51.979 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T21:17:52.336 INFO:teuthology.misc.health.smithi083.stdout: 2024-09-17T21:17:52.336 INFO:teuthology.misc.health.smithi083.stdout:{"epoch":11,"fsid":"224a7a29-e112-472e-8a13-f67345d6d123","created":"2024-09-17T21:17:42.597312+0000","modified":"2024-09-17T21:17:51.476609+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T21:17:44.301318+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5bfd41c9-0cac-47c4-9a6e-8dce774c3314","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"78404e0d-d397-4806-8e90-5f32182643bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"286ed6e0-c121-4ebb-8cf6-37e506a397aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b0af73a1-4bca-4892-a550-2ced87f4c357","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2188361c-e1a1-45fa-bc01-da5a984ffd9f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7feccf4d-d8e4-4d06-ac83-bbc1bc623a5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:17:52.358 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-17T21:17:52.965 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:17:52.961+0000 7f8566793640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T21:17:52.975 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:17:52.969+0000 7f128b176640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T21:17:52.993 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:17:52.989+0000 7f855ddb3640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T21:17:52.997 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:17:52.993+0000 7f1282fa8640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T21:17:53.456 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:17:53.450+0000 7f4bab3f3640 -1 osd.3 0 waiting for initial osdmap 2024-09-17T21:17:53.462 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:17:53.458+0000 7f2f67e0d640 -1 osd.4 0 waiting for initial osdmap 2024-09-17T21:17:53.464 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:17:53.458+0000 7f2b055d3640 -1 osd.5 0 waiting for initial osdmap 2024-09-17T21:17:53.486 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:17:53.482+0000 7f4ba2a13640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T21:17:53.498 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:17:53.494+0000 7f2f5fc3f640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T21:17:53.502 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:17:53.498+0000 7f2afd405640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T21:17:58.964 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T21:17:59.271 INFO:teuthology.misc.health.smithi083.stdout: 2024-09-17T21:17:59.271 INFO:teuthology.misc.health.smithi083.stdout:{"epoch":17,"fsid":"224a7a29-e112-472e-8a13-f67345d6d123","created":"2024-09-17T21:17:42.597312+0000","modified":"2024-09-17T21:17:58.004182+0000","last_up_change":"2024-09-17T21:17:53.976897+0000","last_in_change":"2024-09-17T21:17:44.301318+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T21:17:55.581350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5bfd41c9-0cac-47c4-9a6e-8dce774c3314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6817","nonce":1111036754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6819","nonce":1111036754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6823","nonce":1111036754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6821","nonce":1111036754}]},"public_addr":"172.21.15.83:6817/1111036754","cluster_addr":"172.21.15.83:6819/1111036754","heartbeat_back_addr":"172.21.15.83:6823/1111036754","heartbeat_front_addr":"172.21.15.83:6821/1111036754","state":["exists","up"]},{"osd":1,"uuid":"78404e0d-d397-4806-8e90-5f32182643bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6809","nonce":1460363937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6811","nonce":1460363937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6815","nonce":1460363937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6813","nonce":1460363937}]},"public_addr":"172.21.15.83:6809/1460363937","cluster_addr":"172.21.15.83:6811/1460363937","heartbeat_back_addr":"172.21.15.83:6815/1460363937","heartbeat_front_addr":"172.21.15.83:6813/1460363937","state":["exists","up"]},{"osd":2,"uuid":"286ed6e0-c121-4ebb-8cf6-37e506a397aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2837444034},{"type":"v1","addr":"172.21.15.83:6801","nonce":2837444034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2837444034},{"type":"v1","addr":"172.21.15.83:6803","nonce":2837444034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2837444034},{"type":"v1","addr":"172.21.15.83:6807","nonce":2837444034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2837444034},{"type":"v1","addr":"172.21.15.83:6805","nonce":2837444034}]},"public_addr":"172.21.15.83:6801/2837444034","cluster_addr":"172.21.15.83:6803/2837444034","heartbeat_back_addr":"172.21.15.83:6807/2837444034","heartbeat_front_addr":"172.21.15.83:6805/2837444034","state":["exists","up"]},{"osd":3,"uuid":"b0af73a1-4bca-4892-a550-2ced87f4c357","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6809","nonce":3637204865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6811","nonce":3637204865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6815","nonce":3637204865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6813","nonce":3637204865}]},"public_addr":"172.21.15.97:6809/3637204865","cluster_addr":"172.21.15.97:6811/3637204865","heartbeat_back_addr":"172.21.15.97:6815/3637204865","heartbeat_front_addr":"172.21.15.97:6813/3637204865","state":["exists","up"]},{"osd":4,"uuid":"2188361c-e1a1-45fa-bc01-da5a984ffd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6801","nonce":2391356911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6803","nonce":2391356911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6807","nonce":2391356911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6805","nonce":2391356911}]},"public_addr":"172.21.15.97:6801/2391356911","cluster_addr":"172.21.15.97:6803/2391356911","heartbeat_back_addr":"172.21.15.97:6807/2391356911","heartbeat_front_addr":"172.21.15.97:6805/2391356911","state":["exists","up"]},{"osd":5,"uuid":"7feccf4d-d8e4-4d06-ac83-bbc1bc623a5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6817","nonce":2326316381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6819","nonce":2326316381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6823","nonce":2326316381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6821","nonce":2326316381}]},"public_addr":"172.21.15.97:6817/2326316381","cluster_addr":"172.21.15.97:6819/2326316381","heartbeat_back_addr":"172.21.15.97:6823/2326316381","heartbeat_front_addr":"172.21.15.97:6821/2326316381","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:51.138101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.581632+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.365193+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.718974+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.501878+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:51.150747+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:17:59.289 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-09-17T21:17:59.289 INFO:tasks.ceph:Creating RBD pool 2024-09-17T21:17:59.289 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-17T21:18:00.030 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'rbd' created 2024-09-17T21:18:00.056 DEBUG:teuthology.orchestra.run.smithi083:> rbd --cluster ceph pool init rbd 2024-09-17T21:18:03.084 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-17T21:18:03.085 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-17T21:18:03.085 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-17T21:18:03.396 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:18:03.415 INFO:teuthology.orchestra.run.smithi083.stdout:[{"version":7,"timestamp":"2024-09-17T21:17:53.474293+0000","name":"","changes":[{"name":"osd.5/osd_mclock_max_capacity_iops_ssd","new_value":"3995.211973"}]}] 2024-09-17T21:18:03.415 INFO:tasks.ceph_manager:config epoch is 7 2024-09-17T21:18:03.415 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T21:18:03.415 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-17T21:18:03.416 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-17T21:18:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:18:03.774 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":434723260},{"type":"v1","addr":"172.21.15.97:6825","nonce":434723260}]},"active_addr":"172.21.15.97:6825/434723260","active_change":"2024-09-17T21:17:49.524960+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":3563088051}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":669728366}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":2382780893}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":3143855729}]}]} 2024-09-17T21:18:03.776 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-17T21:18:03.777 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T21:18:03.777 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T21:18:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:18:04.075 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"fsid":"224a7a29-e112-472e-8a13-f67345d6d123","created":"2024-09-17T21:17:42.597312+0000","modified":"2024-09-17T21:18:03.048057+0000","last_up_change":"2024-09-17T21:17:53.976897+0000","last_in_change":"2024-09-17T21:17:44.301318+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T21:17:55.581350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T21:17:59.603025+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5bfd41c9-0cac-47c4-9a6e-8dce774c3314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6817","nonce":1111036754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6819","nonce":1111036754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6823","nonce":1111036754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6821","nonce":1111036754}]},"public_addr":"172.21.15.83:6817/1111036754","cluster_addr":"172.21.15.83:6819/1111036754","heartbeat_back_addr":"172.21.15.83:6823/1111036754","heartbeat_front_addr":"172.21.15.83:6821/1111036754","state":["exists","up"]},{"osd":1,"uuid":"78404e0d-d397-4806-8e90-5f32182643bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6809","nonce":1460363937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6811","nonce":1460363937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6815","nonce":1460363937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6813","nonce":1460363937}]},"public_addr":"172.21.15.83:6809/1460363937","cluster_addr":"172.21.15.83:6811/1460363937","heartbeat_back_addr":"172.21.15.83:6815/1460363937","heartbeat_front_addr":"172.21.15.83:6813/1460363937","state":["exists","up"]},{"osd":2,"uuid":"286ed6e0-c121-4ebb-8cf6-37e506a397aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2837444034},{"type":"v1","addr":"172.21.15.83:6801","nonce":2837444034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2837444034},{"type":"v1","addr":"172.21.15.83:6803","nonce":2837444034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2837444034},{"type":"v1","addr":"172.21.15.83:6807","nonce":2837444034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2837444034},{"type":"v1","addr":"172.21.15.83:6805","nonce":2837444034}]},"public_addr":"172.21.15.83:6801/2837444034","cluster_addr":"172.21.15.83:6803/2837444034","heartbeat_back_addr":"172.21.15.83:6807/2837444034","heartbeat_front_addr":"172.21.15.83:6805/2837444034","state":["exists","up"]},{"osd":3,"uuid":"b0af73a1-4bca-4892-a550-2ced87f4c357","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6809","nonce":3637204865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6811","nonce":3637204865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6815","nonce":3637204865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6813","nonce":3637204865}]},"public_addr":"172.21.15.97:6809/3637204865","cluster_addr":"172.21.15.97:6811/3637204865","heartbeat_back_addr":"172.21.15.97:6815/3637204865","heartbeat_front_addr":"172.21.15.97:6813/3637204865","state":["exists","up"]},{"osd":4,"uuid":"2188361c-e1a1-45fa-bc01-da5a984ffd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6801","nonce":2391356911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6803","nonce":2391356911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6807","nonce":2391356911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6805","nonce":2391356911}]},"public_addr":"172.21.15.97:6801/2391356911","cluster_addr":"172.21.15.97:6803/2391356911","heartbeat_back_addr":"172.21.15.97:6807/2391356911","heartbeat_front_addr":"172.21.15.97:6805/2391356911","state":["exists","up"]},{"osd":5,"uuid":"7feccf4d-d8e4-4d06-ac83-bbc1bc623a5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6817","nonce":2326316381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6819","nonce":2326316381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6823","nonce":2326316381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6821","nonce":2326316381}]},"public_addr":"172.21.15.97:6817/2326316381","cluster_addr":"172.21.15.97:6819/2326316381","heartbeat_back_addr":"172.21.15.97:6823/2326316381","heartbeat_front_addr":"172.21.15.97:6821/2326316381","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:51.138101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.581632+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.365193+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.718974+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.501878+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:51.150747+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:18:04.092 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T21:18:04.092 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T21:18:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:18:04.394 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"fsid":"224a7a29-e112-472e-8a13-f67345d6d123","created":"2024-09-17T21:17:42.597312+0000","modified":"2024-09-17T21:18:03.048057+0000","last_up_change":"2024-09-17T21:17:53.976897+0000","last_in_change":"2024-09-17T21:17:44.301318+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T21:17:55.581350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T21:17:59.603025+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5bfd41c9-0cac-47c4-9a6e-8dce774c3314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6817","nonce":1111036754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6819","nonce":1111036754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6823","nonce":1111036754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6821","nonce":1111036754}]},"public_addr":"172.21.15.83:6817/1111036754","cluster_addr":"172.21.15.83:6819/1111036754","heartbeat_back_addr":"172.21.15.83:6823/1111036754","heartbeat_front_addr":"172.21.15.83:6821/1111036754","state":["exists","up"]},{"osd":1,"uuid":"78404e0d-d397-4806-8e90-5f32182643bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6809","nonce":1460363937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6811","nonce":1460363937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6815","nonce":1460363937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6813","nonce":1460363937}]},"public_addr":"172.21.15.83:6809/1460363937","cluster_addr":"172.21.15.83:6811/1460363937","heartbeat_back_addr":"172.21.15.83:6815/1460363937","heartbeat_front_addr":"172.21.15.83:6813/1460363937","state":["exists","up"]},{"osd":2,"uuid":"286ed6e0-c121-4ebb-8cf6-37e506a397aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2837444034},{"type":"v1","addr":"172.21.15.83:6801","nonce":2837444034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2837444034},{"type":"v1","addr":"172.21.15.83:6803","nonce":2837444034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2837444034},{"type":"v1","addr":"172.21.15.83:6807","nonce":2837444034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2837444034},{"type":"v1","addr":"172.21.15.83:6805","nonce":2837444034}]},"public_addr":"172.21.15.83:6801/2837444034","cluster_addr":"172.21.15.83:6803/2837444034","heartbeat_back_addr":"172.21.15.83:6807/2837444034","heartbeat_front_addr":"172.21.15.83:6805/2837444034","state":["exists","up"]},{"osd":3,"uuid":"b0af73a1-4bca-4892-a550-2ced87f4c357","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6809","nonce":3637204865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6811","nonce":3637204865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6815","nonce":3637204865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6813","nonce":3637204865}]},"public_addr":"172.21.15.97:6809/3637204865","cluster_addr":"172.21.15.97:6811/3637204865","heartbeat_back_addr":"172.21.15.97:6815/3637204865","heartbeat_front_addr":"172.21.15.97:6813/3637204865","state":["exists","up"]},{"osd":4,"uuid":"2188361c-e1a1-45fa-bc01-da5a984ffd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6801","nonce":2391356911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6803","nonce":2391356911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6807","nonce":2391356911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6805","nonce":2391356911}]},"public_addr":"172.21.15.97:6801/2391356911","cluster_addr":"172.21.15.97:6803/2391356911","heartbeat_back_addr":"172.21.15.97:6807/2391356911","heartbeat_front_addr":"172.21.15.97:6805/2391356911","state":["exists","up"]},{"osd":5,"uuid":"7feccf4d-d8e4-4d06-ac83-bbc1bc623a5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6817","nonce":2326316381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6819","nonce":2326316381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6823","nonce":2326316381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6821","nonce":2326316381}]},"public_addr":"172.21.15.97:6817/2326316381","cluster_addr":"172.21.15.97:6819/2326316381","heartbeat_back_addr":"172.21.15.97:6823/2326316381","heartbeat_front_addr":"172.21.15.97:6821/2326316381","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:51.138101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.581632+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.365193+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.718974+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.501878+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:51.150747+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:18:04.413 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T21:18:04.413 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T21:18:04.414 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T21:18:04.414 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T21:18:04.415 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T21:18:04.415 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T21:18:04.582 INFO:teuthology.orchestra.run.smithi083.stdout:55834574853 2024-09-17T21:18:04.582 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T21:18:04.586 INFO:teuthology.orchestra.run.smithi083.stdout:55834574853 2024-09-17T21:18:04.587 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T21:18:04.638 INFO:teuthology.orchestra.run.smithi083.stdout:51539607558 2024-09-17T21:18:04.638 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T21:18:04.641 INFO:teuthology.orchestra.run.smithi083.stdout:55834574853 2024-09-17T21:18:04.641 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T21:18:04.652 INFO:teuthology.orchestra.run.smithi083.stdout:55834574852 2024-09-17T21:18:04.653 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T21:18:04.656 INFO:teuthology.orchestra.run.smithi083.stdout:55834574853 2024-09-17T21:18:04.656 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T21:18:04.941 INFO:teuthology.orchestra.run.smithi083.stdout:55834574851 2024-09-17T21:18:04.959 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574851 for osd.3 2024-09-17T21:18:05.050 INFO:teuthology.orchestra.run.smithi083.stdout:55834574850 2024-09-17T21:18:05.061 INFO:teuthology.orchestra.run.smithi083.stdout:51539607556 2024-09-17T21:18:05.069 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574850 for osd.5 2024-09-17T21:18:05.081 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607556 for osd.2 2024-09-17T21:18:05.095 INFO:teuthology.orchestra.run.smithi083.stdout:55834574851 2024-09-17T21:18:05.111 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574851 for osd.0 2024-09-17T21:18:05.114 INFO:teuthology.orchestra.run.smithi083.stdout:55834574851 2024-09-17T21:18:05.129 INFO:teuthology.orchestra.run.smithi083.stdout:55834574851 2024-09-17T21:18:05.132 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574851 for osd.1 2024-09-17T21:18:05.145 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574851 for osd.4 2024-09-17T21:18:05.960 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T21:18:06.070 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T21:18:06.082 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T21:18:06.112 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T21:18:06.133 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T21:18:06.146 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T21:18:06.370 INFO:teuthology.orchestra.run.smithi083.stdout:55834574853 2024-09-17T21:18:06.399 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2024-09-17T21:18:06.399 DEBUG:teuthology.parallel:result is None 2024-09-17T21:18:06.416 INFO:teuthology.orchestra.run.smithi083.stdout:55834574852 2024-09-17T21:18:06.435 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.5 2024-09-17T21:18:06.435 DEBUG:teuthology.parallel:result is None 2024-09-17T21:18:06.463 INFO:teuthology.orchestra.run.smithi083.stdout:55834574853 2024-09-17T21:18:06.480 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2024-09-17T21:18:06.480 DEBUG:teuthology.parallel:result is None 2024-09-17T21:18:06.481 INFO:teuthology.orchestra.run.smithi083.stdout:51539607558 2024-09-17T21:18:06.498 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607558 for osd.2 2024-09-17T21:18:06.499 DEBUG:teuthology.parallel:result is None 2024-09-17T21:18:06.521 INFO:teuthology.orchestra.run.smithi083.stdout:55834574853 2024-09-17T21:18:06.538 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.4 2024-09-17T21:18:06.538 DEBUG:teuthology.parallel:result is None 2024-09-17T21:18:06.580 INFO:teuthology.orchestra.run.smithi083.stdout:55834574853 2024-09-17T21:18:06.597 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2024-09-17T21:18:06.597 DEBUG:teuthology.parallel:result is None 2024-09-17T21:18:06.597 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T21:18:06.597 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:18:06.912 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:18:06.913 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T21:18:06.928 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-17T21:18:05.535285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":168680,"kb_used_data":2320,"kb_used_omap":9,"kb_used_meta":166198,"kb_avail":566062360,"statfs":{"total":579820584960,"available":579647856640,"internally_reserved":0,"allocated":2375680,"data_stored":1573246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9574,"internal_metadata":170187418},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.500981"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.059715+0000","last_change":"2024-09-17T21:18:03.059715+0000","last_active":"2024-09-17T21:18:03.059715+0000","last_peered":"2024-09-17T21:18:03.059715+0000","last_clean":"2024-09-17T21:18:03.059715+0000","last_became_active":"2024-09-17T21:18:01.213047+0000","last_became_peered":"2024-09-17T21:18:01.213047+0000","last_unstale":"2024-09-17T21:18:03.059715+0000","last_undegraded":"2024-09-17T21:18:03.059715+0000","last_fullsized":"2024-09-17T21:18:03.059715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:46:35.222373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.073440+0000","last_change":"2024-09-17T21:18:03.073440+0000","last_active":"2024-09-17T21:18:03.073440+0000","last_peered":"2024-09-17T21:18:03.073440+0000","last_clean":"2024-09-17T21:18:03.073440+0000","last_became_active":"2024-09-17T21:18:01.048806+0000","last_became_peered":"2024-09-17T21:18:01.048806+0000","last_unstale":"2024-09-17T21:18:03.073440+0000","last_undegraded":"2024-09-17T21:18:03.073440+0000","last_fullsized":"2024-09-17T21:18:03.073440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:26.487163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094161400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.058986+0000","last_change":"2024-09-17T21:18:03.058986+0000","last_active":"2024-09-17T21:18:03.058986+0000","last_peered":"2024-09-17T21:18:03.058986+0000","last_clean":"2024-09-17T21:18:03.058986+0000","last_became_active":"2024-09-17T21:18:01.214272+0000","last_became_peered":"2024-09-17T21:18:01.214272+0000","last_unstale":"2024-09-17T21:18:03.058986+0000","last_undegraded":"2024-09-17T21:18:03.058986+0000","last_fullsized":"2024-09-17T21:18:03.058986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:04:38.082107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099219900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.073508+0000","last_change":"2024-09-17T21:18:03.073508+0000","last_active":"2024-09-17T21:18:03.073508+0000","last_peered":"2024-09-17T21:18:03.073508+0000","last_clean":"2024-09-17T21:18:03.073508+0000","last_became_active":"2024-09-17T21:18:01.213705+0000","last_became_peered":"2024-09-17T21:18:01.213705+0000","last_unstale":"2024-09-17T21:18:03.073508+0000","last_undegraded":"2024-09-17T21:18:03.073508+0000","last_fullsized":"2024-09-17T21:18:03.073508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:46:08.811379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006222,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.071880+0000","last_change":"2024-09-17T21:18:03.070336+0000","last_active":"2024-09-17T21:18:03.071880+0000","last_peered":"2024-09-17T21:18:03.071880+0000","last_clean":"2024-09-17T21:18:03.071880+0000","last_became_active":"2024-09-17T21:18:01.049336+0000","last_became_peered":"2024-09-17T21:18:01.049336+0000","last_unstale":"2024-09-17T21:18:03.071880+0000","last_undegraded":"2024-09-17T21:18:03.071880+0000","last_fullsized":"2024-09-17T21:18:03.071880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:56:24.329470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023986049999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.060818+0000","last_change":"2024-09-17T21:18:03.060818+0000","last_active":"2024-09-17T21:18:03.060818+0000","last_peered":"2024-09-17T21:18:03.060818+0000","last_clean":"2024-09-17T21:18:03.060818+0000","last_became_active":"2024-09-17T21:18:01.048506+0000","last_became_peered":"2024-09-17T21:18:01.048506+0000","last_unstale":"2024-09-17T21:18:03.060818+0000","last_undegraded":"2024-09-17T21:18:03.060818+0000","last_fullsized":"2024-09-17T21:18:03.060818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:47:10.439018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096175600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.059156+0000","last_change":"2024-09-17T21:18:03.059156+0000","last_active":"2024-09-17T21:18:03.059156+0000","last_peered":"2024-09-17T21:18:03.059156+0000","last_clean":"2024-09-17T21:18:03.059156+0000","last_became_active":"2024-09-17T21:18:01.049180+0000","last_became_peered":"2024-09-17T21:18:01.049180+0000","last_unstale":"2024-09-17T21:18:03.059156+0000","last_undegraded":"2024-09-17T21:18:03.059156+0000","last_fullsized":"2024-09-17T21:18:03.059156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:27.069794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077408,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.069091+0000","last_change":"2024-09-17T21:18:03.069091+0000","last_active":"2024-09-17T21:18:03.069091+0000","last_peered":"2024-09-17T21:18:03.069091+0000","last_clean":"2024-09-17T21:18:03.069091+0000","last_became_active":"2024-09-17T21:18:01.044275+0000","last_became_peered":"2024-09-17T21:18:01.044275+0000","last_unstale":"2024-09-17T21:18:03.069091+0000","last_undegraded":"2024-09-17T21:18:03.069091+0000","last_fullsized":"2024-09-17T21:18:03.069091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:28:27.066081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001145884,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.057898+0000","last_change":"2024-09-17T21:17:57.194934+0000","last_active":"2024-09-17T21:18:03.057898+0000","last_peered":"2024-09-17T21:18:03.057898+0000","last_clean":"2024-09-17T21:18:03.057898+0000","last_became_active":"2024-09-17T21:17:57.192405+0000","last_became_peered":"2024-09-17T21:17:57.192405+0000","last_unstale":"2024-09-17T21:18:03.057898+0000","last_undegraded":"2024-09-17T21:18:03.057898+0000","last_fullsized":"2024-09-17T21:18:03.057898+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_clean_scrub_stamp":"2024-09-17T21:17:55.982339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:30.628595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":200704,"data_stored":65431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":65412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":790528,"data_stored":655780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":200704,"data_stored":65431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":790528,"data_stored":655780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":196608,"data_stored":65412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:18:06.929 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:18:07.210 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:18:07.211 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T21:18:07.227 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-17T21:18:05.535285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":168680,"kb_used_data":2320,"kb_used_omap":9,"kb_used_meta":166198,"kb_avail":566062360,"statfs":{"total":579820584960,"available":579647856640,"internally_reserved":0,"allocated":2375680,"data_stored":1573246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9574,"internal_metadata":170187418},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.500981"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.059715+0000","last_change":"2024-09-17T21:18:03.059715+0000","last_active":"2024-09-17T21:18:03.059715+0000","last_peered":"2024-09-17T21:18:03.059715+0000","last_clean":"2024-09-17T21:18:03.059715+0000","last_became_active":"2024-09-17T21:18:01.213047+0000","last_became_peered":"2024-09-17T21:18:01.213047+0000","last_unstale":"2024-09-17T21:18:03.059715+0000","last_undegraded":"2024-09-17T21:18:03.059715+0000","last_fullsized":"2024-09-17T21:18:03.059715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:46:35.222373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.073440+0000","last_change":"2024-09-17T21:18:03.073440+0000","last_active":"2024-09-17T21:18:03.073440+0000","last_peered":"2024-09-17T21:18:03.073440+0000","last_clean":"2024-09-17T21:18:03.073440+0000","last_became_active":"2024-09-17T21:18:01.048806+0000","last_became_peered":"2024-09-17T21:18:01.048806+0000","last_unstale":"2024-09-17T21:18:03.073440+0000","last_undegraded":"2024-09-17T21:18:03.073440+0000","last_fullsized":"2024-09-17T21:18:03.073440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:26.487163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094161400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.058986+0000","last_change":"2024-09-17T21:18:03.058986+0000","last_active":"2024-09-17T21:18:03.058986+0000","last_peered":"2024-09-17T21:18:03.058986+0000","last_clean":"2024-09-17T21:18:03.058986+0000","last_became_active":"2024-09-17T21:18:01.214272+0000","last_became_peered":"2024-09-17T21:18:01.214272+0000","last_unstale":"2024-09-17T21:18:03.058986+0000","last_undegraded":"2024-09-17T21:18:03.058986+0000","last_fullsized":"2024-09-17T21:18:03.058986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:04:38.082107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099219900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.073508+0000","last_change":"2024-09-17T21:18:03.073508+0000","last_active":"2024-09-17T21:18:03.073508+0000","last_peered":"2024-09-17T21:18:03.073508+0000","last_clean":"2024-09-17T21:18:03.073508+0000","last_became_active":"2024-09-17T21:18:01.213705+0000","last_became_peered":"2024-09-17T21:18:01.213705+0000","last_unstale":"2024-09-17T21:18:03.073508+0000","last_undegraded":"2024-09-17T21:18:03.073508+0000","last_fullsized":"2024-09-17T21:18:03.073508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:46:08.811379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006222,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.071880+0000","last_change":"2024-09-17T21:18:03.070336+0000","last_active":"2024-09-17T21:18:03.071880+0000","last_peered":"2024-09-17T21:18:03.071880+0000","last_clean":"2024-09-17T21:18:03.071880+0000","last_became_active":"2024-09-17T21:18:01.049336+0000","last_became_peered":"2024-09-17T21:18:01.049336+0000","last_unstale":"2024-09-17T21:18:03.071880+0000","last_undegraded":"2024-09-17T21:18:03.071880+0000","last_fullsized":"2024-09-17T21:18:03.071880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:56:24.329470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023986049999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.060818+0000","last_change":"2024-09-17T21:18:03.060818+0000","last_active":"2024-09-17T21:18:03.060818+0000","last_peered":"2024-09-17T21:18:03.060818+0000","last_clean":"2024-09-17T21:18:03.060818+0000","last_became_active":"2024-09-17T21:18:01.048506+0000","last_became_peered":"2024-09-17T21:18:01.048506+0000","last_unstale":"2024-09-17T21:18:03.060818+0000","last_undegraded":"2024-09-17T21:18:03.060818+0000","last_fullsized":"2024-09-17T21:18:03.060818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:47:10.439018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096175600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.059156+0000","last_change":"2024-09-17T21:18:03.059156+0000","last_active":"2024-09-17T21:18:03.059156+0000","last_peered":"2024-09-17T21:18:03.059156+0000","last_clean":"2024-09-17T21:18:03.059156+0000","last_became_active":"2024-09-17T21:18:01.049180+0000","last_became_peered":"2024-09-17T21:18:01.049180+0000","last_unstale":"2024-09-17T21:18:03.059156+0000","last_undegraded":"2024-09-17T21:18:03.059156+0000","last_fullsized":"2024-09-17T21:18:03.059156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:27.069794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077408,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.069091+0000","last_change":"2024-09-17T21:18:03.069091+0000","last_active":"2024-09-17T21:18:03.069091+0000","last_peered":"2024-09-17T21:18:03.069091+0000","last_clean":"2024-09-17T21:18:03.069091+0000","last_became_active":"2024-09-17T21:18:01.044275+0000","last_became_peered":"2024-09-17T21:18:01.044275+0000","last_unstale":"2024-09-17T21:18:03.069091+0000","last_undegraded":"2024-09-17T21:18:03.069091+0000","last_fullsized":"2024-09-17T21:18:03.069091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:28:27.066081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001145884,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.057898+0000","last_change":"2024-09-17T21:17:57.194934+0000","last_active":"2024-09-17T21:18:03.057898+0000","last_peered":"2024-09-17T21:18:03.057898+0000","last_clean":"2024-09-17T21:18:03.057898+0000","last_became_active":"2024-09-17T21:17:57.192405+0000","last_became_peered":"2024-09-17T21:17:57.192405+0000","last_unstale":"2024-09-17T21:18:03.057898+0000","last_undegraded":"2024-09-17T21:18:03.057898+0000","last_fullsized":"2024-09-17T21:18:03.057898+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_clean_scrub_stamp":"2024-09-17T21:17:55.982339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:30.628595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":200704,"data_stored":65431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":65412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":790528,"data_stored":655780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":200704,"data_stored":65431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":790528,"data_stored":655780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":196608,"data_stored":65412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:18:07.228 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T21:18:07.228 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T21:18:07.228 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-17T21:18:07.228 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T21:18:07.537 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:18:07.538 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T21:18:07.555 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-17T21:18:07.555 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-17T21:18:07.565 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-17T21:18:07.566 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '08cec6974d86f81dc304360b31113f30b4085bf8'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '08cec6974d86f81dc304360b31113f30b4085bf8'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '5ecc7400dbbb63ce990b663d2a92e87dbc17f30d'}} 2024-09-17T21:18:07.566 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-17T21:18:07.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-17T21:18:07.877 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-17T21:18:07.877 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T21:18:07.878 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:18:08.162 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:18:08.162 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T21:18:08.179 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-17T21:18:07.535860+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":168680,"kb_used_data":2320,"kb_used_omap":9,"kb_used_meta":166198,"kb_avail":566062360,"statfs":{"total":579820584960,"available":579647856640,"internally_reserved":0,"allocated":2375680,"data_stored":1573246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9574,"internal_metadata":170187418},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001658"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.059715+0000","last_change":"2024-09-17T21:18:03.059715+0000","last_active":"2024-09-17T21:18:03.059715+0000","last_peered":"2024-09-17T21:18:03.059715+0000","last_clean":"2024-09-17T21:18:03.059715+0000","last_became_active":"2024-09-17T21:18:01.213047+0000","last_became_peered":"2024-09-17T21:18:01.213047+0000","last_unstale":"2024-09-17T21:18:03.059715+0000","last_undegraded":"2024-09-17T21:18:03.059715+0000","last_fullsized":"2024-09-17T21:18:03.059715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:46:35.222373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.073440+0000","last_change":"2024-09-17T21:18:03.073440+0000","last_active":"2024-09-17T21:18:03.073440+0000","last_peered":"2024-09-17T21:18:03.073440+0000","last_clean":"2024-09-17T21:18:03.073440+0000","last_became_active":"2024-09-17T21:18:01.048806+0000","last_became_peered":"2024-09-17T21:18:01.048806+0000","last_unstale":"2024-09-17T21:18:03.073440+0000","last_undegraded":"2024-09-17T21:18:03.073440+0000","last_fullsized":"2024-09-17T21:18:03.073440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:26.487163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094161400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.058986+0000","last_change":"2024-09-17T21:18:03.058986+0000","last_active":"2024-09-17T21:18:03.058986+0000","last_peered":"2024-09-17T21:18:03.058986+0000","last_clean":"2024-09-17T21:18:03.058986+0000","last_became_active":"2024-09-17T21:18:01.214272+0000","last_became_peered":"2024-09-17T21:18:01.214272+0000","last_unstale":"2024-09-17T21:18:03.058986+0000","last_undegraded":"2024-09-17T21:18:03.058986+0000","last_fullsized":"2024-09-17T21:18:03.058986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:04:38.082107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099219900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.073508+0000","last_change":"2024-09-17T21:18:03.073508+0000","last_active":"2024-09-17T21:18:03.073508+0000","last_peered":"2024-09-17T21:18:03.073508+0000","last_clean":"2024-09-17T21:18:03.073508+0000","last_became_active":"2024-09-17T21:18:01.213705+0000","last_became_peered":"2024-09-17T21:18:01.213705+0000","last_unstale":"2024-09-17T21:18:03.073508+0000","last_undegraded":"2024-09-17T21:18:03.073508+0000","last_fullsized":"2024-09-17T21:18:03.073508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:46:08.811379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006222,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.071880+0000","last_change":"2024-09-17T21:18:03.070336+0000","last_active":"2024-09-17T21:18:03.071880+0000","last_peered":"2024-09-17T21:18:03.071880+0000","last_clean":"2024-09-17T21:18:03.071880+0000","last_became_active":"2024-09-17T21:18:01.049336+0000","last_became_peered":"2024-09-17T21:18:01.049336+0000","last_unstale":"2024-09-17T21:18:03.071880+0000","last_undegraded":"2024-09-17T21:18:03.071880+0000","last_fullsized":"2024-09-17T21:18:03.071880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:56:24.329470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023986049999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.060818+0000","last_change":"2024-09-17T21:18:03.060818+0000","last_active":"2024-09-17T21:18:03.060818+0000","last_peered":"2024-09-17T21:18:03.060818+0000","last_clean":"2024-09-17T21:18:03.060818+0000","last_became_active":"2024-09-17T21:18:01.048506+0000","last_became_peered":"2024-09-17T21:18:01.048506+0000","last_unstale":"2024-09-17T21:18:03.060818+0000","last_undegraded":"2024-09-17T21:18:03.060818+0000","last_fullsized":"2024-09-17T21:18:03.060818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:47:10.439018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096175600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.059156+0000","last_change":"2024-09-17T21:18:03.059156+0000","last_active":"2024-09-17T21:18:03.059156+0000","last_peered":"2024-09-17T21:18:03.059156+0000","last_clean":"2024-09-17T21:18:03.059156+0000","last_became_active":"2024-09-17T21:18:01.049180+0000","last_became_peered":"2024-09-17T21:18:01.049180+0000","last_unstale":"2024-09-17T21:18:03.059156+0000","last_undegraded":"2024-09-17T21:18:03.059156+0000","last_fullsized":"2024-09-17T21:18:03.059156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:27.069794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077408,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.069091+0000","last_change":"2024-09-17T21:18:03.069091+0000","last_active":"2024-09-17T21:18:03.069091+0000","last_peered":"2024-09-17T21:18:03.069091+0000","last_clean":"2024-09-17T21:18:03.069091+0000","last_became_active":"2024-09-17T21:18:01.044275+0000","last_became_peered":"2024-09-17T21:18:01.044275+0000","last_unstale":"2024-09-17T21:18:03.069091+0000","last_undegraded":"2024-09-17T21:18:03.069091+0000","last_fullsized":"2024-09-17T21:18:03.069091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:28:27.066081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001145884,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.057898+0000","last_change":"2024-09-17T21:17:57.194934+0000","last_active":"2024-09-17T21:18:03.057898+0000","last_peered":"2024-09-17T21:18:03.057898+0000","last_clean":"2024-09-17T21:18:03.057898+0000","last_became_active":"2024-09-17T21:17:57.192405+0000","last_became_peered":"2024-09-17T21:17:57.192405+0000","last_unstale":"2024-09-17T21:18:03.057898+0000","last_undegraded":"2024-09-17T21:18:03.057898+0000","last_fullsized":"2024-09-17T21:18:03.057898+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_clean_scrub_stamp":"2024-09-17T21:17:55.982339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:30.628595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":200704,"data_stored":65431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":65412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":790528,"data_stored":655780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":200704,"data_stored":65431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":790528,"data_stored":655780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":196608,"data_stored":65412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:18:08.180 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:18:08.456 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:18:08.456 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T21:18:08.472 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-17T21:18:07.535860+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":168680,"kb_used_data":2320,"kb_used_omap":9,"kb_used_meta":166198,"kb_avail":566062360,"statfs":{"total":579820584960,"available":579647856640,"internally_reserved":0,"allocated":2375680,"data_stored":1573246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9574,"internal_metadata":170187418},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001658"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.059715+0000","last_change":"2024-09-17T21:18:03.059715+0000","last_active":"2024-09-17T21:18:03.059715+0000","last_peered":"2024-09-17T21:18:03.059715+0000","last_clean":"2024-09-17T21:18:03.059715+0000","last_became_active":"2024-09-17T21:18:01.213047+0000","last_became_peered":"2024-09-17T21:18:01.213047+0000","last_unstale":"2024-09-17T21:18:03.059715+0000","last_undegraded":"2024-09-17T21:18:03.059715+0000","last_fullsized":"2024-09-17T21:18:03.059715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:46:35.222373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.073440+0000","last_change":"2024-09-17T21:18:03.073440+0000","last_active":"2024-09-17T21:18:03.073440+0000","last_peered":"2024-09-17T21:18:03.073440+0000","last_clean":"2024-09-17T21:18:03.073440+0000","last_became_active":"2024-09-17T21:18:01.048806+0000","last_became_peered":"2024-09-17T21:18:01.048806+0000","last_unstale":"2024-09-17T21:18:03.073440+0000","last_undegraded":"2024-09-17T21:18:03.073440+0000","last_fullsized":"2024-09-17T21:18:03.073440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:26.487163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094161400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.058986+0000","last_change":"2024-09-17T21:18:03.058986+0000","last_active":"2024-09-17T21:18:03.058986+0000","last_peered":"2024-09-17T21:18:03.058986+0000","last_clean":"2024-09-17T21:18:03.058986+0000","last_became_active":"2024-09-17T21:18:01.214272+0000","last_became_peered":"2024-09-17T21:18:01.214272+0000","last_unstale":"2024-09-17T21:18:03.058986+0000","last_undegraded":"2024-09-17T21:18:03.058986+0000","last_fullsized":"2024-09-17T21:18:03.058986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:04:38.082107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099219900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.073508+0000","last_change":"2024-09-17T21:18:03.073508+0000","last_active":"2024-09-17T21:18:03.073508+0000","last_peered":"2024-09-17T21:18:03.073508+0000","last_clean":"2024-09-17T21:18:03.073508+0000","last_became_active":"2024-09-17T21:18:01.213705+0000","last_became_peered":"2024-09-17T21:18:01.213705+0000","last_unstale":"2024-09-17T21:18:03.073508+0000","last_undegraded":"2024-09-17T21:18:03.073508+0000","last_fullsized":"2024-09-17T21:18:03.073508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:46:08.811379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006222,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.071880+0000","last_change":"2024-09-17T21:18:03.070336+0000","last_active":"2024-09-17T21:18:03.071880+0000","last_peered":"2024-09-17T21:18:03.071880+0000","last_clean":"2024-09-17T21:18:03.071880+0000","last_became_active":"2024-09-17T21:18:01.049336+0000","last_became_peered":"2024-09-17T21:18:01.049336+0000","last_unstale":"2024-09-17T21:18:03.071880+0000","last_undegraded":"2024-09-17T21:18:03.071880+0000","last_fullsized":"2024-09-17T21:18:03.071880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:56:24.329470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023986049999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.060818+0000","last_change":"2024-09-17T21:18:03.060818+0000","last_active":"2024-09-17T21:18:03.060818+0000","last_peered":"2024-09-17T21:18:03.060818+0000","last_clean":"2024-09-17T21:18:03.060818+0000","last_became_active":"2024-09-17T21:18:01.048506+0000","last_became_peered":"2024-09-17T21:18:01.048506+0000","last_unstale":"2024-09-17T21:18:03.060818+0000","last_undegraded":"2024-09-17T21:18:03.060818+0000","last_fullsized":"2024-09-17T21:18:03.060818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:47:10.439018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096175600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.059156+0000","last_change":"2024-09-17T21:18:03.059156+0000","last_active":"2024-09-17T21:18:03.059156+0000","last_peered":"2024-09-17T21:18:03.059156+0000","last_clean":"2024-09-17T21:18:03.059156+0000","last_became_active":"2024-09-17T21:18:01.049180+0000","last_became_peered":"2024-09-17T21:18:01.049180+0000","last_unstale":"2024-09-17T21:18:03.059156+0000","last_undegraded":"2024-09-17T21:18:03.059156+0000","last_fullsized":"2024-09-17T21:18:03.059156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:27.069794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077408,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.069091+0000","last_change":"2024-09-17T21:18:03.069091+0000","last_active":"2024-09-17T21:18:03.069091+0000","last_peered":"2024-09-17T21:18:03.069091+0000","last_clean":"2024-09-17T21:18:03.069091+0000","last_became_active":"2024-09-17T21:18:01.044275+0000","last_became_peered":"2024-09-17T21:18:01.044275+0000","last_unstale":"2024-09-17T21:18:03.069091+0000","last_undegraded":"2024-09-17T21:18:03.069091+0000","last_fullsized":"2024-09-17T21:18:03.069091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:28:27.066081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001145884,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T21:18:03.057898+0000","last_change":"2024-09-17T21:17:57.194934+0000","last_active":"2024-09-17T21:18:03.057898+0000","last_peered":"2024-09-17T21:18:03.057898+0000","last_clean":"2024-09-17T21:18:03.057898+0000","last_became_active":"2024-09-17T21:17:57.192405+0000","last_became_peered":"2024-09-17T21:17:57.192405+0000","last_unstale":"2024-09-17T21:18:03.057898+0000","last_undegraded":"2024-09-17T21:18:03.057898+0000","last_fullsized":"2024-09-17T21:18:03.057898+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_clean_scrub_stamp":"2024-09-17T21:17:55.982339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:30.628595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":200704,"data_stored":65431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":65412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":790528,"data_stored":655780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":200704,"data_stored":65431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":790528,"data_stored":655780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":196608,"data_stored":65412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:18:08.473 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T21:18:08.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-17T21:18:08.752 INFO:teuthology.orchestra.run.smithi083.stdout:epoch 22 2024-09-17T21:18:08.752 INFO:teuthology.orchestra.run.smithi083.stdout:fsid 224a7a29-e112-472e-8a13-f67345d6d123 2024-09-17T21:18:08.752 INFO:teuthology.orchestra.run.smithi083.stdout:created 2024-09-17T21:17:42.597312+0000 2024-09-17T21:18:08.752 INFO:teuthology.orchestra.run.smithi083.stdout:modified 2024-09-17T21:18:07.258122+0000 2024-09-17T21:18:08.753 INFO:teuthology.orchestra.run.smithi083.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-17T21:18:08.753 INFO:teuthology.orchestra.run.smithi083.stdout:crush_version 5 2024-09-17T21:18:08.753 INFO:teuthology.orchestra.run.smithi083.stdout:full_ratio 0.95 2024-09-17T21:18:08.753 INFO:teuthology.orchestra.run.smithi083.stdout:backfillfull_ratio 0.9 2024-09-17T21:18:08.753 INFO:teuthology.orchestra.run.smithi083.stdout:nearfull_ratio 0.85 2024-09-17T21:18:08.753 INFO:teuthology.orchestra.run.smithi083.stdout:require_min_compat_client luminous 2024-09-17T21:18:08.753 INFO:teuthology.orchestra.run.smithi083.stdout:min_compat_client jewel 2024-09-17T21:18:08.753 INFO:teuthology.orchestra.run.smithi083.stdout:require_osd_release squid 2024-09-17T21:18:08.753 INFO:teuthology.orchestra.run.smithi083.stdout:stretch_mode_enabled false 2024-09-17T21:18:08.753 INFO:teuthology.orchestra.run.smithi083.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 6.06 2024-09-17T21:18:08.753 INFO:teuthology.orchestra.run.smithi083.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 21 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.25 2024-09-17T21:18:08.753 INFO:teuthology.orchestra.run.smithi083.stdout:max_osd 6 2024-09-17T21:18:08.753 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6816/1111036754,v1:172.21.15.83:6817/1111036754] [v2:172.21.15.83:6818/1111036754,v1:172.21.15.83:6819/1111036754] exists,up 5bfd41c9-0cac-47c4-9a6e-8dce774c3314 2024-09-17T21:18:08.754 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6808/1460363937,v1:172.21.15.83:6809/1460363937] [v2:172.21.15.83:6810/1460363937,v1:172.21.15.83:6811/1460363937] exists,up 78404e0d-d397-4806-8e90-5f32182643bb 2024-09-17T21:18:08.754 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2 up in weight 1 up_from 12 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6800/2837444034,v1:172.21.15.83:6801/2837444034] [v2:172.21.15.83:6802/2837444034,v1:172.21.15.83:6803/2837444034] exists,up 286ed6e0-c121-4ebb-8cf6-37e506a397aa 2024-09-17T21:18:08.754 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6808/3637204865,v1:172.21.15.97:6809/3637204865] [v2:172.21.15.97:6810/3637204865,v1:172.21.15.97:6811/3637204865] exists,up b0af73a1-4bca-4892-a550-2ced87f4c357 2024-09-17T21:18:08.754 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6800/2391356911,v1:172.21.15.97:6801/2391356911] [v2:172.21.15.97:6802/2391356911,v1:172.21.15.97:6803/2391356911] exists,up 2188361c-e1a1-45fa-bc01-da5a984ffd9f 2024-09-17T21:18:08.754 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6816/2326316381,v1:172.21.15.97:6817/2326316381] [v2:172.21.15.97:6818/2326316381,v1:172.21.15.97:6819/2326316381] exists,up 7feccf4d-d8e4-4d06-ac83-bbc1bc623a5c 2024-09-17T21:18:08.770 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6816/1111036754,v1:172.21.15.83:6817/1111036754] [v2:172.21.15.83:6818/1111036754,v1:172.21.15.83:6819/1111036754] exists,up 5bfd41c9-0cac-47c4-9a6e-8dce774c3314', 'osd.1 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6808/1460363937,v1:172.21.15.83:6809/1460363937] [v2:172.21.15.83:6810/1460363937,v1:172.21.15.83:6811/1460363937] exists,up 78404e0d-d397-4806-8e90-5f32182643bb', 'osd.2 up in weight 1 up_from 12 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6800/2837444034,v1:172.21.15.83:6801/2837444034] [v2:172.21.15.83:6802/2837444034,v1:172.21.15.83:6803/2837444034] exists,up 286ed6e0-c121-4ebb-8cf6-37e506a397aa', 'osd.3 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6808/3637204865,v1:172.21.15.97:6809/3637204865] [v2:172.21.15.97:6810/3637204865,v1:172.21.15.97:6811/3637204865] exists,up b0af73a1-4bca-4892-a550-2ced87f4c357', 'osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6800/2391356911,v1:172.21.15.97:6801/2391356911] [v2:172.21.15.97:6802/2391356911,v1:172.21.15.97:6803/2391356911] exists,up 2188361c-e1a1-45fa-bc01-da5a984ffd9f', 'osd.5 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6816/2326316381,v1:172.21.15.97:6817/2326316381] [v2:172.21.15.97:6818/2326316381,v1:172.21.15.97:6819/2326316381] exists,up 7feccf4d-d8e4-4d06-ac83-bbc1bc623a5c'] 2024-09-17T21:18:08.770 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-17T21:18:08.872 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:08.872 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mon.a", 2024-09-17T21:18:08.872 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster": "ceph", 2024-09-17T21:18:08.872 INFO:teuthology.orchestra.run.smithi083.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-17T21:18:08.872 INFO:teuthology.orchestra.run.smithi083.stdout: "admin_socket_mode": "", 2024-09-17T21:18:08.873 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-17T21:18:08.873 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_client_required": "cephx, none", 2024-09-17T21:18:08.873 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_cluster_required": "cephx", 2024-09-17T21:18:08.873 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_debug": "true", 2024-09-17T21:18:08.873 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-17T21:18:08.873 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-17T21:18:08.873 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_service_required": "cephx", 2024-09-17T21:18:08.873 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-17T21:18:08.873 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_supported": "cephx", 2024-09-17T21:18:08.873 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_aio": "true", 2024-09-17T21:18:08.873 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-17T21:18:08.873 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_aio_poll_ms": "250", 2024-09-17T21:18:08.873 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_aio_reap_max": "16", 2024-09-17T21:18:08.874 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_async_discard_threads": "0", 2024-09-17T21:18:08.874 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_block_size": "4096", 2024-09-17T21:18:08.874 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_debug_aio": "false", 2024-09-17T21:18:08.874 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-17T21:18:08.874 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-17T21:18:08.874 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_debug_inflight_ios": "false", 2024-09-17T21:18:08.874 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_enable_discard": "false", 2024-09-17T21:18:08.874 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_flock_retry": "3", 2024-09-17T21:18:08.874 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-17T21:18:08.874 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_inject_crash": "0", 2024-09-17T21:18:08.874 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-17T21:18:08.875 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_ioring": "false", 2024-09-17T21:18:08.875 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_ioring_hipri": "false", 2024-09-17T21:18:08.875 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-17T21:18:08.875 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-17T21:18:08.875 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-17T21:18:08.875 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-17T21:18:08.875 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-17T21:18:08.875 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-17T21:18:08.875 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_type": "", 2024-09-17T21:18:08.875 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_alloc_size": "1048576", 2024-09-17T21:18:08.875 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_allocator": "hybrid", 2024-09-17T21:18:08.875 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_buffered_io": "true", 2024-09-17T21:18:08.875 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_check_for_zeros": "false", 2024-09-17T21:18:08.875 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-17T21:18:08.876 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-17T21:18:08.876 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_compact_log_sync": "false", 2024-09-17T21:18:08.876 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-17T21:18:08.876 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-17T21:18:08.876 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-17T21:18:08.876 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-17T21:18:08.876 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_max_log_runway": "4194304", 2024-09-17T21:18:08.876 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_max_prefetch": "1048576", 2024-09-17T21:18:08.876 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_min_flush_size": "524288", 2024-09-17T21:18:08.876 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_min_log_runway": "1048576", 2024-09-17T21:18:08.876 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_replay_recovery": "false", 2024-09-17T21:18:08.876 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-17T21:18:08.876 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-17T21:18:08.877 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_sync_write": "false", 2024-09-17T21:18:08.877 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-17T21:18:08.877 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-17T21:18:08.877 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-17T21:18:08.877 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_allocation_from_file": "true", 2024-09-17T21:18:08.877 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_allocator": "hybrid", 2024-09-17T21:18:08.877 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_async_db_compaction": "true", 2024-09-17T21:18:08.877 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-17T21:18:08.877 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-17T21:18:08.877 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-17T21:18:08.877 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-17T21:18:08.877 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_bdev_label_multi": "true", 2024-09-17T21:18:08.877 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-17T21:18:08.877 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-17T21:18:08.878 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-17T21:18:08.878 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-17T21:18:08.878 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-17T21:18:08.878 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_block_create": "true", 2024-09-17T21:18:08.878 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_block_db_create": "false", 2024-09-17T21:18:08.878 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_block_db_path": "", 2024-09-17T21:18:08.878 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_block_db_size": "0", 2024-09-17T21:18:08.878 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_block_path": "", 2024-09-17T21:18:08.878 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_block_preallocate_file": "false", 2024-09-17T21:18:08.878 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_block_size": "107374182400", 2024-09-17T21:18:08.879 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_block_wal_create": "false", 2024-09-17T21:18:08.879 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_block_wal_path": "", 2024-09-17T21:18:08.879 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_block_wal_size": "100663296", 2024-09-17T21:18:08.879 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_bluefs": "true", 2024-09-17T21:18:08.879 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-17T21:18:08.879 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-17T21:18:08.879 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-17T21:18:08.879 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2024-09-17T21:18:08.879 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-17T21:18:08.879 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T21:18:08.879 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T21:18:08.879 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-17T21:18:08.879 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T21:18:08.879 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_autotune": "true", 2024-09-17T21:18:08.880 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-17T21:18:08.880 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-17T21:18:08.880 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-17T21:18:08.880 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-17T21:18:08.880 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_size": "0", 2024-09-17T21:18:08.880 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-17T21:18:08.880 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-17T21:18:08.880 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-17T21:18:08.880 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-17T21:18:08.881 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_type": "2q", 2024-09-17T21:18:08.881 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-17T21:18:08.881 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_clone_cow": "true", 2024-09-17T21:18:08.881 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-17T21:18:08.881 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-17T21:18:08.881 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-17T21:18:08.881 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-17T21:18:08.882 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-17T21:18:08.882 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-17T21:18:08.882 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-17T21:18:08.882 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_compression_mode": "none", 2024-09-17T21:18:08.882 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-17T21:18:08.882 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_csum_type": "crc32c", 2024-09-17T21:18:08.882 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-09-17T21:18:08.882 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-17T21:18:08.882 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_freelist": "false", 2024-09-17T21:18:08.882 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-17T21:18:08.882 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-17T21:18:08.882 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-17T21:18:08.882 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-17T21:18:08.883 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-17T21:18:08.883 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-17T21:18:08.883 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-17T21:18:08.883 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-17T21:18:08.883 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-17T21:18:08.883 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-17T21:18:08.883 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-17T21:18:08.883 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-17T21:18:08.883 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_small_allocations": "0", 2024-09-17T21:18:08.883 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-17T21:18:08.883 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_default_buffered_read": "true", 2024-09-17T21:18:08.883 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_default_buffered_write": "false", 2024-09-17T21:18:08.883 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-17T21:18:08.883 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-17T21:18:08.884 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-17T21:18:08.884 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-17T21:18:08.884 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-17T21:18:08.884 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-17T21:18:08.884 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-17T21:18:08.884 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-17T21:18:08.884 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-17T21:18:08.884 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fail_eio": "false", 2024-09-17T21:18:08.884 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-17T21:18:08.884 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-17T21:18:08.884 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-17T21:18:08.884 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-17T21:18:08.884 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-17T21:18:08.884 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-17T21:18:08.885 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_on_mount": "false", 2024-09-17T21:18:08.885 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-17T21:18:08.885 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_on_umount": "false", 2024-09-17T21:18:08.885 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-17T21:18:08.885 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-17T21:18:08.885 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-17T21:18:08.885 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-17T21:18:08.885 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-17T21:18:08.885 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-17T21:18:08.885 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-17T21:18:08.885 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-17T21:18:08.885 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_ignore_data_csum": "false", 2024-09-17T21:18:08.885 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-17T21:18:08.885 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-17T21:18:08.886 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-17T21:18:08.886 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-17T21:18:08.886 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_log_op_age": "5.000000", 2024-09-17T21:18:08.886 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_max_alloc_size": "0", 2024-09-17T21:18:08.886 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_max_blob_size": "0", 2024-09-17T21:18:08.886 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-17T21:18:08.886 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-17T21:18:08.886 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-17T21:18:08.886 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_max_deferred_txc": "32", 2024-09-17T21:18:08.886 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_min_alloc_size": "0", 2024-09-17T21:18:08.886 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-17T21:18:08.886 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-17T21:18:08.886 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_nid_prealloc": "1024", 2024-09-17T21:18:08.886 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-17T21:18:08.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-17T21:18:08.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-17T21:18:08.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-17T21:18:08.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_retry_disk_reads": "3", 2024-09-17T21:18:08.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_rocksdb_cf": "true", 2024-09-17T21:18:08.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-17T21:18:08.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-17T21:18:08.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-17T21:18:08.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-17T21:18:08.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-17T21:18:08.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-17T21:18:08.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-17T21:18:08.888 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-17T21:18:08.888 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_spdk_mem": "512", 2024-09-17T21:18:08.888 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-17T21:18:08.888 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-17T21:18:08.888 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-17T21:18:08.888 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-17T21:18:08.888 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-17T21:18:08.888 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-17T21:18:08.889 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-17T21:18:08.889 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_tracing": "false", 2024-09-17T21:18:08.889 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-17T21:18:08.889 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-17T21:18:08.889 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-17T21:18:08.889 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-17T21:18:08.889 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-17T21:18:08.889 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-17T21:18:08.889 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-17T21:18:08.889 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-17T21:18:08.889 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-17T21:18:08.890 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_write_v2": "false", 2024-09-17T21:18:08.890 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_write_v2_random": "false", 2024-09-17T21:18:08.890 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_zero_block_detection": "false", 2024-09-17T21:18:08.890 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-17T21:18:08.890 INFO:teuthology.orchestra.run.smithi083.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-17T21:18:08.890 INFO:teuthology.orchestra.run.smithi083.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-17T21:18:08.890 INFO:teuthology.orchestra.run.smithi083.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-17T21:18:08.890 INFO:teuthology.orchestra.run.smithi083.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-17T21:18:08.890 INFO:teuthology.orchestra.run.smithi083.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-17T21:18:08.890 INFO:teuthology.orchestra.run.smithi083.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-17T21:18:08.890 INFO:teuthology.orchestra.run.smithi083.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-17T21:18:08.891 INFO:teuthology.orchestra.run.smithi083.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-17T21:18:08.891 INFO:teuthology.orchestra.run.smithi083.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-17T21:18:08.891 INFO:teuthology.orchestra.run.smithi083.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-17T21:18:08.891 INFO:teuthology.orchestra.run.smithi083.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-17T21:18:08.891 INFO:teuthology.orchestra.run.smithi083.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-17T21:18:08.891 INFO:teuthology.orchestra.run.smithi083.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-17T21:18:08.891 INFO:teuthology.orchestra.run.smithi083.stdout: "cephx_cluster_require_signatures": "false", 2024-09-17T21:18:08.891 INFO:teuthology.orchestra.run.smithi083.stdout: "cephx_cluster_require_version": "2", 2024-09-17T21:18:08.891 INFO:teuthology.orchestra.run.smithi083.stdout: "cephx_require_signatures": "false", 2024-09-17T21:18:08.891 INFO:teuthology.orchestra.run.smithi083.stdout: "cephx_require_version": "2", 2024-09-17T21:18:08.891 INFO:teuthology.orchestra.run.smithi083.stdout: "cephx_service_require_signatures": "false", 2024-09-17T21:18:08.891 INFO:teuthology.orchestra.run.smithi083.stdout: "cephx_service_require_version": "2", 2024-09-17T21:18:08.892 INFO:teuthology.orchestra.run.smithi083.stdout: "cephx_sign_messages": "true", 2024-09-17T21:18:08.892 INFO:teuthology.orchestra.run.smithi083.stdout: "chdir": "", 2024-09-17T21:18:08.892 INFO:teuthology.orchestra.run.smithi083.stdout: "client_acl_type": "", 2024-09-17T21:18:08.892 INFO:teuthology.orchestra.run.smithi083.stdout: "client_asio_thread_count": "2", 2024-09-17T21:18:08.892 INFO:teuthology.orchestra.run.smithi083.stdout: "client_cache_mid": "0.750000", 2024-09-17T21:18:08.892 INFO:teuthology.orchestra.run.smithi083.stdout: "client_cache_size": "16384", 2024-09-17T21:18:08.892 INFO:teuthology.orchestra.run.smithi083.stdout: "client_caps_release_delay": "5", 2024-09-17T21:18:08.892 INFO:teuthology.orchestra.run.smithi083.stdout: "client_check_pool_perm": "true", 2024-09-17T21:18:08.892 INFO:teuthology.orchestra.run.smithi083.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-17T21:18:08.892 INFO:teuthology.orchestra.run.smithi083.stdout: "client_debug_force_sync_read": "false", 2024-09-17T21:18:08.893 INFO:teuthology.orchestra.run.smithi083.stdout: "client_debug_getattr_caps": "false", 2024-09-17T21:18:08.893 INFO:teuthology.orchestra.run.smithi083.stdout: "client_debug_inject_features": "", 2024-09-17T21:18:08.893 INFO:teuthology.orchestra.run.smithi083.stdout: "client_debug_inject_tick_delay": "0", 2024-09-17T21:18:08.893 INFO:teuthology.orchestra.run.smithi083.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-17T21:18:08.893 INFO:teuthology.orchestra.run.smithi083.stdout: "client_die_on_failed_remount": "false", 2024-09-17T21:18:08.893 INFO:teuthology.orchestra.run.smithi083.stdout: "client_dirsize_rbytes": "true", 2024-09-17T21:18:08.893 INFO:teuthology.orchestra.run.smithi083.stdout: "client_force_lazyio": "false", 2024-09-17T21:18:08.893 INFO:teuthology.orchestra.run.smithi083.stdout: "client_fs": "", 2024-09-17T21:18:08.893 INFO:teuthology.orchestra.run.smithi083.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-17T21:18:08.893 INFO:teuthology.orchestra.run.smithi083.stdout: "client_inject_release_failure": "false", 2024-09-17T21:18:08.893 INFO:teuthology.orchestra.run.smithi083.stdout: "client_max_inline_size": "4096", 2024-09-17T21:18:08.894 INFO:teuthology.orchestra.run.smithi083.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-17T21:18:08.894 INFO:teuthology.orchestra.run.smithi083.stdout: "client_mds_namespace": "", 2024-09-17T21:18:08.894 INFO:teuthology.orchestra.run.smithi083.stdout: "client_metadata": "", 2024-09-17T21:18:08.894 INFO:teuthology.orchestra.run.smithi083.stdout: "client_mount_gid": "-1", 2024-09-17T21:18:08.894 INFO:teuthology.orchestra.run.smithi083.stdout: "client_mount_timeout": "300", 2024-09-17T21:18:08.894 INFO:teuthology.orchestra.run.smithi083.stdout: "client_mount_uid": "-1", 2024-09-17T21:18:08.894 INFO:teuthology.orchestra.run.smithi083.stdout: "client_mountpoint": "/", 2024-09-17T21:18:08.894 INFO:teuthology.orchestra.run.smithi083.stdout: "client_notify_timeout": "10", 2024-09-17T21:18:08.894 INFO:teuthology.orchestra.run.smithi083.stdout: "client_oc": "true", 2024-09-17T21:18:08.894 INFO:teuthology.orchestra.run.smithi083.stdout: "client_oc_max_dirty": "104857600", 2024-09-17T21:18:08.894 INFO:teuthology.orchestra.run.smithi083.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-17T21:18:08.895 INFO:teuthology.orchestra.run.smithi083.stdout: "client_oc_max_objects": "1000", 2024-09-17T21:18:08.895 INFO:teuthology.orchestra.run.smithi083.stdout: "client_oc_size": "209715200", 2024-09-17T21:18:08.895 INFO:teuthology.orchestra.run.smithi083.stdout: "client_oc_target_dirty": "8388608", 2024-09-17T21:18:08.895 INFO:teuthology.orchestra.run.smithi083.stdout: "client_permissions": "true", 2024-09-17T21:18:08.895 INFO:teuthology.orchestra.run.smithi083.stdout: "client_quota": "true", 2024-09-17T21:18:08.895 INFO:teuthology.orchestra.run.smithi083.stdout: "client_quota_df": "true", 2024-09-17T21:18:08.895 INFO:teuthology.orchestra.run.smithi083.stdout: "client_readahead_max_bytes": "0", 2024-09-17T21:18:08.895 INFO:teuthology.orchestra.run.smithi083.stdout: "client_readahead_max_periods": "4", 2024-09-17T21:18:08.895 INFO:teuthology.orchestra.run.smithi083.stdout: "client_readahead_min": "131072", 2024-09-17T21:18:08.895 INFO:teuthology.orchestra.run.smithi083.stdout: "client_reconnect_stale": "false", 2024-09-17T21:18:08.895 INFO:teuthology.orchestra.run.smithi083.stdout: "client_shutdown_timeout": "30", 2024-09-17T21:18:08.896 INFO:teuthology.orchestra.run.smithi083.stdout: "client_snapdir": ".snap", 2024-09-17T21:18:08.896 INFO:teuthology.orchestra.run.smithi083.stdout: "client_tick_interval": "1", 2024-09-17T21:18:08.896 INFO:teuthology.orchestra.run.smithi083.stdout: "client_trace": "", 2024-09-17T21:18:08.896 INFO:teuthology.orchestra.run.smithi083.stdout: "client_try_dentry_invalidate": "false", 2024-09-17T21:18:08.896 INFO:teuthology.orchestra.run.smithi083.stdout: "client_use_faked_inos": "false", 2024-09-17T21:18:08.896 INFO:teuthology.orchestra.run.smithi083.stdout: "client_use_random_mds": "false", 2024-09-17T21:18:08.896 INFO:teuthology.orchestra.run.smithi083.stdout: "clog_to_graylog": "false", 2024-09-17T21:18:08.896 INFO:teuthology.orchestra.run.smithi083.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-17T21:18:08.896 INFO:teuthology.orchestra.run.smithi083.stdout: "clog_to_graylog_port": "12201", 2024-09-17T21:18:08.896 INFO:teuthology.orchestra.run.smithi083.stdout: "clog_to_monitors": "default=true", 2024-09-17T21:18:08.896 INFO:teuthology.orchestra.run.smithi083.stdout: "clog_to_syslog": "false", 2024-09-17T21:18:08.897 INFO:teuthology.orchestra.run.smithi083.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-17T21:18:08.897 INFO:teuthology.orchestra.run.smithi083.stdout: "clog_to_syslog_level": "info", 2024-09-17T21:18:08.897 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "-", 2024-09-17T21:18:08.897 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_network": "", 2024-09-17T21:18:08.897 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_network_interface": "", 2024-09-17T21:18:08.897 INFO:teuthology.orchestra.run.smithi083.stdout: "compressor_zlib_isal": "false", 2024-09-17T21:18:08.897 INFO:teuthology.orchestra.run.smithi083.stdout: "compressor_zlib_level": "5", 2024-09-17T21:18:08.897 INFO:teuthology.orchestra.run.smithi083.stdout: "compressor_zlib_winsize": "-15", 2024-09-17T21:18:08.897 INFO:teuthology.orchestra.run.smithi083.stdout: "compressor_zstd_level": "1", 2024-09-17T21:18:08.897 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-17T21:18:08.897 INFO:teuthology.orchestra.run.smithi083.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-17T21:18:08.898 INFO:teuthology.orchestra.run.smithi083.stdout: "crimson_alien_op_num_threads": "6", 2024-09-17T21:18:08.898 INFO:teuthology.orchestra.run.smithi083.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-17T21:18:08.898 INFO:teuthology.orchestra.run.smithi083.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-17T21:18:08.898 INFO:teuthology.orchestra.run.smithi083.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-17T21:18:08.898 INFO:teuthology.orchestra.run.smithi083.stdout: "crimson_osd_stat_interval": "0", 2024-09-17T21:18:08.898 INFO:teuthology.orchestra.run.smithi083.stdout: "crimson_seastar_cpu_cores": "", 2024-09-17T21:18:08.898 INFO:teuthology.orchestra.run.smithi083.stdout: "crimson_seastar_num_threads": "0", 2024-09-17T21:18:08.898 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_location": "", 2024-09-17T21:18:08.899 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_location_hook": "", 2024-09-17T21:18:08.899 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_location_hook_timeout": "10", 2024-09-17T21:18:08.899 INFO:teuthology.orchestra.run.smithi083.stdout: "daemonize": "false", 2024-09-17T21:18:08.899 INFO:teuthology.orchestra.run.smithi083.stdout: "daos_pool": "tank", 2024-09-17T21:18:08.899 INFO:teuthology.orchestra.run.smithi083.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-17T21:18:08.899 INFO:teuthology.orchestra.run.smithi083.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-17T21:18:08.899 INFO:teuthology.orchestra.run.smithi083.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-17T21:18:08.899 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_alienstore": "0/5", 2024-09-17T21:18:08.899 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_allow_any_pool_priority": "false", 2024-09-17T21:18:08.899 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_asok": "1/5", 2024-09-17T21:18:08.899 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_asok_assert_abort": "false", 2024-09-17T21:18:08.899 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_asserts_on_shutdown": "true", 2024-09-17T21:18:08.900 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_auth": "20/20", 2024-09-17T21:18:08.900 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_bdev": "1/3", 2024-09-17T21:18:08.900 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_bluefs": "1/5", 2024-09-17T21:18:08.900 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_bluestore": "1/5", 2024-09-17T21:18:08.900 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_buffer": "0/1", 2024-09-17T21:18:08.900 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_ceph_dedup": "0/5", 2024-09-17T21:18:08.900 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_ceph_exporter": "1/5", 2024-09-17T21:18:08.900 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_cephfs_mirror": "0/5", 2024-09-17T21:18:08.900 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_cephsqlite": "0/5", 2024-09-17T21:18:08.900 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_client": "0/5", 2024-09-17T21:18:08.900 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_compressor": "1/5", 2024-09-17T21:18:08.901 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_context": "0/1", 2024-09-17T21:18:08.901 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_crimson_interrupt": "0/5", 2024-09-17T21:18:08.901 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_crush": "1/1", 2024-09-17T21:18:08.901 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_crypto": "1/5", 2024-09-17T21:18:08.901 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_cyanstore": "0/5", 2024-09-17T21:18:08.901 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_deliberately_leak_memory": "false", 2024-09-17T21:18:08.901 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_disable_randomized_ping": "false", 2024-09-17T21:18:08.901 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_dpdk": "1/5", 2024-09-17T21:18:08.901 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_eventtrace": "1/5", 2024-09-17T21:18:08.901 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_filer": "0/1", 2024-09-17T21:18:08.901 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_filestore": "1/3", 2024-09-17T21:18:08.901 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_finisher": "1/1", 2024-09-17T21:18:08.902 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_fuse": "1/5", 2024-09-17T21:18:08.902 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_heartbeat_testing_span": "0", 2024-09-17T21:18:08.902 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_heartbeatmap": "1/5", 2024-09-17T21:18:08.902 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-17T21:18:08.902 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_javaclient": "1/5", 2024-09-17T21:18:08.902 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_journal": "1/3", 2024-09-17T21:18:08.902 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_journaler": "0/5", 2024-09-17T21:18:08.902 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_kstore": "1/5", 2024-09-17T21:18:08.902 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_lockdep": "0/1", 2024-09-17T21:18:08.902 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_mclock": "1/5", 2024-09-17T21:18:08.902 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_mds": "1/5", 2024-09-17T21:18:08.903 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_mds_balancer": "1/5", 2024-09-17T21:18:08.903 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_mds_locker": "1/5", 2024-09-17T21:18:08.903 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_mds_log": "1/5", 2024-09-17T21:18:08.903 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_mds_log_expire": "1/5", 2024-09-17T21:18:08.903 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_mds_migrator": "1/5", 2024-09-17T21:18:08.903 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_mds_quiesce": "3/5", 2024-09-17T21:18:08.903 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_memstore": "1/5", 2024-09-17T21:18:08.903 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_mgr": "2/5", 2024-09-17T21:18:08.903 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_mgrc": "1/5", 2024-09-17T21:18:08.903 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_mon": "20/20", 2024-09-17T21:18:08.904 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_monc": "0/10", 2024-09-17T21:18:08.904 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_ms": "1/1", 2024-09-17T21:18:08.904 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_none": "0/5", 2024-09-17T21:18:08.904 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_objclass": "0/5", 2024-09-17T21:18:08.904 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_objectcacher": "0/5", 2024-09-17T21:18:08.904 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_objecter": "0/1", 2024-09-17T21:18:08.904 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_optracker": "0/5", 2024-09-17T21:18:08.904 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_osd": "1/5", 2024-09-17T21:18:08.904 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_paxos": "20/20", 2024-09-17T21:18:08.904 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_perfcounter": "1/5", 2024-09-17T21:18:08.904 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_prioritycache": "1/5", 2024-09-17T21:18:08.904 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rados": "0/5", 2024-09-17T21:18:08.905 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rbd": "0/5", 2024-09-17T21:18:08.905 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rbd_mirror": "0/5", 2024-09-17T21:18:08.905 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rbd_pwl": "0/5", 2024-09-17T21:18:08.905 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rbd_replay": "0/5", 2024-09-17T21:18:08.905 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_refs": "0/0", 2024-09-17T21:18:08.905 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_reserver": "1/1", 2024-09-17T21:18:08.905 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rgw": "1/5", 2024-09-17T21:18:08.905 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rgw_access": "1/5", 2024-09-17T21:18:08.905 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rgw_datacache": "1/5", 2024-09-17T21:18:08.905 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rgw_dbstore": "1/5", 2024-09-17T21:18:08.905 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rgw_flight": "1/5", 2024-09-17T21:18:08.905 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-17T21:18:08.906 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rgw_notification": "1/5", 2024-09-17T21:18:08.906 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rgw_sync": "1/5", 2024-09-17T21:18:08.906 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rocksdb": "4/5", 2024-09-17T21:18:08.906 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore": "0/5", 2024-09-17T21:18:08.906 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_backref": "0/5", 2024-09-17T21:18:08.906 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_cache": "0/5", 2024-09-17T21:18:08.906 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_cleaner": "0/5", 2024-09-17T21:18:08.906 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_device": "0/5", 2024-09-17T21:18:08.906 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_epm": "0/5", 2024-09-17T21:18:08.906 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-17T21:18:08.906 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_journal": "0/5", 2024-09-17T21:18:08.907 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_lba": "0/5", 2024-09-17T21:18:08.907 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_odata": "0/5", 2024-09-17T21:18:08.907 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_omap": "0/5", 2024-09-17T21:18:08.907 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_onode": "0/5", 2024-09-17T21:18:08.907 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_t": "0/5", 2024-09-17T21:18:08.907 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_tm": "0/5", 2024-09-17T21:18:08.907 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_striper": "0/1", 2024-09-17T21:18:08.907 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_test": "0/5", 2024-09-17T21:18:08.907 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_throttle": "1/1", 2024-09-17T21:18:08.907 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_timer": "0/1", 2024-09-17T21:18:08.907 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_tp": "0/5", 2024-09-17T21:18:08.907 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_trace": "1/5", 2024-09-17T21:18:08.907 INFO:teuthology.orchestra.run.smithi083.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-17T21:18:08.908 INFO:teuthology.orchestra.run.smithi083.stdout: "device_failure_prediction_mode": "none", 2024-09-17T21:18:08.908 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-17T21:18:08.908 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-17T21:18:08.908 INFO:teuthology.orchestra.run.smithi083.stdout: "err_to_graylog": "false", 2024-09-17T21:18:08.908 INFO:teuthology.orchestra.run.smithi083.stdout: "err_to_journald": "false", 2024-09-17T21:18:08.908 INFO:teuthology.orchestra.run.smithi083.stdout: "err_to_stderr": "true", 2024-09-17T21:18:08.908 INFO:teuthology.orchestra.run.smithi083.stdout: "err_to_syslog": "false", 2024-09-17T21:18:08.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event_tracing": "false", 2024-09-17T21:18:08.908 INFO:teuthology.orchestra.run.smithi083.stdout: "exporter_addr": "0.0.0.0", 2024-09-17T21:18:08.908 INFO:teuthology.orchestra.run.smithi083.stdout: "exporter_cert_file": "", 2024-09-17T21:18:08.908 INFO:teuthology.orchestra.run.smithi083.stdout: "exporter_http_port": "9926", 2024-09-17T21:18:08.909 INFO:teuthology.orchestra.run.smithi083.stdout: "exporter_key_file": "", 2024-09-17T21:18:08.909 INFO:teuthology.orchestra.run.smithi083.stdout: "exporter_prio_limit": "5", 2024-09-17T21:18:08.909 INFO:teuthology.orchestra.run.smithi083.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-17T21:18:08.909 INFO:teuthology.orchestra.run.smithi083.stdout: "exporter_sort_metrics": "true", 2024-09-17T21:18:08.909 INFO:teuthology.orchestra.run.smithi083.stdout: "exporter_stats_period": "5", 2024-09-17T21:18:08.909 INFO:teuthology.orchestra.run.smithi083.stdout: "fake_statfs_for_testing": "0", 2024-09-17T21:18:08.909 INFO:teuthology.orchestra.run.smithi083.stdout: "fatal_signal_handlers": "true", 2024-09-17T21:18:08.909 INFO:teuthology.orchestra.run.smithi083.stdout: "filer_max_purge_ops": "10", 2024-09-17T21:18:08.909 INFO:teuthology.orchestra.run.smithi083.stdout: "filer_max_truncate_ops": "128", 2024-09-17T21:18:08.909 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_apply_finisher_threads": "1", 2024-09-17T21:18:08.909 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_blackhole": "false", 2024-09-17T21:18:08.910 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_btrfs_clone_range": "true", 2024-09-17T21:18:08.910 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_btrfs_snap": "true", 2024-09-17T21:18:08.910 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_caller_concurrency": "10", 2024-09-17T21:18:08.910 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_collect_device_partition_information": "true", 2024-09-17T21:18:08.910 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_commit_timeout": "600.000000", 2024-09-17T21:18:08.910 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_debug_inject_read_err": "false", 2024-09-17T21:18:08.910 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_debug_omap_check": "false", 2024-09-17T21:18:08.910 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_debug_verify_split": "false", 2024-09-17T21:18:08.910 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_dump_file": "", 2024-09-17T21:18:08.910 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-17T21:18:08.910 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-17T21:18:08.910 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_fadvise": "true", 2024-09-17T21:18:08.911 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_fail_eio": "true", 2024-09-17T21:18:08.911 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_fd_cache_shards": "16", 2024-09-17T21:18:08.911 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_fd_cache_size": "128", 2024-09-17T21:18:08.911 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_fiemap": "false", 2024-09-17T21:18:08.911 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_fiemap_threshold": "4096", 2024-09-17T21:18:08.911 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-17T21:18:08.911 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-17T21:18:08.911 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_inject_stall": "0", 2024-09-17T21:18:08.911 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_journal_parallel": "false", 2024-09-17T21:18:08.911 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_journal_trailing": "false", 2024-09-17T21:18:08.911 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_journal_writeahead": "false", 2024-09-17T21:18:08.911 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_kill_at": "0", 2024-09-17T21:18:08.912 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-17T21:18:08.912 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-17T21:18:08.912 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-17T21:18:08.912 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-17T21:18:08.912 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-17T21:18:08.912 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_inline_xattrs": "0", 2024-09-17T21:18:08.912 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-17T21:18:08.912 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-17T21:18:08.912 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-17T21:18:08.912 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-17T21:18:08.912 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_xattr_value_size": "0", 2024-09-17T21:18:08.913 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-17T21:18:08.913 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-17T21:18:08.913 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-17T21:18:08.913 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_merge_threshold": "-10", 2024-09-17T21:18:08.913 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-17T21:18:08.913 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_odsync_write": "false", 2024-09-17T21:18:08.913 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_omap_backend": "rocksdb", 2024-09-17T21:18:08.913 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_omap_backend_path": "", 2024-09-17T21:18:08.913 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-17T21:18:08.913 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-17T21:18:08.913 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-17T21:18:08.913 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_op_thread_timeout": "60", 2024-09-17T21:18:08.914 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_op_threads": "2", 2024-09-17T21:18:08.914 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_punch_hole": "false", 2024-09-17T21:18:08.914 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-17T21:18:08.914 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-17T21:18:08.914 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-17T21:18:08.914 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-17T21:18:08.914 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-17T21:18:08.914 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-17T21:18:08.914 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-17T21:18:08.914 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-17T21:18:08.914 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-17T21:18:08.914 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_queue_max_ops": "50", 2024-09-17T21:18:08.914 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-17T21:18:08.915 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_seek_data_hole": "false", 2024-09-17T21:18:08.915 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_sloppy_crc": "false", 2024-09-17T21:18:08.915 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-17T21:18:08.915 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_splice": "false", 2024-09-17T21:18:08.915 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_split_multiple": "2", 2024-09-17T21:18:08.915 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_split_rand_factor": "20", 2024-09-17T21:18:08.915 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_update_to": "1000", 2024-09-17T21:18:08.915 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-17T21:18:08.915 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-17T21:18:08.915 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-17T21:18:08.915 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-17T21:18:08.915 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-17T21:18:08.916 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-17T21:18:08.916 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_enable": "true", 2024-09-17T21:18:08.916 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-17T21:18:08.916 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-17T21:18:08.916 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-17T21:18:08.916 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-17T21:18:08.916 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-17T21:18:08.916 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-17T21:18:08.916 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_xfs_extsize": "false", 2024-09-17T21:18:08.916 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_zfs_snap": "false", 2024-09-17T21:18:08.916 INFO:teuthology.orchestra.run.smithi083.stdout: "fio_dir": "/tmp/fio", 2024-09-17T21:18:08.917 INFO:teuthology.orchestra.run.smithi083.stdout: "fsid": "224a7a29-e112-472e-8a13-f67345d6d123", 2024-09-17T21:18:08.917 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_allow_other": "true", 2024-09-17T21:18:08.917 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_atomic_o_trunc": "true", 2024-09-17T21:18:08.917 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_big_writes": "true", 2024-09-17T21:18:08.917 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_debug": "false", 2024-09-17T21:18:08.917 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_default_permissions": "false", 2024-09-17T21:18:08.917 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_disable_pagecache": "false", 2024-09-17T21:18:08.917 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_max_write": "0", 2024-09-17T21:18:08.917 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_multithreaded": "true", 2024-09-17T21:18:08.917 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_require_active_mds": "true", 2024-09-17T21:18:08.917 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_set_user_groups": "true", 2024-09-17T21:18:08.917 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_splice_move": "true", 2024-09-17T21:18:08.917 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_splice_read": "true", 2024-09-17T21:18:08.918 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_splice_write": "true", 2024-09-17T21:18:08.918 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-17T21:18:08.918 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_use_invalidate_cb": "true", 2024-09-17T21:18:08.918 INFO:teuthology.orchestra.run.smithi083.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-17T21:18:08.918 INFO:teuthology.orchestra.run.smithi083.stdout: "gss_target_name": "ceph", 2024-09-17T21:18:08.918 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_file": "", 2024-09-17T21:18:08.918 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_inject_failure": "0", 2024-09-17T21:18:08.918 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_interval": "5", 2024-09-17T21:18:08.918 INFO:teuthology.orchestra.run.smithi083.stdout: "host": "smithi083", 2024-09-17T21:18:08.918 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-17T21:18:08.918 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-17T21:18:08.918 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-17T21:18:08.919 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-17T21:18:08.919 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-17T21:18:08.919 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-17T21:18:08.919 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-17T21:18:08.919 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-17T21:18:08.919 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-17T21:18:08.919 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-17T21:18:08.919 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-17T21:18:08.919 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-17T21:18:08.919 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-17T21:18:08.920 INFO:teuthology.orchestra.run.smithi083.stdout: "inject_early_sigterm": "false", 2024-09-17T21:18:08.920 INFO:teuthology.orchestra.run.smithi083.stdout: "jaeger_agent_port": "6799", 2024-09-17T21:18:08.920 INFO:teuthology.orchestra.run.smithi083.stdout: "jaeger_tracing_enable": "false", 2024-09-17T21:18:08.920 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_aio": "true", 2024-09-17T21:18:08.920 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_align_min_size": "65536", 2024-09-17T21:18:08.920 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_block_align": "true", 2024-09-17T21:18:08.920 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_block_size": "4096", 2024-09-17T21:18:08.920 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_dio": "true", 2024-09-17T21:18:08.920 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_discard": "false", 2024-09-17T21:18:08.920 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_force_aio": "false", 2024-09-17T21:18:08.920 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_ignore_corruption": "false", 2024-09-17T21:18:08.920 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_max_write_bytes": "10485760", 2024-09-17T21:18:08.921 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_max_write_entries": "100", 2024-09-17T21:18:08.921 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_replay_from": "0", 2024-09-17T21:18:08.921 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-17T21:18:08.921 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-17T21:18:08.921 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-17T21:18:08.921 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-17T21:18:08.921 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_write_header_frequency": "0", 2024-09-17T21:18:08.921 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_zero_on_create": "false", 2024-09-17T21:18:08.921 INFO:teuthology.orchestra.run.smithi083.stdout: "journaler_prefetch_periods": "10", 2024-09-17T21:18:08.921 INFO:teuthology.orchestra.run.smithi083.stdout: "journaler_prezero_periods": "5", 2024-09-17T21:18:08.921 INFO:teuthology.orchestra.run.smithi083.stdout: "journaler_write_head_interval": "15", 2024-09-17T21:18:08.921 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T21:18:08.922 INFO:teuthology.orchestra.run.smithi083.stdout: "keyfile": "", 2024-09-17T21:18:08.922 INFO:teuthology.orchestra.run.smithi083.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-17T21:18:08.922 INFO:teuthology.orchestra.run.smithi083.stdout: "kstore_backend": "rocksdb", 2024-09-17T21:18:08.922 INFO:teuthology.orchestra.run.smithi083.stdout: "kstore_default_stripe_size": "65536", 2024-09-17T21:18:08.922 INFO:teuthology.orchestra.run.smithi083.stdout: "kstore_fsck_on_mount": "false", 2024-09-17T21:18:08.922 INFO:teuthology.orchestra.run.smithi083.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-17T21:18:08.922 INFO:teuthology.orchestra.run.smithi083.stdout: "kstore_max_bytes": "67108864", 2024-09-17T21:18:08.922 INFO:teuthology.orchestra.run.smithi083.stdout: "kstore_max_ops": "512", 2024-09-17T21:18:08.922 INFO:teuthology.orchestra.run.smithi083.stdout: "kstore_nid_prealloc": "1024", 2024-09-17T21:18:08.922 INFO:teuthology.orchestra.run.smithi083.stdout: "kstore_onode_map_size": "1024", 2024-09-17T21:18:08.922 INFO:teuthology.orchestra.run.smithi083.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-17T21:18:08.923 INFO:teuthology.orchestra.run.smithi083.stdout: "kstore_sync_submit_transaction": "false", 2024-09-17T21:18:08.923 INFO:teuthology.orchestra.run.smithi083.stdout: "kstore_sync_transaction": "false", 2024-09-17T21:18:08.923 INFO:teuthology.orchestra.run.smithi083.stdout: "librados_thread_count": "2", 2024-09-17T21:18:08.923 INFO:teuthology.orchestra.run.smithi083.stdout: "lockdep": "false", 2024-09-17T21:18:08.923 INFO:teuthology.orchestra.run.smithi083.stdout: "lockdep_force_backtrace": "false", 2024-09-17T21:18:08.923 INFO:teuthology.orchestra.run.smithi083.stdout: "log_coarse_timestamps": "true", 2024-09-17T21:18:08.923 INFO:teuthology.orchestra.run.smithi083.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-17T21:18:08.923 INFO:teuthology.orchestra.run.smithi083.stdout: "log_flush_on_exit": "false", 2024-09-17T21:18:08.923 INFO:teuthology.orchestra.run.smithi083.stdout: "log_graylog_host": "127.0.0.1", 2024-09-17T21:18:08.923 INFO:teuthology.orchestra.run.smithi083.stdout: "log_graylog_port": "12201", 2024-09-17T21:18:08.923 INFO:teuthology.orchestra.run.smithi083.stdout: "log_max_new": "1000", 2024-09-17T21:18:08.924 INFO:teuthology.orchestra.run.smithi083.stdout: "log_max_recent": "10000", 2024-09-17T21:18:08.924 INFO:teuthology.orchestra.run.smithi083.stdout: "log_stderr_prefix": "", 2024-09-17T21:18:08.924 INFO:teuthology.orchestra.run.smithi083.stdout: "log_stop_at_utilization": "0.970000", 2024-09-17T21:18:08.924 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": "true", 2024-09-17T21:18:08.924 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_graylog": "false", 2024-09-17T21:18:08.924 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_journald": "false", 2024-09-17T21:18:08.924 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_stderr": "false", 2024-09-17T21:18:08.924 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_syslog": "false", 2024-09-17T21:18:08.924 INFO:teuthology.orchestra.run.smithi083.stdout: "max_rotating_auth_attempts": "10", 2024-09-17T21:18:08.924 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-17T21:18:08.924 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_action_on_write_error": "1", 2024-09-17T21:18:08.925 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_alternate_name_max": "8192", 2024-09-17T21:18:08.925 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_asio_thread_count": "2", 2024-09-17T21:18:08.925 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_export_pin": "true", 2024-09-17T21:18:08.925 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_fragment_dirs": "true", 2024-09-17T21:18:08.925 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-17T21:18:08.925 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_fragment_interval": "5", 2024-09-17T21:18:08.925 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-17T21:18:08.925 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-17T21:18:08.925 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_interval": "10", 2024-09-17T21:18:08.925 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_max": "-1", 2024-09-17T21:18:08.925 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_max_until": "-1", 2024-09-17T21:18:08.926 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_merge_size": "50", 2024-09-17T21:18:08.926 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_midchunk": "0.300000", 2024-09-17T21:18:08.926 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-17T21:18:08.926 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_min_start": "0.200000", 2024-09-17T21:18:08.926 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_minchunk": "0.001000", 2024-09-17T21:18:08.926 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_mode": "0", 2024-09-17T21:18:08.926 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_need_max": "1.200000", 2024-09-17T21:18:08.926 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_need_min": "0.800000", 2024-09-17T21:18:08.926 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_overload_epochs": "2", 2024-09-17T21:18:08.926 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-17T21:18:08.926 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-17T21:18:08.926 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_split_bits": "3", 2024-09-17T21:18:08.927 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-17T21:18:08.927 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_split_size": "10000", 2024-09-17T21:18:08.927 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-17T21:18:08.927 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_target_decay": "10.000000", 2024-09-17T21:18:08.927 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-17T21:18:08.927 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_beacon_grace": "15.000000", 2024-09-17T21:18:08.927 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_beacon_interval": "4.000000", 2024-09-17T21:18:08.927 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-17T21:18:08.927 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-17T21:18:08.927 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_mid": "0.700000", 2024-09-17T21:18:08.927 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-17T21:18:08.927 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_quiesce_delay": "0", 2024-09-17T21:18:08.927 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-17T21:18:08.928 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-17T21:18:08.928 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-17T21:18:08.928 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_release_free_interval": "10", 2024-09-17T21:18:08.928 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_reservation": "0.050000", 2024-09-17T21:18:08.928 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-17T21:18:08.928 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_trim_interval": "1", 2024-09-17T21:18:08.928 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_trim_threshold": "262144", 2024-09-17T21:18:08.928 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-17T21:18:08.928 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-17T21:18:08.928 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-17T21:18:08.928 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_client_prealloc_inos": "1000", 2024-09-17T21:18:08.928 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-17T21:18:08.928 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_connect_bootstrapping": "false", 2024-09-17T21:18:08.929 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_damage_table_max_entries": "10000", 2024-09-17T21:18:08.929 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-17T21:18:08.929 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_debug_auth_pins": "false", 2024-09-17T21:18:08.929 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_debug_frag": "false", 2024-09-17T21:18:08.929 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_debug_scatterstat": "false", 2024-09-17T21:18:08.929 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_debug_subtrees": "false", 2024-09-17T21:18:08.929 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_decay_halflife": "5.000000", 2024-09-17T21:18:08.929 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_default_dir_hash": "2", 2024-09-17T21:18:08.929 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_defer_session_stale": "true", 2024-09-17T21:18:08.929 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_deny_all_reconnect": "false", 2024-09-17T21:18:08.929 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_dir_keys_per_op": "16384", 2024-09-17T21:18:08.929 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_dir_max_commit_size": "10", 2024-09-17T21:18:08.930 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_dir_max_entries": "0", 2024-09-17T21:18:08.930 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_dir_prefetch": "true", 2024-09-17T21:18:08.930 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-17T21:18:08.930 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-17T21:18:08.930 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_dump_cache_on_map": "false", 2024-09-17T21:18:08.930 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-17T21:18:08.930 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-17T21:18:08.930 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_early_reply": "true", 2024-09-17T21:18:08.930 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_enable_op_tracker": "true", 2024-09-17T21:18:08.930 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_enforce_unique_name": "true", 2024-09-17T21:18:08.930 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-17T21:18:08.930 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-17T21:18:08.931 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_export_ephemeral_random": "true", 2024-09-17T21:18:08.931 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-17T21:18:08.931 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-17T21:18:08.931 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-17T21:18:08.931 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-17T21:18:08.931 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-17T21:18:08.931 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-17T21:18:08.931 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-17T21:18:08.931 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-17T21:18:08.931 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_health_summarize_threshold": "10", 2024-09-17T21:18:08.931 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-17T21:18:08.932 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-17T21:18:08.932 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_inject_health_dummy": "false", 2024-09-17T21:18:08.932 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-17T21:18:08.932 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_inject_migrator_session_race": "false", 2024-09-17T21:18:08.932 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-17T21:18:08.932 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-17T21:18:08.932 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-17T21:18:08.932 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_join_fs": "", 2024-09-17T21:18:08.932 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_journal_format": "1", 2024-09-17T21:18:08.932 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-17T21:18:08.932 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_create_at": "0", 2024-09-17T21:18:08.933 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_dirfrag_at": "0", 2024-09-17T21:18:08.933 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_export_at": "0", 2024-09-17T21:18:08.933 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_import_at": "0", 2024-09-17T21:18:08.933 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_journal_at": "0", 2024-09-17T21:18:08.933 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_journal_expire_at": "0", 2024-09-17T21:18:08.933 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_journal_replay_at": "0", 2024-09-17T21:18:08.933 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_link_at": "0", 2024-09-17T21:18:08.933 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_mdstable_at": "0", 2024-09-17T21:18:08.933 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_openc_at": "0", 2024-09-17T21:18:08.933 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_rename_at": "0", 2024-09-17T21:18:08.933 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_shutdown_at": "0", 2024-09-17T21:18:08.934 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_event_large_threshold": "524288", 2024-09-17T21:18:08.934 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_events_per_segment": "1024", 2024-09-17T21:18:08.934 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-17T21:18:08.934 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_max_events": "-1", 2024-09-17T21:18:08.934 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_max_segments": "128", 2024-09-17T21:18:08.934 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_pause": "false", 2024-09-17T21:18:08.934 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_segment_size": "0", 2024-09-17T21:18:08.934 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-17T21:18:08.934 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-17T21:18:08.934 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-17T21:18:08.934 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_trim_threshold": "128", 2024-09-17T21:18:08.935 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-17T21:18:08.935 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_warn_factor": "2.000000", 2024-09-17T21:18:08.935 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_max_caps_per_client": "1048576", 2024-09-17T21:18:08.935 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_max_completed_flushes": "100000", 2024-09-17T21:18:08.935 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_max_completed_requests": "100000", 2024-09-17T21:18:08.935 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_max_export_size": "20971520", 2024-09-17T21:18:08.935 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_max_file_recover": "32", 2024-09-17T21:18:08.935 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_max_purge_files": "64", 2024-09-17T21:18:08.935 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_max_purge_ops": "8192", 2024-09-17T21:18:08.935 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-17T21:18:08.935 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-17T21:18:08.935 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_max_snaps_per_dir": "100", 2024-09-17T21:18:08.936 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_metrics_update_interval": "2", 2024-09-17T21:18:08.936 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_min_caps_per_client": "100", 2024-09-17T21:18:08.936 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_min_caps_working_set": "10000", 2024-09-17T21:18:08.936 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-17T21:18:08.936 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_numa_node": "-1", 2024-09-17T21:18:08.936 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-17T21:18:08.936 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_op_complaint_time": "30.000000", 2024-09-17T21:18:08.936 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_op_history_duration": "600", 2024-09-17T21:18:08.936 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_op_history_size": "20", 2024-09-17T21:18:08.936 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_op_history_slow_op_size": "20", 2024-09-17T21:18:08.936 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-17T21:18:08.936 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_op_log_threshold": "5", 2024-09-17T21:18:08.937 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_ping_grace": "15", 2024-09-17T21:18:08.937 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_ping_interval": "5", 2024-09-17T21:18:08.937 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-17T21:18:08.937 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-17T21:18:08.937 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_recall_max_caps": "30000", 2024-09-17T21:18:08.937 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-17T21:18:08.937 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-17T21:18:08.937 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-17T21:18:08.937 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_recall_warning_threshold": "262144", 2024-09-17T21:18:08.937 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-17T21:18:08.937 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_replay_interval": "1.000000", 2024-09-17T21:18:08.937 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-17T21:18:08.938 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-17T21:18:08.938 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_root_ino_gid": "0", 2024-09-17T21:18:08.938 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_root_ino_uid": "0", 2024-09-17T21:18:08.938 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-17T21:18:08.938 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_scrub_stats_review_period": "1", 2024-09-17T21:18:08.938 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-09-17T21:18:08.938 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-09-17T21:18:08.938 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-17T21:18:08.938 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-17T21:18:08.938 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-17T21:18:08.938 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-17T21:18:08.939 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-17T21:18:08.939 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-17T21:18:08.939 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-17T21:18:08.939 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-17T21:18:08.939 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-17T21:18:08.939 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_shutdown_check": "0", 2024-09-17T21:18:08.939 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_skip_ino": "0", 2024-09-17T21:18:08.939 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-17T21:18:08.939 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_snap_max_uid": "4294967294", 2024-09-17T21:18:08.939 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_snap_min_uid": "0", 2024-09-17T21:18:08.939 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_snap_rstat": "false", 2024-09-17T21:18:08.939 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_standby_replay_damaged": "false", 2024-09-17T21:18:08.940 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_symlink_recovery": "true", 2024-09-17T21:18:08.940 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-17T21:18:08.940 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_thrash_exports": "0", 2024-09-17T21:18:08.940 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_thrash_fragments": "0", 2024-09-17T21:18:08.940 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_tick_interval": "5.000000", 2024-09-17T21:18:08.940 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_valgrind_exit": "false", 2024-09-17T21:18:08.940 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_verify_backtrace": "1", 2024-09-17T21:18:08.940 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_verify_scatter": "false", 2024-09-17T21:18:08.940 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-17T21:18:08.940 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_wipe_sessions": "false", 2024-09-17T21:18:08.940 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool_debug": "false", 2024-09-17T21:18:08.941 INFO:teuthology.orchestra.run.smithi083.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-17T21:18:08.941 INFO:teuthology.orchestra.run.smithi083.stdout: "memstore_device_bytes": "1073741824", 2024-09-17T21:18:08.941 INFO:teuthology.orchestra.run.smithi083.stdout: "memstore_page_set": "false", 2024-09-17T21:18:08.941 INFO:teuthology.orchestra.run.smithi083.stdout: "memstore_page_size": "65536", 2024-09-17T21:18:08.941 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_client_bytes": "134217728", 2024-09-17T21:18:08.941 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_client_messages": "512", 2024-09-17T21:18:08.941 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-17T21:18:08.941 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-17T21:18:08.941 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-17T21:18:08.941 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-17T21:18:08.941 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_disabled_modules": "", 2024-09-17T21:18:08.941 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_enable_op_tracker": "true", 2024-09-17T21:18:08.942 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-17T21:18:08.942 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_max_pg_creating": "1024", 2024-09-17T21:18:08.942 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_max_pg_num_change": "128", 2024-09-17T21:18:08.942 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_mds_bytes": "134217728", 2024-09-17T21:18:08.942 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_mds_messages": "128", 2024-09-17T21:18:08.942 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-17T21:18:08.942 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_mon_bytes": "134217728", 2024-09-17T21:18:08.942 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_mon_messages": "128", 2024-09-17T21:18:08.942 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-17T21:18:08.942 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-17T21:18:08.943 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_op_history_duration": "600", 2024-09-17T21:18:08.943 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_op_history_size": "20", 2024-09-17T21:18:08.943 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-17T21:18:08.943 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-17T21:18:08.943 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_op_log_threshold": "5", 2024-09-17T21:18:08.943 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_osd_bytes": "536870912", 2024-09-17T21:18:08.943 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_osd_messages": "8192", 2024-09-17T21:18:08.943 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_pool": "true", 2024-09-17T21:18:08.943 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-17T21:18:08.943 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_standby_modules": "true", 2024-09-17T21:18:08.943 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_stats_period": "5", 2024-09-17T21:18:08.943 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_stats_threshold": "5", 2024-09-17T21:18:08.944 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_tick_period": "2", 2024-09-17T21:18:08.944 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-17T21:18:08.944 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-17T21:18:08.944 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_allow_pool_delete": "true", 2024-09-17T21:18:08.944 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_allow_pool_size_one": "true", 2024-09-17T21:18:08.944 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_auth_validate_all_caps": "true", 2024-09-17T21:18:08.944 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-17T21:18:08.944 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-17T21:18:08.944 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_bytes": "104857600", 2024-09-17T21:18:08.944 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_directed_command_retry": "5", 2024-09-17T21:18:08.944 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-17T21:18:08.944 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-17T21:18:08.945 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-17T21:18:08.945 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-17T21:18:08.945 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_hunt_parallel": "3", 2024-09-17T21:18:08.945 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_log_interval": "1.000000", 2024-09-17T21:18:08.945 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-17T21:18:08.945 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_ping_interval": "10.000000", 2024-09-17T21:18:08.945 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-17T21:18:08.945 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_target_rank": "-1", 2024-09-17T21:18:08.945 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-17T21:18:08.945 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-17T21:18:08.945 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-17T21:18:08.946 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cluster_log_level": "debug", 2024-09-17T21:18:08.946 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cluster_log_to_file": "true", 2024-09-17T21:18:08.946 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-17T21:18:08.946 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-17T21:18:08.946 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-17T21:18:08.946 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cluster_log_to_journald": "false", 2024-09-17T21:18:08.946 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-17T21:18:08.946 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-17T21:18:08.946 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-17T21:18:08.946 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_compact_on_bootstrap": "false", 2024-09-17T21:18:08.946 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_compact_on_start": "false", 2024-09-17T21:18:08.946 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_compact_on_trim": "true", 2024-09-17T21:18:08.947 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-17T21:18:08.947 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-17T21:18:08.947 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-17T21:18:08.947 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cpu_threads": "4", 2024-09-17T21:18:08.947 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_crush_min_required_version": "hammer", 2024-09-17T21:18:08.947 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_daemon_bytes": "419430400", 2024-09-17T21:18:08.947 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-17T21:18:08.947 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_data_avail_crit": "5", 2024-09-17T21:18:08.947 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_data_avail_warn": "5", 2024-09-17T21:18:08.947 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_data_size_warn": "16106127360", 2024-09-17T21:18:08.947 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-17T21:18:08.948 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-17T21:18:08.948 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_debug_dump_json": "false", 2024-09-17T21:18:08.948 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-17T21:18:08.948 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_debug_dump_transactions": "false", 2024-09-17T21:18:08.948 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_debug_extra_checks": "false", 2024-09-17T21:18:08.948 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-17T21:18:08.948 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-17T21:18:08.948 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_debug_no_require_reef": "false", 2024-09-17T21:18:08.948 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_debug_no_require_squid": "false", 2024-09-17T21:18:08.948 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-17T21:18:08.948 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-17T21:18:08.948 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-17T21:18:08.949 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_down_mkfs_grace": "120", 2024-09-17T21:18:08.949 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_down_uptime_grace": "60", 2024-09-17T21:18:08.949 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_election_default_strategy": "1", 2024-09-17T21:18:08.949 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_election_timeout": "5.000000", 2024-09-17T21:18:08.949 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-17T21:18:08.949 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_elector_ping_divisor": "2", 2024-09-17T21:18:08.949 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-17T21:18:08.949 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_enable_op_tracker": "true", 2024-09-17T21:18:08.949 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_fake_pool_delete": "false", 2024-09-17T21:18:08.949 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_force_quorum_join": "false", 2024-09-17T21:18:08.949 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-17T21:18:08.950 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_globalid_prealloc": "10000", 2024-09-17T21:18:08.950 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_health_detail_to_clog": "false", 2024-09-17T21:18:08.950 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_health_log_update_period": "5", 2024-09-17T21:18:08.950 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_health_max_detail": "50", 2024-09-17T21:18:08.950 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_health_to_clog": "true", 2024-09-17T21:18:08.950 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_health_to_clog_interval": "600", 2024-09-17T21:18:08.950 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-17T21:18:08.950 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_host": "172.21.15.83,[v2:172.21.15.83:3301,v1:172.21.15.83:6790],172.21.15.97", 2024-09-17T21:18:08.950 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_host_override": "", 2024-09-17T21:18:08.950 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_initial_members": "", 2024-09-17T21:18:08.950 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-17T21:18:08.950 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-17T21:18:08.951 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-17T21:18:08.951 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-17T21:18:08.951 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-17T21:18:08.951 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_lease": "5.000000", 2024-09-17T21:18:08.951 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-17T21:18:08.951 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-17T21:18:08.951 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_log_full_interval": "50", 2024-09-17T21:18:08.951 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_log_max": "10000", 2024-09-17T21:18:08.951 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_log_max_summary": "50", 2024-09-17T21:18:08.951 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-17T21:18:08.951 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_max_log_epochs": "500", 2024-09-17T21:18:08.952 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-17T21:18:08.952 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-17T21:18:08.952 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_max_nvmeof_epochs": "500", 2024-09-17T21:18:08.952 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_max_osd": "10000", 2024-09-17T21:18:08.952 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_max_pg_per_osd": "10000", 2024-09-17T21:18:08.952 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_max_pool_pg_num": "65536", 2024-09-17T21:18:08.952 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-17T21:18:08.952 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-17T21:18:08.952 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_mds_force_trim_to": "0", 2024-09-17T21:18:08.952 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_mds_skip_sanity": "false", 2024-09-17T21:18:08.952 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_memory_autotune": "true", 2024-09-17T21:18:08.953 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_memory_target": "2147483648", 2024-09-17T21:18:08.953 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_mgr_beacon_grace": "30", 2024-09-17T21:18:08.953 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-17T21:18:08.953 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_mgr_digest_period": "5", 2024-09-17T21:18:08.953 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_mgr_inactive_grace": "60", 2024-09-17T21:18:08.953 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-17T21:18:08.953 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-17T21:18:08.953 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_min_osdmap_epochs": "500", 2024-09-17T21:18:08.953 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_nvmeofgw_beacon_grace": "10", 2024-09-17T21:18:08.953 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2024-09-17T21:18:08.953 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_op_complaint_time": "30", 2024-09-17T21:18:08.954 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_op_history_duration": "600", 2024-09-17T21:18:08.954 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_op_history_size": "20", 2024-09-17T21:18:08.954 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_op_history_slow_op_size": "20", 2024-09-17T21:18:08.954 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-17T21:18:08.954 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_op_log_threshold": "5", 2024-09-17T21:18:08.954 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-17T21:18:08.954 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-17T21:18:08.954 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-17T21:18:08.954 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_auto_mark_in": "false", 2024-09-17T21:18:08.954 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-17T21:18:08.954 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-17T21:18:08.954 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-17T21:18:08.955 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_cache_size": "500", 2024-09-17T21:18:08.955 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-17T21:18:08.955 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-17T21:18:08.955 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-17T21:18:08.955 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_down_out_interval": "600", 2024-09-17T21:18:08.955 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-17T21:18:08.955 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-17T21:18:08.955 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_force_trim_to": "0", 2024-09-17T21:18:08.955 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-17T21:18:08.955 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-17T21:18:08.955 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-17T21:18:08.956 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-17T21:18:08.956 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-17T21:18:08.956 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-17T21:18:08.956 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-17T21:18:08.956 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_min_down_reporters": "2", 2024-09-17T21:18:08.956 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-17T21:18:08.956 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-17T21:18:08.956 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-17T21:18:08.956 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-17T21:18:08.956 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-17T21:18:08.956 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-17T21:18:08.957 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_report_timeout": "900", 2024-09-17T21:18:08.957 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-17T21:18:08.957 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-17T21:18:08.957 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-17T21:18:08.957 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-17T21:18:08.957 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-17T21:18:08.957 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-17T21:18:08.957 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-17T21:18:08.957 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-17T21:18:08.957 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-17T21:18:08.958 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_pg_stuck_threshold": "60", 2024-09-17T21:18:08.958 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-17T21:18:08.958 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-17T21:18:08.958 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-17T21:18:08.958 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-17T21:18:08.958 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-17T21:18:08.958 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-17T21:18:08.958 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_probe_timeout": "2.000000", 2024-09-17T21:18:08.958 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_reweight_max_change": "0.050000", 2024-09-17T21:18:08.958 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_reweight_max_osds": "4", 2024-09-17T21:18:08.958 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-17T21:18:08.958 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-17T21:18:08.959 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-17T21:18:08.959 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-17T21:18:08.959 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-17T21:18:08.959 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_scrub_interval": "86400", 2024-09-17T21:18:08.959 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_scrub_max_keys": "100", 2024-09-17T21:18:08.959 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_scrub_timeout": "300", 2024-09-17T21:18:08.959 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_session_timeout": "300", 2024-09-17T21:18:08.959 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_smart_report_timeout": "5", 2024-09-17T21:18:08.959 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_stat_smooth_intervals": "6", 2024-09-17T21:18:08.959 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-17T21:18:08.959 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_stretch_pool_min_size": "2", 2024-09-17T21:18:08.960 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_stretch_pool_size": "4", 2024-09-17T21:18:08.960 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-17T21:18:08.960 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-17T21:18:08.960 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_sync_debug": "false", 2024-09-17T21:18:08.960 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-17T21:18:08.960 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-17T21:18:08.960 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_sync_provider_kill_at": "0", 2024-09-17T21:18:08.960 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_sync_requester_kill_at": "0", 2024-09-17T21:18:08.960 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_sync_timeout": "60.000000", 2024-09-17T21:18:08.960 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_target_pg_per_osd": "100", 2024-09-17T21:18:08.960 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_tick_interval": "5", 2024-09-17T21:18:08.960 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_timecheck_interval": "300.000000", 2024-09-17T21:18:08.961 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-17T21:18:08.961 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_use_min_delay_socket": "false", 2024-09-17T21:18:08.961 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_older_version_delay": "604800", 2024-09-17T21:18:08.961 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-17T21:18:08.961 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-17T21:18:08.961 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-17T21:18:08.961 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-17T21:18:08.961 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-17T21:18:08.961 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-17T21:18:08.961 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-17T21:18:08.961 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_misplaced": "false", 2024-09-17T21:18:08.962 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-17T21:18:08.962 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_older_version": "true", 2024-09-17T21:18:08.962 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-17T21:18:08.962 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-17T21:18:08.962 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-17T21:18:08.962 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-17T21:18:08.962 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-17T21:18:08.962 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-17T21:18:08.962 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-17T21:18:08.962 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-17T21:18:08.962 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-17T21:18:08.962 INFO:teuthology.orchestra.run.smithi083.stdout: "monmap": "", 2024-09-17T21:18:08.963 INFO:teuthology.orchestra.run.smithi083.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-17T21:18:08.963 INFO:teuthology.orchestra.run.smithi083.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-17T21:18:08.963 INFO:teuthology.orchestra.run.smithi083.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-17T21:18:08.963 INFO:teuthology.orchestra.run.smithi083.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-17T21:18:08.963 INFO:teuthology.orchestra.run.smithi083.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-17T21:18:08.963 INFO:teuthology.orchestra.run.smithi083.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-17T21:18:08.963 INFO:teuthology.orchestra.run.smithi083.stdout: "motr_tracing_enabled": "false", 2024-09-17T21:18:08.963 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_op_threads": "3", 2024-09-17T21:18:08.963 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-17T21:18:08.963 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_cm": "false", 2024-09-17T21:18:08.963 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_device_name": "", 2024-09-17T21:18:08.963 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_dscp": "96", 2024-09-17T21:18:08.964 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-17T21:18:08.964 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-17T21:18:08.964 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_local_gid": "", 2024-09-17T21:18:08.964 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-17T21:18:08.964 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_port_num": "1", 2024-09-17T21:18:08.964 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-17T21:18:08.964 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-17T21:18:08.965 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-17T21:18:08.965 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-17T21:18:08.965 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_sl": "3", 2024-09-17T21:18:08.965 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_support_srq": "true", 2024-09-17T21:18:08.965 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_type": "ib", 2024-09-17T21:18:08.965 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_reap_threshold": "5", 2024-09-17T21:18:08.965 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_bind_before_connect": "false", 2024-09-17T21:18:08.965 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_bind_ipv4": "true", 2024-09-17T21:18:08.965 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_bind_ipv6": "false", 2024-09-17T21:18:08.965 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_bind_msgr1": "true", 2024-09-17T21:18:08.965 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_bind_msgr2": "true", 2024-09-17T21:18:08.966 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_bind_port_max": "7568", 2024-09-17T21:18:08.966 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_bind_port_min": "6800", 2024-09-17T21:18:08.966 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-17T21:18:08.966 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_bind_retry_count": "3", 2024-09-17T21:18:08.966 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_bind_retry_delay": "5", 2024-09-17T21:18:08.966 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_blackhole_client": "false", 2024-09-17T21:18:08.966 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_blackhole_mds": "false", 2024-09-17T21:18:08.966 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_blackhole_mgr": "false", 2024-09-17T21:18:08.966 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_blackhole_mon": "false", 2024-09-17T21:18:08.966 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_blackhole_osd": "false", 2024-09-17T21:18:08.967 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_client_mode": "crc secure", 2024-09-17T21:18:08.967 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_client_throttle_retry_time_interval": "5000", 2024-09-17T21:18:08.967 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_cluster_mode": "crc secure", 2024-09-17T21:18:08.967 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_cluster_type": "", 2024-09-17T21:18:08.967 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_compress_secure": "false", 2024-09-17T21:18:08.967 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_connection_idle_timeout": "900", 2024-09-17T21:18:08.967 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_connection_ready_timeout": "10", 2024-09-17T21:18:08.967 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_crc_data": "true", 2024-09-17T21:18:08.967 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_crc_header": "true", 2024-09-17T21:18:08.967 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_die_on_bad_msg": "false", 2024-09-17T21:18:08.967 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_die_on_bug": "true", 2024-09-17T21:18:08.968 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_die_on_old_message": "true", 2024-09-17T21:18:08.968 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_die_on_skipped_message": "false", 2024-09-17T21:18:08.968 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-17T21:18:08.968 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-17T21:18:08.968 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_coremask": "0xF", 2024-09-17T21:18:08.968 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-17T21:18:08.968 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-17T21:18:08.968 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_enable_tso": "true", 2024-09-17T21:18:08.968 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-17T21:18:08.968 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-17T21:18:08.968 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_hugepages": "", 2024-09-17T21:18:08.968 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-17T21:18:08.969 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-17T21:18:08.969 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_lro": "true", 2024-09-17T21:18:08.969 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_memory_channel": "4", 2024-09-17T21:18:08.969 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-17T21:18:08.969 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_pmd": "", 2024-09-17T21:18:08.969 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_port_id": "0", 2024-09-17T21:18:08.969 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-17T21:18:08.969 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-17T21:18:08.969 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dump_on_send": "false", 2024-09-17T21:18:08.969 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_initial_backoff": "0.200000", 2024-09-17T21:18:08.969 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_inject_delay_max": "1.000000", 2024-09-17T21:18:08.970 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-17T21:18:08.970 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_inject_delay_type": "", 2024-09-17T21:18:08.970 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-17T21:18:08.970 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_inject_network_congestion": "0", 2024-09-17T21:18:08.970 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_inject_socket_failures": "5000", 2024-09-17T21:18:08.970 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_learn_addr_from_peer": "true", 2024-09-17T21:18:08.970 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_max_accept_failures": "4", 2024-09-17T21:18:08.970 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_max_backoff": "15.000000", 2024-09-17T21:18:08.970 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_mon_client_mode": "secure crc", 2024-09-17T21:18:08.970 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-17T21:18:08.970 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_mon_service_mode": "secure crc", 2024-09-17T21:18:08.970 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_osd_compress_min_size": "1024", 2024-09-17T21:18:08.971 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_osd_compress_mode": "none", 2024-09-17T21:18:08.971 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-17T21:18:08.971 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-17T21:18:08.971 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_pq_min_cost": "65536", 2024-09-17T21:18:08.971 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_public_type": "", 2024-09-17T21:18:08.971 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_service_mode": "crc secure", 2024-09-17T21:18:08.971 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_tcp_listen_backlog": "512", 2024-09-17T21:18:08.971 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_tcp_nodelay": "true", 2024-09-17T21:18:08.971 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-17T21:18:08.971 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_tcp_rcvbuf": "0", 2024-09-17T21:18:08.971 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_time_events_min_wait_interval": "1000", 2024-09-17T21:18:08.971 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_type": "async+posix", 2024-09-17T21:18:08.972 INFO:teuthology.orchestra.run.smithi083.stdout: "no_config_file": "false", 2024-09-17T21:18:08.972 INFO:teuthology.orchestra.run.smithi083.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2024-09-17T21:18:08.972 INFO:teuthology.orchestra.run.smithi083.stdout: "nvmeof_mon_client_tick_period": "2", 2024-09-17T21:18:08.972 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter_completion_locks_per_session": "32", 2024-09-17T21:18:08.972 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-17T21:18:08.972 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-17T21:18:08.972 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter_inflight_ops": "1024", 2024-09-17T21:18:08.972 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-17T21:18:08.972 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-17T21:18:08.972 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter_tick_interval": "5.000000", 2024-09-17T21:18:08.972 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter_timeout": "10.000000", 2024-09-17T21:18:08.972 INFO:teuthology.orchestra.run.smithi083.stdout: "objectstore_blackhole": "false", 2024-09-17T21:18:08.973 INFO:teuthology.orchestra.run.smithi083.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-17T21:18:08.973 INFO:teuthology.orchestra.run.smithi083.stdout: "openssl_engine_opts": "", 2024-09-17T21:18:08.973 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_agent_delay_time": "5.000000", 2024-09-17T21:18:08.973 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_agent_hist_halflife": "1000", 2024-09-17T21:18:08.973 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_agent_max_low_ops": "2", 2024-09-17T21:18:08.973 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_agent_max_ops": "4", 2024-09-17T21:18:08.973 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-17T21:18:08.973 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-17T21:18:08.973 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_agent_slop": "0.020000", 2024-09-17T21:18:08.973 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-17T21:18:08.973 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-17T21:18:08.974 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_asio_thread_count": "2", 2024-09-17T21:18:08.974 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_async_recovery_min_cost": "100", 2024-09-17T21:18:08.974 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-17T21:18:08.974 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-17T21:18:08.974 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_backfill_scan_max": "512", 2024-09-17T21:18:08.974 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_backfill_scan_min": "64", 2024-09-17T21:18:08.974 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_backoff_on_degraded": "false", 2024-09-17T21:18:08.974 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_backoff_on_peering": "false", 2024-09-17T21:18:08.974 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_backoff_on_unfound": "true", 2024-09-17T21:18:08.974 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_beacon_report_interval": "300", 2024-09-17T21:18:08.974 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bench_duration": "30", 2024-09-17T21:18:08.974 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-17T21:18:08.975 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bench_max_block_size": "67108864", 2024-09-17T21:18:08.975 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-17T21:18:08.975 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_blkin_trace_all": "false", 2024-09-17T21:18:08.975 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-17T21:18:08.975 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-17T21:18:08.975 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-17T21:18:08.975 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-17T21:18:08.975 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_check_for_log_corruption": "false", 2024-09-17T21:18:08.975 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-17T21:18:08.975 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-17T21:18:08.975 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-17T21:18:08.976 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-17T21:18:08.976 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_class_update_on_start": "true", 2024-09-17T21:18:08.976 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_client_message_cap": "256", 2024-09-17T21:18:08.976 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_client_message_size_cap": "524288000", 2024-09-17T21:18:08.976 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_client_op_priority": "63", 2024-09-17T21:18:08.976 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_client_watch_timeout": "30", 2024-09-17T21:18:08.976 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_command_max_records": "256", 2024-09-17T21:18:08.976 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_compact_on_start": "false", 2024-09-17T21:18:08.976 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-17T21:18:08.976 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-17T21:18:08.976 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-17T21:18:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_crush_update_on_start": "true", 2024-09-17T21:18:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_crush_update_weight_set": "true", 2024-09-17T21:18:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-17T21:18:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-17T21:18:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-17T21:18:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-17T21:18:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-17T21:18:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_feed_pullee": "-1", 2024-09-17T21:18:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-17T21:18:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-17T21:18:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-17T21:18:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_misdirected_ops": "false", 2024-09-17T21:18:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_no_acting_change": "false", 2024-09-17T21:18:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_no_purge_strays": "false", 2024-09-17T21:18:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_op_order": "false", 2024-09-17T21:18:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-17T21:18:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-17T21:18:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-17T21:18:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-17T21:18:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_shutdown": "false", 2024-09-17T21:18:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-17T21:18:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-17T21:18:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_trim_objects": "false", 2024-09-17T21:18:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-17T21:18:08.979 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-17T21:18:08.979 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_verify_snaps": "false", 2024-09-17T21:18:08.979 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-17T21:18:08.979 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-17T21:18:08.979 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_deep_scrub_keys": "1024", 2024-09-17T21:18:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-17T21:18:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-17T21:18:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-17T21:18:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_deep_scrub_stride": "524288", 2024-09-17T21:18:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-17T21:18:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-17T21:18:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_default_notify_timeout": "30", 2024-09-17T21:18:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_delete_sleep": "0.000000", 2024-09-17T21:18:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-17T21:18:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-17T21:18:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-17T21:18:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_discard_disconnected_ops": "true", 2024-09-17T21:18:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_ec_partial_reads": "true", 2024-09-17T21:18:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_enable_op_tracker": "true", 2024-09-17T21:18:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-17T21:18:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-17T21:18:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-17T21:18:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-17T21:18:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_fast_info": "true", 2024-09-17T21:18:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_fast_shutdown": "true", 2024-09-17T21:18:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-17T21:18:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-17T21:18:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-17T21:18:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-17T21:18:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-17T21:18:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_function_tracing": "false", 2024-09-17T21:18:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_heartbeat_grace": "20", 2024-09-17T21:18:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_heartbeat_interval": "6", 2024-09-17T21:18:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-17T21:18:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_heartbeat_min_peers": "10", 2024-09-17T21:18:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_heartbeat_min_size": "2000", 2024-09-17T21:18:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_heartbeat_stale": "600", 2024-09-17T21:18:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-17T21:18:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_hit_set_max_size": "100000", 2024-09-17T21:18:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_hit_set_min_size": "1000", 2024-09-17T21:18:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-17T21:18:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-17T21:18:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-17T21:18:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-17T21:18:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-17T21:18:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-17T21:18:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_journal_size": "5120", 2024-09-17T21:18:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_kill_backfill_at": "0", 2024-09-17T21:18:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-17T21:18:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_map_cache_size": "50", 2024-09-17T21:18:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_map_dedup": "true", 2024-09-17T21:18:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_map_message_max": "40", 2024-09-17T21:18:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-17T21:18:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_map_share_max_epochs": "40", 2024-09-17T21:18:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_attr_name_len": "100", 2024-09-17T21:18:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_attr_size": "0", 2024-09-17T21:18:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_backfills": "1", 2024-09-17T21:18:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_markdown_count": "5", 2024-09-17T21:18:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_markdown_period": "600", 2024-09-17T21:18:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_object_name_len": "2048", 2024-09-17T21:18:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_object_namespace_len": "256", 2024-09-17T21:18:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_object_size": "134217728", 2024-09-17T21:18:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-17T21:18:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-17T21:18:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_pg_blocked_by": "16", 2024-09-17T21:18:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_pg_log_entries": "10000", 2024-09-17T21:18:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-17T21:18:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_pgls": "1024", 2024-09-17T21:18:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_push_cost": "8388608", 2024-09-17T21:18:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_push_objects": "10", 2024-09-17T21:18:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_scrubs": "3", 2024-09-17T21:18:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-17T21:18:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_trimming_pgs": "2", 2024-09-17T21:18:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_write_op_reply_len": "64", 2024-09-17T21:18:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_write_size": "90", 2024-09-17T21:18:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-17T21:18:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-17T21:18:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-17T21:18:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-17T21:18:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-17T21:18:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-17T21:18:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-17T21:18:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-17T21:18:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_profile": "balanced", 2024-09-17T21:18:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-17T21:18:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-17T21:18:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-17T21:18:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-17T21:18:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-17T21:18:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-17T21:18:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-17T21:18:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-17T21:18:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-17T21:18:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-17T21:18:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-17T21:18:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_memory_base": "805306368", 2024-09-17T21:18:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_memory_cache_min": "134217728", 2024-09-17T21:18:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-17T21:18:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-17T21:18:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_memory_target": "4294967296", 2024-09-17T21:18:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_memory_target_autotune": "false", 2024-09-17T21:18:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-17T21:18:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_min_pg_log_entries": "250", 2024-09-17T21:18:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_min_recovery_priority": "0", 2024-09-17T21:18:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-17T21:18:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-17T21:18:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mon_report_interval": "5", 2024-09-17T21:18:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-17T21:18:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-17T21:18:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_num_cache_shards": "32", 2024-09-17T21:18:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_num_op_tracker_shard": "32", 2024-09-17T21:18:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_numa_auto_affinity": "true", 2024-09-17T21:18:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_numa_node": "-1", 2024-09-17T21:18:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_numa_prefer_iface": "true", 2024-09-17T21:18:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-17T21:18:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_objecter_finishers": "1", 2024-09-17T21:18:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_objectstore": "bluestore", 2024-09-17T21:18:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_objectstore_fuse": "false", 2024-09-17T21:18:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_objectstore_ideal_list_max": "64", 2024-09-17T21:18:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_objectstore_tracing": "false", 2024-09-17T21:18:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_complaint_time": "30.000000", 2024-09-17T21:18:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_history_duration": "600", 2024-09-17T21:18:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_history_size": "20", 2024-09-17T21:18:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_history_slow_op_size": "20", 2024-09-17T21:18:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-17T21:18:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_log_threshold": "5", 2024-09-17T21:18:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_num_shards": "0", 2024-09-17T21:18:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_num_shards_hdd": "5", 2024-09-17T21:18:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_num_shards_ssd": "8", 2024-09-17T21:18:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-17T21:18:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-17T21:18:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-17T21:18:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-17T21:18:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_pq_min_cost": "65536", 2024-09-17T21:18:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-17T21:18:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_queue_cut_off": "high", 2024-09-17T21:18:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-17T21:18:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_thread_timeout": "15", 2024-09-17T21:18:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_open_classes_on_start": "true", 2024-09-17T21:18:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_os_flags": "0", 2024-09-17T21:18:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_peering_op_priority": "255", 2024-09-17T21:18:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pg_delete_cost": "1048576", 2024-09-17T21:18:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pg_delete_priority": "5", 2024-09-17T21:18:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-17T21:18:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-17T21:18:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-17T21:18:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pg_log_trim_max": "10000", 2024-09-17T21:18:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pg_log_trim_min": "100", 2024-09-17T21:18:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-17T21:18:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-17T21:18:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-17T21:18:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-17T21:18:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-17T21:18:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-17T21:18:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-17T21:18:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-17T21:18:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-17T21:18:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-17T21:18:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_crimson": "false", 2024-09-17T21:18:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-17T21:18:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-17T21:18:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-17T21:18:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-17T21:18:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-17T21:18:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-17T21:18:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-17T21:18:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-17T21:18:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_flags": "0", 2024-09-17T21:18:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-17T21:18:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_min_size": "0", 2024-09-17T21:18:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-17T21:18:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_pg_num": "32", 2024-09-17T21:18:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_pgp_num": "0", 2024-09-17T21:18:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-17T21:18:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_read_ratio": "70", 2024-09-17T21:18:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_size": "2", 2024-09-17T21:18:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_type": "replicated", 2024-09-17T21:18:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-17T21:18:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-17T21:18:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_push_per_object_cost": "1000", 2024-09-17T21:18:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-17T21:18:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recover_clone_overlap": "true", 2024-09-17T21:18:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-17T21:18:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_cost": "20971520", 2024-09-17T21:18:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-17T21:18:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_max_active": "0", 2024-09-17T21:18:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-17T21:18:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-17T21:18:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-17T21:18:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-17T21:18:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_max_single_start": "1", 2024-09-17T21:18:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_op_priority": "3", 2024-09-17T21:18:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-17T21:18:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_priority": "5", 2024-09-17T21:18:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-17T21:18:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_sleep": "0.000000", 2024-09-17T21:18:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-17T21:18:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-17T21:18:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-17T21:18:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_repair_during_recovery": "false", 2024-09-17T21:18:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_requested_scrub_priority": "120", 2024-09-17T21:18:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-17T21:18:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-17T21:18:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_auto_repair": "false", 2024-09-17T21:18:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-17T21:18:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-17T21:18:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_begin_hour": "0", 2024-09-17T21:18:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_begin_week_day": "0", 2024-09-17T21:18:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_chunk_max": "15", 2024-09-17T21:18:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_chunk_min": "5", 2024-09-17T21:18:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_cost": "52428800", 2024-09-17T21:18:08.998 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-17T21:18:08.998 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_during_recovery": "false", 2024-09-17T21:18:08.998 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_end_hour": "0", 2024-09-17T21:18:08.998 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_end_week_day": "0", 2024-09-17T21:18:08.998 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_event_cost": "4096", 2024-09-17T21:18:08.998 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-17T21:18:08.998 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-17T21:18:08.998 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_invalid_stats": "true", 2024-09-17T21:18:08.998 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-17T21:18:08.998 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-17T21:18:08.998 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_max_preemptions": "5", 2024-09-17T21:18:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-17T21:18:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_priority": "5", 2024-09-17T21:18:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_retry_after_noscrub": "60", 2024-09-17T21:18:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_retry_delay": "30", 2024-09-17T21:18:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_retry_new_interval": "10", 2024-09-17T21:18:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_retry_pg_state": "60", 2024-09-17T21:18:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_retry_trimming": "10", 2024-09-17T21:18:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_sleep": "0.000000", 2024-09-17T21:18:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-17T21:18:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-17T21:18:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-17T21:18:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-09-17T21:18:09.000 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_skip_data_digest": "false", 2024-09-17T21:18:09.000 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_smart_report_timeout": "5", 2024-09-17T21:18:09.000 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_snap_trim_cost": "1048576", 2024-09-17T21:18:09.000 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_snap_trim_priority": "5", 2024-09-17T21:18:09.000 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-17T21:18:09.000 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-17T21:18:09.000 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-17T21:18:09.000 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-17T21:18:09.000 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-17T21:18:09.000 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-17T21:18:09.000 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-17T21:18:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_target_transaction_size": "30", 2024-09-17T21:18:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-17T21:18:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-17T21:18:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-17T21:18:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-17T21:18:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-17T21:18:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-17T21:18:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-17T21:18:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-17T21:18:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-17T21:18:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-17T21:18:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_tracing": "false", 2024-09-17T21:18:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_use_stale_snap": "false", 2024-09-17T21:18:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-17T21:18:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: "osdc_blkin_trace_all": "false", 2024-09-17T21:18:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: "paxos_kill_at": "0", 2024-09-17T21:18:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: "paxos_max_join_drift": "10", 2024-09-17T21:18:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: "paxos_min": "500", 2024-09-17T21:18:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: "paxos_min_wait": "0.050000", 2024-09-17T21:18:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: "paxos_propose_interval": "1.000000", 2024-09-17T21:18:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: "paxos_service_trim_max": "500", 2024-09-17T21:18:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-17T21:18:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: "paxos_service_trim_min": "250", 2024-09-17T21:18:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: "paxos_stash_full_interval": "25", 2024-09-17T21:18:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: "paxos_trim_max": "500", 2024-09-17T21:18:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: "paxos_trim_min": "250", 2024-09-17T21:18:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: "perf": "true", 2024-09-17T21:18:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-17T21:18:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-17T21:18:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-17T21:18:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "-", 2024-09-17T21:18:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrv": "", 2024-09-17T21:18:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: "public_bind_addr": "-", 2024-09-17T21:18:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: "public_network": "", 2024-09-17T21:18:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: "public_network_interface": "", 2024-09-17T21:18:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: "qat_compressor_busy_polling": "false", 2024-09-17T21:18:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: "qat_compressor_enabled": "false", 2024-09-17T21:18:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: "qat_compressor_session_max_number": "256", 2024-09-17T21:18:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: "rados_mon_op_timeout": "0", 2024-09-17T21:18:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: "rados_osd_op_timeout": "0", 2024-09-17T21:18:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: "rados_tracing": "false", 2024-09-17T21:18:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_atime_update_interval": "60", 2024-09-17T21:18:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-17T21:18:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_balance_parent_reads": "false", 2024-09-17T21:18:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_balance_snap_reads": "false", 2024-09-17T21:18:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_blkin_trace_all": "false", 2024-09-17T21:18:09.005 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-17T21:18:09.005 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-17T21:18:09.005 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_cache": "true", 2024-09-17T21:18:09.005 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-17T21:18:09.005 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-17T21:18:09.005 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-17T21:18:09.005 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-17T21:18:09.005 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_cache_policy": "writearound", 2024-09-17T21:18:09.005 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_cache_size": "33554432", 2024-09-17T21:18:09.005 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-17T21:18:09.005 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-17T21:18:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_clone_copy_on_read": "false", 2024-09-17T21:18:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_compression_hint": "none", 2024-09-17T21:18:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_concurrent_management_ops": "10", 2024-09-17T21:18:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-17T21:18:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_default_clone_format": "auto", 2024-09-17T21:18:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_default_data_pool": "", 2024-09-17T21:18:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_default_features": "61", 2024-09-17T21:18:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_default_format": "2", 2024-09-17T21:18:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_default_map_options": "", 2024-09-17T21:18:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_default_order": "22", 2024-09-17T21:18:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_default_pool": "rbd", 2024-09-17T21:18:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-17T21:18:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_default_stripe_count": "0", 2024-09-17T21:18:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_default_stripe_unit": "0", 2024-09-17T21:18:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-17T21:18:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-17T21:18:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-17T21:18:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_enable_alloc_hint": "true", 2024-09-17T21:18:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-17T21:18:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_io_scheduler": "simple", 2024-09-17T21:18:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-17T21:18:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-17T21:18:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-17T21:18:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-17T21:18:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-17T21:18:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-17T21:18:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-17T21:18:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-17T21:18:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-17T21:18:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_journal_order": "24", 2024-09-17T21:18:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_journal_pool": "", 2024-09-17T21:18:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_journal_splay_width": "4", 2024-09-17T21:18:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_localize_parent_reads": "false", 2024-09-17T21:18:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_localize_snap_reads": "false", 2024-09-17T21:18:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-17T21:18:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-17T21:18:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-17T21:18:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-17T21:18:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-17T21:18:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-17T21:18:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-17T21:18:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-17T21:18:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-17T21:18:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-17T21:18:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-17T21:18:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-17T21:18:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-17T21:18:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-17T21:18:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-17T21:18:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-17T21:18:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-17T21:18:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-17T21:18:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-17T21:18:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-17T21:18:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-17T21:18:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-17T21:18:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-17T21:18:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-17T21:18:09.011 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-17T21:18:09.011 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-17T21:18:09.011 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-17T21:18:09.011 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-17T21:18:09.011 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-17T21:18:09.011 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-17T21:18:09.011 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-17T21:18:09.011 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mtime_update_interval": "60", 2024-09-17T21:18:09.011 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_non_blocking_aio": "true", 2024-09-17T21:18:09.011 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_op_thread_timeout": "60", 2024-09-17T21:18:09.011 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_op_threads": "1", 2024-09-17T21:18:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_parent_cache_enabled": "false", 2024-09-17T21:18:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-17T21:18:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-17T21:18:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-17T21:18:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_plugins": "", 2024-09-17T21:18:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_bps_burst": "0", 2024-09-17T21:18:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-17T21:18:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_bps_limit": "0", 2024-09-17T21:18:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_exclude_ops": "0", 2024-09-17T21:18:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_iops_burst": "0", 2024-09-17T21:18:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-17T21:18:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_iops_limit": "0", 2024-09-17T21:18:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-17T21:18:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-17T21:18:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-17T21:18:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-17T21:18:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-17T21:18:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-17T21:18:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-17T21:18:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-17T21:18:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-17T21:18:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-17T21:18:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-17T21:18:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-17T21:18:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-17T21:18:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-17T21:18:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_read_from_replica_policy": "default", 2024-09-17T21:18:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-17T21:18:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-17T21:18:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-17T21:18:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-17T21:18:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_skip_partial_discard": "true", 2024-09-17T21:18:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-17T21:18:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_tracing": "false", 2024-09-17T21:18:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_validate_names": "true", 2024-09-17T21:18:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_validate_pool": "true", 2024-09-17T21:18:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: "restapi_base_url": "", 2024-09-17T21:18:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: "restapi_log_level": "", 2024-09-17T21:18:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-17T21:18:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-17T21:18:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_acl_grants_max_num": "100", 2024-09-17T21:18:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_admin_entry": "admin", 2024-09-17T21:18:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-17T21:18:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_asio_assert_yielding": "false", 2024-09-17T21:18:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_backend_store": "rados", 2024-09-17T21:18:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_barbican_url": "", 2024-09-17T21:18:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_beast_enable_async": "true", 2024-09-17T21:18:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_bucket_counters_cache": "false", 2024-09-17T21:18:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-17T21:18:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-17T21:18:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-17T21:18:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-17T21:18:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-17T21:18:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-17T21:18:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-17T21:18:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-17T21:18:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_cache_enabled": "true", 2024-09-17T21:18:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_cache_expiry_interval": "900", 2024-09-17T21:18:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_cache_lru_size": "25000", 2024-09-17T21:18:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_config_store": "rados", 2024-09-17T21:18:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_content_length_compat": "false", 2024-09-17T21:18:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_copy_obj_progress": "true", 2024-09-17T21:18:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-17T21:18:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_cors_rules_max_num": "100", 2024-09-17T21:18:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_cross_domain_policy": "", 2024-09-17T21:18:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-17T21:18:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_kmip_addr": "", 2024-09-17T21:18:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-17T21:18:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-17T21:18:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-17T21:18:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-17T21:18:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_kmip_password": "", 2024-09-17T21:18:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-17T21:18:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_kmip_username": "", 2024-09-17T21:18:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_require_ssl": "true", 2024-09-17T21:18:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-17T21:18:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-17T21:18:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-17T21:18:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-17T21:18:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-17T21:18:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-17T21:18:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-17T21:18:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-17T21:18:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-17T21:18:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-17T21:18:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-17T21:18:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-17T21:18:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-17T21:18:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-17T21:18:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-17T21:18:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_vault_addr": "", 2024-09-17T21:18:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_vault_auth": "token", 2024-09-17T21:18:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_vault_namespace": "", 2024-09-17T21:18:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_vault_prefix": "", 2024-09-17T21:18:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-17T21:18:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-17T21:18:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-17T21:18:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-17T21:18:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_vault_token_file": "", 2024-09-17T21:18:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-17T21:18:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_curl_buffersize": "524288", 2024-09-17T21:18:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-17T21:18:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_curl_low_speed_time": "300", 2024-09-17T21:18:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-17T21:18:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-17T21:18:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-17T21:18:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-17T21:18:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-17T21:18:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-17T21:18:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-17T21:18:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-17T21:18:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-17T21:18:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-17T21:18:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2024-09-17T21:18:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2024-09-17T21:18:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-09-17T21:18:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-09-17T21:18:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-09-17T21:18:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d4n_l1_fadvise": "4", 2024-09-17T21:18:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-09-17T21:18:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-09-17T21:18:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-17T21:18:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_data_log_changes_size": "1000", 2024-09-17T21:18:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_data_log_num_shards": "128", 2024-09-17T21:18:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-17T21:18:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_data_log_window": "30", 2024-09-17T21:18:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-17T21:18:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-17T21:18:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-17T21:18:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-17T21:18:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-17T21:18:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-17T21:18:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-17T21:18:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-17T21:18:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-17T21:18:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-17T21:18:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-17T21:18:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-17T21:18:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-17T21:18:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_disable_s3select": "false", 2024-09-17T21:18:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-17T21:18:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-17T21:18:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-17T21:18:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-17T21:18:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-17T21:18:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-17T21:18:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-17T21:18:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-17T21:18:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-17T21:18:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-17T21:18:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-17T21:18:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-17T21:18:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dns_name": "", 2024-09-17T21:18:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dns_s3website_name": "", 2024-09-17T21:18:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dynamic_resharding": "true", 2024-09-17T21:18:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2024-09-17T21:18:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2024-09-17T21:18:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-17T21:18:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_enable_gc_threads": "true", 2024-09-17T21:18:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_enable_lc_threads": "true", 2024-09-17T21:18:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_enable_mdsearch": "true", 2024-09-17T21:18:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_enable_ops_log": "false", 2024-09-17T21:18:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_enable_quota_threads": "true", 2024-09-17T21:18:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_enable_static_website": "false", 2024-09-17T21:18:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_enable_usage_log": "false", 2024-09-17T21:18:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_enforce_swift_acls": "true", 2024-09-17T21:18:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_exit_timeout_secs": "120", 2024-09-17T21:18:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_expose_bucket": "false", 2024-09-17T21:18:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_extended_http_attrs": "", 2024-09-17T21:18:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_filter": "none", 2024-09-17T21:18:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-17T21:18:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_frontends": "beast port=7480", 2024-09-17T21:18:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-17T21:18:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_gc_max_deferred": "50", 2024-09-17T21:18:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-17T21:18:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_gc_max_objs": "32", 2024-09-17T21:18:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-17T21:18:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-17T21:18:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-17T21:18:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-17T21:18:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_gc_processor_period": "3600", 2024-09-17T21:18:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-17T21:18:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-17T21:18:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-17T21:18:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-17T21:18:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_init_timeout": "300", 2024-09-17T21:18:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-17T21:18:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-17T21:18:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_kafka_connection_idle": "300", 2024-09-17T21:18:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-17T21:18:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-17T21:18:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-17T21:18:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-17T21:18:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-17T21:18:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_admin_domain": "", 2024-09-17T21:18:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_admin_password": "", 2024-09-17T21:18:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_admin_password_path": "", 2024-09-17T21:18:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_admin_project": "", 2024-09-17T21:18:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_admin_tenant": "", 2024-09-17T21:18:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_admin_token": "", 2024-09-17T21:18:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_admin_token_path": "", 2024-09-17T21:18:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_admin_user": "", 2024-09-17T21:18:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_barbican_domain": "", 2024-09-17T21:18:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_barbican_password": "", 2024-09-17T21:18:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_barbican_project": "", 2024-09-17T21:18:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-17T21:18:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_barbican_user": "", 2024-09-17T21:18:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-17T21:18:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-17T21:18:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-17T21:18:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-17T21:18:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-17T21:18:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_url": "", 2024-09-17T21:18:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-17T21:18:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_lc_debug_interval": "-1", 2024-09-17T21:18:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_lc_lock_max_time": "90", 2024-09-17T21:18:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_lc_max_objs": "32", 2024-09-17T21:18:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_lc_max_rules": "1000", 2024-09-17T21:18:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_lc_max_worker": "3", 2024-09-17T21:18:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-17T21:18:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_lc_thread_delay": "0", 2024-09-17T21:18:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-17T21:18:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_ldap_dnattr": "uid", 2024-09-17T21:18:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-17T21:18:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_ldap_searchfilter": "", 2024-09-17T21:18:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-17T21:18:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_ldap_uri": "", 2024-09-17T21:18:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_lfuda_sync_frequency": "60", 2024-09-17T21:18:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-17T21:18:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-17T21:18:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-17T21:18:09.032 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_log_http_headers": "", 2024-09-17T21:18:09.032 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-17T21:18:09.032 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-17T21:18:09.032 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_log_object_name_utc": "false", 2024-09-17T21:18:09.032 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-17T21:18:09.032 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-17T21:18:09.032 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_attr_name_len": "0", 2024-09-17T21:18:09.032 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_attr_size": "0", 2024-09-17T21:18:09.032 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-17T21:18:09.032 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_chunk_size": "4194304", 2024-09-17T21:18:09.032 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-17T21:18:09.032 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-17T21:18:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-17T21:18:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_listing_results": "5000", 2024-09-17T21:18:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_notify_retries": "10", 2024-09-17T21:18:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-17T21:18:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_put_param_size": "1048576", 2024-09-17T21:18:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_put_size": "5368709120", 2024-09-17T21:18:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_slo_entries": "1000", 2024-09-17T21:18:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_md_log_max_shards": "64", 2024-09-17T21:18:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-17T21:18:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-17T21:18:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-17T21:18:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-17T21:18:09.034 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_mp_lock_max_time": "600", 2024-09-17T21:18:09.034 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-17T21:18:09.034 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-17T21:18:09.034 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-17T21:18:09.034 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-17T21:18:09.034 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-17T21:18:09.034 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-17T21:18:09.034 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-17T21:18:09.034 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-17T21:18:09.034 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_max_gc": "300", 2024-09-17T21:18:09.034 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-17T21:18:09.035 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-17T21:18:09.035 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-17T21:18:09.035 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-17T21:18:09.035 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-17T21:18:09.035 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-17T21:18:09.035 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-17T21:18:09.035 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_num_async_rados_threads": "32", 2024-09-17T21:18:09.035 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_num_control_oids": "8", 2024-09-17T21:18:09.035 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_numa_node": "-1", 2024-09-17T21:18:09.035 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-17T21:18:09.035 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-17T21:18:09.035 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_objexp_chunk_size": "100", 2024-09-17T21:18:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_objexp_gc_interval": "600", 2024-09-17T21:18:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-17T21:18:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-17T21:18:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-17T21:18:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_op_thread_timeout": "600", 2024-09-17T21:18:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_op_tracing": "false", 2024-09-17T21:18:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_opa_token": "", 2024-09-17T21:18:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_opa_url": "", 2024-09-17T21:18:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_opa_verify_ssl": "true", 2024-09-17T21:18:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-17T21:18:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-17T21:18:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_ops_log_rados": "false", 2024-09-17T21:18:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_ops_log_socket_path": "", 2024-09-17T21:18:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-17T21:18:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-17T21:18:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-17T21:18:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_period_push_interval": "2.000000", 2024-09-17T21:18:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-17T21:18:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-17T21:18:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-17T21:18:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-17T21:18:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_posix_cache_lanes": "3", 2024-09-17T21:18:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-17T21:18:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-17T21:18:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_posix_cache_partitions": "3", 2024-09-17T21:18:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-17T21:18:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_print_continue": "true", 2024-09-17T21:18:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-17T21:18:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-17T21:18:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-17T21:18:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-17T21:18:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-17T21:18:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_rados_tracing": "false", 2024-09-17T21:18:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_realm": "", 2024-09-17T21:18:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_realm_id": "", 2024-09-17T21:18:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-17T21:18:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_region": "", 2024-09-17T21:18:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-17T21:18:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-17T21:18:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-17T21:18:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-17T21:18:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_request_uri": "", 2024-09-17T21:18:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_reshard_batch_size": "64", 2024-09-17T21:18:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-17T21:18:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_reshard_debug_interval": "-1", 2024-09-17T21:18:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_reshard_max_aio": "128", 2024-09-17T21:18:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_reshard_num_logs": "16", 2024-09-17T21:18:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_reshard_progress_judge_interval": "120", 2024-09-17T21:18:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_reshard_progress_judge_ratio": "0.500000", 2024-09-17T21:18:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_reshard_thread_interval": "600", 2024-09-17T21:18:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_reshardlog_threshold": "30000", 2024-09-17T21:18:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_resolve_cname": "false", 2024-09-17T21:18:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-17T21:18:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_run_sync_thread": "true", 2024-09-17T21:18:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-17T21:18:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-17T21:18:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-17T21:18:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-17T21:18:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-17T21:18:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-17T21:18:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-17T21:18:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-17T21:18:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-17T21:18:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_scheduler_type": "throttler", 2024-09-17T21:18:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_script_uri": "", 2024-09-17T21:18:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_service_provider_name": "", 2024-09-17T21:18:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-17T21:18:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sts_client_id": "", 2024-09-17T21:18:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sts_client_secret": "", 2024-09-17T21:18:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sts_entry": "sts", 2024-09-17T21:18:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sts_key": "sts", 2024-09-17T21:18:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-17T21:18:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sts_min_session_duration": "900", 2024-09-17T21:18:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sts_token_introspection_url": "", 2024-09-17T21:18:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_swift_account_in_url": "false", 2024-09-17T21:18:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_swift_auth_entry": "auth", 2024-09-17T21:18:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_swift_auth_url": "", 2024-09-17T21:18:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_swift_custom_header": "", 2024-09-17T21:18:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-17T21:18:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_swift_need_stats": "true", 2024-09-17T21:18:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_swift_tenant_name": "", 2024-09-17T21:18:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_swift_token_expiration": "86400", 2024-09-17T21:18:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_swift_url": "", 2024-09-17T21:18:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_swift_url_prefix": "swift", 2024-09-17T21:18:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-17T21:18:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-17T21:18:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-17T21:18:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_lease_period": "120", 2024-09-17T21:18:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-17T21:18:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-17T21:18:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-17T21:18:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-17T21:18:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-17T21:18:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-17T21:18:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-17T21:18:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-17T21:18:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-17T21:18:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_thread_pool_size": "512", 2024-09-17T21:18:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-17T21:18:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-17T21:18:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-17T21:18:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-17T21:18:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_torrent_comment": "", 2024-09-17T21:18:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_torrent_createby": "", 2024-09-17T21:18:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_torrent_encoding": "", 2024-09-17T21:18:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_torrent_flag": "false", 2024-09-17T21:18:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-17T21:18:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_torrent_origin": "", 2024-09-17T21:18:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-17T21:18:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_torrent_tracker": "", 2024-09-17T21:18:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_trust_forwarded_https": "false", 2024-09-17T21:18:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-17T21:18:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-17T21:18:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_usage_max_shards": "32", 2024-09-17T21:18:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_usage_max_user_shards": "1", 2024-09-17T21:18:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_use_opa_authz": "false", 2024-09-17T21:18:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_user_counters_cache": "false", 2024-09-17T21:18:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-17T21:18:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-17T21:18:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-17T21:18:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_user_max_buckets": "1000", 2024-09-17T21:18:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_user_policies_max_num": "100", 2024-09-17T21:18:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-17T21:18:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-17T21:18:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-17T21:18:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-17T21:18:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_user_unique_email": "true", 2024-09-17T21:18:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_verify_ssl": "true", 2024-09-17T21:18:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-17T21:18:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_zone": "", 2024-09-17T21:18:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_zone_id": "", 2024-09-17T21:18:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-17T21:18:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_zonegroup": "", 2024-09-17T21:18:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_zonegroup_id": "", 2024-09-17T21:18:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-17T21:18:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: "rgwlc_auto_session_clear": "true", 2024-09-17T21:18:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-17T21:18:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_block_size": "4096", 2024-09-17T21:18:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-17T21:18:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-17T21:18:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-17T21:18:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-17T21:18:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-17T21:18:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_cache_size": "536870912", 2024-09-17T21:18:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-17T21:18:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-17T21:18:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-17T21:18:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-17T21:18:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-17T21:18:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-17T21:18:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-17T21:18:09.049 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-17T21:18:09.049 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_index_type": "binary_search", 2024-09-17T21:18:09.049 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-17T21:18:09.049 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-17T21:18:09.049 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_partition_filters": "false", 2024-09-17T21:18:09.049 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_perf": "false", 2024-09-17T21:18:09.049 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-17T21:18:09.049 INFO:teuthology.orchestra.run.smithi083.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-17T21:18:09.049 INFO:teuthology.orchestra.run.smithi083.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-17T21:18:09.049 INFO:teuthology.orchestra.run.smithi083.stdout: "run_dir": "/var/run/ceph", 2024-09-17T21:18:09.049 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_block_create": "true", 2024-09-17T21:18:09.049 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_cache_lru_size": "67108864", 2024-09-17T21:18:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-17T21:18:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-17T21:18:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_default_max_object_size": "16777216", 2024-09-17T21:18:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-17T21:18:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_device_size": "53687091200", 2024-09-17T21:18:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-17T21:18:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_full_integrity_check": "false", 2024-09-17T21:18:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_journal_batch_capacity": "16", 2024-09-17T21:18:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-17T21:18:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-17T21:18:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-17T21:18:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_main_device_type": "SSD", 2024-09-17T21:18:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-17T21:18:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-17T21:18:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-17T21:18:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-17T21:18:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-17T21:18:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-17T21:18:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_segment_size": "67108864", 2024-09-17T21:18:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: "set_keepcaps": "false", 2024-09-17T21:18:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: "setgroup": "", 2024-09-17T21:18:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: "setuser": "", 2024-09-17T21:18:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: "setuser_match_path": "", 2024-09-17T21:18:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-17T21:18:09.052 INFO:teuthology.orchestra.run.smithi083.stdout: "thp": "false", 2024-09-17T21:18:09.052 INFO:teuthology.orchestra.run.smithi083.stdout: "threadpool_default_timeout": "60", 2024-09-17T21:18:09.052 INFO:teuthology.orchestra.run.smithi083.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-17T21:18:09.052 INFO:teuthology.orchestra.run.smithi083.stdout: "throttler_perf_counter": "true", 2024-09-17T21:18:09.052 INFO:teuthology.orchestra.run.smithi083.stdout: "tmp_dir": "/tmp", 2024-09-17T21:18:09.052 INFO:teuthology.orchestra.run.smithi083.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2024-09-17T21:18:09.052 INFO:teuthology.orchestra.run.smithi083.stdout: "uadk_compressor_enabled": "false", 2024-09-17T21:18:09.052 INFO:teuthology.orchestra.run.smithi083.stdout: "uadk_wd_sync_ctx_num": "2" 2024-09-17T21:18:09.052 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:09.054 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-17T21:18:09.194 INFO:teuthology.orchestra.run.smithi083.stdout:mon.a: {} 2024-09-17T21:18:09.194 INFO:teuthology.orchestra.run.smithi083.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T21:18:09.206 INFO:teuthology.orchestra.run.smithi083.stdout:mon.b: {} 2024-09-17T21:18:09.206 INFO:teuthology.orchestra.run.smithi083.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T21:18:09.215 INFO:teuthology.orchestra.run.smithi083.stdout:mon.c: {} 2024-09-17T21:18:09.215 INFO:teuthology.orchestra.run.smithi083.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T21:18:09.231 DEBUG:teuthology.orchestra.run.smithi097:> type ceph-objectstore-tool 2024-09-17T21:18:09.237 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T21:18:09.237 DEBUG:teuthology.orchestra.run.smithi083:> type ceph-objectstore-tool 2024-09-17T21:18:09.242 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T21:18:09.242 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-17T21:18:09.256 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-17T21:18:09.256 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-17T21:18:09.256 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:18:09.257 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:18:09.257 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-17T21:18:09.257 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-17T21:18:09.257 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-17T21:18:09.258 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-17T21:18:09.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:18:09.258 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-17T21:18:09.259 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:18:09.259 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-17T21:18:09.259 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:18:09.260 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0 2024-09-17T21:18:09.289 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:09.284+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:09.360 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:09.358+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:09.404 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:09.404 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:09.404 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:09.405 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:09.416 INFO:teuthology.orchestra.run.smithi184.stdout:mapped writes DISABLED 2024-09-17T21:18:09.416 INFO:teuthology.orchestra.run.smithi184.stdout:mapped reads DISABLED 2024-09-17T21:18:09.416 INFO:teuthology.orchestra.run.smithi184.stdout:Seed set to 6659 2024-09-17T21:18:09.416 INFO:teuthology.orchestra.run.smithi184.stdout:rbd-nbd mode enabled 2024-09-17T21:18:09.420 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:18:09.431 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:18:09.432 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:18:09.457 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-17T21:18:09.457 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-17T21:18:09.458 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-17T21:18:09.458 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-17T21:18:09.458 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-17T21:18:09.458 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-17T21:18:09.458 INFO:teuthology.orchestra.run.smithi083.stderr:bench [] [] [] [] 2024-09-17T21:18:09.458 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs debug_inject_read_zeros 2024-09-17T21:18:09.458 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs files list 2024-09-17T21:18:09.458 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs stats 2024-09-17T21:18:09.458 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator dump block 2024-09-17T21:18:09.458 INFO:teuthology.orchestra.run.smithi083.stderr:Error EINVAL: invalid command 2024-09-17T21:18:09.461 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:09.456+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:09.461 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T21:18:09.462 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T21:18:09.462 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-17T21:18:09.463 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:18:09.464 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:18:09.481 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:18:09.481 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:18:09.501 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:18:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:18:09.533 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:09.533 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:09.536 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:18:09.536 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:18:09.547 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:18:09.572 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:18:09.573 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:18:09.573 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:09.568+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:09.617 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-17T21:18:09.617 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-17T21:18:09.617 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-17T21:18:09.618 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-17T21:18:09.618 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-17T21:18:09.618 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-17T21:18:09.618 INFO:teuthology.orchestra.run.smithi083.stderr:bench [] [] [] [] 2024-09-17T21:18:09.618 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs debug_inject_read_zeros 2024-09-17T21:18:09.618 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs files list 2024-09-17T21:18:09.618 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs stats 2024-09-17T21:18:09.618 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator dump block 2024-09-17T21:18:09.618 INFO:teuthology.orchestra.run.smithi083.stderr:Error EINVAL: invalid command 2024-09-17T21:18:09.620 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T21:18:09.620 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T21:18:09.620 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-17T21:18:09.664 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:09.660+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:09.681 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:09.682 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:09.694 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:18:09.764 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:09.758+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:09.812 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-17T21:18:09.812 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-17T21:18:09.813 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-17T21:18:09.813 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-17T21:18:09.813 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-17T21:18:09.813 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-17T21:18:09.813 INFO:teuthology.orchestra.run.smithi083.stderr:bench [] [] [] [] 2024-09-17T21:18:09.813 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs debug_inject_read_zeros 2024-09-17T21:18:09.813 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs files list 2024-09-17T21:18:09.813 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs stats 2024-09-17T21:18:09.813 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator dump block 2024-09-17T21:18:09.813 INFO:teuthology.orchestra.run.smithi083.stderr:Error EINVAL: invalid command 2024-09-17T21:18:09.815 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T21:18:09.816 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T21:18:09.816 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-17T21:18:09.853 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:09.853 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:09.865 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:09.858+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:09.867 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:18:09.951 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-17T21:18:09.951 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-17T21:18:09.951 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-17T21:18:09.951 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-17T21:18:09.951 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-17T21:18:09.951 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-17T21:18:09.952 INFO:teuthology.orchestra.run.smithi083.stderr:bench [] [] [] [] 2024-09-17T21:18:09.952 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs debug_inject_read_zeros 2024-09-17T21:18:09.952 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs files list 2024-09-17T21:18:09.952 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs stats 2024-09-17T21:18:09.952 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator dump block 2024-09-17T21:18:09.952 INFO:teuthology.orchestra.run.smithi083.stderr:Error EINVAL: invalid command 2024-09-17T21:18:09.953 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T21:18:09.954 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T21:18:09.954 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-17T21:18:09.965 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:09.962+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:10.025 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:10.025 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:10.039 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:18:10.084 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:10.080+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:10.139 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-17T21:18:10.139 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-17T21:18:10.139 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-17T21:18:10.139 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-17T21:18:10.139 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-17T21:18:10.139 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-17T21:18:10.139 INFO:teuthology.orchestra.run.smithi083.stderr:bench [] [] [] [] 2024-09-17T21:18:10.139 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs debug_inject_read_zeros 2024-09-17T21:18:10.139 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs files list 2024-09-17T21:18:10.139 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs stats 2024-09-17T21:18:10.139 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator dump block 2024-09-17T21:18:10.139 INFO:teuthology.orchestra.run.smithi083.stderr:Error EINVAL: invalid command 2024-09-17T21:18:10.141 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T21:18:10.141 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T21:18:10.142 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-17T21:18:10.167 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:10.162+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:10.199 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:10.199 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:10.214 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:18:10.268 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:10.260+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:10.275 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-17T21:18:10.304 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-17T21:18:10.304 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-17T21:18:10.304 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-17T21:18:10.304 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-17T21:18:10.305 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-17T21:18:10.305 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-17T21:18:10.305 INFO:teuthology.orchestra.run.smithi083.stderr:bench [] [] [] [] 2024-09-17T21:18:10.305 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs debug_inject_read_zeros 2024-09-17T21:18:10.305 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs files list 2024-09-17T21:18:10.305 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs stats 2024-09-17T21:18:10.305 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator dump block 2024-09-17T21:18:10.305 INFO:teuthology.orchestra.run.smithi083.stderr:Error EINVAL: invalid command 2024-09-17T21:18:10.307 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T21:18:10.307 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T21:18:10.341 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:10.341 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:10.355 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:18:10.396 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:10.392+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:10.471 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:10.466+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:10.515 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:10.515 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:10.529 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:18:10.576 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:10.572+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:10.645 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:10.645 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:10.659 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:18:10.701 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:10.696+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:10.774 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:10.770+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:10.790 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:10.790 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:10.804 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:18:10.890 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:10.886+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:10.967 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:10.967 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:10.975 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:10.972+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:10.980 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:18:11.076 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:11.068+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:11.146 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:11.146 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:11.161 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:18:11.206 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:11.202+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:11.278 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:11.274+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:11.289 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:11.289 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:11.311 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:18:11.379 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:11.376+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:11.471 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:11.471 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:11.479 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:11.472+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:11.485 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:18:11.581 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:11.574+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:11.591 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:18:11.607 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:11.607 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:11.621 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:18:11.713 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:11.710+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:11.742 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:18:11.743 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:18:11.759 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:18:11.759 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:18:11.776 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:18:11.776 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:18:11.782 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:11.778+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:11.794 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:11.794 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:11.796 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:18:11.796 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:18:11.809 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:18:11.814 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:18:11.814 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:18:11.834 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:18:11.834 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:18:11.883 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:11.876+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:11.972 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:11.972 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:11.972 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:18:11.972 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:18:11.972 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:11.984 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:11.980+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:11.985 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:18:12.085 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:12.080+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:12.104 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:12.104 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4190.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-17T21:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:01.049433+0000", 2024-09-17T21:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 11.053795069, 2024-09-17T21:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0064286639999999997, 2024-09-17T21:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4190", 2024-09-17T21:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.83:0/956669820", 2024-09-17T21:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.049433+0000", 2024-09-17T21:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.049433+0000", 2024-09-17T21:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.049444+0000", 2024-09-17T21:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1559999999999999e-05 2024-09-17T21:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.049458+0000", 2024-09-17T21:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3312999999999999e-05 2024-09-17T21:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.049487+0000", 2024-09-17T21:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9689999999999999e-05 2024-09-17T21:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.049520+0000", 2024-09-17T21:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3012000000000002e-05 2024-09-17T21:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.049699+0000", 2024-09-17T21:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017853300000000001 2024-09-17T21:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.050087+0000", 2024-09-17T21:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038779299999999999 2024-09-17T21:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.050435+0000", 2024-09-17T21:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00034841700000000001 2024-09-17T21:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.051936+0000", 2024-09-17T21:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001501053 2024-09-17T21:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.055707+0000", 2024-09-17T21:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0037706340000000001 2024-09-17T21:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.055790+0000", 2024-09-17T21:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3442000000000005e-05 2024-09-17T21:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.055861+0000", 2024-09-17T21:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1217000000000006e-05 2024-09-17T21:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4190.0:1 2.3 e19/18 ondisk, result = 0)", 2024-09-17T21:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:01.055523+0000", 2024-09-17T21:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 11.047704578999999, 2024-09-17T21:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035326699999999999, 2024-09-17T21:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.055523+0000", 2024-09-17T21:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.055523+0000", 2024-09-17T21:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.055525+0000", 2024-09-17T21:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.72e-06 2024-09-17T21:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.055534+0000", 2024-09-17T21:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4960000000000002e-06 2024-09-17T21:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.055563+0000", 2024-09-17T21:18:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8527e-05 2024-09-17T21:18:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.055577+0000", 2024-09-17T21:18:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3883000000000001e-05 2024-09-17T21:18:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.055675+0000", 2024-09-17T21:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8072000000000006e-05 2024-09-17T21:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.055696+0000", 2024-09-17T21:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1310000000000001e-05 2024-09-17T21:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.055876+0000", 2024-09-17T21:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000180259 2024-09-17T21:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4190.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-17T21:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:01.059475+0000", 2024-09-17T21:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 11.043752509999999, 2024-09-17T21:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037196700000000001, 2024-09-17T21:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "reached pg", 2024-09-17T21:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4190", 2024-09-17T21:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.83:0/956669820", 2024-09-17T21:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2 2024-09-17T21:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.059475+0000", 2024-09-17T21:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.059475+0000", 2024-09-17T21:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.059478+0000", 2024-09-17T21:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3e-06 2024-09-17T21:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.059489+0000", 2024-09-17T21:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1569999999999999e-05 2024-09-17T21:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.059520+0000", 2024-09-17T21:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0902999999999999e-05 2024-09-17T21:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.059536+0000", 2024-09-17T21:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5726000000000002e-05 2024-09-17T21:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.059644+0000", 2024-09-17T21:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000107936 2024-09-17T21:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.059847+0000", 2024-09-17T21:18:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000203532 2024-09-17T21:18:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4190.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-17T21:18:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:01.060355+0000", 2024-09-17T21:18:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 11.042872843, 2024-09-17T21:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00049474199999999995, 2024-09-17T21:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "reached pg", 2024-09-17T21:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4190", 2024-09-17T21:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.83:0/956669820", 2024-09-17T21:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3 2024-09-17T21:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.060355+0000", 2024-09-17T21:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.060355+0000", 2024-09-17T21:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.060357+0000", 2024-09-17T21:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4099999999999998e-06 2024-09-17T21:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.060368+0000", 2024-09-17T21:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0360000000000001e-05 2024-09-17T21:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.060399+0000", 2024-09-17T21:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1532000000000002e-05 2024-09-17T21:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.060416+0000", 2024-09-17T21:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6526999999999999e-05 2024-09-17T21:18:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.060559+0000", 2024-09-17T21:18:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000143659 2024-09-17T21:18:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.060850+0000", 2024-09-17T21:18:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029025399999999999 2024-09-17T21:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4190.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-17T21:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:02.056803+0000", 2024-09-17T21:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.046425149999999, 2024-09-17T21:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0070956250000000004, 2024-09-17T21:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4190", 2024-09-17T21:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.83:0/956669820", 2024-09-17T21:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 5 2024-09-17T21:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.056803+0000", 2024-09-17T21:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.056803+0000", 2024-09-17T21:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.056806+0000", 2024-09-17T21:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6830000000000001e-06 2024-09-17T21:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.056829+0000", 2024-09-17T21:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2235999999999999e-05 2024-09-17T21:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.056863+0000", 2024-09-17T21:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4542999999999998e-05 2024-09-17T21:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.056885+0000", 2024-09-17T21:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2113999999999999e-05 2024-09-17T21:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.057027+0000", 2024-09-17T21:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014203499999999999 2024-09-17T21:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.057039+0000", 2024-09-17T21:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1297e-05 2024-09-17T21:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.060489+0000", 2024-09-17T21:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0034507330000000001 2024-09-17T21:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.060696+0000", 2024-09-17T21:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020716899999999999 2024-09-17T21:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.060908+0000", 2024-09-17T21:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000211735 2024-09-17T21:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.062413+0000", 2024-09-17T21:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0015043820000000001 2024-09-17T21:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.063746+0000", 2024-09-17T21:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0013335689999999999 2024-09-17T21:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.063830+0000", 2024-09-17T21:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4049999999999997e-05 2024-09-17T21:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.063898+0000", 2024-09-17T21:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8078999999999997e-05 2024-09-17T21:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4190.0:5 2.2 e19/18 ondisk, result = 0)", 2024-09-17T21:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:02.063559+0000", 2024-09-17T21:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.039668402, 2024-09-17T21:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035510400000000002, 2024-09-17T21:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.063559+0000", 2024-09-17T21:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.063559+0000", 2024-09-17T21:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.063561+0000", 2024-09-17T21:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.657e-06 2024-09-17T21:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.063571+0000", 2024-09-17T21:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0163000000000001e-05 2024-09-17T21:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.063592+0000", 2024-09-17T21:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0780000000000001e-05 2024-09-17T21:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.063605+0000", 2024-09-17T21:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2713e-05 2024-09-17T21:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.063714+0000", 2024-09-17T21:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010904900000000001 2024-09-17T21:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.063729+0000", 2024-09-17T21:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5760000000000002e-05 2024-09-17T21:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.063914+0000", 2024-09-17T21:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018498200000000001 2024-09-17T21:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4190.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-17T21:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:03.060815+0000", 2024-09-17T21:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0424131289999998, 2024-09-17T21:18:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011124612000000001, 2024-09-17T21:18:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4190", 2024-09-17T21:18:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.83:0/956669820", 2024-09-17T21:18:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 7 2024-09-17T21:18:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060815+0000", 2024-09-17T21:18:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060815+0000", 2024-09-17T21:18:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060819+0000", 2024-09-17T21:18:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2640000000000002e-06 2024-09-17T21:18:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:12.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060838+0000", 2024-09-17T21:18:12.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9406e-05 2024-09-17T21:18:12.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:12.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060878+0000", 2024-09-17T21:18:12.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0120000000000002e-05 2024-09-17T21:18:12.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060896+0000", 2024-09-17T21:18:12.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7986999999999999e-05 2024-09-17T21:18:12.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.061030+0000", 2024-09-17T21:18:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013395200000000001 2024-09-17T21:18:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.061041+0000", 2024-09-17T21:18:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1053000000000001e-05 2024-09-17T21:18:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.068612+0000", 2024-09-17T21:18:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0075704049999999997 2024-09-17T21:18:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.068806+0000", 2024-09-17T21:18:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019402500000000001 2024-09-17T21:18:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:18:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.069031+0000", 2024-09-17T21:18:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022502199999999999 2024-09-17T21:18:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.070704+0000", 2024-09-17T21:18:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001672974 2024-09-17T21:18:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071795+0000", 2024-09-17T21:18:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001091648 2024-09-17T21:18:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071878+0000", 2024-09-17T21:18:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2225999999999995e-05 2024-09-17T21:18:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071939+0000", 2024-09-17T21:18:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1530000000000002e-05 2024-09-17T21:18:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4190.0:7 2.2 e20/18 ondisk, result = 0)", 2024-09-17T21:18:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:03.071626+0000", 2024-09-17T21:18:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0316020689999998, 2024-09-17T21:18:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032978199999999998, 2024-09-17T21:18:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071626+0000", 2024-09-17T21:18:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071626+0000", 2024-09-17T21:18:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071628+0000", 2024-09-17T21:18:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.824e-06 2024-09-17T21:18:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071637+0000", 2024-09-17T21:18:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6700000000000006e-06 2024-09-17T21:18:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071663+0000", 2024-09-17T21:18:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.62e-05 2024-09-17T21:18:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071689+0000", 2024-09-17T21:18:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5333e-05 2024-09-17T21:18:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071772+0000", 2024-09-17T21:18:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3011999999999997e-05 2024-09-17T21:18:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071787+0000", 2024-09-17T21:18:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.541e-05 2024-09-17T21:18:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071955+0000", 2024-09-17T21:18:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000168333 2024-09-17T21:18:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:12.148 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:12.148 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:12.149 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:18:12.196 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:12.192+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:12.280 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:12.280 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:12.280 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:12.280 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:12.287 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:12.282+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:12.296 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:18:12.301 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:18:12.388 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:12.380+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:12.457 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:12.457 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:12.457 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:12.457 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:12.457 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:12.472 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:18:12.489 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:12.482+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:12.589 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:12.586+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:12.633 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:12.633 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:18:12.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:18:12.633 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:18:12.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.634 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:20 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'10, mlcod=16'8)", 2024-09-17T21:18:12.634 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.233425+0000", 2024-09-17T21:18:12.634 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.397767035999999, 2024-09-17T21:18:12.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0026528390000000002, 2024-09-17T21:18:12.634 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:12.634 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:12.634 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:12.634 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:12.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:12.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.233425+0000", 2024-09-17T21:18:12.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:12.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.233425+0000", 2024-09-17T21:18:12.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:12.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.233427+0000", 2024-09-17T21:18:12.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2539999999999999e-06 2024-09-17T21:18:12.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:12.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.233739+0000", 2024-09-17T21:18:12.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031145400000000002 2024-09-17T21:18:12.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:12.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.233767+0000", 2024-09-17T21:18:12.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8543000000000001e-05 2024-09-17T21:18:12.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.233782+0000", 2024-09-17T21:18:12.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4705999999999999e-05 2024-09-17T21:18:12.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:12.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.233930+0000", 2024-09-17T21:18:12.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000148166 2024-09-17T21:18:12.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:12.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.233967+0000", 2024-09-17T21:18:12.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6492999999999998e-05 2024-09-17T21:18:12.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:12.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.235995+0000", 2024-09-17T21:18:12.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0020283409999999999 2024-09-17T21:18:12.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:12.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.236078+0000", 2024-09-17T21:18:12.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2881999999999997e-05 2024-09-17T21:18:12.638 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.638 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:12.638 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.638 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:21 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'11, mlcod=16'8)", 2024-09-17T21:18:12.638 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.234905+0000", 2024-09-17T21:18:12.638 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.396287434, 2024-09-17T21:18:12.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002519884, 2024-09-17T21:18:12.639 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:12.639 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:12.639 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:12.639 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:12.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:12.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.234905+0000", 2024-09-17T21:18:12.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:12.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.234905+0000", 2024-09-17T21:18:12.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:12.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.234907+0000", 2024-09-17T21:18:12.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0269999999999998e-06 2024-09-17T21:18:12.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:12.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.235051+0000", 2024-09-17T21:18:12.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014395799999999999 2024-09-17T21:18:12.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:12.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.235079+0000", 2024-09-17T21:18:12.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8487e-05 2024-09-17T21:18:12.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.235092+0000", 2024-09-17T21:18:12.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2969999999999999e-05 2024-09-17T21:18:12.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:12.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.235229+0000", 2024-09-17T21:18:12.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136582 2024-09-17T21:18:12.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:12.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.235262+0000", 2024-09-17T21:18:12.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3524e-05 2024-09-17T21:18:12.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:12.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.237311+0000", 2024-09-17T21:18:12.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0020488440000000002 2024-09-17T21:18:12.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:12.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.237425+0000", 2024-09-17T21:18:12.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113492 2024-09-17T21:18:12.643 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.643 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:12.643 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.643 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:22 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'12, mlcod=16'8)", 2024-09-17T21:18:12.644 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.236414+0000", 2024-09-17T21:18:12.644 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.394777791999999, 2024-09-17T21:18:12.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0026532790000000001, 2024-09-17T21:18:12.644 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:12.644 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:12.644 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:12.644 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:12.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:12.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.236414+0000", 2024-09-17T21:18:12.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:12.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.236414+0000", 2024-09-17T21:18:12.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:12.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.236416+0000", 2024-09-17T21:18:12.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.64e-06 2024-09-17T21:18:12.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:12.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.236559+0000", 2024-09-17T21:18:12.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014252500000000001 2024-09-17T21:18:12.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:12.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.236583+0000", 2024-09-17T21:18:12.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4473000000000001e-05 2024-09-17T21:18:12.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.236596+0000", 2024-09-17T21:18:12.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2517000000000001e-05 2024-09-17T21:18:12.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:12.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.236739+0000", 2024-09-17T21:18:12.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014371499999999999 2024-09-17T21:18:12.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:12.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.236771+0000", 2024-09-17T21:18:12.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.201e-05 2024-09-17T21:18:12.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.648 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:12.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.238997+0000", 2024-09-17T21:18:12.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022256229999999999 2024-09-17T21:18:12.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.648 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:12.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.239068+0000", 2024-09-17T21:18:12.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0776000000000001e-05 2024-09-17T21:18:12.648 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.648 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:12.649 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.649 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:40 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'22, mlcod=16'19)", 2024-09-17T21:18:12.649 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.335506+0000", 2024-09-17T21:18:12.649 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.295685809, 2024-09-17T21:18:12.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002817543, 2024-09-17T21:18:12.649 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:12.649 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:12.649 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:12.649 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:12.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:12.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.335506+0000", 2024-09-17T21:18:12.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:12.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.335506+0000", 2024-09-17T21:18:12.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:12.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.335507+0000", 2024-09-17T21:18:12.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0160000000000001e-06 2024-09-17T21:18:12.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:12.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.335520+0000", 2024-09-17T21:18:12.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3004e-05 2024-09-17T21:18:12.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:12.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.335532+0000", 2024-09-17T21:18:12.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1148999999999999e-05 2024-09-17T21:18:12.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.335566+0000", 2024-09-17T21:18:12.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4601999999999998e-05 2024-09-17T21:18:12.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:12.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.335623+0000", 2024-09-17T21:18:12.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6771999999999998e-05 2024-09-17T21:18:12.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:12.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.335640+0000", 2024-09-17T21:18:12.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6708999999999999e-05 2024-09-17T21:18:12.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:12.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.338244+0000", 2024-09-17T21:18:12.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002604749 2024-09-17T21:18:12.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.653 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:12.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.338324+0000", 2024-09-17T21:18:12.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9542000000000005e-05 2024-09-17T21:18:12.653 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.653 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:12.653 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.653 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:42 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'24, mlcod=16'19)", 2024-09-17T21:18:12.653 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.338268+0000", 2024-09-17T21:18:12.654 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.292924268, 2024-09-17T21:18:12.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002564733, 2024-09-17T21:18:12.654 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:12.654 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:12.654 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:12.654 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:12.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:12.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.338268+0000", 2024-09-17T21:18:12.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:12.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.338268+0000", 2024-09-17T21:18:12.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:12.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.338270+0000", 2024-09-17T21:18:12.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0200000000000001e-06 2024-09-17T21:18:12.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:12.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.338304+0000", 2024-09-17T21:18:12.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3776000000000003e-05 2024-09-17T21:18:12.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:12.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.338343+0000", 2024-09-17T21:18:12.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8955999999999998e-05 2024-09-17T21:18:12.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.338361+0000", 2024-09-17T21:18:12.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7816999999999999e-05 2024-09-17T21:18:12.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:12.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.338453+0000", 2024-09-17T21:18:12.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2449999999999997e-05 2024-09-17T21:18:12.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:12.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.338478+0000", 2024-09-17T21:18:12.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4521000000000001e-05 2024-09-17T21:18:12.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:12.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.340765+0000", 2024-09-17T21:18:12.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022871720000000001 2024-09-17T21:18:12.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:12.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.340833+0000", 2024-09-17T21:18:12.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8021000000000005e-05 2024-09-17T21:18:12.658 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.658 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:12.658 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.658 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:64 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'40)", 2024-09-17T21:18:12.658 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.530644+0000", 2024-09-17T21:18:12.658 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.100548691, 2024-09-17T21:18:12.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0027941960000000001, 2024-09-17T21:18:12.658 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:12.659 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:12.659 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:12.659 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:12.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:12.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.530644+0000", 2024-09-17T21:18:12.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:12.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.530644+0000", 2024-09-17T21:18:12.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:12.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.530645+0000", 2024-09-17T21:18:12.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0860000000000001e-06 2024-09-17T21:18:12.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:12.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.530651+0000", 2024-09-17T21:18:12.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4320000000000004e-06 2024-09-17T21:18:12.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:12.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.530665+0000", 2024-09-17T21:18:12.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4355000000000001e-05 2024-09-17T21:18:12.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.530672+0000", 2024-09-17T21:18:12.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4690000000000002e-06 2024-09-17T21:18:12.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:12.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.530778+0000", 2024-09-17T21:18:12.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106523 2024-09-17T21:18:12.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:12.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.530799+0000", 2024-09-17T21:18:12.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.067e-05 2024-09-17T21:18:12.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:12.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.533359+0000", 2024-09-17T21:18:12.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025601450000000002 2024-09-17T21:18:12.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:12.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.533438+0000", 2024-09-17T21:18:12.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8516000000000005e-05 2024-09-17T21:18:12.662 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.662 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:12.662 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.663 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:66 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'40)", 2024-09-17T21:18:12.663 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.531975+0000", 2024-09-17T21:18:12.663 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.099217591, 2024-09-17T21:18:12.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002901842, 2024-09-17T21:18:12.663 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:12.663 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:12.663 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:12.663 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:12.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:12.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.531975+0000", 2024-09-17T21:18:12.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:12.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.531975+0000", 2024-09-17T21:18:12.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:12.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.531977+0000", 2024-09-17T21:18:12.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9369999999999998e-06 2024-09-17T21:18:12.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:12.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.532039+0000", 2024-09-17T21:18:12.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2156999999999995e-05 2024-09-17T21:18:12.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:12.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.532070+0000", 2024-09-17T21:18:12.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1609000000000002e-05 2024-09-17T21:18:12.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.532086+0000", 2024-09-17T21:18:12.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5160000000000001e-05 2024-09-17T21:18:12.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:12.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.532216+0000", 2024-09-17T21:18:12.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000130159 2024-09-17T21:18:12.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:12.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.532251+0000", 2024-09-17T21:18:12.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4922999999999998e-05 2024-09-17T21:18:12.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:12.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.534804+0000", 2024-09-17T21:18:12.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025534210000000002 2024-09-17T21:18:12.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:12.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.534877+0000", 2024-09-17T21:18:12.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2476000000000002e-05 2024-09-17T21:18:12.667 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.667 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:12.667 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.667 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:68 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'46, mlcod=16'40)", 2024-09-17T21:18:12.667 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.533420+0000", 2024-09-17T21:18:12.667 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.097772414, 2024-09-17T21:18:12.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002853151, 2024-09-17T21:18:12.667 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:12.668 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:12.668 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:12.668 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:12.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:12.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.533420+0000", 2024-09-17T21:18:12.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:12.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.533420+0000", 2024-09-17T21:18:12.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:12.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.533421+0000", 2024-09-17T21:18:12.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0899999999999999e-06 2024-09-17T21:18:12.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:12.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.533429+0000", 2024-09-17T21:18:12.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6199999999999999e-06 2024-09-17T21:18:12.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:12.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.533439+0000", 2024-09-17T21:18:12.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0185e-05 2024-09-17T21:18:12.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.533445+0000", 2024-09-17T21:18:12.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5590000000000002e-06 2024-09-17T21:18:12.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:12.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.534295+0000", 2024-09-17T21:18:12.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084953199999999996 2024-09-17T21:18:12.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:12.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.534330+0000", 2024-09-17T21:18:12.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4829999999999997e-05 2024-09-17T21:18:12.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:12.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.536179+0000", 2024-09-17T21:18:12.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001849166 2024-09-17T21:18:12.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:12.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.536273+0000", 2024-09-17T21:18:12.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4168999999999999e-05 2024-09-17T21:18:12.671 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.671 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:12.671 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.672 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'47, mlcod=16'40)", 2024-09-17T21:18:12.672 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.534075+0000", 2024-09-17T21:18:12.672 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.09711695, 2024-09-17T21:18:12.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0033556950000000001, 2024-09-17T21:18:12.672 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:12.672 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:12.672 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:12.672 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:12.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.672 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:12.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.534075+0000", 2024-09-17T21:18:12.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:12.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.534075+0000", 2024-09-17T21:18:12.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:12.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.534076+0000", 2024-09-17T21:18:12.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2500000000000005e-07 2024-09-17T21:18:12.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:12.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.534116+0000", 2024-09-17T21:18:12.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9876000000000002e-05 2024-09-17T21:18:12.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:12.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.534125+0000", 2024-09-17T21:18:12.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8030000000000003e-06 2024-09-17T21:18:12.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.534130+0000", 2024-09-17T21:18:12.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:18:12.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:12.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.534974+0000", 2024-09-17T21:18:12.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084409600000000002 2024-09-17T21:18:12.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:12.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535017+0000", 2024-09-17T21:18:12.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2735999999999999e-05 2024-09-17T21:18:12.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:12.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.537353+0000", 2024-09-17T21:18:12.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023363419999999999 2024-09-17T21:18:12.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:12.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.537431+0000", 2024-09-17T21:18:12.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7676999999999995e-05 2024-09-17T21:18:12.676 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.676 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:12.676 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.676 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:70 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'48, mlcod=16'42)", 2024-09-17T21:18:12.676 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.535353+0000", 2024-09-17T21:18:12.676 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.095839274999999, 2024-09-17T21:18:12.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035956500000000001, 2024-09-17T21:18:12.676 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:12.676 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:12.676 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:12.677 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:12.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:12.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535353+0000", 2024-09-17T21:18:12.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:12.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535353+0000", 2024-09-17T21:18:12.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:12.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535354+0000", 2024-09-17T21:18:12.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.375e-06 2024-09-17T21:18:12.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:12.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535362+0000", 2024-09-17T21:18:12.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3139999999999998e-06 2024-09-17T21:18:12.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:12.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535378+0000", 2024-09-17T21:18:12.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6189999999999999e-05 2024-09-17T21:18:12.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535387+0000", 2024-09-17T21:18:12.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9080000000000001e-06 2024-09-17T21:18:12.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:12.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.536329+0000", 2024-09-17T21:18:12.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00094202900000000004 2024-09-17T21:18:12.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:12.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.536356+0000", 2024-09-17T21:18:12.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6837000000000001e-05 2024-09-17T21:18:12.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:12.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.538911+0000", 2024-09-17T21:18:12.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002555599 2024-09-17T21:18:12.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:12.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.538949+0000", 2024-09-17T21:18:12.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7397999999999997e-05 2024-09-17T21:18:12.681 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.681 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:12.681 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.682 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:71 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'49, mlcod=16'42)", 2024-09-17T21:18:12.682 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.535870+0000", 2024-09-17T21:18:12.682 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.09532243, 2024-09-17T21:18:12.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0040377670000000003, 2024-09-17T21:18:12.682 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:12.682 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:12.682 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:12.682 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:12.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:12.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535870+0000", 2024-09-17T21:18:12.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:12.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535870+0000", 2024-09-17T21:18:12.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:12.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535871+0000", 2024-09-17T21:18:12.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6800000000000009e-07 2024-09-17T21:18:12.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:12.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535877+0000", 2024-09-17T21:18:12.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5320000000000003e-06 2024-09-17T21:18:12.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:12.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535888+0000", 2024-09-17T21:18:12.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1129e-05 2024-09-17T21:18:12.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535896+0000", 2024-09-17T21:18:12.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1999999999999997e-06 2024-09-17T21:18:12.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:12.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.536717+0000", 2024-09-17T21:18:12.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00082162800000000003 2024-09-17T21:18:12.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:12.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.536761+0000", 2024-09-17T21:18:12.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3803000000000001e-05 2024-09-17T21:18:12.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:12.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539879+0000", 2024-09-17T21:18:12.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031181160000000002 2024-09-17T21:18:12.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:12.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539908+0000", 2024-09-17T21:18:12.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8391e-05 2024-09-17T21:18:12.686 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.686 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:12.686 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.686 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:72 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'50, mlcod=16'44)", 2024-09-17T21:18:12.686 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.536726+0000", 2024-09-17T21:18:12.687 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.094466736999999, 2024-09-17T21:18:12.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003215748, 2024-09-17T21:18:12.687 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:12.687 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:12.687 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:12.687 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:12.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:12.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.536726+0000", 2024-09-17T21:18:12.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:12.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.536726+0000", 2024-09-17T21:18:12.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:12.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.536727+0000", 2024-09-17T21:18:12.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7969999999999999e-06 2024-09-17T21:18:12.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:12.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.536842+0000", 2024-09-17T21:18:12.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114702 2024-09-17T21:18:12.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:12.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.536863+0000", 2024-09-17T21:18:12.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0530000000000002e-05 2024-09-17T21:18:12.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.536878+0000", 2024-09-17T21:18:12.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5834000000000001e-05 2024-09-17T21:18:12.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:12.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.537505+0000", 2024-09-17T21:18:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00062617499999999997 2024-09-17T21:18:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.537587+0000", 2024-09-17T21:18:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2212000000000005e-05 2024-09-17T21:18:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539914+0000", 2024-09-17T21:18:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023268130000000001 2024-09-17T21:18:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539941+0000", 2024-09-17T21:18:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7685e-05 2024-09-17T21:18:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:73 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'51, mlcod=16'44)", 2024-09-17T21:18:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.537690+0000", 2024-09-17T21:18:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.093502614, 2024-09-17T21:18:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003343015, 2024-09-17T21:18:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.537690+0000", 2024-09-17T21:18:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.537690+0000", 2024-09-17T21:18:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.537692+0000", 2024-09-17T21:18:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9829999999999999e-06 2024-09-17T21:18:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.537704+0000", 2024-09-17T21:18:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2584e-05 2024-09-17T21:18:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.537734+0000", 2024-09-17T21:18:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9886000000000002e-05 2024-09-17T21:18:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.537751+0000", 2024-09-17T21:18:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6699999999999999e-05 2024-09-17T21:18:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539015+0000", 2024-09-17T21:18:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0012645060000000001 2024-09-17T21:18:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539051+0000", 2024-09-17T21:18:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5444000000000001e-05 2024-09-17T21:18:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.541002+0000", 2024-09-17T21:18:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019515540000000001 2024-09-17T21:18:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.541033+0000", 2024-09-17T21:18:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0358e-05 2024-09-17T21:18:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:74 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'52, mlcod=16'46)", 2024-09-17T21:18:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.538545+0000", 2024-09-17T21:18:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.092647077000001, 2024-09-17T21:18:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002516491, 2024-09-17T21:18:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.538545+0000", 2024-09-17T21:18:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.538545+0000", 2024-09-17T21:18:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.538546+0000", 2024-09-17T21:18:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.28e-06 2024-09-17T21:18:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.538554+0000", 2024-09-17T21:18:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.96e-06 2024-09-17T21:18:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.538568+0000", 2024-09-17T21:18:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869999999999999e-05 2024-09-17T21:18:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.538583+0000", 2024-09-17T21:18:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5075999999999999e-05 2024-09-17T21:18:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539655+0000", 2024-09-17T21:18:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001071792 2024-09-17T21:18:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539670+0000", 2024-09-17T21:18:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4642e-05 2024-09-17T21:18:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.541035+0000", 2024-09-17T21:18:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00136498 2024-09-17T21:18:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.541062+0000", 2024-09-17T21:18:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6891000000000001e-05 2024-09-17T21:18:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:75 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'53, mlcod=16'46)", 2024-09-17T21:18:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.539197+0000", 2024-09-17T21:18:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.091994998000001, 2024-09-17T21:18:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002668368, 2024-09-17T21:18:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539197+0000", 2024-09-17T21:18:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539197+0000", 2024-09-17T21:18:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539198+0000", 2024-09-17T21:18:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2900000000000003e-07 2024-09-17T21:18:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539269+0000", 2024-09-17T21:18:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1470000000000005e-05 2024-09-17T21:18:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539286+0000", 2024-09-17T21:18:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6710000000000001e-05 2024-09-17T21:18:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539293+0000", 2024-09-17T21:18:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5189999999999998e-06 2024-09-17T21:18:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539983+0000", 2024-09-17T21:18:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00069051500000000005 2024-09-17T21:18:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.540013+0000", 2024-09-17T21:18:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0035999999999998e-05 2024-09-17T21:18:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.541839+0000", 2024-09-17T21:18:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0018256150000000001 2024-09-17T21:18:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.541866+0000", 2024-09-17T21:18:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6774000000000001e-05 2024-09-17T21:18:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:76 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'46)", 2024-09-17T21:18:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.539925+0000", 2024-09-17T21:18:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.091266827, 2024-09-17T21:18:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002506164, 2024-09-17T21:18:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539925+0000", 2024-09-17T21:18:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539925+0000", 2024-09-17T21:18:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539926+0000", 2024-09-17T21:18:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8000000000000005e-07 2024-09-17T21:18:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539933+0000", 2024-09-17T21:18:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:18:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539942+0000", 2024-09-17T21:18:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2159999999999995e-06 2024-09-17T21:18:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539950+0000", 2024-09-17T21:18:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:18:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.540803+0000", 2024-09-17T21:18:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00085252500000000003 2024-09-17T21:18:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.540816+0000", 2024-09-17T21:18:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3749e-05 2024-09-17T21:18:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.542402+0000", 2024-09-17T21:18:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015857040000000001 2024-09-17T21:18:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.542432+0000", 2024-09-17T21:18:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9403999999999998e-05 2024-09-17T21:18:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:77 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'55, mlcod=16'53)", 2024-09-17T21:18:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.543335+0000", 2024-09-17T21:18:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.087856821999999, 2024-09-17T21:18:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030410110000000001, 2024-09-17T21:18:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.543335+0000", 2024-09-17T21:18:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.543335+0000", 2024-09-17T21:18:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.543337+0000", 2024-09-17T21:18:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.093e-06 2024-09-17T21:18:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.543399+0000", 2024-09-17T21:18:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2801999999999999e-05 2024-09-17T21:18:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.543415+0000", 2024-09-17T21:18:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5362999999999998e-05 2024-09-17T21:18:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.543421+0000", 2024-09-17T21:18:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5649999999999998e-06 2024-09-17T21:18:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.543494+0000", 2024-09-17T21:18:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2719999999999995e-05 2024-09-17T21:18:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.543527+0000", 2024-09-17T21:18:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3115999999999999e-05 2024-09-17T21:18:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.546292+0000", 2024-09-17T21:18:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0027651260000000001 2024-09-17T21:18:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.546376+0000", 2024-09-17T21:18:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4226000000000003e-05 2024-09-17T21:18:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:78 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'56, mlcod=16'53)", 2024-09-17T21:18:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.543974+0000", 2024-09-17T21:18:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.087218163999999, 2024-09-17T21:18:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024999660000000002, 2024-09-17T21:18:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.543974+0000", 2024-09-17T21:18:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.543974+0000", 2024-09-17T21:18:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.543975+0000", 2024-09-17T21:18:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1399999999999997e-07 2024-09-17T21:18:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.544037+0000", 2024-09-17T21:18:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1909999999999995e-05 2024-09-17T21:18:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.544046+0000", 2024-09-17T21:18:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8630000000000003e-06 2024-09-17T21:18:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.544051+0000", 2024-09-17T21:18:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6240000000000001e-06 2024-09-17T21:18:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.544127+0000", 2024-09-17T21:18:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6083999999999999e-05 2024-09-17T21:18:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.544145+0000", 2024-09-17T21:18:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7858000000000001e-05 2024-09-17T21:18:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.546387+0000", 2024-09-17T21:18:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022417499999999998 2024-09-17T21:18:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.546474+0000", 2024-09-17T21:18:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7262999999999994e-05 2024-09-17T21:18:12.718 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.718 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:12.718 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.718 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:89 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'63, mlcod=16'60)", 2024-09-17T21:18:12.718 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.810075+0000", 2024-09-17T21:18:12.718 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.821117637, 2024-09-17T21:18:12.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0029819740000000001, 2024-09-17T21:18:12.718 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:12.718 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:12.718 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:12.718 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:12.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:12.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.810075+0000", 2024-09-17T21:18:12.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:12.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.810075+0000", 2024-09-17T21:18:12.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:12.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.810076+0000", 2024-09-17T21:18:12.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8670000000000001e-06 2024-09-17T21:18:12.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.810088+0000", 2024-09-17T21:18:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1457000000000001e-05 2024-09-17T21:18:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.810114+0000", 2024-09-17T21:18:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6316e-05 2024-09-17T21:18:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.810130+0000", 2024-09-17T21:18:12.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5739999999999998e-05 2024-09-17T21:18:12.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:12.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.810185+0000", 2024-09-17T21:18:12.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5402000000000003e-05 2024-09-17T21:18:12.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:12.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.810200+0000", 2024-09-17T21:18:12.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4646e-05 2024-09-17T21:18:12.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:12.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.812955+0000", 2024-09-17T21:18:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0027554369999999999 2024-09-17T21:18:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.813057+0000", 2024-09-17T21:18:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000101109 2024-09-17T21:18:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:90 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'64, mlcod=16'60)", 2024-09-17T21:18:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.810710+0000", 2024-09-17T21:18:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.820482522000001, 2024-09-17T21:18:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002429119, 2024-09-17T21:18:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.810710+0000", 2024-09-17T21:18:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.810710+0000", 2024-09-17T21:18:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.810711+0000", 2024-09-17T21:18:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2899999999999999e-06 2024-09-17T21:18:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.810823+0000", 2024-09-17T21:18:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000112156 2024-09-17T21:18:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.810841+0000", 2024-09-17T21:18:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7897000000000002e-05 2024-09-17T21:18:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.810851+0000", 2024-09-17T21:18:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0180000000000001e-05 2024-09-17T21:18:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.810902+0000", 2024-09-17T21:18:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1223000000000001e-05 2024-09-17T21:18:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.810917+0000", 2024-09-17T21:18:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4477e-05 2024-09-17T21:18:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.813064+0000", 2024-09-17T21:18:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021470830000000002 2024-09-17T21:18:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:12.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.813139+0000", 2024-09-17T21:18:12.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4813000000000004e-05 2024-09-17T21:18:12.730 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.730 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:12.730 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.730 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:12.730 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:12.730 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:12.732 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:18:12.732 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:12.728+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:12.832 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:12.830+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:12.894 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:12.895 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:12.895 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:12.895 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:12.908 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:18:12.956 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:12.952+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:13.028 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:13.028 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:13.028 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:13.028 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:13.028 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:13.035 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:13.028+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:13.042 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:18:13.135 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:13.134+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:13.204 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:18:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:18:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:18:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4190.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-17T21:18:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:02.060792+0000", 2024-09-17T21:18:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 11.142856306000001, 2024-09-17T21:18:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021485530000000001, 2024-09-17T21:18:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:02.060792+0000", 2024-09-17T21:18:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:02.060792+0000", 2024-09-17T21:18:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:02.060793+0000", 2024-09-17T21:18:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8299999999999997e-07 2024-09-17T21:18:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:02.060808+0000", 2024-09-17T21:18:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5563999999999999e-05 2024-09-17T21:18:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:02.060852+0000", 2024-09-17T21:18:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3739999999999998e-05 2024-09-17T21:18:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:02.060871+0000", 2024-09-17T21:18:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9018999999999999e-05 2024-09-17T21:18:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:02.061048+0000", 2024-09-17T21:18:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017683900000000001 2024-09-17T21:18:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:02.061093+0000", 2024-09-17T21:18:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5096000000000003e-05 2024-09-17T21:18:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:02.062833+0000", 2024-09-17T21:18:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001740356 2024-09-17T21:18:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:02.062941+0000", 2024-09-17T21:18:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010735599999999999 2024-09-17T21:18:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4190.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-17T21:18:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:03.068953+0000", 2024-09-17T21:18:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 10.134695191, 2024-09-17T21:18:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0020738810000000001, 2024-09-17T21:18:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:03.068953+0000", 2024-09-17T21:18:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:03.068953+0000", 2024-09-17T21:18:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:03.068954+0000", 2024-09-17T21:18:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0699999999999996e-07 2024-09-17T21:18:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:03.068965+0000", 2024-09-17T21:18:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0703000000000001e-05 2024-09-17T21:18:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:03.069017+0000", 2024-09-17T21:18:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2587000000000003e-05 2024-09-17T21:18:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:03.069041+0000", 2024-09-17T21:18:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3583000000000001e-05 2024-09-17T21:18:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:03.069215+0000", 2024-09-17T21:18:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017434800000000001 2024-09-17T21:18:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:03.069264+0000", 2024-09-17T21:18:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8874e-05 2024-09-17T21:18:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:03.070951+0000", 2024-09-17T21:18:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016864899999999999 2024-09-17T21:18:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:03.071027+0000", 2024-09-17T21:18:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6489000000000006e-05 2024-09-17T21:18:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4434.0:7 3.1a e25/23 3:5cea7035:::rbd_info:head v 25'1, mlcod=0'0)", 2024-09-17T21:18:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:12.297541+0000", 2024-09-17T21:18:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.90610761299999998, 2024-09-17T21:18:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00108709, 2024-09-17T21:18:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:12.297541+0000", 2024-09-17T21:18:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:12.297541+0000", 2024-09-17T21:18:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:12.297542+0000", 2024-09-17T21:18:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3069999999999999e-06 2024-09-17T21:18:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:12.297554+0000", 2024-09-17T21:18:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2140000000000001e-05 2024-09-17T21:18:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:12.297577+0000", 2024-09-17T21:18:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2682999999999999e-05 2024-09-17T21:18:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:12.297597+0000", 2024-09-17T21:18:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0089e-05 2024-09-17T21:18:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:12.297711+0000", 2024-09-17T21:18:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114263 2024-09-17T21:18:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:12.297733+0000", 2024-09-17T21:18:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1857000000000001e-05 2024-09-17T21:18:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:12.298556+0000", 2024-09-17T21:18:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00082286600000000001 2024-09-17T21:18:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:12.298628+0000", 2024-09-17T21:18:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1884999999999994e-05 2024-09-17T21:18:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:13.218 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:13.219 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:18:13.236 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:13.234+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:13.300 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-17T21:18:13.337 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:13.334+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:13.343 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:13.343 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:13.356 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:18:13.448 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:13.444+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:13.514 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:13.514 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:13.514 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:13.514 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:13.514 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:13.527 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:18:13.540 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:13.534+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:13.639 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:13.632+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:13.687 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:13.688 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:18:13.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:18:13.688 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:18:13.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.688 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4190.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-17T21:18:13.688 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:01.050597+0000", 2024-09-17T21:18:13.688 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.634561776, 2024-09-17T21:18:13.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042911440000000002, 2024-09-17T21:18:13.688 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:13.688 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:13.688 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050597+0000", 2024-09-17T21:18:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050597+0000", 2024-09-17T21:18:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050597+0000", 2024-09-17T21:18:13.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6599999999999996e-07 2024-09-17T21:18:13.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:13.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050604+0000", 2024-09-17T21:18:13.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0400000000000004e-06 2024-09-17T21:18:13.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:13.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050623+0000", 2024-09-17T21:18:13.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8691000000000001e-05 2024-09-17T21:18:13.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:13.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050639+0000", 2024-09-17T21:18:13.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5719999999999999e-05 2024-09-17T21:18:13.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:13.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050709+0000", 2024-09-17T21:18:13.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9898000000000001e-05 2024-09-17T21:18:13.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:13.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050738+0000", 2024-09-17T21:18:13.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9023e-05 2024-09-17T21:18:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.054823+0000", 2024-09-17T21:18:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0040854940000000003 2024-09-17T21:18:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.054888+0000", 2024-09-17T21:18:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4712e-05 2024-09-17T21:18:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:12 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.35282409399999998, 2024-09-17T21:18:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061340989999999996, 2024-09-17T21:18:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12 2024-09-17T21:18:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:13.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:13.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:13.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:13.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:13.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332336+0000", 2024-09-17T21:18:13.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5519999999999999e-06 2024-09-17T21:18:13.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:13.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332342+0000", 2024-09-17T21:18:13.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7180000000000001e-06 2024-09-17T21:18:13.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:13.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332352+0000", 2024-09-17T21:18:13.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:18:13.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:13.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332359+0000", 2024-09-17T21:18:13.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3479999999999998e-06 2024-09-17T21:18:13.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:13.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332413+0000", 2024-09-17T21:18:13.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4447999999999998e-05 2024-09-17T21:18:13.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-17T21:18:13.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332417+0000", 2024-09-17T21:18:13.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.895e-06 2024-09-17T21:18:13.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:13.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334755+0000", 2024-09-17T21:18:13.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023377189999999998 2024-09-17T21:18:13.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:13.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334903+0000", 2024-09-17T21:18:13.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014778999999999999 2024-09-17T21:18:13.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:18:13.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.335187+0000", 2024-09-17T21:18:13.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000284715 2024-09-17T21:18:13.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.337731+0000", 2024-09-17T21:18:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002543741 2024-09-17T21:18:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338382+0000", 2024-09-17T21:18:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00065029200000000001 2024-09-17T21:18:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338418+0000", 2024-09-17T21:18:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.697e-05 2024-09-17T21:18:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338469+0000", 2024-09-17T21:18:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0154999999999997e-05 2024-09-17T21:18:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.4434.0:12 3.1e e26/23 ondisk, result = 0)", 2024-09-17T21:18:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.337668+0000", 2024-09-17T21:18:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.34749094200000002, 2024-09-17T21:18:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8880999999999993e-05, 2024-09-17T21:18:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:13.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:13.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.337668+0000", 2024-09-17T21:18:13.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:13.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.337668+0000", 2024-09-17T21:18:13.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:13.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.337668+0000", 2024-09-17T21:18:13.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7299999999999995e-07 2024-09-17T21:18:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.337671+0000", 2024-09-17T21:18:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.988e-06 2024-09-17T21:18:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.337682+0000", 2024-09-17T21:18:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0572e-05 2024-09-17T21:18:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.337687+0000", 2024-09-17T21:18:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8810000000000004e-06 2024-09-17T21:18:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.337722+0000", 2024-09-17T21:18:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5438999999999997e-05 2024-09-17T21:18:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.337728+0000", 2024-09-17T21:18:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3079999999999998e-06 2024-09-17T21:18:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.337737+0000", 2024-09-17T21:18:13.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.02e-06 2024-09-17T21:18:13.703 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.703 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:13.703 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.703 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:16 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:13.703 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.349043+0000", 2024-09-17T21:18:13.703 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.336115846, 2024-09-17T21:18:13.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00065732200000000001, 2024-09-17T21:18:13.703 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:13.703 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:13.703 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 16 2024-09-17T21:18:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.349043+0000", 2024-09-17T21:18:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.349043+0000", 2024-09-17T21:18:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:13.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.349045+0000", 2024-09-17T21:18:13.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.215e-06 2024-09-17T21:18:13.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:13.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.349057+0000", 2024-09-17T21:18:13.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1994000000000001e-05 2024-09-17T21:18:13.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:13.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.349075+0000", 2024-09-17T21:18:13.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8182999999999999e-05 2024-09-17T21:18:13.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:13.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.349096+0000", 2024-09-17T21:18:13.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0355000000000001e-05 2024-09-17T21:18:13.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:13.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.349176+0000", 2024-09-17T21:18:13.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0888000000000002e-05 2024-09-17T21:18:13.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:13.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.349304+0000", 2024-09-17T21:18:13.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000127775 2024-09-17T21:18:13.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:13.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.349700+0000", 2024-09-17T21:18:13.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000395912 2024-09-17T21:18:13.707 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.707 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:13.707 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.707 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:17 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:13.707 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.350116+0000", 2024-09-17T21:18:13.707 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.33504287100000002, 2024-09-17T21:18:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00037470400000000001, 2024-09-17T21:18:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 17 2024-09-17T21:18:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350116+0000", 2024-09-17T21:18:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:13.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350116+0000", 2024-09-17T21:18:13.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:13.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350117+0000", 2024-09-17T21:18:13.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0860000000000001e-06 2024-09-17T21:18:13.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:13.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350122+0000", 2024-09-17T21:18:13.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0479999999999998e-06 2024-09-17T21:18:13.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:13.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350136+0000", 2024-09-17T21:18:13.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4581e-05 2024-09-17T21:18:13.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:13.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350144+0000", 2024-09-17T21:18:13.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3350000000000001e-06 2024-09-17T21:18:13.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:13.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350208+0000", 2024-09-17T21:18:13.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4331000000000005e-05 2024-09-17T21:18:13.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:13.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350316+0000", 2024-09-17T21:18:13.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107402 2024-09-17T21:18:13.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:13.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350490+0000", 2024-09-17T21:18:13.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000174921 2024-09-17T21:18:13.711 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:18 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call,call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.350940+0000", 2024-09-17T21:18:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.33421814900000002, 2024-09-17T21:18:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041480400000000001, 2024-09-17T21:18:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 18 2024-09-17T21:18:13.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.713 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:13.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:13.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350940+0000", 2024-09-17T21:18:13.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:13.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350940+0000", 2024-09-17T21:18:13.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350942+0000", 2024-09-17T21:18:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0440000000000001e-06 2024-09-17T21:18:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350949+0000", 2024-09-17T21:18:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:18:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350964+0000", 2024-09-17T21:18:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4136e-05 2024-09-17T21:18:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:13.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350973+0000", 2024-09-17T21:18:13.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5699999999999999e-06 2024-09-17T21:18:13.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:13.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351031+0000", 2024-09-17T21:18:13.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7395999999999997e-05 2024-09-17T21:18:13.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:13.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351138+0000", 2024-09-17T21:18:13.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010709699999999999 2024-09-17T21:18:13.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:13.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351355+0000", 2024-09-17T21:18:13.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021766799999999999 2024-09-17T21:18:13.716 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.716 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:13.716 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.716 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:19 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:13.716 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.351745+0000", 2024-09-17T21:18:13.717 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.33341373800000002, 2024-09-17T21:18:13.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024653700000000001, 2024-09-17T21:18:13.717 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:13.717 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:13.717 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:13.717 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:13.717 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:13.717 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:13.717 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 19 2024-09-17T21:18:13.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.717 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:13.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351745+0000", 2024-09-17T21:18:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351745+0000", 2024-09-17T21:18:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351746+0000", 2024-09-17T21:18:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5000000000000001e-07 2024-09-17T21:18:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351749+0000", 2024-09-17T21:18:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5580000000000001e-06 2024-09-17T21:18:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351758+0000", 2024-09-17T21:18:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0799999999999995e-06 2024-09-17T21:18:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351767+0000", 2024-09-17T21:18:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4100000000000008e-06 2024-09-17T21:18:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351811+0000", 2024-09-17T21:18:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4317000000000003e-05 2024-09-17T21:18:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351872+0000", 2024-09-17T21:18:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1099999999999994e-05 2024-09-17T21:18:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351991+0000", 2024-09-17T21:18:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119222 2024-09-17T21:18:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:20 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.352989+0000", 2024-09-17T21:18:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.33216944300000001, 2024-09-17T21:18:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00079692800000000002, 2024-09-17T21:18:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 20 2024-09-17T21:18:13.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.722 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:13.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:13.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.352989+0000", 2024-09-17T21:18:13.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:13.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.352989+0000", 2024-09-17T21:18:13.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:13.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.352990+0000", 2024-09-17T21:18:13.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.004e-06 2024-09-17T21:18:13.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:13.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.352995+0000", 2024-09-17T21:18:13.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0459999999999996e-06 2024-09-17T21:18:13.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:13.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.353009+0000", 2024-09-17T21:18:13.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4115e-05 2024-09-17T21:18:13.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:13.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.353016+0000", 2024-09-17T21:18:13.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7050000000000002e-06 2024-09-17T21:18:13.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:13.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.353076+0000", 2024-09-17T21:18:13.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0282999999999999e-05 2024-09-17T21:18:13.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.353202+0000", 2024-09-17T21:18:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000125155 2024-09-17T21:18:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.353786+0000", 2024-09-17T21:18:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00058462 2024-09-17T21:18:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:21 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.354293+0000", 2024-09-17T21:18:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.33086600999999999, 2024-09-17T21:18:13.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00059341799999999996, 2024-09-17T21:18:13.726 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:13.726 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:13.726 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:13.726 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:13.726 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:13.726 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:13.726 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 21 2024-09-17T21:18:13.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.726 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:13.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:13.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.354293+0000", 2024-09-17T21:18:13.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:13.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.354293+0000", 2024-09-17T21:18:13.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:13.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.354294+0000", 2024-09-17T21:18:13.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0330000000000001e-06 2024-09-17T21:18:13.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:13.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.354302+0000", 2024-09-17T21:18:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6249999999999996e-06 2024-09-17T21:18:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.354317+0000", 2024-09-17T21:18:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4559000000000001e-05 2024-09-17T21:18:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.354326+0000", 2024-09-17T21:18:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6180000000000001e-06 2024-09-17T21:18:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.354388+0000", 2024-09-17T21:18:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1418e-05 2024-09-17T21:18:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.354499+0000", 2024-09-17T21:18:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111548 2024-09-17T21:18:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.354886+0000", 2024-09-17T21:18:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00038661700000000002 2024-09-17T21:18:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.730 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:22 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:13.730 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.355335+0000", 2024-09-17T21:18:13.730 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.32982339999999999, 2024-09-17T21:18:13.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024841700000000002, 2024-09-17T21:18:13.730 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:13.730 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:13.730 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:13.730 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:13.730 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:13.730 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:13.730 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 22 2024-09-17T21:18:13.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.731 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:13.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:13.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.355335+0000", 2024-09-17T21:18:13.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:13.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.355335+0000", 2024-09-17T21:18:13.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.355336+0000", 2024-09-17T21:18:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2900000000000002e-07 2024-09-17T21:18:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.355340+0000", 2024-09-17T21:18:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2540000000000003e-06 2024-09-17T21:18:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.355351+0000", 2024-09-17T21:18:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0739e-05 2024-09-17T21:18:13.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:13.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.355356+0000", 2024-09-17T21:18:13.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4099999999999999e-06 2024-09-17T21:18:13.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:13.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.355404+0000", 2024-09-17T21:18:13.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7073999999999997e-05 2024-09-17T21:18:13.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:13.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.355464+0000", 2024-09-17T21:18:13.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0668999999999998e-05 2024-09-17T21:18:13.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:13.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.355584+0000", 2024-09-17T21:18:13.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119442 2024-09-17T21:18:13.734 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.734 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:13.734 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.734 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:24 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:13.735 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.357708+0000", 2024-09-17T21:18:13.735 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.327450256, 2024-09-17T21:18:13.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00047135200000000002, 2024-09-17T21:18:13.735 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:13.735 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:13.735 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:13.735 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:13.735 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:13.735 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:13.735 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 24 2024-09-17T21:18:13.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.735 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:13.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:13.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.357708+0000", 2024-09-17T21:18:13.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:13.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.357708+0000", 2024-09-17T21:18:13.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:13.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.357710+0000", 2024-09-17T21:18:13.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.874e-06 2024-09-17T21:18:13.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:13.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.357719+0000", 2024-09-17T21:18:13.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9660000000000002e-06 2024-09-17T21:18:13.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:13.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.357739+0000", 2024-09-17T21:18:13.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0103e-05 2024-09-17T21:18:13.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:13.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.357751+0000", 2024-09-17T21:18:13.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1403000000000001e-05 2024-09-17T21:18:13.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:13.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.357845+0000", 2024-09-17T21:18:13.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4285999999999998e-05 2024-09-17T21:18:13.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:13.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.357959+0000", 2024-09-17T21:18:13.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0001142 2024-09-17T21:18:13.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:13.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.358180+0000", 2024-09-17T21:18:13.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022052000000000001 2024-09-17T21:18:13.739 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.739 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:13.739 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.739 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch watch cookie 139860845289440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:13.739 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.358802+0000", 2024-09-17T21:18:13.740 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.32635680900000003, 2024-09-17T21:18:13.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023391890000000002, 2024-09-17T21:18:13.740 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:13.740 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:13.740 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:13.740 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:13.740 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:13.740 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:13.740 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25 2024-09-17T21:18:13.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.740 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:13.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:13.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.358802+0000", 2024-09-17T21:18:13.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:13.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.358802+0000", 2024-09-17T21:18:13.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:13.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.358803+0000", 2024-09-17T21:18:13.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9699999999999995e-07 2024-09-17T21:18:13.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:13.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.358806+0000", 2024-09-17T21:18:13.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6880000000000001e-06 2024-09-17T21:18:13.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:13.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.358816+0000", 2024-09-17T21:18:13.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:18:13.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.358822+0000", 2024-09-17T21:18:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1519999999999997e-06 2024-09-17T21:18:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.358893+0000", 2024-09-17T21:18:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0711999999999997e-05 2024-09-17T21:18:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.359009+0000", 2024-09-17T21:18:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011584899999999999 2024-09-17T21:18:13.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:18:13.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.359236+0000", 2024-09-17T21:18:13.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000226973 2024-09-17T21:18:13.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:13.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.359997+0000", 2024-09-17T21:18:13.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000761388 2024-09-17T21:18:13.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:13.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.360969+0000", 2024-09-17T21:18:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00097161800000000005 2024-09-17T21:18:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.361043+0000", 2024-09-17T21:18:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4781000000000001e-05 2024-09-17T21:18:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.361141+0000", 2024-09-17T21:18:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7617999999999999e-05 2024-09-17T21:18:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.4434.0:25 3.1e e26/23 ondisk, result = 0)", 2024-09-17T21:18:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.360868+0000", 2024-09-17T21:18:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.32429027100000002, 2024-09-17T21:18:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028039500000000001, 2024-09-17T21:18:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.360868+0000", 2024-09-17T21:18:13.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:13.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.360868+0000", 2024-09-17T21:18:13.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:13.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:13.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.360869+0000", 2024-09-17T21:18:13.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.048e-06 2024-09-17T21:18:13.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:13.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.360873+0000", 2024-09-17T21:18:13.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0090000000000001e-06 2024-09-17T21:18:13.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:13.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.360883+0000", 2024-09-17T21:18:13.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8810000000000008e-06 2024-09-17T21:18:13.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:13.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.360889+0000", 2024-09-17T21:18:13.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:18:13.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.360941+0000", 2024-09-17T21:18:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2262999999999998e-05 2024-09-17T21:18:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.360954+0000", 2024-09-17T21:18:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3044000000000001e-05 2024-09-17T21:18:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.361149+0000", 2024-09-17T21:18:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000194816 2024-09-17T21:18:13.752 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.752 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:13.752 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.752 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:13.752 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:13.752 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:13.754 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:18:13.754 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:13.748+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:13.853 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:18:13.854 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:13.852+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:13.884 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:18:13.884 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:13.897 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:18:13.955 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:13.950+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:13.999 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:18:14.000 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:18:14.014 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:14.015 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:14.015 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:18:14.015 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:18:14.015 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:14.016 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:18:14.016 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:18:14.029 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:18:14.032 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:18:14.033 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:18:14.050 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:18:14.051 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:18:14.056 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:14.052+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:14.068 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:18:14.069 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:18:14.091 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:18:14.152 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:18:14.188 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:14.188 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:14.196 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:14.192+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:14.201 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:18:14.258 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:14.254+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:14.320 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:14.320 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:14.334 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:18:14.360 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:14.356+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:14.461 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:14.460+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:14.497 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:14.497 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:14.512 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:18:14.562 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:14.556+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:14.635 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:14.635 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:14.649 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:18:14.663 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:14.660+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:14.764 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:14.764 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:14.764 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:14.758+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:14.778 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:18:14.865 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:14.860+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:14.947 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:14.948 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:14.963 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:18:14.966 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:14.962+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:15.067 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:15.062+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:15.079 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:15.079 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:15.093 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:18:15.167 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:15.160+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:15.225 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:15.226 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:15.240 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:18:15.285 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:15.280+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:15.308 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:18:15.308 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:18:15.309 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T21:18:15.309 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.276905 2024-09-17T21:18:15.309 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 0.2769045115130603 2024-09-17T21:18:15.325 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:18:15.369 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:15.366+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:15.411 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:15.411 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:15.428 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:18:15.470 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:15.466+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:15.472 INFO:teuthology.orchestra.run.smithi184.stdout:skipping zero length punch hole 2024-09-17T21:18:15.472 INFO:teuthology.orchestra.run.smithi184.stdout:skipping zero size read 2024-09-17T21:18:15.472 INFO:teuthology.orchestra.run.smithi184.stdout:truncating to largest ever: 0x2e38800 2024-09-17T21:18:15.475 INFO:teuthology.orchestra.run.smithi184.stdout:8 trunc from 0x0 to 0x2e38800 2024-09-17T21:18:15.494 INFO:teuthology.orchestra.run.smithi184.stdout:truncating to largest ever: 0xc3be600 2024-09-17T21:18:15.506 INFO:teuthology.orchestra.run.smithi184.stdout:10 trunc from 0x2e38800 to 0xc3be600 2024-09-17T21:18:15.523 INFO:teuthology.orchestra.run.smithi184.stdout:11 write 0x8ff4200 thru 0x8ffc6a1 (0x84a2 bytes) 2024-09-17T21:18:15.525 INFO:teuthology.orchestra.run.smithi184.stdout:13 punch from 0x8727600 to 0x872ca00, (0x5400 bytes) 2024-09-17T21:18:15.549 INFO:teuthology.orchestra.run.smithi184.stdout:15 write 0xd1e3200 thru 0xd1e7e84 (0x4c85 bytes) 2024-09-17T21:18:15.551 INFO:teuthology.orchestra.run.smithi184.stdout:16 write 0x8cd2c00 thru 0x8cd725e (0x465f bytes) 2024-09-17T21:18:15.552 INFO:teuthology.orchestra.run.smithi184.stdout:18 read 0xc084e00 thru 0xc091399 (0xc59a bytes) 2024-09-17T21:18:15.553 INFO:teuthology.orchestra.run.smithi184.stdout:19 read 0x6b8b600 thru 0x6b9ac7d (0xf67e bytes) 2024-09-17T21:18:15.553 INFO:teuthology.orchestra.run.smithi184.stdout:22 write 0x6b44e00 thru 0x6b4d891 (0x8a92 bytes) 2024-09-17T21:18:15.553 INFO:teuthology.orchestra.run.smithi184.stdout:25 write 0xb903600 thru 0xb90712d (0x3b2e bytes) 2024-09-17T21:18:15.553 INFO:teuthology.orchestra.run.smithi184.stdout:26 punch from 0x70b400 to 0x71ae00, (0xfa00 bytes) 2024-09-17T21:18:15.555 INFO:teuthology.orchestra.run.smithi184.stdout:27 read 0xa7bf000 thru 0xa7c84f3 (0x94f4 bytes) 2024-09-17T21:18:15.555 INFO:teuthology.orchestra.run.smithi184.stdout:28 read 0x1be2400 thru 0x1be4755 (0x2356 bytes) 2024-09-17T21:18:15.555 INFO:teuthology.orchestra.run.smithi184.stdout:29 read 0x1be7400 thru 0x1bf4f22 (0xdb23 bytes) 2024-09-17T21:18:15.555 INFO:teuthology.orchestra.run.smithi184.stdout:30 punch from 0x4fb1600 to 0x4fba800, (0x9200 bytes) 2024-09-17T21:18:15.555 INFO:teuthology.orchestra.run.smithi184.stdout:truncating to largest ever: 0xddcf200 2024-09-17T21:18:15.556 INFO:teuthology.orchestra.run.smithi184.stdout:31 trunc from 0xd1e8000 to 0xddcf200 2024-09-17T21:18:15.571 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:15.564+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:15.583 INFO:teuthology.orchestra.run.smithi184.stdout:32 write 0x2befa00 thru 0x2bf004e (0x64f bytes) 2024-09-17T21:18:15.585 INFO:teuthology.orchestra.run.smithi184.stdout:33 write 0x281e200 thru 0x2821f5f (0x3d60 bytes) 2024-09-17T21:18:15.602 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:15.602 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:15.606 INFO:teuthology.orchestra.run.smithi184.stdout:36 trunc from 0xddcf200 to 0x3402e00 2024-09-17T21:18:15.618 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:18:15.672 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:15.666+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:15.676 INFO:teuthology.orchestra.run.smithi184.stdout:38 write 0x9ca1800 thru 0x9ca47e2 (0x2fe3 bytes) 2024-09-17T21:18:15.678 INFO:teuthology.orchestra.run.smithi184.stdout:39 write 0x1c4e00 thru 0x1d2957 (0xdb58 bytes) 2024-09-17T21:18:15.679 INFO:teuthology.orchestra.run.smithi184.stdout:41 read 0xfa8800 thru 0xfb05c7 (0x7dc8 bytes) 2024-09-17T21:18:15.679 INFO:teuthology.orchestra.run.smithi184.stdout:43 punch from 0x9011400 to 0x901a400, (0x9000 bytes) 2024-09-17T21:18:15.679 INFO:teuthology.orchestra.run.smithi184.stdout:44 read 0x4e54a00 thru 0x4e60a39 (0xc03a bytes) 2024-09-17T21:18:15.690 INFO:teuthology.orchestra.run.smithi184.stdout:45 trunc from 0x9ca4800 to 0x1f52200 2024-09-17T21:18:15.716 INFO:teuthology.orchestra.run.smithi184.stdout:47 read 0x383c00 thru 0x393861 (0xfc62 bytes) 2024-09-17T21:18:15.718 INFO:teuthology.orchestra.run.smithi184.stdout:48 write 0x1462400 thru 0x14715c3 (0xf1c4 bytes) 2024-09-17T21:18:15.755 INFO:teuthology.orchestra.run.smithi184.stdout:49 write 0xacb1200 thru 0xacbbe7a (0xac7b bytes) 2024-09-17T21:18:15.756 INFO:teuthology.orchestra.run.smithi184.stdout:50 write 0x299c00 thru 0x2a257a (0x897b bytes) 2024-09-17T21:18:15.773 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:15.766+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:15.777 INFO:teuthology.orchestra.run.smithi184.stdout:53 trunc from 0xacbc000 to 0x15c1400 2024-09-17T21:18:15.781 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:15.782 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:15.796 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:18:15.820 INFO:teuthology.orchestra.run.smithi184.stdout:55 punch from 0xdada00 to 0xdb2200, (0x4800 bytes) 2024-09-17T21:18:15.822 INFO:teuthology.orchestra.run.smithi184.stdout:56 punch from 0x10e200 to 0x110e00, (0x2c00 bytes) 2024-09-17T21:18:15.822 INFO:teuthology.orchestra.run.smithi184.stdout:57 punch from 0xc97c00 to 0xc9a000, (0x2400 bytes) 2024-09-17T21:18:15.847 INFO:teuthology.orchestra.run.smithi184.stdout:61 write 0x4685000 thru 0x4688ae1 (0x3ae2 bytes) 2024-09-17T21:18:15.874 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:15.870+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:15.894 INFO:teuthology.orchestra.run.smithi184.stdout:62 write 0xd064000 thru 0xd0664ff (0x2500 bytes) 2024-09-17T21:18:15.923 INFO:teuthology.orchestra.run.smithi184.stdout:63 write 0xd32e400 thru 0xd33adc5 (0xc9c6 bytes) 2024-09-17T21:18:15.927 INFO:teuthology.orchestra.run.smithi184.stdout:64 read 0x9dde600 thru 0x9ddf3bc (0xdbd bytes) 2024-09-17T21:18:15.927 INFO:teuthology.orchestra.run.smithi184.stdout:66 write 0x19ba00 thru 0x1a1dce (0x63cf bytes) 2024-09-17T21:18:15.928 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:15.928 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:15.929 INFO:teuthology.orchestra.run.smithi184.stdout:67 write 0xbe10200 thru 0xbe1e120 (0xdf21 bytes) 2024-09-17T21:18:15.929 INFO:teuthology.orchestra.run.smithi184.stdout:68 punch from 0xacf1200 to 0xacffe00, (0xec00 bytes) 2024-09-17T21:18:15.930 INFO:teuthology.orchestra.run.smithi184.stdout:69 punch from 0x1027200 to 0x102a600, (0x3400 bytes) 2024-09-17T21:18:15.930 INFO:teuthology.orchestra.run.smithi184.stdout:70 read 0x4f88a00 thru 0x4f8d29c (0x489d bytes) 2024-09-17T21:18:15.931 INFO:teuthology.orchestra.run.smithi184.stdout:71 write 0xd219400 thru 0xd21cacf (0x36d0 bytes) 2024-09-17T21:18:15.937 INFO:teuthology.orchestra.run.smithi184.stdout:72 read 0x2d47e00 thru 0x2d552e8 (0xd4e9 bytes) 2024-09-17T21:18:15.938 INFO:teuthology.orchestra.run.smithi184.stdout:74 read 0x99ff200 thru 0x9a025bb (0x33bc bytes) 2024-09-17T21:18:15.938 INFO:teuthology.orchestra.run.smithi184.stdout:75 write 0xcf66200 thru 0xcf666e7 (0x4e8 bytes) 2024-09-17T21:18:15.938 INFO:teuthology.orchestra.run.smithi184.stdout:76 read 0x1727a00 thru 0x173180f (0x9e10 bytes) 2024-09-17T21:18:15.938 INFO:teuthology.orchestra.run.smithi184.stdout:77 clone 1 order 20 su 524288 sc 9 2024-09-17T21:18:15.944 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:18:15.988 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:15.984+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:16.066 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:16.066 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:16.075 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:16.070+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:16.080 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:18:16.154 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:18:16.177 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:16.172+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:16.255 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:16.255 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:16.269 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:18:16.320 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:16.316+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:16.325 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:18:16.326 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:18:16.344 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:18:16.344 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:18:16.361 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:18:16.362 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:18:16.378 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:16.372+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:16.383 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:18:16.383 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:18:16.389 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:16.389 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:16.389 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:16.389 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:16.389 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:16.403 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:18:16.405 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:18:16.405 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:18:16.426 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:18:16.426 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:18:16.479 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:16.474+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:16.524 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:16.524 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:18:16.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:18:16.524 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:18:16.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.525 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4190.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-17T21:18:16.525 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:01.050597+0000", 2024-09-17T21:18:16.525 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.471051882999999, 2024-09-17T21:18:16.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042911440000000002, 2024-09-17T21:18:16.525 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:16.525 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:16.525 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:16.525 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:16.525 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.525 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:16.525 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050597+0000", 2024-09-17T21:18:16.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:16.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050597+0000", 2024-09-17T21:18:16.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:16.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050597+0000", 2024-09-17T21:18:16.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6599999999999996e-07 2024-09-17T21:18:16.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:16.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050604+0000", 2024-09-17T21:18:16.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0400000000000004e-06 2024-09-17T21:18:16.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:16.527 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050623+0000", 2024-09-17T21:18:16.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8691000000000001e-05 2024-09-17T21:18:16.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:16.527 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050639+0000", 2024-09-17T21:18:16.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5719999999999999e-05 2024-09-17T21:18:16.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:16.528 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050709+0000", 2024-09-17T21:18:16.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9898000000000001e-05 2024-09-17T21:18:16.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.528 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:16.528 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050738+0000", 2024-09-17T21:18:16.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9023e-05 2024-09-17T21:18:16.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.528 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:16.528 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.054823+0000", 2024-09-17T21:18:16.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0040854940000000003 2024-09-17T21:18:16.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.529 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.529 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:16.529 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.054888+0000", 2024-09-17T21:18:16.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4712e-05 2024-09-17T21:18:16.529 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.529 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:16.529 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.529 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.529 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:12 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:16.529 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:16.529 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.1893142010000002, 2024-09-17T21:18:16.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061340989999999996, 2024-09-17T21:18:16.529 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:16.530 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:16.530 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:16.530 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:16.530 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:16.530 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:16.530 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12 2024-09-17T21:18:16.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.530 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:16.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.530 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:16.530 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:16.530 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:16.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:16.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:16.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332336+0000", 2024-09-17T21:18:16.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5519999999999999e-06 2024-09-17T21:18:16.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:16.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332342+0000", 2024-09-17T21:18:16.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7180000000000001e-06 2024-09-17T21:18:16.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:16.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332352+0000", 2024-09-17T21:18:16.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:18:16.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:16.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332359+0000", 2024-09-17T21:18:16.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3479999999999998e-06 2024-09-17T21:18:16.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:16.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332413+0000", 2024-09-17T21:18:16.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4447999999999998e-05 2024-09-17T21:18:16.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-17T21:18:16.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332417+0000", 2024-09-17T21:18:16.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.895e-06 2024-09-17T21:18:16.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:16.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334755+0000", 2024-09-17T21:18:16.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023377189999999998 2024-09-17T21:18:16.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:16.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334903+0000", 2024-09-17T21:18:16.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014778999999999999 2024-09-17T21:18:16.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:18:16.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.335187+0000", 2024-09-17T21:18:16.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000284715 2024-09-17T21:18:16.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:16.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.337731+0000", 2024-09-17T21:18:16.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002543741 2024-09-17T21:18:16.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.535 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:16.535 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338382+0000", 2024-09-17T21:18:16.535 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00065029200000000001 2024-09-17T21:18:16.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.535 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:16.535 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338418+0000", 2024-09-17T21:18:16.535 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.697e-05 2024-09-17T21:18:16.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.535 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:16.535 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338469+0000", 2024-09-17T21:18:16.536 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0154999999999997e-05 2024-09-17T21:18:16.536 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.536 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:16.536 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.536 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:16 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:16.536 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.349043+0000", 2024-09-17T21:18:16.536 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.1726059530000001, 2024-09-17T21:18:16.536 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00065732200000000001, 2024-09-17T21:18:16.536 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:16.536 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:16.536 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:16.536 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:16.537 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:16.537 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:16.537 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 16 2024-09-17T21:18:16.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.537 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:16.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:16.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.349043+0000", 2024-09-17T21:18:16.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:16.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.349043+0000", 2024-09-17T21:18:16.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:16.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.349045+0000", 2024-09-17T21:18:16.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.215e-06 2024-09-17T21:18:16.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:16.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.349057+0000", 2024-09-17T21:18:16.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1994000000000001e-05 2024-09-17T21:18:16.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:16.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.349075+0000", 2024-09-17T21:18:16.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8182999999999999e-05 2024-09-17T21:18:16.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:16.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.349096+0000", 2024-09-17T21:18:16.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0355000000000001e-05 2024-09-17T21:18:16.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:16.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.349176+0000", 2024-09-17T21:18:16.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0888000000000002e-05 2024-09-17T21:18:16.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:16.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.349304+0000", 2024-09-17T21:18:16.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000127775 2024-09-17T21:18:16.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:16.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.349700+0000", 2024-09-17T21:18:16.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000395912 2024-09-17T21:18:16.540 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.540 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:16.541 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.541 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:17 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:16.541 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.350116+0000", 2024-09-17T21:18:16.541 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.1715329780000001, 2024-09-17T21:18:16.541 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00037470400000000001, 2024-09-17T21:18:16.541 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:16.541 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:16.541 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:16.541 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:16.541 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:16.541 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:16.542 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 17 2024-09-17T21:18:16.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.542 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:16.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.542 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:16.542 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350116+0000", 2024-09-17T21:18:16.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.542 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:16.542 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350116+0000", 2024-09-17T21:18:16.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:16.543 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350117+0000", 2024-09-17T21:18:16.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0860000000000001e-06 2024-09-17T21:18:16.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:16.543 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350122+0000", 2024-09-17T21:18:16.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0479999999999998e-06 2024-09-17T21:18:16.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:16.543 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350136+0000", 2024-09-17T21:18:16.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4581e-05 2024-09-17T21:18:16.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.544 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:16.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350144+0000", 2024-09-17T21:18:16.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3350000000000001e-06 2024-09-17T21:18:16.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.544 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:16.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350208+0000", 2024-09-17T21:18:16.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4331000000000005e-05 2024-09-17T21:18:16.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.544 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:16.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350316+0000", 2024-09-17T21:18:16.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107402 2024-09-17T21:18:16.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:16.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350490+0000", 2024-09-17T21:18:16.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000174921 2024-09-17T21:18:16.545 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.545 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:16.545 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.545 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:18 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call,call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:16.546 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.350940+0000", 2024-09-17T21:18:16.546 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.1707082560000002, 2024-09-17T21:18:16.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041480400000000001, 2024-09-17T21:18:16.546 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:16.546 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:16.546 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:16.546 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:16.546 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:16.546 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:16.546 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 18 2024-09-17T21:18:16.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.546 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:16.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:16.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350940+0000", 2024-09-17T21:18:16.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:16.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350940+0000", 2024-09-17T21:18:16.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:16.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350942+0000", 2024-09-17T21:18:16.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0440000000000001e-06 2024-09-17T21:18:16.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:16.548 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350949+0000", 2024-09-17T21:18:16.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:18:16.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:16.548 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350964+0000", 2024-09-17T21:18:16.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4136e-05 2024-09-17T21:18:16.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:16.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.350973+0000", 2024-09-17T21:18:16.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5699999999999999e-06 2024-09-17T21:18:16.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:16.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351031+0000", 2024-09-17T21:18:16.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7395999999999997e-05 2024-09-17T21:18:16.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:16.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351138+0000", 2024-09-17T21:18:16.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010709699999999999 2024-09-17T21:18:16.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:16.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351355+0000", 2024-09-17T21:18:16.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021766799999999999 2024-09-17T21:18:16.550 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.550 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:16.550 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.550 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:19 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:16.550 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.351745+0000", 2024-09-17T21:18:16.550 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.1699038449999999, 2024-09-17T21:18:16.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024653700000000001, 2024-09-17T21:18:16.550 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:16.550 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 19 2024-09-17T21:18:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351745+0000", 2024-09-17T21:18:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351745+0000", 2024-09-17T21:18:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351746+0000", 2024-09-17T21:18:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5000000000000001e-07 2024-09-17T21:18:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351749+0000", 2024-09-17T21:18:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5580000000000001e-06 2024-09-17T21:18:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:16.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351758+0000", 2024-09-17T21:18:16.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0799999999999995e-06 2024-09-17T21:18:16.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:16.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351767+0000", 2024-09-17T21:18:16.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4100000000000008e-06 2024-09-17T21:18:16.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:16.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351811+0000", 2024-09-17T21:18:16.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4317000000000003e-05 2024-09-17T21:18:16.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:16.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351872+0000", 2024-09-17T21:18:16.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1099999999999994e-05 2024-09-17T21:18:16.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:16.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.351991+0000", 2024-09-17T21:18:16.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119222 2024-09-17T21:18:16.554 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.554 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:16.555 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.555 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:20 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:16.555 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.352989+0000", 2024-09-17T21:18:16.555 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.1686595500000001, 2024-09-17T21:18:16.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00079692800000000002, 2024-09-17T21:18:16.555 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:16.555 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:16.555 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:16.555 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:16.555 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:16.555 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:16.556 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 20 2024-09-17T21:18:16.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.556 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:16.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:16.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.352989+0000", 2024-09-17T21:18:16.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:16.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.352989+0000", 2024-09-17T21:18:16.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:16.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.352990+0000", 2024-09-17T21:18:16.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.004e-06 2024-09-17T21:18:16.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:16.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.352995+0000", 2024-09-17T21:18:16.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0459999999999996e-06 2024-09-17T21:18:16.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:16.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.353009+0000", 2024-09-17T21:18:16.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4115e-05 2024-09-17T21:18:16.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:16.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.353016+0000", 2024-09-17T21:18:16.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7050000000000002e-06 2024-09-17T21:18:16.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:16.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.353076+0000", 2024-09-17T21:18:16.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0282999999999999e-05 2024-09-17T21:18:16.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:16.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.353202+0000", 2024-09-17T21:18:16.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000125155 2024-09-17T21:18:16.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:16.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.353786+0000", 2024-09-17T21:18:16.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00058462 2024-09-17T21:18:16.560 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.560 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:16.560 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.560 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:21 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:16.560 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.354293+0000", 2024-09-17T21:18:16.560 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.1673561170000002, 2024-09-17T21:18:16.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00059341799999999996, 2024-09-17T21:18:16.560 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:16.560 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:16.560 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:16.560 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:16.561 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:16.561 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:16.561 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 21 2024-09-17T21:18:16.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.561 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:16.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:16.561 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.354293+0000", 2024-09-17T21:18:16.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:16.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.354293+0000", 2024-09-17T21:18:16.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:16.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.354294+0000", 2024-09-17T21:18:16.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0330000000000001e-06 2024-09-17T21:18:16.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:16.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.354302+0000", 2024-09-17T21:18:16.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6249999999999996e-06 2024-09-17T21:18:16.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:16.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.354317+0000", 2024-09-17T21:18:16.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4559000000000001e-05 2024-09-17T21:18:16.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:16.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.354326+0000", 2024-09-17T21:18:16.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6180000000000001e-06 2024-09-17T21:18:16.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:16.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.354388+0000", 2024-09-17T21:18:16.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1418e-05 2024-09-17T21:18:16.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:16.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.354499+0000", 2024-09-17T21:18:16.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111548 2024-09-17T21:18:16.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:16.565 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.354886+0000", 2024-09-17T21:18:16.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00038661700000000002 2024-09-17T21:18:16.565 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.565 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:16.565 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.565 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:22 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:16.565 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.355335+0000", 2024-09-17T21:18:16.565 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.1663135069999999, 2024-09-17T21:18:16.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024841700000000002, 2024-09-17T21:18:16.565 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:16.565 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:16.566 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:16.566 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:16.566 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:16.566 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:16.566 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 22 2024-09-17T21:18:16.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.566 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:16.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:16.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.355335+0000", 2024-09-17T21:18:16.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:16.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.355335+0000", 2024-09-17T21:18:16.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.567 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:16.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.355336+0000", 2024-09-17T21:18:16.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2900000000000002e-07 2024-09-17T21:18:16.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.568 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:16.568 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.355340+0000", 2024-09-17T21:18:16.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2540000000000003e-06 2024-09-17T21:18:16.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.568 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.568 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:16.568 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.355351+0000", 2024-09-17T21:18:16.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0739e-05 2024-09-17T21:18:16.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.568 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.569 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:16.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.355356+0000", 2024-09-17T21:18:16.569 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4099999999999999e-06 2024-09-17T21:18:16.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.569 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.569 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:16.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.355404+0000", 2024-09-17T21:18:16.569 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7073999999999997e-05 2024-09-17T21:18:16.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.569 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.569 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:16.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.355464+0000", 2024-09-17T21:18:16.569 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0668999999999998e-05 2024-09-17T21:18:16.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.570 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:16.570 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.355584+0000", 2024-09-17T21:18:16.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119442 2024-09-17T21:18:16.570 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.570 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:16.570 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.570 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:24 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:16.571 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.357708+0000", 2024-09-17T21:18:16.571 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.163940363, 2024-09-17T21:18:16.571 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00047135200000000002, 2024-09-17T21:18:16.571 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:16.571 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:16.571 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:16.571 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:16.571 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:16.571 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:16.571 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 24 2024-09-17T21:18:16.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.572 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:16.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.572 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:16.572 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.357708+0000", 2024-09-17T21:18:16.572 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.572 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:16.572 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.357708+0000", 2024-09-17T21:18:16.572 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.573 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.573 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:16.573 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.357710+0000", 2024-09-17T21:18:16.573 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.874e-06 2024-09-17T21:18:16.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.573 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.573 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:16.573 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.357719+0000", 2024-09-17T21:18:16.573 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9660000000000002e-06 2024-09-17T21:18:16.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.573 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.573 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:16.574 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.357739+0000", 2024-09-17T21:18:16.574 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0103e-05 2024-09-17T21:18:16.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.574 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:16.574 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.357751+0000", 2024-09-17T21:18:16.574 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1403000000000001e-05 2024-09-17T21:18:16.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.574 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:16.574 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.357845+0000", 2024-09-17T21:18:16.574 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4285999999999998e-05 2024-09-17T21:18:16.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.575 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.575 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:16.575 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.357959+0000", 2024-09-17T21:18:16.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0001142 2024-09-17T21:18:16.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.575 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.575 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:16.575 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.358180+0000", 2024-09-17T21:18:16.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022052000000000001 2024-09-17T21:18:16.575 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.575 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:16.575 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.576 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.576 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch watch cookie 139860845289440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:16.576 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.358802+0000", 2024-09-17T21:18:16.576 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.1628469159999999, 2024-09-17T21:18:16.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023391890000000002, 2024-09-17T21:18:16.576 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:16.576 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:16.576 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:16.576 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:16.576 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:16.576 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:16.576 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25 2024-09-17T21:18:16.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.577 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:16.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.577 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:16.577 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.358802+0000", 2024-09-17T21:18:16.577 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.577 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:16.577 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.358802+0000", 2024-09-17T21:18:16.577 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.577 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:16.578 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.358803+0000", 2024-09-17T21:18:16.578 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9699999999999995e-07 2024-09-17T21:18:16.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.578 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.578 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:16.578 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.358806+0000", 2024-09-17T21:18:16.578 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6880000000000001e-06 2024-09-17T21:18:16.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.578 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.578 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:16.578 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.358816+0000", 2024-09-17T21:18:16.578 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:18:16.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.579 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:16.579 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.358822+0000", 2024-09-17T21:18:16.579 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1519999999999997e-06 2024-09-17T21:18:16.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.579 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:16.579 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.358893+0000", 2024-09-17T21:18:16.579 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0711999999999997e-05 2024-09-17T21:18:16.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.579 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:16.579 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.359009+0000", 2024-09-17T21:18:16.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011584899999999999 2024-09-17T21:18:16.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.580 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.580 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:18:16.580 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.359236+0000", 2024-09-17T21:18:16.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000226973 2024-09-17T21:18:16.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.580 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.580 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:16.580 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.359997+0000", 2024-09-17T21:18:16.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000761388 2024-09-17T21:18:16.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.580 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.580 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:16.581 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.360969+0000", 2024-09-17T21:18:16.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00097161800000000005 2024-09-17T21:18:16.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.581 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:16.581 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.361043+0000", 2024-09-17T21:18:16.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4781000000000001e-05 2024-09-17T21:18:16.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.581 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:16.581 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.361141+0000", 2024-09-17T21:18:16.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7617999999999999e-05 2024-09-17T21:18:16.581 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.582 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:16.582 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.582 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.582 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.4434.0:25 3.1e e26/23 ondisk, result = 0)", 2024-09-17T21:18:16.582 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.360868+0000", 2024-09-17T21:18:16.582 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.1607803780000001, 2024-09-17T21:18:16.582 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028039500000000001, 2024-09-17T21:18:16.582 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:16.582 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:16.582 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:16.582 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:16.582 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.582 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:16.583 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.360868+0000", 2024-09-17T21:18:16.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.583 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:16.583 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.360868+0000", 2024-09-17T21:18:16.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.583 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:16.583 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.360869+0000", 2024-09-17T21:18:16.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.048e-06 2024-09-17T21:18:16.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.584 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.584 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:16.584 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.360873+0000", 2024-09-17T21:18:16.584 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0090000000000001e-06 2024-09-17T21:18:16.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.584 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.584 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:16.584 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.360883+0000", 2024-09-17T21:18:16.584 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8810000000000008e-06 2024-09-17T21:18:16.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.584 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.584 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:16.584 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.360889+0000", 2024-09-17T21:18:16.585 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:18:16.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.585 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.585 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:16.585 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.360941+0000", 2024-09-17T21:18:16.585 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2262999999999998e-05 2024-09-17T21:18:16.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.585 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.585 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:16.585 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.360954+0000", 2024-09-17T21:18:16.585 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3044000000000001e-05 2024-09-17T21:18:16.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.585 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.586 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:16.586 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.361149+0000", 2024-09-17T21:18:16.586 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000194816 2024-09-17T21:18:16.586 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.586 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:16.586 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.586 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.586 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4488.0:3 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:16.586 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.546860+0000", 2024-09-17T21:18:16.586 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 2.9747884290000002, 2024-09-17T21:18:16.586 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034755700000000002, 2024-09-17T21:18:16.586 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:16.586 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:16.587 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:16.587 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:16.587 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4488", 2024-09-17T21:18:16.587 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:16.587 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3 2024-09-17T21:18:16.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.587 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:16.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.587 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:16.587 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.546860+0000", 2024-09-17T21:18:16.587 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.588 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:16.588 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.546860+0000", 2024-09-17T21:18:16.588 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.588 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:16.588 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.546863+0000", 2024-09-17T21:18:16.588 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3159999999999999e-06 2024-09-17T21:18:16.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.588 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:16.588 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.546871+0000", 2024-09-17T21:18:16.588 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6200000000000005e-06 2024-09-17T21:18:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.546889+0000", 2024-09-17T21:18:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8216999999999999e-05 2024-09-17T21:18:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.546922+0000", 2024-09-17T21:18:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2305999999999999e-05 2024-09-17T21:18:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.546991+0000", 2024-09-17T21:18:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9628000000000005e-05 2024-09-17T21:18:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.547047+0000", 2024-09-17T21:18:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5572000000000003e-05 2024-09-17T21:18:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.547208+0000", 2024-09-17T21:18:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000160898 2024-09-17T21:18:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4488.0:7 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.549456+0000", 2024-09-17T21:18:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 2.9721928360000001, 2024-09-17T21:18:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00096293599999999996, 2024-09-17T21:18:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4488", 2024-09-17T21:18:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 7 2024-09-17T21:18:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.549456+0000", 2024-09-17T21:18:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.549456+0000", 2024-09-17T21:18:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.593 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:16.593 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.549457+0000", 2024-09-17T21:18:16.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4640000000000001e-06 2024-09-17T21:18:16.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.593 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:16.593 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.549464+0000", 2024-09-17T21:18:16.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9829999999999999e-06 2024-09-17T21:18:16.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.593 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:16.593 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.549486+0000", 2024-09-17T21:18:16.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1878999999999998e-05 2024-09-17T21:18:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.549498+0000", 2024-09-17T21:18:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1953999999999999e-05 2024-09-17T21:18:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.549626+0000", 2024-09-17T21:18:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000127305 2024-09-17T21:18:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.549776+0000", 2024-09-17T21:18:16.595 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015096599999999999 2024-09-17T21:18:16.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.595 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:16.595 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.550419+0000", 2024-09-17T21:18:16.595 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00064238499999999998 2024-09-17T21:18:16.595 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.595 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:16.595 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.596 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4488.0:8 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:16.596 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.550674+0000", 2024-09-17T21:18:16.596 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 2.9709748820000002, 2024-09-17T21:18:16.596 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072017100000000003, 2024-09-17T21:18:16.596 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:16.596 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:16.596 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:16.596 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:16.596 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4488", 2024-09-17T21:18:16.596 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:16.596 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 8 2024-09-17T21:18:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.550674+0000", 2024-09-17T21:18:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.550674+0000", 2024-09-17T21:18:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.550675+0000", 2024-09-17T21:18:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.61e-06 2024-09-17T21:18:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.550682+0000", 2024-09-17T21:18:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5899999999999996e-06 2024-09-17T21:18:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.550698+0000", 2024-09-17T21:18:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6073e-05 2024-09-17T21:18:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.550709+0000", 2024-09-17T21:18:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1073000000000001e-05 2024-09-17T21:18:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.550788+0000", 2024-09-17T21:18:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8802999999999998e-05 2024-09-17T21:18:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.550928+0000", 2024-09-17T21:18:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000139565 2024-09-17T21:18:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.551394+0000", 2024-09-17T21:18:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046645699999999998 2024-09-17T21:18:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4488.0:9 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.551617+0000", 2024-09-17T21:18:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 2.9700317360000001, 2024-09-17T21:18:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00055064199999999995, 2024-09-17T21:18:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4488", 2024-09-17T21:18:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 9 2024-09-17T21:18:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:16.605 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.551617+0000", 2024-09-17T21:18:16.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.551617+0000", 2024-09-17T21:18:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.551619+0000", 2024-09-17T21:18:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.607e-06 2024-09-17T21:18:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.551625+0000", 2024-09-17T21:18:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3860000000000004e-06 2024-09-17T21:18:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.551640+0000", 2024-09-17T21:18:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5447e-05 2024-09-17T21:18:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.551651+0000", 2024-09-17T21:18:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0946e-05 2024-09-17T21:18:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.551747+0000", 2024-09-17T21:18:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5942999999999999e-05 2024-09-17T21:18:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.551885+0000", 2024-09-17T21:18:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013750200000000001 2024-09-17T21:18:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.552168+0000", 2024-09-17T21:18:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000282811 2024-09-17T21:18:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4488.0:11 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.554513+0000", 2024-09-17T21:18:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 2.9671355859999999, 2024-09-17T21:18:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000259103, 2024-09-17T21:18:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:16.610 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:16.610 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:16.610 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4488", 2024-09-17T21:18:16.610 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:16.610 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 11 2024-09-17T21:18:16.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.610 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:16.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:16.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.554513+0000", 2024-09-17T21:18:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.554513+0000", 2024-09-17T21:18:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.554515+0000", 2024-09-17T21:18:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:18:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.554527+0000", 2024-09-17T21:18:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1837e-05 2024-09-17T21:18:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.554555+0000", 2024-09-17T21:18:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7852999999999999e-05 2024-09-17T21:18:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.554571+0000", 2024-09-17T21:18:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6016999999999999e-05 2024-09-17T21:18:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.554653+0000", 2024-09-17T21:18:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2279000000000003e-05 2024-09-17T21:18:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.554699+0000", 2024-09-17T21:18:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6059000000000001e-05 2024-09-17T21:18:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.554772+0000", 2024-09-17T21:18:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2947999999999993e-05 2024-09-17T21:18:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4488.0:12 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch watch cookie 140256452023056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.555149+0000", 2024-09-17T21:18:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 2.9664998000000002, 2024-09-17T21:18:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003529936, 2024-09-17T21:18:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4488", 2024-09-17T21:18:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12 2024-09-17T21:18:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.555149+0000", 2024-09-17T21:18:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.555149+0000", 2024-09-17T21:18:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.555151+0000", 2024-09-17T21:18:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5960000000000001e-06 2024-09-17T21:18:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.555157+0000", 2024-09-17T21:18:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4400000000000002e-06 2024-09-17T21:18:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.555175+0000", 2024-09-17T21:18:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8386999999999999e-05 2024-09-17T21:18:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.555188+0000", 2024-09-17T21:18:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2186e-05 2024-09-17T21:18:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.555291+0000", 2024-09-17T21:18:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010387600000000001 2024-09-17T21:18:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.555424+0000", 2024-09-17T21:18:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000133029 2024-09-17T21:18:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:18:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.555697+0000", 2024-09-17T21:18:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00027259799999999998 2024-09-17T21:18:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.556742+0000", 2024-09-17T21:18:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001045282 2024-09-17T21:18:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.558309+0000", 2024-09-17T21:18:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015671210000000001 2024-09-17T21:18:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.558470+0000", 2024-09-17T21:18:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016069599999999999 2024-09-17T21:18:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.558679+0000", 2024-09-17T21:18:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020872499999999999 2024-09-17T21:18:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.4488.0:12 3.1e e26/23 ondisk, result = 0)", 2024-09-17T21:18:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.558046+0000", 2024-09-17T21:18:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 2.963602699, 2024-09-17T21:18:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00065047499999999997, 2024-09-17T21:18:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.558046+0000", 2024-09-17T21:18:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.558046+0000", 2024-09-17T21:18:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.558048+0000", 2024-09-17T21:18:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.08e-06 2024-09-17T21:18:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.558059+0000", 2024-09-17T21:18:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:18:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.558086+0000", 2024-09-17T21:18:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7112999999999999e-05 2024-09-17T21:18:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.558121+0000", 2024-09-17T21:18:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5540000000000002e-05 2024-09-17T21:18:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.558266+0000", 2024-09-17T21:18:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000144382 2024-09-17T21:18:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.558292+0000", 2024-09-17T21:18:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5843e-05 2024-09-17T21:18:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.558697+0000", 2024-09-17T21:18:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00040499399999999999 2024-09-17T21:18:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4488.0:14 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.593988+0000", 2024-09-17T21:18:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 2.9276609169999999, 2024-09-17T21:18:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051729300000000005, 2024-09-17T21:18:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4488", 2024-09-17T21:18:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 14 2024-09-17T21:18:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.593988+0000", 2024-09-17T21:18:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.593988+0000", 2024-09-17T21:18:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.593990+0000", 2024-09-17T21:18:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3939999999999999e-06 2024-09-17T21:18:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.594000+0000", 2024-09-17T21:18:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0189e-05 2024-09-17T21:18:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.594022+0000", 2024-09-17T21:18:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1858e-05 2024-09-17T21:18:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.594032+0000", 2024-09-17T21:18:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0074e-05 2024-09-17T21:18:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.594123+0000", 2024-09-17T21:18:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0438000000000004e-05 2024-09-17T21:18:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.594257+0000", 2024-09-17T21:18:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013417 2024-09-17T21:18:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.594505+0000", 2024-09-17T21:18:16.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024816999999999998 2024-09-17T21:18:16.629 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.629 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:16.629 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.629 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:16.629 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:16.629 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:16.631 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:18:16.631 INFO:teuthology.orchestra.run.smithi083.stderr:set osd.2 primary-affinity to 0.276905 (8181472) 2024-09-17T21:18:16.631 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-17T21:18:16.632 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:16.624+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:16.731 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:16.724+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:16.765 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3.19bfc4de (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:18:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:18:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.17048084599999999, 2024-09-17T21:18:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17052429299999999, 2024-09-17T21:18:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "waiting for peered", 2024-09-17T21:18:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:18:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:18:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:18:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:18:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:18:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:18:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:18:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:18:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:18:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:18:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:18:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:18:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:18:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:18:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:18:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:18:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:18:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:18:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:18:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:18:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3.19bfc4de (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:18:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:18:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.170383534, 2024-09-17T21:18:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17054630000000001, 2024-09-17T21:18:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "waiting for peered", 2024-09-17T21:18:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:18:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:18:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:18:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:18:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:18:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:18:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:18:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:18:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:18:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:18:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:18:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:18:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:18:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:18:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:18:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:18:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:18:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:18:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:18:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3.19bfc4de (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:18:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:18:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.168296733, 2024-09-17T21:18:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.16855990500000001, 2024-09-17T21:18:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "waiting for peered", 2024-09-17T21:18:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:18:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:18:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:18:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:18:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:18:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:18:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:18:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:18:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:18:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:18:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:18:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:18:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:18:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:18:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:18:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:18:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:18:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:18:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:18:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:18:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:18:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-09-17T21:18:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 3 2024-09-17T21:18:16.797 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:16.799 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:18:16.832 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:16.828+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:16.916 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:16.917 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:16.917 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:18:16.917 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:18:16.917 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:16.931 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:18:16.933 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:16.928+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:17.034 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:17.028+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:17.052 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:17.053 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:18:17.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:18:17.053 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:17.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.053 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:17.054 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:18:17.054 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 20.034173915, 2024-09-17T21:18:17.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:18:17.054 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.054 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.054 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.055 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.055 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:17.055 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:17.055 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:18:17.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.055 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:18:17.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:18:17.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:18:17.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:18:17.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:18:17.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:18:17.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:18:17.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:18:17.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:18:17.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:18:17.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:18:17.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:18:17.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:17.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:18:17.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:18:17.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:18:17.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:18:17.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:18:17.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:18:17.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:17.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:18:17.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:18:17.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:17.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:18:17.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:18:17.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:17.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:18:17.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:18:17.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:18:17.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:18:17.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:18:17.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:18:17.068 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.068 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.068 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.069 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:17.069 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.233810+0000", 2024-09-17T21:18:17.069 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.814129479999998, 2024-09-17T21:18:17.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0096383549999999995, 2024-09-17T21:18:17.070 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.070 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.070 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.070 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.070 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:17.070 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:17.070 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 24 2024-09-17T21:18:17.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.071 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.233810+0000", 2024-09-17T21:18:17.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.233810+0000", 2024-09-17T21:18:17.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.233811+0000", 2024-09-17T21:18:17.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3230000000000001e-06 2024-09-17T21:18:17.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.233985+0000", 2024-09-17T21:18:17.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000173609 2024-09-17T21:18:17.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234013+0000", 2024-09-17T21:18:17.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8396000000000001e-05 2024-09-17T21:18:17.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234035+0000", 2024-09-17T21:18:17.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1756999999999999e-05 2024-09-17T21:18:17.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.239983+0000", 2024-09-17T21:18:17.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059477030000000004 2024-09-17T21:18:17.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.240111+0000", 2024-09-17T21:18:17.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012860899999999999 2024-09-17T21:18:17.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:17.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.240317+0000", 2024-09-17T21:18:17.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020541900000000001 2024-09-17T21:18:17.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:17.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.243187+0000", 2024-09-17T21:18:17.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002870531 2024-09-17T21:18:17.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:17.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.243295+0000", 2024-09-17T21:18:17.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000107606 2024-09-17T21:18:17.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.243381+0000", 2024-09-17T21:18:17.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5995999999999995e-05 2024-09-17T21:18:17.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.243448+0000", 2024-09-17T21:18:17.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7405999999999998e-05 2024-09-17T21:18:17.080 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.080 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.080 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.080 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:17.081 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.234074+0000", 2024-09-17T21:18:17.081 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.813864879, 2024-09-17T21:18:17.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011808219, 2024-09-17T21:18:17.081 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.081 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.081 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.081 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.082 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:17.082 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:17.082 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 25 2024-09-17T21:18:17.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.082 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234074+0000", 2024-09-17T21:18:17.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234074+0000", 2024-09-17T21:18:17.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234076+0000", 2024-09-17T21:18:17.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1799999999999999e-06 2024-09-17T21:18:17.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234094+0000", 2024-09-17T21:18:17.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8240000000000002e-05 2024-09-17T21:18:17.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234116+0000", 2024-09-17T21:18:17.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2296000000000001e-05 2024-09-17T21:18:17.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234140+0000", 2024-09-17T21:18:17.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4204000000000001e-05 2024-09-17T21:18:17.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.241931+0000", 2024-09-17T21:18:17.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0077910419999999998 2024-09-17T21:18:17.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.242167+0000", 2024-09-17T21:18:17.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000236025 2024-09-17T21:18:17.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:17.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.242544+0000", 2024-09-17T21:18:17.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037610699999999998 2024-09-17T21:18:17.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:17.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.243799+0000", 2024-09-17T21:18:17.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001255611 2024-09-17T21:18:17.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:17.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.245610+0000", 2024-09-17T21:18:17.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001811099 2024-09-17T21:18:17.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.245758+0000", 2024-09-17T21:18:17.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148212 2024-09-17T21:18:17.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.245883+0000", 2024-09-17T21:18:17.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124203 2024-09-17T21:18:17.093 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.093 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.094 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.094 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:17.094 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.334704+0000", 2024-09-17T21:18:17.094 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.713235253000001, 2024-09-17T21:18:17.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0087013230000000004, 2024-09-17T21:18:17.095 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.095 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.095 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.095 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.096 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:17.096 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:17.096 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 42 2024-09-17T21:18:17.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.096 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.334704+0000", 2024-09-17T21:18:17.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.334704+0000", 2024-09-17T21:18:17.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.334706+0000", 2024-09-17T21:18:17.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:18:17.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.334716+0000", 2024-09-17T21:18:17.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0720000000000001e-05 2024-09-17T21:18:17.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.334745+0000", 2024-09-17T21:18:17.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9013000000000002e-05 2024-09-17T21:18:17.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.334760+0000", 2024-09-17T21:18:17.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4797000000000001e-05 2024-09-17T21:18:17.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.338163+0000", 2024-09-17T21:18:17.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0034028700000000001 2024-09-17T21:18:17.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.338293+0000", 2024-09-17T21:18:17.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000129596 2024-09-17T21:18:17.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:17.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.338583+0000", 2024-09-17T21:18:17.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029060700000000002 2024-09-17T21:18:17.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:17.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.342292+0000", 2024-09-17T21:18:17.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0037091720000000002 2024-09-17T21:18:17.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:17.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.343240+0000", 2024-09-17T21:18:17.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00094782199999999997 2024-09-17T21:18:17.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.343341+0000", 2024-09-17T21:18:17.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010048300000000001 2024-09-17T21:18:17.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.343405+0000", 2024-09-17T21:18:17.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4642999999999998e-05 2024-09-17T21:18:17.107 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.107 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.107 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.107 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:17.107 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.335442+0000", 2024-09-17T21:18:17.108 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.712497785, 2024-09-17T21:18:17.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0096596089999999996, 2024-09-17T21:18:17.108 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.108 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.108 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.108 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.108 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:17.109 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:17.109 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 44 2024-09-17T21:18:17.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.109 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335442+0000", 2024-09-17T21:18:17.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335442+0000", 2024-09-17T21:18:17.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335443+0000", 2024-09-17T21:18:17.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.747e-06 2024-09-17T21:18:17.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335562+0000", 2024-09-17T21:18:17.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000118656 2024-09-17T21:18:17.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335592+0000", 2024-09-17T21:18:17.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9743e-05 2024-09-17T21:18:17.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335609+0000", 2024-09-17T21:18:17.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7558999999999999e-05 2024-09-17T21:18:17.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.340968+0000", 2024-09-17T21:18:17.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0053589220000000003 2024-09-17T21:18:17.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.341101+0000", 2024-09-17T21:18:17.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013255600000000001 2024-09-17T21:18:17.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:17.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.341308+0000", 2024-09-17T21:18:17.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020729099999999999 2024-09-17T21:18:17.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:17.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.344815+0000", 2024-09-17T21:18:17.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003506594 2024-09-17T21:18:17.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:17.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.344960+0000", 2024-09-17T21:18:17.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145105 2024-09-17T21:18:17.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.345037+0000", 2024-09-17T21:18:17.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6987e-05 2024-09-17T21:18:17.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.345101+0000", 2024-09-17T21:18:17.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4449000000000006e-05 2024-09-17T21:18:17.119 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.119 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.119 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.120 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:17.120 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.335647+0000", 2024-09-17T21:18:17.120 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.712292247000001, 2024-09-17T21:18:17.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012049351, 2024-09-17T21:18:17.120 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.120 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.120 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.120 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.120 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:17.120 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:17.120 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 45 2024-09-17T21:18:17.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.121 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335647+0000", 2024-09-17T21:18:17.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335647+0000", 2024-09-17T21:18:17.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335649+0000", 2024-09-17T21:18:17.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.624e-06 2024-09-17T21:18:17.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335680+0000", 2024-09-17T21:18:17.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0913e-05 2024-09-17T21:18:17.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335701+0000", 2024-09-17T21:18:17.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1307000000000001e-05 2024-09-17T21:18:17.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335718+0000", 2024-09-17T21:18:17.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6586e-05 2024-09-17T21:18:17.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.342345+0000", 2024-09-17T21:18:17.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006627712 2024-09-17T21:18:17.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.342463+0000", 2024-09-17T21:18:17.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000117439 2024-09-17T21:18:17.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:17.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.342647+0000", 2024-09-17T21:18:17.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000184692 2024-09-17T21:18:17.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:17.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.344938+0000", 2024-09-17T21:18:17.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022909190000000002 2024-09-17T21:18:17.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:17.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.347555+0000", 2024-09-17T21:18:17.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026162429999999999 2024-09-17T21:18:17.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.347635+0000", 2024-09-17T21:18:17.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0253000000000006e-05 2024-09-17T21:18:17.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.347696+0000", 2024-09-17T21:18:17.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1662999999999996e-05 2024-09-17T21:18:17.125 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.125 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.125 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.126 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:17.126 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.335761+0000", 2024-09-17T21:18:17.126 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.712178808000001, 2024-09-17T21:18:17.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013102346000000001, 2024-09-17T21:18:17.126 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.126 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.126 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.126 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.126 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:17.126 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:17.126 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 46 2024-09-17T21:18:17.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.127 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335761+0000", 2024-09-17T21:18:17.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335761+0000", 2024-09-17T21:18:17.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335762+0000", 2024-09-17T21:18:17.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:18:17.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335787+0000", 2024-09-17T21:18:17.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5360000000000001e-05 2024-09-17T21:18:17.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335808+0000", 2024-09-17T21:18:17.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0333e-05 2024-09-17T21:18:17.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335868+0000", 2024-09-17T21:18:17.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0463e-05 2024-09-17T21:18:17.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.343865+0000", 2024-09-17T21:18:17.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0079966350000000002 2024-09-17T21:18:17.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.343989+0000", 2024-09-17T21:18:17.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124449 2024-09-17T21:18:17.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:17.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.344190+0000", 2024-09-17T21:18:17.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020082499999999999 2024-09-17T21:18:17.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:17.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.347528+0000", 2024-09-17T21:18:17.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003337998 2024-09-17T21:18:17.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:17.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.348737+0000", 2024-09-17T21:18:17.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0012084470000000001 2024-09-17T21:18:17.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.348805+0000", 2024-09-17T21:18:17.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8306000000000006e-05 2024-09-17T21:18:17.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.348863+0000", 2024-09-17T21:18:17.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8006e-05 2024-09-17T21:18:17.131 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.132 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.132 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.132 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:17.132 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.335903+0000", 2024-09-17T21:18:17.132 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.712036195, 2024-09-17T21:18:17.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013146595000000001, 2024-09-17T21:18:17.132 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.132 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.132 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.133 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.133 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:17.133 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:17.133 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 47 2024-09-17T21:18:17.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.133 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335903+0000", 2024-09-17T21:18:17.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335903+0000", 2024-09-17T21:18:17.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335905+0000", 2024-09-17T21:18:17.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.556e-06 2024-09-17T21:18:17.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336023+0000", 2024-09-17T21:18:17.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000118136 2024-09-17T21:18:17.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336050+0000", 2024-09-17T21:18:17.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7472999999999999e-05 2024-09-17T21:18:17.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336066+0000", 2024-09-17T21:18:17.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5934e-05 2024-09-17T21:18:17.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.345186+0000", 2024-09-17T21:18:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0091198389999999994 2024-09-17T21:18:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.345337+0000", 2024-09-17T21:18:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015087600000000001 2024-09-17T21:18:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.345549+0000", 2024-09-17T21:18:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021209500000000001 2024-09-17T21:18:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:17.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.347760+0000", 2024-09-17T21:18:17.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022113900000000001 2024-09-17T21:18:17.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:17.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.348877+0000", 2024-09-17T21:18:17.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011163340000000001 2024-09-17T21:18:17.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.348951+0000", 2024-09-17T21:18:17.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4666000000000006e-05 2024-09-17T21:18:17.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.349050+0000", 2024-09-17T21:18:17.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8295999999999995e-05 2024-09-17T21:18:17.138 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.138 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.138 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.138 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:17.138 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.336089+0000", 2024-09-17T21:18:17.138 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.711850463000001, 2024-09-17T21:18:17.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014613607000000001, 2024-09-17T21:18:17.139 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.139 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.139 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.139 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.139 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:17.139 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:17.139 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 48 2024-09-17T21:18:17.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.139 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336089+0000", 2024-09-17T21:18:17.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336089+0000", 2024-09-17T21:18:17.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336090+0000", 2024-09-17T21:18:17.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.083e-06 2024-09-17T21:18:17.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336097+0000", 2024-09-17T21:18:17.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0299999999999996e-06 2024-09-17T21:18:17.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336151+0000", 2024-09-17T21:18:17.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4292999999999997e-05 2024-09-17T21:18:17.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336163+0000", 2024-09-17T21:18:17.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.136e-05 2024-09-17T21:18:17.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.346568+0000", 2024-09-17T21:18:17.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010405009999999999 2024-09-17T21:18:17.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.346687+0000", 2024-09-17T21:18:17.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011885899999999999 2024-09-17T21:18:17.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:17.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.346865+0000", 2024-09-17T21:18:17.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017869500000000001 2024-09-17T21:18:17.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:17.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.350554+0000", 2024-09-17T21:18:17.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036889879999999998 2024-09-17T21:18:17.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:17.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.350575+0000", 2024-09-17T21:18:17.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.052e-05 2024-09-17T21:18:17.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.350645+0000", 2024-09-17T21:18:17.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0506000000000006e-05 2024-09-17T21:18:17.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.350702+0000", 2024-09-17T21:18:17.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7262999999999997e-05 2024-09-17T21:18:17.144 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.144 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.144 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.144 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:17.144 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.336187+0000", 2024-09-17T21:18:17.144 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.711752294, 2024-09-17T21:18:17.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015759305000000001, 2024-09-17T21:18:17.145 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.145 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.145 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.145 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.145 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:17.145 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:17.145 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 49 2024-09-17T21:18:17.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.145 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336187+0000", 2024-09-17T21:18:17.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336187+0000", 2024-09-17T21:18:17.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336188+0000", 2024-09-17T21:18:17.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3230000000000001e-06 2024-09-17T21:18:17.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336198+0000", 2024-09-17T21:18:17.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7440000000000002e-06 2024-09-17T21:18:17.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336210+0000", 2024-09-17T21:18:17.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1950000000000001e-05 2024-09-17T21:18:17.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336218+0000", 2024-09-17T21:18:17.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6329999999999996e-06 2024-09-17T21:18:17.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.347858+0000", 2024-09-17T21:18:17.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011640322999999999 2024-09-17T21:18:17.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.347979+0000", 2024-09-17T21:18:17.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012050899999999999 2024-09-17T21:18:17.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:17.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.348185+0000", 2024-09-17T21:18:17.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020600499999999999 2024-09-17T21:18:17.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:17.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.350801+0000", 2024-09-17T21:18:17.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026167830000000001 2024-09-17T21:18:17.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:17.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.351816+0000", 2024-09-17T21:18:17.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001015019 2024-09-17T21:18:17.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.351886+0000", 2024-09-17T21:18:17.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9952999999999999e-05 2024-09-17T21:18:17.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.351946+0000", 2024-09-17T21:18:17.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0062999999999997e-05 2024-09-17T21:18:17.150 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.150 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.150 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.150 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:17.150 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.336258+0000", 2024-09-17T21:18:17.150 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.711681542000001, 2024-09-17T21:18:17.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016750900999999999, 2024-09-17T21:18:17.151 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.151 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.151 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.151 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.151 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:17.151 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:17.151 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 50 2024-09-17T21:18:17.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.151 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336258+0000", 2024-09-17T21:18:17.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336258+0000", 2024-09-17T21:18:17.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336259+0000", 2024-09-17T21:18:17.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.11e-06 2024-09-17T21:18:17.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336369+0000", 2024-09-17T21:18:17.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010995 2024-09-17T21:18:17.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336384+0000", 2024-09-17T21:18:17.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5353e-05 2024-09-17T21:18:17.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336395+0000", 2024-09-17T21:18:17.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0995999999999999e-05 2024-09-17T21:18:17.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.349183+0000", 2024-09-17T21:18:17.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012788152000000001 2024-09-17T21:18:17.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.349310+0000", 2024-09-17T21:18:17.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012668600000000001 2024-09-17T21:18:17.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:17.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.349523+0000", 2024-09-17T21:18:17.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021328499999999999 2024-09-17T21:18:17.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:17.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.352854+0000", 2024-09-17T21:18:17.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033302140000000002 2024-09-17T21:18:17.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:17.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.352877+0000", 2024-09-17T21:18:17.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3195999999999999e-05 2024-09-17T21:18:17.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.352949+0000", 2024-09-17T21:18:17.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2086000000000001e-05 2024-09-17T21:18:17.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353009+0000", 2024-09-17T21:18:17.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9873000000000001e-05 2024-09-17T21:18:17.156 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.156 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.156 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.156 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:17.156 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.336415+0000", 2024-09-17T21:18:17.157 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.711523972999998, 2024-09-17T21:18:17.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016888507000000001, 2024-09-17T21:18:17.157 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.157 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.157 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.157 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.157 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:17.157 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:17.157 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 51 2024-09-17T21:18:17.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.157 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336415+0000", 2024-09-17T21:18:17.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336415+0000", 2024-09-17T21:18:17.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336416+0000", 2024-09-17T21:18:17.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7000000000000003e-07 2024-09-17T21:18:17.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336423+0000", 2024-09-17T21:18:17.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0940000000000002e-06 2024-09-17T21:18:17.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336433+0000", 2024-09-17T21:18:17.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.73e-06 2024-09-17T21:18:17.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336440+0000", 2024-09-17T21:18:17.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.973e-06 2024-09-17T21:18:17.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.350896+0000", 2024-09-17T21:18:17.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014456101000000001 2024-09-17T21:18:17.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.351027+0000", 2024-09-17T21:18:17.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000131063 2024-09-17T21:18:17.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:17.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.351249+0000", 2024-09-17T21:18:17.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000221568 2024-09-17T21:18:17.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:17.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353101+0000", 2024-09-17T21:18:17.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018519020000000001 2024-09-17T21:18:17.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:17.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353150+0000", 2024-09-17T21:18:17.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9607000000000001e-05 2024-09-17T21:18:17.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353249+0000", 2024-09-17T21:18:17.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8176000000000003e-05 2024-09-17T21:18:17.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353304+0000", 2024-09-17T21:18:17.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5322999999999999e-05 2024-09-17T21:18:17.162 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.162 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.162 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.162 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:17.162 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.336482+0000", 2024-09-17T21:18:17.163 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.711456949999999, 2024-09-17T21:18:17.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017076834999999999, 2024-09-17T21:18:17.163 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.163 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.163 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.163 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.163 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:17.163 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:17.163 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 52 2024-09-17T21:18:17.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.163 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336482+0000", 2024-09-17T21:18:17.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336482+0000", 2024-09-17T21:18:17.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336483+0000", 2024-09-17T21:18:17.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1699999999999997e-07 2024-09-17T21:18:17.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336490+0000", 2024-09-17T21:18:17.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5030000000000002e-06 2024-09-17T21:18:17.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336501+0000", 2024-09-17T21:18:17.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0827e-05 2024-09-17T21:18:17.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336508+0000", 2024-09-17T21:18:17.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4630000000000004e-06 2024-09-17T21:18:17.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.352032+0000", 2024-09-17T21:18:17.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015524263 2024-09-17T21:18:17.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.352175+0000", 2024-09-17T21:18:17.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000143206 2024-09-17T21:18:17.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:17.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.352363+0000", 2024-09-17T21:18:17.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000187579 2024-09-17T21:18:17.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:17.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353377+0000", 2024-09-17T21:18:17.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001013772 2024-09-17T21:18:17.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:17.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353424+0000", 2024-09-17T21:18:17.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7456000000000001e-05 2024-09-17T21:18:17.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353499+0000", 2024-09-17T21:18:17.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4518999999999996e-05 2024-09-17T21:18:17.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353559+0000", 2024-09-17T21:18:17.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0430000000000002e-05 2024-09-17T21:18:17.168 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.168 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.168 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.169 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:17.169 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.531414+0000", 2024-09-17T21:18:17.169 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.516525532999999, 2024-09-17T21:18:17.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092795340000000007, 2024-09-17T21:18:17.169 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.169 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.169 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.169 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.169 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:17.169 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:17.169 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 70 2024-09-17T21:18:17.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.169 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531414+0000", 2024-09-17T21:18:17.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531414+0000", 2024-09-17T21:18:17.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531415+0000", 2024-09-17T21:18:17.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.186e-06 2024-09-17T21:18:17.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531433+0000", 2024-09-17T21:18:17.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7808e-05 2024-09-17T21:18:17.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531449+0000", 2024-09-17T21:18:17.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5704000000000001e-05 2024-09-17T21:18:17.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531460+0000", 2024-09-17T21:18:17.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1005999999999999e-05 2024-09-17T21:18:17.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.535548+0000", 2024-09-17T21:18:17.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040887600000000003 2024-09-17T21:18:17.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.535619+0000", 2024-09-17T21:18:17.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0596e-05 2024-09-17T21:18:17.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:17.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.535724+0000", 2024-09-17T21:18:17.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104981 2024-09-17T21:18:17.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:17.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.538697+0000", 2024-09-17T21:18:17.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0029726869999999999 2024-09-17T21:18:17.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:17.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.540626+0000", 2024-09-17T21:18:17.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00192942 2024-09-17T21:18:17.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.540664+0000", 2024-09-17T21:18:17.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7707999999999999e-05 2024-09-17T21:18:17.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.540693+0000", 2024-09-17T21:18:17.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9677999999999999e-05 2024-09-17T21:18:17.174 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.174 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.174 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.174 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:17.174 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.531478+0000", 2024-09-17T21:18:17.174 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.516461622000001, 2024-09-17T21:18:17.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093886179999999996, 2024-09-17T21:18:17.175 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.175 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.175 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.175 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.175 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:17.175 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:17.175 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 71 2024-09-17T21:18:17.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.175 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531478+0000", 2024-09-17T21:18:17.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531478+0000", 2024-09-17T21:18:17.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531479+0000", 2024-09-17T21:18:17.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6400000000000001e-07 2024-09-17T21:18:17.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531484+0000", 2024-09-17T21:18:17.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8470000000000001e-06 2024-09-17T21:18:17.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531494+0000", 2024-09-17T21:18:17.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0001e-05 2024-09-17T21:18:17.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531502+0000", 2024-09-17T21:18:17.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4699999999999996e-06 2024-09-17T21:18:17.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.536100+0000", 2024-09-17T21:18:17.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045982219999999999 2024-09-17T21:18:17.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.536164+0000", 2024-09-17T21:18:17.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4300000000000004e-05 2024-09-17T21:18:17.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:17.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.536262+0000", 2024-09-17T21:18:17.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7745999999999996e-05 2024-09-17T21:18:17.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:17.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.538700+0000", 2024-09-17T21:18:17.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002437874 2024-09-17T21:18:17.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:17.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.540750+0000", 2024-09-17T21:18:17.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0020496989999999999 2024-09-17T21:18:17.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.540808+0000", 2024-09-17T21:18:17.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7911999999999997e-05 2024-09-17T21:18:17.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.540866+0000", 2024-09-17T21:18:17.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8683e-05 2024-09-17T21:18:17.180 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.180 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.180 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.180 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:17.180 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.531534+0000", 2024-09-17T21:18:17.180 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.516405030000001, 2024-09-17T21:18:17.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0094631569999999998, 2024-09-17T21:18:17.181 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.181 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.181 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.181 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.181 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:17.181 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:17.181 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 72 2024-09-17T21:18:17.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.181 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531534+0000", 2024-09-17T21:18:17.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531534+0000", 2024-09-17T21:18:17.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531535+0000", 2024-09-17T21:18:17.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2699999999999998e-07 2024-09-17T21:18:17.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531610+0000", 2024-09-17T21:18:17.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5240000000000005e-05 2024-09-17T21:18:17.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531619+0000", 2024-09-17T21:18:17.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9239999999999996e-06 2024-09-17T21:18:17.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531624+0000", 2024-09-17T21:18:17.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0629999999999996e-06 2024-09-17T21:18:17.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.536924+0000", 2024-09-17T21:18:17.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052993859999999997 2024-09-17T21:18:17.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.536997+0000", 2024-09-17T21:18:17.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3073999999999999e-05 2024-09-17T21:18:17.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:17.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.537090+0000", 2024-09-17T21:18:17.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3405000000000004e-05 2024-09-17T21:18:17.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:17.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.540704+0000", 2024-09-17T21:18:17.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003613326 2024-09-17T21:18:17.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:17.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.540926+0000", 2024-09-17T21:18:17.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022201400000000001 2024-09-17T21:18:17.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.540966+0000", 2024-09-17T21:18:17.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0760000000000003e-05 2024-09-17T21:18:17.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.540997+0000", 2024-09-17T21:18:17.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1038000000000003e-05 2024-09-17T21:18:17.186 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.186 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.186 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.186 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:17.186 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.531642+0000", 2024-09-17T21:18:17.186 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.516296852, 2024-09-17T21:18:17.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010095023999999999, 2024-09-17T21:18:17.187 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.187 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.187 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.187 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.187 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:17.187 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:17.187 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 73 2024-09-17T21:18:17.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.187 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531642+0000", 2024-09-17T21:18:17.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531642+0000", 2024-09-17T21:18:17.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531643+0000", 2024-09-17T21:18:17.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3099999999999998e-07 2024-09-17T21:18:17.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531647+0000", 2024-09-17T21:18:17.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6370000000000001e-06 2024-09-17T21:18:17.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531652+0000", 2024-09-17T21:18:17.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4790000000000004e-06 2024-09-17T21:18:17.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531656+0000", 2024-09-17T21:18:17.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8530000000000002e-06 2024-09-17T21:18:17.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.537928+0000", 2024-09-17T21:18:17.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062718560000000001 2024-09-17T21:18:17.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.537992+0000", 2024-09-17T21:18:17.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4284999999999999e-05 2024-09-17T21:18:17.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:17.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.538086+0000", 2024-09-17T21:18:17.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4094999999999999e-05 2024-09-17T21:18:17.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:17.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.540707+0000", 2024-09-17T21:18:17.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002621057 2024-09-17T21:18:17.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:17.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541673+0000", 2024-09-17T21:18:17.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00096603399999999995 2024-09-17T21:18:17.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541709+0000", 2024-09-17T21:18:17.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5790999999999997e-05 2024-09-17T21:18:17.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541738+0000", 2024-09-17T21:18:17.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8405999999999999e-05 2024-09-17T21:18:17.192 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.192 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.192 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.192 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:17.193 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.531665+0000", 2024-09-17T21:18:17.193 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.516274558999999, 2024-09-17T21:18:17.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010231192, 2024-09-17T21:18:17.193 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.193 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.193 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.193 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.193 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:17.193 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:17.193 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 74 2024-09-17T21:18:17.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.193 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531665+0000", 2024-09-17T21:18:17.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531665+0000", 2024-09-17T21:18:17.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531665+0000", 2024-09-17T21:18:17.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.34e-07 2024-09-17T21:18:17.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531670+0000", 2024-09-17T21:18:17.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8230000000000002e-06 2024-09-17T21:18:17.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531675+0000", 2024-09-17T21:18:17.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8810000000000004e-06 2024-09-17T21:18:17.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531680+0000", 2024-09-17T21:18:17.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2959999999999998e-06 2024-09-17T21:18:17.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.538749+0000", 2024-09-17T21:18:17.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0070683309999999997 2024-09-17T21:18:17.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.538834+0000", 2024-09-17T21:18:17.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5620999999999999e-05 2024-09-17T21:18:17.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:17.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.538934+0000", 2024-09-17T21:18:17.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9508999999999999e-05 2024-09-17T21:18:17.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:17.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541012+0000", 2024-09-17T21:18:17.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0020783099999999999 2024-09-17T21:18:17.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:17.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541781+0000", 2024-09-17T21:18:17.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00076862700000000003 2024-09-17T21:18:17.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541835+0000", 2024-09-17T21:18:17.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4283999999999997e-05 2024-09-17T21:18:17.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541896+0000", 2024-09-17T21:18:17.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1075999999999996e-05 2024-09-17T21:18:17.198 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.199 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.199 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.199 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:17.199 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.531687+0000", 2024-09-17T21:18:17.199 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.516252297000001, 2024-09-17T21:18:17.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010899883000000001, 2024-09-17T21:18:17.199 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.199 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.199 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.199 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.199 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:17.200 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:17.200 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 75 2024-09-17T21:18:17.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.200 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531687+0000", 2024-09-17T21:18:17.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531687+0000", 2024-09-17T21:18:17.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531687+0000", 2024-09-17T21:18:17.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8500000000000002e-07 2024-09-17T21:18:17.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531748+0000", 2024-09-17T21:18:17.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0588000000000003e-05 2024-09-17T21:18:17.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531756+0000", 2024-09-17T21:18:17.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6399999999999997e-06 2024-09-17T21:18:17.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531760+0000", 2024-09-17T21:18:17.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4870000000000004e-06 2024-09-17T21:18:17.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.539287+0000", 2024-09-17T21:18:17.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0075267379999999998 2024-09-17T21:18:17.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.539353+0000", 2024-09-17T21:18:17.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6577999999999999e-05 2024-09-17T21:18:17.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:17.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.539470+0000", 2024-09-17T21:18:17.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000116662 2024-09-17T21:18:17.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:17.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541287+0000", 2024-09-17T21:18:17.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018171839999999999 2024-09-17T21:18:17.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:17.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.542494+0000", 2024-09-17T21:18:17.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0012064650000000001 2024-09-17T21:18:17.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.542533+0000", 2024-09-17T21:18:17.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9196999999999998e-05 2024-09-17T21:18:17.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.542587+0000", 2024-09-17T21:18:17.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3959000000000003e-05 2024-09-17T21:18:17.204 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.204 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.205 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.205 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:17.205 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.531769+0000", 2024-09-17T21:18:17.205 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.516169875999999, 2024-09-17T21:18:17.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011381054, 2024-09-17T21:18:17.205 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.205 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.205 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.205 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.205 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:17.205 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:17.206 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 76 2024-09-17T21:18:17.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.206 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531769+0000", 2024-09-17T21:18:17.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531769+0000", 2024-09-17T21:18:17.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531770+0000", 2024-09-17T21:18:17.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8100000000000003e-07 2024-09-17T21:18:17.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531782+0000", 2024-09-17T21:18:17.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1832e-05 2024-09-17T21:18:17.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531787+0000", 2024-09-17T21:18:17.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3870000000000003e-06 2024-09-17T21:18:17.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531797+0000", 2024-09-17T21:18:17.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0036e-05 2024-09-17T21:18:17.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.540161+0000", 2024-09-17T21:18:17.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083641919999999995 2024-09-17T21:18:17.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.540220+0000", 2024-09-17T21:18:17.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8829000000000002e-05 2024-09-17T21:18:17.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:17.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.540312+0000", 2024-09-17T21:18:17.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1983000000000003e-05 2024-09-17T21:18:17.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:17.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.542152+0000", 2024-09-17T21:18:17.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018396840000000001 2024-09-17T21:18:17.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:17.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.543063+0000", 2024-09-17T21:18:17.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00091107600000000003 2024-09-17T21:18:17.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.543102+0000", 2024-09-17T21:18:17.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8767999999999999e-05 2024-09-17T21:18:17.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.543151+0000", 2024-09-17T21:18:17.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8785999999999997e-05 2024-09-17T21:18:17.210 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.210 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.211 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.211 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.211 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.211 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:17.212 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:18:17.216 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:17.208+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:17.313 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:17.306+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:17.357 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:17.357 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:17.357 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:18:17.358 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:17.371 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:18:17.414 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:17.408+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:17.515 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:17.510+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:17.532 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:17.532 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:17.532 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:18:17.532 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:18:17.532 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:17.546 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:18:17.616 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:17.612+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:17.713 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:17.714 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:18:17.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:18:17.714 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:17.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.714 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-17T21:18:17.714 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:12.297057+0000", 2024-09-17T21:18:17.714 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.4135739640000002, 2024-09-17T21:18:17.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043531430000000003, 2024-09-17T21:18:17.714 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.714 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.714 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.714 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.715 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:17.715 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:17.715 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 7 2024-09-17T21:18:17.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.715 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:12.297057+0000", 2024-09-17T21:18:17.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:12.297057+0000", 2024-09-17T21:18:17.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:12.297060+0000", 2024-09-17T21:18:17.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8469999999999998e-06 2024-09-17T21:18:17.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:12.297078+0000", 2024-09-17T21:18:17.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7343e-05 2024-09-17T21:18:17.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:12.297111+0000", 2024-09-17T21:18:17.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3123000000000001e-05 2024-09-17T21:18:17.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:12.297134+0000", 2024-09-17T21:18:17.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.319e-05 2024-09-17T21:18:17.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:12.297243+0000", 2024-09-17T21:18:17.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000109422 2024-09-17T21:18:17.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:12.297423+0000", 2024-09-17T21:18:17.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017918900000000001 2024-09-17T21:18:17.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:18:17.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:12.297635+0000", 2024-09-17T21:18:17.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021186100000000001 2024-09-17T21:18:17.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:17.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:12.299394+0000", 2024-09-17T21:18:17.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017590209999999999 2024-09-17T21:18:17.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:17.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:12.301250+0000", 2024-09-17T21:18:17.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018559679999999999 2024-09-17T21:18:17.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:12.301344+0000", 2024-09-17T21:18:17.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4399000000000002e-05 2024-09-17T21:18:17.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:12.301410+0000", 2024-09-17T21:18:17.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5779999999999997e-05 2024-09-17T21:18:17.720 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.720 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.720 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.720 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:17.720 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:13.306888+0000", 2024-09-17T21:18:17.720 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.4037428140000001, 2024-09-17T21:18:17.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010487935, 2024-09-17T21:18:17.720 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.721 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.721 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.721 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.721 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:17.721 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:17.721 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 9 2024-09-17T21:18:17.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.721 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.306888+0000", 2024-09-17T21:18:17.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.306888+0000", 2024-09-17T21:18:17.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.306892+0000", 2024-09-17T21:18:17.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8460000000000001e-06 2024-09-17T21:18:17.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.306916+0000", 2024-09-17T21:18:17.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4522999999999999e-05 2024-09-17T21:18:17.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.306958+0000", 2024-09-17T21:18:17.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1909999999999997e-05 2024-09-17T21:18:17.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.306983+0000", 2024-09-17T21:18:17.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5035999999999999e-05 2024-09-17T21:18:17.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.307155+0000", 2024-09-17T21:18:17.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017227599999999999 2024-09-17T21:18:17.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:17.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.307174+0000", 2024-09-17T21:18:17.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8940000000000002e-05 2024-09-17T21:18:17.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.311885+0000", 2024-09-17T21:18:17.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0047109860000000003 2024-09-17T21:18:17.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.312129+0000", 2024-09-17T21:18:17.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024392500000000001 2024-09-17T21:18:17.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:18:17.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.312487+0000", 2024-09-17T21:18:17.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035794000000000001 2024-09-17T21:18:17.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:17.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.314290+0000", 2024-09-17T21:18:17.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001803303 2024-09-17T21:18:17.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:17.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.317157+0000", 2024-09-17T21:18:17.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028671220000000002 2024-09-17T21:18:17.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.317303+0000", 2024-09-17T21:18:17.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014547200000000001 2024-09-17T21:18:17.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.317376+0000", 2024-09-17T21:18:17.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2656000000000004e-05 2024-09-17T21:18:17.727 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.727 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.727 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.727 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4434.0:9 3.1a e25/23 ondisk, result = 0)", 2024-09-17T21:18:17.727 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:13.314019+0000", 2024-09-17T21:18:17.727 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.3966118439999997, 2024-09-17T21:18:17.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029901500000000002, 2024-09-17T21:18:17.727 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.727 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.728 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:17.728 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.314019+0000", 2024-09-17T21:18:17.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.314019+0000", 2024-09-17T21:18:17.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.314020+0000", 2024-09-17T21:18:17.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6169999999999999e-06 2024-09-17T21:18:17.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.314030+0000", 2024-09-17T21:18:17.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4970000000000007e-06 2024-09-17T21:18:17.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.314049+0000", 2024-09-17T21:18:17.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8873e-05 2024-09-17T21:18:17.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.314070+0000", 2024-09-17T21:18:17.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.154e-05 2024-09-17T21:18:17.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.314235+0000", 2024-09-17T21:18:17.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016473599999999999 2024-09-17T21:18:17.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.314271+0000", 2024-09-17T21:18:17.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5929000000000001e-05 2024-09-17T21:18:17.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.314318+0000", 2024-09-17T21:18:17.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6823000000000002e-05 2024-09-17T21:18:17.731 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.731 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.732 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.732 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4434.0:10 3.1c e26/23 3:3831950c:::rbd_directory:head v 26'1, mlcod=0'0)", 2024-09-17T21:18:17.732 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:13.325368+0000", 2024-09-17T21:18:17.732 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.385262451, 2024-09-17T21:18:17.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018063560000000001, 2024-09-17T21:18:17.732 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.732 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.732 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.732 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.325368+0000", 2024-09-17T21:18:17.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.325368+0000", 2024-09-17T21:18:17.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.325369+0000", 2024-09-17T21:18:17.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1540000000000001e-06 2024-09-17T21:18:17.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.325380+0000", 2024-09-17T21:18:17.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1253000000000001e-05 2024-09-17T21:18:17.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.325407+0000", 2024-09-17T21:18:17.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6279999999999999e-05 2024-09-17T21:18:17.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.325427+0000", 2024-09-17T21:18:17.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9966000000000002e-05 2024-09-17T21:18:17.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.325562+0000", 2024-09-17T21:18:17.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000134912 2024-09-17T21:18:17.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.325609+0000", 2024-09-17T21:18:17.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6922999999999998e-05 2024-09-17T21:18:17.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.327072+0000", 2024-09-17T21:18:17.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0014633090000000001 2024-09-17T21:18:17.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.327174+0000", 2024-09-17T21:18:17.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010255900000000001 2024-09-17T21:18:17.737 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.737 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.737 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.737 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4434.0:13 3.1 e26/23 3:81054d48:::rbd_object_map.1152c01721e5:head v 26'1, mlcod=0'0)", 2024-09-17T21:18:17.737 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:13.339738+0000", 2024-09-17T21:18:17.737 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.3708921580000002, 2024-09-17T21:18:17.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017218660000000001, 2024-09-17T21:18:17.737 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.737 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.738 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.738 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.339738+0000", 2024-09-17T21:18:17.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.339738+0000", 2024-09-17T21:18:17.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.339739+0000", 2024-09-17T21:18:17.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8999999999999996e-07 2024-09-17T21:18:17.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.339747+0000", 2024-09-17T21:18:17.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9500000000000001e-06 2024-09-17T21:18:17.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.339768+0000", 2024-09-17T21:18:17.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1365999999999998e-05 2024-09-17T21:18:17.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.339783+0000", 2024-09-17T21:18:17.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4562999999999999e-05 2024-09-17T21:18:17.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.339886+0000", 2024-09-17T21:18:17.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010276 2024-09-17T21:18:17.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.339918+0000", 2024-09-17T21:18:17.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2262999999999999e-05 2024-09-17T21:18:17.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.341396+0000", 2024-09-17T21:18:17.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001478271 2024-09-17T21:18:17.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.341460+0000", 2024-09-17T21:18:17.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4003000000000004e-05 2024-09-17T21:18:17.741 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.741 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.741 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.742 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:23 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:17.742 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:13.356542+0000", 2024-09-17T21:18:17.742 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.3540884310000001, 2024-09-17T21:18:17.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079154699999999998, 2024-09-17T21:18:17.742 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.742 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.742 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:17.742 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.742 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:17.742 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:17.742 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 23 2024-09-17T21:18:17.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.742 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.356542+0000", 2024-09-17T21:18:17.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.356542+0000", 2024-09-17T21:18:17.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.356545+0000", 2024-09-17T21:18:17.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5730000000000002e-06 2024-09-17T21:18:17.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.356556+0000", 2024-09-17T21:18:17.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1334e-05 2024-09-17T21:18:17.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.356587+0000", 2024-09-17T21:18:17.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1131999999999999e-05 2024-09-17T21:18:17.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.356602+0000", 2024-09-17T21:18:17.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.452e-05 2024-09-17T21:18:17.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.356766+0000", 2024-09-17T21:18:17.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000164513 2024-09-17T21:18:17.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.357015+0000", 2024-09-17T21:18:17.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024885100000000002 2024-09-17T21:18:17.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.357334+0000", 2024-09-17T21:18:17.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031862399999999999 2024-09-17T21:18:17.746 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.746 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.746 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.746 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:10 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:17.746 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:13.553846+0000", 2024-09-17T21:18:17.746 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.1567849270000004, 2024-09-17T21:18:17.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00055910200000000001, 2024-09-17T21:18:17.746 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.746 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.746 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:17.746 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.746 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:18:17.747 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:17.747 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10 2024-09-17T21:18:17.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.747 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.553846+0000", 2024-09-17T21:18:17.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.553846+0000", 2024-09-17T21:18:17.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.553848+0000", 2024-09-17T21:18:17.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2670000000000001e-06 2024-09-17T21:18:17.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.553874+0000", 2024-09-17T21:18:17.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6622999999999999e-05 2024-09-17T21:18:17.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.553897+0000", 2024-09-17T21:18:17.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3023e-05 2024-09-17T21:18:17.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.553922+0000", 2024-09-17T21:18:17.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4808999999999999e-05 2024-09-17T21:18:17.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.554082+0000", 2024-09-17T21:18:17.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016000300000000001 2024-09-17T21:18:17.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.554223+0000", 2024-09-17T21:18:17.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000140605 2024-09-17T21:18:17.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.554405+0000", 2024-09-17T21:18:17.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018177199999999999 2024-09-17T21:18:17.750 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.750 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.750 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.750 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:13 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:17.750 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:13.593126+0000", 2024-09-17T21:18:17.750 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.1175048900000002, 2024-09-17T21:18:17.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079187699999999997, 2024-09-17T21:18:17.751 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.751 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.751 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:17.751 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.751 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:18:17.751 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:17.751 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 13 2024-09-17T21:18:17.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.751 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.593126+0000", 2024-09-17T21:18:17.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.593126+0000", 2024-09-17T21:18:17.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.593128+0000", 2024-09-17T21:18:17.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:18:17.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.593145+0000", 2024-09-17T21:18:17.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7309e-05 2024-09-17T21:18:17.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.593176+0000", 2024-09-17T21:18:17.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0817e-05 2024-09-17T21:18:17.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.593191+0000", 2024-09-17T21:18:17.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5033e-05 2024-09-17T21:18:17.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.593350+0000", 2024-09-17T21:18:17.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015881599999999999 2024-09-17T21:18:17.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.593592+0000", 2024-09-17T21:18:17.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024263799999999999 2024-09-17T21:18:17.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.593917+0000", 2024-09-17T21:18:17.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032502399999999998 2024-09-17T21:18:17.754 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.754 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.754 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.755 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4488.0:565 3.1 e28/23 3:81054d48:::rbd_object_map.1152c01721e5:head v 28'43, mlcod=26'41)", 2024-09-17T21:18:17.755 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.292998+0000", 2024-09-17T21:18:17.755 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.41763230400000001, 2024-09-17T21:18:17.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0012218109999999999, 2024-09-17T21:18:17.755 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.755 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.755 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.755 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.292998+0000", 2024-09-17T21:18:17.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.292998+0000", 2024-09-17T21:18:17.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.292998+0000", 2024-09-17T21:18:17.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6199999999999999e-07 2024-09-17T21:18:17.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.293004+0000", 2024-09-17T21:18:17.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.096e-06 2024-09-17T21:18:17.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.293021+0000", 2024-09-17T21:18:17.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7529000000000001e-05 2024-09-17T21:18:17.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.293034+0000", 2024-09-17T21:18:17.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3081999999999999e-05 2024-09-17T21:18:17.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.293118+0000", 2024-09-17T21:18:17.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3836000000000002e-05 2024-09-17T21:18:17.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.293141+0000", 2024-09-17T21:18:17.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3011e-05 2024-09-17T21:18:17.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.294172+0000", 2024-09-17T21:18:17.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0010307599999999999 2024-09-17T21:18:17.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.294220+0000", 2024-09-17T21:18:17.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8235000000000002e-05 2024-09-17T21:18:17.759 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.759 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.759 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.759 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:575 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:17.759 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.299145+0000", 2024-09-17T21:18:17.759 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.41148583100000002, 2024-09-17T21:18:17.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031443500000000001, 2024-09-17T21:18:17.759 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.759 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.759 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:17.760 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.760 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:18:17.760 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:17.760 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 575 2024-09-17T21:18:17.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.760 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.299145+0000", 2024-09-17T21:18:17.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.299145+0000", 2024-09-17T21:18:17.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.299147+0000", 2024-09-17T21:18:17.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1059999999999998e-06 2024-09-17T21:18:17.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.299153+0000", 2024-09-17T21:18:17.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1999999999999999e-06 2024-09-17T21:18:17.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.299167+0000", 2024-09-17T21:18:17.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4246999999999999e-05 2024-09-17T21:18:17.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.299177+0000", 2024-09-17T21:18:17.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5370000000000003e-06 2024-09-17T21:18:17.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.299260+0000", 2024-09-17T21:18:17.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2987999999999999e-05 2024-09-17T21:18:17.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.299351+0000", 2024-09-17T21:18:17.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1425e-05 2024-09-17T21:18:17.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.299459+0000", 2024-09-17T21:18:17.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000107932 2024-09-17T21:18:17.763 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.763 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.763 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.763 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:236 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:17.764 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.301145+0000", 2024-09-17T21:18:17.764 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.40948528299999998, 2024-09-17T21:18:17.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031121600000000001, 2024-09-17T21:18:17.764 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.764 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.764 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:17.764 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.764 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:17.764 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:17.764 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 236 2024-09-17T21:18:17.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.764 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.301145+0000", 2024-09-17T21:18:17.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.301145+0000", 2024-09-17T21:18:17.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.301147+0000", 2024-09-17T21:18:17.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3659999999999999e-06 2024-09-17T21:18:17.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.301154+0000", 2024-09-17T21:18:17.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4560000000000003e-06 2024-09-17T21:18:17.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.301172+0000", 2024-09-17T21:18:17.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8212000000000001e-05 2024-09-17T21:18:17.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.301183+0000", 2024-09-17T21:18:17.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0942e-05 2024-09-17T21:18:17.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.301261+0000", 2024-09-17T21:18:17.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7928000000000003e-05 2024-09-17T21:18:17.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.301348+0000", 2024-09-17T21:18:17.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7241e-05 2024-09-17T21:18:17.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.301456+0000", 2024-09-17T21:18:17.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000108071 2024-09-17T21:18:17.767 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.768 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.768 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.768 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:581 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:17.768 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.303522+0000", 2024-09-17T21:18:17.768 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.40710817300000002, 2024-09-17T21:18:17.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000346671, 2024-09-17T21:18:17.768 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.768 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.768 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:17.768 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.768 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:18:17.768 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:17.769 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 581 2024-09-17T21:18:17.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.769 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.303522+0000", 2024-09-17T21:18:17.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.303522+0000", 2024-09-17T21:18:17.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.303523+0000", 2024-09-17T21:18:17.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1000000000000001e-06 2024-09-17T21:18:17.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.303529+0000", 2024-09-17T21:18:17.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9259999999999997e-06 2024-09-17T21:18:17.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.303549+0000", 2024-09-17T21:18:17.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0024e-05 2024-09-17T21:18:17.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.303562+0000", 2024-09-17T21:18:17.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2867000000000001e-05 2024-09-17T21:18:17.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.303641+0000", 2024-09-17T21:18:17.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8590999999999993e-05 2024-09-17T21:18:17.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.303743+0000", 2024-09-17T21:18:17.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102546 2024-09-17T21:18:17.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.303869+0000", 2024-09-17T21:18:17.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012561699999999999 2024-09-17T21:18:17.772 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.772 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.772 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.773 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:588 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:17.773 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.309342+0000", 2024-09-17T21:18:17.773 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.40128843600000003, 2024-09-17T21:18:17.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029670100000000002, 2024-09-17T21:18:17.773 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.773 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.773 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:17.773 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.773 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:18:17.773 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:17.773 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 588 2024-09-17T21:18:17.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.773 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.309342+0000", 2024-09-17T21:18:17.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.309342+0000", 2024-09-17T21:18:17.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.309343+0000", 2024-09-17T21:18:17.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1310000000000001e-06 2024-09-17T21:18:17.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.309350+0000", 2024-09-17T21:18:17.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8870000000000003e-06 2024-09-17T21:18:17.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.309364+0000", 2024-09-17T21:18:17.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3983e-05 2024-09-17T21:18:17.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.309371+0000", 2024-09-17T21:18:17.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0550000000000002e-06 2024-09-17T21:18:17.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.309447+0000", 2024-09-17T21:18:17.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5504000000000001e-05 2024-09-17T21:18:17.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.309538+0000", 2024-09-17T21:18:17.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1719999999999996e-05 2024-09-17T21:18:17.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.309639+0000", 2024-09-17T21:18:17.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010042100000000001 2024-09-17T21:18:17.777 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.777 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.777 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.777 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:243 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:17.777 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.314796+0000", 2024-09-17T21:18:17.777 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.39583464000000002, 2024-09-17T21:18:17.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029511499999999998, 2024-09-17T21:18:17.777 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.777 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.777 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:17.778 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.778 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:17.778 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:17.778 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 243 2024-09-17T21:18:17.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.778 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.314796+0000", 2024-09-17T21:18:17.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.314796+0000", 2024-09-17T21:18:17.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.314797+0000", 2024-09-17T21:18:17.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.421e-06 2024-09-17T21:18:17.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.314805+0000", 2024-09-17T21:18:17.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0730000000000005e-06 2024-09-17T21:18:17.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.314822+0000", 2024-09-17T21:18:17.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6443000000000002e-05 2024-09-17T21:18:17.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.314829+0000", 2024-09-17T21:18:17.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3860000000000001e-06 2024-09-17T21:18:17.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.314905+0000", 2024-09-17T21:18:17.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6171e-05 2024-09-17T21:18:17.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.314992+0000", 2024-09-17T21:18:17.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.674e-05 2024-09-17T21:18:17.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.315091+0000", 2024-09-17T21:18:17.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8881000000000004e-05 2024-09-17T21:18:17.781 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.781 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.781 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.781 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:252 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:17.782 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.329670+0000", 2024-09-17T21:18:17.782 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.38096059900000001, 2024-09-17T21:18:17.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033804100000000003, 2024-09-17T21:18:17.782 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.782 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.782 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:17.782 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.782 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:17.782 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:17.782 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 252 2024-09-17T21:18:17.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.782 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.329670+0000", 2024-09-17T21:18:17.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.329670+0000", 2024-09-17T21:18:17.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.329672+0000", 2024-09-17T21:18:17.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3029999999999998e-06 2024-09-17T21:18:17.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.329683+0000", 2024-09-17T21:18:17.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1236999999999999e-05 2024-09-17T21:18:17.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.329712+0000", 2024-09-17T21:18:17.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8102999999999998e-05 2024-09-17T21:18:17.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.329725+0000", 2024-09-17T21:18:17.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.381e-05 2024-09-17T21:18:17.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.329816+0000", 2024-09-17T21:18:17.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0673000000000004e-05 2024-09-17T21:18:17.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.329903+0000", 2024-09-17T21:18:17.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6978000000000006e-05 2024-09-17T21:18:17.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.330008+0000", 2024-09-17T21:18:17.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104937 2024-09-17T21:18:17.785 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.786 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.786 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.786 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:258 3.1a 3:5cea7035:::rbd_info:head [read 0~19] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:17.786 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.336826+0000", 2024-09-17T21:18:17.786 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.37380407500000001, 2024-09-17T21:18:17.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000605319, 2024-09-17T21:18:17.786 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.786 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.786 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:17.786 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.786 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:17.786 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:17.787 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 258 2024-09-17T21:18:17.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.787 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.336826+0000", 2024-09-17T21:18:17.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.336826+0000", 2024-09-17T21:18:17.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.336828+0000", 2024-09-17T21:18:17.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0729999999999999e-06 2024-09-17T21:18:17.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.336839+0000", 2024-09-17T21:18:17.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0706999999999999e-05 2024-09-17T21:18:17.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.336868+0000", 2024-09-17T21:18:17.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8843000000000002e-05 2024-09-17T21:18:17.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.336882+0000", 2024-09-17T21:18:17.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4042999999999999e-05 2024-09-17T21:18:17.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.336968+0000", 2024-09-17T21:18:17.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6363e-05 2024-09-17T21:18:17.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.337044+0000", 2024-09-17T21:18:17.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5524000000000004e-05 2024-09-17T21:18:17.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.337432+0000", 2024-09-17T21:18:17.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000387766 2024-09-17T21:18:17.790 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.790 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.790 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.790 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4434.0:259 3.1c e28/23 3:3831950c:::rbd_directory:head v 28'2, mlcod=0'0)", 2024-09-17T21:18:17.790 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.338971+0000", 2024-09-17T21:18:17.790 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.37165948500000001, 2024-09-17T21:18:17.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001026772, 2024-09-17T21:18:17.791 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.791 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.791 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.791 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.338971+0000", 2024-09-17T21:18:17.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.338971+0000", 2024-09-17T21:18:17.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.338972+0000", 2024-09-17T21:18:17.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4300000000000003e-07 2024-09-17T21:18:17.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.338980+0000", 2024-09-17T21:18:17.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4940000000000003e-06 2024-09-17T21:18:17.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.339004+0000", 2024-09-17T21:18:17.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4321999999999999e-05 2024-09-17T21:18:17.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.339017+0000", 2024-09-17T21:18:17.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2687000000000001e-05 2024-09-17T21:18:17.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.339090+0000", 2024-09-17T21:18:17.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2444e-05 2024-09-17T21:18:17.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.339112+0000", 2024-09-17T21:18:17.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2155e-05 2024-09-17T21:18:17.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.339922+0000", 2024-09-17T21:18:17.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00081014100000000001 2024-09-17T21:18:17.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.339998+0000", 2024-09-17T21:18:17.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5986000000000001e-05 2024-09-17T21:18:17.794 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.795 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.795 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.795 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:263 3.9 3:9797fd44:::rbd_object_map.1152c6af092e:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:17.795 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.352451+0000", 2024-09-17T21:18:17.795 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.35817991599999999, 2024-09-17T21:18:17.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031096539999999999, 2024-09-17T21:18:17.795 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.795 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.795 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.795 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.795 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:17.795 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:17.796 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 263 2024-09-17T21:18:17.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.796 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.352451+0000", 2024-09-17T21:18:17.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.352451+0000", 2024-09-17T21:18:17.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.352453+0000", 2024-09-17T21:18:17.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.367e-06 2024-09-17T21:18:17.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.352464+0000", 2024-09-17T21:18:17.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.092e-05 2024-09-17T21:18:17.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.352494+0000", 2024-09-17T21:18:17.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9909000000000001e-05 2024-09-17T21:18:17.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.352515+0000", 2024-09-17T21:18:17.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1047e-05 2024-09-17T21:18:17.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.352612+0000", 2024-09-17T21:18:17.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7145999999999995e-05 2024-09-17T21:18:17.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.352753+0000", 2024-09-17T21:18:17.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014065400000000001 2024-09-17T21:18:17.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:17.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.352945+0000", 2024-09-17T21:18:17.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019261500000000001 2024-09-17T21:18:17.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:17.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.353894+0000", 2024-09-17T21:18:17.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00094915799999999997 2024-09-17T21:18:17.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:17.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.355460+0000", 2024-09-17T21:18:17.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0015660240000000001 2024-09-17T21:18:17.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.355521+0000", 2024-09-17T21:18:17.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0847999999999997e-05 2024-09-17T21:18:17.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.355560+0000", 2024-09-17T21:18:17.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8965999999999999e-05 2024-09-17T21:18:17.800 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.800 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.800 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.801 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:274 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:17.801 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.376566+0000", 2024-09-17T21:18:17.801 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.33406457499999997, 2024-09-17T21:18:17.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00052807899999999998, 2024-09-17T21:18:17.801 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.801 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.801 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:17.801 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:17.801 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:17.801 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:17.801 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 274 2024-09-17T21:18:17.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.802 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.376566+0000", 2024-09-17T21:18:17.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.376566+0000", 2024-09-17T21:18:17.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.376569+0000", 2024-09-17T21:18:17.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:18:17.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.376587+0000", 2024-09-17T21:18:17.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8012999999999998e-05 2024-09-17T21:18:17.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.376624+0000", 2024-09-17T21:18:17.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6675999999999999e-05 2024-09-17T21:18:17.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.376649+0000", 2024-09-17T21:18:17.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5109999999999998e-05 2024-09-17T21:18:17.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.376786+0000", 2024-09-17T21:18:17.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000137355 2024-09-17T21:18:17.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.376920+0000", 2024-09-17T21:18:17.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013411599999999999 2024-09-17T21:18:17.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.377094+0000", 2024-09-17T21:18:17.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017359499999999999 2024-09-17T21:18:17.805 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.805 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.805 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.805 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4434.0:278 3.a e28/23 3:535385f0:::rbd_children:head v 28'7, mlcod=26'5)", 2024-09-17T21:18:17.805 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.385300+0000", 2024-09-17T21:18:17.805 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.32533007200000003, 2024-09-17T21:18:17.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00085101199999999995, 2024-09-17T21:18:17.806 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:17.806 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:17.806 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:17.806 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:17.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:17.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.385300+0000", 2024-09-17T21:18:17.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:17.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.385300+0000", 2024-09-17T21:18:17.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:17.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:17.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.385301+0000", 2024-09-17T21:18:17.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9999999999999997e-07 2024-09-17T21:18:17.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:17.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.385311+0000", 2024-09-17T21:18:17.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4960000000000002e-06 2024-09-17T21:18:17.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:17.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.385345+0000", 2024-09-17T21:18:17.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4143000000000002e-05 2024-09-17T21:18:17.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:17.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.385365+0000", 2024-09-17T21:18:17.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9786999999999999e-05 2024-09-17T21:18:17.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:17.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.385451+0000", 2024-09-17T21:18:17.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6321000000000002e-05 2024-09-17T21:18:17.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:17.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.385475+0000", 2024-09-17T21:18:17.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3765e-05 2024-09-17T21:18:17.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:17.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.386121+0000", 2024-09-17T21:18:17.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00064595400000000002 2024-09-17T21:18:17.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:17.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:17.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:17.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.386151+0000", 2024-09-17T21:18:17.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0846000000000002e-05 2024-09-17T21:18:17.809 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.809 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.809 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.810 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:17.810 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:17.810 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:17.811 INFO:teuthology.orchestra.run.smithi184.stdout:82 read 0x1262c00 thru 0x126cbd5 (0x9fd6 bytes) 2024-09-17T21:18:17.811 INFO:teuthology.orchestra.run.smithi184.stdout:84 read 0x7bb1600 thru 0x7bb5e76 (0x4877 bytes) 2024-09-17T21:18:17.811 INFO:teuthology.orchestra.run.smithi184.stdout:85 write 0x384e00 thru 0x388d04 (0x3f05 bytes) 2024-09-17T21:18:17.811 INFO:teuthology.orchestra.run.smithi184.stdout:86 write 0xba52800 thru 0xba5c969 (0xa16a bytes) 2024-09-17T21:18:17.812 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:18:17.812 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:17.806+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:17.814 INFO:teuthology.orchestra.run.smithi184.stdout:87 trunc from 0xd33ae00 to 0xdcdf400 2024-09-17T21:18:17.848 INFO:teuthology.orchestra.run.smithi184.stdout:88 punch from 0x1550e00 to 0x1553200, (0x2400 bytes) 2024-09-17T21:18:17.849 INFO:teuthology.orchestra.run.smithi184.stdout:90 write 0x6d8ca00 thru 0x6d8fa9b (0x309c bytes) 2024-09-17T21:18:17.850 INFO:teuthology.orchestra.run.smithi184.stdout:91 write 0x5717a00 thru 0x57233eb (0xb9ec bytes) 2024-09-17T21:18:17.850 INFO:teuthology.orchestra.run.smithi184.stdout:92 punch from 0xad87000 to 0xad8ee00, (0x7e00 bytes) 2024-09-17T21:18:17.897 INFO:teuthology.orchestra.run.smithi184.stdout:97 trunc from 0xdcdf400 to 0x6091000 2024-09-17T21:18:17.912 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:17.908+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:17.936 INFO:teuthology.orchestra.run.smithi184.stdout:102 punch from 0x3c2b000 to 0x3c2c000, (0x1000 bytes) 2024-09-17T21:18:17.938 INFO:teuthology.orchestra.run.smithi184.stdout:103 write 0x2050600 thru 0x205ae3d (0xa83e bytes) 2024-09-17T21:18:17.939 INFO:teuthology.orchestra.run.smithi184.stdout:104 punch from 0x847200 to 0x848400, (0x1200 bytes) 2024-09-17T21:18:17.969 INFO:teuthology.orchestra.run.smithi184.stdout:106 write 0xddb8e00 thru 0xddb9c27 (0xe28 bytes) 2024-09-17T21:18:17.973 INFO:teuthology.orchestra.run.smithi184.stdout:107 write 0x3057a00 thru 0x3064a3d (0xd03e bytes) 2024-09-17T21:18:17.973 INFO:teuthology.orchestra.run.smithi184.stdout:109 write 0x4028600 thru 0x40314a6 (0x8ea7 bytes) 2024-09-17T21:18:17.973 INFO:teuthology.orchestra.run.smithi184.stdout:113 punch from 0x8fd5800 to 0x8fe5400, (0xfc00 bytes) 2024-09-17T21:18:17.974 INFO:teuthology.orchestra.run.smithi184.stdout:116 write 0x189bc00 thru 0x189ec8f (0x3090 bytes) 2024-09-17T21:18:17.974 INFO:teuthology.orchestra.run.smithi184.stdout:118 read 0xb4e4400 thru 0xb4ef3cd (0xafce bytes) 2024-09-17T21:18:17.974 INFO:teuthology.orchestra.run.smithi184.stdout:122 read 0x69cfe00 thru 0x69d4d84 (0x4f85 bytes) 2024-09-17T21:18:17.974 INFO:teuthology.orchestra.run.smithi184.stdout:123 write 0xb9f7800 thru 0xb9fb803 (0x4004 bytes) 2024-09-17T21:18:17.974 INFO:teuthology.orchestra.run.smithi184.stdout:124 read 0x14ff400 thru 0x150ac28 (0xb829 bytes) 2024-09-17T21:18:17.976 INFO:teuthology.orchestra.run.smithi184.stdout:125 write 0x4758c00 thru 0x475b565 (0x2966 bytes) 2024-09-17T21:18:17.976 INFO:teuthology.orchestra.run.smithi184.stdout:127 read 0x2764800 thru 0x276761d (0x2e1e bytes) 2024-09-17T21:18:17.995 INFO:teuthology.orchestra.run.smithi184.stdout:128 trunc from 0xddb9e00 to 0x5132600 2024-09-17T21:18:18.000 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:18.000 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:18.000 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:18:18.000 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:18.014 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:18.008+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:18.018 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:18:18.045 INFO:teuthology.orchestra.run.smithi184.stdout:129 read 0x350da00 thru 0x3516214 (0x8815 bytes) 2024-09-17T21:18:18.050 INFO:teuthology.orchestra.run.smithi184.stdout:131 trunc from 0x5132600 to 0x6d1e800 2024-09-17T21:18:18.064 INFO:teuthology.orchestra.run.smithi184.stdout:133 read 0x3fd3800 thru 0x3fde553 (0xad54 bytes) 2024-09-17T21:18:18.066 INFO:teuthology.orchestra.run.smithi184.stdout:136 read 0x68c1200 thru 0x68d0fdb (0xfddc bytes) 2024-09-17T21:18:18.067 INFO:teuthology.orchestra.run.smithi184.stdout:137 punch from 0x37aba00 to 0x37ac800, (0xe00 bytes) 2024-09-17T21:18:18.067 INFO:teuthology.orchestra.run.smithi184.stdout:138 read 0x4e33600 thru 0x4e3a990 (0x7391 bytes) 2024-09-17T21:18:18.067 INFO:teuthology.orchestra.run.smithi184.stdout:140 write 0x5b9fe00 thru 0x5ba1536 (0x1737 bytes) 2024-09-17T21:18:18.067 INFO:teuthology.orchestra.run.smithi184.stdout:143 read 0x19e7000 thru 0x19ee696 (0x7697 bytes) 2024-09-17T21:18:18.068 INFO:teuthology.orchestra.run.smithi184.stdout:144 read 0xa3b400 thru 0xa469d4 (0xb5d5 bytes) 2024-09-17T21:18:18.098 INFO:teuthology.orchestra.run.smithi184.stdout:145 write 0xbc5b600 thru 0xbc6a790 (0xf191 bytes) 2024-09-17T21:18:18.100 INFO:teuthology.orchestra.run.smithi184.stdout:147 write 0x7e2600 thru 0x7e8be7 (0x65e8 bytes) 2024-09-17T21:18:18.101 INFO:teuthology.orchestra.run.smithi184.stdout:149 write 0x8f94400 thru 0x8fa2ba8 (0xe7a9 bytes) 2024-09-17T21:18:18.101 INFO:teuthology.orchestra.run.smithi184.stdout:152 read 0xb7bc000 thru 0xb7c0769 (0x476a bytes) 2024-09-17T21:18:18.108 INFO:teuthology.orchestra.run.smithi184.stdout:153 trunc from 0xbc6a800 to 0x787e000 2024-09-17T21:18:18.114 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:18.108+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:18.143 INFO:teuthology.orchestra.run.smithi184.stdout:155 read 0x46fe200 thru 0x47060cc (0x7ecd bytes) 2024-09-17T21:18:18.157 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:18.157 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:18.157 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:18:18.157 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:18:18.157 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:18.166 INFO:teuthology.orchestra.run.smithi184.stdout:157 write 0xabd0a00 thru 0xabd7fd5 (0x75d6 bytes) 2024-09-17T21:18:18.169 INFO:teuthology.orchestra.run.smithi184.stdout:158 punch from 0x9e59000 to 0x9e5fc00, (0x6c00 bytes) 2024-09-17T21:18:18.170 INFO:teuthology.orchestra.run.smithi184.stdout:161 write 0x9d30600 thru 0x9d3e92c (0xe32d bytes) 2024-09-17T21:18:18.175 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:18:18.191 INFO:teuthology.orchestra.run.smithi184.stdout:162 trunc from 0xabd8000 to 0x323aa00 2024-09-17T21:18:18.221 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:18.218+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:18.250 INFO:teuthology.orchestra.run.smithi184.stdout:163 trunc from 0x323aa00 to 0x21c1e00 2024-09-17T21:18:18.310 INFO:teuthology.orchestra.run.smithi184.stdout:168 write 0x30b1000 thru 0x30b6ea5 (0x5ea6 bytes) 2024-09-17T21:18:18.311 INFO:teuthology.orchestra.run.smithi184.stdout:169 punch from 0x4a3c00 to 0x4a5600, (0x1a00 bytes) 2024-09-17T21:18:18.316 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:18.312+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:18.316 INFO:teuthology.orchestra.run.smithi184.stdout:172 trunc from 0x30b7000 to 0x4b5be00 2024-09-17T21:18:18.321 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:18.322 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:18:18.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:18:18.322 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:18.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.322 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4190.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-17T21:18:18.322 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:01.049433+0000", 2024-09-17T21:18:18.322 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.270668661999999, 2024-09-17T21:18:18.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0064286639999999997, 2024-09-17T21:18:18.322 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:18.322 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:18.322 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4190", 2024-09-17T21:18:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.83:0/956669820", 2024-09-17T21:18:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:18:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.049433+0000", 2024-09-17T21:18:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.049433+0000", 2024-09-17T21:18:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.049444+0000", 2024-09-17T21:18:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1559999999999999e-05 2024-09-17T21:18:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.049458+0000", 2024-09-17T21:18:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3312999999999999e-05 2024-09-17T21:18:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.049487+0000", 2024-09-17T21:18:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9689999999999999e-05 2024-09-17T21:18:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.049520+0000", 2024-09-17T21:18:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3012000000000002e-05 2024-09-17T21:18:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.049699+0000", 2024-09-17T21:18:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017853300000000001 2024-09-17T21:18:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.050087+0000", 2024-09-17T21:18:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038779299999999999 2024-09-17T21:18:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.050435+0000", 2024-09-17T21:18:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00034841700000000001 2024-09-17T21:18:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:18.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.051936+0000", 2024-09-17T21:18:18.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001501053 2024-09-17T21:18:18.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:18.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.055707+0000", 2024-09-17T21:18:18.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0037706340000000001 2024-09-17T21:18:18.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:18.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.055790+0000", 2024-09-17T21:18:18.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3442000000000005e-05 2024-09-17T21:18:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.055861+0000", 2024-09-17T21:18:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1217000000000006e-05 2024-09-17T21:18:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4190.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-17T21:18:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:02.056803+0000", 2024-09-17T21:18:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 16.263298743, 2024-09-17T21:18:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0070956250000000004, 2024-09-17T21:18:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4190", 2024-09-17T21:18:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.83:0/956669820", 2024-09-17T21:18:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 5 2024-09-17T21:18:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.056803+0000", 2024-09-17T21:18:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.056803+0000", 2024-09-17T21:18:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.056806+0000", 2024-09-17T21:18:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6830000000000001e-06 2024-09-17T21:18:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.056829+0000", 2024-09-17T21:18:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2235999999999999e-05 2024-09-17T21:18:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.056863+0000", 2024-09-17T21:18:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4542999999999998e-05 2024-09-17T21:18:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.056885+0000", 2024-09-17T21:18:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2113999999999999e-05 2024-09-17T21:18:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.057027+0000", 2024-09-17T21:18:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014203499999999999 2024-09-17T21:18:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.057039+0000", 2024-09-17T21:18:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1297e-05 2024-09-17T21:18:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.060489+0000", 2024-09-17T21:18:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0034507330000000001 2024-09-17T21:18:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.060696+0000", 2024-09-17T21:18:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020716899999999999 2024-09-17T21:18:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:18:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.060908+0000", 2024-09-17T21:18:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000211735 2024-09-17T21:18:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.062413+0000", 2024-09-17T21:18:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0015043820000000001 2024-09-17T21:18:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.063746+0000", 2024-09-17T21:18:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0013335689999999999 2024-09-17T21:18:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.063830+0000", 2024-09-17T21:18:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4049999999999997e-05 2024-09-17T21:18:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.063898+0000", 2024-09-17T21:18:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8078999999999997e-05 2024-09-17T21:18:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4190.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-17T21:18:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:03.060815+0000", 2024-09-17T21:18:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.259286722000001, 2024-09-17T21:18:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011124612000000001, 2024-09-17T21:18:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4190", 2024-09-17T21:18:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.83:0/956669820", 2024-09-17T21:18:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 7 2024-09-17T21:18:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060815+0000", 2024-09-17T21:18:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060815+0000", 2024-09-17T21:18:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060819+0000", 2024-09-17T21:18:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2640000000000002e-06 2024-09-17T21:18:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060838+0000", 2024-09-17T21:18:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9406e-05 2024-09-17T21:18:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060878+0000", 2024-09-17T21:18:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0120000000000002e-05 2024-09-17T21:18:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060896+0000", 2024-09-17T21:18:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7986999999999999e-05 2024-09-17T21:18:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.061030+0000", 2024-09-17T21:18:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013395200000000001 2024-09-17T21:18:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.061041+0000", 2024-09-17T21:18:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1053000000000001e-05 2024-09-17T21:18:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.068612+0000", 2024-09-17T21:18:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0075704049999999997 2024-09-17T21:18:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.068806+0000", 2024-09-17T21:18:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019402500000000001 2024-09-17T21:18:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:18:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.069031+0000", 2024-09-17T21:18:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022502199999999999 2024-09-17T21:18:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.070704+0000", 2024-09-17T21:18:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001672974 2024-09-17T21:18:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071795+0000", 2024-09-17T21:18:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001091648 2024-09-17T21:18:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071878+0000", 2024-09-17T21:18:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2225999999999995e-05 2024-09-17T21:18:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071939+0000", 2024-09-17T21:18:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1530000000000002e-05 2024-09-17T21:18:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:13.319078+0000", 2024-09-17T21:18:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.0010232290000003, 2024-09-17T21:18:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089439210000000005, 2024-09-17T21:18:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10 2024-09-17T21:18:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.319078+0000", 2024-09-17T21:18:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.319078+0000", 2024-09-17T21:18:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.319081+0000", 2024-09-17T21:18:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.79e-06 2024-09-17T21:18:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.319094+0000", 2024-09-17T21:18:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.345e-05 2024-09-17T21:18:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.319115+0000", 2024-09-17T21:18:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0585999999999999e-05 2024-09-17T21:18:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.319130+0000", 2024-09-17T21:18:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5362999999999998e-05 2024-09-17T21:18:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.319278+0000", 2024-09-17T21:18:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000147205 2024-09-17T21:18:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.319287+0000", 2024-09-17T21:18:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9340000000000001e-06 2024-09-17T21:18:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.324174+0000", 2024-09-17T21:18:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0048864520000000003 2024-09-17T21:18:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.324547+0000", 2024-09-17T21:18:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037309300000000001 2024-09-17T21:18:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-17T21:18:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.325020+0000", 2024-09-17T21:18:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000473237 2024-09-17T21:18:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.327294+0000", 2024-09-17T21:18:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002273476 2024-09-17T21:18:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.327843+0000", 2024-09-17T21:18:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000549199 2024-09-17T21:18:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.327946+0000", 2024-09-17T21:18:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103322 2024-09-17T21:18:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.328022+0000", 2024-09-17T21:18:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5814000000000003e-05 2024-09-17T21:18:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:59 3.1d 3:bf155903:::rbd_data.1152c01721e5.0000000000000000:3 [read 7680~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.791960+0000", 2024-09-17T21:18:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.528141058, 2024-09-17T21:18:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0032856740000000001, 2024-09-17T21:18:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 59 2024-09-17T21:18:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.791960+0000", 2024-09-17T21:18:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.791960+0000", 2024-09-17T21:18:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.791961+0000", 2024-09-17T21:18:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8299999999999996e-07 2024-09-17T21:18:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.791964+0000", 2024-09-17T21:18:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.706e-06 2024-09-17T21:18:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.791971+0000", 2024-09-17T21:18:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8750000000000002e-06 2024-09-17T21:18:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.791975+0000", 2024-09-17T21:18:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7559999999999999e-06 2024-09-17T21:18:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.794944+0000", 2024-09-17T21:18:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0029681659999999999 2024-09-17T21:18:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.795083+0000", 2024-09-17T21:18:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013932300000000001 2024-09-17T21:18:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.795246+0000", 2024-09-17T21:18:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163065 2024-09-17T21:18:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:74 3.1d 3:bf155903:::rbd_data.1152c01721e5.0000000000000000:3 [read 15360~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.797652+0000", 2024-09-17T21:18:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.52244940100000004, 2024-09-17T21:18:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033456430000000001, 2024-09-17T21:18:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 74 2024-09-17T21:18:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.797652+0000", 2024-09-17T21:18:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.797652+0000", 2024-09-17T21:18:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.797653+0000", 2024-09-17T21:18:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6000000000000002e-07 2024-09-17T21:18:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.797656+0000", 2024-09-17T21:18:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.202e-06 2024-09-17T21:18:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.797663+0000", 2024-09-17T21:18:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2529999999999998e-06 2024-09-17T21:18:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.797668+0000", 2024-09-17T21:18:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8169999999999998e-06 2024-09-17T21:18:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.800505+0000", 2024-09-17T21:18:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028370980000000001 2024-09-17T21:18:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.800740+0000", 2024-09-17T21:18:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000234708 2024-09-17T21:18:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.800998+0000", 2024-09-17T21:18:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000257705 2024-09-17T21:18:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:75 3.1d 3:bf155903:::rbd_data.1152c01721e5.0000000000000000:3 [read 15872~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.797812+0000", 2024-09-17T21:18:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.52228961299999999, 2024-09-17T21:18:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003593022, 2024-09-17T21:18:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 75 2024-09-17T21:18:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.797812+0000", 2024-09-17T21:18:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.797812+0000", 2024-09-17T21:18:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.797812+0000", 2024-09-17T21:18:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3200000000000005e-07 2024-09-17T21:18:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.797815+0000", 2024-09-17T21:18:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3300000000000001e-06 2024-09-17T21:18:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.797821+0000", 2024-09-17T21:18:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1e-06 2024-09-17T21:18:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.797825+0000", 2024-09-17T21:18:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8510000000000001e-06 2024-09-17T21:18:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.801110+0000", 2024-09-17T21:18:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003285437 2024-09-17T21:18:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.801241+0000", 2024-09-17T21:18:18.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013060599999999999 2024-09-17T21:18:18.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:18.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.801405+0000", 2024-09-17T21:18:18.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016406600000000001 2024-09-17T21:18:18.362 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.362 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:18.362 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.362 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:173 3.1d 3:bf155903:::rbd_data.1152c01721e5.0000000000000000:3 [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.850337+0000", 2024-09-17T21:18:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.469763869, 2024-09-17T21:18:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035003930000000001, 2024-09-17T21:18:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 173 2024-09-17T21:18:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.850337+0000", 2024-09-17T21:18:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.850337+0000", 2024-09-17T21:18:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.850338+0000", 2024-09-17T21:18:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6599999999999995e-07 2024-09-17T21:18:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.850344+0000", 2024-09-17T21:18:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.31e-06 2024-09-17T21:18:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.850360+0000", 2024-09-17T21:18:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6574000000000001e-05 2024-09-17T21:18:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.850373+0000", 2024-09-17T21:18:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2723e-05 2024-09-17T21:18:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.853301+0000", 2024-09-17T21:18:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0029284210000000001 2024-09-17T21:18:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.853530+0000", 2024-09-17T21:18:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022830800000000001 2024-09-17T21:18:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.853838+0000", 2024-09-17T21:18:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00030829099999999998 2024-09-17T21:18:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:174 3.1d 3:bf155903:::rbd_data.1152c01721e5.0000000000000000:3 [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.850415+0000", 2024-09-17T21:18:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.469685874, 2024-09-17T21:18:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038787579999999999, 2024-09-17T21:18:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 174 2024-09-17T21:18:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.850415+0000", 2024-09-17T21:18:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.850415+0000", 2024-09-17T21:18:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.850417+0000", 2024-09-17T21:18:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4139999999999999e-06 2024-09-17T21:18:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.850423+0000", 2024-09-17T21:18:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0299999999999999e-06 2024-09-17T21:18:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.850440+0000", 2024-09-17T21:18:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.749e-05 2024-09-17T21:18:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.850454+0000", 2024-09-17T21:18:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3976e-05 2024-09-17T21:18:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.853923+0000", 2024-09-17T21:18:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0034685639999999999 2024-09-17T21:18:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.854052+0000", 2024-09-17T21:18:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012938899999999999 2024-09-17T21:18:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.854294+0000", 2024-09-17T21:18:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000241895 2024-09-17T21:18:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:202 3.1d 3:bf155903:::rbd_data.1152c01721e5.0000000000000000:3 [read 15360~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.858798+0000", 2024-09-17T21:18:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.46130363099999999, 2024-09-17T21:18:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033810099999999998, 2024-09-17T21:18:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 202 2024-09-17T21:18:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.858798+0000", 2024-09-17T21:18:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.858798+0000", 2024-09-17T21:18:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.858798+0000", 2024-09-17T21:18:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7000000000000004e-07 2024-09-17T21:18:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.858801+0000", 2024-09-17T21:18:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9730000000000002e-06 2024-09-17T21:18:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.858811+0000", 2024-09-17T21:18:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2040000000000003e-06 2024-09-17T21:18:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.858818+0000", 2024-09-17T21:18:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0199999999999997e-06 2024-09-17T21:18:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.861885+0000", 2024-09-17T21:18:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0030671589999999999 2024-09-17T21:18:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.862012+0000", 2024-09-17T21:18:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012735900000000001 2024-09-17T21:18:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.862179+0000", 2024-09-17T21:18:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000166625 2024-09-17T21:18:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:203 3.1d 3:bf155903:::rbd_data.1152c01721e5.0000000000000000:3 [read 15872~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.858828+0000", 2024-09-17T21:18:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.46127328499999998, 2024-09-17T21:18:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0037857170000000001, 2024-09-17T21:18:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 203 2024-09-17T21:18:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.858828+0000", 2024-09-17T21:18:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.858828+0000", 2024-09-17T21:18:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.858829+0000", 2024-09-17T21:18:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9999999999999997e-07 2024-09-17T21:18:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.858832+0000", 2024-09-17T21:18:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.157e-06 2024-09-17T21:18:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.858841+0000", 2024-09-17T21:18:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2830000000000007e-06 2024-09-17T21:18:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.858852+0000", 2024-09-17T21:18:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1017e-05 2024-09-17T21:18:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.862249+0000", 2024-09-17T21:18:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003396807 2024-09-17T21:18:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.862383+0000", 2024-09-17T21:18:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000133585 2024-09-17T21:18:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.862614+0000", 2024-09-17T21:18:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023116799999999999 2024-09-17T21:18:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:293 3.1d 3:bcd54fb7:::rbd_data.1152c6af092e.000000000000001c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 853504~43520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.944520+0000", 2024-09-17T21:18:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.37558114199999998, 2024-09-17T21:18:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036295160000000002, 2024-09-17T21:18:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 293 2024-09-17T21:18:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.944520+0000", 2024-09-17T21:18:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.944520+0000", 2024-09-17T21:18:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.944521+0000", 2024-09-17T21:18:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1500000000000003e-07 2024-09-17T21:18:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.944646+0000", 2024-09-17T21:18:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124788 2024-09-17T21:18:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.944659+0000", 2024-09-17T21:18:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.273e-05 2024-09-17T21:18:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.944668+0000", 2024-09-17T21:18:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3190000000000001e-06 2024-09-17T21:18:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.946361+0000", 2024-09-17T21:18:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00169298 2024-09-17T21:18:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.946457+0000", 2024-09-17T21:18:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6272000000000003e-05 2024-09-17T21:18:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-17T21:18:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.946600+0000", 2024-09-17T21:18:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142948 2024-09-17T21:18:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.947463+0000", 2024-09-17T21:18:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00086271899999999996 2024-09-17T21:18:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.948081+0000", 2024-09-17T21:18:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00061789799999999997 2024-09-17T21:18:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.948123+0000", 2024-09-17T21:18:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2648999999999998e-05 2024-09-17T21:18:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.948150+0000", 2024-09-17T21:18:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6298000000000001e-05 2024-09-17T21:18:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4423.0:411 3.17 e28/23 3:e80c86d5:::rbd_data.1152c6af092e.0000000000000061:head v 28'2, mlcod=0'0)", 2024-09-17T21:18:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.013868+0000", 2024-09-17T21:18:18.388 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.30623346600000001, 2024-09-17T21:18:18.388 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031973570000000001, 2024-09-17T21:18:18.388 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:18.388 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:18.388 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:18.388 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:18.388 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.388 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:18.388 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.013868+0000", 2024-09-17T21:18:18.388 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.388 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.388 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:18.388 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.013868+0000", 2024-09-17T21:18:18.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.389 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:18.389 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.013869+0000", 2024-09-17T21:18:18.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4410000000000001e-06 2024-09-17T21:18:18.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.389 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:18.389 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.013880+0000", 2024-09-17T21:18:18.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0454e-05 2024-09-17T21:18:18.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.390 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:18.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.013900+0000", 2024-09-17T21:18:18.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9816000000000001e-05 2024-09-17T21:18:18.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.390 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:18.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.013911+0000", 2024-09-17T21:18:18.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1178e-05 2024-09-17T21:18:18.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.390 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:18.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.014763+0000", 2024-09-17T21:18:18.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00085201700000000005 2024-09-17T21:18:18.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:18.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.014792+0000", 2024-09-17T21:18:18.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9268000000000002e-05 2024-09-17T21:18:18.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:18.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.017006+0000", 2024-09-17T21:18:18.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022135789999999998 2024-09-17T21:18:18.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:18.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.017065+0000", 2024-09-17T21:18:18.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9604000000000001e-05 2024-09-17T21:18:18.392 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.392 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:18.392 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.392 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:758 3.16 3:6aacae9d:::rbd_data.1152c6af092e.0000000000000039:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:18.392 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.203202+0000", 2024-09-17T21:18:18.392 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.11689946599999999, 2024-09-17T21:18:18.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041466790000000003, 2024-09-17T21:18:18.392 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:18.392 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:18.392 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:18.393 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:18.393 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:18.393 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:18.393 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 758 2024-09-17T21:18:18.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.393 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:18.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:18.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203202+0000", 2024-09-17T21:18:18.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:18.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203202+0000", 2024-09-17T21:18:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203204+0000", 2024-09-17T21:18:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7060000000000001e-06 2024-09-17T21:18:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203210+0000", 2024-09-17T21:18:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8000000000000001e-06 2024-09-17T21:18:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:18.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203232+0000", 2024-09-17T21:18:18.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1297999999999998e-05 2024-09-17T21:18:18.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:18.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203243+0000", 2024-09-17T21:18:18.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1396000000000001e-05 2024-09-17T21:18:18.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:18.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203313+0000", 2024-09-17T21:18:18.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0114e-05 2024-09-17T21:18:18.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203406+0000", 2024-09-17T21:18:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3153999999999996e-05 2024-09-17T21:18:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 3", 2024-09-17T21:18:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203539+0000", 2024-09-17T21:18:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000133053 2024-09-17T21:18:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.204432+0000", 2024-09-17T21:18:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00089268599999999996 2024-09-17T21:18:18.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:18.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.207240+0000", 2024-09-17T21:18:18.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028078000000000001 2024-09-17T21:18:18.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:18.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.207303+0000", 2024-09-17T21:18:18.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3501000000000001e-05 2024-09-17T21:18:18.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:18.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.207349+0000", 2024-09-17T21:18:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5170999999999998e-05 2024-09-17T21:18:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:770 3.e 3:71afa771:::rbd_data.1152c6af092e.00000000000000ac:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.203356+0000", 2024-09-17T21:18:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.11674557000000001, 2024-09-17T21:18:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0070554060000000002, 2024-09-17T21:18:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 770 2024-09-17T21:18:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203356+0000", 2024-09-17T21:18:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203356+0000", 2024-09-17T21:18:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203357+0000", 2024-09-17T21:18:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1000000000000001e-06 2024-09-17T21:18:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203362+0000", 2024-09-17T21:18:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.977e-06 2024-09-17T21:18:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203374+0000", 2024-09-17T21:18:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2296000000000001e-05 2024-09-17T21:18:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203383+0000", 2024-09-17T21:18:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3440000000000001e-06 2024-09-17T21:18:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203439+0000", 2024-09-17T21:18:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6096e-05 2024-09-17T21:18:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203521+0000", 2024-09-17T21:18:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1908000000000003e-05 2024-09-17T21:18:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-17T21:18:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203634+0000", 2024-09-17T21:18:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011326 2024-09-17T21:18:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.207402+0000", 2024-09-17T21:18:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0037684049999999998 2024-09-17T21:18:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.210322+0000", 2024-09-17T21:18:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002919839 2024-09-17T21:18:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.210362+0000", 2024-09-17T21:18:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0243000000000001e-05 2024-09-17T21:18:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.210411+0000", 2024-09-17T21:18:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8937999999999998e-05 2024-09-17T21:18:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4423.0:761 3.2 e28/23 3:449b5906:::rbd_data.1152c6af092e.0000000000000046:head v 28'3, mlcod=28'1)", 2024-09-17T21:18:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.204347+0000", 2024-09-17T21:18:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.115754257, 2024-09-17T21:18:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057932399999999998, 2024-09-17T21:18:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.204347+0000", 2024-09-17T21:18:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.204347+0000", 2024-09-17T21:18:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.204348+0000", 2024-09-17T21:18:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1900000000000001e-07 2024-09-17T21:18:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.204355+0000", 2024-09-17T21:18:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4359999999999996e-06 2024-09-17T21:18:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.204368+0000", 2024-09-17T21:18:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3005e-05 2024-09-17T21:18:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:18.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.204377+0000", 2024-09-17T21:18:18.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5909999999999996e-06 2024-09-17T21:18:18.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:18.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.204465+0000", 2024-09-17T21:18:18.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7960000000000005e-05 2024-09-17T21:18:18.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:18.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.204486+0000", 2024-09-17T21:18:18.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1109999999999999e-05 2024-09-17T21:18:18.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.408 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:18.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.210089+0000", 2024-09-17T21:18:18.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0056028450000000004 2024-09-17T21:18:18.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.408 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:18.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.210140+0000", 2024-09-17T21:18:18.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1374e-05 2024-09-17T21:18:18.408 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:790 3.5 3:a14b1a37:::rbd_data.1152c6af092e.000000000000002f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.221998+0000", 2024-09-17T21:18:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.098103196000000004, 2024-09-17T21:18:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006953018, 2024-09-17T21:18:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 790 2024-09-17T21:18:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.221998+0000", 2024-09-17T21:18:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.221998+0000", 2024-09-17T21:18:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.222000+0000", 2024-09-17T21:18:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1210000000000001e-06 2024-09-17T21:18:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.222009+0000", 2024-09-17T21:18:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4309999999999994e-06 2024-09-17T21:18:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.222030+0000", 2024-09-17T21:18:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1645999999999999e-05 2024-09-17T21:18:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.222042+0000", 2024-09-17T21:18:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1273e-05 2024-09-17T21:18:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.222116+0000", 2024-09-17T21:18:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4187e-05 2024-09-17T21:18:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.222288+0000", 2024-09-17T21:18:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000171991 2024-09-17T21:18:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.222488+0000", 2024-09-17T21:18:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000199831 2024-09-17T21:18:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.223213+0000", 2024-09-17T21:18:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00072577500000000001 2024-09-17T21:18:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.228886+0000", 2024-09-17T21:18:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0056722949999999999 2024-09-17T21:18:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.228921+0000", 2024-09-17T21:18:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5707999999999998e-05 2024-09-17T21:18:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.228951+0000", 2024-09-17T21:18:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.976e-05 2024-09-17T21:18:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:850 3.5 3:a14b1a37:::rbd_data.1152c6af092e.000000000000002f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.257546+0000", 2024-09-17T21:18:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.062555414000000004, 2024-09-17T21:18:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0047133050000000001, 2024-09-17T21:18:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:18:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257546+0000", 2024-09-17T21:18:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257546+0000", 2024-09-17T21:18:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257547+0000", 2024-09-17T21:18:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6400000000000001e-07 2024-09-17T21:18:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257550+0000", 2024-09-17T21:18:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-06 2024-09-17T21:18:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257558+0000", 2024-09-17T21:18:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:18:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257563+0000", 2024-09-17T21:18:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2360000000000004e-06 2024-09-17T21:18:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257625+0000", 2024-09-17T21:18:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1919000000000001e-05 2024-09-17T21:18:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257711+0000", 2024-09-17T21:18:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5811999999999998e-05 2024-09-17T21:18:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:18.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257877+0000", 2024-09-17T21:18:18.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000166324 2024-09-17T21:18:18.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:18.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.259279+0000", 2024-09-17T21:18:18.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001402396 2024-09-17T21:18:18.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.262211+0000", 2024-09-17T21:18:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002931534 2024-09-17T21:18:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.262239+0000", 2024-09-17T21:18:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7708999999999998e-05 2024-09-17T21:18:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.262259+0000", 2024-09-17T21:18:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0687e-05 2024-09-17T21:18:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:858 3.1d 3:bab05a5d:::rbd_data.1152c6af092e.0000000000000034:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.257734+0000", 2024-09-17T21:18:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.062367792999999998, 2024-09-17T21:18:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035786820000000001, 2024-09-17T21:18:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 858 2024-09-17T21:18:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257734+0000", 2024-09-17T21:18:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257734+0000", 2024-09-17T21:18:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257735+0000", 2024-09-17T21:18:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2550000000000001e-06 2024-09-17T21:18:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257740+0000", 2024-09-17T21:18:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2920000000000003e-06 2024-09-17T21:18:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:18.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257754+0000", 2024-09-17T21:18:18.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3875e-05 2024-09-17T21:18:18.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:18.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257762+0000", 2024-09-17T21:18:18.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0749999999999998e-06 2024-09-17T21:18:18.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:18.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.258253+0000", 2024-09-17T21:18:18.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00049078499999999998 2024-09-17T21:18:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.258333+0000", 2024-09-17T21:18:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0325000000000001e-05 2024-09-17T21:18:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-17T21:18:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.258451+0000", 2024-09-17T21:18:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000118192 2024-09-17T21:18:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.261086+0000", 2024-09-17T21:18:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026350359999999999 2024-09-17T21:18:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.261261+0000", 2024-09-17T21:18:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000174115 2024-09-17T21:18:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.261290+0000", 2024-09-17T21:18:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9728000000000001e-05 2024-09-17T21:18:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.261312+0000", 2024-09-17T21:18:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2004000000000001e-05 2024-09-17T21:18:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:859 3.e 3:752b81b8:::rbd_data.1152c6af092e.0000000000000035:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.257784+0000", 2024-09-17T21:18:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.062317764999999997, 2024-09-17T21:18:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0034215750000000001, 2024-09-17T21:18:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 859 2024-09-17T21:18:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257784+0000", 2024-09-17T21:18:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:18.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257784+0000", 2024-09-17T21:18:18.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:18.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:18.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257784+0000", 2024-09-17T21:18:18.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1099999999999995e-07 2024-09-17T21:18:18.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:18.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257787+0000", 2024-09-17T21:18:18.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.976e-06 2024-09-17T21:18:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257795+0000", 2024-09-17T21:18:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2130000000000008e-06 2024-09-17T21:18:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257801+0000", 2024-09-17T21:18:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4179999999999996e-06 2024-09-17T21:18:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257886+0000", 2024-09-17T21:18:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4627000000000001e-05 2024-09-17T21:18:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257959+0000", 2024-09-17T21:18:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3701000000000005e-05 2024-09-17T21:18:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-17T21:18:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.258057+0000", 2024-09-17T21:18:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7662000000000001e-05 2024-09-17T21:18:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.259246+0000", 2024-09-17T21:18:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011893699999999999 2024-09-17T21:18:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.261092+0000", 2024-09-17T21:18:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018460390000000001 2024-09-17T21:18:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.261157+0000", 2024-09-17T21:18:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4692999999999999e-05 2024-09-17T21:18:18.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:18.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:18.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:18.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.261205+0000", 2024-09-17T21:18:18.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8164999999999997e-05 2024-09-17T21:18:18.434 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.434 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:18.434 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.434 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:18.434 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:18.434 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:18.436 INFO:teuthology.orchestra.run.smithi184.stdout:174 write 0x5902000 thru 0x590ea16 (0xca17 bytes) 2024-09-17T21:18:18.436 INFO:teuthology.orchestra.run.smithi184.stdout:177 trunc from 0x590ec00 to 0x3c56400 2024-09-17T21:18:18.436 INFO:teuthology.orchestra.run.smithi184.stdout:178 write 0x9fbea00 thru 0x9fbf4bc (0xabd bytes) 2024-09-17T21:18:18.436 INFO:teuthology.orchestra.run.smithi184.stdout:179 read 0x7a9de00 thru 0x7aa4317 (0x6518 bytes) 2024-09-17T21:18:18.436 INFO:teuthology.orchestra.run.smithi184.stdout:181 trunc from 0x9fbf600 to 0x8d4ba00 2024-09-17T21:18:18.437 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:18.430+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:18.437 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:18:18.444 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:18:18.451 INFO:teuthology.orchestra.run.smithi184.stdout:182 punch from 0x3daa600 to 0x3daf400, (0x4e00 bytes) 2024-09-17T21:18:18.474 INFO:teuthology.orchestra.run.smithi184.stdout:183 write 0xac9d800 thru 0xaca131a (0x3b1b bytes) 2024-09-17T21:18:18.505 INFO:teuthology.orchestra.run.smithi184.stdout:184 write 0xc22f600 thru 0xc238dfe (0x97ff bytes) 2024-09-17T21:18:18.508 INFO:teuthology.orchestra.run.smithi184.stdout:185 read 0x45f1a00 thru 0x45ff1c9 (0xd7ca bytes) 2024-09-17T21:18:18.509 INFO:teuthology.orchestra.run.smithi184.stdout:186 trunc from 0xc238e00 to 0xb760000 2024-09-17T21:18:18.537 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:18.530+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:18.542 INFO:teuthology.orchestra.run.smithi184.stdout:187 punch from 0x8782000 to 0x8787000, (0x5000 bytes) 2024-09-17T21:18:18.543 INFO:teuthology.orchestra.run.smithi184.stdout:188 write 0xaf70800 thru 0xaf77e7a (0x767b bytes) 2024-09-17T21:18:18.544 INFO:teuthology.orchestra.run.smithi184.stdout:189 read 0x2c6c800 thru 0x2c77157 (0xa958 bytes) 2024-09-17T21:18:18.556 INFO:teuthology.orchestra.run.smithi184.stdout:190 trunc from 0xb760000 to 0x4f3c200 2024-09-17T21:18:18.570 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:18.570 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:18.570 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:18.570 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:18.586 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:18:18.593 INFO:teuthology.orchestra.run.smithi184.stdout:truncating to largest ever: 0xe8fd600 2024-09-17T21:18:18.618 INFO:teuthology.orchestra.run.smithi184.stdout:192 trunc from 0x4f3c200 to 0xe8fd600 2024-09-17T21:18:18.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:18:18.634 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:18:18.635 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:18:18.640 INFO:teuthology.orchestra.run.smithi184.stdout:193 punch from 0x4ea6000 to 0x4eae600, (0x8600 bytes) 2024-09-17T21:18:18.650 INFO:teuthology.orchestra.run.smithi184.stdout:196 trunc from 0xe8fd600 to 0xadb1200 2024-09-17T21:18:18.652 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:18:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:18:18.652 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:18.648+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:18.670 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:18:18.670 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:18:18.677 INFO:teuthology.orchestra.run.smithi184.stdout:197 read 0x4848a00 thru 0x4848b7d (0x17e bytes) 2024-09-17T21:18:18.678 INFO:teuthology.orchestra.run.smithi184.stdout:200 read 0x75db800 thru 0x75e8b32 (0xd333 bytes) 2024-09-17T21:18:18.678 INFO:teuthology.orchestra.run.smithi184.stdout:201 write 0x1c31600 thru 0x1c3399f (0x23a0 bytes) 2024-09-17T21:18:18.679 INFO:teuthology.orchestra.run.smithi184.stdout:202 write 0x8acd600 thru 0x8ad265e (0x505f bytes) 2024-09-17T21:18:18.679 INFO:teuthology.orchestra.run.smithi184.stdout:203 read 0x4cf9e00 thru 0x4d066dc (0xc8dd bytes) 2024-09-17T21:18:18.679 INFO:teuthology.orchestra.run.smithi184.stdout:205 write 0x1647a00 thru 0x1656afc (0xf0fd bytes) 2024-09-17T21:18:18.691 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:18:18.691 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:18:18.709 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:18.709 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:18.712 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:18:18.712 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:18:18.721 INFO:teuthology.orchestra.run.smithi184.stdout:209 write 0xbaf8600 thru 0xbb00358 (0x7d59 bytes) 2024-09-17T21:18:18.723 INFO:teuthology.orchestra.run.smithi184.stdout:210 write 0x6c6c600 thru 0x6c7a8fb (0xe2fc bytes) 2024-09-17T21:18:18.724 INFO:teuthology.orchestra.run.smithi184.stdout:212 read 0xe7be00 thru 0xe8335a (0x755b bytes) 2024-09-17T21:18:18.725 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:18:18.728 INFO:teuthology.orchestra.run.smithi184.stdout:213 read 0xa537800 thru 0xa53e3d5 (0x6bd6 bytes) 2024-09-17T21:18:18.729 INFO:teuthology.orchestra.run.smithi184.stdout:214 punch from 0xa064000 to 0xa06a000, (0x6000 bytes) 2024-09-17T21:18:18.730 INFO:teuthology.orchestra.run.smithi184.stdout:215 write 0x13bf000 thru 0x13c682e (0x782f bytes) 2024-09-17T21:18:18.731 INFO:teuthology.orchestra.run.smithi184.stdout:216 write 0xb08b200 thru 0xb08dd6d (0x2b6e bytes) 2024-09-17T21:18:18.731 INFO:teuthology.orchestra.run.smithi184.stdout:217 punch from 0x5f55e00 to 0x5f5ee00, (0x9000 bytes) 2024-09-17T21:18:18.731 INFO:teuthology.orchestra.run.smithi184.stdout:219 read 0x6142e00 thru 0x614ea52 (0xbc53 bytes) 2024-09-17T21:18:18.732 INFO:teuthology.orchestra.run.smithi184.stdout:220 read 0x58fe200 thru 0x5903212 (0x5013 bytes) 2024-09-17T21:18:18.732 INFO:teuthology.orchestra.run.smithi184.stdout:221 write 0xb756200 thru 0xb75d6a8 (0x74a9 bytes) 2024-09-17T21:18:18.732 INFO:teuthology.orchestra.run.smithi184.stdout:223 punch from 0x1778000 to 0x1778800, (0x800 bytes) 2024-09-17T21:18:18.732 INFO:teuthology.orchestra.run.smithi184.stdout:226 read 0x5c80e00 thru 0x5c8c033 (0xb234 bytes) 2024-09-17T21:18:18.732 INFO:teuthology.orchestra.run.smithi184.stdout:227 clone 2 order 24 su 524288 sc 5 2024-09-17T21:18:18.735 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:18:18.735 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:18:18.759 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:18.752+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:18.840 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:18.838+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:18.856 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:18.856 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:18.869 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:18:18.941 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:18.936+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:18.989 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:18.989 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:19.003 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:18:19.044 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:19.040+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:19.143 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:19.142+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:19.165 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:19.165 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:19.180 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:18:19.244 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:19.242+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:19.299 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:19.299 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:19.313 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:18:19.345 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:19.338+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:19.436 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:19.436 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:19.446 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:19.442+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:19.450 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:18:19.547 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:19.542+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:19.572 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:19.572 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:19.586 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:18:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-17T21:18:19.680 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:19.676+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:19.705 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:19.705 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:19.722 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:18:19.766 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone1 from 0xbb00400 (overlap 0x21c1e00) to 0x1b01800 2024-09-17T21:18:19.769 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:19.766+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:19.849 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:19.842+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:19.868 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:19.869 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:19.883 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:18:19.973 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:19.970+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:20.049 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:20.050 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:20.051 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:20.044+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:20.064 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:18:20.152 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:20.144+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:20.237 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:20.237 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:20.252 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:18:20.252 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:20.244+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:20.271 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-17T21:18:20.353 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:20.352+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:20.376 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:20.376 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:20.391 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:18:20.454 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:20.450+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:20.514 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:20.514 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:20.528 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:18:20.574 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:20.570+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:20.656 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:20.650+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:20.699 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:20.700 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:20.713 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:18:20.758 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:20.754+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:20.760 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:18:20.858 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:20.854+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:20.884 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:20.884 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:20.898 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:18:20.919 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:18:20.919 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:18:20.935 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:18:20.936 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:18:20.952 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:18:20.952 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:18:20.974 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:18:20.974 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:18:20.986 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:20.982+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:20.995 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:18:20.995 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:18:21.015 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:18:21.016 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:18:21.059 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:21.054+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:21.068 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:21.068 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:21.068 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:18:21.068 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:18:21.068 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:21.083 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:18:21.160 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:21.156+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:21.250 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:18:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:18:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4190.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-17T21:18:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:01.049433+0000", 2024-09-17T21:18:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 20.198586035000002, 2024-09-17T21:18:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0064286639999999997, 2024-09-17T21:18:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4190", 2024-09-17T21:18:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.83:0/956669820", 2024-09-17T21:18:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:18:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.049433+0000", 2024-09-17T21:18:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.049433+0000", 2024-09-17T21:18:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.049444+0000", 2024-09-17T21:18:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1559999999999999e-05 2024-09-17T21:18:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.049458+0000", 2024-09-17T21:18:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3312999999999999e-05 2024-09-17T21:18:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.049487+0000", 2024-09-17T21:18:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9689999999999999e-05 2024-09-17T21:18:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.049520+0000", 2024-09-17T21:18:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3012000000000002e-05 2024-09-17T21:18:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.049699+0000", 2024-09-17T21:18:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017853300000000001 2024-09-17T21:18:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.050087+0000", 2024-09-17T21:18:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038779299999999999 2024-09-17T21:18:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.050435+0000", 2024-09-17T21:18:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00034841700000000001 2024-09-17T21:18:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.051936+0000", 2024-09-17T21:18:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001501053 2024-09-17T21:18:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.055707+0000", 2024-09-17T21:18:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0037706340000000001 2024-09-17T21:18:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.055790+0000", 2024-09-17T21:18:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3442000000000005e-05 2024-09-17T21:18:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:01.055861+0000", 2024-09-17T21:18:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1217000000000006e-05 2024-09-17T21:18:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4190.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-17T21:18:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:02.056803+0000", 2024-09-17T21:18:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.191216116, 2024-09-17T21:18:21.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0070956250000000004, 2024-09-17T21:18:21.257 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:21.257 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:21.257 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.257 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:21.257 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4190", 2024-09-17T21:18:21.257 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.83:0/956669820", 2024-09-17T21:18:21.257 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 5 2024-09-17T21:18:21.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.257 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:21.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:21.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.056803+0000", 2024-09-17T21:18:21.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:21.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.056803+0000", 2024-09-17T21:18:21.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:21.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.056806+0000", 2024-09-17T21:18:21.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6830000000000001e-06 2024-09-17T21:18:21.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:21.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.056829+0000", 2024-09-17T21:18:21.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2235999999999999e-05 2024-09-17T21:18:21.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:21.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.056863+0000", 2024-09-17T21:18:21.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4542999999999998e-05 2024-09-17T21:18:21.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.056885+0000", 2024-09-17T21:18:21.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2113999999999999e-05 2024-09-17T21:18:21.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:21.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.057027+0000", 2024-09-17T21:18:21.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014203499999999999 2024-09-17T21:18:21.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:21.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.057039+0000", 2024-09-17T21:18:21.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1297e-05 2024-09-17T21:18:21.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:21.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.060489+0000", 2024-09-17T21:18:21.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0034507330000000001 2024-09-17T21:18:21.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:21.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.060696+0000", 2024-09-17T21:18:21.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020716899999999999 2024-09-17T21:18:21.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:18:21.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.060908+0000", 2024-09-17T21:18:21.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000211735 2024-09-17T21:18:21.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:21.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.062413+0000", 2024-09-17T21:18:21.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0015043820000000001 2024-09-17T21:18:21.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:21.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.063746+0000", 2024-09-17T21:18:21.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0013335689999999999 2024-09-17T21:18:21.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:21.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.063830+0000", 2024-09-17T21:18:21.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4049999999999997e-05 2024-09-17T21:18:21.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:21.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:02.063898+0000", 2024-09-17T21:18:21.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8078999999999997e-05 2024-09-17T21:18:21.263 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.263 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:21.263 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.263 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4190.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-17T21:18:21.263 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:03.060815+0000", 2024-09-17T21:18:21.263 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.187204094999998, 2024-09-17T21:18:21.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011124612000000001, 2024-09-17T21:18:21.264 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:21.264 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:21.264 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.264 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:21.264 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4190", 2024-09-17T21:18:21.264 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.83:0/956669820", 2024-09-17T21:18:21.264 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 7 2024-09-17T21:18:21.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.264 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:21.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:21.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060815+0000", 2024-09-17T21:18:21.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:21.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060815+0000", 2024-09-17T21:18:21.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:21.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060819+0000", 2024-09-17T21:18:21.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2640000000000002e-06 2024-09-17T21:18:21.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:21.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060838+0000", 2024-09-17T21:18:21.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9406e-05 2024-09-17T21:18:21.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:21.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060878+0000", 2024-09-17T21:18:21.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0120000000000002e-05 2024-09-17T21:18:21.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060896+0000", 2024-09-17T21:18:21.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7986999999999999e-05 2024-09-17T21:18:21.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:21.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.061030+0000", 2024-09-17T21:18:21.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013395200000000001 2024-09-17T21:18:21.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:21.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.061041+0000", 2024-09-17T21:18:21.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1053000000000001e-05 2024-09-17T21:18:21.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:21.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.068612+0000", 2024-09-17T21:18:21.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0075704049999999997 2024-09-17T21:18:21.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:21.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.068806+0000", 2024-09-17T21:18:21.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019402500000000001 2024-09-17T21:18:21.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:18:21.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.069031+0000", 2024-09-17T21:18:21.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022502199999999999 2024-09-17T21:18:21.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:21.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.070704+0000", 2024-09-17T21:18:21.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001672974 2024-09-17T21:18:21.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:21.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071795+0000", 2024-09-17T21:18:21.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001091648 2024-09-17T21:18:21.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:21.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071878+0000", 2024-09-17T21:18:21.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2225999999999995e-05 2024-09-17T21:18:21.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:21.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071939+0000", 2024-09-17T21:18:21.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1530000000000002e-05 2024-09-17T21:18:21.270 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.270 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:21.270 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.270 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:21.270 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:13.319078+0000", 2024-09-17T21:18:21.270 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.9289406019999999, 2024-09-17T21:18:21.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089439210000000005, 2024-09-17T21:18:21.271 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:21.271 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:21.271 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.271 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:21.271 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:21.271 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:21.271 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10 2024-09-17T21:18:21.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.271 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:21.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:21.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.319078+0000", 2024-09-17T21:18:21.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:21.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.319078+0000", 2024-09-17T21:18:21.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:21.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.319081+0000", 2024-09-17T21:18:21.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.79e-06 2024-09-17T21:18:21.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:21.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.319094+0000", 2024-09-17T21:18:21.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.345e-05 2024-09-17T21:18:21.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:21.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.319115+0000", 2024-09-17T21:18:21.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0585999999999999e-05 2024-09-17T21:18:21.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.319130+0000", 2024-09-17T21:18:21.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5362999999999998e-05 2024-09-17T21:18:21.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:21.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.319278+0000", 2024-09-17T21:18:21.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000147205 2024-09-17T21:18:21.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:21.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.319287+0000", 2024-09-17T21:18:21.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9340000000000001e-06 2024-09-17T21:18:21.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:21.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.324174+0000", 2024-09-17T21:18:21.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0048864520000000003 2024-09-17T21:18:21.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:21.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.324547+0000", 2024-09-17T21:18:21.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037309300000000001 2024-09-17T21:18:21.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-17T21:18:21.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.325020+0000", 2024-09-17T21:18:21.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000473237 2024-09-17T21:18:21.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:21.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.327294+0000", 2024-09-17T21:18:21.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002273476 2024-09-17T21:18:21.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:21.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.327843+0000", 2024-09-17T21:18:21.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000549199 2024-09-17T21:18:21.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:21.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.327946+0000", 2024-09-17T21:18:21.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103322 2024-09-17T21:18:21.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:21.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.328022+0000", 2024-09-17T21:18:21.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5814000000000003e-05 2024-09-17T21:18:21.279 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.279 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:21.279 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.279 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:75 3.1d 3:bf155903:::rbd_data.1152c01721e5.0000000000000000:3 [read 15872~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.279 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.797812+0000", 2024-09-17T21:18:21.280 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.450206986, 2024-09-17T21:18:21.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003593022, 2024-09-17T21:18:21.280 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:21.280 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:21.280 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:21.280 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:21.280 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:21.280 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.280 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 75 2024-09-17T21:18:21.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.280 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:21.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:21.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.797812+0000", 2024-09-17T21:18:21.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:21.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.797812+0000", 2024-09-17T21:18:21.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:21.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.797812+0000", 2024-09-17T21:18:21.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3200000000000005e-07 2024-09-17T21:18:21.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:21.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.797815+0000", 2024-09-17T21:18:21.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3300000000000001e-06 2024-09-17T21:18:21.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:21.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.797821+0000", 2024-09-17T21:18:21.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1e-06 2024-09-17T21:18:21.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.797825+0000", 2024-09-17T21:18:21.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8510000000000001e-06 2024-09-17T21:18:21.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:21.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.801110+0000", 2024-09-17T21:18:21.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003285437 2024-09-17T21:18:21.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:21.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.801241+0000", 2024-09-17T21:18:21.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013060599999999999 2024-09-17T21:18:21.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:21.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.801405+0000", 2024-09-17T21:18:21.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016406600000000001 2024-09-17T21:18:21.284 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.284 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:21.284 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.284 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:174 3.1d 3:bf155903:::rbd_data.1152c01721e5.0000000000000000:3 [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.284 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.850415+0000", 2024-09-17T21:18:21.284 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.3976032470000002, 2024-09-17T21:18:21.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038787579999999999, 2024-09-17T21:18:21.284 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:21.284 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:21.284 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:21.284 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:21.284 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:21.284 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.285 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 174 2024-09-17T21:18:21.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.285 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:21.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:21.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.850415+0000", 2024-09-17T21:18:21.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:21.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.850415+0000", 2024-09-17T21:18:21.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:21.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.850417+0000", 2024-09-17T21:18:21.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4139999999999999e-06 2024-09-17T21:18:21.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:21.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.850423+0000", 2024-09-17T21:18:21.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0299999999999999e-06 2024-09-17T21:18:21.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:21.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.850440+0000", 2024-09-17T21:18:21.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.749e-05 2024-09-17T21:18:21.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.850454+0000", 2024-09-17T21:18:21.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3976e-05 2024-09-17T21:18:21.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:21.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.853923+0000", 2024-09-17T21:18:21.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0034685639999999999 2024-09-17T21:18:21.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:21.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.854052+0000", 2024-09-17T21:18:21.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012938899999999999 2024-09-17T21:18:21.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:21.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.854294+0000", 2024-09-17T21:18:21.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000241895 2024-09-17T21:18:21.288 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.288 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:21.288 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.288 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:203 3.1d 3:bf155903:::rbd_data.1152c01721e5.0000000000000000:3 [read 15872~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.288 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.858828+0000", 2024-09-17T21:18:21.288 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.389190658, 2024-09-17T21:18:21.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0037857170000000001, 2024-09-17T21:18:21.288 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:21.288 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:21.288 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:21.289 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:21.289 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:21.289 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.289 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 203 2024-09-17T21:18:21.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.289 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:21.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:21.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.858828+0000", 2024-09-17T21:18:21.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:21.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.858828+0000", 2024-09-17T21:18:21.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:21.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.858829+0000", 2024-09-17T21:18:21.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9999999999999997e-07 2024-09-17T21:18:21.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:21.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.858832+0000", 2024-09-17T21:18:21.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.157e-06 2024-09-17T21:18:21.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:21.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.858841+0000", 2024-09-17T21:18:21.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2830000000000007e-06 2024-09-17T21:18:21.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.858852+0000", 2024-09-17T21:18:21.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1017e-05 2024-09-17T21:18:21.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:21.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.862249+0000", 2024-09-17T21:18:21.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003396807 2024-09-17T21:18:21.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:21.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.862383+0000", 2024-09-17T21:18:21.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000133585 2024-09-17T21:18:21.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:21.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.862614+0000", 2024-09-17T21:18:21.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023116799999999999 2024-09-17T21:18:21.292 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.292 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:21.292 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.292 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:293 3.1d 3:bcd54fb7:::rbd_data.1152c6af092e.000000000000001c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 853504~43520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.292 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.944520+0000", 2024-09-17T21:18:21.292 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.3034985150000002, 2024-09-17T21:18:21.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036295160000000002, 2024-09-17T21:18:21.292 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:21.292 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:21.293 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.293 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:21.293 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:21.293 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.293 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 293 2024-09-17T21:18:21.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.293 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:21.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:21.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.944520+0000", 2024-09-17T21:18:21.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:21.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.944520+0000", 2024-09-17T21:18:21.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:21.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.944521+0000", 2024-09-17T21:18:21.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1500000000000003e-07 2024-09-17T21:18:21.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:21.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.944646+0000", 2024-09-17T21:18:21.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124788 2024-09-17T21:18:21.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:21.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.944659+0000", 2024-09-17T21:18:21.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.273e-05 2024-09-17T21:18:21.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.944668+0000", 2024-09-17T21:18:21.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3190000000000001e-06 2024-09-17T21:18:21.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:21.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.946361+0000", 2024-09-17T21:18:21.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00169298 2024-09-17T21:18:21.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:21.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.946457+0000", 2024-09-17T21:18:21.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6272000000000003e-05 2024-09-17T21:18:21.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-17T21:18:21.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.946600+0000", 2024-09-17T21:18:21.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142948 2024-09-17T21:18:21.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:21.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.947463+0000", 2024-09-17T21:18:21.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00086271899999999996 2024-09-17T21:18:21.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:21.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.948081+0000", 2024-09-17T21:18:21.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00061789799999999997 2024-09-17T21:18:21.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:21.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.948123+0000", 2024-09-17T21:18:21.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2648999999999998e-05 2024-09-17T21:18:21.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:21.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.948150+0000", 2024-09-17T21:18:21.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6298000000000001e-05 2024-09-17T21:18:21.298 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.298 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:21.298 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.298 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:758 3.16 3:6aacae9d:::rbd_data.1152c6af092e.0000000000000039:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.298 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.203202+0000", 2024-09-17T21:18:21.298 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.0448168390000001, 2024-09-17T21:18:21.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041466790000000003, 2024-09-17T21:18:21.299 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:21.299 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:21.299 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:21.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:21.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.299 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 758 2024-09-17T21:18:21.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.299 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:21.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:21.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203202+0000", 2024-09-17T21:18:21.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:21.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203202+0000", 2024-09-17T21:18:21.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:21.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203204+0000", 2024-09-17T21:18:21.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7060000000000001e-06 2024-09-17T21:18:21.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:21.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203210+0000", 2024-09-17T21:18:21.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8000000000000001e-06 2024-09-17T21:18:21.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:21.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203232+0000", 2024-09-17T21:18:21.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1297999999999998e-05 2024-09-17T21:18:21.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203243+0000", 2024-09-17T21:18:21.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1396000000000001e-05 2024-09-17T21:18:21.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:21.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203313+0000", 2024-09-17T21:18:21.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0114e-05 2024-09-17T21:18:21.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:21.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203406+0000", 2024-09-17T21:18:21.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3153999999999996e-05 2024-09-17T21:18:21.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 3", 2024-09-17T21:18:21.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203539+0000", 2024-09-17T21:18:21.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000133053 2024-09-17T21:18:21.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:21.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.204432+0000", 2024-09-17T21:18:21.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00089268599999999996 2024-09-17T21:18:21.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:21.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.207240+0000", 2024-09-17T21:18:21.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028078000000000001 2024-09-17T21:18:21.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:21.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.207303+0000", 2024-09-17T21:18:21.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3501000000000001e-05 2024-09-17T21:18:21.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:21.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.207349+0000", 2024-09-17T21:18:21.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5170999999999998e-05 2024-09-17T21:18:21.304 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.304 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:21.304 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.304 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:770 3.e 3:71afa771:::rbd_data.1152c6af092e.00000000000000ac:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.304 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.203356+0000", 2024-09-17T21:18:21.304 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.0446629430000001, 2024-09-17T21:18:21.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0070554060000000002, 2024-09-17T21:18:21.304 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:21.304 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:21.304 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.304 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:21.304 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:21.305 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.305 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 770 2024-09-17T21:18:21.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.305 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:21.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:21.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203356+0000", 2024-09-17T21:18:21.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:21.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203356+0000", 2024-09-17T21:18:21.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:21.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203357+0000", 2024-09-17T21:18:21.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1000000000000001e-06 2024-09-17T21:18:21.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:21.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203362+0000", 2024-09-17T21:18:21.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.977e-06 2024-09-17T21:18:21.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:21.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203374+0000", 2024-09-17T21:18:21.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2296000000000001e-05 2024-09-17T21:18:21.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203383+0000", 2024-09-17T21:18:21.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3440000000000001e-06 2024-09-17T21:18:21.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:21.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203439+0000", 2024-09-17T21:18:21.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6096e-05 2024-09-17T21:18:21.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:21.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203521+0000", 2024-09-17T21:18:21.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1908000000000003e-05 2024-09-17T21:18:21.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-17T21:18:21.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.203634+0000", 2024-09-17T21:18:21.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011326 2024-09-17T21:18:21.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:21.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.207402+0000", 2024-09-17T21:18:21.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0037684049999999998 2024-09-17T21:18:21.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:21.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.210322+0000", 2024-09-17T21:18:21.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002919839 2024-09-17T21:18:21.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:21.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.210362+0000", 2024-09-17T21:18:21.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0243000000000001e-05 2024-09-17T21:18:21.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:21.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.210411+0000", 2024-09-17T21:18:21.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8937999999999998e-05 2024-09-17T21:18:21.309 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.309 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:21.310 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.310 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4423.0:761 3.2 e28/23 3:449b5906:::rbd_data.1152c6af092e.0000000000000046:head v 28'3, mlcod=28'1)", 2024-09-17T21:18:21.310 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.204347+0000", 2024-09-17T21:18:21.310 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.04367163, 2024-09-17T21:18:21.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057932399999999998, 2024-09-17T21:18:21.310 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:21.310 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:21.310 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.310 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:21.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:21.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.204347+0000", 2024-09-17T21:18:21.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:21.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.204347+0000", 2024-09-17T21:18:21.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:21.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.204348+0000", 2024-09-17T21:18:21.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1900000000000001e-07 2024-09-17T21:18:21.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:21.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.204355+0000", 2024-09-17T21:18:21.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4359999999999996e-06 2024-09-17T21:18:21.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:21.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.204368+0000", 2024-09-17T21:18:21.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3005e-05 2024-09-17T21:18:21.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.204377+0000", 2024-09-17T21:18:21.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5909999999999996e-06 2024-09-17T21:18:21.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:21.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.204465+0000", 2024-09-17T21:18:21.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7960000000000005e-05 2024-09-17T21:18:21.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:21.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.204486+0000", 2024-09-17T21:18:21.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1109999999999999e-05 2024-09-17T21:18:21.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:21.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.210089+0000", 2024-09-17T21:18:21.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0056028450000000004 2024-09-17T21:18:21.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:21.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.210140+0000", 2024-09-17T21:18:21.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1374e-05 2024-09-17T21:18:21.314 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.314 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:21.314 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.314 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:790 3.5 3:a14b1a37:::rbd_data.1152c6af092e.000000000000002f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.314 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.221998+0000", 2024-09-17T21:18:21.314 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.0260205689999999, 2024-09-17T21:18:21.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006953018, 2024-09-17T21:18:21.314 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:21.315 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:21.315 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.315 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:21.315 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:21.315 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.315 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 790 2024-09-17T21:18:21.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.315 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:21.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:21.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.221998+0000", 2024-09-17T21:18:21.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:21.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.221998+0000", 2024-09-17T21:18:21.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:21.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.222000+0000", 2024-09-17T21:18:21.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1210000000000001e-06 2024-09-17T21:18:21.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:21.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.222009+0000", 2024-09-17T21:18:21.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4309999999999994e-06 2024-09-17T21:18:21.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:21.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.222030+0000", 2024-09-17T21:18:21.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1645999999999999e-05 2024-09-17T21:18:21.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.222042+0000", 2024-09-17T21:18:21.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1273e-05 2024-09-17T21:18:21.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:21.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.222116+0000", 2024-09-17T21:18:21.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4187e-05 2024-09-17T21:18:21.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:21.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.222288+0000", 2024-09-17T21:18:21.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000171991 2024-09-17T21:18:21.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:21.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.222488+0000", 2024-09-17T21:18:21.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000199831 2024-09-17T21:18:21.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:21.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.223213+0000", 2024-09-17T21:18:21.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00072577500000000001 2024-09-17T21:18:21.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:21.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.228886+0000", 2024-09-17T21:18:21.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0056722949999999999 2024-09-17T21:18:21.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:21.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.228921+0000", 2024-09-17T21:18:21.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5707999999999998e-05 2024-09-17T21:18:21.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:21.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.228951+0000", 2024-09-17T21:18:21.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.976e-05 2024-09-17T21:18:21.323 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.323 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:21.323 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.323 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:850 3.5 3:a14b1a37:::rbd_data.1152c6af092e.000000000000002f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.323 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.257546+0000", 2024-09-17T21:18:21.323 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.9904727869999999, 2024-09-17T21:18:21.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0047133050000000001, 2024-09-17T21:18:21.324 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:21.324 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:21.324 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.324 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:21.324 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:21.324 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.324 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:18:21.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.324 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:21.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:21.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257546+0000", 2024-09-17T21:18:21.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:21.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257546+0000", 2024-09-17T21:18:21.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:21.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257547+0000", 2024-09-17T21:18:21.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6400000000000001e-07 2024-09-17T21:18:21.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:21.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257550+0000", 2024-09-17T21:18:21.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-06 2024-09-17T21:18:21.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:21.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257558+0000", 2024-09-17T21:18:21.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:18:21.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257563+0000", 2024-09-17T21:18:21.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2360000000000004e-06 2024-09-17T21:18:21.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:21.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257625+0000", 2024-09-17T21:18:21.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1919000000000001e-05 2024-09-17T21:18:21.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:21.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257711+0000", 2024-09-17T21:18:21.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5811999999999998e-05 2024-09-17T21:18:21.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:21.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257877+0000", 2024-09-17T21:18:21.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000166324 2024-09-17T21:18:21.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:21.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.259279+0000", 2024-09-17T21:18:21.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001402396 2024-09-17T21:18:21.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:21.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.262211+0000", 2024-09-17T21:18:21.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002931534 2024-09-17T21:18:21.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:21.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.262239+0000", 2024-09-17T21:18:21.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7708999999999998e-05 2024-09-17T21:18:21.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:21.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.262259+0000", 2024-09-17T21:18:21.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0687e-05 2024-09-17T21:18:21.329 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.329 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:21.329 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.329 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:858 3.1d 3:bab05a5d:::rbd_data.1152c6af092e.0000000000000034:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.329 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.257734+0000", 2024-09-17T21:18:21.329 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.990285166, 2024-09-17T21:18:21.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035786820000000001, 2024-09-17T21:18:21.330 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:21.330 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:21.330 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.330 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:21.330 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:21.330 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.330 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 858 2024-09-17T21:18:21.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.330 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:21.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:21.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257734+0000", 2024-09-17T21:18:21.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:21.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257734+0000", 2024-09-17T21:18:21.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:21.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257735+0000", 2024-09-17T21:18:21.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2550000000000001e-06 2024-09-17T21:18:21.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:21.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257740+0000", 2024-09-17T21:18:21.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2920000000000003e-06 2024-09-17T21:18:21.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:21.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257754+0000", 2024-09-17T21:18:21.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3875e-05 2024-09-17T21:18:21.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.257762+0000", 2024-09-17T21:18:21.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0749999999999998e-06 2024-09-17T21:18:21.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:21.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.258253+0000", 2024-09-17T21:18:21.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00049078499999999998 2024-09-17T21:18:21.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:21.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.258333+0000", 2024-09-17T21:18:21.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0325000000000001e-05 2024-09-17T21:18:21.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-17T21:18:21.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.258451+0000", 2024-09-17T21:18:21.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000118192 2024-09-17T21:18:21.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:21.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.261086+0000", 2024-09-17T21:18:21.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026350359999999999 2024-09-17T21:18:21.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:21.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.261261+0000", 2024-09-17T21:18:21.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000174115 2024-09-17T21:18:21.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:21.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.261290+0000", 2024-09-17T21:18:21.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9728000000000001e-05 2024-09-17T21:18:21.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:21.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.261312+0000", 2024-09-17T21:18:21.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2004000000000001e-05 2024-09-17T21:18:21.335 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.335 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:21.335 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.335 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:1529 3.c 3:33af988e:::rbd_data.1152c6af092e.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.335 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.596259+0000", 2024-09-17T21:18:21.335 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.6517597880000001, 2024-09-17T21:18:21.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035901280000000002, 2024-09-17T21:18:21.336 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:21.336 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:21.336 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:21.336 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:21.336 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:21.336 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.336 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1529 2024-09-17T21:18:21.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.336 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:21.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:21.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.596259+0000", 2024-09-17T21:18:21.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:21.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.596259+0000", 2024-09-17T21:18:21.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:21.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.596260+0000", 2024-09-17T21:18:21.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:18:21.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:21.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.596262+0000", 2024-09-17T21:18:21.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8679999999999999e-06 2024-09-17T21:18:21.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:21.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.596271+0000", 2024-09-17T21:18:21.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3569999999999998e-06 2024-09-17T21:18:21.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.596277+0000", 2024-09-17T21:18:21.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2349999999999996e-06 2024-09-17T21:18:21.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:21.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.599377+0000", 2024-09-17T21:18:21.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0030999980000000001 2024-09-17T21:18:21.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:21.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.599579+0000", 2024-09-17T21:18:21.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202105 2024-09-17T21:18:21.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:21.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.599849+0000", 2024-09-17T21:18:21.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026990800000000002 2024-09-17T21:18:21.339 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.340 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:21.340 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.340 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:1530 3.c 3:33af988e:::rbd_data.1152c6af092e.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.340 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.596286+0000", 2024-09-17T21:18:21.340 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.6517324540000002, 2024-09-17T21:18:21.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0039369180000000002, 2024-09-17T21:18:21.340 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:21.340 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:21.340 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:21.340 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:21.341 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:21.341 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.341 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1530 2024-09-17T21:18:21.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.341 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:21.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:21.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.596286+0000", 2024-09-17T21:18:21.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:21.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.596286+0000", 2024-09-17T21:18:21.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:21.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.596287+0000", 2024-09-17T21:18:21.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5400000000000001e-07 2024-09-17T21:18:21.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:21.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.596290+0000", 2024-09-17T21:18:21.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6510000000000001e-06 2024-09-17T21:18:21.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:21.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.596298+0000", 2024-09-17T21:18:21.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3659999999999992e-06 2024-09-17T21:18:21.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.596304+0000", 2024-09-17T21:18:21.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2709999999999998e-06 2024-09-17T21:18:21.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:21.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.599927+0000", 2024-09-17T21:18:21.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036230009999999998 2024-09-17T21:18:21.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:21.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.600057+0000", 2024-09-17T21:18:21.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013020600000000001 2024-09-17T21:18:21.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:21.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.600223+0000", 2024-09-17T21:18:21.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016576899999999999 2024-09-17T21:18:21.344 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.344 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:21.344 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.345 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:1538 3.c 3:33af988e:::rbd_data.1152c6af092e.0000000000000000:head [read 7680~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.345 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.601116+0000", 2024-09-17T21:18:21.345 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.6469029630000001, 2024-09-17T21:18:21.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0039221760000000003, 2024-09-17T21:18:21.345 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:21.345 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:21.345 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:21.345 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:21.345 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:21.345 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.345 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1538 2024-09-17T21:18:21.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.346 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:21.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:21.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.601116+0000", 2024-09-17T21:18:21.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:21.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.601116+0000", 2024-09-17T21:18:21.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:21.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.601116+0000", 2024-09-17T21:18:21.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4899999999999995e-07 2024-09-17T21:18:21.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:21.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.601119+0000", 2024-09-17T21:18:21.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.88e-06 2024-09-17T21:18:21.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:21.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.601128+0000", 2024-09-17T21:18:21.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5480000000000002e-06 2024-09-17T21:18:21.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.601135+0000", 2024-09-17T21:18:21.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6939999999999998e-06 2024-09-17T21:18:21.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:21.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.604556+0000", 2024-09-17T21:18:21.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0034214419999999998 2024-09-17T21:18:21.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:21.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.604769+0000", 2024-09-17T21:18:21.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000213122 2024-09-17T21:18:21.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:21.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.605038+0000", 2024-09-17T21:18:21.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026884100000000002 2024-09-17T21:18:21.349 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.349 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:21.349 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.349 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:1690 3.c 3:33af988e:::rbd_data.1152c6af092e.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.349 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.724388+0000", 2024-09-17T21:18:21.349 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.5236309649999997, 2024-09-17T21:18:21.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003559993, 2024-09-17T21:18:21.350 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:21.350 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:21.350 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:21.350 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:21.350 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:21.350 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.350 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1690 2024-09-17T21:18:21.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.350 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:21.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:21.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.724388+0000", 2024-09-17T21:18:21.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:21.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.724388+0000", 2024-09-17T21:18:21.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:21.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.724389+0000", 2024-09-17T21:18:21.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6000000000000003e-07 2024-09-17T21:18:21.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:21.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.724392+0000", 2024-09-17T21:18:21.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:18:21.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:21.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.724415+0000", 2024-09-17T21:18:21.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2917000000000001e-05 2024-09-17T21:18:21.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.724423+0000", 2024-09-17T21:18:21.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2029999999999992e-06 2024-09-17T21:18:21.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:21.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.727652+0000", 2024-09-17T21:18:21.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0032287050000000001 2024-09-17T21:18:21.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:21.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.727780+0000", 2024-09-17T21:18:21.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000128906 2024-09-17T21:18:21.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:21.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.727948+0000", 2024-09-17T21:18:21.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016733199999999999 2024-09-17T21:18:21.354 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.354 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:21.354 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.354 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:1691 3.c 3:3202476b:::rbd_data.1152c01721e5.0000000000000003:3 [read 2600960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.354 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.724774+0000", 2024-09-17T21:18:21.354 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.523245191, 2024-09-17T21:18:21.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003718828, 2024-09-17T21:18:21.354 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:21.354 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:21.355 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:21.355 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:21.355 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:21.355 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.355 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1691 2024-09-17T21:18:21.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.355 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:21.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:21.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.724774+0000", 2024-09-17T21:18:21.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:21.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.724774+0000", 2024-09-17T21:18:21.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:21.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.724775+0000", 2024-09-17T21:18:21.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.277e-06 2024-09-17T21:18:21.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:21.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.724780+0000", 2024-09-17T21:18:21.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3929999999999999e-06 2024-09-17T21:18:21.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:21.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.724795+0000", 2024-09-17T21:18:21.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4623e-05 2024-09-17T21:18:21.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.724804+0000", 2024-09-17T21:18:21.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4800000000000007e-06 2024-09-17T21:18:21.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:21.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.728058+0000", 2024-09-17T21:18:21.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0032539779999999998 2024-09-17T21:18:21.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:21.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.728280+0000", 2024-09-17T21:18:21.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022166800000000001 2024-09-17T21:18:21.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:21.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.728492+0000", 2024-09-17T21:18:21.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021240899999999999 2024-09-17T21:18:21.358 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.358 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:21.358 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.359 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:1692 3.c 3:3202476b:::rbd_data.1152c01721e5.0000000000000003:3 [read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.359 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.724821+0000", 2024-09-17T21:18:21.359 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.523198211, 2024-09-17T21:18:21.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0039700489999999998, 2024-09-17T21:18:21.359 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:21.359 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:21.359 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:21.359 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:21.359 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:21.359 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.359 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1692 2024-09-17T21:18:21.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.359 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:21.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:21.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.724821+0000", 2024-09-17T21:18:21.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:21.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.724821+0000", 2024-09-17T21:18:21.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:21.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:21.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.724821+0000", 2024-09-17T21:18:21.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5300000000000003e-07 2024-09-17T21:18:21.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:21.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.724825+0000", 2024-09-17T21:18:21.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3000000000000002e-06 2024-09-17T21:18:21.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:21.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.724835+0000", 2024-09-17T21:18:21.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0332999999999999e-05 2024-09-17T21:18:21.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.724842+0000", 2024-09-17T21:18:21.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5839999999999997e-06 2024-09-17T21:18:21.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:21.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.728566+0000", 2024-09-17T21:18:21.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003723381 2024-09-17T21:18:21.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:21.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.728679+0000", 2024-09-17T21:18:21.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011355500000000001 2024-09-17T21:18:21.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:21.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:21.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:21.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.728791+0000", 2024-09-17T21:18:21.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111143 2024-09-17T21:18:21.362 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.362 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:21.363 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.363 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:21.363 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:21.363 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:21.364 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:18:21.365 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:21.360+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:21.410 INFO:teuthology.orchestra.run.smithi184.stdout:229 read 0x9b75e00 thru 0x9b8190d (0xbb0e bytes) 2024-09-17T21:18:21.412 INFO:teuthology.orchestra.run.smithi184.stdout:232 punch from 0xa286800 to 0xa287c00, (0x1400 bytes) 2024-09-17T21:18:21.426 INFO:teuthology.orchestra.run.smithi184.stdout:234 read 0x7c30000 thru 0x7c3204f (0x2050 bytes) 2024-09-17T21:18:21.432 INFO:teuthology.orchestra.run.smithi184.stdout:235 trunc from 0xbb00400 to 0xcb28e00 2024-09-17T21:18:21.454 INFO:teuthology.orchestra.run.smithi184.stdout:236 read 0x4750200 thru 0x4753240 (0x3041 bytes) 2024-09-17T21:18:21.456 INFO:teuthology.orchestra.run.smithi184.stdout:237 write 0xb7e1800 thru 0xb7f04d8 (0xecd9 bytes) 2024-09-17T21:18:21.456 INFO:teuthology.orchestra.run.smithi184.stdout:238 punch from 0xc97f600 to 0xc982800, (0x3200 bytes) 2024-09-17T21:18:21.465 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:21.460+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:21.487 INFO:teuthology.orchestra.run.smithi184.stdout:239 trunc from 0xcb28e00 to 0x6a28a00 2024-09-17T21:18:21.505 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:21.505 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:21.505 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:21.505 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:21.525 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:18:21.550 INFO:teuthology.orchestra.run.smithi184.stdout:241 read 0x5cb6c00 thru 0x5cc3759 (0xcb5a bytes) 2024-09-17T21:18:21.553 INFO:teuthology.orchestra.run.smithi184.stdout:243 read 0x61f0400 thru 0x61f535c (0x4f5d bytes) 2024-09-17T21:18:21.566 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:21.562+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:21.587 INFO:teuthology.orchestra.run.smithi184.stdout:245 write 0xba76c00 thru 0xba807af (0x9bb0 bytes) 2024-09-17T21:18:21.589 INFO:teuthology.orchestra.run.smithi184.stdout:247 punch from 0x4e29c00 to 0x4e35000, (0xb400 bytes) 2024-09-17T21:18:21.590 INFO:teuthology.orchestra.run.smithi184.stdout:248 read 0xb555a00 thru 0xb5625a8 (0xcba9 bytes) 2024-09-17T21:18:21.591 INFO:teuthology.orchestra.run.smithi184.stdout:249 write 0x770a00 thru 0x772e8e (0x248f bytes) 2024-09-17T21:18:21.593 INFO:teuthology.orchestra.run.smithi184.stdout:250 read 0x4537e00 thru 0x45423a9 (0xa5aa bytes) 2024-09-17T21:18:21.593 INFO:teuthology.orchestra.run.smithi184.stdout:251 write 0x73e8c00 thru 0x73f2fcf (0xa3d0 bytes) 2024-09-17T21:18:21.594 INFO:teuthology.orchestra.run.smithi184.stdout:252 clone 3 order 22 su 524288 sc 3 2024-09-17T21:18:21.633 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:18:21.633 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:18:21.633 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-17T21:18:21.633 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T21:18:21.633 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T21:18:21.633 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-17T21:18:21.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:18:21.667 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:21.662+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:21.700 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:21.700 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:21.700 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:21.700 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:21.700 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:21.715 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:18:21.752 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:21.752 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T21:18:21.753 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:21.770 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:21.764+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:21.869 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:21.862+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:21.872 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:21.872 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:18:21.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:18:21.872 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:18:21.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.872 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'47, mlcod=16'40)", 2024-09-17T21:18:21.872 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.534075+0000", 2024-09-17T21:18:21.872 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 24.334726238999998, 2024-09-17T21:18:21.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0033556950000000001, 2024-09-17T21:18:21.873 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:21.873 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:21.873 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.873 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:21.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.873 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:21.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.534075+0000", 2024-09-17T21:18:21.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.874 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.874 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:21.874 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.534075+0000", 2024-09-17T21:18:21.874 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.874 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.874 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:21.874 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.534076+0000", 2024-09-17T21:18:21.874 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2500000000000005e-07 2024-09-17T21:18:21.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.874 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.874 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:21.875 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.534116+0000", 2024-09-17T21:18:21.875 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9876000000000002e-05 2024-09-17T21:18:21.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.875 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.875 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:21.875 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.534125+0000", 2024-09-17T21:18:21.875 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8030000000000003e-06 2024-09-17T21:18:21.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.875 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.875 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.875 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.534130+0000", 2024-09-17T21:18:21.875 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:18:21.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.876 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:21.876 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.534974+0000", 2024-09-17T21:18:21.876 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084409600000000002 2024-09-17T21:18:21.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.876 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:21.876 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535017+0000", 2024-09-17T21:18:21.876 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2735999999999999e-05 2024-09-17T21:18:21.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.876 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:21.876 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.537353+0000", 2024-09-17T21:18:21.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023363419999999999 2024-09-17T21:18:21.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:21.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.537431+0000", 2024-09-17T21:18:21.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7676999999999995e-05 2024-09-17T21:18:21.877 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.877 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:21.877 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.878 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:70 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'48, mlcod=16'42)", 2024-09-17T21:18:21.878 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.535353+0000", 2024-09-17T21:18:21.878 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 24.333448564000001, 2024-09-17T21:18:21.878 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035956500000000001, 2024-09-17T21:18:21.878 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:21.878 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:21.878 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.878 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:21.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.878 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:21.878 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535353+0000", 2024-09-17T21:18:21.878 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.879 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:21.879 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535353+0000", 2024-09-17T21:18:21.879 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.879 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:21.879 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535354+0000", 2024-09-17T21:18:21.879 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.375e-06 2024-09-17T21:18:21.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.879 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:21.880 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535362+0000", 2024-09-17T21:18:21.880 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3139999999999998e-06 2024-09-17T21:18:21.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.880 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:21.880 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535378+0000", 2024-09-17T21:18:21.880 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6189999999999999e-05 2024-09-17T21:18:21.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.880 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.880 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535387+0000", 2024-09-17T21:18:21.880 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9080000000000001e-06 2024-09-17T21:18:21.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:21.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.536329+0000", 2024-09-17T21:18:21.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00094202900000000004 2024-09-17T21:18:21.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:21.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.536356+0000", 2024-09-17T21:18:21.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6837000000000001e-05 2024-09-17T21:18:21.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:21.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.538911+0000", 2024-09-17T21:18:21.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002555599 2024-09-17T21:18:21.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.882 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:21.882 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.538949+0000", 2024-09-17T21:18:21.882 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7397999999999997e-05 2024-09-17T21:18:21.882 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.882 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:21.882 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.882 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:71 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'49, mlcod=16'42)", 2024-09-17T21:18:21.882 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.535870+0000", 2024-09-17T21:18:21.882 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 24.332931719000001, 2024-09-17T21:18:21.883 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0040377670000000003, 2024-09-17T21:18:21.883 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:21.883 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:21.883 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.883 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:21.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.883 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:21.883 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535870+0000", 2024-09-17T21:18:21.883 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.884 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:21.884 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535870+0000", 2024-09-17T21:18:21.884 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.884 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:21.884 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535871+0000", 2024-09-17T21:18:21.884 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6800000000000009e-07 2024-09-17T21:18:21.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.885 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:21.885 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535877+0000", 2024-09-17T21:18:21.885 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5320000000000003e-06 2024-09-17T21:18:21.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.885 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:21.885 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535888+0000", 2024-09-17T21:18:21.885 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1129e-05 2024-09-17T21:18:21.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.885 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.885 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.535896+0000", 2024-09-17T21:18:21.885 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1999999999999997e-06 2024-09-17T21:18:21.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.886 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:21.886 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.536717+0000", 2024-09-17T21:18:21.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00082162800000000003 2024-09-17T21:18:21.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.886 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:21.886 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.536761+0000", 2024-09-17T21:18:21.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3803000000000001e-05 2024-09-17T21:18:21.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.887 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:21.887 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539879+0000", 2024-09-17T21:18:21.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031181160000000002 2024-09-17T21:18:21.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.887 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:21.887 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539908+0000", 2024-09-17T21:18:21.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8391e-05 2024-09-17T21:18:21.887 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.887 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:21.887 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.888 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:72 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'50, mlcod=16'44)", 2024-09-17T21:18:21.888 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.536726+0000", 2024-09-17T21:18:21.888 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 24.332076025999999, 2024-09-17T21:18:21.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003215748, 2024-09-17T21:18:21.888 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:21.888 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:21.888 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.888 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:21.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:21.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.536726+0000", 2024-09-17T21:18:21.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:21.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.536726+0000", 2024-09-17T21:18:21.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:21.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.536727+0000", 2024-09-17T21:18:21.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7969999999999999e-06 2024-09-17T21:18:21.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:21.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.536842+0000", 2024-09-17T21:18:21.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114702 2024-09-17T21:18:21.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:21.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.536863+0000", 2024-09-17T21:18:21.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0530000000000002e-05 2024-09-17T21:18:21.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.536878+0000", 2024-09-17T21:18:21.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5834000000000001e-05 2024-09-17T21:18:21.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:21.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.537505+0000", 2024-09-17T21:18:21.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00062617499999999997 2024-09-17T21:18:21.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:21.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.537587+0000", 2024-09-17T21:18:21.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2212000000000005e-05 2024-09-17T21:18:21.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:21.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539914+0000", 2024-09-17T21:18:21.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023268130000000001 2024-09-17T21:18:21.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:21.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539941+0000", 2024-09-17T21:18:21.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7685e-05 2024-09-17T21:18:21.892 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.892 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:21.892 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.892 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4195.0:73 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'51, mlcod=16'44)", 2024-09-17T21:18:21.892 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:17:57.537690+0000", 2024-09-17T21:18:21.892 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 24.331111903, 2024-09-17T21:18:21.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003343015, 2024-09-17T21:18:21.892 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:21.892 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:21.892 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.893 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:21.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.893 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:21.893 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.537690+0000", 2024-09-17T21:18:21.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.893 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:21.893 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.537690+0000", 2024-09-17T21:18:21.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.893 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:21.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.537692+0000", 2024-09-17T21:18:21.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9829999999999999e-06 2024-09-17T21:18:21.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:21.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.537704+0000", 2024-09-17T21:18:21.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2584e-05 2024-09-17T21:18:21.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:21.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.537734+0000", 2024-09-17T21:18:21.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9886000000000002e-05 2024-09-17T21:18:21.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.537751+0000", 2024-09-17T21:18:21.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6699999999999999e-05 2024-09-17T21:18:21.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:21.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539015+0000", 2024-09-17T21:18:21.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0012645060000000001 2024-09-17T21:18:21.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:21.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.539051+0000", 2024-09-17T21:18:21.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5444000000000001e-05 2024-09-17T21:18:21.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:21.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.541002+0000", 2024-09-17T21:18:21.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019515540000000001 2024-09-17T21:18:21.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:21.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:17:57.541033+0000", 2024-09-17T21:18:21.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0358e-05 2024-09-17T21:18:21.897 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.897 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:21.897 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.897 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4423.0:358 3.6 3:65aa8b1b:::rbd_data.1152c6af092e.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 358912~53760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.897 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:17.978559+0000", 2024-09-17T21:18:21.897 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.8902427610000001, 2024-09-17T21:18:21.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0045407879999999996, 2024-09-17T21:18:21.897 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:21.897 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:21.897 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.897 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:21.898 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4423", 2024-09-17T21:18:21.898 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.898 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 358 2024-09-17T21:18:21.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.898 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:21.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:21.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.978559+0000", 2024-09-17T21:18:21.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:21.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.978559+0000", 2024-09-17T21:18:21.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:21.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.978566+0000", 2024-09-17T21:18:21.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0899999999999999e-06 2024-09-17T21:18:21.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:21.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.978763+0000", 2024-09-17T21:18:21.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019667900000000001 2024-09-17T21:18:21.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:21.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.978794+0000", 2024-09-17T21:18:21.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1578999999999997e-05 2024-09-17T21:18:21.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.978810+0000", 2024-09-17T21:18:21.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.571e-05 2024-09-17T21:18:21.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:21.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.978929+0000", 2024-09-17T21:18:21.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119129 2024-09-17T21:18:21.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:21.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.979149+0000", 2024-09-17T21:18:21.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021964200000000001 2024-09-17T21:18:21.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1", 2024-09-17T21:18:21.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.979485+0000", 2024-09-17T21:18:21.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00033633399999999998 2024-09-17T21:18:21.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:21.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.982591+0000", 2024-09-17T21:18:21.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003106279 2024-09-17T21:18:21.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:21.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.982908+0000", 2024-09-17T21:18:21.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000317134 2024-09-17T21:18:21.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:21.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.982987+0000", 2024-09-17T21:18:21.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8202999999999996e-05 2024-09-17T21:18:21.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:21.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.983100+0000", 2024-09-17T21:18:21.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113009 2024-09-17T21:18:21.902 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.902 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:21.903 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.903 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4423.0:366 3.18 3:19cb91e4:::rbd_data.1152c6af092e.00000000000000da:head [set-alloc-hint object_size 1048576 write_size 1048576,write 757248~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.903 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:17.978849+0000", 2024-09-17T21:18:21.903 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.8899520729999999, 2024-09-17T21:18:21.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042929969999999998, 2024-09-17T21:18:21.903 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:21.903 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:21.903 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.903 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:21.903 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4423", 2024-09-17T21:18:21.903 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.903 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 366 2024-09-17T21:18:21.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.904 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:21.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:21.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.978849+0000", 2024-09-17T21:18:21.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:21.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.978849+0000", 2024-09-17T21:18:21.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:21.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.978851+0000", 2024-09-17T21:18:21.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1400000000000001e-06 2024-09-17T21:18:21.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:21.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.978864+0000", 2024-09-17T21:18:21.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3509999999999999e-05 2024-09-17T21:18:21.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:21.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.978877+0000", 2024-09-17T21:18:21.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2876e-05 2024-09-17T21:18:21.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.978889+0000", 2024-09-17T21:18:21.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2374e-05 2024-09-17T21:18:21.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:21.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.978994+0000", 2024-09-17T21:18:21.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000104535 2024-09-17T21:18:21.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:21.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.979224+0000", 2024-09-17T21:18:21.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023056500000000001 2024-09-17T21:18:21.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:21.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.979423+0000", 2024-09-17T21:18:21.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019806199999999999 2024-09-17T21:18:21.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:21.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.981812+0000", 2024-09-17T21:18:21.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002389838 2024-09-17T21:18:21.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:21.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.982847+0000", 2024-09-17T21:18:21.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001034522 2024-09-17T21:18:21.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:21.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.982985+0000", 2024-09-17T21:18:21.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013797899999999999 2024-09-17T21:18:21.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:21.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.983142+0000", 2024-09-17T21:18:21.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000157596 2024-09-17T21:18:21.908 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.909 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:21.909 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.909 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4423.0:396 3.1b 3:d94e04f5:::rbd_data.1152c6af092e.0000000000000016:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 637952~12800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.909 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.001865+0000", 2024-09-17T21:18:21.909 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.866936071, 2024-09-17T21:18:21.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038767099999999998, 2024-09-17T21:18:21.909 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:21.909 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:21.909 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.909 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:21.909 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4423", 2024-09-17T21:18:21.909 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.910 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 396 2024-09-17T21:18:21.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.910 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:21.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:21.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.001865+0000", 2024-09-17T21:18:21.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:21.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.001865+0000", 2024-09-17T21:18:21.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:21.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.001868+0000", 2024-09-17T21:18:21.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2000000000000001e-06 2024-09-17T21:18:21.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:21.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.002107+0000", 2024-09-17T21:18:21.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023940100000000001 2024-09-17T21:18:21.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:21.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.002137+0000", 2024-09-17T21:18:21.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0303000000000001e-05 2024-09-17T21:18:21.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.002151+0000", 2024-09-17T21:18:21.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3164e-05 2024-09-17T21:18:21.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:21.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.002310+0000", 2024-09-17T21:18:21.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159825 2024-09-17T21:18:21.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:21.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.002620+0000", 2024-09-17T21:18:21.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00030929800000000001 2024-09-17T21:18:21.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:18:21.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.002960+0000", 2024-09-17T21:18:21.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000340787 2024-09-17T21:18:21.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:21.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.005227+0000", 2024-09-17T21:18:21.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002266583 2024-09-17T21:18:21.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:21.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.005589+0000", 2024-09-17T21:18:21.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00036183400000000001 2024-09-17T21:18:21.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:21.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.005681+0000", 2024-09-17T21:18:21.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1712000000000006e-05 2024-09-17T21:18:21.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:21.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.005742+0000", 2024-09-17T21:18:21.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1603e-05 2024-09-17T21:18:21.915 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.915 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:21.915 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.915 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4423.0:411 3.17 3:e80c86d5:::rbd_data.1152c6af092e.0000000000000061:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.915 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.012771+0000", 2024-09-17T21:18:21.915 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.8560306049999999, 2024-09-17T21:18:21.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0043258430000000002, 2024-09-17T21:18:21.915 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:21.915 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:21.915 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.915 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:21.915 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4423", 2024-09-17T21:18:21.916 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.916 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 411 2024-09-17T21:18:21.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.916 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:21.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:21.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.012771+0000", 2024-09-17T21:18:21.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:21.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.012771+0000", 2024-09-17T21:18:21.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:21.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.012773+0000", 2024-09-17T21:18:21.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.216e-06 2024-09-17T21:18:21.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:21.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.012783+0000", 2024-09-17T21:18:21.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5370000000000003e-06 2024-09-17T21:18:21.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:21.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.012814+0000", 2024-09-17T21:18:21.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1183000000000003e-05 2024-09-17T21:18:21.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.012829+0000", 2024-09-17T21:18:21.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.473e-05 2024-09-17T21:18:21.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:21.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.012961+0000", 2024-09-17T21:18:21.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000132199 2024-09-17T21:18:21.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:21.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.013102+0000", 2024-09-17T21:18:21.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014107200000000001 2024-09-17T21:18:21.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:18:21.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.013254+0000", 2024-09-17T21:18:21.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015185599999999999 2024-09-17T21:18:21.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:21.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.015647+0000", 2024-09-17T21:18:21.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023929110000000002 2024-09-17T21:18:21.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:21.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.016966+0000", 2024-09-17T21:18:21.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001319347 2024-09-17T21:18:21.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:21.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.017040+0000", 2024-09-17T21:18:21.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3715999999999997e-05 2024-09-17T21:18:21.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:21.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.017097+0000", 2024-09-17T21:18:21.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7076e-05 2024-09-17T21:18:21.921 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.921 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:21.921 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.921 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4423.0:432 3.18 3:183145ee:::rbd_data.1152c6af092e.0000000000000055:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.921 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.022698+0000", 2024-09-17T21:18:21.921 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.8461037149999999, 2024-09-17T21:18:21.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0037877240000000001, 2024-09-17T21:18:21.921 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:21.921 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:21.922 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:21.922 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:21.922 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4423", 2024-09-17T21:18:21.922 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.922 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 432 2024-09-17T21:18:21.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.922 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:21.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:21.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.022698+0000", 2024-09-17T21:18:21.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:21.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.022698+0000", 2024-09-17T21:18:21.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:21.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.022700+0000", 2024-09-17T21:18:21.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3800000000000001e-06 2024-09-17T21:18:21.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:21.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.022709+0000", 2024-09-17T21:18:21.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0059999999999998e-06 2024-09-17T21:18:21.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:21.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.022737+0000", 2024-09-17T21:18:21.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8127e-05 2024-09-17T21:18:21.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.022758+0000", 2024-09-17T21:18:21.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0489999999999999e-05 2024-09-17T21:18:21.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:21.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.022915+0000", 2024-09-17T21:18:21.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015739199999999999 2024-09-17T21:18:21.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:21.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.023174+0000", 2024-09-17T21:18:21.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025918099999999998 2024-09-17T21:18:21.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:21.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.026486+0000", 2024-09-17T21:18:21.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0033111479999999999 2024-09-17T21:18:21.925 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.925 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:21.925 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.926 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4423.0:615 3.1b 3:dc87e5a4:::rbd_data.1152c6af092e.00000000000000bb:head [set-alloc-hint object_size 1048576 write_size 1048576,write 898560~61952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.926 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.115104+0000", 2024-09-17T21:18:21.926 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.7536977509999998, 2024-09-17T21:18:21.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057109120000000003, 2024-09-17T21:18:21.926 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:21.926 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:21.926 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.926 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:21.926 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4423", 2024-09-17T21:18:21.926 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.926 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 615 2024-09-17T21:18:21.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.926 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:21.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:21.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115104+0000", 2024-09-17T21:18:21.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:21.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115104+0000", 2024-09-17T21:18:21.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:21.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115106+0000", 2024-09-17T21:18:21.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.616e-06 2024-09-17T21:18:21.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:21.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115163+0000", 2024-09-17T21:18:21.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6209999999999999e-05 2024-09-17T21:18:21.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:21.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115204+0000", 2024-09-17T21:18:21.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0949999999999999e-05 2024-09-17T21:18:21.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115220+0000", 2024-09-17T21:18:21.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6209999999999999e-05 2024-09-17T21:18:21.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:21.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115330+0000", 2024-09-17T21:18:21.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010982899999999999 2024-09-17T21:18:21.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:21.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115667+0000", 2024-09-17T21:18:21.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00033735299999999998 2024-09-17T21:18:21.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:18:21.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115983+0000", 2024-09-17T21:18:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031603499999999999 2024-09-17T21:18:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.119404+0000", 2024-09-17T21:18:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0034212930000000002 2024-09-17T21:18:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.120563+0000", 2024-09-17T21:18:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001158898 2024-09-17T21:18:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.120706+0000", 2024-09-17T21:18:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014283199999999999 2024-09-17T21:18:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.120815+0000", 2024-09-17T21:18:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108686 2024-09-17T21:18:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4423.0:768 3.1f 3:fc094fa7:::rbd_data.1152c6af092e.00000000000000a1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.202990+0000", 2024-09-17T21:18:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.6658116569999999, 2024-09-17T21:18:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0071941649999999998, 2024-09-17T21:18:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4423", 2024-09-17T21:18:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 768 2024-09-17T21:18:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.202990+0000", 2024-09-17T21:18:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.202990+0000", 2024-09-17T21:18:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.202992+0000", 2024-09-17T21:18:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1940000000000001e-06 2024-09-17T21:18:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.203002+0000", 2024-09-17T21:18:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0183000000000001e-05 2024-09-17T21:18:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.203029+0000", 2024-09-17T21:18:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6299999999999999e-05 2024-09-17T21:18:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.203044+0000", 2024-09-17T21:18:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5690000000000001e-05 2024-09-17T21:18:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.203182+0000", 2024-09-17T21:18:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000137305 2024-09-17T21:18:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.203305+0000", 2024-09-17T21:18:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000123299 2024-09-17T21:18:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:18:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.203453+0000", 2024-09-17T21:18:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000148462 2024-09-17T21:18:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.204907+0000", 2024-09-17T21:18:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014536519999999999 2024-09-17T21:18:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.209938+0000", 2024-09-17T21:18:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0050309059999999999 2024-09-17T21:18:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.210077+0000", 2024-09-17T21:18:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013953500000000001 2024-09-17T21:18:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.210184+0000", 2024-09-17T21:18:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010663899999999999 2024-09-17T21:18:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:791 3.b e28/23 3:d422d7f5:::rbd_data.1152c6af092e.0000000000000030:head v 28'6, mlcod=28'4)", 2024-09-17T21:18:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.222809+0000", 2024-09-17T21:18:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.6459925659999999, 2024-09-17T21:18:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0034905259999999999, 2024-09-17T21:18:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222809+0000", 2024-09-17T21:18:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222809+0000", 2024-09-17T21:18:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222811+0000", 2024-09-17T21:18:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:18:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222822+0000", 2024-09-17T21:18:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.094e-05 2024-09-17T21:18:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222847+0000", 2024-09-17T21:18:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5550000000000001e-05 2024-09-17T21:18:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222861+0000", 2024-09-17T21:18:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.363e-05 2024-09-17T21:18:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222983+0000", 2024-09-17T21:18:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000122442 2024-09-17T21:18:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223038+0000", 2024-09-17T21:18:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4342999999999998e-05 2024-09-17T21:18:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.226219+0000", 2024-09-17T21:18:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003181222 2024-09-17T21:18:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:21.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.226300+0000", 2024-09-17T21:18:21.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0659000000000001e-05 2024-09-17T21:18:21.942 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.942 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:21.942 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.942 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4423.0:856 3.6 3:65aa8b1b:::rbd_data.1152c6af092e.0000000000000033:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.942 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.257289+0000", 2024-09-17T21:18:21.942 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.611512082, 2024-09-17T21:18:21.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0050312209999999998, 2024-09-17T21:18:21.943 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:21.943 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:21.943 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.943 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:21.943 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4423", 2024-09-17T21:18:21.943 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.943 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 856 2024-09-17T21:18:21.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.943 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:21.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:21.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.257289+0000", 2024-09-17T21:18:21.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:21.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.257289+0000", 2024-09-17T21:18:21.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:21.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.257293+0000", 2024-09-17T21:18:21.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4630000000000001e-06 2024-09-17T21:18:21.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:21.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.257306+0000", 2024-09-17T21:18:21.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2763e-05 2024-09-17T21:18:21.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:21.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.257344+0000", 2024-09-17T21:18:21.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8112999999999999e-05 2024-09-17T21:18:21.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.257366+0000", 2024-09-17T21:18:21.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2552999999999999e-05 2024-09-17T21:18:21.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:21.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.257521+0000", 2024-09-17T21:18:21.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015429899999999999 2024-09-17T21:18:21.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:21.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.257739+0000", 2024-09-17T21:18:21.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021791200000000001 2024-09-17T21:18:21.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1", 2024-09-17T21:18:21.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.258025+0000", 2024-09-17T21:18:21.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028601700000000001 2024-09-17T21:18:21.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:21.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.262005+0000", 2024-09-17T21:18:21.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039805700000000001 2024-09-17T21:18:21.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:21.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.262101+0000", 2024-09-17T21:18:21.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5573000000000001e-05 2024-09-17T21:18:21.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:21.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.262209+0000", 2024-09-17T21:18:21.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010816500000000001 2024-09-17T21:18:21.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:21.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.262321+0000", 2024-09-17T21:18:21.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011179300000000001 2024-09-17T21:18:21.948 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.948 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:21.948 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.948 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:852 3.b e28/23 3:d422d7f5:::rbd_data.1152c6af092e.0000000000000030:head v 28'7, mlcod=28'5)", 2024-09-17T21:18:21.948 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.258575+0000", 2024-09-17T21:18:21.948 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.6102266090000001, 2024-09-17T21:18:21.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035427359999999999, 2024-09-17T21:18:21.948 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:21.949 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:21.949 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.949 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:21.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:21.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.258575+0000", 2024-09-17T21:18:21.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:21.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.258575+0000", 2024-09-17T21:18:21.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:21.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.258577+0000", 2024-09-17T21:18:21.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.137e-06 2024-09-17T21:18:21.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:21.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.258590+0000", 2024-09-17T21:18:21.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3332999999999999e-05 2024-09-17T21:18:21.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:21.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.258631+0000", 2024-09-17T21:18:21.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0800000000000002e-05 2024-09-17T21:18:21.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.258648+0000", 2024-09-17T21:18:21.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6603e-05 2024-09-17T21:18:21.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:21.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.258794+0000", 2024-09-17T21:18:21.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000146508 2024-09-17T21:18:21.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:21.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.258842+0000", 2024-09-17T21:18:21.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7320000000000001e-05 2024-09-17T21:18:21.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:21.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.262049+0000", 2024-09-17T21:18:21.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032075490000000001 2024-09-17T21:18:21.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:21.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.262118+0000", 2024-09-17T21:18:21.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8485999999999994e-05 2024-09-17T21:18:21.952 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.952 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4423.0:888 3.1b 3:d92b94f9:::rbd_data.1152c6af092e.0000000000000023:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.273664+0000", 2024-09-17T21:18:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.5951378859999998, 2024-09-17T21:18:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0033193049999999998, 2024-09-17T21:18:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4423", 2024-09-17T21:18:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.954 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 888 2024-09-17T21:18:21.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.954 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:21.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:21.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.273664+0000", 2024-09-17T21:18:21.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.273664+0000", 2024-09-17T21:18:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.273667+0000", 2024-09-17T21:18:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5769999999999998e-06 2024-09-17T21:18:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.273687+0000", 2024-09-17T21:18:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0205999999999999e-05 2024-09-17T21:18:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.273729+0000", 2024-09-17T21:18:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1600000000000002e-05 2024-09-17T21:18:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.273747+0000", 2024-09-17T21:18:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8099999999999999e-05 2024-09-17T21:18:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.273906+0000", 2024-09-17T21:18:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159152 2024-09-17T21:18:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.274244+0000", 2024-09-17T21:18:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338081 2024-09-17T21:18:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:18:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.274730+0000", 2024-09-17T21:18:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00048588599999999999 2024-09-17T21:18:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.276810+0000", 2024-09-17T21:18:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0020797239999999998 2024-09-17T21:18:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.276837+0000", 2024-09-17T21:18:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6910000000000002e-05 2024-09-17T21:18:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.276915+0000", 2024-09-17T21:18:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8108999999999994e-05 2024-09-17T21:18:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.276983+0000", 2024-09-17T21:18:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7960000000000007e-05 2024-09-17T21:18:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4423.0:1391 3.1f 3:fd7906ce:::rbd_data.1152c6af092e.00000000000000be:head [set-alloc-hint object_size 1048576 write_size 1048576,write 718336~38912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.513141+0000", 2024-09-17T21:18:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.3556605949999998, 2024-09-17T21:18:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003189669, 2024-09-17T21:18:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4423", 2024-09-17T21:18:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 1391 2024-09-17T21:18:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.513141+0000", 2024-09-17T21:18:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.513141+0000", 2024-09-17T21:18:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.513144+0000", 2024-09-17T21:18:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9560000000000002e-06 2024-09-17T21:18:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.513177+0000", 2024-09-17T21:18:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2598999999999997e-05 2024-09-17T21:18:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.513203+0000", 2024-09-17T21:18:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6842000000000002e-05 2024-09-17T21:18:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.513217+0000", 2024-09-17T21:18:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4054e-05 2024-09-17T21:18:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.513310+0000", 2024-09-17T21:18:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2633000000000006e-05 2024-09-17T21:18:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.513495+0000", 2024-09-17T21:18:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018485800000000001 2024-09-17T21:18:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:18:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.513687+0000", 2024-09-17T21:18:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000191931 2024-09-17T21:18:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.514870+0000", 2024-09-17T21:18:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011835319999999999 2024-09-17T21:18:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.516199+0000", 2024-09-17T21:18:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0013286890000000001 2024-09-17T21:18:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.516274+0000", 2024-09-17T21:18:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5109000000000002e-05 2024-09-17T21:18:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.516331+0000", 2024-09-17T21:18:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6465999999999998e-05 2024-09-17T21:18:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4423.0:1499 3.1 3:81876b9e:::rbd_data.1152c6af092e.00000000000000b3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.571152+0000", 2024-09-17T21:18:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.2976494619999999, 2024-09-17T21:18:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00360969, 2024-09-17T21:18:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4423", 2024-09-17T21:18:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 1499 2024-09-17T21:18:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.571152+0000", 2024-09-17T21:18:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.571152+0000", 2024-09-17T21:18:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.571154+0000", 2024-09-17T21:18:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2840000000000001e-06 2024-09-17T21:18:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.571161+0000", 2024-09-17T21:18:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3659999999999997e-06 2024-09-17T21:18:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.571180+0000", 2024-09-17T21:18:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9690999999999998e-05 2024-09-17T21:18:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.571192+0000", 2024-09-17T21:18:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1950000000000001e-05 2024-09-17T21:18:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.571266+0000", 2024-09-17T21:18:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4048000000000007e-05 2024-09-17T21:18:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.571351+0000", 2024-09-17T21:18:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4981000000000005e-05 2024-09-17T21:18:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:18:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.571451+0000", 2024-09-17T21:18:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9792999999999998e-05 2024-09-17T21:18:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.572129+0000", 2024-09-17T21:18:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00067818700000000002 2024-09-17T21:18:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.574661+0000", 2024-09-17T21:18:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025320099999999999 2024-09-17T21:18:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.574723+0000", 2024-09-17T21:18:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1748999999999994e-05 2024-09-17T21:18:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.574762+0000", 2024-09-17T21:18:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8631000000000003e-05 2024-09-17T21:18:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4447.0:210 3.17 3:e866433c:::rbd_data.1152772b7dab.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8388608~10240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T21:18:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:21.465796+0000", 2024-09-17T21:18:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.40300583899999998, 2024-09-17T21:18:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056553360000000004, 2024-09-17T21:18:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4447", 2024-09-17T21:18:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 210 2024-09-17T21:18:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.465796+0000", 2024-09-17T21:18:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.465796+0000", 2024-09-17T21:18:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.465804+0000", 2024-09-17T21:18:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3699999999999995e-06 2024-09-17T21:18:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.466058+0000", 2024-09-17T21:18:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025415099999999999 2024-09-17T21:18:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.466090+0000", 2024-09-17T21:18:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2249999999999998e-05 2024-09-17T21:18:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.466105+0000", 2024-09-17T21:18:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4980000000000001e-05 2024-09-17T21:18:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.466247+0000", 2024-09-17T21:18:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014153899999999999 2024-09-17T21:18:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.466477+0000", 2024-09-17T21:18:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000230301 2024-09-17T21:18:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:18:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.466825+0000", 2024-09-17T21:18:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034766799999999998 2024-09-17T21:18:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.470093+0000", 2024-09-17T21:18:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032684709999999998 2024-09-17T21:18:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.471219+0000", 2024-09-17T21:18:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011256149999999999 2024-09-17T21:18:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.471317+0000", 2024-09-17T21:18:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8065000000000004e-05 2024-09-17T21:18:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.471451+0000", 2024-09-17T21:18:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000133926 2024-09-17T21:18:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "pg_update_log_missing(3.c epoch 29/23 rep_tid 48 entries 29'15 (0'0) error 3:37a1d4ff:::rbd_header.1152772b7dab:head by client.4434.0:809 0.000000 -16 [r=-16+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 29'13)", 2024-09-17T21:18:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:21.489374+0000", 2024-09-17T21:18:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.37942774499999998, 2024-09-17T21:18:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035204870000000001, 2024-09-17T21:18:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:18:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.489374+0000", 2024-09-17T21:18:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.489374+0000", 2024-09-17T21:18:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.489376+0000", 2024-09-17T21:18:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7740000000000001e-06 2024-09-17T21:18:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.489394+0000", 2024-09-17T21:18:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8133000000000001e-05 2024-09-17T21:18:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.489441+0000", 2024-09-17T21:18:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7268999999999998e-05 2024-09-17T21:18:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.489466+0000", 2024-09-17T21:18:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5230000000000001e-05 2024-09-17T21:18:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.489680+0000", 2024-09-17T21:18:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021387199999999999 2024-09-17T21:18:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.492894+0000", 2024-09-17T21:18:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003214209 2024-09-17T21:18:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:21.980 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:21.982 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:18:21.983 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:21.978+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:22.083 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:22.082+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:22.110 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:22.110 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:22.111 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:22.111 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:22.125 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:18:22.184 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:22.182+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:22.240 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:22.241 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:22.241 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:22.241 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:22.241 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:22.255 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:18:22.285 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:22.278+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:22.378 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:22.378 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:18:22.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:18:22.378 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:18:22.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.378 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4423.0:125 3.0 3:0303f0ef:::rbd_data.1152c6af092e.00000000000000b7:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 862208~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:22.378 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:17.825884+0000", 2024-09-17T21:18:22.378 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.549927179, 2024-09-17T21:18:22.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0040912049999999997, 2024-09-17T21:18:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4423", 2024-09-17T21:18:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 125 2024-09-17T21:18:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.825884+0000", 2024-09-17T21:18:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.825884+0000", 2024-09-17T21:18:22.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.825887+0000", 2024-09-17T21:18:22.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5600000000000001e-06 2024-09-17T21:18:22.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.826030+0000", 2024-09-17T21:18:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014348200000000001 2024-09-17T21:18:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.826064+0000", 2024-09-17T21:18:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3330000000000001e-05 2024-09-17T21:18:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.826079+0000", 2024-09-17T21:18:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5366000000000002e-05 2024-09-17T21:18:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.826200+0000", 2024-09-17T21:18:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121313 2024-09-17T21:18:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.826411+0000", 2024-09-17T21:18:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021080200000000001 2024-09-17T21:18:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.826728+0000", 2024-09-17T21:18:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031652999999999998 2024-09-17T21:18:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.828960+0000", 2024-09-17T21:18:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022320199999999999 2024-09-17T21:18:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.829817+0000", 2024-09-17T21:18:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00085699300000000001 2024-09-17T21:18:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.829910+0000", 2024-09-17T21:18:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3308999999999997e-05 2024-09-17T21:18:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.829976+0000", 2024-09-17T21:18:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5500000000000006e-05 2024-09-17T21:18:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4423.0:195 3.b 3:d3b51548:::rbd_data.1152c6af092e.0000000000000018:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 331264~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:17.857623+0000", 2024-09-17T21:18:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.5181888709999996, 2024-09-17T21:18:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042017269999999997, 2024-09-17T21:18:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4423", 2024-09-17T21:18:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 195 2024-09-17T21:18:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.857623+0000", 2024-09-17T21:18:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.857623+0000", 2024-09-17T21:18:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.857633+0000", 2024-09-17T21:18:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0176999999999999e-05 2024-09-17T21:18:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.857693+0000", 2024-09-17T21:18:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0229000000000002e-05 2024-09-17T21:18:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.857725+0000", 2024-09-17T21:18:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2286999999999998e-05 2024-09-17T21:18:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.857740+0000", 2024-09-17T21:18:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4735999999999999e-05 2024-09-17T21:18:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.857860+0000", 2024-09-17T21:18:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119919 2024-09-17T21:18:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.858084+0000", 2024-09-17T21:18:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022427800000000001 2024-09-17T21:18:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.858436+0000", 2024-09-17T21:18:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000351477 2024-09-17T21:18:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.860358+0000", 2024-09-17T21:18:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001921929 2024-09-17T21:18:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.861529+0000", 2024-09-17T21:18:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001170768 2024-09-17T21:18:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.861698+0000", 2024-09-17T21:18:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016968500000000001 2024-09-17T21:18:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.861824+0000", 2024-09-17T21:18:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012624199999999999 2024-09-17T21:18:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4423.0:433 3.7 3:e3b59cb0:::rbd_data.1152c6af092e.0000000000000056:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.022868+0000", 2024-09-17T21:18:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.3529440729999997, 2024-09-17T21:18:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0043000199999999999, 2024-09-17T21:18:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4423", 2024-09-17T21:18:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 433 2024-09-17T21:18:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.022868+0000", 2024-09-17T21:18:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.022868+0000", 2024-09-17T21:18:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.022870+0000", 2024-09-17T21:18:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.323e-06 2024-09-17T21:18:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.022884+0000", 2024-09-17T21:18:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4303e-05 2024-09-17T21:18:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.022923+0000", 2024-09-17T21:18:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8417000000000002e-05 2024-09-17T21:18:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.022940+0000", 2024-09-17T21:18:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7093000000000001e-05 2024-09-17T21:18:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.023097+0000", 2024-09-17T21:18:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000157365 2024-09-17T21:18:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.023456+0000", 2024-09-17T21:18:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00035912500000000001 2024-09-17T21:18:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.027168+0000", 2024-09-17T21:18:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0037113939999999998 2024-09-17T21:18:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4423.0:616 3.4 3:26405e96:::rbd_data.1152c6af092e.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 927232~26112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.115639+0000", 2024-09-17T21:18:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.2601723460000001, 2024-09-17T21:18:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0054126479999999999, 2024-09-17T21:18:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4423", 2024-09-17T21:18:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 616 2024-09-17T21:18:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115639+0000", 2024-09-17T21:18:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115639+0000", 2024-09-17T21:18:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115642+0000", 2024-09-17T21:18:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:18:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115704+0000", 2024-09-17T21:18:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2086000000000002e-05 2024-09-17T21:18:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115736+0000", 2024-09-17T21:18:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2087e-05 2024-09-17T21:18:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115757+0000", 2024-09-17T21:18:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1416e-05 2024-09-17T21:18:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115874+0000", 2024-09-17T21:18:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000116426 2024-09-17T21:18:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.116110+0000", 2024-09-17T21:18:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000236145 2024-09-17T21:18:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.116501+0000", 2024-09-17T21:18:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039088700000000003 2024-09-17T21:18:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.120103+0000", 2024-09-17T21:18:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0036019189999999999 2024-09-17T21:18:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.120744+0000", 2024-09-17T21:18:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00064118499999999995 2024-09-17T21:18:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.120930+0000", 2024-09-17T21:18:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000186448 2024-09-17T21:18:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.121052+0000", 2024-09-17T21:18:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121432 2024-09-17T21:18:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4423.0:761 3.2 3:449b5906:::rbd_data.1152c6af092e.0000000000000046:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.202863+0000", 2024-09-17T21:18:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.1729483759999999, 2024-09-17T21:18:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075071690000000002, 2024-09-17T21:18:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4423", 2024-09-17T21:18:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 761 2024-09-17T21:18:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.202863+0000", 2024-09-17T21:18:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.202863+0000", 2024-09-17T21:18:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.202866+0000", 2024-09-17T21:18:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.39e-06 2024-09-17T21:18:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.202879+0000", 2024-09-17T21:18:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2977e-05 2024-09-17T21:18:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.202909+0000", 2024-09-17T21:18:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0667000000000003e-05 2024-09-17T21:18:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.202924+0000", 2024-09-17T21:18:22.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4888999999999999e-05 2024-09-17T21:18:22.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.203039+0000", 2024-09-17T21:18:22.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011533599999999999 2024-09-17T21:18:22.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.203278+0000", 2024-09-17T21:18:22.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000238958 2024-09-17T21:18:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:18:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.203612+0000", 2024-09-17T21:18:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000333381 2024-09-17T21:18:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.205275+0000", 2024-09-17T21:18:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001663 2024-09-17T21:18:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.210162+0000", 2024-09-17T21:18:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0048876900000000001 2024-09-17T21:18:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.210292+0000", 2024-09-17T21:18:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012907500000000001 2024-09-17T21:18:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.210370+0000", 2024-09-17T21:18:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8806000000000004e-05 2024-09-17T21:18:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4423.0:789 3.7 3:e5b7e4f7:::rbd_data.1152c6af092e.000000000000002e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 764416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.218896+0000", 2024-09-17T21:18:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.1569157729999997, 2024-09-17T21:18:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0053813020000000001, 2024-09-17T21:18:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4423", 2024-09-17T21:18:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 789 2024-09-17T21:18:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218896+0000", 2024-09-17T21:18:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218896+0000", 2024-09-17T21:18:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218898+0000", 2024-09-17T21:18:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4430000000000002e-06 2024-09-17T21:18:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218910+0000", 2024-09-17T21:18:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1270000000000001e-05 2024-09-17T21:18:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218940+0000", 2024-09-17T21:18:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:18:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218954+0000", 2024-09-17T21:18:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.434e-05 2024-09-17T21:18:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.219092+0000", 2024-09-17T21:18:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000138269 2024-09-17T21:18:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.219386+0000", 2024-09-17T21:18:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029309099999999999 2024-09-17T21:18:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.219688+0000", 2024-09-17T21:18:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00030272399999999998 2024-09-17T21:18:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222142+0000", 2024-09-17T21:18:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002453935 2024-09-17T21:18:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224052+0000", 2024-09-17T21:18:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001909532 2024-09-17T21:18:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224189+0000", 2024-09-17T21:18:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013693599999999999 2024-09-17T21:18:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224277+0000", 2024-09-17T21:18:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8451999999999999e-05 2024-09-17T21:18:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4423.0:791 3.b 3:d422d7f5:::rbd_data.1152c6af092e.0000000000000030:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.221664+0000", 2024-09-17T21:18:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.1541472739999996, 2024-09-17T21:18:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0050912509999999998, 2024-09-17T21:18:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4423", 2024-09-17T21:18:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 791 2024-09-17T21:18:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221664+0000", 2024-09-17T21:18:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221664+0000", 2024-09-17T21:18:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221667+0000", 2024-09-17T21:18:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7829999999999999e-06 2024-09-17T21:18:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221680+0000", 2024-09-17T21:18:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.31e-05 2024-09-17T21:18:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221712+0000", 2024-09-17T21:18:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1542999999999999e-05 2024-09-17T21:18:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221732+0000", 2024-09-17T21:18:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0080000000000001e-05 2024-09-17T21:18:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221857+0000", 2024-09-17T21:18:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012519199999999999 2024-09-17T21:18:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222226+0000", 2024-09-17T21:18:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00036861100000000003 2024-09-17T21:18:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222621+0000", 2024-09-17T21:18:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039495099999999999 2024-09-17T21:18:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223918+0000", 2024-09-17T21:18:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0012976330000000001 2024-09-17T21:18:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.226596+0000", 2024-09-17T21:18:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002677919 2024-09-17T21:18:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.226689+0000", 2024-09-17T21:18:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3171999999999995e-05 2024-09-17T21:18:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.226756+0000", 2024-09-17T21:18:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6266999999999994e-05 2024-09-17T21:18:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4423.0:852 3.b 3:d422d7f5:::rbd_data.1152c6af092e.0000000000000030:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.257311+0000", 2024-09-17T21:18:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.1185001320000003, 2024-09-17T21:18:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0054559109999999999, 2024-09-17T21:18:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4423", 2024-09-17T21:18:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 852 2024-09-17T21:18:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.257311+0000", 2024-09-17T21:18:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.257311+0000", 2024-09-17T21:18:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.257313+0000", 2024-09-17T21:18:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.223e-06 2024-09-17T21:18:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.257318+0000", 2024-09-17T21:18:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5500000000000002e-06 2024-09-17T21:18:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.257331+0000", 2024-09-17T21:18:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.328e-05 2024-09-17T21:18:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.257341+0000", 2024-09-17T21:18:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9429999999999995e-06 2024-09-17T21:18:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.257484+0000", 2024-09-17T21:18:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014226199999999999 2024-09-17T21:18:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.257686+0000", 2024-09-17T21:18:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000202105 2024-09-17T21:18:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.258043+0000", 2024-09-17T21:18:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00035698099999999998 2024-09-17T21:18:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.259944+0000", 2024-09-17T21:18:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001901088 2024-09-17T21:18:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.262457+0000", 2024-09-17T21:18:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002512818 2024-09-17T21:18:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.262605+0000", 2024-09-17T21:18:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014815499999999999 2024-09-17T21:18:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.262767+0000", 2024-09-17T21:18:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016250600000000001 2024-09-17T21:18:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4423.0:885 3.19 3:9a28128b:::rbd_data.1152c6af092e.000000000000001f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 794112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.269579+0000", 2024-09-17T21:18:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.1062329310000001, 2024-09-17T21:18:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0045356190000000003, 2024-09-17T21:18:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4423", 2024-09-17T21:18:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 885 2024-09-17T21:18:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269579+0000", 2024-09-17T21:18:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269579+0000", 2024-09-17T21:18:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269582+0000", 2024-09-17T21:18:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9869999999999999e-06 2024-09-17T21:18:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269597+0000", 2024-09-17T21:18:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5676999999999999e-05 2024-09-17T21:18:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269630+0000", 2024-09-17T21:18:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3036e-05 2024-09-17T21:18:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269652+0000", 2024-09-17T21:18:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1433999999999998e-05 2024-09-17T21:18:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269811+0000", 2024-09-17T21:18:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159421 2024-09-17T21:18:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.270127+0000", 2024-09-17T21:18:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031572499999999999 2024-09-17T21:18:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:18:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.270650+0000", 2024-09-17T21:18:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052335900000000004 2024-09-17T21:18:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.273694+0000", 2024-09-17T21:18:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030434559999999999 2024-09-17T21:18:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.273851+0000", 2024-09-17T21:18:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015721600000000001 2024-09-17T21:18:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.273990+0000", 2024-09-17T21:18:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000138725 2024-09-17T21:18:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.274114+0000", 2024-09-17T21:18:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000124583 2024-09-17T21:18:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4447.0:194 3.7 3:e276f830:::rbd_object_map.1152772b7dab:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T21:18:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:21.456911+0000", 2024-09-17T21:18:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.91890107600000004, 2024-09-17T21:18:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0048726339999999998, 2024-09-17T21:18:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4447", 2024-09-17T21:18:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 194 2024-09-17T21:18:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.456911+0000", 2024-09-17T21:18:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.456911+0000", 2024-09-17T21:18:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.456913+0000", 2024-09-17T21:18:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2749999999999998e-06 2024-09-17T21:18:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.456922+0000", 2024-09-17T21:18:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6099999999999995e-06 2024-09-17T21:18:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.456946+0000", 2024-09-17T21:18:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3802999999999999e-05 2024-09-17T21:18:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.456974+0000", 2024-09-17T21:18:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7767e-05 2024-09-17T21:18:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.457132+0000", 2024-09-17T21:18:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015799499999999999 2024-09-17T21:18:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.457350+0000", 2024-09-17T21:18:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000217655 2024-09-17T21:18:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.458103+0000", 2024-09-17T21:18:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00075332800000000005 2024-09-17T21:18:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.461400+0000", 2024-09-17T21:18:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032965910000000002 2024-09-17T21:18:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.461506+0000", 2024-09-17T21:18:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010637900000000001 2024-09-17T21:18:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.461666+0000", 2024-09-17T21:18:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015973199999999999 2024-09-17T21:18:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.461783+0000", 2024-09-17T21:18:22.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011749899999999999 2024-09-17T21:18:22.437 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.437 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.437 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.437 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4447.0:197 3.7 3:e276f830:::rbd_object_map.1152772b7dab:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T21:18:22.437 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:21.458302+0000", 2024-09-17T21:18:22.437 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.91751007500000004, 2024-09-17T21:18:22.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0071999280000000004, 2024-09-17T21:18:22.437 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.437 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.437 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.438 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:22.438 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4447", 2024-09-17T21:18:22.438 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:22.438 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 197 2024-09-17T21:18:22.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.438 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.458302+0000", 2024-09-17T21:18:22.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.458302+0000", 2024-09-17T21:18:22.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.458304+0000", 2024-09-17T21:18:22.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4660000000000002e-06 2024-09-17T21:18:22.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.458319+0000", 2024-09-17T21:18:22.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4613999999999999e-05 2024-09-17T21:18:22.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.458346+0000", 2024-09-17T21:18:22.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7229999999999998e-05 2024-09-17T21:18:22.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.458365+0000", 2024-09-17T21:18:22.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9533000000000001e-05 2024-09-17T21:18:22.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.459034+0000", 2024-09-17T21:18:22.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00066830500000000001 2024-09-17T21:18:22.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.441 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.459275+0000", 2024-09-17T21:18:22.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000241528 2024-09-17T21:18:22.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.441 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:22.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.460278+0000", 2024-09-17T21:18:22.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010031420000000001 2024-09-17T21:18:22.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.441 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:22.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.465015+0000", 2024-09-17T21:18:22.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047370470000000003 2024-09-17T21:18:22.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:22.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.465353+0000", 2024-09-17T21:18:22.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000337974 2024-09-17T21:18:22.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.465435+0000", 2024-09-17T21:18:22.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1559999999999998e-05 2024-09-17T21:18:22.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.465501+0000", 2024-09-17T21:18:22.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6528999999999999e-05 2024-09-17T21:18:22.443 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.443 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.443 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.443 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4447.0:206 3.b 3:d273c544:::rbd_data.1152772b7dab.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5117952~61440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T21:18:22.443 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:21.462045+0000", 2024-09-17T21:18:22.443 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.91376635299999998, 2024-09-17T21:18:22.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0046281270000000001, 2024-09-17T21:18:22.443 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.443 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.443 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.443 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:22.443 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4447", 2024-09-17T21:18:22.443 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:22.444 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 206 2024-09-17T21:18:22.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.444 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.462045+0000", 2024-09-17T21:18:22.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.462045+0000", 2024-09-17T21:18:22.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.462052+0000", 2024-09-17T21:18:22.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5590000000000002e-06 2024-09-17T21:18:22.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.462355+0000", 2024-09-17T21:18:22.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00030307400000000002 2024-09-17T21:18:22.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.462384+0000", 2024-09-17T21:18:22.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8767000000000001e-05 2024-09-17T21:18:22.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.462408+0000", 2024-09-17T21:18:22.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4657000000000001e-05 2024-09-17T21:18:22.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.462591+0000", 2024-09-17T21:18:22.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018253200000000001 2024-09-17T21:18:22.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.462957+0000", 2024-09-17T21:18:22.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00036637399999999999 2024-09-17T21:18:22.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:22.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.463503+0000", 2024-09-17T21:18:22.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00054555200000000004 2024-09-17T21:18:22.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:22.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.466136+0000", 2024-09-17T21:18:22.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0026330059999999998 2024-09-17T21:18:22.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:22.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.466523+0000", 2024-09-17T21:18:22.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00038752699999999998 2024-09-17T21:18:22.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.466607+0000", 2024-09-17T21:18:22.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3679999999999998e-05 2024-09-17T21:18:22.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.466673+0000", 2024-09-17T21:18:22.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6398999999999999e-05 2024-09-17T21:18:22.448 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.449 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.449 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.449 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4447.0:257 3.7 3:e276f830:::rbd_object_map.1152772b7dab:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T21:18:22.449 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:21.502273+0000", 2024-09-17T21:18:22.449 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.87353891100000003, 2024-09-17T21:18:22.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044688719999999996, 2024-09-17T21:18:22.449 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.449 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.449 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.449 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:22.449 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4447", 2024-09-17T21:18:22.449 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:22.450 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 257 2024-09-17T21:18:22.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.450 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.502273+0000", 2024-09-17T21:18:22.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.502273+0000", 2024-09-17T21:18:22.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.502275+0000", 2024-09-17T21:18:22.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:18:22.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.502285+0000", 2024-09-17T21:18:22.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.927e-06 2024-09-17T21:18:22.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.502311+0000", 2024-09-17T21:18:22.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.635e-05 2024-09-17T21:18:22.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.502325+0000", 2024-09-17T21:18:22.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3729e-05 2024-09-17T21:18:22.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.502398+0000", 2024-09-17T21:18:22.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2879999999999993e-05 2024-09-17T21:18:22.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.502488+0000", 2024-09-17T21:18:22.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0048000000000002e-05 2024-09-17T21:18:22.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:22.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.502851+0000", 2024-09-17T21:18:22.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00036292099999999999 2024-09-17T21:18:22.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:22.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.503926+0000", 2024-09-17T21:18:22.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001075552 2024-09-17T21:18:22.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:22.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.506587+0000", 2024-09-17T21:18:22.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00266089 2024-09-17T21:18:22.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.454 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.454 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.506675+0000", 2024-09-17T21:18:22.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7423000000000006e-05 2024-09-17T21:18:22.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.454 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.454 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.506742+0000", 2024-09-17T21:18:22.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6809000000000004e-05 2024-09-17T21:18:22.454 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.454 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.454 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4447.0:264 3.7 3:e276f830:::rbd_object_map.1152772b7dab:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T21:18:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:21.508630+0000", 2024-09-17T21:18:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.86718198700000004, 2024-09-17T21:18:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0053378039999999998, 2024-09-17T21:18:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4447", 2024-09-17T21:18:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 264 2024-09-17T21:18:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.508630+0000", 2024-09-17T21:18:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.508630+0000", 2024-09-17T21:18:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.508631+0000", 2024-09-17T21:18:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.68e-06 2024-09-17T21:18:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.508640+0000", 2024-09-17T21:18:22.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4440000000000008e-06 2024-09-17T21:18:22.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.508661+0000", 2024-09-17T21:18:22.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0809999999999999e-05 2024-09-17T21:18:22.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.508672+0000", 2024-09-17T21:18:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1695999999999999e-05 2024-09-17T21:18:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.508768+0000", 2024-09-17T21:18:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5608999999999999e-05 2024-09-17T21:18:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.508933+0000", 2024-09-17T21:18:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016559300000000001 2024-09-17T21:18:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.509442+0000", 2024-09-17T21:18:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00050810899999999997 2024-09-17T21:18:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.511082+0000", 2024-09-17T21:18:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00164065 2024-09-17T21:18:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.513852+0000", 2024-09-17T21:18:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0027700979999999999 2024-09-17T21:18:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.513914+0000", 2024-09-17T21:18:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2195999999999999e-05 2024-09-17T21:18:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.513967+0000", 2024-09-17T21:18:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2918999999999999e-05 2024-09-17T21:18:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4447.0:265 3.7 3:e276f830:::rbd_object_map.1152772b7dab:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T21:18:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:21.509751+0000", 2024-09-17T21:18:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.86606036200000003, 2024-09-17T21:18:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049041980000000002, 2024-09-17T21:18:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4447", 2024-09-17T21:18:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 265 2024-09-17T21:18:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.509751+0000", 2024-09-17T21:18:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.509751+0000", 2024-09-17T21:18:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.509753+0000", 2024-09-17T21:18:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4470000000000001e-06 2024-09-17T21:18:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.509759+0000", 2024-09-17T21:18:22.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0859999999999998e-06 2024-09-17T21:18:22.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.509773+0000", 2024-09-17T21:18:22.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.43e-05 2024-09-17T21:18:22.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.509782+0000", 2024-09-17T21:18:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5469999999999997e-06 2024-09-17T21:18:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.510201+0000", 2024-09-17T21:18:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000419737 2024-09-17T21:18:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.510288+0000", 2024-09-17T21:18:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6784000000000001e-05 2024-09-17T21:18:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.510683+0000", 2024-09-17T21:18:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039532800000000002 2024-09-17T21:18:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.513751+0000", 2024-09-17T21:18:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00306785 2024-09-17T21:18:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.514511+0000", 2024-09-17T21:18:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00075966100000000002 2024-09-17T21:18:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.514593+0000", 2024-09-17T21:18:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2366000000000004e-05 2024-09-17T21:18:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.514655+0000", 2024-09-17T21:18:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2092000000000001e-05 2024-09-17T21:18:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4447.0:269 3.7 3:e276f830:::rbd_object_map.1152772b7dab:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T21:18:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:21.518733+0000", 2024-09-17T21:18:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.85707820099999998, 2024-09-17T21:18:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039444930000000003, 2024-09-17T21:18:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4447", 2024-09-17T21:18:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 269 2024-09-17T21:18:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.518733+0000", 2024-09-17T21:18:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.518733+0000", 2024-09-17T21:18:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.518736+0000", 2024-09-17T21:18:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7429999999999998e-06 2024-09-17T21:18:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.518751+0000", 2024-09-17T21:18:22.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4613000000000001e-05 2024-09-17T21:18:22.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.518797+0000", 2024-09-17T21:18:22.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6493999999999999e-05 2024-09-17T21:18:22.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.518820+0000", 2024-09-17T21:18:22.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2592999999999999e-05 2024-09-17T21:18:22.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.518921+0000", 2024-09-17T21:18:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010165 2024-09-17T21:18:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.519008+0000", 2024-09-17T21:18:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6797000000000003e-05 2024-09-17T21:18:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.519380+0000", 2024-09-17T21:18:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00037206199999999999 2024-09-17T21:18:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.522289+0000", 2024-09-17T21:18:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0029087240000000001 2024-09-17T21:18:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.522404+0000", 2024-09-17T21:18:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114499 2024-09-17T21:18:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.522554+0000", 2024-09-17T21:18:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015081900000000001 2024-09-17T21:18:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.472 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.472 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.522678+0000", 2024-09-17T21:18:22.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000123499 2024-09-17T21:18:22.472 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.472 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.472 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.472 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4447.0:270 3.7 3:e276f830:::rbd_object_map.1152772b7dab:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T21:18:22.472 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:21.518890+0000", 2024-09-17T21:18:22.472 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.856921296, 2024-09-17T21:18:22.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.005299939, 2024-09-17T21:18:22.473 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.473 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.473 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.473 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:22.473 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4447", 2024-09-17T21:18:22.473 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:22.473 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 270 2024-09-17T21:18:22.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.473 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.473 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.473 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.518890+0000", 2024-09-17T21:18:22.473 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.518890+0000", 2024-09-17T21:18:22.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.518900+0000", 2024-09-17T21:18:22.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.234e-06 2024-09-17T21:18:22.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.518909+0000", 2024-09-17T21:18:22.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.516e-06 2024-09-17T21:18:22.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.518932+0000", 2024-09-17T21:18:22.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3030000000000001e-05 2024-09-17T21:18:22.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.518949+0000", 2024-09-17T21:18:22.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7203000000000001e-05 2024-09-17T21:18:22.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.519871+0000", 2024-09-17T21:18:22.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00092138300000000001 2024-09-17T21:18:22.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.476 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.476 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.520153+0000", 2024-09-17T21:18:22.476 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028263499999999999 2024-09-17T21:18:22.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.476 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:22.476 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.521205+0000", 2024-09-17T21:18:22.476 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010517809999999999 2024-09-17T21:18:22.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.476 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:22.476 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.523483+0000", 2024-09-17T21:18:22.476 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022778830000000001 2024-09-17T21:18:22.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.477 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:22.477 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.524014+0000", 2024-09-17T21:18:22.477 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00053131900000000004 2024-09-17T21:18:22.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.477 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.477 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.524100+0000", 2024-09-17T21:18:22.477 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5612999999999995e-05 2024-09-17T21:18:22.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.477 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.477 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.524190+0000", 2024-09-17T21:18:22.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0341999999999997e-05 2024-09-17T21:18:22.478 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.478 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.478 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.478 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4447.0:387 3.2 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 16423936~46080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T21:18:22.478 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:21.597931+0000", 2024-09-17T21:18:22.478 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.77788056299999997, 2024-09-17T21:18:22.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041500774999999997, 2024-09-17T21:18:22.478 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.478 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.478 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.478 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:22.479 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4447", 2024-09-17T21:18:22.479 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:22.479 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 387 2024-09-17T21:18:22.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.479 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.597931+0000", 2024-09-17T21:18:22.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.597931+0000", 2024-09-17T21:18:22.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.597963+0000", 2024-09-17T21:18:22.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1754999999999997e-05 2024-09-17T21:18:22.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.602945+0000", 2024-09-17T21:18:22.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049825319999999996 2024-09-17T21:18:22.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.602970+0000", 2024-09-17T21:18:22.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4956999999999998e-05 2024-09-17T21:18:22.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.481 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.602980+0000", 2024-09-17T21:18:22.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5899999999999997e-06 2024-09-17T21:18:22.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.481 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.603060+0000", 2024-09-17T21:18:22.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0412000000000002e-05 2024-09-17T21:18:22.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.482 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.482 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.603182+0000", 2024-09-17T21:18:22.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121302 2024-09-17T21:18:22.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.482 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:18:22.482 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.603419+0000", 2024-09-17T21:18:22.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023761499999999999 2024-09-17T21:18:22.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.482 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:22.482 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.611819+0000", 2024-09-17T21:18:22.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00839992 2024-09-17T21:18:22.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:22.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.639315+0000", 2024-09-17T21:18:22.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.027496181000000001 2024-09-17T21:18:22.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.639369+0000", 2024-09-17T21:18:22.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3909999999999997e-05 2024-09-17T21:18:22.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.639432+0000", 2024-09-17T21:18:22.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2601000000000006e-05 2024-09-17T21:18:22.484 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.484 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.484 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.484 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4447.0:393 3.7 3:e276f830:::rbd_object_map.1152772b7dab:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T21:18:22.484 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:21.603008+0000", 2024-09-17T21:18:22.484 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.77280337499999996, 2024-09-17T21:18:22.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004518818, 2024-09-17T21:18:22.484 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.484 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.484 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.484 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:22.485 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4447", 2024-09-17T21:18:22.485 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:22.485 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 393 2024-09-17T21:18:22.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.485 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.603008+0000", 2024-09-17T21:18:22.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.603008+0000", 2024-09-17T21:18:22.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.603010+0000", 2024-09-17T21:18:22.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6029999999999999e-06 2024-09-17T21:18:22.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.603014+0000", 2024-09-17T21:18:22.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.817e-06 2024-09-17T21:18:22.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.603021+0000", 2024-09-17T21:18:22.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1339999999999998e-06 2024-09-17T21:18:22.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.603025+0000", 2024-09-17T21:18:22.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3290000000000004e-06 2024-09-17T21:18:22.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.603104+0000", 2024-09-17T21:18:22.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8751999999999994e-05 2024-09-17T21:18:22.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.603225+0000", 2024-09-17T21:18:22.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012088599999999999 2024-09-17T21:18:22.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:22.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.603734+0000", 2024-09-17T21:18:22.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00050970099999999999 2024-09-17T21:18:22.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:22.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.605427+0000", 2024-09-17T21:18:22.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001692741 2024-09-17T21:18:22.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:22.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.607379+0000", 2024-09-17T21:18:22.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019515660000000001 2024-09-17T21:18:22.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.607465+0000", 2024-09-17T21:18:22.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6415000000000005e-05 2024-09-17T21:18:22.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.607527+0000", 2024-09-17T21:18:22.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1873999999999997e-05 2024-09-17T21:18:22.490 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.490 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.490 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.490 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4447.0:404 3.2 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 1509376~10240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T21:18:22.490 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:21.608323+0000", 2024-09-17T21:18:22.490 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.767488701, 2024-09-17T21:18:22.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.033032983000000002, 2024-09-17T21:18:22.490 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.490 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.490 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.490 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:22.490 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4447", 2024-09-17T21:18:22.491 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:22.491 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 404 2024-09-17T21:18:22.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.491 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.608323+0000", 2024-09-17T21:18:22.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.608323+0000", 2024-09-17T21:18:22.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.608325+0000", 2024-09-17T21:18:22.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4880000000000001e-06 2024-09-17T21:18:22.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.612116+0000", 2024-09-17T21:18:22.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0037909599999999999 2024-09-17T21:18:22.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.612148+0000", 2024-09-17T21:18:22.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1977999999999998e-05 2024-09-17T21:18:22.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.612160+0000", 2024-09-17T21:18:22.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1426999999999999e-05 2024-09-17T21:18:22.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.612228+0000", 2024-09-17T21:18:22.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8023999999999998e-05 2024-09-17T21:18:22.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.612394+0000", 2024-09-17T21:18:22.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016662299999999999 2024-09-17T21:18:22.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:18:22.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.612659+0000", 2024-09-17T21:18:22.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00026454899999999999 2024-09-17T21:18:22.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:22.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.621870+0000", 2024-09-17T21:18:22.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0092112429999999992 2024-09-17T21:18:22.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:22.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.641219+0000", 2024-09-17T21:18:22.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019349235999999999 2024-09-17T21:18:22.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.495 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.641297+0000", 2024-09-17T21:18:22.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7824999999999994e-05 2024-09-17T21:18:22.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.495 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.495 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.641356+0000", 2024-09-17T21:18:22.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8629999999999999e-05 2024-09-17T21:18:22.495 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.495 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.495 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.495 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.495 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.495 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:22.497 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:18:22.497 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:22.490+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:22.598 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:22.594+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:22.621 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-17T21:18:22.625 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:22.625 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:22.625 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:22.625 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:22.645 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:18:22.699 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:22.694+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:22.761 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:22.762 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:22.762 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:22.762 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:22.762 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:22.777 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:18:22.799 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:22.794+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:22.808 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone2 from 0xba80800 (overlap 0x6a28a00) to 0x4a4fa00 2024-09-17T21:18:22.901 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:22.894+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:22.920 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:22.920 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:18:22.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:18:22.920 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:18:22.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.921 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4190.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-17T21:18:22.921 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:01.050597+0000", 2024-09-17T21:18:22.921 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 21.867465955, 2024-09-17T21:18:22.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042911440000000002, 2024-09-17T21:18:22.921 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.921 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.921 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.921 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050597+0000", 2024-09-17T21:18:22.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050597+0000", 2024-09-17T21:18:22.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050597+0000", 2024-09-17T21:18:22.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6599999999999996e-07 2024-09-17T21:18:22.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050604+0000", 2024-09-17T21:18:22.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0400000000000004e-06 2024-09-17T21:18:22.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050623+0000", 2024-09-17T21:18:22.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8691000000000001e-05 2024-09-17T21:18:22.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050639+0000", 2024-09-17T21:18:22.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5719999999999999e-05 2024-09-17T21:18:22.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050709+0000", 2024-09-17T21:18:22.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9898000000000001e-05 2024-09-17T21:18:22.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050738+0000", 2024-09-17T21:18:22.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9023e-05 2024-09-17T21:18:22.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.054823+0000", 2024-09-17T21:18:22.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0040854940000000003 2024-09-17T21:18:22.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.054888+0000", 2024-09-17T21:18:22.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4712e-05 2024-09-17T21:18:22.926 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.926 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.926 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.926 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:12 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:22.926 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:22.926 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.5857282730000009, 2024-09-17T21:18:22.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061340989999999996, 2024-09-17T21:18:22.926 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.926 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.927 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.927 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:22.927 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:22.927 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:22.927 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12 2024-09-17T21:18:22.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.927 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:22.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:22.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332336+0000", 2024-09-17T21:18:22.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5519999999999999e-06 2024-09-17T21:18:22.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332342+0000", 2024-09-17T21:18:22.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7180000000000001e-06 2024-09-17T21:18:22.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332352+0000", 2024-09-17T21:18:22.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:18:22.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332359+0000", 2024-09-17T21:18:22.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3479999999999998e-06 2024-09-17T21:18:22.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332413+0000", 2024-09-17T21:18:22.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4447999999999998e-05 2024-09-17T21:18:22.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-17T21:18:22.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332417+0000", 2024-09-17T21:18:22.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.895e-06 2024-09-17T21:18:22.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334755+0000", 2024-09-17T21:18:22.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023377189999999998 2024-09-17T21:18:22.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334903+0000", 2024-09-17T21:18:22.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014778999999999999 2024-09-17T21:18:22.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:18:22.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.335187+0000", 2024-09-17T21:18:22.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000284715 2024-09-17T21:18:22.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:22.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.337731+0000", 2024-09-17T21:18:22.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002543741 2024-09-17T21:18:22.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:22.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338382+0000", 2024-09-17T21:18:22.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00065029200000000001 2024-09-17T21:18:22.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338418+0000", 2024-09-17T21:18:22.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.697e-05 2024-09-17T21:18:22.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338469+0000", 2024-09-17T21:18:22.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0154999999999997e-05 2024-09-17T21:18:22.934 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.934 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.934 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.934 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "pg_update_log_missing(3.1e epoch 28/27 rep_tid 81 entries 28'41 (0'0) error 3:7b23fd98:::rbd_header.1152c01721e5:head by client.4488.0:561 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-17T21:18:22.934 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:18:22.934 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.6364541460000002, 2024-09-17T21:18:22.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0051146969999999996, 2024-09-17T21:18:22.935 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.935 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.935 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:18:22.935 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:18:22.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:18:22.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281610+0000", 2024-09-17T21:18:22.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.16e-07 2024-09-17T21:18:22.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281626+0000", 2024-09-17T21:18:22.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6039999999999999e-05 2024-09-17T21:18:22.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281666+0000", 2024-09-17T21:18:22.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0364000000000002e-05 2024-09-17T21:18:22.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281710+0000", 2024-09-17T21:18:22.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3596000000000001e-05 2024-09-17T21:18:22.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281992+0000", 2024-09-17T21:18:22.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028273099999999999 2024-09-17T21:18:22.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286723+0000", 2024-09-17T21:18:22.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047310499999999997 2024-09-17T21:18:22.938 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.938 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.938 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.939 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4488.0:562 3.1e e28/27 3:7b23fd98:::rbd_header.1152c01721e5:head v 28'42, mlcod=0'0)", 2024-09-17T21:18:22.939 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:18:22.939 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.6362103579999996, 2024-09-17T21:18:22.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049275020000000003, 2024-09-17T21:18:22.939 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.939 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.939 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.939 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:18:22.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:18:22.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281853+0000", 2024-09-17T21:18:22.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2999999999999999e-07 2024-09-17T21:18:22.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281875+0000", 2024-09-17T21:18:22.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1386000000000002e-05 2024-09-17T21:18:22.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281892+0000", 2024-09-17T21:18:22.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7637000000000001e-05 2024-09-17T21:18:22.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281908+0000", 2024-09-17T21:18:22.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5187e-05 2024-09-17T21:18:22.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.282517+0000", 2024-09-17T21:18:22.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00060989799999999999 2024-09-17T21:18:22.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.282548+0000", 2024-09-17T21:18:22.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0910000000000001e-05 2024-09-17T21:18:22.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286729+0000", 2024-09-17T21:18:22.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0041810850000000002 2024-09-17T21:18:22.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286780+0000", 2024-09-17T21:18:22.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0569000000000003e-05 2024-09-17T21:18:22.944 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.944 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.944 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.944 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:609 3.d e28/23 3:b7278be9:::rbd_data.1152c6af092e.000000000000008f:head v 28'6, mlcod=28'4)", 2024-09-17T21:18:22.944 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:18:22.944 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.8032660710000004, 2024-09-17T21:18:22.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049797280000000001, 2024-09-17T21:18:22.944 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.944 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.944 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.944 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:18:22.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:18:22.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114800+0000", 2024-09-17T21:18:22.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:18:22.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114915+0000", 2024-09-17T21:18:22.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115053 2024-09-17T21:18:22.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114951+0000", 2024-09-17T21:18:22.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6165999999999999e-05 2024-09-17T21:18:22.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114965+0000", 2024-09-17T21:18:22.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4364e-05 2024-09-17T21:18:22.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115097+0000", 2024-09-17T21:18:22.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013139200000000001 2024-09-17T21:18:22.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115151+0000", 2024-09-17T21:18:22.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4422000000000002e-05 2024-09-17T21:18:22.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.119633+0000", 2024-09-17T21:18:22.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044818820000000004 2024-09-17T21:18:22.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.119776+0000", 2024-09-17T21:18:22.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000143609 2024-09-17T21:18:22.949 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.949 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.949 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.950 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:781 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'62, mlcod=28'60)", 2024-09-17T21:18:22.950 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.213959+0000", 2024-09-17T21:18:22.950 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.7041042730000004, 2024-09-17T21:18:22.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0043389400000000003, 2024-09-17T21:18:22.950 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.950 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.950 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.950 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.213959+0000", 2024-09-17T21:18:22.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.213959+0000", 2024-09-17T21:18:22.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.213960+0000", 2024-09-17T21:18:22.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:18:22.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.213969+0000", 2024-09-17T21:18:22.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1300000000000007e-06 2024-09-17T21:18:22.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.213991+0000", 2024-09-17T21:18:22.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2180000000000001e-05 2024-09-17T21:18:22.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214007+0000", 2024-09-17T21:18:22.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5722999999999998e-05 2024-09-17T21:18:22.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214141+0000", 2024-09-17T21:18:22.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013425899999999999 2024-09-17T21:18:22.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214201+0000", 2024-09-17T21:18:22.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0223000000000003e-05 2024-09-17T21:18:22.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218231+0000", 2024-09-17T21:18:22.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0040299719999999997 2024-09-17T21:18:22.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218297+0000", 2024-09-17T21:18:22.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6073000000000002e-05 2024-09-17T21:18:22.955 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.955 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.955 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.955 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:782 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'63, mlcod=28'60)", 2024-09-17T21:18:22.955 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:18:22.955 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.7032980430000002, 2024-09-17T21:18:22.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006167283, 2024-09-17T21:18:22.955 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.956 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.956 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.956 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:18:22.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:18:22.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214767+0000", 2024-09-17T21:18:22.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:18:22.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214779+0000", 2024-09-17T21:18:22.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2627e-05 2024-09-17T21:18:22.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214805+0000", 2024-09-17T21:18:22.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5570000000000001e-05 2024-09-17T21:18:22.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214823+0000", 2024-09-17T21:18:22.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8202999999999998e-05 2024-09-17T21:18:22.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215170+0000", 2024-09-17T21:18:22.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034720399999999999 2024-09-17T21:18:22.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215235+0000", 2024-09-17T21:18:22.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4276000000000006e-05 2024-09-17T21:18:22.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220859+0000", 2024-09-17T21:18:22.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056240500000000002 2024-09-17T21:18:22.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220932+0000", 2024-09-17T21:18:22.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3503000000000004e-05 2024-09-17T21:18:22.962 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.962 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.962 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.962 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:783 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'64, mlcod=28'60)", 2024-09-17T21:18:22.962 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:18:22.962 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.7026112009999999, 2024-09-17T21:18:22.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0055240869999999996, 2024-09-17T21:18:22.963 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.963 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.963 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.963 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:18:22.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:18:22.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215454+0000", 2024-09-17T21:18:22.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0269999999999998e-06 2024-09-17T21:18:22.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215468+0000", 2024-09-17T21:18:22.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4816e-05 2024-09-17T21:18:22.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215501+0000", 2024-09-17T21:18:22.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2957000000000003e-05 2024-09-17T21:18:22.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215524+0000", 2024-09-17T21:18:22.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2782999999999999e-05 2024-09-17T21:18:22.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216675+0000", 2024-09-17T21:18:22.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011507869999999999 2024-09-17T21:18:22.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216728+0000", 2024-09-17T21:18:22.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3112999999999998e-05 2024-09-17T21:18:22.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220938+0000", 2024-09-17T21:18:22.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042097380000000002 2024-09-17T21:18:22.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220976+0000", 2024-09-17T21:18:22.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7866e-05 2024-09-17T21:18:22.968 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.968 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.968 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.968 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:785 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'65, mlcod=28'60)", 2024-09-17T21:18:22.968 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:18:22.968 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.702025152, 2024-09-17T21:18:22.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049795409999999997, 2024-09-17T21:18:22.968 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.968 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.969 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.969 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:18:22.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:18:22.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216039+0000", 2024-09-17T21:18:22.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.793e-06 2024-09-17T21:18:22.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216053+0000", 2024-09-17T21:18:22.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.345e-05 2024-09-17T21:18:22.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216092+0000", 2024-09-17T21:18:22.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9180000000000001e-05 2024-09-17T21:18:22.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216106+0000", 2024-09-17T21:18:22.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:18:22.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217569+0000", 2024-09-17T21:18:22.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001462938 2024-09-17T21:18:22.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217607+0000", 2024-09-17T21:18:22.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7267000000000001e-05 2024-09-17T21:18:22.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220980+0000", 2024-09-17T21:18:22.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003373544 2024-09-17T21:18:22.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221017+0000", 2024-09-17T21:18:22.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7116000000000002e-05 2024-09-17T21:18:22.973 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.973 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.973 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.973 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:786 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'66, mlcod=28'60)", 2024-09-17T21:18:22.973 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:18:22.974 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.701324348, 2024-09-17T21:18:22.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057966650000000003, 2024-09-17T21:18:22.974 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.974 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.974 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.974 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:18:22.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:18:22.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216740+0000", 2024-09-17T21:18:22.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5600000000000001e-06 2024-09-17T21:18:22.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216752+0000", 2024-09-17T21:18:22.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1477e-05 2024-09-17T21:18:22.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216779+0000", 2024-09-17T21:18:22.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7093e-05 2024-09-17T21:18:22.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216817+0000", 2024-09-17T21:18:22.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8275999999999997e-05 2024-09-17T21:18:22.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218387+0000", 2024-09-17T21:18:22.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015703749999999999 2024-09-17T21:18:22.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218432+0000", 2024-09-17T21:18:22.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4910000000000002e-05 2024-09-17T21:18:22.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222432+0000", 2024-09-17T21:18:22.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039997189999999997 2024-09-17T21:18:22.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222535+0000", 2024-09-17T21:18:22.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103255 2024-09-17T21:18:22.978 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.978 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.979 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.979 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:18:22.979 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:18:22.979 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.7007025159999998, 2024-09-17T21:18:22.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:18:22.979 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.979 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.979 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.979 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:18:22.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:18:22.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:18:22.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:18:22.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:18:22.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:18:22.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:18:22.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:18:22.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:18:22.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:18:22.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:18:22.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:18:22.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:18:22.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:18:22.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:18:22.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:18:22.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:18:22.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:18:22.984 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.984 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.984 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.985 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:788 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'68, mlcod=28'60)", 2024-09-17T21:18:22.985 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:18:22.985 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.7000680839999998, 2024-09-17T21:18:22.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058495320000000002, 2024-09-17T21:18:22.985 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.985 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.985 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.985 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:18:22.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:18:22.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217996+0000", 2024-09-17T21:18:22.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.376e-06 2024-09-17T21:18:22.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218006+0000", 2024-09-17T21:18:22.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.01e-05 2024-09-17T21:18:22.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218030+0000", 2024-09-17T21:18:22.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4026999999999999e-05 2024-09-17T21:18:22.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218042+0000", 2024-09-17T21:18:22.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1977000000000001e-05 2024-09-17T21:18:22.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221081+0000", 2024-09-17T21:18:22.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030385960000000002 2024-09-17T21:18:22.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221115+0000", 2024-09-17T21:18:22.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3772999999999997e-05 2024-09-17T21:18:22.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223749+0000", 2024-09-17T21:18:22.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0026340629999999999 2024-09-17T21:18:22.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223844+0000", 2024-09-17T21:18:22.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5619999999999996e-05 2024-09-17T21:18:22.989 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.989 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.990 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.990 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:790 3.5 e28/23 3:a14b1a37:::rbd_data.1152c6af092e.000000000000002f:head v 28'1, mlcod=0'0)", 2024-09-17T21:18:22.990 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:18:22.990 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.6956637580000002, 2024-09-17T21:18:22.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059622240000000003, 2024-09-17T21:18:22.990 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.990 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.990 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.990 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:18:22.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:18:22.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222400+0000", 2024-09-17T21:18:22.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.02e-06 2024-09-17T21:18:22.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222411+0000", 2024-09-17T21:18:22.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.044e-05 2024-09-17T21:18:22.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222440+0000", 2024-09-17T21:18:22.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.932e-05 2024-09-17T21:18:22.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222458+0000", 2024-09-17T21:18:22.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8615999999999999e-05 2024-09-17T21:18:22.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222609+0000", 2024-09-17T21:18:22.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015102299999999999 2024-09-17T21:18:22.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.994 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.994 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222657+0000", 2024-09-17T21:18:22.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7206000000000002e-05 2024-09-17T21:18:22.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.994 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.994 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228278+0000", 2024-09-17T21:18:22.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056208100000000004 2024-09-17T21:18:22.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:22.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228361+0000", 2024-09-17T21:18:22.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3788999999999993e-05 2024-09-17T21:18:22.995 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.995 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:22.995 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:22.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.995 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:793 3.1e e28/27 3:793c7d73:::rbd_data.1152c6af092e.0000000000000032:head v 28'48, mlcod=28'46)", 2024-09-17T21:18:22.995 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.224161+0000", 2024-09-17T21:18:22.995 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.6939023049999999, 2024-09-17T21:18:22.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004730723, 2024-09-17T21:18:22.996 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:22.996 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:22.996 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:22.996 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:22.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:22.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224161+0000", 2024-09-17T21:18:22.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:22.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224161+0000", 2024-09-17T21:18:22.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:22.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:22.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224163+0000", 2024-09-17T21:18:22.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.244e-06 2024-09-17T21:18:22.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:22.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224178+0000", 2024-09-17T21:18:22.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:18:22.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:22.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224210+0000", 2024-09-17T21:18:22.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1152999999999998e-05 2024-09-17T21:18:22.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:22.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224227+0000", 2024-09-17T21:18:22.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7863000000000002e-05 2024-09-17T21:18:22.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:22.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224347+0000", 2024-09-17T21:18:22.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119376 2024-09-17T21:18:22.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:22.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224385+0000", 2024-09-17T21:18:22.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8408999999999998e-05 2024-09-17T21:18:22.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:22.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:22.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:22.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228842+0000", 2024-09-17T21:18:23.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044566859999999996 2024-09-17T21:18:23.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:23.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228891+0000", 2024-09-17T21:18:23.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9292000000000002e-05 2024-09-17T21:18:23.000 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:23.000 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:23.000 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:23.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.000 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:884 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'76, mlcod=28'70)", 2024-09-17T21:18:23.001 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:18:23.001 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.649078877, 2024-09-17T21:18:23.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056451990000000001, 2024-09-17T21:18:23.001 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:23.001 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:23.001 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:23.001 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:23.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:23.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:18:23.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:23.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:23.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:18:23.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:23.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:23.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268986+0000", 2024-09-17T21:18:23.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9360000000000002e-06 2024-09-17T21:18:23.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:23.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269001+0000", 2024-09-17T21:18:23.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4684e-05 2024-09-17T21:18:23.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:23.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269026+0000", 2024-09-17T21:18:23.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5193000000000001e-05 2024-09-17T21:18:23.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:23.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269043+0000", 2024-09-17T21:18:23.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7306e-05 2024-09-17T21:18:23.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:23.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.270024+0000", 2024-09-17T21:18:23.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00098120600000000009 2024-09-17T21:18:23.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:23.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.270085+0000", 2024-09-17T21:18:23.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0486000000000003e-05 2024-09-17T21:18:23.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:23.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.274557+0000", 2024-09-17T21:18:23.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044723419999999998 2024-09-17T21:18:23.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:23.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.274629+0000", 2024-09-17T21:18:23.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2045999999999994e-05 2024-09-17T21:18:23.005 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:23.006 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:23.006 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:23.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.006 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:197 3.7 e29/27 3:e276f830:::rbd_object_map.1152772b7dab:head v 29'12, mlcod=29'9)", 2024-09-17T21:18:23.006 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:21.460475+0000", 2024-09-17T21:18:23.006 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.4575878870000001, 2024-09-17T21:18:23.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0045300849999999997, 2024-09-17T21:18:23.006 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:23.006 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:23.006 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:23.006 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:23.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:23.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.460475+0000", 2024-09-17T21:18:23.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:23.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:23.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.460475+0000", 2024-09-17T21:18:23.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:23.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:23.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.460476+0000", 2024-09-17T21:18:23.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1599999999999999e-06 2024-09-17T21:18:23.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.008 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:23.008 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.460486+0000", 2024-09-17T21:18:23.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6569999999999992e-06 2024-09-17T21:18:23.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.008 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:23.008 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.460509+0000", 2024-09-17T21:18:23.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3226000000000001e-05 2024-09-17T21:18:23.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:23.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.460527+0000", 2024-09-17T21:18:23.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8462999999999999e-05 2024-09-17T21:18:23.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:23.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.460615+0000", 2024-09-17T21:18:23.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7503000000000006e-05 2024-09-17T21:18:23.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:23.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.460637+0000", 2024-09-17T21:18:23.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1610000000000001e-05 2024-09-17T21:18:23.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:23.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.464958+0000", 2024-09-17T21:18:23.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0043218939999999997 2024-09-17T21:18:23.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:23.011 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.465005+0000", 2024-09-17T21:18:23.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6572000000000001e-05 2024-09-17T21:18:23.011 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:23.011 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:23.011 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:23.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.011 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:18:23.011 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:18:23.011 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.050401851999999997, 2024-09-17T21:18:23.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:18:23.012 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:23.012 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:23.012 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:23.012 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:23.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.012 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:23.012 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:18:23.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:23.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:23.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:18:23.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:23.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:23.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:18:23.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:18:23.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:23.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:18:23.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:18:23.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:23.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:18:23.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:18:23.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:23.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:18:23.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:18:23.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:23.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:18:23.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:18:23.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:23.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:18:23.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:18:23.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:23.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:18:23.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:18:23.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.016 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:23.016 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:18:23.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:18:23.016 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:23.016 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:23.016 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:23.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.017 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:18:23.017 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:18:23.017 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.049136768999999997, 2024-09-17T21:18:23.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:18:23.017 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:23.017 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:23.017 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:23.017 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:23.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.017 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:23.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:18:23.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:23.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:23.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:18:23.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:23.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:23.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:18:23.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:18:23.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:23.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:18:23.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:18:23.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:23.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:18:23.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:18:23.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:23.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:18:23.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:18:23.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:23.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:18:23.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:18:23.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.021 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:23.021 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:18:23.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:18:23.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.021 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:23.021 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:18:23.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:18:23.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.021 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:23.022 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:18:23.022 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:18:23.022 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:23.022 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:23.022 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:23.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.022 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:18:23.022 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:18:23.022 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.039357359000000001, 2024-09-17T21:18:23.022 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:18:23.023 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:23.023 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:23.023 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:23.023 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:23.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.023 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:23.023 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:18:23.023 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:23.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.023 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:23.023 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:18:23.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:23.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:23.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:18:23.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:18:23.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:23.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:18:23.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:18:23.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:23.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:18:23.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:18:23.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:23.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:18:23.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:18:23.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.026 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:23.026 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:18:23.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:18:23.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.026 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:23.026 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:18:23.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:18:23.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.026 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:23.027 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:18:23.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:18:23.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.027 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:23.027 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:18:23.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:18:23.027 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:23.027 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:23.027 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:23.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.027 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:18:23.028 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:18:23.028 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.030634956000000001, 2024-09-17T21:18:23.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:18:23.028 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:23.028 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:23.028 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:23.028 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:23.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:23.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:18:23.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:23.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:23.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:18:23.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:23.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:23.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:18:23.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:18:23.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:23.030 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:18:23.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:18:23.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:23.030 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:18:23.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:18:23.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:23.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:18:23.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:18:23.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:23.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:18:23.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:18:23.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:23.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:18:23.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:18:23.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:23.032 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:18:23.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:18:23.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:23.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:23.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:23.032 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:18:23.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:18:23.032 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:23.032 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:23.033 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:23.033 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:23.033 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:23.033 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:23.033 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:18:23.035 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:23.030+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:23.036 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:18:23.135 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:23.128+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:23.165 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:23.165 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:23.165 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:18:23.165 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:23.179 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:18:23.231 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:18:23.231 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:18:23.236 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:23.230+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:23.248 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:18:23.248 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:18:23.264 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:18:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:18:23.285 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:18:23.285 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:18:23.305 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:18:23.305 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:18:23.309 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:23.310 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:23.325 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:18:23.326 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:18:23.326 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:18:23.374 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:23.370+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:23.416 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-17T21:18:23.438 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:23.432+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:23.506 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:23.506 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:23.520 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:18:23.538 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:23.532+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:23.639 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:23.632+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:23.648 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:23.648 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:23.662 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:18:23.706 INFO:teuthology.orchestra.run.smithi184.stdout:254 write 0x8cac800 thru 0x8caf604 (0x2e05 bytes) 2024-09-17T21:18:23.708 INFO:teuthology.orchestra.run.smithi184.stdout:255 write 0x5592800 thru 0x55a113a (0xe93b bytes) 2024-09-17T21:18:23.709 INFO:teuthology.orchestra.run.smithi184.stdout:256 write 0x13ab800 thru 0x13b2638 (0x6e39 bytes) 2024-09-17T21:18:23.712 INFO:teuthology.orchestra.run.smithi184.stdout:259 write 0x6c73c00 thru 0x6c797cb (0x5bcc bytes) 2024-09-17T21:18:23.740 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:23.734+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:23.781 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:23.781 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:23.787 INFO:teuthology.orchestra.run.smithi184.stdout:264 trunc from 0xba80800 to 0x1522800 2024-09-17T21:18:23.795 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:18:23.840 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:23.836+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:23.941 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:23.936+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:23.965 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:23.965 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:23.973 INFO:teuthology.orchestra.run.smithi184.stdout:266 write 0xed0c200 thru 0xed123b5 (0x61b6 bytes) 2024-09-17T21:18:23.975 INFO:teuthology.orchestra.run.smithi184.stdout:267 punch from 0x408fc00 to 0x4090a00, (0xe00 bytes) 2024-09-17T21:18:23.975 INFO:teuthology.orchestra.run.smithi184.stdout:268 write 0x2fa8e00 thru 0x2fac9a2 (0x3ba3 bytes) 2024-09-17T21:18:23.976 INFO:teuthology.orchestra.run.smithi184.stdout:269 write 0xe958c00 thru 0xe95b993 (0x2d94 bytes) 2024-09-17T21:18:23.976 INFO:teuthology.orchestra.run.smithi184.stdout:274 read 0xa7aca00 thru 0xa7b1cf2 (0x52f3 bytes) 2024-09-17T21:18:23.976 INFO:teuthology.orchestra.run.smithi184.stdout:275 read 0x5255a00 thru 0x525b2bf (0x58c0 bytes) 2024-09-17T21:18:23.976 INFO:teuthology.orchestra.run.smithi184.stdout:276 write 0x76f1600 thru 0x76f932c (0x7d2d bytes) 2024-09-17T21:18:23.981 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:18:23.995 INFO:teuthology.orchestra.run.smithi184.stdout:277 trunc from 0xed12400 to 0x4faf600 2024-09-17T21:18:24.042 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:24.036+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:24.079 INFO:teuthology.orchestra.run.smithi184.stdout:282 write 0x7d71c00 thru 0x7d7472e (0x2b2f bytes) 2024-09-17T21:18:24.080 INFO:teuthology.orchestra.run.smithi184.stdout:283 write 0xe4dc00 thru 0xe55704 (0x7b05 bytes) 2024-09-17T21:18:24.106 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:24.107 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:24.121 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:18:24.134 INFO:teuthology.orchestra.run.smithi184.stdout:284 write 0xed66000 thru 0xed71287 (0xb288 bytes) 2024-09-17T21:18:24.135 INFO:teuthology.orchestra.run.smithi184.stdout:285 punch from 0xa070600 to 0xa07c600, (0xc000 bytes) 2024-09-17T21:18:24.139 INFO:teuthology.orchestra.run.smithi184.stdout:286 trunc from 0xed71400 to 0xd5a6c00 2024-09-17T21:18:24.143 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:24.136+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:24.179 INFO:teuthology.orchestra.run.smithi184.stdout:288 punch from 0x7549600 to 0x7552000, (0x8a00 bytes) 2024-09-17T21:18:24.185 INFO:teuthology.orchestra.run.smithi184.stdout:289 punch from 0x95ca00 to 0x95dc00, (0x1200 bytes) 2024-09-17T21:18:24.185 INFO:teuthology.orchestra.run.smithi184.stdout:290 write 0x4800000 thru 0x48082a3 (0x82a4 bytes) 2024-09-17T21:18:24.185 INFO:teuthology.orchestra.run.smithi184.stdout:294 write 0x8a6f400 thru 0x8a76acb (0x76cc bytes) 2024-09-17T21:18:24.185 INFO:teuthology.orchestra.run.smithi184.stdout:297 punch from 0x993e600 to 0x9946c00, (0x8600 bytes) 2024-09-17T21:18:24.185 INFO:teuthology.orchestra.run.smithi184.stdout:302 punch from 0x1956800 to 0x1958200, (0x1a00 bytes) 2024-09-17T21:18:24.185 INFO:teuthology.orchestra.run.smithi184.stdout:304 read 0x7e8f000 thru 0x7e9c37d (0xd37e bytes) 2024-09-17T21:18:24.186 INFO:teuthology.orchestra.run.smithi184.stdout:306 write 0x3338400 thru 0x333a305 (0x1f06 bytes) 2024-09-17T21:18:24.186 INFO:teuthology.orchestra.run.smithi184.stdout:308 write 0xc33fa00 thru 0xc33fab5 (0xb6 bytes) 2024-09-17T21:18:24.186 INFO:teuthology.orchestra.run.smithi184.stdout:311 write 0x540a000 thru 0x540da88 (0x3a89 bytes) 2024-09-17T21:18:24.187 INFO:teuthology.orchestra.run.smithi184.stdout:313 write 0xbe34c00 thru 0xbe3ae92 (0x6293 bytes) 2024-09-17T21:18:24.187 INFO:teuthology.orchestra.run.smithi184.stdout:316 read 0x42c4c00 thru 0x42c8663 (0x3a64 bytes) 2024-09-17T21:18:24.187 INFO:teuthology.orchestra.run.smithi184.stdout:317 read 0x808800 thru 0x80cc8d (0x448e bytes) 2024-09-17T21:18:24.189 INFO:teuthology.orchestra.run.smithi184.stdout:318 clone 4 order 23 su 131072 sc 15 2024-09-17T21:18:24.244 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:24.240+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:24.251 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:24.251 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:24.265 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:18:24.345 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:24.342+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:24.383 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:24.383 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:24.397 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:18:24.446 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:24.440+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:24.530 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:24.530 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:24.545 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:18:24.547 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:24.542+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:24.647 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:18:24.648 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:24.642+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:24.717 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:24.717 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:24.732 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:18:24.748 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:24.742+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:24.826 INFO:teuthology.orchestra.run.smithi184.stdout:flattening image image_client.0-clone3 2024-09-17T21:18:24.849 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:24.846+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:24.855 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:24.855 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:24.870 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:18:24.950 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:24.946+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:24.987 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:24.988 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:25.002 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:18:25.051 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:25.046+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:25.123 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:25.124 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:25.139 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:18:25.152 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:25.146+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:25.253 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:25.248+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:25.264 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:25.265 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:25.279 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:18:25.343 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:18:25.354 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:25.350+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:25.444 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:25.444 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:25.455 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:25.450+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:25.459 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:18:25.500 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-17T21:18:25.501 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:18:25.502 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:18:25.519 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:18:25.519 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:18:25.536 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:18:25.536 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:18:25.555 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:25.548+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:25.558 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:18:25.558 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:18:25.580 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:18:25.580 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:18:25.581 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:25.581 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:25.581 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:25.581 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:25.581 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:25.597 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:18:25.602 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:18:25.602 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:18:25.647 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-17T21:18:25.656 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:25.652+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:25.733 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:25.733 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:18:25.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:18:25.733 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:18:25.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.733 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4190.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-17T21:18:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:01.050597+0000", 2024-09-17T21:18:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 24.680744446999999, 2024-09-17T21:18:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042911440000000002, 2024-09-17T21:18:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050597+0000", 2024-09-17T21:18:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050597+0000", 2024-09-17T21:18:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050597+0000", 2024-09-17T21:18:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6599999999999996e-07 2024-09-17T21:18:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050604+0000", 2024-09-17T21:18:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0400000000000004e-06 2024-09-17T21:18:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050623+0000", 2024-09-17T21:18:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8691000000000001e-05 2024-09-17T21:18:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050639+0000", 2024-09-17T21:18:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5719999999999999e-05 2024-09-17T21:18:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050709+0000", 2024-09-17T21:18:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9898000000000001e-05 2024-09-17T21:18:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.050738+0000", 2024-09-17T21:18:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9023e-05 2024-09-17T21:18:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.054823+0000", 2024-09-17T21:18:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0040854940000000003 2024-09-17T21:18:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:01.054888+0000", 2024-09-17T21:18:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4712e-05 2024-09-17T21:18:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:12 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.399006764999999, 2024-09-17T21:18:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061340989999999996, 2024-09-17T21:18:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12 2024-09-17T21:18:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:25.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:25.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332336+0000", 2024-09-17T21:18:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5519999999999999e-06 2024-09-17T21:18:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332342+0000", 2024-09-17T21:18:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7180000000000001e-06 2024-09-17T21:18:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:25.742 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332352+0000", 2024-09-17T21:18:25.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:18:25.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:25.742 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332359+0000", 2024-09-17T21:18:25.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3479999999999998e-06 2024-09-17T21:18:25.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332413+0000", 2024-09-17T21:18:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4447999999999998e-05 2024-09-17T21:18:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-17T21:18:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332417+0000", 2024-09-17T21:18:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.895e-06 2024-09-17T21:18:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334755+0000", 2024-09-17T21:18:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023377189999999998 2024-09-17T21:18:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334903+0000", 2024-09-17T21:18:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014778999999999999 2024-09-17T21:18:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:18:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.335187+0000", 2024-09-17T21:18:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000284715 2024-09-17T21:18:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.337731+0000", 2024-09-17T21:18:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002543741 2024-09-17T21:18:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338382+0000", 2024-09-17T21:18:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00065029200000000001 2024-09-17T21:18:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338418+0000", 2024-09-17T21:18:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.697e-05 2024-09-17T21:18:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338469+0000", 2024-09-17T21:18:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0154999999999997e-05 2024-09-17T21:18:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "pg_update_log_missing(3.1e epoch 28/27 rep_tid 81 entries 28'41 (0'0) error 3:7b23fd98:::rbd_header.1152c01721e5:head by client.4488.0:561 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-17T21:18:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:18:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.4497326380000004, 2024-09-17T21:18:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0051146969999999996, 2024-09-17T21:18:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:18:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:18:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:18:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281610+0000", 2024-09-17T21:18:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.16e-07 2024-09-17T21:18:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281626+0000", 2024-09-17T21:18:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6039999999999999e-05 2024-09-17T21:18:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281666+0000", 2024-09-17T21:18:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0364000000000002e-05 2024-09-17T21:18:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281710+0000", 2024-09-17T21:18:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3596000000000001e-05 2024-09-17T21:18:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281992+0000", 2024-09-17T21:18:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028273099999999999 2024-09-17T21:18:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286723+0000", 2024-09-17T21:18:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047310499999999997 2024-09-17T21:18:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4488.0:562 3.1e e28/27 3:7b23fd98:::rbd_header.1152c01721e5:head v 28'42, mlcod=0'0)", 2024-09-17T21:18:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:18:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.4494888499999998, 2024-09-17T21:18:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049275020000000003, 2024-09-17T21:18:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:18:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.752 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:25.752 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:18:25.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.752 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:25.752 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281853+0000", 2024-09-17T21:18:25.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2999999999999999e-07 2024-09-17T21:18:25.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.752 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:25.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281875+0000", 2024-09-17T21:18:25.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1386000000000002e-05 2024-09-17T21:18:25.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:25.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281892+0000", 2024-09-17T21:18:25.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7637000000000001e-05 2024-09-17T21:18:25.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:25.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281908+0000", 2024-09-17T21:18:25.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5187e-05 2024-09-17T21:18:25.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:25.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.282517+0000", 2024-09-17T21:18:25.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00060989799999999999 2024-09-17T21:18:25.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:25.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.282548+0000", 2024-09-17T21:18:25.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0910000000000001e-05 2024-09-17T21:18:25.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:25.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286729+0000", 2024-09-17T21:18:25.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0041810850000000002 2024-09-17T21:18:25.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:25.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286780+0000", 2024-09-17T21:18:25.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0569000000000003e-05 2024-09-17T21:18:25.755 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.755 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:25.755 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.755 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:609 3.d e28/23 3:b7278be9:::rbd_data.1152c6af092e.000000000000008f:head v 28'6, mlcod=28'4)", 2024-09-17T21:18:25.755 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:18:25.755 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 7.6165445629999997, 2024-09-17T21:18:25.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049797280000000001, 2024-09-17T21:18:25.756 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:25.756 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:25.756 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:25.756 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:25.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:25.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:18:25.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:25.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:18:25.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:25.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114800+0000", 2024-09-17T21:18:25.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:18:25.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:25.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114915+0000", 2024-09-17T21:18:25.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115053 2024-09-17T21:18:25.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:25.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114951+0000", 2024-09-17T21:18:25.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6165999999999999e-05 2024-09-17T21:18:25.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:25.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114965+0000", 2024-09-17T21:18:25.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4364e-05 2024-09-17T21:18:25.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:25.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115097+0000", 2024-09-17T21:18:25.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013139200000000001 2024-09-17T21:18:25.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:25.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115151+0000", 2024-09-17T21:18:25.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4422000000000002e-05 2024-09-17T21:18:25.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:25.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.119633+0000", 2024-09-17T21:18:25.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044818820000000004 2024-09-17T21:18:25.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:25.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.119776+0000", 2024-09-17T21:18:25.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000143609 2024-09-17T21:18:25.760 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.760 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:25.760 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.760 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:781 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'62, mlcod=28'60)", 2024-09-17T21:18:25.760 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.213959+0000", 2024-09-17T21:18:25.760 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 7.5173827649999998, 2024-09-17T21:18:25.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0043389400000000003, 2024-09-17T21:18:25.760 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:25.760 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:25.760 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:25.761 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:25.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:25.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.213959+0000", 2024-09-17T21:18:25.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:25.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.213959+0000", 2024-09-17T21:18:25.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:25.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.213960+0000", 2024-09-17T21:18:25.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:18:25.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:25.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.213969+0000", 2024-09-17T21:18:25.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1300000000000007e-06 2024-09-17T21:18:25.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:25.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.213991+0000", 2024-09-17T21:18:25.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2180000000000001e-05 2024-09-17T21:18:25.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:25.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214007+0000", 2024-09-17T21:18:25.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5722999999999998e-05 2024-09-17T21:18:25.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:25.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214141+0000", 2024-09-17T21:18:25.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013425899999999999 2024-09-17T21:18:25.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:25.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214201+0000", 2024-09-17T21:18:25.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0223000000000003e-05 2024-09-17T21:18:25.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:25.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218231+0000", 2024-09-17T21:18:25.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0040299719999999997 2024-09-17T21:18:25.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:25.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218297+0000", 2024-09-17T21:18:25.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6073000000000002e-05 2024-09-17T21:18:25.765 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.765 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:25.765 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.765 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:782 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'63, mlcod=28'60)", 2024-09-17T21:18:25.765 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:18:25.765 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 7.5165765350000004, 2024-09-17T21:18:25.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006167283, 2024-09-17T21:18:25.765 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:25.765 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:25.765 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:25.766 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:25.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:25.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:18:25.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:25.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:18:25.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:25.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214767+0000", 2024-09-17T21:18:25.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:18:25.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:25.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214779+0000", 2024-09-17T21:18:25.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2627e-05 2024-09-17T21:18:25.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:25.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214805+0000", 2024-09-17T21:18:25.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5570000000000001e-05 2024-09-17T21:18:25.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:25.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214823+0000", 2024-09-17T21:18:25.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8202999999999998e-05 2024-09-17T21:18:25.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:25.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215170+0000", 2024-09-17T21:18:25.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034720399999999999 2024-09-17T21:18:25.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:25.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215235+0000", 2024-09-17T21:18:25.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4276000000000006e-05 2024-09-17T21:18:25.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:25.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220859+0000", 2024-09-17T21:18:25.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056240500000000002 2024-09-17T21:18:25.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:25.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220932+0000", 2024-09-17T21:18:25.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3503000000000004e-05 2024-09-17T21:18:25.769 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.769 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:25.769 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.770 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:783 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'64, mlcod=28'60)", 2024-09-17T21:18:25.770 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:18:25.770 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 7.5158896930000001, 2024-09-17T21:18:25.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0055240869999999996, 2024-09-17T21:18:25.770 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:25.770 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:25.770 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:25.770 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:25.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:25.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:18:25.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:25.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:18:25.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:25.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215454+0000", 2024-09-17T21:18:25.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0269999999999998e-06 2024-09-17T21:18:25.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:25.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215468+0000", 2024-09-17T21:18:25.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4816e-05 2024-09-17T21:18:25.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:25.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215501+0000", 2024-09-17T21:18:25.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2957000000000003e-05 2024-09-17T21:18:25.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:25.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215524+0000", 2024-09-17T21:18:25.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2782999999999999e-05 2024-09-17T21:18:25.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:25.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216675+0000", 2024-09-17T21:18:25.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011507869999999999 2024-09-17T21:18:25.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:25.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216728+0000", 2024-09-17T21:18:25.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3112999999999998e-05 2024-09-17T21:18:25.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:25.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220938+0000", 2024-09-17T21:18:25.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042097380000000002 2024-09-17T21:18:25.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:25.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220976+0000", 2024-09-17T21:18:25.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7866e-05 2024-09-17T21:18:25.774 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.774 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:25.774 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.774 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:785 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'65, mlcod=28'60)", 2024-09-17T21:18:25.774 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:18:25.774 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 7.5153036440000003, 2024-09-17T21:18:25.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049795409999999997, 2024-09-17T21:18:25.775 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:25.775 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:25.775 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:25.775 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:25.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:25.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:18:25.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:25.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:18:25.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:25.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216039+0000", 2024-09-17T21:18:25.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.793e-06 2024-09-17T21:18:25.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:25.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216053+0000", 2024-09-17T21:18:25.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.345e-05 2024-09-17T21:18:25.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:25.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216092+0000", 2024-09-17T21:18:25.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9180000000000001e-05 2024-09-17T21:18:25.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:25.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216106+0000", 2024-09-17T21:18:25.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:18:25.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:25.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217569+0000", 2024-09-17T21:18:25.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001462938 2024-09-17T21:18:25.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:25.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217607+0000", 2024-09-17T21:18:25.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7267000000000001e-05 2024-09-17T21:18:25.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:25.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220980+0000", 2024-09-17T21:18:25.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003373544 2024-09-17T21:18:25.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:25.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221017+0000", 2024-09-17T21:18:25.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7116000000000002e-05 2024-09-17T21:18:25.779 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.779 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:25.779 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.779 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:786 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'66, mlcod=28'60)", 2024-09-17T21:18:25.779 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:18:25.779 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 7.5146028400000002, 2024-09-17T21:18:25.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057966650000000003, 2024-09-17T21:18:25.779 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:25.779 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:25.779 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:25.780 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:25.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:25.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:18:25.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:25.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:18:25.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:25.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216740+0000", 2024-09-17T21:18:25.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5600000000000001e-06 2024-09-17T21:18:25.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:25.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216752+0000", 2024-09-17T21:18:25.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1477e-05 2024-09-17T21:18:25.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:25.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216779+0000", 2024-09-17T21:18:25.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7093e-05 2024-09-17T21:18:25.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:25.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216817+0000", 2024-09-17T21:18:25.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8275999999999997e-05 2024-09-17T21:18:25.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:25.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218387+0000", 2024-09-17T21:18:25.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015703749999999999 2024-09-17T21:18:25.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:25.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218432+0000", 2024-09-17T21:18:25.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4910000000000002e-05 2024-09-17T21:18:25.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:25.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222432+0000", 2024-09-17T21:18:25.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039997189999999997 2024-09-17T21:18:25.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:25.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222535+0000", 2024-09-17T21:18:25.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103255 2024-09-17T21:18:25.783 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.784 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:25.784 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.784 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:18:25.784 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:18:25.784 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 7.513981008, 2024-09-17T21:18:25.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:18:25.784 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:25.784 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:25.784 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:25.784 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:25.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:25.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:18:25.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:25.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:18:25.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:25.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:18:25.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:18:25.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:25.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:18:25.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:18:25.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:25.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:18:25.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:18:25.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:25.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:18:25.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:18:25.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:25.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:18:25.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:18:25.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:25.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:18:25.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:18:25.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:25.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:18:25.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:18:25.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:25.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:18:25.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:18:25.788 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.788 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:25.788 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.789 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:788 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'68, mlcod=28'60)", 2024-09-17T21:18:25.789 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:18:25.789 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 7.513346576, 2024-09-17T21:18:25.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058495320000000002, 2024-09-17T21:18:25.789 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:25.789 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:25.789 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:25.789 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:25.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:25.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:18:25.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:25.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:18:25.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:25.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217996+0000", 2024-09-17T21:18:25.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.376e-06 2024-09-17T21:18:25.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:25.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218006+0000", 2024-09-17T21:18:25.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.01e-05 2024-09-17T21:18:25.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:25.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218030+0000", 2024-09-17T21:18:25.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4026999999999999e-05 2024-09-17T21:18:25.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:25.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218042+0000", 2024-09-17T21:18:25.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1977000000000001e-05 2024-09-17T21:18:25.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:25.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221081+0000", 2024-09-17T21:18:25.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030385960000000002 2024-09-17T21:18:25.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:25.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221115+0000", 2024-09-17T21:18:25.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3772999999999997e-05 2024-09-17T21:18:25.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:25.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223749+0000", 2024-09-17T21:18:25.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0026340629999999999 2024-09-17T21:18:25.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:25.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223844+0000", 2024-09-17T21:18:25.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5619999999999996e-05 2024-09-17T21:18:25.793 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.793 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:25.793 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.793 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:790 3.5 e28/23 3:a14b1a37:::rbd_data.1152c6af092e.000000000000002f:head v 28'1, mlcod=0'0)", 2024-09-17T21:18:25.793 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:18:25.793 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 7.5089422500000005, 2024-09-17T21:18:25.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059622240000000003, 2024-09-17T21:18:25.794 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:25.794 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:25.794 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:25.794 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:25.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:25.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:18:25.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:25.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:18:25.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:25.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222400+0000", 2024-09-17T21:18:25.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.02e-06 2024-09-17T21:18:25.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:25.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222411+0000", 2024-09-17T21:18:25.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.044e-05 2024-09-17T21:18:25.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:25.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222440+0000", 2024-09-17T21:18:25.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.932e-05 2024-09-17T21:18:25.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:25.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222458+0000", 2024-09-17T21:18:25.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8615999999999999e-05 2024-09-17T21:18:25.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:25.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222609+0000", 2024-09-17T21:18:25.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015102299999999999 2024-09-17T21:18:25.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:25.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222657+0000", 2024-09-17T21:18:25.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7206000000000002e-05 2024-09-17T21:18:25.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:25.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228278+0000", 2024-09-17T21:18:25.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056208100000000004 2024-09-17T21:18:25.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:25.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228361+0000", 2024-09-17T21:18:25.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3788999999999993e-05 2024-09-17T21:18:25.798 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.798 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:25.798 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.798 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:793 3.1e e28/27 3:793c7d73:::rbd_data.1152c6af092e.0000000000000032:head v 28'48, mlcod=28'46)", 2024-09-17T21:18:25.798 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.224161+0000", 2024-09-17T21:18:25.798 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 7.5071807970000002, 2024-09-17T21:18:25.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004730723, 2024-09-17T21:18:25.798 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:25.798 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:25.798 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:25.798 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:25.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:25.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224161+0000", 2024-09-17T21:18:25.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:25.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224161+0000", 2024-09-17T21:18:25.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:25.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224163+0000", 2024-09-17T21:18:25.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.244e-06 2024-09-17T21:18:25.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:25.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224178+0000", 2024-09-17T21:18:25.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:18:25.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:25.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224210+0000", 2024-09-17T21:18:25.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1152999999999998e-05 2024-09-17T21:18:25.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:25.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224227+0000", 2024-09-17T21:18:25.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7863000000000002e-05 2024-09-17T21:18:25.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:25.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224347+0000", 2024-09-17T21:18:25.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119376 2024-09-17T21:18:25.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:25.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224385+0000", 2024-09-17T21:18:25.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8408999999999998e-05 2024-09-17T21:18:25.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:25.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228842+0000", 2024-09-17T21:18:25.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044566859999999996 2024-09-17T21:18:25.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:25.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228891+0000", 2024-09-17T21:18:25.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9292000000000002e-05 2024-09-17T21:18:25.802 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.802 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:25.802 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.803 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:884 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'76, mlcod=28'70)", 2024-09-17T21:18:25.803 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:18:25.803 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 7.4623573690000002, 2024-09-17T21:18:25.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056451990000000001, 2024-09-17T21:18:25.803 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:25.803 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:25.803 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:25.803 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:25.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:25.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:18:25.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:25.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:18:25.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:25.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268986+0000", 2024-09-17T21:18:25.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9360000000000002e-06 2024-09-17T21:18:25.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:25.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269001+0000", 2024-09-17T21:18:25.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4684e-05 2024-09-17T21:18:25.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:25.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269026+0000", 2024-09-17T21:18:25.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5193000000000001e-05 2024-09-17T21:18:25.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:25.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269043+0000", 2024-09-17T21:18:25.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7306e-05 2024-09-17T21:18:25.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:25.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.270024+0000", 2024-09-17T21:18:25.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00098120600000000009 2024-09-17T21:18:25.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:25.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.270085+0000", 2024-09-17T21:18:25.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0486000000000003e-05 2024-09-17T21:18:25.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:25.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.274557+0000", 2024-09-17T21:18:25.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044723419999999998 2024-09-17T21:18:25.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:25.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.274629+0000", 2024-09-17T21:18:25.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2045999999999994e-05 2024-09-17T21:18:25.807 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.807 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:25.807 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.807 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:197 3.7 e29/27 3:e276f830:::rbd_object_map.1152772b7dab:head v 29'12, mlcod=29'9)", 2024-09-17T21:18:25.807 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:21.460475+0000", 2024-09-17T21:18:25.807 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.2708663790000001, 2024-09-17T21:18:25.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0045300849999999997, 2024-09-17T21:18:25.807 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:25.808 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:25.808 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:25.808 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:25.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:25.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.460475+0000", 2024-09-17T21:18:25.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:25.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.460475+0000", 2024-09-17T21:18:25.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:25.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.460476+0000", 2024-09-17T21:18:25.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1599999999999999e-06 2024-09-17T21:18:25.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:25.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.460486+0000", 2024-09-17T21:18:25.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6569999999999992e-06 2024-09-17T21:18:25.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:25.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.460509+0000", 2024-09-17T21:18:25.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3226000000000001e-05 2024-09-17T21:18:25.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:25.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.460527+0000", 2024-09-17T21:18:25.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8462999999999999e-05 2024-09-17T21:18:25.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:25.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.460615+0000", 2024-09-17T21:18:25.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7503000000000006e-05 2024-09-17T21:18:25.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:25.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.460637+0000", 2024-09-17T21:18:25.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1610000000000001e-05 2024-09-17T21:18:25.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:25.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.464958+0000", 2024-09-17T21:18:25.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0043218939999999997 2024-09-17T21:18:25.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:25.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.465005+0000", 2024-09-17T21:18:25.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6572000000000001e-05 2024-09-17T21:18:25.811 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.811 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:25.812 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.812 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:18:25.812 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:18:25.812 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 2.863680344, 2024-09-17T21:18:25.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:18:25.812 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:25.812 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:25.812 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:25.812 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:25.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:25.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:18:25.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:25.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:18:25.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:25.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:18:25.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:18:25.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:25.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:18:25.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:18:25.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:25.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:18:25.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:18:25.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:25.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:18:25.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:18:25.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:25.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:18:25.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:18:25.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:25.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:18:25.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:18:25.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:25.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:18:25.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:18:25.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:25.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:18:25.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:18:25.816 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.816 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:25.816 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.816 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:18:25.816 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:18:25.817 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 2.8624152609999998, 2024-09-17T21:18:25.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:18:25.817 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:25.817 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:25.817 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:25.817 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:25.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:25.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:18:25.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:25.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:18:25.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:25.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:18:25.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:18:25.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:25.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:18:25.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:18:25.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:25.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:18:25.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:18:25.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:25.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:18:25.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:18:25.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:25.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:18:25.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:18:25.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.820 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:25.820 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:18:25.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:18:25.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.820 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:25.820 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:18:25.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:18:25.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.820 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:25.820 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:18:25.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:18:25.821 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.821 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:25.821 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.821 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.821 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:18:25.821 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:18:25.821 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 2.8526358510000001, 2024-09-17T21:18:25.821 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:18:25.821 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:25.822 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:25.822 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:25.822 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:25.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.822 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:25.822 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:18:25.822 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.822 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:25.822 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:18:25.822 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.822 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:25.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:18:25.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:18:25.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:25.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:18:25.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:18:25.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:25.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:18:25.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:18:25.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:25.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:18:25.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:18:25.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:25.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:18:25.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:18:25.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:25.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:18:25.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:18:25.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:25.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:18:25.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:18:25.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:25.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:18:25.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:18:25.825 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.825 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:25.825 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.826 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.826 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:18:25.826 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:18:25.826 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 2.8439134479999999, 2024-09-17T21:18:25.826 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:18:25.826 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:25.826 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:25.826 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:25.826 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:25.826 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.826 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:25.826 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:18:25.826 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.827 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:25.827 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:18:25.827 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:25.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.827 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:25.827 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:18:25.827 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:18:25.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.827 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:25.827 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:18:25.827 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:18:25.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.828 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:25.828 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:18:25.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:18:25.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.828 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:25.828 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:18:25.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:18:25.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.828 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:25.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:18:25.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:18:25.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:25.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:18:25.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:18:25.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:25.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:18:25.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:18:25.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:25.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:25.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:25.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:18:25.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:18:25.830 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.830 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:25.830 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.830 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:25.830 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:25.830 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:25.831 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:18:25.832 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:25.826+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:25.932 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:25.928+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:25.964 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:25.964 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:25.964 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:18:25.964 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:25.979 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:18:26.033 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:26.026+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:26.103 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:26.103 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:26.103 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:18:26.103 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:18:26.103 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:26.109 INFO:teuthology.orchestra.run.smithi184.stdout:320 read 0x4f81200 thru 0x4f8f51a (0xe31b bytes) 2024-09-17T21:18:26.112 INFO:teuthology.orchestra.run.smithi184.stdout:321 write 0x81e2800 thru 0x81e60af (0x38b0 bytes) 2024-09-17T21:18:26.116 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:18:26.154 INFO:teuthology.orchestra.run.smithi184.stdout:322 trunc from 0xd5a6c00 to 0x7043600 2024-09-17T21:18:26.157 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:26.154+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:26.235 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:26.230+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:26.275 INFO:teuthology.orchestra.run.smithi184.stdout:323 read 0x35e8200 thru 0x35e9c9b (0x1a9c bytes) 2024-09-17T21:18:26.277 INFO:teuthology.orchestra.run.smithi184.stdout:325 read 0x4815600 thru 0x481f64e (0xa04f bytes) 2024-09-17T21:18:26.278 INFO:teuthology.orchestra.run.smithi184.stdout:326 write 0x19adc00 thru 0x19b5b46 (0x7f47 bytes) 2024-09-17T21:18:26.279 INFO:teuthology.orchestra.run.smithi184.stdout:327 write 0x6efb200 thru 0x6f04f2a (0x9d2b bytes) 2024-09-17T21:18:26.289 INFO:teuthology.orchestra.run.smithi184.stdout:328 trunc from 0x7043600 to 0xad90600 2024-09-17T21:18:26.304 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:18:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:18:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:18:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 29.287880659999999, 2024-09-17T21:18:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:18:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:18:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:18:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:18:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:18:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:18:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:18:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:18:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:18:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:18:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:18:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:18:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:18:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:18:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:18:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:18:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:18:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:18:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:18:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:18:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:18:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:18:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:18:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:18:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:18:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:18:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:18:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:18:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:18:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:18:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.233810+0000", 2024-09-17T21:18:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 29.067836225000001, 2024-09-17T21:18:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0096383549999999995, 2024-09-17T21:18:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 24 2024-09-17T21:18:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.233810+0000", 2024-09-17T21:18:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.233810+0000", 2024-09-17T21:18:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.233811+0000", 2024-09-17T21:18:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3230000000000001e-06 2024-09-17T21:18:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.233985+0000", 2024-09-17T21:18:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000173609 2024-09-17T21:18:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234013+0000", 2024-09-17T21:18:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8396000000000001e-05 2024-09-17T21:18:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234035+0000", 2024-09-17T21:18:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1756999999999999e-05 2024-09-17T21:18:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.239983+0000", 2024-09-17T21:18:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059477030000000004 2024-09-17T21:18:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.240111+0000", 2024-09-17T21:18:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012860899999999999 2024-09-17T21:18:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.240317+0000", 2024-09-17T21:18:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020541900000000001 2024-09-17T21:18:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.243187+0000", 2024-09-17T21:18:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002870531 2024-09-17T21:18:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.243295+0000", 2024-09-17T21:18:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000107606 2024-09-17T21:18:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.243381+0000", 2024-09-17T21:18:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5995999999999995e-05 2024-09-17T21:18:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.243448+0000", 2024-09-17T21:18:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7405999999999998e-05 2024-09-17T21:18:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.234074+0000", 2024-09-17T21:18:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 29.067571623999999, 2024-09-17T21:18:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011808219, 2024-09-17T21:18:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 25 2024-09-17T21:18:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234074+0000", 2024-09-17T21:18:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234074+0000", 2024-09-17T21:18:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234076+0000", 2024-09-17T21:18:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1799999999999999e-06 2024-09-17T21:18:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234094+0000", 2024-09-17T21:18:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8240000000000002e-05 2024-09-17T21:18:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234116+0000", 2024-09-17T21:18:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2296000000000001e-05 2024-09-17T21:18:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234140+0000", 2024-09-17T21:18:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4204000000000001e-05 2024-09-17T21:18:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.241931+0000", 2024-09-17T21:18:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0077910419999999998 2024-09-17T21:18:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.242167+0000", 2024-09-17T21:18:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000236025 2024-09-17T21:18:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.242544+0000", 2024-09-17T21:18:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037610699999999998 2024-09-17T21:18:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.243799+0000", 2024-09-17T21:18:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001255611 2024-09-17T21:18:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.245610+0000", 2024-09-17T21:18:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001811099 2024-09-17T21:18:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.245758+0000", 2024-09-17T21:18:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148212 2024-09-17T21:18:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.245883+0000", 2024-09-17T21:18:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124203 2024-09-17T21:18:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.335442+0000", 2024-09-17T21:18:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.966204529999999, 2024-09-17T21:18:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0096596089999999996, 2024-09-17T21:18:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 44 2024-09-17T21:18:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335442+0000", 2024-09-17T21:18:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335442+0000", 2024-09-17T21:18:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335443+0000", 2024-09-17T21:18:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.747e-06 2024-09-17T21:18:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335562+0000", 2024-09-17T21:18:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000118656 2024-09-17T21:18:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335592+0000", 2024-09-17T21:18:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9743e-05 2024-09-17T21:18:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335609+0000", 2024-09-17T21:18:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7558999999999999e-05 2024-09-17T21:18:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.340968+0000", 2024-09-17T21:18:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0053589220000000003 2024-09-17T21:18:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.341101+0000", 2024-09-17T21:18:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013255600000000001 2024-09-17T21:18:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.341308+0000", 2024-09-17T21:18:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020729099999999999 2024-09-17T21:18:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.344815+0000", 2024-09-17T21:18:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003506594 2024-09-17T21:18:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.344960+0000", 2024-09-17T21:18:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145105 2024-09-17T21:18:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.345037+0000", 2024-09-17T21:18:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6987e-05 2024-09-17T21:18:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.345101+0000", 2024-09-17T21:18:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4449000000000006e-05 2024-09-17T21:18:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.335647+0000", 2024-09-17T21:18:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.965998991999999, 2024-09-17T21:18:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012049351, 2024-09-17T21:18:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 45 2024-09-17T21:18:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335647+0000", 2024-09-17T21:18:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335647+0000", 2024-09-17T21:18:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335649+0000", 2024-09-17T21:18:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.624e-06 2024-09-17T21:18:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335680+0000", 2024-09-17T21:18:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0913e-05 2024-09-17T21:18:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335701+0000", 2024-09-17T21:18:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1307000000000001e-05 2024-09-17T21:18:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335718+0000", 2024-09-17T21:18:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6586e-05 2024-09-17T21:18:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.342345+0000", 2024-09-17T21:18:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006627712 2024-09-17T21:18:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.342463+0000", 2024-09-17T21:18:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000117439 2024-09-17T21:18:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.342647+0000", 2024-09-17T21:18:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000184692 2024-09-17T21:18:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.344938+0000", 2024-09-17T21:18:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022909190000000002 2024-09-17T21:18:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.347555+0000", 2024-09-17T21:18:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026162429999999999 2024-09-17T21:18:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.347635+0000", 2024-09-17T21:18:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0253000000000006e-05 2024-09-17T21:18:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.347696+0000", 2024-09-17T21:18:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1662999999999996e-05 2024-09-17T21:18:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.335761+0000", 2024-09-17T21:18:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.965885553, 2024-09-17T21:18:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013102346000000001, 2024-09-17T21:18:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 46 2024-09-17T21:18:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335761+0000", 2024-09-17T21:18:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335761+0000", 2024-09-17T21:18:26.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:26.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335762+0000", 2024-09-17T21:18:26.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:18:26.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:26.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335787+0000", 2024-09-17T21:18:26.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5360000000000001e-05 2024-09-17T21:18:26.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:26.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335808+0000", 2024-09-17T21:18:26.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0333e-05 2024-09-17T21:18:26.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:26.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335868+0000", 2024-09-17T21:18:26.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0463e-05 2024-09-17T21:18:26.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.343865+0000", 2024-09-17T21:18:26.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0079966350000000002 2024-09-17T21:18:26.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:26.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.343989+0000", 2024-09-17T21:18:26.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124449 2024-09-17T21:18:26.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:26.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.344190+0000", 2024-09-17T21:18:26.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020082499999999999 2024-09-17T21:18:26.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:26.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.347528+0000", 2024-09-17T21:18:26.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003337998 2024-09-17T21:18:26.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:26.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.348737+0000", 2024-09-17T21:18:26.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0012084470000000001 2024-09-17T21:18:26.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:26.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.348805+0000", 2024-09-17T21:18:26.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8306000000000006e-05 2024-09-17T21:18:26.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:26.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.348863+0000", 2024-09-17T21:18:26.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8006e-05 2024-09-17T21:18:26.340 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.335903+0000", 2024-09-17T21:18:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.965742939999998, 2024-09-17T21:18:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013146595000000001, 2024-09-17T21:18:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 47 2024-09-17T21:18:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335903+0000", 2024-09-17T21:18:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335903+0000", 2024-09-17T21:18:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335905+0000", 2024-09-17T21:18:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.556e-06 2024-09-17T21:18:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336023+0000", 2024-09-17T21:18:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000118136 2024-09-17T21:18:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336050+0000", 2024-09-17T21:18:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7472999999999999e-05 2024-09-17T21:18:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336066+0000", 2024-09-17T21:18:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5934e-05 2024-09-17T21:18:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.345186+0000", 2024-09-17T21:18:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0091198389999999994 2024-09-17T21:18:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.345337+0000", 2024-09-17T21:18:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015087600000000001 2024-09-17T21:18:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.345549+0000", 2024-09-17T21:18:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021209500000000001 2024-09-17T21:18:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.347760+0000", 2024-09-17T21:18:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022113900000000001 2024-09-17T21:18:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.348877+0000", 2024-09-17T21:18:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011163340000000001 2024-09-17T21:18:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.348951+0000", 2024-09-17T21:18:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4666000000000006e-05 2024-09-17T21:18:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.349050+0000", 2024-09-17T21:18:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8295999999999995e-05 2024-09-17T21:18:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.336089+0000", 2024-09-17T21:18:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.965557208, 2024-09-17T21:18:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014613607000000001, 2024-09-17T21:18:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 48 2024-09-17T21:18:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336089+0000", 2024-09-17T21:18:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336089+0000", 2024-09-17T21:18:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336090+0000", 2024-09-17T21:18:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.083e-06 2024-09-17T21:18:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336097+0000", 2024-09-17T21:18:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0299999999999996e-06 2024-09-17T21:18:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336151+0000", 2024-09-17T21:18:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4292999999999997e-05 2024-09-17T21:18:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336163+0000", 2024-09-17T21:18:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.136e-05 2024-09-17T21:18:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.346568+0000", 2024-09-17T21:18:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010405009999999999 2024-09-17T21:18:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.346687+0000", 2024-09-17T21:18:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011885899999999999 2024-09-17T21:18:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.346865+0000", 2024-09-17T21:18:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017869500000000001 2024-09-17T21:18:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.350554+0000", 2024-09-17T21:18:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036889879999999998 2024-09-17T21:18:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.350575+0000", 2024-09-17T21:18:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.052e-05 2024-09-17T21:18:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.350645+0000", 2024-09-17T21:18:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0506000000000006e-05 2024-09-17T21:18:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.350702+0000", 2024-09-17T21:18:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7262999999999997e-05 2024-09-17T21:18:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.336187+0000", 2024-09-17T21:18:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.965459038999999, 2024-09-17T21:18:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015759305000000001, 2024-09-17T21:18:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 49 2024-09-17T21:18:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336187+0000", 2024-09-17T21:18:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336187+0000", 2024-09-17T21:18:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336188+0000", 2024-09-17T21:18:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3230000000000001e-06 2024-09-17T21:18:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336198+0000", 2024-09-17T21:18:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7440000000000002e-06 2024-09-17T21:18:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336210+0000", 2024-09-17T21:18:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1950000000000001e-05 2024-09-17T21:18:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336218+0000", 2024-09-17T21:18:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6329999999999996e-06 2024-09-17T21:18:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.347858+0000", 2024-09-17T21:18:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011640322999999999 2024-09-17T21:18:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.347979+0000", 2024-09-17T21:18:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012050899999999999 2024-09-17T21:18:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.348185+0000", 2024-09-17T21:18:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020600499999999999 2024-09-17T21:18:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.350801+0000", 2024-09-17T21:18:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026167830000000001 2024-09-17T21:18:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.351816+0000", 2024-09-17T21:18:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001015019 2024-09-17T21:18:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.351886+0000", 2024-09-17T21:18:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9952999999999999e-05 2024-09-17T21:18:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.351946+0000", 2024-09-17T21:18:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0062999999999997e-05 2024-09-17T21:18:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.336258+0000", 2024-09-17T21:18:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.965388287, 2024-09-17T21:18:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016750900999999999, 2024-09-17T21:18:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 50 2024-09-17T21:18:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336258+0000", 2024-09-17T21:18:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336258+0000", 2024-09-17T21:18:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336259+0000", 2024-09-17T21:18:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.11e-06 2024-09-17T21:18:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336369+0000", 2024-09-17T21:18:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010995 2024-09-17T21:18:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336384+0000", 2024-09-17T21:18:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5353e-05 2024-09-17T21:18:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336395+0000", 2024-09-17T21:18:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0995999999999999e-05 2024-09-17T21:18:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.349183+0000", 2024-09-17T21:18:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012788152000000001 2024-09-17T21:18:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.349310+0000", 2024-09-17T21:18:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012668600000000001 2024-09-17T21:18:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.349523+0000", 2024-09-17T21:18:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021328499999999999 2024-09-17T21:18:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.352854+0000", 2024-09-17T21:18:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033302140000000002 2024-09-17T21:18:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.352877+0000", 2024-09-17T21:18:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3195999999999999e-05 2024-09-17T21:18:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.352949+0000", 2024-09-17T21:18:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2086000000000001e-05 2024-09-17T21:18:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353009+0000", 2024-09-17T21:18:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9873000000000001e-05 2024-09-17T21:18:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.336415+0000", 2024-09-17T21:18:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.965230718000001, 2024-09-17T21:18:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016888507000000001, 2024-09-17T21:18:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 51 2024-09-17T21:18:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336415+0000", 2024-09-17T21:18:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336415+0000", 2024-09-17T21:18:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336416+0000", 2024-09-17T21:18:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7000000000000003e-07 2024-09-17T21:18:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336423+0000", 2024-09-17T21:18:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0940000000000002e-06 2024-09-17T21:18:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336433+0000", 2024-09-17T21:18:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.73e-06 2024-09-17T21:18:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336440+0000", 2024-09-17T21:18:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.973e-06 2024-09-17T21:18:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.350896+0000", 2024-09-17T21:18:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014456101000000001 2024-09-17T21:18:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.351027+0000", 2024-09-17T21:18:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000131063 2024-09-17T21:18:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.351249+0000", 2024-09-17T21:18:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000221568 2024-09-17T21:18:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353101+0000", 2024-09-17T21:18:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018519020000000001 2024-09-17T21:18:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353150+0000", 2024-09-17T21:18:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9607000000000001e-05 2024-09-17T21:18:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353249+0000", 2024-09-17T21:18:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8176000000000003e-05 2024-09-17T21:18:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353304+0000", 2024-09-17T21:18:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5322999999999999e-05 2024-09-17T21:18:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.336482+0000", 2024-09-17T21:18:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.965163695000001, 2024-09-17T21:18:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017076834999999999, 2024-09-17T21:18:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 52 2024-09-17T21:18:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336482+0000", 2024-09-17T21:18:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336482+0000", 2024-09-17T21:18:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336483+0000", 2024-09-17T21:18:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1699999999999997e-07 2024-09-17T21:18:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336490+0000", 2024-09-17T21:18:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5030000000000002e-06 2024-09-17T21:18:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336501+0000", 2024-09-17T21:18:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0827e-05 2024-09-17T21:18:26.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:26.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336508+0000", 2024-09-17T21:18:26.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4630000000000004e-06 2024-09-17T21:18:26.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.352032+0000", 2024-09-17T21:18:26.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015524263 2024-09-17T21:18:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.352175+0000", 2024-09-17T21:18:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000143206 2024-09-17T21:18:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.352363+0000", 2024-09-17T21:18:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000187579 2024-09-17T21:18:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353377+0000", 2024-09-17T21:18:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001013772 2024-09-17T21:18:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353424+0000", 2024-09-17T21:18:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7456000000000001e-05 2024-09-17T21:18:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353499+0000", 2024-09-17T21:18:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4518999999999996e-05 2024-09-17T21:18:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353559+0000", 2024-09-17T21:18:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0430000000000002e-05 2024-09-17T21:18:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.531642+0000", 2024-09-17T21:18:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.770003596999999, 2024-09-17T21:18:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010095023999999999, 2024-09-17T21:18:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 73 2024-09-17T21:18:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531642+0000", 2024-09-17T21:18:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531642+0000", 2024-09-17T21:18:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531643+0000", 2024-09-17T21:18:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3099999999999998e-07 2024-09-17T21:18:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531647+0000", 2024-09-17T21:18:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6370000000000001e-06 2024-09-17T21:18:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531652+0000", 2024-09-17T21:18:26.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4790000000000004e-06 2024-09-17T21:18:26.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:26.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531656+0000", 2024-09-17T21:18:26.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8530000000000002e-06 2024-09-17T21:18:26.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.537928+0000", 2024-09-17T21:18:26.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062718560000000001 2024-09-17T21:18:26.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:26.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.537992+0000", 2024-09-17T21:18:26.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4284999999999999e-05 2024-09-17T21:18:26.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:26.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.538086+0000", 2024-09-17T21:18:26.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4094999999999999e-05 2024-09-17T21:18:26.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:26.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.540707+0000", 2024-09-17T21:18:26.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002621057 2024-09-17T21:18:26.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:26.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541673+0000", 2024-09-17T21:18:26.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00096603399999999995 2024-09-17T21:18:26.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:26.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541709+0000", 2024-09-17T21:18:26.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5790999999999997e-05 2024-09-17T21:18:26.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:26.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541738+0000", 2024-09-17T21:18:26.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8405999999999999e-05 2024-09-17T21:18:26.381 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.381 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:26.382 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.382 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:26.382 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.531665+0000", 2024-09-17T21:18:26.382 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.769981304000002, 2024-09-17T21:18:26.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010231192, 2024-09-17T21:18:26.382 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:26.382 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:26.382 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:26.382 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:26.382 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:26.382 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:26.382 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 74 2024-09-17T21:18:26.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.383 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:26.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:26.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531665+0000", 2024-09-17T21:18:26.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:26.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531665+0000", 2024-09-17T21:18:26.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:26.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531665+0000", 2024-09-17T21:18:26.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.34e-07 2024-09-17T21:18:26.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:26.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531670+0000", 2024-09-17T21:18:26.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8230000000000002e-06 2024-09-17T21:18:26.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:26.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531675+0000", 2024-09-17T21:18:26.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8810000000000004e-06 2024-09-17T21:18:26.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:26.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531680+0000", 2024-09-17T21:18:26.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2959999999999998e-06 2024-09-17T21:18:26.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.538749+0000", 2024-09-17T21:18:26.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0070683309999999997 2024-09-17T21:18:26.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:26.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.538834+0000", 2024-09-17T21:18:26.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5620999999999999e-05 2024-09-17T21:18:26.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:26.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.538934+0000", 2024-09-17T21:18:26.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9508999999999999e-05 2024-09-17T21:18:26.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:26.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541012+0000", 2024-09-17T21:18:26.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0020783099999999999 2024-09-17T21:18:26.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:26.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541781+0000", 2024-09-17T21:18:26.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00076862700000000003 2024-09-17T21:18:26.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.387 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.387 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:26.387 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541835+0000", 2024-09-17T21:18:26.387 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4283999999999997e-05 2024-09-17T21:18:26.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.387 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.387 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:26.387 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541896+0000", 2024-09-17T21:18:26.387 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1075999999999996e-05 2024-09-17T21:18:26.387 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.387 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:26.387 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.388 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.388 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:26.388 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.531687+0000", 2024-09-17T21:18:26.388 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.769959042, 2024-09-17T21:18:26.388 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010899883000000001, 2024-09-17T21:18:26.388 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:26.388 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:26.388 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:26.388 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:26.388 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:26.388 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:26.388 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 75 2024-09-17T21:18:26.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.388 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:26.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.389 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:26.389 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531687+0000", 2024-09-17T21:18:26.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.389 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:26.389 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531687+0000", 2024-09-17T21:18:26.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.389 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:26.389 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531687+0000", 2024-09-17T21:18:26.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8500000000000002e-07 2024-09-17T21:18:26.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.390 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:26.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531748+0000", 2024-09-17T21:18:26.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0588000000000003e-05 2024-09-17T21:18:26.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.390 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:26.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531756+0000", 2024-09-17T21:18:26.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6399999999999997e-06 2024-09-17T21:18:26.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.390 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:26.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531760+0000", 2024-09-17T21:18:26.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4870000000000004e-06 2024-09-17T21:18:26.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.539287+0000", 2024-09-17T21:18:26.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0075267379999999998 2024-09-17T21:18:26.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:26.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.539353+0000", 2024-09-17T21:18:26.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6577999999999999e-05 2024-09-17T21:18:26.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:26.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.539470+0000", 2024-09-17T21:18:26.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000116662 2024-09-17T21:18:26.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.392 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:26.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541287+0000", 2024-09-17T21:18:26.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018171839999999999 2024-09-17T21:18:26.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.392 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:26.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.542494+0000", 2024-09-17T21:18:26.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0012064650000000001 2024-09-17T21:18:26.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:26.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.542533+0000", 2024-09-17T21:18:26.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9196999999999998e-05 2024-09-17T21:18:26.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:26.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.542587+0000", 2024-09-17T21:18:26.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3959000000000003e-05 2024-09-17T21:18:26.393 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.393 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:26.393 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.393 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:26.394 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.531769+0000", 2024-09-17T21:18:26.394 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.769876621000002, 2024-09-17T21:18:26.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011381054, 2024-09-17T21:18:26.394 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:26.394 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:26.394 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:26.394 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:26.394 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:26.394 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:26.394 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 76 2024-09-17T21:18:26.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.394 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:26.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:26.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531769+0000", 2024-09-17T21:18:26.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:26.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531769+0000", 2024-09-17T21:18:26.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:26.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531770+0000", 2024-09-17T21:18:26.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8100000000000003e-07 2024-09-17T21:18:26.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:26.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531782+0000", 2024-09-17T21:18:26.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1832e-05 2024-09-17T21:18:26.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:26.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531787+0000", 2024-09-17T21:18:26.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3870000000000003e-06 2024-09-17T21:18:26.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:26.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531797+0000", 2024-09-17T21:18:26.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0036e-05 2024-09-17T21:18:26.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.540161+0000", 2024-09-17T21:18:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083641919999999995 2024-09-17T21:18:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.540220+0000", 2024-09-17T21:18:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8829000000000002e-05 2024-09-17T21:18:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.540312+0000", 2024-09-17T21:18:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1983000000000003e-05 2024-09-17T21:18:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:26.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.542152+0000", 2024-09-17T21:18:26.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018396840000000001 2024-09-17T21:18:26.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:26.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.543063+0000", 2024-09-17T21:18:26.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00091107600000000003 2024-09-17T21:18:26.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:26.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.543102+0000", 2024-09-17T21:18:26.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8767999999999999e-05 2024-09-17T21:18:26.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:26.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.543151+0000", 2024-09-17T21:18:26.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8785999999999997e-05 2024-09-17T21:18:26.399 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.399 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:26.399 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.399 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:18:26.399 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:18:26.399 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.7077163800000008, 2024-09-17T21:18:26.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:18:26.400 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:26.400 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:26.400 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:26.400 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:26.400 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:18:26.400 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:26.400 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:18:26.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.400 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:26.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:26.400 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:18:26.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:26.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:18:26.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:26.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:18:26.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:18:26.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:26.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:18:26.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:18:26.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:26.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:18:26.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:18:26.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:26.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:18:26.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:18:26.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:18:26.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:18:26.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:26.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:18:26.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:18:26.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:18:26.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:18:26.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:26.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:18:26.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:18:26.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.404 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.404 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:18:26.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:18:26.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.404 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:26.404 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:18:26.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:18:26.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.404 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:26.404 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:18:26.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:18:26.405 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.405 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:26.405 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.405 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:18:26.405 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:18:26.405 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.7076190679999996, 2024-09-17T21:18:26.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:18:26.405 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:26.405 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:26.405 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:26.406 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:26.406 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:26.406 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:26.406 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:18:26.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.406 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:26.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:26.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:18:26.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:26.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:18:26.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:26.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:18:26.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:18:26.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:26.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:18:26.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:18:26.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:26.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:18:26.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:18:26.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.408 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:26.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:18:26.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:18:26.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.408 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:18:26.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:18:26.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:26.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:18:26.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:18:26.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:18:26.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:18:26.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:26.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:18:26.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:18:26.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:18:26.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:18:26.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:18:26.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:18:26.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:18:26.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:18:26.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:18:26.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:26.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:18:26.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:18:26.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:26.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:18:26.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:18:26.411 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.411 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:26.411 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.412 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:18:26.412 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:18:26.412 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.7055322670000006, 2024-09-17T21:18:26.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:18:26.412 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:26.412 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:26.412 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:26.412 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:26.412 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:18:26.412 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:26.412 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:18:26.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.413 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:26.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:26.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:18:26.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:26.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:18:26.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:26.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:18:26.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:18:26.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:26.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:18:26.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:18:26.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:26.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:18:26.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:18:26.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:26.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:18:26.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:18:26.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:18:26.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:18:26.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:18:26.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:18:26.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:18:26.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:18:26.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:18:26.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:26.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:18:26.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:18:26.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:18:26.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:18:26.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:26.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:18:26.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:18:26.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:26.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:18:26.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:18:26.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:26.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:18:26.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:18:26.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:26.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:18:26.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:18:26.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:26.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:18:26.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:18:26.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:26.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:18:26.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:18:26.419 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.419 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:26.419 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.419 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:575 3.15 3:ac921cb1:::rbd_data.1152b51cb396.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:26.419 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.213766+0000", 2024-09-17T21:18:26.419 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.087879688999999997, 2024-09-17T21:18:26.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015902976999999999, 2024-09-17T21:18:26.419 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:26.419 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:26.419 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:26.420 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:26.420 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:26.420 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:26.420 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 575 2024-09-17T21:18:26.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.420 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:26.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:26.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.213766+0000", 2024-09-17T21:18:26.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:26.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.213766+0000", 2024-09-17T21:18:26.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:26.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:26.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.213768+0000", 2024-09-17T21:18:26.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1149999999999999e-06 2024-09-17T21:18:26.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:26.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.216434+0000", 2024-09-17T21:18:26.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026669580000000001 2024-09-17T21:18:26.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:26.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.216450+0000", 2024-09-17T21:18:26.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5122e-05 2024-09-17T21:18:26.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.422 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:26.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.216458+0000", 2024-09-17T21:18:26.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7949999999999996e-06 2024-09-17T21:18:26.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.422 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:26.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.216604+0000", 2024-09-17T21:18:26.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145313 2024-09-17T21:18:26.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:26.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.216940+0000", 2024-09-17T21:18:26.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033653099999999999 2024-09-17T21:18:26.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:26.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.217594+0000", 2024-09-17T21:18:26.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00065363499999999996 2024-09-17T21:18:26.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:26.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.224088+0000", 2024-09-17T21:18:26.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0064940270000000003 2024-09-17T21:18:26.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:26.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.229575+0000", 2024-09-17T21:18:26.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0054871620000000003 2024-09-17T21:18:26.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:26.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.229635+0000", 2024-09-17T21:18:26.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9781000000000002e-05 2024-09-17T21:18:26.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:26.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:26.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:26.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.229669+0000", 2024-09-17T21:18:26.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4538e-05 2024-09-17T21:18:26.424 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.425 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:26.425 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.425 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:26.425 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:26.425 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:26.427 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:18:26.428 INFO:teuthology.orchestra.run.smithi184.stdout:329 write 0x6f03c00 thru 0x6f0af6d (0x736e bytes) 2024-09-17T21:18:26.428 INFO:teuthology.orchestra.run.smithi184.stdout:330 trunc from 0xad90600 to 0xe431600 2024-09-17T21:18:26.428 INFO:teuthology.orchestra.run.smithi184.stdout:331 write 0x43f6000 thru 0x43ff84a (0x984b bytes) 2024-09-17T21:18:26.428 INFO:teuthology.orchestra.run.smithi184.stdout:333 punch from 0xd16a00 to 0xd21e00, (0xb400 bytes) 2024-09-17T21:18:26.428 INFO:teuthology.orchestra.run.smithi184.stdout:334 read 0x46de000 thru 0x46ed574 (0xf575 bytes) 2024-09-17T21:18:26.428 INFO:teuthology.orchestra.run.smithi184.stdout:335 trunc from 0xe431600 to 0x7504400 2024-09-17T21:18:26.428 INFO:teuthology.orchestra.run.smithi184.stdout:336 punch from 0x585000 to 0x592800, (0xd800 bytes) 2024-09-17T21:18:26.428 INFO:teuthology.orchestra.run.smithi184.stdout:337 read 0x1ebc000 thru 0x1eca1df (0xe1e0 bytes) 2024-09-17T21:18:26.428 INFO:teuthology.orchestra.run.smithi184.stdout:340 write 0x9c32a00 thru 0x9c429bb (0xffbc bytes) 2024-09-17T21:18:26.428 INFO:teuthology.orchestra.run.smithi184.stdout:342 punch from 0x5ef3600 to 0x5efe200, (0xac00 bytes) 2024-09-17T21:18:26.429 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:26.420+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:26.429 INFO:teuthology.orchestra.run.smithi184.stdout:343 read 0x3092800 thru 0x309a2d8 (0x7ad9 bytes) 2024-09-17T21:18:26.429 INFO:teuthology.orchestra.run.smithi184.stdout:344 write 0x63bbc00 thru 0x63be2f1 (0x26f2 bytes) 2024-09-17T21:18:26.430 INFO:teuthology.orchestra.run.smithi184.stdout:345 read 0xf52600 thru 0xf5d7e8 (0xb1e9 bytes) 2024-09-17T21:18:26.432 INFO:teuthology.orchestra.run.smithi184.stdout:348 write 0x6737c00 thru 0x6742582 (0xa983 bytes) 2024-09-17T21:18:26.432 INFO:teuthology.orchestra.run.smithi184.stdout:350 read 0x889b600 thru 0x88a6e2a (0xb82b bytes) 2024-09-17T21:18:26.433 INFO:teuthology.orchestra.run.smithi184.stdout:351 punch from 0x9221200 to 0x922b600, (0xa400 bytes) 2024-09-17T21:18:26.462 INFO:teuthology.orchestra.run.smithi184.stdout:355 write 0xc2a3200 thru 0xc2a4cdb (0x1adc bytes) 2024-09-17T21:18:26.469 INFO:teuthology.orchestra.run.smithi184.stdout:356 trunc from 0xc2a4e00 to 0x935fc00 2024-09-17T21:18:26.499 INFO:teuthology.orchestra.run.smithi184.stdout:357 punch from 0x5036000 to 0x5043a00, (0xda00 bytes) 2024-09-17T21:18:26.500 INFO:teuthology.orchestra.run.smithi184.stdout:359 read 0x8ecce00 thru 0x8ed0ccc (0x3ecd bytes) 2024-09-17T21:18:26.500 INFO:teuthology.orchestra.run.smithi184.stdout:364 read 0x7ef000 thru 0x7f3549 (0x454a bytes) 2024-09-17T21:18:26.500 INFO:teuthology.orchestra.run.smithi184.stdout:370 read 0x5fd5400 thru 0x5fdf785 (0xa386 bytes) 2024-09-17T21:18:26.501 INFO:teuthology.orchestra.run.smithi184.stdout:371 read 0x44fb800 thru 0x45087c5 (0xcfc6 bytes) 2024-09-17T21:18:26.503 INFO:teuthology.orchestra.run.smithi184.stdout:372 read 0x125800 thru 0x127715 (0x1f16 bytes) 2024-09-17T21:18:26.526 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:26.522+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:26.532 INFO:teuthology.orchestra.run.smithi184.stdout:373 trunc from 0x935fc00 to 0x2fad600 2024-09-17T21:18:26.577 INFO:teuthology.orchestra.run.smithi184.stdout:375 punch from 0xf8a200 to 0xf94a00, (0xa800 bytes) 2024-09-17T21:18:26.599 INFO:teuthology.orchestra.run.smithi184.stdout:377 write 0x49aec00 thru 0x49bce77 (0xe278 bytes) 2024-09-17T21:18:26.601 INFO:teuthology.orchestra.run.smithi184.stdout:378 punch from 0x45a5e00 to 0x45a8400, (0x2600 bytes) 2024-09-17T21:18:26.601 INFO:teuthology.orchestra.run.smithi184.stdout:381 read 0xe17e00 thru 0xe1f171 (0x7372 bytes) 2024-09-17T21:18:26.603 INFO:teuthology.orchestra.run.smithi184.stdout:382 write 0xaba400 thru 0xabaa1c (0x61d bytes) 2024-09-17T21:18:26.614 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:26.614 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:26.614 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:18:26.614 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:26.628 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:26.622+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:26.632 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:18:26.640 INFO:teuthology.orchestra.run.smithi184.stdout:391 write 0xcec7400 thru 0xced24df (0xb0e0 bytes) 2024-09-17T21:18:26.642 INFO:teuthology.orchestra.run.smithi184.stdout:392 trunc from 0xced2600 to 0xc493c00 2024-09-17T21:18:26.668 INFO:teuthology.orchestra.run.smithi184.stdout:393 write 0x77ec800 thru 0x77fb4f7 (0xecf8 bytes) 2024-09-17T21:18:26.681 INFO:teuthology.orchestra.run.smithi184.stdout:394 trunc from 0xc493c00 to 0x824d200 2024-09-17T21:18:26.719 INFO:teuthology.orchestra.run.smithi184.stdout:395 write 0xb0b7600 thru 0xb0c31a8 (0xbba9 bytes) 2024-09-17T21:18:26.721 INFO:teuthology.orchestra.run.smithi184.stdout:396 punch from 0x9858c00 to 0x9867c00, (0xf000 bytes) 2024-09-17T21:18:26.729 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:26.724+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:26.742 INFO:teuthology.orchestra.run.smithi184.stdout:397 write 0xea5b800 thru 0xea636d4 (0x7ed5 bytes) 2024-09-17T21:18:26.744 INFO:teuthology.orchestra.run.smithi184.stdout:398 write 0x37f3c00 thru 0x37fc1b2 (0x85b3 bytes) 2024-09-17T21:18:26.745 INFO:teuthology.orchestra.run.smithi184.stdout:399 read 0xd737e00 thru 0xd746b70 (0xed71 bytes) 2024-09-17T21:18:26.746 INFO:teuthology.orchestra.run.smithi184.stdout:400 trunc from 0xea63800 to 0xde2d200 2024-09-17T21:18:26.771 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:18:26.771 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:18:26.771 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-17T21:18:26.771 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T21:18:26.771 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T21:18:26.771 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-17T21:18:26.777 INFO:teuthology.orchestra.run.smithi184.stdout:403 read 0xaa95800 thru 0xaa99425 (0x3c26 bytes) 2024-09-17T21:18:26.780 INFO:teuthology.orchestra.run.smithi184.stdout:404 punch from 0x5d9a600 to 0x5da0000, (0x5a00 bytes) 2024-09-17T21:18:26.785 INFO:teuthology.orchestra.run.smithi184.stdout:405 trunc from 0xde2d200 to 0xbb68e00 2024-09-17T21:18:26.801 INFO:teuthology.orchestra.run.smithi184.stdout:406 write 0x2071e00 thru 0x207363e (0x183f bytes) 2024-09-17T21:18:26.805 INFO:teuthology.orchestra.run.smithi184.stdout:408 trunc from 0xbb68e00 to 0xc214a00 2024-09-17T21:18:26.810 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:26.810 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:26.810 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:18:26.810 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:18:26.810 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:26.822 INFO:teuthology.orchestra.run.smithi184.stdout:409 write 0x6c17600 thru 0x6c1879a (0x119b bytes) 2024-09-17T21:18:26.824 INFO:teuthology.orchestra.run.smithi184.stdout:410 write 0x9927600 thru 0x9932d7a (0xb77b bytes) 2024-09-17T21:18:26.828 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:18:26.829 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:26.824+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:26.857 INFO:teuthology.orchestra.run.smithi184.stdout:413 trunc from 0xc214a00 to 0x11f5e00 2024-09-17T21:18:26.909 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:26.910 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T21:18:26.910 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:26.926 INFO:teuthology.orchestra.run.smithi184.stdout:416 write 0x55d9c00 thru 0x55e2898 (0x8c99 bytes) 2024-09-17T21:18:26.929 INFO:teuthology.orchestra.run.smithi184.stdout:417 write 0xfaf400 thru 0xfb2152 (0x2d53 bytes) 2024-09-17T21:18:26.937 INFO:teuthology.orchestra.run.smithi184.stdout:418 trunc from 0x55e2a00 to 0x89e7200 2024-09-17T21:18:26.966 INFO:teuthology.orchestra.run.smithi184.stdout:422 trunc from 0x89e7200 to 0xc61e400 2024-09-17T21:18:26.969 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:26.966+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:26.978 INFO:teuthology.orchestra.run.smithi184.stdout:425 read 0x3e53e00 thru 0x3e5e6fa (0xa8fb bytes) 2024-09-17T21:18:26.979 INFO:teuthology.orchestra.run.smithi184.stdout:426 read 0x9e8a600 thru 0x9e98025 (0xda26 bytes) 2024-09-17T21:18:26.980 INFO:teuthology.orchestra.run.smithi184.stdout:427 write 0x393c000 thru 0x39466a2 (0xa6a3 bytes) 2024-09-17T21:18:26.980 INFO:teuthology.orchestra.run.smithi184.stdout:429 read 0x9616600 thru 0x9618b61 (0x2562 bytes) 2024-09-17T21:18:26.981 INFO:teuthology.orchestra.run.smithi184.stdout:430 punch from 0x2aee000 to 0x2af2a00, (0x4a00 bytes) 2024-09-17T21:18:26.982 INFO:teuthology.orchestra.run.smithi184.stdout:431 read 0x4921000 thru 0x49237dc (0x27dd bytes) 2024-09-17T21:18:26.983 INFO:teuthology.orchestra.run.smithi184.stdout:433 write 0x1a1ba00 thru 0x1a218d0 (0x5ed1 bytes) 2024-09-17T21:18:26.986 INFO:teuthology.orchestra.run.smithi184.stdout:434 read 0x6860400 thru 0x6861d27 (0x1928 bytes) 2024-09-17T21:18:26.986 INFO:teuthology.orchestra.run.smithi184.stdout:435 punch from 0x52c9400 to 0x52d3000, (0x9c00 bytes) 2024-09-17T21:18:26.987 INFO:teuthology.orchestra.run.smithi184.stdout:436 punch from 0x583b400 to 0x5840e00, (0x5a00 bytes) 2024-09-17T21:18:26.987 INFO:teuthology.orchestra.run.smithi184.stdout:438 punch from 0x7c53000 to 0x7c56c00, (0x3c00 bytes) 2024-09-17T21:18:26.987 INFO:teuthology.orchestra.run.smithi184.stdout:439 write 0xa7b4800 thru 0xa7bde75 (0x9676 bytes) 2024-09-17T21:18:26.987 INFO:teuthology.orchestra.run.smithi184.stdout:440 write 0xbb93400 thru 0xbb994f0 (0x60f1 bytes) 2024-09-17T21:18:27.004 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:27.004 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:18:27.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:18:27.004 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:27.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.005 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:27.005 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:13.306888+0000", 2024-09-17T21:18:27.005 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.695420581, 2024-09-17T21:18:27.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010487935, 2024-09-17T21:18:27.005 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.005 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.005 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.005 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.005 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:27.005 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:27.005 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 9 2024-09-17T21:18:27.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.005 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.306888+0000", 2024-09-17T21:18:27.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.306888+0000", 2024-09-17T21:18:27.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.306892+0000", 2024-09-17T21:18:27.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8460000000000001e-06 2024-09-17T21:18:27.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.306916+0000", 2024-09-17T21:18:27.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4522999999999999e-05 2024-09-17T21:18:27.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.306958+0000", 2024-09-17T21:18:27.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1909999999999997e-05 2024-09-17T21:18:27.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.306983+0000", 2024-09-17T21:18:27.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5035999999999999e-05 2024-09-17T21:18:27.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.307155+0000", 2024-09-17T21:18:27.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017227599999999999 2024-09-17T21:18:27.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:27.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.307174+0000", 2024-09-17T21:18:27.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8940000000000002e-05 2024-09-17T21:18:27.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.311885+0000", 2024-09-17T21:18:27.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0047109860000000003 2024-09-17T21:18:27.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.312129+0000", 2024-09-17T21:18:27.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024392500000000001 2024-09-17T21:18:27.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:18:27.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.312487+0000", 2024-09-17T21:18:27.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035794000000000001 2024-09-17T21:18:27.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:27.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.314290+0000", 2024-09-17T21:18:27.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001803303 2024-09-17T21:18:27.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:27.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.317157+0000", 2024-09-17T21:18:27.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028671220000000002 2024-09-17T21:18:27.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.317303+0000", 2024-09-17T21:18:27.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014547200000000001 2024-09-17T21:18:27.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:13.317376+0000", 2024-09-17T21:18:27.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2656000000000004e-05 2024-09-17T21:18:27.011 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.011 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.011 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.011 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:180 3.9 3:9797fd44:::rbd_object_map.1152c6af092e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:27.011 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.851441+0000", 2024-09-17T21:18:27.011 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.150867044, 2024-09-17T21:18:27.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011056965, 2024-09-17T21:18:27.012 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.012 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.012 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.012 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.012 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:27.012 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:27.012 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 180 2024-09-17T21:18:27.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.012 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.851441+0000", 2024-09-17T21:18:27.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.851441+0000", 2024-09-17T21:18:27.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.851443+0000", 2024-09-17T21:18:27.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.917e-06 2024-09-17T21:18:27.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.851455+0000", 2024-09-17T21:18:27.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1516e-05 2024-09-17T21:18:27.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.851495+0000", 2024-09-17T21:18:27.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9903000000000001e-05 2024-09-17T21:18:27.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.851512+0000", 2024-09-17T21:18:27.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7907e-05 2024-09-17T21:18:27.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.855062+0000", 2024-09-17T21:18:27.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035499360000000001 2024-09-17T21:18:27.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.855292+0000", 2024-09-17T21:18:27.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000229838 2024-09-17T21:18:27.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:27.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.856353+0000", 2024-09-17T21:18:27.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0010605219999999999 2024-09-17T21:18:27.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:27.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.862322+0000", 2024-09-17T21:18:27.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059689970000000002 2024-09-17T21:18:27.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:27.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.862344+0000", 2024-09-17T21:18:27.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2223000000000001e-05 2024-09-17T21:18:27.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.862420+0000", 2024-09-17T21:18:27.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5562999999999995e-05 2024-09-17T21:18:27.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.862498+0000", 2024-09-17T21:18:27.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8642999999999999e-05 2024-09-17T21:18:27.017 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.017 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.017 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.017 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:181 3.9 3:9797fd44:::rbd_object_map.1152c6af092e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:27.017 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.851564+0000", 2024-09-17T21:18:27.017 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.1507447249999991, 2024-09-17T21:18:27.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01168189, 2024-09-17T21:18:27.018 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.018 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.018 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.018 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.018 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:27.018 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:27.018 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 181 2024-09-17T21:18:27.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.018 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.851564+0000", 2024-09-17T21:18:27.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.851564+0000", 2024-09-17T21:18:27.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.851565+0000", 2024-09-17T21:18:27.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6640000000000001e-06 2024-09-17T21:18:27.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.851573+0000", 2024-09-17T21:18:27.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2660000000000001e-06 2024-09-17T21:18:27.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.851594+0000", 2024-09-17T21:18:27.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0769999999999999e-05 2024-09-17T21:18:27.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.851608+0000", 2024-09-17T21:18:27.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4202999999999999e-05 2024-09-17T21:18:27.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.857823+0000", 2024-09-17T21:18:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006214369 2024-09-17T21:18:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.858080+0000", 2024-09-17T21:18:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025708099999999999 2024-09-17T21:18:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.859105+0000", 2024-09-17T21:18:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0010255889999999999 2024-09-17T21:18:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.862608+0000", 2024-09-17T21:18:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035022920000000002 2024-09-17T21:18:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.863048+0000", 2024-09-17T21:18:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00043999099999999998 2024-09-17T21:18:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.863123+0000", 2024-09-17T21:18:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4806000000000002e-05 2024-09-17T21:18:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.863245+0000", 2024-09-17T21:18:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012285900000000001 2024-09-17T21:18:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:183 3.9 3:9797fd44:::rbd_object_map.1152c6af092e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.852505+0000", 2024-09-17T21:18:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.1498028920000003, 2024-09-17T21:18:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012063043000000001, 2024-09-17T21:18:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 183 2024-09-17T21:18:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.852505+0000", 2024-09-17T21:18:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.852505+0000", 2024-09-17T21:18:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.852508+0000", 2024-09-17T21:18:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2630000000000002e-06 2024-09-17T21:18:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.852519+0000", 2024-09-17T21:18:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.172e-05 2024-09-17T21:18:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.852550+0000", 2024-09-17T21:18:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0202999999999999e-05 2024-09-17T21:18:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.852565+0000", 2024-09-17T21:18:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5254e-05 2024-09-17T21:18:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.860240+0000", 2024-09-17T21:18:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0076750339999999998 2024-09-17T21:18:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.860490+0000", 2024-09-17T21:18:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024978499999999998 2024-09-17T21:18:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.861233+0000", 2024-09-17T21:18:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00074317399999999998 2024-09-17T21:18:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.864236+0000", 2024-09-17T21:18:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0030035800000000001 2024-09-17T21:18:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.864378+0000", 2024-09-17T21:18:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014144200000000001 2024-09-17T21:18:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.864491+0000", 2024-09-17T21:18:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113582 2024-09-17T21:18:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.864568+0000", 2024-09-17T21:18:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7006000000000001e-05 2024-09-17T21:18:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:184 3.9 3:92888d62:::rbd_data.1152c01721e5.0000000000000034:3 [read 2891776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.854146+0000", 2024-09-17T21:18:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.1481624010000004, 2024-09-17T21:18:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0091124589999999998, 2024-09-17T21:18:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 184 2024-09-17T21:18:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.854146+0000", 2024-09-17T21:18:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.854146+0000", 2024-09-17T21:18:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.854148+0000", 2024-09-17T21:18:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:18:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.854160+0000", 2024-09-17T21:18:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2123999999999999e-05 2024-09-17T21:18:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.854213+0000", 2024-09-17T21:18:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2156e-05 2024-09-17T21:18:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.854238+0000", 2024-09-17T21:18:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5153000000000001e-05 2024-09-17T21:18:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.862708+0000", 2024-09-17T21:18:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0084699270000000004 2024-09-17T21:18:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.862864+0000", 2024-09-17T21:18:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015633199999999999 2024-09-17T21:18:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.863258+0000", 2024-09-17T21:18:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00039436400000000002 2024-09-17T21:18:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:185 3.9 3:92888d62:::rbd_data.1152c01721e5.0000000000000034:3 [read 3018752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.854275+0000", 2024-09-17T21:18:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.1480330490000004, 2024-09-17T21:18:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095492029999999992, 2024-09-17T21:18:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 185 2024-09-17T21:18:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.854275+0000", 2024-09-17T21:18:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.854275+0000", 2024-09-17T21:18:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.854277+0000", 2024-09-17T21:18:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.457e-06 2024-09-17T21:18:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.854284+0000", 2024-09-17T21:18:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.357e-06 2024-09-17T21:18:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.854310+0000", 2024-09-17T21:18:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6452999999999999e-05 2024-09-17T21:18:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.854328+0000", 2024-09-17T21:18:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7365999999999999e-05 2024-09-17T21:18:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.863365+0000", 2024-09-17T21:18:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0090375330000000004 2024-09-17T21:18:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.863509+0000", 2024-09-17T21:18:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014347200000000001 2024-09-17T21:18:27.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.863824+0000", 2024-09-17T21:18:27.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031556500000000002 2024-09-17T21:18:27.040 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.040 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.040 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:186 3.9 3:92888d62:::rbd_data.1152c01721e5.0000000000000034:3 [read 2904064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.854349+0000", 2024-09-17T21:18:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.1479596989999994, 2024-09-17T21:18:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0098944359999999995, 2024-09-17T21:18:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 186 2024-09-17T21:18:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.854349+0000", 2024-09-17T21:18:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.854349+0000", 2024-09-17T21:18:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.854350+0000", 2024-09-17T21:18:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9600000000000008e-07 2024-09-17T21:18:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.854354+0000", 2024-09-17T21:18:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.194e-06 2024-09-17T21:18:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.854370+0000", 2024-09-17T21:18:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6538999999999999e-05 2024-09-17T21:18:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.854405+0000", 2024-09-17T21:18:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4507e-05 2024-09-17T21:18:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.863919+0000", 2024-09-17T21:18:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095144830000000007 2024-09-17T21:18:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.864036+0000", 2024-09-17T21:18:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000116889 2024-09-17T21:18:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.864243+0000", 2024-09-17T21:18:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020682799999999999 2024-09-17T21:18:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:242 3.9 3:9797fd44:::rbd_object_map.1152c6af092e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.912155+0000", 2024-09-17T21:18:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0901537290000007, 2024-09-17T21:18:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0071028949999999997, 2024-09-17T21:18:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 242 2024-09-17T21:18:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.912155+0000", 2024-09-17T21:18:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.912155+0000", 2024-09-17T21:18:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.912155+0000", 2024-09-17T21:18:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.16e-07 2024-09-17T21:18:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.912161+0000", 2024-09-17T21:18:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4929999999999998e-06 2024-09-17T21:18:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.912170+0000", 2024-09-17T21:18:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1130000000000007e-06 2024-09-17T21:18:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.912177+0000", 2024-09-17T21:18:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.652e-06 2024-09-17T21:18:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.913948+0000", 2024-09-17T21:18:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017715840000000001 2024-09-17T21:18:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.914005+0000", 2024-09-17T21:18:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6387000000000001e-05 2024-09-17T21:18:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.914228+0000", 2024-09-17T21:18:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000223126 2024-09-17T21:18:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.917117+0000", 2024-09-17T21:18:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028896550000000001 2024-09-17T21:18:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.919203+0000", 2024-09-17T21:18:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002085866 2024-09-17T21:18:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.919234+0000", 2024-09-17T21:18:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0972000000000001e-05 2024-09-17T21:18:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.919257+0000", 2024-09-17T21:18:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3230999999999998e-05 2024-09-17T21:18:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:243 3.9 3:9797fd44:::rbd_object_map.1152c6af092e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.912187+0000", 2024-09-17T21:18:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0901209929999993, 2024-09-17T21:18:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0071289329999999996, 2024-09-17T21:18:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 243 2024-09-17T21:18:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.912187+0000", 2024-09-17T21:18:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.912187+0000", 2024-09-17T21:18:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.912188+0000", 2024-09-17T21:18:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3499999999999996e-07 2024-09-17T21:18:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.912191+0000", 2024-09-17T21:18:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1760000000000002e-06 2024-09-17T21:18:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.912198+0000", 2024-09-17T21:18:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3470000000000001e-06 2024-09-17T21:18:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.912205+0000", 2024-09-17T21:18:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.798e-06 2024-09-17T21:18:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.914532+0000", 2024-09-17T21:18:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0023266599999999999 2024-09-17T21:18:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.914589+0000", 2024-09-17T21:18:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6617999999999999e-05 2024-09-17T21:18:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.914812+0000", 2024-09-17T21:18:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022364800000000001 2024-09-17T21:18:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.917794+0000", 2024-09-17T21:18:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002982139 2024-09-17T21:18:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.919262+0000", 2024-09-17T21:18:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001467896 2024-09-17T21:18:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.919295+0000", 2024-09-17T21:18:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:18:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.919316+0000", 2024-09-17T21:18:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1554000000000001e-05 2024-09-17T21:18:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:354 3.9 3:9797fd44:::rbd_object_map.1152c6af092e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:17.975175+0000", 2024-09-17T21:18:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0271331749999995, 2024-09-17T21:18:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0071922499999999999, 2024-09-17T21:18:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 354 2024-09-17T21:18:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.975175+0000", 2024-09-17T21:18:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.975175+0000", 2024-09-17T21:18:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.975177+0000", 2024-09-17T21:18:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.88e-06 2024-09-17T21:18:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.975195+0000", 2024-09-17T21:18:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8379000000000001e-05 2024-09-17T21:18:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.975222+0000", 2024-09-17T21:18:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7132999999999999e-05 2024-09-17T21:18:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.975240+0000", 2024-09-17T21:18:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7496999999999999e-05 2024-09-17T21:18:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.977149+0000", 2024-09-17T21:18:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00190883 2024-09-17T21:18:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.977233+0000", 2024-09-17T21:18:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4615000000000002e-05 2024-09-17T21:18:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.977569+0000", 2024-09-17T21:18:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033551099999999998 2024-09-17T21:18:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.978430+0000", 2024-09-17T21:18:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00086076900000000005 2024-09-17T21:18:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.982276+0000", 2024-09-17T21:18:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038463009999999999 2024-09-17T21:18:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.982328+0000", 2024-09-17T21:18:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1576000000000002e-05 2024-09-17T21:18:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.982367+0000", 2024-09-17T21:18:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9759000000000003e-05 2024-09-17T21:18:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:782 3.9 3:9797fd44:::rbd_object_map.1152c6af092e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.213866+0000", 2024-09-17T21:18:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.7884427370000004, 2024-09-17T21:18:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0077229159999999998, 2024-09-17T21:18:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 782 2024-09-17T21:18:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.213866+0000", 2024-09-17T21:18:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.213866+0000", 2024-09-17T21:18:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.213866+0000", 2024-09-17T21:18:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1500000000000003e-07 2024-09-17T21:18:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.213871+0000", 2024-09-17T21:18:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.724e-06 2024-09-17T21:18:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.213892+0000", 2024-09-17T21:18:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0667999999999999e-05 2024-09-17T21:18:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.213900+0000", 2024-09-17T21:18:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5539999999999998e-06 2024-09-17T21:18:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.214564+0000", 2024-09-17T21:18:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066420400000000005 2024-09-17T21:18:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.214632+0000", 2024-09-17T21:18:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7317999999999995e-05 2024-09-17T21:18:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.214980+0000", 2024-09-17T21:18:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00034822400000000001 2024-09-17T21:18:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.217897+0000", 2024-09-17T21:18:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002917008 2024-09-17T21:18:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.221497+0000", 2024-09-17T21:18:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036003950000000002 2024-09-17T21:18:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.221549+0000", 2024-09-17T21:18:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1678000000000002e-05 2024-09-17T21:18:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.221588+0000", 2024-09-17T21:18:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9328e-05 2024-09-17T21:18:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:783 3.9 3:9797fd44:::rbd_object_map.1152c6af092e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.213916+0000", 2024-09-17T21:18:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.7883919309999996, 2024-09-17T21:18:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0078090520000000004, 2024-09-17T21:18:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 783 2024-09-17T21:18:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.213916+0000", 2024-09-17T21:18:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.213916+0000", 2024-09-17T21:18:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.213917+0000", 2024-09-17T21:18:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1900000000000002e-07 2024-09-17T21:18:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.213921+0000", 2024-09-17T21:18:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7869999999999999e-06 2024-09-17T21:18:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.213930+0000", 2024-09-17T21:18:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6589999999999996e-06 2024-09-17T21:18:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.213936+0000", 2024-09-17T21:18:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3980000000000004e-06 2024-09-17T21:18:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.215350+0000", 2024-09-17T21:18:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00141424 2024-09-17T21:18:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.215424+0000", 2024-09-17T21:18:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3731000000000003e-05 2024-09-17T21:18:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.215673+0000", 2024-09-17T21:18:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024957100000000003 2024-09-17T21:18:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.217900+0000", 2024-09-17T21:18:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002226572 2024-09-17T21:18:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.221643+0000", 2024-09-17T21:18:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0037430549999999999 2024-09-17T21:18:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.221682+0000", 2024-09-17T21:18:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8574e-05 2024-09-17T21:18:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.221725+0000", 2024-09-17T21:18:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3745999999999998e-05 2024-09-17T21:18:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:785 3.9 3:9797fd44:::rbd_object_map.1152c6af092e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.213968+0000", 2024-09-17T21:18:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.7883398019999994, 2024-09-17T21:18:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0078991500000000006, 2024-09-17T21:18:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 785 2024-09-17T21:18:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.213968+0000", 2024-09-17T21:18:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.213968+0000", 2024-09-17T21:18:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.213969+0000", 2024-09-17T21:18:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.44e-07 2024-09-17T21:18:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.213973+0000", 2024-09-17T21:18:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3819999999999997e-06 2024-09-17T21:18:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.213984+0000", 2024-09-17T21:18:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0944000000000001e-05 2024-09-17T21:18:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.213991+0000", 2024-09-17T21:18:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5230000000000001e-06 2024-09-17T21:18:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.215999+0000", 2024-09-17T21:18:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0020081550000000002 2024-09-17T21:18:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.216078+0000", 2024-09-17T21:18:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9420999999999997e-05 2024-09-17T21:18:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.216291+0000", 2024-09-17T21:18:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021295099999999999 2024-09-17T21:18:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.217905+0000", 2024-09-17T21:18:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001613333 2024-09-17T21:18:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.221770+0000", 2024-09-17T21:18:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038654700000000002 2024-09-17T21:18:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.221821+0000", 2024-09-17T21:18:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1298000000000003e-05 2024-09-17T21:18:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.221868+0000", 2024-09-17T21:18:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6128999999999998e-05 2024-09-17T21:18:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:786 3.9 3:9797fd44:::rbd_object_map.1152c6af092e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.214004+0000", 2024-09-17T21:18:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.7883047100000002, 2024-09-17T21:18:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0096441779999999998, 2024-09-17T21:18:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 786 2024-09-17T21:18:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.214004+0000", 2024-09-17T21:18:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.214004+0000", 2024-09-17T21:18:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.214004+0000", 2024-09-17T21:18:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0900000000000002e-07 2024-09-17T21:18:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.214011+0000", 2024-09-17T21:18:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9280000000000004e-06 2024-09-17T21:18:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.214019+0000", 2024-09-17T21:18:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9540000000000004e-06 2024-09-17T21:18:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.214025+0000", 2024-09-17T21:18:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5790000000000003e-06 2024-09-17T21:18:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.216619+0000", 2024-09-17T21:18:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0025941010000000001 2024-09-17T21:18:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.216679+0000", 2024-09-17T21:18:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0751000000000002e-05 2024-09-17T21:18:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.216973+0000", 2024-09-17T21:18:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029319799999999999 2024-09-17T21:18:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.217909+0000", 2024-09-17T21:18:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00093656900000000005 2024-09-17T21:18:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.223564+0000", 2024-09-17T21:18:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0056548320000000003 2024-09-17T21:18:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.223602+0000", 2024-09-17T21:18:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7830000000000002e-05 2024-09-17T21:18:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.223648+0000", 2024-09-17T21:18:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5927000000000003e-05 2024-09-17T21:18:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:787 3.9 3:9797fd44:::rbd_object_map.1152c6af092e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.214034+0000", 2024-09-17T21:18:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.7882747600000002, 2024-09-17T21:18:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010409554, 2024-09-17T21:18:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 787 2024-09-17T21:18:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.214034+0000", 2024-09-17T21:18:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.214034+0000", 2024-09-17T21:18:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.214034+0000", 2024-09-17T21:18:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2099999999999997e-07 2024-09-17T21:18:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.214037+0000", 2024-09-17T21:18:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.219e-06 2024-09-17T21:18:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.214048+0000", 2024-09-17T21:18:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0261000000000001e-05 2024-09-17T21:18:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.214054+0000", 2024-09-17T21:18:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.28e-06 2024-09-17T21:18:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.217318+0000", 2024-09-17T21:18:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0032637209999999998 2024-09-17T21:18:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.217386+0000", 2024-09-17T21:18:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8590000000000006e-05 2024-09-17T21:18:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.217594+0000", 2024-09-17T21:18:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000207987 2024-09-17T21:18:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.218578+0000", 2024-09-17T21:18:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00098424499999999991 2024-09-17T21:18:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.224306+0000", 2024-09-17T21:18:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057278149999999998 2024-09-17T21:18:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.224394+0000", 2024-09-17T21:18:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8041000000000006e-05 2024-09-17T21:18:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.224443+0000", 2024-09-17T21:18:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8874e-05 2024-09-17T21:18:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:788 3.9 3:9797fd44:::rbd_object_map.1152c6af092e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.214066+0000", 2024-09-17T21:18:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.7882418500000004, 2024-09-17T21:18:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010550551, 2024-09-17T21:18:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 788 2024-09-17T21:18:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.214066+0000", 2024-09-17T21:18:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.214066+0000", 2024-09-17T21:18:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.214067+0000", 2024-09-17T21:18:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.89e-07 2024-09-17T21:18:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.214070+0000", 2024-09-17T21:18:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5719999999999999e-06 2024-09-17T21:18:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.214080+0000", 2024-09-17T21:18:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5319999999999995e-06 2024-09-17T21:18:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.214086+0000", 2024-09-17T21:18:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9830000000000001e-06 2024-09-17T21:18:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.217934+0000", 2024-09-17T21:18:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038484579999999999 2024-09-17T21:18:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.218003+0000", 2024-09-17T21:18:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8194999999999993e-05 2024-09-17T21:18:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.218240+0000", 2024-09-17T21:18:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000237216 2024-09-17T21:18:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.218995+0000", 2024-09-17T21:18:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00075540899999999996 2024-09-17T21:18:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.224508+0000", 2024-09-17T21:18:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005512421 2024-09-17T21:18:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.224579+0000", 2024-09-17T21:18:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0970000000000007e-05 2024-09-17T21:18:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.224617+0000", 2024-09-17T21:18:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8306000000000002e-05 2024-09-17T21:18:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:882 3.9 3:9797fd44:::rbd_object_map.1152c6af092e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.266537+0000", 2024-09-17T21:18:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.7357709470000007, 2024-09-17T21:18:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0070884880000000004, 2024-09-17T21:18:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 882 2024-09-17T21:18:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.266537+0000", 2024-09-17T21:18:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.266537+0000", 2024-09-17T21:18:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.266538+0000", 2024-09-17T21:18:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:18:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.266540+0000", 2024-09-17T21:18:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2089999999999999e-06 2024-09-17T21:18:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.266545+0000", 2024-09-17T21:18:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8949999999999997e-06 2024-09-17T21:18:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.266551+0000", 2024-09-17T21:18:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7479999999999999e-06 2024-09-17T21:18:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.267765+0000", 2024-09-17T21:18:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0012144079999999999 2024-09-17T21:18:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.267820+0000", 2024-09-17T21:18:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4947000000000001e-05 2024-09-17T21:18:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.268028+0000", 2024-09-17T21:18:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000208283 2024-09-17T21:18:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.271631+0000", 2024-09-17T21:18:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036027630000000001 2024-09-17T21:18:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.273570+0000", 2024-09-17T21:18:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0019394729999999999 2024-09-17T21:18:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.273604+0000", 2024-09-17T21:18:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3946999999999999e-05 2024-09-17T21:18:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.273626+0000", 2024-09-17T21:18:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1441e-05 2024-09-17T21:18:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:883 3.9 3:9797fd44:::rbd_object_map.1152c6af092e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.266559+0000", 2024-09-17T21:18:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.735748869, 2024-09-17T21:18:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0071147470000000003, 2024-09-17T21:18:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 883 2024-09-17T21:18:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.266559+0000", 2024-09-17T21:18:27.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.266559+0000", 2024-09-17T21:18:27.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.266560+0000", 2024-09-17T21:18:27.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1199999999999998e-07 2024-09-17T21:18:27.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.266565+0000", 2024-09-17T21:18:27.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6239999999999999e-06 2024-09-17T21:18:27.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.266570+0000", 2024-09-17T21:18:27.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.566e-06 2024-09-17T21:18:27.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.266574+0000", 2024-09-17T21:18:27.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.512e-06 2024-09-17T21:18:27.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.268313+0000", 2024-09-17T21:18:27.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001739892 2024-09-17T21:18:27.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.268366+0000", 2024-09-17T21:18:27.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2383999999999999e-05 2024-09-17T21:18:27.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:27.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.268573+0000", 2024-09-17T21:18:27.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020724900000000001 2024-09-17T21:18:27.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:27.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.271915+0000", 2024-09-17T21:18:27.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033420820000000001 2024-09-17T21:18:27.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:27.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.273631+0000", 2024-09-17T21:18:27.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001715486 2024-09-17T21:18:27.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.273655+0000", 2024-09-17T21:18:27.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4861000000000001e-05 2024-09-17T21:18:27.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.273674+0000", 2024-09-17T21:18:27.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8678999999999999e-05 2024-09-17T21:18:27.111 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.111 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.111 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.111 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4423.0:884 3.9 3:9797fd44:::rbd_object_map.1152c6af092e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:18:27.111 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:18.266578+0000", 2024-09-17T21:18:27.111 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.7357299780000002, 2024-09-17T21:18:27.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0086558069999999997, 2024-09-17T21:18:27.112 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.112 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.112 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.112 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.112 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4423", 2024-09-17T21:18:27.112 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1812572179", 2024-09-17T21:18:27.112 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 884 2024-09-17T21:18:27.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.112 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.266578+0000", 2024-09-17T21:18:27.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.266578+0000", 2024-09-17T21:18:27.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.266579+0000", 2024-09-17T21:18:27.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0499999999999999e-07 2024-09-17T21:18:27.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.266581+0000", 2024-09-17T21:18:27.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7470000000000001e-06 2024-09-17T21:18:27.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.266589+0000", 2024-09-17T21:18:27.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7400000000000004e-06 2024-09-17T21:18:27.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.266595+0000", 2024-09-17T21:18:27.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6509999999999998e-06 2024-09-17T21:18:27.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.268863+0000", 2024-09-17T21:18:27.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022679000000000002 2024-09-17T21:18:27.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.268913+0000", 2024-09-17T21:18:27.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0294999999999999e-05 2024-09-17T21:18:27.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:27.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.269117+0000", 2024-09-17T21:18:27.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020392699999999999 2024-09-17T21:18:27.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:27.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.273680+0000", 2024-09-17T21:18:27.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045627719999999997 2024-09-17T21:18:27.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:27.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.275175+0000", 2024-09-17T21:18:27.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001495389 2024-09-17T21:18:27.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.275209+0000", 2024-09-17T21:18:27.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3531999999999997e-05 2024-09-17T21:18:27.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:18.275234+0000", 2024-09-17T21:18:27.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5548999999999999e-05 2024-09-17T21:18:27.117 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.117 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.117 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.117 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:578 3.3 3:c651b0ce:::rbd_data.1152b51cb396.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:27.117 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.222156+0000", 2024-09-17T21:18:27.117 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.78015195299999995, 2024-09-17T21:18:27.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013260160999999999, 2024-09-17T21:18:27.117 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.117 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.117 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.118 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.118 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:27.118 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:27.118 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 578 2024-09-17T21:18:27.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.118 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.222156+0000", 2024-09-17T21:18:27.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.222156+0000", 2024-09-17T21:18:27.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.222159+0000", 2024-09-17T21:18:27.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6070000000000002e-06 2024-09-17T21:18:27.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.224314+0000", 2024-09-17T21:18:27.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021548439999999999 2024-09-17T21:18:27.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.224340+0000", 2024-09-17T21:18:27.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6483999999999999e-05 2024-09-17T21:18:27.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.224353+0000", 2024-09-17T21:18:27.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2316e-05 2024-09-17T21:18:27.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.224437+0000", 2024-09-17T21:18:27.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4914999999999996e-05 2024-09-17T21:18:27.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.224607+0000", 2024-09-17T21:18:27.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016928900000000001 2024-09-17T21:18:27.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:27.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.224863+0000", 2024-09-17T21:18:27.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025646499999999999 2024-09-17T21:18:27.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:27.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.228024+0000", 2024-09-17T21:18:27.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031604269999999999 2024-09-17T21:18:27.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:27.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.235158+0000", 2024-09-17T21:18:27.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0071339280000000003 2024-09-17T21:18:27.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.235305+0000", 2024-09-17T21:18:27.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000147751 2024-09-17T21:18:27.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.235416+0000", 2024-09-17T21:18:27.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111135 2024-09-17T21:18:27.122 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.123 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.123 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.123 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.123 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.123 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:27.125 INFO:teuthology.orchestra.run.smithi184.stdout:441 trunc from 0xc61e400 to 0xf40200 2024-09-17T21:18:27.125 INFO:teuthology.orchestra.run.smithi184.stdout:442 trunc from 0xf40200 to 0xa7b1200 2024-09-17T21:18:27.125 INFO:teuthology.orchestra.run.smithi184.stdout:444 read 0x1289c00 thru 0x1290f26 (0x7327 bytes) 2024-09-17T21:18:27.125 INFO:teuthology.orchestra.run.smithi184.stdout:445 write 0x9bfa400 thru 0x9c03dc6 (0x99c7 bytes) 2024-09-17T21:18:27.125 INFO:teuthology.orchestra.run.smithi184.stdout:446 trunc from 0xa7b1200 to 0x8fed200 2024-09-17T21:18:27.125 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:18:27.126 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:27.120+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:27.129 INFO:teuthology.orchestra.run.smithi184.stdout:447 trunc from 0x8fed200 to 0xdabb800 2024-09-17T21:18:27.150 INFO:teuthology.orchestra.run.smithi184.stdout:448 flatten 2024-09-17T21:18:27.173 INFO:teuthology.orchestra.run.smithi184.stdout:450 read 0xca9d200 thru 0xcaa1d0d (0x4b0e bytes) 2024-09-17T21:18:27.173 INFO:teuthology.orchestra.run.smithi184.stdout:451 read 0x8fd2a00 thru 0x8fddf17 (0xb518 bytes) 2024-09-17T21:18:27.173 INFO:teuthology.orchestra.run.smithi184.stdout:452 write 0x8a4bc00 thru 0x8a5421d (0x861e bytes) 2024-09-17T21:18:27.173 INFO:teuthology.orchestra.run.smithi184.stdout:453 clone 5 order 19 su 32768 sc 8 2024-09-17T21:18:27.225 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:27.220+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:27.292 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:27.292 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:27.292 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:18:27.292 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:27.308 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:18:27.326 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:27.320+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:27.427 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:27.420+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:27.430 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:27.430 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:27.430 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:18:27.430 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:18:27.430 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:27.444 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:18:27.528 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:27.522+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:27.569 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:27.569 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:18:27.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:18:27.569 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:27.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.570 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4190.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-17T21:18:27.570 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:03.060815+0000", 2024-09-17T21:18:27.570 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.505346255999999, 2024-09-17T21:18:27.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011124612000000001, 2024-09-17T21:18:27.570 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.570 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.570 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.570 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.570 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4190", 2024-09-17T21:18:27.570 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.83:0/956669820", 2024-09-17T21:18:27.570 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 7 2024-09-17T21:18:27.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.570 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060815+0000", 2024-09-17T21:18:27.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060815+0000", 2024-09-17T21:18:27.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060819+0000", 2024-09-17T21:18:27.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2640000000000002e-06 2024-09-17T21:18:27.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060838+0000", 2024-09-17T21:18:27.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9406e-05 2024-09-17T21:18:27.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060878+0000", 2024-09-17T21:18:27.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0120000000000002e-05 2024-09-17T21:18:27.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060896+0000", 2024-09-17T21:18:27.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7986999999999999e-05 2024-09-17T21:18:27.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.061030+0000", 2024-09-17T21:18:27.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013395200000000001 2024-09-17T21:18:27.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:27.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.061041+0000", 2024-09-17T21:18:27.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1053000000000001e-05 2024-09-17T21:18:27.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.068612+0000", 2024-09-17T21:18:27.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0075704049999999997 2024-09-17T21:18:27.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.068806+0000", 2024-09-17T21:18:27.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019402500000000001 2024-09-17T21:18:27.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:18:27.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.069031+0000", 2024-09-17T21:18:27.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022502199999999999 2024-09-17T21:18:27.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:27.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.070704+0000", 2024-09-17T21:18:27.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001672974 2024-09-17T21:18:27.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:27.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071795+0000", 2024-09-17T21:18:27.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001091648 2024-09-17T21:18:27.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071878+0000", 2024-09-17T21:18:27.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2225999999999995e-05 2024-09-17T21:18:27.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071939+0000", 2024-09-17T21:18:27.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1530000000000002e-05 2024-09-17T21:18:27.576 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.576 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.576 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.576 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:18:27.576 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:18:27.577 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.9619704799999997, 2024-09-17T21:18:27.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:18:27.577 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.577 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.577 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.577 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:18:27.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:18:27.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:18:27.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:18:27.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:18:27.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:18:27.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:18:27.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:18:27.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:18:27.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:18:27.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:18:27.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:18:27.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:18:27.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:18:27.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:18:27.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:18:27.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:18:27.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:18:27.581 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.581 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.581 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.581 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:18:27.581 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:18:27.581 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.9520238440000002, 2024-09-17T21:18:27.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:18:27.581 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.581 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.581 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.581 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:18:27.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:18:27.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:18:27.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:18:27.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:18:27.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:18:27.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:18:27.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:18:27.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:18:27.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:18:27.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:18:27.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:18:27.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:18:27.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:18:27.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:18:27.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:18:27.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:18:27.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:18:27.585 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.585 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.585 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.586 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:532 3.2 e30/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 30'7, mlcod=29'4)", 2024-09-17T21:18:27.586 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.851302+0000", 2024-09-17T21:18:27.586 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.7148593309999995, 2024-09-17T21:18:27.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099840699999999994, 2024-09-17T21:18:27.586 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.586 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.586 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.586 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.851302+0000", 2024-09-17T21:18:27.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.851302+0000", 2024-09-17T21:18:27.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.851302+0000", 2024-09-17T21:18:27.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.37e-07 2024-09-17T21:18:27.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.851311+0000", 2024-09-17T21:18:27.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4230000000000005e-06 2024-09-17T21:18:27.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.851332+0000", 2024-09-17T21:18:27.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1549999999999999e-05 2024-09-17T21:18:27.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.851360+0000", 2024-09-17T21:18:27.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8314e-05 2024-09-17T21:18:27.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.851488+0000", 2024-09-17T21:18:27.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012728199999999999 2024-09-17T21:18:27.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.851527+0000", 2024-09-17T21:18:27.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8785999999999998e-05 2024-09-17T21:18:27.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.861217+0000", 2024-09-17T21:18:27.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0096907790000000001 2024-09-17T21:18:27.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.861286+0000", 2024-09-17T21:18:27.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8299000000000005e-05 2024-09-17T21:18:27.590 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.590 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.590 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.591 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:18:27.591 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:18:27.591 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.708873004, 2024-09-17T21:18:27.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:18:27.591 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.591 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.591 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.591 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.591 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:18:27.591 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:27.591 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:18:27.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.591 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:18:27.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:18:27.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:18:27.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:18:27.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:18:27.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:18:27.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:18:27.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:18:27.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:18:27.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:18:27.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:18:27.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:18:27.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:18:27.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:18:27.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:27.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:18:27.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:18:27.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:27.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:18:27.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:18:27.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:27.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:18:27.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:18:27.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:18:27.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:18:27.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:18:27.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:18:27.597 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.597 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.597 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.597 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:18:27.597 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:18:27.598 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.6992275179999998, 2024-09-17T21:18:27.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:18:27.598 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.598 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.598 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.598 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.598 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:18:27.598 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:27.598 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:18:27.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.598 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:18:27.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:18:27.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:18:27.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:18:27.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:18:27.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:18:27.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:18:27.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:18:27.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:18:27.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:18:27.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:18:27.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:18:27.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:18:27.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:18:27.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:27.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:18:27.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:18:27.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:27.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:18:27.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:18:27.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:27.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:18:27.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:18:27.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:18:27.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:18:27.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:18:27.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:18:27.603 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.603 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.603 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.603 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:378 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 2621440~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:27.603 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.194494+0000", 2024-09-17T21:18:27.603 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.3716664060000001, 2024-09-17T21:18:27.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010449484, 2024-09-17T21:18:27.604 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.604 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.604 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:27.604 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.604 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:27.604 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:27.604 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 378 2024-09-17T21:18:27.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.604 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.194494+0000", 2024-09-17T21:18:27.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.194494+0000", 2024-09-17T21:18:27.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.194496+0000", 2024-09-17T21:18:27.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.15e-06 2024-09-17T21:18:27.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.194502+0000", 2024-09-17T21:18:27.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7120000000000003e-06 2024-09-17T21:18:27.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.194511+0000", 2024-09-17T21:18:27.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8310000000000007e-06 2024-09-17T21:18:27.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.194518+0000", 2024-09-17T21:18:27.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0260000000000002e-06 2024-09-17T21:18:27.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.203849+0000", 2024-09-17T21:18:27.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093307170000000005 2024-09-17T21:18:27.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.203944+0000", 2024-09-17T21:18:27.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4769e-05 2024-09-17T21:18:27.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.204944+0000", 2024-09-17T21:18:27.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001000279 2024-09-17T21:18:27.607 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.607 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.607 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.608 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:379 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 3538944~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:27.608 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.194529+0000", 2024-09-17T21:18:27.608 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.371631507, 2024-09-17T21:18:27.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010866806, 2024-09-17T21:18:27.608 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.608 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.608 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:27.608 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.608 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:27.608 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:27.608 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 379 2024-09-17T21:18:27.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.608 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.194529+0000", 2024-09-17T21:18:27.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.194529+0000", 2024-09-17T21:18:27.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.194530+0000", 2024-09-17T21:18:27.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5099999999999999e-07 2024-09-17T21:18:27.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.610 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.610 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.194533+0000", 2024-09-17T21:18:27.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.18e-06 2024-09-17T21:18:27.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.610 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.610 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.194543+0000", 2024-09-17T21:18:27.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.893e-06 2024-09-17T21:18:27.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.610 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.610 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.194550+0000", 2024-09-17T21:18:27.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1749999999999999e-06 2024-09-17T21:18:27.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.204637+0000", 2024-09-17T21:18:27.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010086797999999999 2024-09-17T21:18:27.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.204770+0000", 2024-09-17T21:18:27.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000132929 2024-09-17T21:18:27.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.205396+0000", 2024-09-17T21:18:27.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00062608 2024-09-17T21:18:27.612 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.612 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.612 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.612 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:407 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:27.612 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.195241+0000", 2024-09-17T21:18:27.612 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.3709201069999999, 2024-09-17T21:18:27.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012283132, 2024-09-17T21:18:27.612 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.612 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.612 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:27.612 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.613 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:27.613 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:27.613 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 407 2024-09-17T21:18:27.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.613 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195241+0000", 2024-09-17T21:18:27.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195241+0000", 2024-09-17T21:18:27.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.614 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195242+0000", 2024-09-17T21:18:27.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.474e-06 2024-09-17T21:18:27.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.614 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195248+0000", 2024-09-17T21:18:27.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.485e-06 2024-09-17T21:18:27.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.614 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195262+0000", 2024-09-17T21:18:27.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4082e-05 2024-09-17T21:18:27.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195270+0000", 2024-09-17T21:18:27.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9980000000000003e-06 2024-09-17T21:18:27.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.205454+0000", 2024-09-17T21:18:27.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010184551 2024-09-17T21:18:27.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.205554+0000", 2024-09-17T21:18:27.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9827000000000005e-05 2024-09-17T21:18:27.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.207524+0000", 2024-09-17T21:18:27.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0019697149999999999 2024-09-17T21:18:27.616 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.616 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.616 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.616 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:408 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 2752512~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:27.616 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.195284+0000", 2024-09-17T21:18:27.617 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.3708767070000001, 2024-09-17T21:18:27.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013044501, 2024-09-17T21:18:27.617 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.617 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.617 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:27.617 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.617 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:27.617 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:27.617 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 408 2024-09-17T21:18:27.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.617 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195284+0000", 2024-09-17T21:18:27.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195284+0000", 2024-09-17T21:18:27.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195285+0000", 2024-09-17T21:18:27.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4600000000000004e-07 2024-09-17T21:18:27.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195288+0000", 2024-09-17T21:18:27.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.117e-06 2024-09-17T21:18:27.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195298+0000", 2024-09-17T21:18:27.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.031e-05 2024-09-17T21:18:27.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195305+0000", 2024-09-17T21:18:27.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8079999999999999e-06 2024-09-17T21:18:27.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.207570+0000", 2024-09-17T21:18:27.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012265287999999999 2024-09-17T21:18:27.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.207675+0000", 2024-09-17T21:18:27.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104695 2024-09-17T21:18:27.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.208329+0000", 2024-09-17T21:18:27.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00065363699999999999 2024-09-17T21:18:27.620 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.621 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.621 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.621 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:441 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:27.621 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.196306+0000", 2024-09-17T21:18:27.621 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.369854425, 2024-09-17T21:18:27.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012879454, 2024-09-17T21:18:27.621 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.621 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.621 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:27.621 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.621 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:27.621 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:27.622 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 441 2024-09-17T21:18:27.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.622 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196306+0000", 2024-09-17T21:18:27.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196306+0000", 2024-09-17T21:18:27.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196307+0000", 2024-09-17T21:18:27.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5799999999999998e-07 2024-09-17T21:18:27.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196312+0000", 2024-09-17T21:18:27.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2610000000000004e-06 2024-09-17T21:18:27.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196326+0000", 2024-09-17T21:18:27.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4656e-05 2024-09-17T21:18:27.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196334+0000", 2024-09-17T21:18:27.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5789999999999998e-06 2024-09-17T21:18:27.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.208385+0000", 2024-09-17T21:18:27.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012051134 2024-09-17T21:18:27.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.208480+0000", 2024-09-17T21:18:27.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5402000000000006e-05 2024-09-17T21:18:27.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.209186+0000", 2024-09-17T21:18:27.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000705564 2024-09-17T21:18:27.625 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.625 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.625 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.625 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:442 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:27.625 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.196349+0000", 2024-09-17T21:18:27.625 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.3698120389999999, 2024-09-17T21:18:27.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014411445, 2024-09-17T21:18:27.625 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.626 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.626 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:27.626 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.626 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:27.626 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:27.626 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 442 2024-09-17T21:18:27.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.626 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196349+0000", 2024-09-17T21:18:27.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196349+0000", 2024-09-17T21:18:27.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196349+0000", 2024-09-17T21:18:27.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7999999999999995e-07 2024-09-17T21:18:27.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196353+0000", 2024-09-17T21:18:27.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:18:27.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196362+0000", 2024-09-17T21:18:27.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6859999999999995e-06 2024-09-17T21:18:27.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196368+0000", 2024-09-17T21:18:27.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.314e-06 2024-09-17T21:18:27.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.209256+0000", 2024-09-17T21:18:27.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012888303 2024-09-17T21:18:27.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.209382+0000", 2024-09-17T21:18:27.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012542900000000001 2024-09-17T21:18:27.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.210760+0000", 2024-09-17T21:18:27.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001378708 2024-09-17T21:18:27.629 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.629 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.629 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.630 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:465 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 393216~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:27.630 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.196983+0000", 2024-09-17T21:18:27.630 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.36917742, 2024-09-17T21:18:27.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016159079999999999, 2024-09-17T21:18:27.630 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.630 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.630 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:27.630 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.630 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:27.630 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:27.630 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 465 2024-09-17T21:18:27.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.630 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196983+0000", 2024-09-17T21:18:27.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196983+0000", 2024-09-17T21:18:27.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196985+0000", 2024-09-17T21:18:27.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3340000000000001e-06 2024-09-17T21:18:27.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196990+0000", 2024-09-17T21:18:27.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5589999999999996e-06 2024-09-17T21:18:27.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197005+0000", 2024-09-17T21:18:27.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4783999999999999e-05 2024-09-17T21:18:27.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197015+0000", 2024-09-17T21:18:27.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6900000000000004e-06 2024-09-17T21:18:27.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.210810+0000", 2024-09-17T21:18:27.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013794969000000001 2024-09-17T21:18:27.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.210911+0000", 2024-09-17T21:18:27.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100752 2024-09-17T21:18:27.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.213143+0000", 2024-09-17T21:18:27.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022319919999999999 2024-09-17T21:18:27.635 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.635 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.635 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.635 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:466 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 2097152~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:27.635 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.197038+0000", 2024-09-17T21:18:27.635 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.3691229279999999, 2024-09-17T21:18:27.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019239187000000001, 2024-09-17T21:18:27.635 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.635 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.635 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:27.635 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.636 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:27.636 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:27.636 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 466 2024-09-17T21:18:27.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.636 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197038+0000", 2024-09-17T21:18:27.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197038+0000", 2024-09-17T21:18:27.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197039+0000", 2024-09-17T21:18:27.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6700000000000003e-07 2024-09-17T21:18:27.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197042+0000", 2024-09-17T21:18:27.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6959999999999998e-06 2024-09-17T21:18:27.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197059+0000", 2024-09-17T21:18:27.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6614000000000001e-05 2024-09-17T21:18:27.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197071+0000", 2024-09-17T21:18:27.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851000000000001e-05 2024-09-17T21:18:27.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.213205+0000", 2024-09-17T21:18:27.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016134030000000001 2024-09-17T21:18:27.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.213289+0000", 2024-09-17T21:18:27.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3850999999999994e-05 2024-09-17T21:18:27.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.216277+0000", 2024-09-17T21:18:27.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0029883779999999999 2024-09-17T21:18:27.639 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.639 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.640 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.640 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:467 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 3014656~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:27.640 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.197083+0000", 2024-09-17T21:18:27.640 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.3690777350000001, 2024-09-17T21:18:27.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021451300999999999, 2024-09-17T21:18:27.640 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.640 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.640 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:27.640 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.640 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:27.640 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:27.641 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 467 2024-09-17T21:18:27.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.641 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197083+0000", 2024-09-17T21:18:27.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197083+0000", 2024-09-17T21:18:27.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197084+0000", 2024-09-17T21:18:27.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2099999999999995e-07 2024-09-17T21:18:27.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197087+0000", 2024-09-17T21:18:27.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4640000000000002e-06 2024-09-17T21:18:27.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:18:27.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1511999999999999e-05 2024-09-17T21:18:27.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197106+0000", 2024-09-17T21:18:27.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9820000000000002e-06 2024-09-17T21:18:27.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.216327+0000", 2024-09-17T21:18:27.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019221201 2024-09-17T21:18:27.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.216427+0000", 2024-09-17T21:18:27.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010032800000000001 2024-09-17T21:18:27.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.218534+0000", 2024-09-17T21:18:27.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021069930000000001 2024-09-17T21:18:27.644 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.644 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.644 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.644 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:495 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 2228224~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:27.644 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.197359+0000", 2024-09-17T21:18:27.644 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.368801685, 2024-09-17T21:18:27.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021840167000000001, 2024-09-17T21:18:27.645 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.645 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.645 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:27.645 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.645 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:27.645 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:27.645 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 495 2024-09-17T21:18:27.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.645 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197359+0000", 2024-09-17T21:18:27.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197359+0000", 2024-09-17T21:18:27.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197360+0000", 2024-09-17T21:18:27.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0380000000000001e-06 2024-09-17T21:18:27.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197376+0000", 2024-09-17T21:18:27.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6147999999999999e-05 2024-09-17T21:18:27.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197392+0000", 2024-09-17T21:18:27.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6025999999999999e-05 2024-09-17T21:18:27.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197401+0000", 2024-09-17T21:18:27.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6640000000000004e-06 2024-09-17T21:18:27.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.218539+0000", 2024-09-17T21:18:27.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021137808000000001 2024-09-17T21:18:27.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.218611+0000", 2024-09-17T21:18:27.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2471000000000005e-05 2024-09-17T21:18:27.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.219199+0000", 2024-09-17T21:18:27.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00058801199999999995 2024-09-17T21:18:27.648 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.648 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.649 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.649 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:526 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 2359296~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:27.649 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.200199+0000", 2024-09-17T21:18:27.649 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.3659623519999999, 2024-09-17T21:18:27.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019695863000000001, 2024-09-17T21:18:27.649 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.649 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.649 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:27.649 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.649 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:27.649 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:27.649 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 526 2024-09-17T21:18:27.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.650 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200199+0000", 2024-09-17T21:18:27.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200199+0000", 2024-09-17T21:18:27.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200199+0000", 2024-09-17T21:18:27.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:18:27.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200205+0000", 2024-09-17T21:18:27.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.268e-06 2024-09-17T21:18:27.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200222+0000", 2024-09-17T21:18:27.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6562000000000002e-05 2024-09-17T21:18:27.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:18:27.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:18:27.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.219244+0000", 2024-09-17T21:18:27.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019016314999999999 2024-09-17T21:18:27.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.219317+0000", 2024-09-17T21:18:27.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3391000000000003e-05 2024-09-17T21:18:27.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.219894+0000", 2024-09-17T21:18:27.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00057747699999999996 2024-09-17T21:18:27.653 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.653 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.653 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.653 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:547 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:27.653 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.200266+0000", 2024-09-17T21:18:27.653 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.365894723, 2024-09-17T21:18:27.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021208113000000001, 2024-09-17T21:18:27.653 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.654 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.654 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:27.654 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.654 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:27.654 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:27.654 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 547 2024-09-17T21:18:27.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.654 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200266+0000", 2024-09-17T21:18:27.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200266+0000", 2024-09-17T21:18:27.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200267+0000", 2024-09-17T21:18:27.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:18:27.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200268+0000", 2024-09-17T21:18:27.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5540000000000001e-06 2024-09-17T21:18:27.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200273+0000", 2024-09-17T21:18:27.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5549999999999996e-06 2024-09-17T21:18:27.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200276+0000", 2024-09-17T21:18:27.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.461e-06 2024-09-17T21:18:27.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.219998+0000", 2024-09-17T21:18:27.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019721596000000001 2024-09-17T21:18:27.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.220117+0000", 2024-09-17T21:18:27.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011937799999999999 2024-09-17T21:18:27.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.221474+0000", 2024-09-17T21:18:27.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0013572289999999999 2024-09-17T21:18:27.657 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.657 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.658 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.658 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:548 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:27.658 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.200281+0000", 2024-09-17T21:18:27.658 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.365880027, 2024-09-17T21:18:27.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021975320999999999, 2024-09-17T21:18:27.658 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.658 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.658 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:27.658 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.658 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:27.658 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:27.658 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 548 2024-09-17T21:18:27.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.659 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200281+0000", 2024-09-17T21:18:27.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200281+0000", 2024-09-17T21:18:27.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200281+0000", 2024-09-17T21:18:27.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5100000000000001e-07 2024-09-17T21:18:27.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200283+0000", 2024-09-17T21:18:27.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4699999999999999e-06 2024-09-17T21:18:27.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200287+0000", 2024-09-17T21:18:27.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3939999999999998e-06 2024-09-17T21:18:27.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200290+0000", 2024-09-17T21:18:27.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3160000000000001e-06 2024-09-17T21:18:27.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.221522+0000", 2024-09-17T21:18:27.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021231782000000001 2024-09-17T21:18:27.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.221624+0000", 2024-09-17T21:18:27.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101693 2024-09-17T21:18:27.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.222256+0000", 2024-09-17T21:18:27.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00063231500000000005 2024-09-17T21:18:27.665 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.665 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.665 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.665 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:588 3.1d 3:bf885c9b:::rbd_data.1152b51cb396.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:27.665 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.243843+0000", 2024-09-17T21:18:27.666 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.3223181849999999, 2024-09-17T21:18:27.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0096383570000000002, 2024-09-17T21:18:27.666 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:27.666 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:27.666 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:27.666 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:27.666 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:27.666 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:27.666 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 588 2024-09-17T21:18:27.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.666 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:27.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:27.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.243843+0000", 2024-09-17T21:18:27.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:27.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.243843+0000", 2024-09-17T21:18:27.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:27.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:27.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.243845+0000", 2024-09-17T21:18:27.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:18:27.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:27.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.245189+0000", 2024-09-17T21:18:27.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0013433659999999999 2024-09-17T21:18:27.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:27.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.245212+0000", 2024-09-17T21:18:27.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3329e-05 2024-09-17T21:18:27.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:27.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.245225+0000", 2024-09-17T21:18:27.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2574e-05 2024-09-17T21:18:27.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:27.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.245309+0000", 2024-09-17T21:18:27.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4065000000000002e-05 2024-09-17T21:18:27.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:27.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.245497+0000", 2024-09-17T21:18:27.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018829400000000001 2024-09-17T21:18:27.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-17T21:18:27.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.245876+0000", 2024-09-17T21:18:27.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037877800000000002 2024-09-17T21:18:27.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:27.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.252927+0000", 2024-09-17T21:18:27.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.007050962 2024-09-17T21:18:27.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:27.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.253332+0000", 2024-09-17T21:18:27.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00040551699999999999 2024-09-17T21:18:27.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:27.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.253420+0000", 2024-09-17T21:18:27.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7410000000000005e-05 2024-09-17T21:18:27.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:27.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:27.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:27.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.253481+0000", 2024-09-17T21:18:27.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1508999999999997e-05 2024-09-17T21:18:27.672 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.672 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.672 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.672 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:27.672 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:27.672 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:27.672 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:18:27.673 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:18:27.675 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:27.670+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:27.675 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:18:27.775 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:27.770+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:27.805 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:27.805 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:27.805 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:27.805 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:27.827 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:18:27.835 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:18:27.835 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:18:27.852 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:18:27.852 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:18:27.869 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:18:27.869 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:18:27.876 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:27.870+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:27.889 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:18:27.890 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:18:27.910 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:18:27.911 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:18:27.931 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:18:27.932 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:18:27.943 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:27.943 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:27.959 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:18:28.000 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:27.996+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:28.077 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:28.072+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:28.114 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:28.114 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:28.129 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:18:28.178 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:28.174+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:28.279 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:28.274+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:28.306 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:28.306 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:28.320 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:18:28.339 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-17T21:18:28.380 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:28.374+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:28.443 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:28.443 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:28.460 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:18:28.480 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:28.474+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:28.581 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:28.574+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:28.587 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:28.587 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:28.604 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:18:28.682 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:28.676+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:28.711 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-17T21:18:28.732 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:28.733 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:28.748 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:18:28.783 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:28.778+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:28.884 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:28.876+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:28.918 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:28.918 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:28.933 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:18:29.020 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:29.016+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:29.086 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:29.082+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:29.101 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:29.101 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:29.116 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:18:29.187 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:29.180+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:29.250 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:29.250 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:29.264 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:18:29.309 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:29.306+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:29.388 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:29.384+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:29.390 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:29.390 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:29.405 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:18:29.489 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:29.486+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:29.561 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:29.561 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:29.575 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:18:29.590 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:29.584+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:29.690 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:29.686+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:29.742 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:29.742 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:29.756 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:18:29.791 INFO:teuthology.orchestra.run.smithi184.stdout:455 write 0xe2dd800 thru 0xe2e1032 (0x3833 bytes) 2024-09-17T21:18:29.791 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:29.784+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:29.794 INFO:teuthology.orchestra.run.smithi184.stdout:457 write 0x1358c00 thru 0x1363726 (0xab27 bytes) 2024-09-17T21:18:29.795 INFO:teuthology.orchestra.run.smithi184.stdout:461 punch from 0xadb5a00 to 0xadbdc00, (0x8200 bytes) 2024-09-17T21:18:29.798 INFO:teuthology.orchestra.run.smithi184.stdout:465 flatten 2024-09-17T21:18:29.892 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:29.888+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:29.949 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:18:29.985 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:29.985 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:30.000 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:29.996+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:30.006 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:18:30.094 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:30.088+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:30.110 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:18:30.110 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:18:30.127 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:18:30.127 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:18:30.143 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:18:30.144 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:18:30.157 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:30.157 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:30.160 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:18:30.161 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:18:30.171 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:18:30.178 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:18:30.178 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:18:30.193 INFO:teuthology.orchestra.run.smithi184.stdout:468 write 0xada6200 thru 0xadb183b (0xb63c bytes) 2024-09-17T21:18:30.194 INFO:teuthology.orchestra.run.smithi184.stdout:470 read 0x9068200 thru 0x907144b (0x924c bytes) 2024-09-17T21:18:30.194 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:30.188+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:30.196 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:18:30.197 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:18:30.265 INFO:teuthology.orchestra.run.smithi184.stdout:471 trunc from 0xe2e1200 to 0x5017200 2024-09-17T21:18:30.290 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:30.290 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:30.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.290 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4520.0:5919 3.16 3:6d429141:::rbd_data.11528c3b0529.00000000000001c3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-17T21:18:30.290 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:30.282033+0000", 2024-09-17T21:18:30.290 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0068388729999999997, 2024-09-17T21:18:30.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0068656189999999999, 2024-09-17T21:18:30.290 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:18:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4520", 2024-09-17T21:18:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2592434746", 2024-09-17T21:18:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 5919 2024-09-17T21:18:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.282033+0000", 2024-09-17T21:18:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.282033+0000", 2024-09-17T21:18:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.282035+0000", 2024-09-17T21:18:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.322e-06 2024-09-17T21:18:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.282040+0000", 2024-09-17T21:18:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8719999999999999e-06 2024-09-17T21:18:30.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.282057+0000", 2024-09-17T21:18:30.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6537000000000001e-05 2024-09-17T21:18:30.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.282072+0000", 2024-09-17T21:18:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4951e-05 2024-09-17T21:18:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.282844+0000", 2024-09-17T21:18:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00077206599999999996 2024-09-17T21:18:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.282919+0000", 2024-09-17T21:18:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5178000000000004e-05 2024-09-17T21:18:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 3", 2024-09-17T21:18:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.283040+0000", 2024-09-17T21:18:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000121342 2024-09-17T21:18:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.286267+0000", 2024-09-17T21:18:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0032267839999999999 2024-09-17T21:18:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4520.0:5920 3.f 3:f2ad9bda:::rbd_data.11528c3b0529.00000000000001c4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-17T21:18:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:30.282107+0000", 2024-09-17T21:18:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0067646970000000001, 2024-09-17T21:18:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0068957239999999998, 2024-09-17T21:18:30.297 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.297 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.297 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:18:30.297 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.297 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4520", 2024-09-17T21:18:30.297 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2592434746", 2024-09-17T21:18:30.297 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 5920 2024-09-17T21:18:30.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.297 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.282107+0000", 2024-09-17T21:18:30.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.282107+0000", 2024-09-17T21:18:30.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.282109+0000", 2024-09-17T21:18:30.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.494e-06 2024-09-17T21:18:30.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.282115+0000", 2024-09-17T21:18:30.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6980000000000003e-06 2024-09-17T21:18:30.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.282144+0000", 2024-09-17T21:18:30.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9142e-05 2024-09-17T21:18:30.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.282160+0000", 2024-09-17T21:18:30.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6489000000000001e-05 2024-09-17T21:18:30.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.282267+0000", 2024-09-17T21:18:30.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106865 2024-09-17T21:18:30.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.282457+0000", 2024-09-17T21:18:30.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018958899999999999 2024-09-17T21:18:30.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 3", 2024-09-17T21:18:30.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.282684+0000", 2024-09-17T21:18:30.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000226896 2024-09-17T21:18:30.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:30.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.286257+0000", 2024-09-17T21:18:30.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035734949999999999 2024-09-17T21:18:30.301 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.301 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.301 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.302 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:2322 3.10 3:0a3e7718:::rbd_header.11528c3b0529:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-17T21:18:30.302 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:30.284354+0000", 2024-09-17T21:18:30.302 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0045182800000000004, 2024-09-17T21:18:30.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0047347020000000004, 2024-09-17T21:18:30.302 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.302 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.302 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:30.302 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.302 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:30.302 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:30.302 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2322 2024-09-17T21:18:30.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.302 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.284354+0000", 2024-09-17T21:18:30.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.284354+0000", 2024-09-17T21:18:30.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.284355+0000", 2024-09-17T21:18:30.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9100000000000002e-07 2024-09-17T21:18:30.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.284360+0000", 2024-09-17T21:18:30.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4029999999999998e-06 2024-09-17T21:18:30.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.284374+0000", 2024-09-17T21:18:30.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4194e-05 2024-09-17T21:18:30.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.284383+0000", 2024-09-17T21:18:30.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1919999999999994e-06 2024-09-17T21:18:30.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.288692+0000", 2024-09-17T21:18:30.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043090009999999998 2024-09-17T21:18:30.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.288871+0000", 2024-09-17T21:18:30.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017873800000000001 2024-09-17T21:18:30.306 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.306 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.306 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.306 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4520.0:5929 3.10 3:0a3e7718:::rbd_header.11528c3b0529:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-17T21:18:30.306 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:30.284550+0000", 2024-09-17T21:18:30.306 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0043221850000000001, 2024-09-17T21:18:30.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0046019069999999997, 2024-09-17T21:18:30.306 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.306 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.306 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "reached pg", 2024-09-17T21:18:30.306 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.306 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4520", 2024-09-17T21:18:30.307 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2592434746", 2024-09-17T21:18:30.307 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 5929 2024-09-17T21:18:30.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.307 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.284550+0000", 2024-09-17T21:18:30.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.284550+0000", 2024-09-17T21:18:30.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.284551+0000", 2024-09-17T21:18:30.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.184e-06 2024-09-17T21:18:30.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.284557+0000", 2024-09-17T21:18:30.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1720000000000004e-06 2024-09-17T21:18:30.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.284570+0000", 2024-09-17T21:18:30.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2330000000000001e-05 2024-09-17T21:18:30.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.284577+0000", 2024-09-17T21:18:30.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3499999999999999e-06 2024-09-17T21:18:30.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.289123+0000", 2024-09-17T21:18:30.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045457060000000001 2024-09-17T21:18:30.309 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.309 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.309 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.310 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:2323 3.10 3.18ee7c50 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-17T21:18:30.310 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:30.284652+0000", 2024-09-17T21:18:30.310 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.004219988, 2024-09-17T21:18:30.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045511529999999996, 2024-09-17T21:18:30.310 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.310 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.310 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:18:30.310 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.310 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:30.310 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:30.310 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2323 2024-09-17T21:18:30.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.311 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.284652+0000", 2024-09-17T21:18:30.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.284652+0000", 2024-09-17T21:18:30.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.284654+0000", 2024-09-17T21:18:30.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:18:30.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.284660+0000", 2024-09-17T21:18:30.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.122e-06 2024-09-17T21:18:30.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.284672+0000", 2024-09-17T21:18:30.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2575000000000001e-05 2024-09-17T21:18:30.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.284683+0000", 2024-09-17T21:18:30.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0336999999999999e-05 2024-09-17T21:18:30.313 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.313 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.313 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.314 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4520.0:5930 3.10 3.18ee7c50 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-17T21:18:30.314 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:30.285006+0000", 2024-09-17T21:18:30.314 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.003865832, 2024-09-17T21:18:30.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042515449999999998, 2024-09-17T21:18:30.314 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.314 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.314 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:18:30.314 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.314 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4520", 2024-09-17T21:18:30.314 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2592434746", 2024-09-17T21:18:30.314 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 5930 2024-09-17T21:18:30.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.314 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285006+0000", 2024-09-17T21:18:30.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285006+0000", 2024-09-17T21:18:30.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285007+0000", 2024-09-17T21:18:30.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.082e-06 2024-09-17T21:18:30.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285012+0000", 2024-09-17T21:18:30.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7299999999999996e-06 2024-09-17T21:18:30.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285025+0000", 2024-09-17T21:18:30.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3448000000000001e-05 2024-09-17T21:18:30.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285035+0000", 2024-09-17T21:18:30.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6099999999999995e-06 2024-09-17T21:18:30.316 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.317 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.317 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.321 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:2324 3.10 3.18ee7c50 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-17T21:18:30.321 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:30.285105+0000", 2024-09-17T21:18:30.321 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0037665659999999998, 2024-09-17T21:18:30.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041970499999999999, 2024-09-17T21:18:30.321 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.321 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.321 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:18:30.321 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.321 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:30.321 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:30.321 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2324 2024-09-17T21:18:30.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.322 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285105+0000", 2024-09-17T21:18:30.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285105+0000", 2024-09-17T21:18:30.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285107+0000", 2024-09-17T21:18:30.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3969999999999999e-06 2024-09-17T21:18:30.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285113+0000", 2024-09-17T21:18:30.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7969999999999997e-06 2024-09-17T21:18:30.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285125+0000", 2024-09-17T21:18:30.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2480000000000001e-05 2024-09-17T21:18:30.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285134+0000", 2024-09-17T21:18:30.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5150000000000006e-06 2024-09-17T21:18:30.324 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.324 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.324 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.324 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4520.0:5931 3.10 3.18ee7c50 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-17T21:18:30.324 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:30.285344+0000", 2024-09-17T21:18:30.324 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0035282429999999999, 2024-09-17T21:18:30.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.004026997, 2024-09-17T21:18:30.324 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.324 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.324 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:18:30.325 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.325 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4520", 2024-09-17T21:18:30.325 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2592434746", 2024-09-17T21:18:30.325 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 5931 2024-09-17T21:18:30.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.325 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285344+0000", 2024-09-17T21:18:30.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285344+0000", 2024-09-17T21:18:30.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285345+0000", 2024-09-17T21:18:30.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.04e-06 2024-09-17T21:18:30.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285350+0000", 2024-09-17T21:18:30.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1080000000000001e-06 2024-09-17T21:18:30.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285359+0000", 2024-09-17T21:18:30.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4560000000000006e-06 2024-09-17T21:18:30.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285366+0000", 2024-09-17T21:18:30.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8539999999999999e-06 2024-09-17T21:18:30.327 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.327 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.327 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.327 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:2325 3.10 3.18ee7c50 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-17T21:18:30.328 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:30.285438+0000", 2024-09-17T21:18:30.328 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0034342460000000002, 2024-09-17T21:18:30.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040225549999999997, 2024-09-17T21:18:30.328 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.328 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.328 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:18:30.328 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.328 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:30.328 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:30.328 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2325 2024-09-17T21:18:30.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.328 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285438+0000", 2024-09-17T21:18:30.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285438+0000", 2024-09-17T21:18:30.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285439+0000", 2024-09-17T21:18:30.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.46e-06 2024-09-17T21:18:30.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285445+0000", 2024-09-17T21:18:30.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7919999999999998e-06 2024-09-17T21:18:30.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285462+0000", 2024-09-17T21:18:30.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7082999999999999e-05 2024-09-17T21:18:30.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.285470+0000", 2024-09-17T21:18:30.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1620000000000008e-06 2024-09-17T21:18:30.331 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.331 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.331 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.331 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4520.0:5920 3.f e34/23 ondisk, result = 0)", 2024-09-17T21:18:30.331 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:30.289394+0000", 2024-09-17T21:18:30.331 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4294967295.9994783, 2024-09-17T21:18:30.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015262699999999999, 2024-09-17T21:18:30.331 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.331 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.331 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:30.331 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.289394+0000", 2024-09-17T21:18:30.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.289394+0000", 2024-09-17T21:18:30.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.289395+0000", 2024-09-17T21:18:30.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7700000000000003e-07 2024-09-17T21:18:30.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.289398+0000", 2024-09-17T21:18:30.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6049999999999998e-06 2024-09-17T21:18:30.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.289412+0000", 2024-09-17T21:18:30.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3725e-05 2024-09-17T21:18:30.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.289417+0000", 2024-09-17T21:18:30.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6759999999999998e-06 2024-09-17T21:18:30.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.289494+0000", 2024-09-17T21:18:30.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7239999999999999e-05 2024-09-17T21:18:30.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.289504+0000", 2024-09-17T21:18:30.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6139999999999998e-06 2024-09-17T21:18:30.334 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.334 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.334 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.334 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.335 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-09-17T21:18:30.335 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 10 2024-09-17T21:18:30.335 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:30.336 INFO:teuthology.orchestra.run.smithi184.stdout:472 write 0x6e2d600 thru 0x6e34731 (0x7132 bytes) 2024-09-17T21:18:30.336 INFO:teuthology.orchestra.run.smithi184.stdout:473 read 0x526ea00 thru 0x527e49e (0xfa9f bytes) 2024-09-17T21:18:30.336 INFO:teuthology.orchestra.run.smithi184.stdout:474 trunc from 0x6e34800 to 0x6495800 2024-09-17T21:18:30.337 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:18:30.337 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:30.330+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:30.365 INFO:teuthology.orchestra.run.smithi184.stdout:475 trunc from 0x6495800 to 0x59ff600 2024-09-17T21:18:30.392 INFO:teuthology.orchestra.run.smithi184.stdout:477 read 0x4052400 thru 0x405cd97 (0xa998 bytes) 2024-09-17T21:18:30.394 INFO:teuthology.orchestra.run.smithi184.stdout:478 read 0x56e800 thru 0x57df36 (0xf737 bytes) 2024-09-17T21:18:30.437 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:30.434+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:30.441 INFO:teuthology.orchestra.run.smithi184.stdout:479 write 0xe95a600 thru 0xe966fa9 (0xc9aa bytes) 2024-09-17T21:18:30.442 INFO:teuthology.orchestra.run.smithi184.stdout:480 write 0x9879000 thru 0x9886477 (0xd478 bytes) 2024-09-17T21:18:30.443 INFO:teuthology.orchestra.run.smithi184.stdout:481 read 0x5f79000 thru 0x5f84fc9 (0xbfca bytes) 2024-09-17T21:18:30.443 INFO:teuthology.orchestra.run.smithi184.stdout:482 write 0x4adcc00 thru 0x4ae3f62 (0x7363 bytes) 2024-09-17T21:18:30.443 INFO:teuthology.orchestra.run.smithi184.stdout:483 write 0x29b8e00 thru 0x29ba7c7 (0x19c8 bytes) 2024-09-17T21:18:30.443 INFO:teuthology.orchestra.run.smithi184.stdout:484 punch from 0x3e6b800 to 0x3e74800, (0x9000 bytes) 2024-09-17T21:18:30.444 INFO:teuthology.orchestra.run.smithi184.stdout:485 read 0x98baa00 thru 0x98c91d4 (0xe7d5 bytes) 2024-09-17T21:18:30.445 INFO:teuthology.orchestra.run.smithi184.stdout:486 read 0xcf70200 thru 0xcf74e6d (0x4c6e bytes) 2024-09-17T21:18:30.445 INFO:teuthology.orchestra.run.smithi184.stdout:487 read 0x7397c00 thru 0x739dfa3 (0x63a4 bytes) 2024-09-17T21:18:30.445 INFO:teuthology.orchestra.run.smithi184.stdout:488 read 0xb28e00 thru 0xb33ce9 (0xaeea bytes) 2024-09-17T21:18:30.445 INFO:teuthology.orchestra.run.smithi184.stdout:489 write 0x2394e00 thru 0x239e018 (0x9219 bytes) 2024-09-17T21:18:30.490 INFO:teuthology.orchestra.run.smithi184.stdout:491 trunc from 0xe967000 to 0x820e00 2024-09-17T21:18:30.516 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:30.516 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:30.516 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:18:30.516 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:18:30.517 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:30.532 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:18:30.564 INFO:teuthology.orchestra.run.smithi184.stdout:493 trunc from 0x820e00 to 0xb9c8e00 2024-09-17T21:18:30.573 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:30.570+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:30.579 INFO:teuthology.orchestra.run.smithi184.stdout:496 write 0x33e3c00 thru 0x33ec5c4 (0x89c5 bytes) 2024-09-17T21:18:30.579 INFO:teuthology.orchestra.run.smithi184.stdout:497 read 0xdca00 thru 0xea4fe (0xdaff bytes) 2024-09-17T21:18:30.594 INFO:teuthology.orchestra.run.smithi184.stdout:498 trunc from 0xb9c8e00 to 0x5ef7000 2024-09-17T21:18:30.617 INFO:teuthology.orchestra.run.smithi184.stdout:502 trunc from 0x5ef7000 to 0x4e34800 2024-09-17T21:18:30.630 INFO:teuthology.orchestra.run.smithi184.stdout:506 punch from 0x3c42e00 to 0x3c4b000, (0x8200 bytes) 2024-09-17T21:18:30.632 INFO:teuthology.orchestra.run.smithi184.stdout:510 write 0x223ca00 thru 0x2249250 (0xc851 bytes) 2024-09-17T21:18:30.632 INFO:teuthology.orchestra.run.smithi184.stdout:511 write 0x13aec00 thru 0x13b335f (0x4760 bytes) 2024-09-17T21:18:30.632 INFO:teuthology.orchestra.run.smithi184.stdout:512 clone 6 order 24 su 131072 sc 11 2024-09-17T21:18:30.639 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:30.632+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:30.701 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:30.701 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:18:30.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:18:30.701 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:30.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.701 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4190.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-17T21:18:30.701 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:03.060815+0000", 2024-09-17T21:18:30.701 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.638450689999999, 2024-09-17T21:18:30.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011124612000000001, 2024-09-17T21:18:30.702 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.702 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.702 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:30.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4190", 2024-09-17T21:18:30.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.83:0/956669820", 2024-09-17T21:18:30.702 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 7 2024-09-17T21:18:30.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.702 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060815+0000", 2024-09-17T21:18:30.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060815+0000", 2024-09-17T21:18:30.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060819+0000", 2024-09-17T21:18:30.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2640000000000002e-06 2024-09-17T21:18:30.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060838+0000", 2024-09-17T21:18:30.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9406e-05 2024-09-17T21:18:30.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060878+0000", 2024-09-17T21:18:30.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0120000000000002e-05 2024-09-17T21:18:30.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.060896+0000", 2024-09-17T21:18:30.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7986999999999999e-05 2024-09-17T21:18:30.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.061030+0000", 2024-09-17T21:18:30.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013395200000000001 2024-09-17T21:18:30.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:30.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.061041+0000", 2024-09-17T21:18:30.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1053000000000001e-05 2024-09-17T21:18:30.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.068612+0000", 2024-09-17T21:18:30.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0075704049999999997 2024-09-17T21:18:30.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.068806+0000", 2024-09-17T21:18:30.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019402500000000001 2024-09-17T21:18:30.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:18:30.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.069031+0000", 2024-09-17T21:18:30.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022502199999999999 2024-09-17T21:18:30.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:30.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.070704+0000", 2024-09-17T21:18:30.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001672974 2024-09-17T21:18:30.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:30.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071795+0000", 2024-09-17T21:18:30.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001091648 2024-09-17T21:18:30.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:30.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071878+0000", 2024-09-17T21:18:30.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2225999999999995e-05 2024-09-17T21:18:30.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:30.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:03.071939+0000", 2024-09-17T21:18:30.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1530000000000002e-05 2024-09-17T21:18:30.708 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.708 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.708 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.708 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:18:30.708 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:18:30.709 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0950749139999996, 2024-09-17T21:18:30.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:18:30.709 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.709 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.709 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:30.709 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:18:30.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:18:30.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:18:30.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:18:30.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:18:30.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:18:30.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:18:30.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:18:30.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:18:30.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:18:30.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:18:30.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:18:30.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:18:30.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:18:30.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:30.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:18:30.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:18:30.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:30.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:18:30.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:18:30.713 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.713 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.713 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.713 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:18:30.713 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:18:30.713 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0851282779999991, 2024-09-17T21:18:30.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:18:30.713 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.713 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.713 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:30.713 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:18:30.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:18:30.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:18:30.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:18:30.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:18:30.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:18:30.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:18:30.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:18:30.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:18:30.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:18:30.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:18:30.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:18:30.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:18:30.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:18:30.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:30.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:18:30.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:18:30.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:30.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:18:30.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:18:30.717 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.717 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.717 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.717 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:18:30.718 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:18:30.718 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.8419774379999998, 2024-09-17T21:18:30.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:18:30.718 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.718 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.718 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:30.718 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.718 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:18:30.718 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:30.718 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:18:30.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.718 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:18:30.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:18:30.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:18:30.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:18:30.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:18:30.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:18:30.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:18:30.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:18:30.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:18:30.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:18:30.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:18:30.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:18:30.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:18:30.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:18:30.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:30.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:18:30.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:18:30.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:30.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:18:30.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:18:30.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:30.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:18:30.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:18:30.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:30.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:18:30.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:18:30.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:30.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:18:30.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:18:30.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.723 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.723 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:18:30.724 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:18:30.724 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.8323319519999997, 2024-09-17T21:18:30.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:18:30.724 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.724 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.724 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:30.724 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.724 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:18:30.724 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:30.724 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:18:30.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.724 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:18:30.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:18:30.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:18:30.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:18:30.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:18:30.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:18:30.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:18:30.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:18:30.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:18:30.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:18:30.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:18:30.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:18:30.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:18:30.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:18:30.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:30.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:18:30.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:18:30.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:30.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:18:30.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:18:30.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:30.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:18:30.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:18:30.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:30.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:18:30.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:18:30.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:30.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:18:30.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:18:30.733 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.733 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.733 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.733 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:407 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:30.733 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.195241+0000", 2024-09-17T21:18:30.733 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.5040245409999997, 2024-09-17T21:18:30.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012283132, 2024-09-17T21:18:30.733 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.734 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.734 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:30.734 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.734 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:30.734 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:30.734 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 407 2024-09-17T21:18:30.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.734 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195241+0000", 2024-09-17T21:18:30.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195241+0000", 2024-09-17T21:18:30.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195242+0000", 2024-09-17T21:18:30.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.474e-06 2024-09-17T21:18:30.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195248+0000", 2024-09-17T21:18:30.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.485e-06 2024-09-17T21:18:30.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195262+0000", 2024-09-17T21:18:30.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4082e-05 2024-09-17T21:18:30.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195270+0000", 2024-09-17T21:18:30.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9980000000000003e-06 2024-09-17T21:18:30.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.205454+0000", 2024-09-17T21:18:30.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010184551 2024-09-17T21:18:30.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.205554+0000", 2024-09-17T21:18:30.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9827000000000005e-05 2024-09-17T21:18:30.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:30.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.207524+0000", 2024-09-17T21:18:30.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0019697149999999999 2024-09-17T21:18:30.737 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.737 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.737 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.738 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:408 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 2752512~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:30.738 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.195284+0000", 2024-09-17T21:18:30.738 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.5039811409999997, 2024-09-17T21:18:30.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013044501, 2024-09-17T21:18:30.738 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.738 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.738 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:30.738 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.738 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:30.738 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:30.738 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 408 2024-09-17T21:18:30.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.739 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195284+0000", 2024-09-17T21:18:30.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195284+0000", 2024-09-17T21:18:30.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195285+0000", 2024-09-17T21:18:30.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4600000000000004e-07 2024-09-17T21:18:30.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195288+0000", 2024-09-17T21:18:30.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.117e-06 2024-09-17T21:18:30.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195298+0000", 2024-09-17T21:18:30.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.031e-05 2024-09-17T21:18:30.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.195305+0000", 2024-09-17T21:18:30.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8079999999999999e-06 2024-09-17T21:18:30.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.207570+0000", 2024-09-17T21:18:30.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012265287999999999 2024-09-17T21:18:30.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.207675+0000", 2024-09-17T21:18:30.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104695 2024-09-17T21:18:30.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:30.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.208329+0000", 2024-09-17T21:18:30.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00065363699999999999 2024-09-17T21:18:30.742 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.742 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.742 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.743 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:441 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:30.743 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.196306+0000", 2024-09-17T21:18:30.743 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.5029588589999996, 2024-09-17T21:18:30.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012879454, 2024-09-17T21:18:30.743 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.743 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.743 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:30.743 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.743 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:30.743 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:30.743 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 441 2024-09-17T21:18:30.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.743 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196306+0000", 2024-09-17T21:18:30.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196306+0000", 2024-09-17T21:18:30.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196307+0000", 2024-09-17T21:18:30.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5799999999999998e-07 2024-09-17T21:18:30.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196312+0000", 2024-09-17T21:18:30.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2610000000000004e-06 2024-09-17T21:18:30.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196326+0000", 2024-09-17T21:18:30.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4656e-05 2024-09-17T21:18:30.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196334+0000", 2024-09-17T21:18:30.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5789999999999998e-06 2024-09-17T21:18:30.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.208385+0000", 2024-09-17T21:18:30.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012051134 2024-09-17T21:18:30.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.208480+0000", 2024-09-17T21:18:30.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5402000000000006e-05 2024-09-17T21:18:30.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:30.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.209186+0000", 2024-09-17T21:18:30.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000705564 2024-09-17T21:18:30.747 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.747 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.747 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.747 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:442 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:30.748 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.196349+0000", 2024-09-17T21:18:30.748 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.502916473, 2024-09-17T21:18:30.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014411445, 2024-09-17T21:18:30.748 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.748 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.748 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:30.748 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.748 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:30.748 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:30.748 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 442 2024-09-17T21:18:30.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.748 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196349+0000", 2024-09-17T21:18:30.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196349+0000", 2024-09-17T21:18:30.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196349+0000", 2024-09-17T21:18:30.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7999999999999995e-07 2024-09-17T21:18:30.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196353+0000", 2024-09-17T21:18:30.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:18:30.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196362+0000", 2024-09-17T21:18:30.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6859999999999995e-06 2024-09-17T21:18:30.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196368+0000", 2024-09-17T21:18:30.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.314e-06 2024-09-17T21:18:30.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.209256+0000", 2024-09-17T21:18:30.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012888303 2024-09-17T21:18:30.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.209382+0000", 2024-09-17T21:18:30.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012542900000000001 2024-09-17T21:18:30.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:30.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.210760+0000", 2024-09-17T21:18:30.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001378708 2024-09-17T21:18:30.752 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.752 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.752 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.752 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:465 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 393216~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:30.752 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.196983+0000", 2024-09-17T21:18:30.752 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.5022818539999996, 2024-09-17T21:18:30.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016159079999999999, 2024-09-17T21:18:30.752 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.752 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.752 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:30.753 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.753 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:30.753 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:30.753 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 465 2024-09-17T21:18:30.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.753 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196983+0000", 2024-09-17T21:18:30.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196983+0000", 2024-09-17T21:18:30.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196985+0000", 2024-09-17T21:18:30.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3340000000000001e-06 2024-09-17T21:18:30.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.196990+0000", 2024-09-17T21:18:30.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5589999999999996e-06 2024-09-17T21:18:30.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197005+0000", 2024-09-17T21:18:30.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4783999999999999e-05 2024-09-17T21:18:30.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197015+0000", 2024-09-17T21:18:30.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6900000000000004e-06 2024-09-17T21:18:30.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.210810+0000", 2024-09-17T21:18:30.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013794969000000001 2024-09-17T21:18:30.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.210911+0000", 2024-09-17T21:18:30.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100752 2024-09-17T21:18:30.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:30.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.213143+0000", 2024-09-17T21:18:30.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022319919999999999 2024-09-17T21:18:30.756 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.756 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.756 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.757 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:466 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 2097152~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:30.757 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.197038+0000", 2024-09-17T21:18:30.757 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.5022273620000002, 2024-09-17T21:18:30.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019239187000000001, 2024-09-17T21:18:30.757 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.757 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.757 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:30.757 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.757 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:30.757 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:30.757 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 466 2024-09-17T21:18:30.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.757 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197038+0000", 2024-09-17T21:18:30.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197038+0000", 2024-09-17T21:18:30.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197039+0000", 2024-09-17T21:18:30.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6700000000000003e-07 2024-09-17T21:18:30.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197042+0000", 2024-09-17T21:18:30.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6959999999999998e-06 2024-09-17T21:18:30.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197059+0000", 2024-09-17T21:18:30.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6614000000000001e-05 2024-09-17T21:18:30.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197071+0000", 2024-09-17T21:18:30.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851000000000001e-05 2024-09-17T21:18:30.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.213205+0000", 2024-09-17T21:18:30.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016134030000000001 2024-09-17T21:18:30.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.213289+0000", 2024-09-17T21:18:30.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3850999999999994e-05 2024-09-17T21:18:30.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:30.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.216277+0000", 2024-09-17T21:18:30.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0029883779999999999 2024-09-17T21:18:30.761 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.761 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.761 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.761 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:467 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 3014656~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:30.761 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.197083+0000", 2024-09-17T21:18:30.761 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.5021821690000001, 2024-09-17T21:18:30.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021451300999999999, 2024-09-17T21:18:30.761 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.762 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.762 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:30.762 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.762 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:30.762 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:30.762 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 467 2024-09-17T21:18:30.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.762 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197083+0000", 2024-09-17T21:18:30.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197083+0000", 2024-09-17T21:18:30.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197084+0000", 2024-09-17T21:18:30.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2099999999999995e-07 2024-09-17T21:18:30.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197087+0000", 2024-09-17T21:18:30.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4640000000000002e-06 2024-09-17T21:18:30.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:18:30.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1511999999999999e-05 2024-09-17T21:18:30.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197106+0000", 2024-09-17T21:18:30.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9820000000000002e-06 2024-09-17T21:18:30.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.216327+0000", 2024-09-17T21:18:30.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019221201 2024-09-17T21:18:30.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.216427+0000", 2024-09-17T21:18:30.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010032800000000001 2024-09-17T21:18:30.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:30.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.218534+0000", 2024-09-17T21:18:30.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021069930000000001 2024-09-17T21:18:30.765 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.766 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.766 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.766 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:495 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 2228224~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:30.766 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.197359+0000", 2024-09-17T21:18:30.766 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.501906119, 2024-09-17T21:18:30.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021840167000000001, 2024-09-17T21:18:30.766 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.766 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.766 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:30.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:30.767 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:30.767 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 495 2024-09-17T21:18:30.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.767 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197359+0000", 2024-09-17T21:18:30.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197359+0000", 2024-09-17T21:18:30.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197360+0000", 2024-09-17T21:18:30.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0380000000000001e-06 2024-09-17T21:18:30.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197376+0000", 2024-09-17T21:18:30.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6147999999999999e-05 2024-09-17T21:18:30.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197392+0000", 2024-09-17T21:18:30.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6025999999999999e-05 2024-09-17T21:18:30.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197401+0000", 2024-09-17T21:18:30.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6640000000000004e-06 2024-09-17T21:18:30.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.218539+0000", 2024-09-17T21:18:30.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021137808000000001 2024-09-17T21:18:30.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.218611+0000", 2024-09-17T21:18:30.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2471000000000005e-05 2024-09-17T21:18:30.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:30.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.219199+0000", 2024-09-17T21:18:30.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00058801199999999995 2024-09-17T21:18:30.770 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.770 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.770 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.770 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:526 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 2359296~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:30.771 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.200199+0000", 2024-09-17T21:18:30.771 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.4990667860000002, 2024-09-17T21:18:30.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019695863000000001, 2024-09-17T21:18:30.771 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.771 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.771 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:30.771 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.771 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:30.771 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:30.771 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 526 2024-09-17T21:18:30.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.771 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200199+0000", 2024-09-17T21:18:30.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200199+0000", 2024-09-17T21:18:30.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200199+0000", 2024-09-17T21:18:30.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:18:30.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200205+0000", 2024-09-17T21:18:30.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.268e-06 2024-09-17T21:18:30.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200222+0000", 2024-09-17T21:18:30.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6562000000000002e-05 2024-09-17T21:18:30.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:18:30.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:18:30.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.219244+0000", 2024-09-17T21:18:30.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019016314999999999 2024-09-17T21:18:30.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.219317+0000", 2024-09-17T21:18:30.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3391000000000003e-05 2024-09-17T21:18:30.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:30.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.219894+0000", 2024-09-17T21:18:30.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00057747699999999996 2024-09-17T21:18:30.775 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.775 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.775 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.775 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:547 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:30.775 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.200266+0000", 2024-09-17T21:18:30.775 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.4989991570000001, 2024-09-17T21:18:30.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021208113000000001, 2024-09-17T21:18:30.775 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.775 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.775 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:30.775 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.775 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:30.776 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:30.776 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 547 2024-09-17T21:18:30.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.776 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200266+0000", 2024-09-17T21:18:30.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200266+0000", 2024-09-17T21:18:30.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200267+0000", 2024-09-17T21:18:30.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:18:30.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200268+0000", 2024-09-17T21:18:30.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5540000000000001e-06 2024-09-17T21:18:30.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200273+0000", 2024-09-17T21:18:30.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5549999999999996e-06 2024-09-17T21:18:30.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200276+0000", 2024-09-17T21:18:30.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.461e-06 2024-09-17T21:18:30.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.219998+0000", 2024-09-17T21:18:30.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019721596000000001 2024-09-17T21:18:30.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.220117+0000", 2024-09-17T21:18:30.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011937799999999999 2024-09-17T21:18:30.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:30.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.221474+0000", 2024-09-17T21:18:30.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0013572289999999999 2024-09-17T21:18:30.780 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.780 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.780 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.780 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:548 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:30.780 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.200281+0000", 2024-09-17T21:18:30.780 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.498984461, 2024-09-17T21:18:30.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021975320999999999, 2024-09-17T21:18:30.780 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.780 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.780 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:30.780 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.781 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:30.781 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:30.781 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 548 2024-09-17T21:18:30.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.781 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200281+0000", 2024-09-17T21:18:30.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200281+0000", 2024-09-17T21:18:30.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200281+0000", 2024-09-17T21:18:30.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5100000000000001e-07 2024-09-17T21:18:30.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200283+0000", 2024-09-17T21:18:30.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4699999999999999e-06 2024-09-17T21:18:30.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200287+0000", 2024-09-17T21:18:30.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3939999999999998e-06 2024-09-17T21:18:30.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200290+0000", 2024-09-17T21:18:30.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3160000000000001e-06 2024-09-17T21:18:30.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.221522+0000", 2024-09-17T21:18:30.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021231782000000001 2024-09-17T21:18:30.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.221624+0000", 2024-09-17T21:18:30.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101693 2024-09-17T21:18:30.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:30.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.222256+0000", 2024-09-17T21:18:30.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00063231500000000005 2024-09-17T21:18:30.784 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.784 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.784 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.785 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4520.0:6121 3.16 3:6d2e09a6:::rbd_object_map.11528c3b0529:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-17T21:18:30.785 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:30.444629+0000", 2024-09-17T21:18:30.785 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.25463619599999998, 2024-09-17T21:18:30.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011071902999999999, 2024-09-17T21:18:30.785 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.785 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.785 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:30.785 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.785 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4520", 2024-09-17T21:18:30.785 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2592434746", 2024-09-17T21:18:30.786 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6121 2024-09-17T21:18:30.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.786 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.444629+0000", 2024-09-17T21:18:30.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.444629+0000", 2024-09-17T21:18:30.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.444630+0000", 2024-09-17T21:18:30.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6699999999999999e-07 2024-09-17T21:18:30.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.444632+0000", 2024-09-17T21:18:30.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1969999999999999e-06 2024-09-17T21:18:30.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.444636+0000", 2024-09-17T21:18:30.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8850000000000001e-06 2024-09-17T21:18:30.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.444639+0000", 2024-09-17T21:18:30.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.207e-06 2024-09-17T21:18:30.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.449825+0000", 2024-09-17T21:18:30.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0051865710000000001 2024-09-17T21:18:30.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.449921+0000", 2024-09-17T21:18:30.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5698000000000004e-05 2024-09-17T21:18:30.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 3", 2024-09-17T21:18:30.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.450243+0000", 2024-09-17T21:18:30.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032173999999999999 2024-09-17T21:18:30.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:30.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.455395+0000", 2024-09-17T21:18:30.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0051522180000000001 2024-09-17T21:18:30.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:30.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.455641+0000", 2024-09-17T21:18:30.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024570499999999998 2024-09-17T21:18:30.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:30.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.455680+0000", 2024-09-17T21:18:30.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9533000000000002e-05 2024-09-17T21:18:30.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:30.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.455701+0000", 2024-09-17T21:18:30.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0682e-05 2024-09-17T21:18:30.790 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.790 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.790 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.791 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4520.0:6123 3.16 3:6d2e09a6:::rbd_object_map.11528c3b0529:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-17T21:18:30.791 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:30.444720+0000", 2024-09-17T21:18:30.791 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.254545717, 2024-09-17T21:18:30.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011063401, 2024-09-17T21:18:30.791 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.791 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.791 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:30.791 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.791 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4520", 2024-09-17T21:18:30.791 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2592434746", 2024-09-17T21:18:30.791 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6123 2024-09-17T21:18:30.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.791 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.444720+0000", 2024-09-17T21:18:30.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.444720+0000", 2024-09-17T21:18:30.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.444720+0000", 2024-09-17T21:18:30.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7799999999999997e-07 2024-09-17T21:18:30.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.444721+0000", 2024-09-17T21:18:30.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5910000000000001e-06 2024-09-17T21:18:30.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.444725+0000", 2024-09-17T21:18:30.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6500000000000002e-06 2024-09-17T21:18:30.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.444728+0000", 2024-09-17T21:18:30.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7779999999999999e-06 2024-09-17T21:18:30.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.450634+0000", 2024-09-17T21:18:30.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059060240000000002 2024-09-17T21:18:30.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.450685+0000", 2024-09-17T21:18:30.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1411e-05 2024-09-17T21:18:30.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 3", 2024-09-17T21:18:30.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.450894+0000", 2024-09-17T21:18:30.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000208765 2024-09-17T21:18:30.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:30.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.455401+0000", 2024-09-17T21:18:30.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045065210000000003 2024-09-17T21:18:30.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:30.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.455736+0000", 2024-09-17T21:18:30.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033579999999999998 2024-09-17T21:18:30.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:30.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.455762+0000", 2024-09-17T21:18:30.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5627e-05 2024-09-17T21:18:30.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:30.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.455783+0000", 2024-09-17T21:18:30.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0956000000000001e-05 2024-09-17T21:18:30.796 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.796 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.797 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.797 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4520.0:6125 3.16 3:6d2e09a6:::rbd_object_map.11528c3b0529:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-17T21:18:30.797 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:30.445097+0000", 2024-09-17T21:18:30.797 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.254168689, 2024-09-17T21:18:30.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010925167, 2024-09-17T21:18:30.797 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.797 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.797 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:30.797 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.797 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4520", 2024-09-17T21:18:30.798 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2592434746", 2024-09-17T21:18:30.798 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6125 2024-09-17T21:18:30.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.798 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.445097+0000", 2024-09-17T21:18:30.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.445097+0000", 2024-09-17T21:18:30.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.445098+0000", 2024-09-17T21:18:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8479999999999999e-06 2024-09-17T21:18:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.445108+0000", 2024-09-17T21:18:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3239999999999992e-06 2024-09-17T21:18:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.445128+0000", 2024-09-17T21:18:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0548e-05 2024-09-17T21:18:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.445142+0000", 2024-09-17T21:18:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3494e-05 2024-09-17T21:18:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.451479+0000", 2024-09-17T21:18:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0063375310000000004 2024-09-17T21:18:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.451523+0000", 2024-09-17T21:18:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3961999999999997e-05 2024-09-17T21:18:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 3", 2024-09-17T21:18:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.451765+0000", 2024-09-17T21:18:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024155199999999999 2024-09-17T21:18:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.455406+0000", 2024-09-17T21:18:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036408500000000002 2024-09-17T21:18:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.455880+0000", 2024-09-17T21:18:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00047426 2024-09-17T21:18:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.455963+0000", 2024-09-17T21:18:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2594000000000002e-05 2024-09-17T21:18:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.456022+0000", 2024-09-17T21:18:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9203999999999998e-05 2024-09-17T21:18:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4520.0:6126 3.16 3:6d2e09a6:::rbd_object_map.11528c3b0529:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-17T21:18:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:30.445166+0000", 2024-09-17T21:18:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.254099397, 2024-09-17T21:18:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011204421000000001, 2024-09-17T21:18:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4520", 2024-09-17T21:18:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2592434746", 2024-09-17T21:18:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6126 2024-09-17T21:18:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.445166+0000", 2024-09-17T21:18:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.445166+0000", 2024-09-17T21:18:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.445167+0000", 2024-09-17T21:18:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1069999999999999e-06 2024-09-17T21:18:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.445173+0000", 2024-09-17T21:18:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.57e-06 2024-09-17T21:18:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.445187+0000", 2024-09-17T21:18:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4494999999999999e-05 2024-09-17T21:18:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.445198+0000", 2024-09-17T21:18:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1e-05 2024-09-17T21:18:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.452315+0000", 2024-09-17T21:18:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.007116977 2024-09-17T21:18:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.452400+0000", 2024-09-17T21:18:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4573999999999994e-05 2024-09-17T21:18:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 3", 2024-09-17T21:18:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.452815+0000", 2024-09-17T21:18:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00041492900000000002 2024-09-17T21:18:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.456044+0000", 2024-09-17T21:18:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0032298629999999999 2024-09-17T21:18:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.456197+0000", 2024-09-17T21:18:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015273500000000001 2024-09-17T21:18:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.456304+0000", 2024-09-17T21:18:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010658300000000001 2024-09-17T21:18:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:30.456370+0000", 2024-09-17T21:18:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6587999999999993e-05 2024-09-17T21:18:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:30.808 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:30.809 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:18:30.810 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:30.806+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:30.811 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:18:30.912 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:30.906+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:30.940 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:30.940 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:30.941 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:30.941 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:30.956 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:18:31.012 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:31.006+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:31.073 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:31.073 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:31.073 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:31.073 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:31.073 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:31.087 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:18:31.113 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:31.110+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:31.215 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:31.215 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:18:31.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:18:31.215 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:18:31.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.215 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:389 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.215 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.194936+0000", 2024-09-17T21:18:31.215 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.0167636900000003, 2024-09-17T21:18:31.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041176510999999999, 2024-09-17T21:18:31.215 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.215 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.215 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.216 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.216 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.216 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.216 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 389 2024-09-17T21:18:31.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.216 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194936+0000", 2024-09-17T21:18:31.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194936+0000", 2024-09-17T21:18:31.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194937+0000", 2024-09-17T21:18:31.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6270000000000001e-06 2024-09-17T21:18:31.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194944+0000", 2024-09-17T21:18:31.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5729999999999996e-06 2024-09-17T21:18:31.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194961+0000", 2024-09-17T21:18:31.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7623000000000001e-05 2024-09-17T21:18:31.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194974+0000", 2024-09-17T21:18:31.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2989999999999999e-05 2024-09-17T21:18:31.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.235848+0000", 2024-09-17T21:18:31.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.040874099999999997 2024-09-17T21:18:31.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.235971+0000", 2024-09-17T21:18:31.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012254400000000001 2024-09-17T21:18:31.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236112+0000", 2024-09-17T21:18:31.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014105399999999999 2024-09-17T21:18:31.219 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.219 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.219 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.219 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:406 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 2359296~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.219 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:18:31.220 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.0166448839999997, 2024-09-17T21:18:31.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042448018999999997, 2024-09-17T21:18:31.220 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.220 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.220 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.220 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.220 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.220 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.220 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 406 2024-09-17T21:18:31.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.220 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:18:31.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:18:31.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195056+0000", 2024-09-17T21:18:31.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2729999999999999e-06 2024-09-17T21:18:31.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195060+0000", 2024-09-17T21:18:31.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5399999999999997e-06 2024-09-17T21:18:31.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195076+0000", 2024-09-17T21:18:31.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6217000000000001e-05 2024-09-17T21:18:31.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195093+0000", 2024-09-17T21:18:31.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.683e-05 2024-09-17T21:18:31.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236217+0000", 2024-09-17T21:18:31.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041124102000000003 2024-09-17T21:18:31.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236339+0000", 2024-09-17T21:18:31.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121876 2024-09-17T21:18:31.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237502+0000", 2024-09-17T21:18:31.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011631809999999999 2024-09-17T21:18:31.223 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.223 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.224 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.224 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:416 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2228224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.224 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:18:31.224 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.0161468579999999, 2024-09-17T21:18:31.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042361990000000002, 2024-09-17T21:18:31.224 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.224 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.224 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.224 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.224 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.224 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.224 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 416 2024-09-17T21:18:31.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.225 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:18:31.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:18:31.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195554+0000", 2024-09-17T21:18:31.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9300000000000002e-06 2024-09-17T21:18:31.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195563+0000", 2024-09-17T21:18:31.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2770000000000005e-06 2024-09-17T21:18:31.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195586+0000", 2024-09-17T21:18:31.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3889000000000001e-05 2024-09-17T21:18:31.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195603+0000", 2024-09-17T21:18:31.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6084000000000001e-05 2024-09-17T21:18:31.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237563+0000", 2024-09-17T21:18:31.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041960856999999997 2024-09-17T21:18:31.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237695+0000", 2024-09-17T21:18:31.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013144400000000001 2024-09-17T21:18:31.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237914+0000", 2024-09-17T21:18:31.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000219509 2024-09-17T21:18:31.228 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.228 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.228 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.228 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:439 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.228 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:18:31.228 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.0155829560000003, 2024-09-17T21:18:31.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043053811999999997, 2024-09-17T21:18:31.228 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.228 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.228 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.229 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.229 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.229 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.229 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 439 2024-09-17T21:18:31.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.229 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:18:31.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:18:31.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196118+0000", 2024-09-17T21:18:31.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3969999999999999e-06 2024-09-17T21:18:31.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196157+0000", 2024-09-17T21:18:31.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9826999999999997e-05 2024-09-17T21:18:31.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196179+0000", 2024-09-17T21:18:31.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1756e-05 2024-09-17T21:18:31.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196194+0000", 2024-09-17T21:18:31.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4707e-05 2024-09-17T21:18:31.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237980+0000", 2024-09-17T21:18:31.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041786219999999999 2024-09-17T21:18:31.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.238065+0000", 2024-09-17T21:18:31.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4560000000000004e-05 2024-09-17T21:18:31.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239170+0000", 2024-09-17T21:18:31.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001105345 2024-09-17T21:18:31.232 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.232 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.232 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.232 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:440 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.232 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:18:31.232 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.0154804530000003, 2024-09-17T21:18:31.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043890039999999998, 2024-09-17T21:18:31.233 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.233 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.233 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.233 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.233 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.233 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.233 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 440 2024-09-17T21:18:31.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.233 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:18:31.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:18:31.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196220+0000", 2024-09-17T21:18:31.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.254e-06 2024-09-17T21:18:31.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196225+0000", 2024-09-17T21:18:31.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7829999999999998e-06 2024-09-17T21:18:31.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196241+0000", 2024-09-17T21:18:31.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6322999999999999e-05 2024-09-17T21:18:31.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196258+0000", 2024-09-17T21:18:31.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7056000000000001e-05 2024-09-17T21:18:31.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239229+0000", 2024-09-17T21:18:31.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042971058999999999 2024-09-17T21:18:31.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239403+0000", 2024-09-17T21:18:31.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017399200000000001 2024-09-17T21:18:31.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240109+0000", 2024-09-17T21:18:31.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00070557300000000003 2024-09-17T21:18:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:450 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:18:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.0151195020000001, 2024-09-17T21:18:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043994521000000002, 2024-09-17T21:18:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 450 2024-09-17T21:18:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:18:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:18:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196581+0000", 2024-09-17T21:18:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4500000000000001e-06 2024-09-17T21:18:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196588+0000", 2024-09-17T21:18:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7530000000000004e-06 2024-09-17T21:18:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196607+0000", 2024-09-17T21:18:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8987e-05 2024-09-17T21:18:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196620+0000", 2024-09-17T21:18:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3589e-05 2024-09-17T21:18:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240225+0000", 2024-09-17T21:18:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043604878999999999 2024-09-17T21:18:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240377+0000", 2024-09-17T21:18:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015146699999999999 2024-09-17T21:18:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240574+0000", 2024-09-17T21:18:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019739599999999999 2024-09-17T21:18:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:464 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:18:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.0148690809999996, 2024-09-17T21:18:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044651692999999999, 2024-09-17T21:18:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 464 2024-09-17T21:18:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:18:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:18:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196838+0000", 2024-09-17T21:18:31.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4970000000000004e-06 2024-09-17T21:18:31.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196859+0000", 2024-09-17T21:18:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1166e-05 2024-09-17T21:18:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196879+0000", 2024-09-17T21:18:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.995e-05 2024-09-17T21:18:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196896+0000", 2024-09-17T21:18:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6929999999999999e-05 2024-09-17T21:18:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240617+0000", 2024-09-17T21:18:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043721784999999999 2024-09-17T21:18:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240702+0000", 2024-09-17T21:18:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4060999999999994e-05 2024-09-17T21:18:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241482+0000", 2024-09-17T21:18:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000780304 2024-09-17T21:18:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:475 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:18:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.0145999489999999, 2024-09-17T21:18:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044661981000000003, 2024-09-17T21:18:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 475 2024-09-17T21:18:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:18:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:18:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197100+0000", 2024-09-17T21:18:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.206e-06 2024-09-17T21:18:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197105+0000", 2024-09-17T21:18:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.527e-06 2024-09-17T21:18:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197124+0000", 2024-09-17T21:18:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:18:31.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197136+0000", 2024-09-17T21:18:31.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2857000000000001e-05 2024-09-17T21:18:31.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241539+0000", 2024-09-17T21:18:31.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044402077999999998 2024-09-17T21:18:31.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241653+0000", 2024-09-17T21:18:31.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114931 2024-09-17T21:18:31.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241761+0000", 2024-09-17T21:18:31.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107753 2024-09-17T21:18:31.250 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.250 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.250 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.250 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:476 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.251 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:18:31.251 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.0145461239999998, 2024-09-17T21:18:31.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045038976000000001, 2024-09-17T21:18:31.251 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.251 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.251 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.251 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.251 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.251 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.251 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 476 2024-09-17T21:18:31.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.251 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:18:31.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:18:31.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197154+0000", 2024-09-17T21:18:31.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.127e-06 2024-09-17T21:18:31.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197159+0000", 2024-09-17T21:18:31.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4569999999999998e-06 2024-09-17T21:18:31.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197174+0000", 2024-09-17T21:18:31.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5173e-05 2024-09-17T21:18:31.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197186+0000", 2024-09-17T21:18:31.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:18:31.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241863+0000", 2024-09-17T21:18:31.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044676826000000003 2024-09-17T21:18:31.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241998+0000", 2024-09-17T21:18:31.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013476100000000001 2024-09-17T21:18:31.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242192+0000", 2024-09-17T21:18:31.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019436900000000001 2024-09-17T21:18:31.254 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.254 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.254 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.255 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:494 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.255 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:18:31.255 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.0143709410000001, 2024-09-17T21:18:31.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046244995999999997, 2024-09-17T21:18:31.255 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.255 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.255 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.255 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.255 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.255 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.255 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 494 2024-09-17T21:18:31.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.256 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:18:31.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:18:31.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197329+0000", 2024-09-17T21:18:31.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.173e-06 2024-09-17T21:18:31.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197334+0000", 2024-09-17T21:18:31.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0499999999999999e-06 2024-09-17T21:18:31.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197352+0000", 2024-09-17T21:18:31.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7133e-05 2024-09-17T21:18:31.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197367+0000", 2024-09-17T21:18:31.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5123000000000001e-05 2024-09-17T21:18:31.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:18:31.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044989829000000002 2024-09-17T21:18:31.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242526+0000", 2024-09-17T21:18:31.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016968199999999999 2024-09-17T21:18:31.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243573+0000", 2024-09-17T21:18:31.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010470060000000001 2024-09-17T21:18:31.259 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.259 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.259 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.259 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:502 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.259 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:18:31.259 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.0121372820000003, 2024-09-17T21:18:31.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044346086999999999, 2024-09-17T21:18:31.259 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.259 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.259 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.259 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.260 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.260 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.260 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 502 2024-09-17T21:18:31.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.260 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:18:31.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:18:31.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199563+0000", 2024-09-17T21:18:31.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2929999999999999e-06 2024-09-17T21:18:31.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199569+0000", 2024-09-17T21:18:31.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5330000000000002e-06 2024-09-17T21:18:31.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199614+0000", 2024-09-17T21:18:31.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4793000000000003e-05 2024-09-17T21:18:31.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199628+0000", 2024-09-17T21:18:31.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4037e-05 2024-09-17T21:18:31.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243665+0000", 2024-09-17T21:18:31.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044037106999999999 2024-09-17T21:18:31.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243767+0000", 2024-09-17T21:18:31.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010279499999999999 2024-09-17T21:18:31.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243908+0000", 2024-09-17T21:18:31.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140529 2024-09-17T21:18:31.263 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.263 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.263 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.263 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:523 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.263 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:18:31.263 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.0116911289999999, 2024-09-17T21:18:31.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044789645000000003, 2024-09-17T21:18:31.263 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.263 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.263 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.263 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.264 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.264 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.264 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 523 2024-09-17T21:18:31.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.264 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:18:31.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:18:31.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:18:31.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9e-06 2024-09-17T21:18:31.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200021+0000", 2024-09-17T21:18:31.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9499999999999998e-06 2024-09-17T21:18:31.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200038+0000", 2024-09-17T21:18:31.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.73e-05 2024-09-17T21:18:31.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200051+0000", 2024-09-17T21:18:31.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.322e-05 2024-09-17T21:18:31.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243987+0000", 2024-09-17T21:18:31.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043935386999999999 2024-09-17T21:18:31.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244136+0000", 2024-09-17T21:18:31.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014868300000000001 2024-09-17T21:18:31.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244798+0000", 2024-09-17T21:18:31.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00066220500000000002 2024-09-17T21:18:31.267 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.267 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.267 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.267 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:524 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.267 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:18:31.267 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.0116249130000003, 2024-09-17T21:18:31.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045737414999999997, 2024-09-17T21:18:31.268 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.268 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.268 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.268 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.268 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.268 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.268 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 524 2024-09-17T21:18:31.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.268 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:18:31.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:18:31.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200076+0000", 2024-09-17T21:18:31.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:18:31.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.270 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:31.208+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:31.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200081+0000", 2024-09-17T21:18:31.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7030000000000003e-06 2024-09-17T21:18:31.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200099+0000", 2024-09-17T21:18:31.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7836999999999999e-05 2024-09-17T21:18:31.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200113+0000", 2024-09-17T21:18:31.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3570000000000001e-05 2024-09-17T21:18:31.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244866+0000", 2024-09-17T21:18:31.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044753573999999997 2024-09-17T21:18:31.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244972+0000", 2024-09-17T21:18:31.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106111 2024-09-17T21:18:31.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245812+0000", 2024-09-17T21:18:31.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083923999999999999 2024-09-17T21:18:31.272 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.272 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.272 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.273 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:525 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3670016~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.273 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:18:31.273 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.011473531, 2024-09-17T21:18:31.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046645697, 2024-09-17T21:18:31.273 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.273 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.273 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.273 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.273 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.273 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.273 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 525 2024-09-17T21:18:31.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.273 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:18:31.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:18:31.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:18:31.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:18:31.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200234+0000", 2024-09-17T21:18:31.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:18:31.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200254+0000", 2024-09-17T21:18:31.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9959999999999999e-05 2024-09-17T21:18:31.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200293+0000", 2024-09-17T21:18:31.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9755999999999997e-05 2024-09-17T21:18:31.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245871+0000", 2024-09-17T21:18:31.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045577871999999998 2024-09-17T21:18:31.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246030+0000", 2024-09-17T21:18:31.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158975 2024-09-17T21:18:31.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246871+0000", 2024-09-17T21:18:31.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084111699999999995 2024-09-17T21:18:31.277 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.277 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.277 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.277 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.277 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:18:31.277 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.0112278559999996, 2024-09-17T21:18:31.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:18:31.278 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.278 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.278 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.278 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.278 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.278 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.278 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:18:31.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.278 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:18:31.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:18:31.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:18:31.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:18:31.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:18:31.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:18:31.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:18:31.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:18:31.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:18:31.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:18:31.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:18:31.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:18:31.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:18:31.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:18:31.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:18:31.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:18:31.282 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.282 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.282 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.282 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.282 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:18:31.282 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.0110491740000001, 2024-09-17T21:18:31.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:18:31.282 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.282 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.283 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.283 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.283 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.283 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.283 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:18:31.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.283 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:18:31.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:18:31.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:18:31.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:18:31.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:18:31.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:18:31.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:18:31.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:18:31.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:18:31.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:18:31.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:18:31.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:18:31.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:18:31.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:18:31.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:18:31.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:18:31.286 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.286 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.286 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.286 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.287 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:18:31.287 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.0109899010000003, 2024-09-17T21:18:31.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:18:31.287 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.287 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.287 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.287 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.287 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.287 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.287 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:18:31.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.287 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:18:31.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:18:31.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:18:31.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:18:31.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:18:31.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:18:31.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:18:31.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:18:31.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:18:31.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:18:31.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:18:31.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:18:31.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:18:31.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:18:31.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:18:31.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:18:31.291 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.291 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.291 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.291 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.291 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:18:31.291 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.0108477059999998, 2024-09-17T21:18:31.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:18:31.291 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.291 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.291 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.291 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.291 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.292 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.292 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:18:31.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.292 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:18:31.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:18:31.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:18:31.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:18:31.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:18:31.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:18:31.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:18:31.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:18:31.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:18:31.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:18:31.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:18:31.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:18:31.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:18:31.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:18:31.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:18:31.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:18:31.295 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.295 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.295 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.295 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.295 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:18:31.295 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.0107917830000002, 2024-09-17T21:18:31.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:18:31.296 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.296 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.296 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.296 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.296 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.296 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.296 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:18:31.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.296 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:18:31.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:18:31.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:18:31.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:18:31.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:18:31.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:18:31.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:18:31.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:18:31.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:18:31.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:18:31.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:18:31.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:18:31.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:18:31.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:18:31.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:18:31.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:18:31.299 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.299 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.299 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.300 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.300 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:18:31.300 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.0107381660000003, 2024-09-17T21:18:31.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:18:31.300 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.300 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.300 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.300 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.300 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.300 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.300 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:18:31.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.301 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:18:31.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:18:31.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:18:31.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:18:31.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:18:31.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:18:31.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:18:31.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:18:31.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:18:31.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:18:31.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:18:31.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:18:31.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:18:31.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:18:31.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:18:31.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:18:31.304 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.304 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.304 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.304 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.304 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.304 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:31.304 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:18:31.310 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-17T21:18:31.314 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:31.310+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:31.415 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:31.408+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:31.441 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:31.441 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:31.441 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:31.441 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:31.456 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:18:31.516 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:31.510+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:31.617 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:31.612+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:31.629 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:31.629 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:31.629 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:31.629 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:31.629 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:31.644 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:18:31.717 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:31.714+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:31.728 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-17T21:18:31.818 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:31.812+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:31.826 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:31.826 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:18:31.827 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:18:31.827 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:18:31.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.827 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4447.0:387 3.2 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 16423936~46080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T21:18:31.827 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:21.597931+0000", 2024-09-17T21:18:31.827 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 10.226910996999999, 2024-09-17T21:18:31.827 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041500774999999997, 2024-09-17T21:18:31.827 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.827 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.827 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:31.827 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.827 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4447", 2024-09-17T21:18:31.827 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:31.827 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 387 2024-09-17T21:18:31.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.828 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.828 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.828 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.597931+0000", 2024-09-17T21:18:31.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.828 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.828 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.597931+0000", 2024-09-17T21:18:31.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.597963+0000", 2024-09-17T21:18:31.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1754999999999997e-05 2024-09-17T21:18:31.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.602945+0000", 2024-09-17T21:18:31.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049825319999999996 2024-09-17T21:18:31.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.602970+0000", 2024-09-17T21:18:31.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4956999999999998e-05 2024-09-17T21:18:31.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.602980+0000", 2024-09-17T21:18:31.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5899999999999997e-06 2024-09-17T21:18:31.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.603060+0000", 2024-09-17T21:18:31.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0412000000000002e-05 2024-09-17T21:18:31.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.603182+0000", 2024-09-17T21:18:31.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121302 2024-09-17T21:18:31.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:18:31.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.603419+0000", 2024-09-17T21:18:31.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023761499999999999 2024-09-17T21:18:31.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:31.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.611819+0000", 2024-09-17T21:18:31.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00839992 2024-09-17T21:18:31.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.832 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:31.832 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.639315+0000", 2024-09-17T21:18:31.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.027496181000000001 2024-09-17T21:18:31.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.832 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:31.832 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.639369+0000", 2024-09-17T21:18:31.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3909999999999997e-05 2024-09-17T21:18:31.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.832 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.832 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.639432+0000", 2024-09-17T21:18:31.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2601000000000006e-05 2024-09-17T21:18:31.833 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.833 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.833 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.833 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:468 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 2359296~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.833 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197658+0000", 2024-09-17T21:18:31.833 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.627183982, 2024-09-17T21:18:31.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041681556000000002, 2024-09-17T21:18:31.833 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.833 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.833 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.833 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.834 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.834 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.834 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 468 2024-09-17T21:18:31.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.834 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197658+0000", 2024-09-17T21:18:31.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197658+0000", 2024-09-17T21:18:31.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.835 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.835 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197659+0000", 2024-09-17T21:18:31.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.144e-06 2024-09-17T21:18:31.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.835 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.835 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197666+0000", 2024-09-17T21:18:31.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.323e-06 2024-09-17T21:18:31.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197683+0000", 2024-09-17T21:18:31.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6699999999999999e-05 2024-09-17T21:18:31.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197697+0000", 2024-09-17T21:18:31.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3713000000000001e-05 2024-09-17T21:18:31.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236385+0000", 2024-09-17T21:18:31.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.038687988 2024-09-17T21:18:31.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236571+0000", 2024-09-17T21:18:31.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000186102 2024-09-17T21:18:31.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239340+0000", 2024-09-17T21:18:31.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002768586 2024-09-17T21:18:31.838 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.838 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.838 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.838 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:491 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.838 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:18:31.838 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.6265281030000001, 2024-09-17T21:18:31.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041839013000000001, 2024-09-17T21:18:31.838 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.838 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.839 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.839 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.839 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.839 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.839 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 491 2024-09-17T21:18:31.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.839 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:18:31.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:18:31.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198315+0000", 2024-09-17T21:18:31.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6599999999999996e-07 2024-09-17T21:18:31.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198317+0000", 2024-09-17T21:18:31.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6639999999999998e-06 2024-09-17T21:18:31.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.841 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.841 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198327+0000", 2024-09-17T21:18:31.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5629999999999998e-06 2024-09-17T21:18:31.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.841 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.841 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198337+0000", 2024-09-17T21:18:31.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0667e-05 2024-09-17T21:18:31.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.841 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.842 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239793+0000", 2024-09-17T21:18:31.842 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041455519000000003 2024-09-17T21:18:31.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.842 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.842 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239929+0000", 2024-09-17T21:18:31.842 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135716 2024-09-17T21:18:31.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.842 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.842 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240153+0000", 2024-09-17T21:18:31.842 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022421799999999999 2024-09-17T21:18:31.843 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.843 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.843 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.843 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:492 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.843 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:18:31.843 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.6264897329999997, 2024-09-17T21:18:31.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042947165000000002, 2024-09-17T21:18:31.843 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.843 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.843 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.843 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.843 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.844 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.844 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 492 2024-09-17T21:18:31.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.844 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:18:31.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:18:31.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198354+0000", 2024-09-17T21:18:31.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:18:31.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198357+0000", 2024-09-17T21:18:31.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6969999999999999e-06 2024-09-17T21:18:31.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198367+0000", 2024-09-17T21:18:31.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8430000000000005e-06 2024-09-17T21:18:31.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198374+0000", 2024-09-17T21:18:31.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9600000000000003e-06 2024-09-17T21:18:31.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240226+0000", 2024-09-17T21:18:31.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041852157000000001 2024-09-17T21:18:31.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.847 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.847 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240409+0000", 2024-09-17T21:18:31.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018244800000000001 2024-09-17T21:18:31.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.847 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.847 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.847 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241299+0000", 2024-09-17T21:18:31.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000890554 2024-09-17T21:18:31.847 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.847 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.847 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.847 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.847 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:493 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 4063232~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.848 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:18:31.848 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.6264526469999998, 2024-09-17T21:18:31.848 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043897209, 2024-09-17T21:18:31.848 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.848 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.848 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.848 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.848 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.848 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.849 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 493 2024-09-17T21:18:31.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.849 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.849 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.849 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:18:31.849 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.849 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.849 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:18:31.849 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198391+0000", 2024-09-17T21:18:31.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4699999999999999e-06 2024-09-17T21:18:31.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198396+0000", 2024-09-17T21:18:31.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:18:31.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198412+0000", 2024-09-17T21:18:31.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6507e-05 2024-09-17T21:18:31.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.851 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198425+0000", 2024-09-17T21:18:31.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.256e-05 2024-09-17T21:18:31.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.851 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241383+0000", 2024-09-17T21:18:31.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042958216 2024-09-17T21:18:31.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.852 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241525+0000", 2024-09-17T21:18:31.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014204500000000001 2024-09-17T21:18:31.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.852 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.852 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242287+0000", 2024-09-17T21:18:31.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076125800000000001 2024-09-17T21:18:31.852 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.852 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.852 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.852 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:496 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.853 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:18:31.853 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.6263622479999995, 2024-09-17T21:18:31.853 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043732500000000001, 2024-09-17T21:18:31.853 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.853 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.853 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.853 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.853 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.853 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.853 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 496 2024-09-17T21:18:31.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.853 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.853 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.853 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.854 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:18:31.854 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.854 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.854 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.854 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:18:31.854 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.854 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.854 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.854 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198481+0000", 2024-09-17T21:18:31.854 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-17T21:18:31.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.855 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198488+0000", 2024-09-17T21:18:31.855 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8499999999999996e-06 2024-09-17T21:18:31.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.855 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198507+0000", 2024-09-17T21:18:31.855 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9276999999999999e-05 2024-09-17T21:18:31.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.855 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198521+0000", 2024-09-17T21:18:31.855 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3376e-05 2024-09-17T21:18:31.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.856 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.856 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241339+0000", 2024-09-17T21:18:31.856 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042818119000000002 2024-09-17T21:18:31.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.856 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.856 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241449+0000", 2024-09-17T21:18:31.856 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110209 2024-09-17T21:18:31.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.857 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.857 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242212+0000", 2024-09-17T21:18:31.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076329900000000005 2024-09-17T21:18:31.857 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.857 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.857 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.857 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.857 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:497 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.857 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:18:31.857 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.6262812990000004, 2024-09-17T21:18:31.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044584278999999997, 2024-09-17T21:18:31.857 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.857 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.858 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.858 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.858 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.858 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.858 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 497 2024-09-17T21:18:31.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.858 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.858 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.858 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.858 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:18:31.858 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.858 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.859 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.859 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:18:31.859 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.859 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.859 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.859 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198562+0000", 2024-09-17T21:18:31.859 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-17T21:18:31.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.859 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.859 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.859 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198569+0000", 2024-09-17T21:18:31.859 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.494e-06 2024-09-17T21:18:31.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198588+0000", 2024-09-17T21:18:31.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8955999999999999e-05 2024-09-17T21:18:31.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198621+0000", 2024-09-17T21:18:31.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3837000000000001e-05 2024-09-17T21:18:31.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242258+0000", 2024-09-17T21:18:31.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043636900999999999 2024-09-17T21:18:31.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242343+0000", 2024-09-17T21:18:31.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5048000000000003e-05 2024-09-17T21:18:31.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243145+0000", 2024-09-17T21:18:31.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00080167300000000003 2024-09-17T21:18:31.861 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.862 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.862 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.862 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:503 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.862 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:18:31.862 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.6261967750000004, 2024-09-17T21:18:31.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044220582000000001, 2024-09-17T21:18:31.862 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.862 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.862 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.862 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.862 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.863 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.863 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 503 2024-09-17T21:18:31.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.863 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.863 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.863 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:18:31.863 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.863 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.863 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:18:31.863 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198647+0000", 2024-09-17T21:18:31.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.483e-06 2024-09-17T21:18:31.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198653+0000", 2024-09-17T21:18:31.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4529999999999999e-06 2024-09-17T21:18:31.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198672+0000", 2024-09-17T21:18:31.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9193000000000001e-05 2024-09-17T21:18:31.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198685+0000", 2024-09-17T21:18:31.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.291e-05 2024-09-17T21:18:31.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:18:31.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043672141999999997 2024-09-17T21:18:31.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.866 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.866 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242716+0000", 2024-09-17T21:18:31.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00035901800000000001 2024-09-17T21:18:31.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.866 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.866 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242866+0000", 2024-09-17T21:18:31.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000149383 2024-09-17T21:18:31.866 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.866 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.866 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.867 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:504 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.867 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:18:31.867 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.6261369830000003, 2024-09-17T21:18:31.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044611406999999999, 2024-09-17T21:18:31.867 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.867 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.867 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.867 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.867 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.867 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.867 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 504 2024-09-17T21:18:31.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.867 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:18:31.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:18:31.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198706+0000", 2024-09-17T21:18:31.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:18:31.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198712+0000", 2024-09-17T21:18:31.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5770000000000001e-06 2024-09-17T21:18:31.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198729+0000", 2024-09-17T21:18:31.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6673000000000001e-05 2024-09-17T21:18:31.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198742+0000", 2024-09-17T21:18:31.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3543000000000001e-05 2024-09-17T21:18:31.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.870 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242981+0000", 2024-09-17T21:18:31.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044238953999999997 2024-09-17T21:18:31.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.870 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243155+0000", 2024-09-17T21:18:31.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017401600000000001 2024-09-17T21:18:31.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.871 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243316+0000", 2024-09-17T21:18:31.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016137399999999999 2024-09-17T21:18:31.871 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.871 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.871 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.871 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:521 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.871 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:18:31.871 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.6255898440000003, 2024-09-17T21:18:31.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045043685999999999, 2024-09-17T21:18:31.871 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.871 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.871 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.872 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.872 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.872 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.872 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 521 2024-09-17T21:18:31.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.872 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:18:31.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:18:31.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.873 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199253+0000", 2024-09-17T21:18:31.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1400000000000001e-06 2024-09-17T21:18:31.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.873 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199258+0000", 2024-09-17T21:18:31.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1800000000000004e-06 2024-09-17T21:18:31.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.873 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.874 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199276+0000", 2024-09-17T21:18:31.874 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7686999999999999e-05 2024-09-17T21:18:31.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.874 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.874 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.874 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199289+0000", 2024-09-17T21:18:31.874 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3263e-05 2024-09-17T21:18:31.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.874 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.874 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.874 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243397+0000", 2024-09-17T21:18:31.874 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044107636999999998 2024-09-17T21:18:31.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.875 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.875 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.875 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243527+0000", 2024-09-17T21:18:31.875 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013038399999999999 2024-09-17T21:18:31.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.875 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.875 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.875 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244296+0000", 2024-09-17T21:18:31.875 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076839499999999995 2024-09-17T21:18:31.875 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.875 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.875 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.875 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.876 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:522 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.876 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:18:31.876 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.6255345410000004, 2024-09-17T21:18:31.876 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046049870999999999, 2024-09-17T21:18:31.876 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.876 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.876 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.876 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.876 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.876 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.876 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 522 2024-09-17T21:18:31.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.876 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:18:31.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:18:31.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199309+0000", 2024-09-17T21:18:31.878 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:18:31.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.878 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.878 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199315+0000", 2024-09-17T21:18:31.878 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2440000000000001e-06 2024-09-17T21:18:31.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.878 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.878 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199333+0000", 2024-09-17T21:18:31.878 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7886000000000001e-05 2024-09-17T21:18:31.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.879 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.879 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199379+0000", 2024-09-17T21:18:31.879 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6383e-05 2024-09-17T21:18:31.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.879 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.879 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244394+0000", 2024-09-17T21:18:31.879 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045014851000000002 2024-09-17T21:18:31.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.879 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.879 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244491+0000", 2024-09-17T21:18:31.879 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7245999999999997e-05 2024-09-17T21:18:31.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.880 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.880 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245357+0000", 2024-09-17T21:18:31.880 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00086626800000000001 2024-09-17T21:18:31.880 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.880 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.880 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.880 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:527 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.880 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:18:31.881 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.6254378750000003, 2024-09-17T21:18:31.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045812522000000001, 2024-09-17T21:18:31.881 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.881 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.881 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.881 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.881 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.881 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.881 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 527 2024-09-17T21:18:31.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.881 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.882 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:18:31.882 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.882 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.882 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:18:31.882 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.882 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.882 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199406+0000", 2024-09-17T21:18:31.882 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.367e-06 2024-09-17T21:18:31.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.883 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.883 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199412+0000", 2024-09-17T21:18:31.883 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:18:31.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.883 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.883 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199429+0000", 2024-09-17T21:18:31.883 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7396000000000001e-05 2024-09-17T21:18:31.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.883 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.883 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199442+0000", 2024-09-17T21:18:31.883 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3327e-05 2024-09-17T21:18:31.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.884 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.884 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244361+0000", 2024-09-17T21:18:31.884 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044918197 2024-09-17T21:18:31.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.884 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.884 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244482+0000", 2024-09-17T21:18:31.884 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121945 2024-09-17T21:18:31.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.884 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.885 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245217+0000", 2024-09-17T21:18:31.885 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00073415999999999998 2024-09-17T21:18:31.885 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.885 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.885 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.885 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:534 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.885 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:18:31.885 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.6253791690000003, 2024-09-17T21:18:31.885 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046292505999999997, 2024-09-17T21:18:31.885 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.885 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.885 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.886 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.886 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.886 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.886 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:18:31.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.886 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.886 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.886 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:18:31.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.886 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.887 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:18:31.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.887 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.887 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199464+0000", 2024-09-17T21:18:31.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:18:31.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.887 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.887 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199473+0000", 2024-09-17T21:18:31.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5629999999999992e-06 2024-09-17T21:18:31.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199492+0000", 2024-09-17T21:18:31.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9273999999999999e-05 2024-09-17T21:18:31.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199505+0000", 2024-09-17T21:18:31.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3443e-05 2024-09-17T21:18:31.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245357+0000", 2024-09-17T21:18:31.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045851714000000002 2024-09-17T21:18:31.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245550+0000", 2024-09-17T21:18:31.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019301600000000001 2024-09-17T21:18:31.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245755+0000", 2024-09-17T21:18:31.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205226 2024-09-17T21:18:31.889 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.889 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.889 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.890 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:535 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.890 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:18:31.890 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.6252746130000002, 2024-09-17T21:18:31.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046637492000000003, 2024-09-17T21:18:31.890 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.890 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.890 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.890 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.890 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.890 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.890 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 535 2024-09-17T21:18:31.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.890 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:18:31.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:18:31.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199568+0000", 2024-09-17T21:18:31.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.037e-06 2024-09-17T21:18:31.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199572+0000", 2024-09-17T21:18:31.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5729999999999999e-06 2024-09-17T21:18:31.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199585+0000", 2024-09-17T21:18:31.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.279e-05 2024-09-17T21:18:31.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.893 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199592+0000", 2024-09-17T21:18:31.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.61e-06 2024-09-17T21:18:31.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.893 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.893 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245873+0000", 2024-09-17T21:18:31.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046280494999999998 2024-09-17T21:18:31.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.893 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.893 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246018+0000", 2024-09-17T21:18:31.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000144697 2024-09-17T21:18:31.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246205+0000", 2024-09-17T21:18:31.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018729 2024-09-17T21:18:31.894 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.894 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.894 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.894 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:543 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.894 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:18:31.894 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.6251793670000003, 2024-09-17T21:18:31.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047564097999999999, 2024-09-17T21:18:31.895 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.895 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.895 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.895 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.895 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.895 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.895 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 543 2024-09-17T21:18:31.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.895 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:18:31.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:18:31.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:18:31.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9699999999999995e-07 2024-09-17T21:18:31.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199677+0000", 2024-09-17T21:18:31.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:18:31.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199690+0000", 2024-09-17T21:18:31.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2977e-05 2024-09-17T21:18:31.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199705+0000", 2024-09-17T21:18:31.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4656e-05 2024-09-17T21:18:31.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246307+0000", 2024-09-17T21:18:31.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046601780000000002 2024-09-17T21:18:31.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246475+0000", 2024-09-17T21:18:31.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016834200000000001 2024-09-17T21:18:31.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247227+0000", 2024-09-17T21:18:31.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00075173299999999998 2024-09-17T21:18:31.899 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.899 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.899 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.899 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:544 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.899 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:18:31.899 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.6251209580000001, 2024-09-17T21:18:31.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049283559999999997, 2024-09-17T21:18:31.899 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.899 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.899 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.899 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.899 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.900 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.900 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 544 2024-09-17T21:18:31.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.900 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:18:31.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:18:31.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199722+0000", 2024-09-17T21:18:31.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.08e-06 2024-09-17T21:18:31.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199738+0000", 2024-09-17T21:18:31.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5943999999999998e-05 2024-09-17T21:18:31.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199748+0000", 2024-09-17T21:18:31.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0193e-05 2024-09-17T21:18:31.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199756+0000", 2024-09-17T21:18:31.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4200000000000001e-06 2024-09-17T21:18:31.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247311+0000", 2024-09-17T21:18:31.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047555300000000002 2024-09-17T21:18:31.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247571+0000", 2024-09-17T21:18:31.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025996999999999999 2024-09-17T21:18:31.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249005+0000", 2024-09-17T21:18:31.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001433653 2024-09-17T21:18:31.903 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.903 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.903 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.904 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:549 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.904 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:18:31.904 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.6250753339999999, 2024-09-17T21:18:31.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048464528999999999, 2024-09-17T21:18:31.904 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.904 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.904 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.904 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.904 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.904 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.904 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 549 2024-09-17T21:18:31.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.904 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:18:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:18:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:18:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:18:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199770+0000", 2024-09-17T21:18:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9969999999999999e-06 2024-09-17T21:18:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199780+0000", 2024-09-17T21:18:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0003000000000001e-05 2024-09-17T21:18:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199793+0000", 2024-09-17T21:18:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2074e-05 2024-09-17T21:18:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247289+0000", 2024-09-17T21:18:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047496691000000001 2024-09-17T21:18:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247437+0000", 2024-09-17T21:18:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014807899999999999 2024-09-17T21:18:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248231+0000", 2024-09-17T21:18:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00079390899999999998 2024-09-17T21:18:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:558 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:18:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.6250348209999999, 2024-09-17T21:18:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049709821000000001, 2024-09-17T21:18:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 558 2024-09-17T21:18:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:18:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:18:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199809+0000", 2024-09-17T21:18:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.333e-06 2024-09-17T21:18:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199814+0000", 2024-09-17T21:18:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0930000000000002e-06 2024-09-17T21:18:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199831+0000", 2024-09-17T21:18:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7580000000000001e-05 2024-09-17T21:18:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199860+0000", 2024-09-17T21:18:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8609999999999999e-05 2024-09-17T21:18:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249106+0000", 2024-09-17T21:18:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049246545000000003 2024-09-17T21:18:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249274+0000", 2024-09-17T21:18:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016806699999999999 2024-09-17T21:18:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249517+0000", 2024-09-17T21:18:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024259300000000001 2024-09-17T21:18:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:559 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:18:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.6249154590000003, 2024-09-17T21:18:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049990694000000002, 2024-09-17T21:18:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.914 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 559 2024-09-17T21:18:31.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.914 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:18:31.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:18:31.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199928+0000", 2024-09-17T21:18:31.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:18:31.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199935+0000", 2024-09-17T21:18:31.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5429999999999999e-06 2024-09-17T21:18:31.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199954+0000", 2024-09-17T21:18:31.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.912e-05 2024-09-17T21:18:31.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199969+0000", 2024-09-17T21:18:31.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5305999999999999e-05 2024-09-17T21:18:31.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249604+0000", 2024-09-17T21:18:31.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049634931 2024-09-17T21:18:31.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249738+0000", 2024-09-17T21:18:31.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013411599999999999 2024-09-17T21:18:31.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249917+0000", 2024-09-17T21:18:31.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179111 2024-09-17T21:18:31.917 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.917 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.917 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.917 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:560 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:31.918 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:18:31.918 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.6248510429999996, 2024-09-17T21:18:31.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050173059999999998, 2024-09-17T21:18:31.918 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:31.918 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:31.918 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:31.918 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:31.918 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:31.918 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:31.918 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 560 2024-09-17T21:18:31.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.918 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:31.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:31.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:18:31.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:31.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:18:31.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:31.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:31.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199992+0000", 2024-09-17T21:18:31.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.517e-06 2024-09-17T21:18:31.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:31.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199998+0000", 2024-09-17T21:18:31.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1460000000000004e-06 2024-09-17T21:18:31.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:31.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:18:31.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7407000000000001e-05 2024-09-17T21:18:31.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:31.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200029+0000", 2024-09-17T21:18:31.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3499999999999999e-05 2024-09-17T21:18:31.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:31.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249968+0000", 2024-09-17T21:18:31.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049939799 2024-09-17T21:18:31.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:31.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250055+0000", 2024-09-17T21:18:31.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6731999999999996e-05 2024-09-17T21:18:31.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:31.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:31.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:31.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250164+0000", 2024-09-17T21:18:31.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108959 2024-09-17T21:18:31.922 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.922 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.922 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.922 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:31.922 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:31.922 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:31.923 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:18:31.924 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:31.918+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:31.927 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:18:31.927 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:18:31.927 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-17T21:18:31.928 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T21:18:32.024 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:32.018+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:32.106 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:32.107 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:32.107 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:32.107 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:32.122 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:18:32.172 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:32.168+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:32.214 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:18:32.241 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:32.241 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:32.242 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:32.236+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:32.256 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:18:32.291 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:18:32.291 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:32.291 INFO:teuthology.orchestra.run.smithi083.stdout: "epoch": 36, 2024-09-17T21:18:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "fsid": "224a7a29-e112-472e-8a13-f67345d6d123", 2024-09-17T21:18:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "created": "2024-09-17T21:17:42.597312+0000", 2024-09-17T21:18:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "modified": "2024-09-17T21:18:31.720715+0000", 2024-09-17T21:18:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "last_up_change": "2024-09-17T21:17:53.976897+0000", 2024-09-17T21:18:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "last_in_change": "2024-09-17T21:17:44.301318+0000", 2024-09-17T21:18:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T21:18:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_num": 5799936, 2024-09-17T21:18:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_set": [ 2024-09-17T21:18:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "pglog_hardlimit", 2024-09-17T21:18:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "purged_snapdirs", 2024-09-17T21:18:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_deletes", 2024-09-17T21:18:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "sortbitwise" 2024-09-17T21:18:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:18:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_version": 5, 2024-09-17T21:18:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T21:18:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T21:18:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T21:18:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_snapshot": "", 2024-09-17T21:18:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_max": 3, 2024-09-17T21:18:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd": 6, 2024-09-17T21:18:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "require_min_compat_client": "luminous", 2024-09-17T21:18:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "min_compat_client": "jewel", 2024-09-17T21:18:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "require_osd_release": "squid", 2024-09-17T21:18:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_crimson": false, 2024-09-17T21:18:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "pools": [ 2024-09-17T21:18:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 1, 2024-09-17T21:18:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": ".mgr", 2024-09-17T21:18:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-17T21:17:55.581350+0000", 2024-09-17T21:18:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-17T21:18:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-17T21:18:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-17T21:18:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 2, 2024-09-17T21:18:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 1, 2024-09-17T21:18:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-17T21:18:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:18:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:18:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:18:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:18:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-17T21:18:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-17T21:18:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:18:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 1, 2024-09-17T21:18:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 1, 2024-09-17T21:18:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 1, 2024-09-17T21:18:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 1, 2024-09-17T21:18:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 1, 2024-09-17T21:18:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-17T21:18:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-17T21:18:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-17T21:18:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-17T21:18:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-17T21:18:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-17T21:18:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-17T21:18:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "17", 2024-09-17T21:18:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-17T21:18:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:18:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:18:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-17T21:18:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:18:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-17T21:18:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-17T21:18:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-17T21:18:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-17T21:18:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-17T21:18:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-17T21:18:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-17T21:18:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-17T21:18:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-17T21:18:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-17T21:18:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-17T21:18:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-17T21:18:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-17T21:18:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:18:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:18:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:18:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-17T21:18:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-17T21:18:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-17T21:18:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-17T21:18:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-17T21:18:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-17T21:18:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-17T21:18:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-17T21:18:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:18:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:18:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:18:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:18:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-17T21:18:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-17T21:18:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-17T21:18:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-17T21:18:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: "options": { 2024-09-17T21:18:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_max": 32, 2024-09-17T21:18:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_min": 1 2024-09-17T21:18:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-17T21:18:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": {} 2024-09-17T21:18:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-17T21:18:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-17T21:18:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 6.059999942779541, 2024-09-17T21:18:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 6.059999942779541, 2024-09-17T21:18:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 0.33000001311302185, 2024-09-17T21:18:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2, 2024-09-17T21:18:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2, 2024-09-17T21:18:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 0.87948358058929443, 2024-09-17T21:18:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-17T21:18:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-17T21:18:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T21:18:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "rbd", 2024-09-17T21:18:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-17T21:17:59.603025+0000", 2024-09-17T21:18:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 8193, 2024-09-17T21:18:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:18:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-17T21:18:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 2, 2024-09-17T21:18:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 1, 2024-09-17T21:18:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-17T21:18:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:18:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:18:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:18:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:18:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-17T21:18:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-17T21:18:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:18:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 8, 2024-09-17T21:18:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 8, 2024-09-17T21:18:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 8, 2024-09-17T21:18:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 8, 2024-09-17T21:18:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 8, 2024-09-17T21:18:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-17T21:18:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-17T21:18:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-17T21:18:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-17T21:18:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-17T21:18:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-17T21:18:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-17T21:18:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "21", 2024-09-17T21:18:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-17T21:18:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:18:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:18:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-17T21:18:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:18:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 2, 2024-09-17T21:18:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 21, 2024-09-17T21:18:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-17T21:18:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-17T21:18:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-17T21:18:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-17T21:18:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-17T21:18:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-17T21:18:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-17T21:18:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-17T21:18:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-17T21:18:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-17T21:18:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-17T21:18:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:18:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:18:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:18:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-17T21:18:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-17T21:18:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-17T21:18:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-17T21:18:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-17T21:18:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-17T21:18:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-17T21:18:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-17T21:18:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:18:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:18:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:18:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:18:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-17T21:18:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-17T21:18:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-17T21:18:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-17T21:18:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-17T21:18:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-17T21:18:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd": {} 2024-09-17T21:18:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-17T21:18:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-17T21:18:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 2.25, 2024-09-17T21:18:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 2.25, 2024-09-17T21:18:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-17T21:18:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2.25, 2024-09-17T21:18:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2.25, 2024-09-17T21:18:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 0.87948358058929443, 2024-09-17T21:18:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-17T21:18:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-17T21:18:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 3, 2024-09-17T21:18:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "pool_client.0", 2024-09-17T21:18:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-17T21:18:09.441137+0000", 2024-09-17T21:18:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 8193, 2024-09-17T21:18:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:18:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-17T21:18:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 2, 2024-09-17T21:18:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 1, 2024-09-17T21:18:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-17T21:18:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:18:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:18:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:18:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:18:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-17T21:18:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-17T21:18:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:18:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-17T21:18:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-17T21:18:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-17T21:18:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-17T21:18:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-17T21:18:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-17T21:18:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-17T21:18:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-17T21:18:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-17T21:18:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-17T21:18:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-17T21:18:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-17T21:18:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "35", 2024-09-17T21:18:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-17T21:18:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:18:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:18:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-17T21:18:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:18:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 8, 2024-09-17T21:18:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 35, 2024-09-17T21:18:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-17T21:18:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-17T21:18:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-17T21:18:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-17T21:18:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-17T21:18:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-17T21:18:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-17T21:18:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-17T21:18:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-17T21:18:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-17T21:18:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-17T21:18:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:18:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:18:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:18:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-17T21:18:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-17T21:18:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-17T21:18:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-17T21:18:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-17T21:18:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-17T21:18:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-17T21:18:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-17T21:18:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:18:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:18:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:18:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:18:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-17T21:18:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-17T21:18:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-17T21:18:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-17T21:18:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-17T21:18:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-17T21:18:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd": {} 2024-09-17T21:18:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-17T21:18:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-17T21:18:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.5, 2024-09-17T21:18:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.5, 2024-09-17T21:18:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-17T21:18:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.5, 2024-09-17T21:18:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.5, 2024-09-17T21:18:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 0.87948358058929443, 2024-09-17T21:18:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-17T21:18:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-17T21:18:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:18:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-17T21:18:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-17T21:18:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "5bfd41c9-0cac-47c4-9a6e-8dce774c3314", 2024-09-17T21:18:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:18:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:18:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:18:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-17T21:18:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:18:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:18:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:18:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 27, 2024-09-17T21:18:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:18:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:18:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:18:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6816", 2024-09-17T21:18:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6817", 2024-09-17T21:18:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:18:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6818", 2024-09-17T21:18:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6819", 2024-09-17T21:18:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:18:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6822", 2024-09-17T21:18:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6823", 2024-09-17T21:18:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:18:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6820", 2024-09-17T21:18:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6821", 2024-09-17T21:18:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6817/1111036754", 2024-09-17T21:18:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6819/1111036754", 2024-09-17T21:18:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6823/1111036754", 2024-09-17T21:18:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6821/1111036754", 2024-09-17T21:18:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:18:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:18:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:18:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-17T21:18:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "78404e0d-d397-4806-8e90-5f32182643bb", 2024-09-17T21:18:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:18:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:18:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:18:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-17T21:18:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:18:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:18:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:18:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 27, 2024-09-17T21:18:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:18:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:18:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:18:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6808", 2024-09-17T21:18:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6809", 2024-09-17T21:18:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:18:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6810", 2024-09-17T21:18:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6811", 2024-09-17T21:18:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:18:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6814", 2024-09-17T21:18:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6815", 2024-09-17T21:18:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:18:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6812", 2024-09-17T21:18:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6813", 2024-09-17T21:18:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6809/1460363937", 2024-09-17T21:18:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6811/1460363937", 2024-09-17T21:18:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6815/1460363937", 2024-09-17T21:18:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6813/1460363937", 2024-09-17T21:18:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:18:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:18:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:18:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-17T21:18:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "286ed6e0-c121-4ebb-8cf6-37e506a397aa", 2024-09-17T21:18:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:18:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:18:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:18:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 0.2769012451171875, 2024-09-17T21:18:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:18:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:18:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 12, 2024-09-17T21:18:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 23, 2024-09-17T21:18:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:18:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:18:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:18:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6800", 2024-09-17T21:18:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6801", 2024-09-17T21:18:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:18:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6802", 2024-09-17T21:18:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6803", 2024-09-17T21:18:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:18:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6806", 2024-09-17T21:18:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6807", 2024-09-17T21:18:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:18:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6804", 2024-09-17T21:18:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6805", 2024-09-17T21:18:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6801/2837444034", 2024-09-17T21:18:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6803/2837444034", 2024-09-17T21:18:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6807/2837444034", 2024-09-17T21:18:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6805/2837444034", 2024-09-17T21:18:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:18:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:18:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:18:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-17T21:18:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "b0af73a1-4bca-4892-a550-2ced87f4c357", 2024-09-17T21:18:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:18:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:18:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:18:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-17T21:18:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:18:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:18:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:18:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 27, 2024-09-17T21:18:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:18:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:18:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:18:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6808", 2024-09-17T21:18:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6809", 2024-09-17T21:18:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:18:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6810", 2024-09-17T21:18:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6811", 2024-09-17T21:18:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:18:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6814", 2024-09-17T21:18:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6815", 2024-09-17T21:18:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:18:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6812", 2024-09-17T21:18:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6813", 2024-09-17T21:18:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.97:6809/3637204865", 2024-09-17T21:18:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.97:6811/3637204865", 2024-09-17T21:18:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.97:6815/3637204865", 2024-09-17T21:18:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.97:6813/3637204865", 2024-09-17T21:18:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:18:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:18:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:18:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-17T21:18:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "2188361c-e1a1-45fa-bc01-da5a984ffd9f", 2024-09-17T21:18:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:18:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:18:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:18:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-17T21:18:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:18:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:18:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:18:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 23, 2024-09-17T21:18:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:18:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:18:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:18:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6800", 2024-09-17T21:18:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6801", 2024-09-17T21:18:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:18:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6802", 2024-09-17T21:18:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6803", 2024-09-17T21:18:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:18:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6806", 2024-09-17T21:18:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6807", 2024-09-17T21:18:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:18:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6804", 2024-09-17T21:18:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6805", 2024-09-17T21:18:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.97:6801/2391356911", 2024-09-17T21:18:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.97:6803/2391356911", 2024-09-17T21:18:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.97:6807/2391356911", 2024-09-17T21:18:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.97:6805/2391356911", 2024-09-17T21:18:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:18:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:18:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:18:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-17T21:18:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "7feccf4d-d8e4-4d06-ac83-bbc1bc623a5c", 2024-09-17T21:18:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:18:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:18:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:18:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-17T21:18:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:18:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:18:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:18:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 23, 2024-09-17T21:18:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:18:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:18:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:18:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6816", 2024-09-17T21:18:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6817", 2024-09-17T21:18:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:18:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6818", 2024-09-17T21:18:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6819", 2024-09-17T21:18:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:18:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6822", 2024-09-17T21:18:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6823", 2024-09-17T21:18:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:18:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6820", 2024-09-17T21:18:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6821", 2024-09-17T21:18:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.97:6817/2326316381", 2024-09-17T21:18:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.97:6819/2326316381", 2024-09-17T21:18:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.97:6823/2326316381", 2024-09-17T21:18:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.97:6821/2326316381", 2024-09-17T21:18:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:18:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:18:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:18:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:18:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_xinfo": [ 2024-09-17T21:18:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-17T21:18:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:18:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:18:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:18:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:18:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:18:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:51.138101+0000", 2024-09-17T21:18:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:18:32.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.350 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-17T21:18:32.350 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:18:32.350 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:18:32.350 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:18:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:18:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:18:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:50.581632+0000", 2024-09-17T21:18:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:18:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-17T21:18:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:18:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:18:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:18:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:18:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:18:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:50.365193+0000", 2024-09-17T21:18:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:18:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-17T21:18:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:18:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:18:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:18:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:18:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:18:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:50.718974+0000", 2024-09-17T21:18:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:18:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-17T21:18:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:18:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:18:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:18:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:18:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:18:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:50.501878+0000", 2024-09-17T21:18:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:18:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-17T21:18:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:18:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:18:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:18:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:18:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:18:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:51.150747+0000", 2024-09-17T21:18:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:18:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:18:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap": [], 2024-09-17T21:18:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_items": [], 2024-09-17T21:18:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_primaries": [], 2024-09-17T21:18:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_temp": [], 2024-09-17T21:18:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_temp": [], 2024-09-17T21:18:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklist": {}, 2024-09-17T21:18:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "range_blocklist": {}, 2024-09-17T21:18:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profiles": { 2024-09-17T21:18:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "default": { 2024-09-17T21:18:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-failure-domain": "osd", 2024-09-17T21:18:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "k": "2", 2024-09-17T21:18:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "m": "1", 2024-09-17T21:18:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "plugin": "jerasure", 2024-09-17T21:18:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "technique": "reed_sol_van" 2024-09-17T21:18:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps_queue": [], 2024-09-17T21:18:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "new_removed_snaps": [], 2024-09-17T21:18:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: "new_purged_snaps": [], 2024-09-17T21:18:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_node_flags": {}, 2024-09-17T21:18:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: "device_class_flags": {}, 2024-09-17T21:18:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode": { 2024-09-17T21:18:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_enabled": false, 2024-09-17T21:18:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_bucket_count": 0, 2024-09-17T21:18:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: "degraded_stretch_mode": 0, 2024-09-17T21:18:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: "recovering_stretch_mode": 0, 2024-09-17T21:18:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_bucket": 0 2024-09-17T21:18:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:32.357 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:32.359 INFO:tasks.thrashosds.thrasher:j is {'epoch': 36, 'fsid': '224a7a29-e112-472e-8a13-f67345d6d123', 'created': '2024-09-17T21:17:42.597312+0000', 'modified': '2024-09-17T21:18:31.720715+0000', 'last_up_change': '2024-09-17T21:17:53.976897+0000', 'last_in_change': '2024-09-17T21:17:44.301318+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T21:17:55.581350+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8794835805892944, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T21:17:59.603025+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.8794835805892944, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T21:18:09.441137+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 8, 'snap_epoch': 35, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.8794835805892944, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '5bfd41c9-0cac-47c4-9a6e-8dce774c3314', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6816', 'nonce': 1111036754}, {'type': 'v1', 'addr': '172.21.15.83:6817', 'nonce': 1111036754}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6818', 'nonce': 1111036754}, {'type': 'v1', 'addr': '172.21.15.83:6819', 'nonce': 1111036754}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6822', 'nonce': 1111036754}, {'type': 'v1', 'addr': '172.21.15.83:6823', 'nonce': 1111036754}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6820', 'nonce': 1111036754}, {'type': 'v1', 'addr': '172.21.15.83:6821', 'nonce': 1111036754}]}, 'public_addr': '172.21.15.83:6817/1111036754', 'cluster_addr': '172.21.15.83:6819/1111036754', 'heartbeat_back_addr': '172.21.15.83:6823/1111036754', 'heartbeat_front_addr': '172.21.15.83:6821/1111036754', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '78404e0d-d397-4806-8e90-5f32182643bb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6808', 'nonce': 1460363937}, {'type': 'v1', 'addr': '172.21.15.83:6809', 'nonce': 1460363937}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6810', 'nonce': 1460363937}, {'type': 'v1', 'addr': '172.21.15.83:6811', 'nonce': 1460363937}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6814', 'nonce': 1460363937}, {'type': 'v1', 'addr': '172.21.15.83:6815', 'nonce': 1460363937}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6812', 'nonce': 1460363937}, {'type': 'v1', 'addr': '172.21.15.83:6813', 'nonce': 1460363937}]}, 'public_addr': '172.21.15.83:6809/1460363937', 'cluster_addr': '172.21.15.83:6811/1460363937', 'heartbeat_back_addr': '172.21.15.83:6815/1460363937', 'heartbeat_front_addr': '172.21.15.83:6813/1460363937', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '286ed6e0-c121-4ebb-8cf6-37e506a397aa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.2769012451171875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6800', 'nonce': 2837444034}, {'type': 'v1', 'addr': '172.21.15.83:6801', 'nonce': 2837444034}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6802', 'nonce': 2837444034}, {'type': 'v1', 'addr': '172.21.15.83:6803', 'nonce': 2837444034}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6806', 'nonce': 2837444034}, {'type': 'v1', 'addr': '172.21.15.83:6807', 'nonce': 2837444034}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6804', 'nonce': 2837444034}, {'type': 'v1', 'addr': '172.21.15.83:6805', 'nonce': 2837444034}]}, 'public_addr': '172.21.15.83:6801/2837444034', 'cluster_addr': '172.21.15.83:6803/2837444034', 'heartbeat_back_addr': '172.21.15.83:6807/2837444034', 'heartbeat_front_addr': '172.21.15.83:6805/2837444034', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'b0af73a1-4bca-4892-a550-2ced87f4c357', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6808', 'nonce': 3637204865}, {'type': 'v1', 'addr': '172.21.15.97:6809', 'nonce': 3637204865}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6810', 'nonce': 3637204865}, {'type': 'v1', 'addr': '172.21.15.97:6811', 'nonce': 3637204865}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6814', 'nonce': 3637204865}, {'type': 'v1', 'addr': '172.21.15.97:6815', 'nonce': 3637204865}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6812', 'nonce': 3637204865}, {'type': 'v1', 'addr': '172.21.15.97:6813', 'nonce': 3637204865}]}, 'public_addr': '172.21.15.97:6809/3637204865', 'cluster_addr': '172.21.15.97:6811/3637204865', 'heartbeat_back_addr': '172.21.15.97:6815/3637204865', 'heartbeat_front_addr': '172.21.15.97:6813/3637204865', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2188361c-e1a1-45fa-bc01-da5a984ffd9f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6800', 'nonce': 2391356911}, {'type': 'v1', 'addr': '172.21.15.97:6801', 'nonce': 2391356911}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6802', 'nonce': 2391356911}, {'type': 'v1', 'addr': '172.21.15.97:6803', 'nonce': 2391356911}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6806', 'nonce': 2391356911}, {'type': 'v1', 'addr': '172.21.15.97:6807', 'nonce': 2391356911}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6804', 'nonce': 2391356911}, {'type': 'v1', 'addr': '172.21.15.97:6805', 'nonce': 2391356911}]}, 'public_addr': '172.21.15.97:6801/2391356911', 'cluster_addr': '172.21.15.97:6803/2391356911', 'heartbeat_back_addr': '172.21.15.97:6807/2391356911', 'heartbeat_front_addr': '172.21.15.97:6805/2391356911', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '7feccf4d-d8e4-4d06-ac83-bbc1bc623a5c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6816', 'nonce': 2326316381}, {'type': 'v1', 'addr': '172.21.15.97:6817', 'nonce': 2326316381}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6818', 'nonce': 2326316381}, {'type': 'v1', 'addr': '172.21.15.97:6819', 'nonce': 2326316381}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6822', 'nonce': 2326316381}, {'type': 'v1', 'addr': '172.21.15.97:6823', 'nonce': 2326316381}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6820', 'nonce': 2326316381}, {'type': 'v1', 'addr': '172.21.15.97:6821', 'nonce': 2326316381}]}, 'public_addr': '172.21.15.97:6817/2326316381', 'cluster_addr': '172.21.15.97:6819/2326316381', 'heartbeat_back_addr': '172.21.15.97:6823/2326316381', 'heartbeat_front_addr': '172.21.15.97:6821/2326316381', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:51.138101+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:50.581632+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:50.365193+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:50.718974+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:50.501878+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:51.150747+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T21:18:32.359 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:18:32.360 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:32.354+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:32.360 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:18:32.361 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:18:32.379 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:18:32.380 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:18:32.412 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:18:32.413 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:18:32.435 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:18:32.435 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:18:32.458 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:18:32.459 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:18:32.459 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:32.454+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:32.478 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:32.478 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:32.479 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:18:32.480 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:18:32.498 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:18:32.559 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:32.554+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:32.630 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:32.630 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:32.644 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:18:32.676 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:18:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T21:18:32.686 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:32.682+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:32.694 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2024-09-17T21:18:31.724654+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":227034383,"num_objects":149,"num_object_clones":32,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":26,"num_read":14706,"num_read_kb":95484,"num_write":1270,"num_write_kb":52716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1099,"ondisk_log_size":1099,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":282352,"kb_used_data":94620,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":565948688,"statfs":{"total":579820584960,"available":579531456512,"internally_reserved":0,"allocated":96890880,"data_stored":154406965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9574,"internal_metadata":191486618},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29050676,"num_objects":30,"num_object_clones":2,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":6177,"num_read_kb":66808,"num_write":377,"num_write_kb":18433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.183536"},"pg_stats":[{"pgid":"3.1f","version":"29'11","reported_seq":212,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.723333+0000","last_change":"2024-09-17T21:18:13.306161+0000","last_active":"2024-09-17T21:18:28.723333+0000","last_peered":"2024-09-17T21:18:28.723333+0000","last_clean":"2024-09-17T21:18:28.723333+0000","last_became_active":"2024-09-17T21:18:11.313776+0000","last_became_peered":"2024-09-17T21:18:11.313776+0000","last_unstale":"2024-09-17T21:18:28.723333+0000","last_undegraded":"2024-09-17T21:18:28.723333+0000","last_fullsized":"2024-09-17T21:18:28.723333+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:17:59.235244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088525599999999996,"stat_sum":{"num_bytes":2888768,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":182,"num_read_kb":1019,"num_write":10,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"33'131","reported_seq":805,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:29.206936+0000","last_change":"2024-09-17T21:18:17.280405+0000","last_active":"2024-09-17T21:18:29.206936+0000","last_peered":"2024-09-17T21:18:29.206936+0000","last_clean":"2024-09-17T21:18:29.206936+0000","last_became_active":"2024-09-17T21:18:17.279586+0000","last_became_peered":"2024-09-17T21:18:17.279586+0000","last_unstale":"2024-09-17T21:18:29.206936+0000","last_undegraded":"2024-09-17T21:18:29.206936+0000","last_fullsized":"2024-09-17T21:18:29.206936+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:45:03.556356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013256229999999999,"stat_sum":{"num_bytes":4484138,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1793,"num_read_kb":1548,"num_write":139,"num_write_kb":231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"32'23","reported_seq":941,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.913161+0000","last_change":"2024-09-17T21:18:13.324697+0000","last_active":"2024-09-17T21:18:28.913161+0000","last_peered":"2024-09-17T21:18:28.913161+0000","last_clean":"2024-09-17T21:18:28.913161+0000","last_became_active":"2024-09-17T21:18:11.302254+0000","last_became_peered":"2024-09-17T21:18:11.302254+0000","last_unstale":"2024-09-17T21:18:28.913161+0000","last_undegraded":"2024-09-17T21:18:28.913161+0000","last_fullsized":"2024-09-17T21:18:28.913161+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:15:29.027128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018973079999999999,"stat_sum":{"num_bytes":5282304,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":890,"num_read_kb":3378,"num_write":23,"num_write_kb":1262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"33'6","reported_seq":85,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.716399+0000","last_change":"2024-09-17T21:18:13.326517+0000","last_active":"2024-09-17T21:18:28.716399+0000","last_peered":"2024-09-17T21:18:28.716399+0000","last_clean":"2024-09-17T21:18:28.716399+0000","last_became_active":"2024-09-17T21:18:11.302134+0000","last_became_peered":"2024-09-17T21:18:11.302134+0000","last_unstale":"2024-09-17T21:18:28.716399+0000","last_undegraded":"2024-09-17T21:18:28.716399+0000","last_fullsized":"2024-09-17T21:18:28.716399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:21:16.985898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034186630000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"32'20","reported_seq":170,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.723141+0000","last_change":"2024-09-17T21:18:13.305723+0000","last_active":"2024-09-17T21:18:28.723141+0000","last_peered":"2024-09-17T21:18:28.723141+0000","last_clean":"2024-09-17T21:18:28.723141+0000","last_became_active":"2024-09-17T21:18:11.310098+0000","last_became_peered":"2024-09-17T21:18:11.310098+0000","last_unstale":"2024-09-17T21:18:28.723141+0000","last_undegraded":"2024-09-17T21:18:28.723141+0000","last_fullsized":"2024-09-17T21:18:28.723141+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:37:01.934328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089504999999999999,"stat_sum":{"num_bytes":2921472,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":123,"num_read_kb":465,"num_write":19,"num_write_kb":227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"30'18","reported_seq":514,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:29.447213+0000","last_change":"2024-09-17T21:18:13.314063+0000","last_active":"2024-09-17T21:18:29.447213+0000","last_peered":"2024-09-17T21:18:29.447213+0000","last_clean":"2024-09-17T21:18:29.447213+0000","last_became_active":"2024-09-17T21:18:11.301426+0000","last_became_peered":"2024-09-17T21:18:11.301426+0000","last_unstale":"2024-09-17T21:18:29.447213+0000","last_undegraded":"2024-09-17T21:18:29.447213+0000","last_fullsized":"2024-09-17T21:18:29.447213+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:05:39.686762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027366090000000001,"stat_sum":{"num_bytes":5898771,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":459,"num_read_kb":473,"num_write":17,"num_write_kb":165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"29'6","reported_seq":68,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.728765+0000","last_change":"2024-09-17T21:18:13.314862+0000","last_active":"2024-09-17T21:18:28.728765+0000","last_peered":"2024-09-17T21:18:28.728765+0000","last_clean":"2024-09-17T21:18:28.728765+0000","last_became_active":"2024-09-17T21:18:11.300793+0000","last_became_peered":"2024-09-17T21:18:11.300793+0000","last_unstale":"2024-09-17T21:18:28.728765+0000","last_undegraded":"2024-09-17T21:18:28.728765+0000","last_fullsized":"2024-09-17T21:18:28.728765+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:50:14.556278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047673000000000002,"stat_sum":{"num_bytes":800256,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":25,"num_read_kb":68,"num_write":5,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"31'48","reported_seq":87,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.722859+0000","last_change":"2024-09-17T21:18:17.280452+0000","last_active":"2024-09-17T21:18:28.722859+0000","last_peered":"2024-09-17T21:18:28.722859+0000","last_clean":"2024-09-17T21:18:28.722859+0000","last_became_active":"2024-09-17T21:18:17.279644+0000","last_became_peered":"2024-09-17T21:18:17.279644+0000","last_unstale":"2024-09-17T21:18:28.722859+0000","last_undegraded":"2024-09-17T21:18:28.722859+0000","last_fullsized":"2024-09-17T21:18:28.722859+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:32:23.867966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":337968,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":229,"num_read_kb":230,"num_write":78,"num_write_kb":97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"32'20","reported_seq":79,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.723425+0000","last_change":"2024-09-17T21:18:13.306286+0000","last_active":"2024-09-17T21:18:28.723425+0000","last_peered":"2024-09-17T21:18:28.723425+0000","last_clean":"2024-09-17T21:18:28.723425+0000","last_became_active":"2024-09-17T21:18:11.312014+0000","last_became_peered":"2024-09-17T21:18:11.312014+0000","last_unstale":"2024-09-17T21:18:28.723425+0000","last_undegraded":"2024-09-17T21:18:28.723425+0000","last_fullsized":"2024-09-17T21:18:28.723425+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:51:02.527779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088962999999999998,"stat_sum":{"num_bytes":6293504,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":26,"num_read_kb":159,"num_write":26,"num_write_kb":1070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"33'8","reported_seq":91,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:29.151099+0000","last_change":"2024-09-17T21:18:13.324809+0000","last_active":"2024-09-17T21:18:29.151099+0000","last_peered":"2024-09-17T21:18:29.151099+0000","last_clean":"2024-09-17T21:18:29.151099+0000","last_became_active":"2024-09-17T21:18:11.303835+0000","last_became_peered":"2024-09-17T21:18:11.303835+0000","last_unstale":"2024-09-17T21:18:29.151099+0000","last_undegraded":"2024-09-17T21:18:29.151099+0000","last_fullsized":"2024-09-17T21:18:29.151099+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:42:03.038636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001825658,"stat_sum":{"num_bytes":2584720,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":52,"num_write":8,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"32'16","reported_seq":112,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.977940+0000","last_change":"2024-09-17T21:18:13.307979+0000","last_active":"2024-09-17T21:18:28.977940+0000","last_peered":"2024-09-17T21:18:28.977940+0000","last_clean":"2024-09-17T21:18:28.977940+0000","last_became_active":"2024-09-17T21:18:11.312815+0000","last_became_peered":"2024-09-17T21:18:11.312815+0000","last_unstale":"2024-09-17T21:18:28.977940+0000","last_undegraded":"2024-09-17T21:18:28.977940+0000","last_fullsized":"2024-09-17T21:18:28.977940+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:03.976512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027112149999999999,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":66,"num_read_kb":249,"num_write":19,"num_write_kb":1155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"33'27","reported_seq":105,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.977606+0000","last_change":"2024-09-17T21:18:13.307234+0000","last_active":"2024-09-17T21:18:28.977606+0000","last_peered":"2024-09-17T21:18:28.977606+0000","last_clean":"2024-09-17T21:18:28.977606+0000","last_became_active":"2024-09-17T21:18:11.299817+0000","last_became_peered":"2024-09-17T21:18:11.299817+0000","last_unstale":"2024-09-17T21:18:28.977606+0000","last_undegraded":"2024-09-17T21:18:28.977606+0000","last_fullsized":"2024-09-17T21:18:28.977606+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:03:50.587253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002281113,"stat_sum":{"num_bytes":6291472,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":52,"num_read_kb":96,"num_write":31,"num_write_kb":1135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":46,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.977964+0000","last_change":"2024-09-17T21:18:03.059715+0000","last_active":"2024-09-17T21:18:28.977964+0000","last_peered":"2024-09-17T21:18:28.977964+0000","last_clean":"2024-09-17T21:18:28.977964+0000","last_became_active":"2024-09-17T21:18:01.213047+0000","last_became_peered":"2024-09-17T21:18:01.213047+0000","last_unstale":"2024-09-17T21:18:28.977964+0000","last_undegraded":"2024-09-17T21:18:28.977964+0000","last_fullsized":"2024-09-17T21:18:28.977964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:46:35.222373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"30'13","reported_seq":355,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.723456+0000","last_change":"2024-09-17T21:18:13.305969+0000","last_active":"2024-09-17T21:18:28.723456+0000","last_peered":"2024-09-17T21:18:28.723456+0000","last_clean":"2024-09-17T21:18:28.723456+0000","last_became_active":"2024-09-17T21:18:11.316919+0000","last_became_peered":"2024-09-17T21:18:11.316919+0000","last_unstale":"2024-09-17T21:18:28.723456+0000","last_undegraded":"2024-09-17T21:18:28.723456+0000","last_fullsized":"2024-09-17T21:18:28.723456+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:41:31.214976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084327200000000005,"stat_sum":{"num_bytes":19694118,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":328,"num_read_kb":14625,"num_write":19,"num_write_kb":7346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":46,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.729063+0000","last_change":"2024-09-17T21:18:03.073440+0000","last_active":"2024-09-17T21:18:28.729063+0000","last_peered":"2024-09-17T21:18:28.729063+0000","last_clean":"2024-09-17T21:18:28.729063+0000","last_became_active":"2024-09-17T21:18:01.048806+0000","last_became_peered":"2024-09-17T21:18:01.048806+0000","last_unstale":"2024-09-17T21:18:28.729063+0000","last_undegraded":"2024-09-17T21:18:28.729063+0000","last_fullsized":"2024-09-17T21:18:28.729063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:26.487163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094161400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"30'37","reported_seq":72,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.728983+0000","last_change":"2024-09-17T21:18:17.280432+0000","last_active":"2024-09-17T21:18:28.728983+0000","last_peered":"2024-09-17T21:18:28.728983+0000","last_clean":"2024-09-17T21:18:28.728983+0000","last_became_active":"2024-09-17T21:18:17.279625+0000","last_became_peered":"2024-09-17T21:18:17.279625+0000","last_unstale":"2024-09-17T21:18:28.728983+0000","last_undegraded":"2024-09-17T21:18:28.728983+0000","last_fullsized":"2024-09-17T21:18:28.728983+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:18:01.251948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383012,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":154,"num_read_kb":141,"num_write":56,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":46,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.978086+0000","last_change":"2024-09-17T21:18:03.058986+0000","last_active":"2024-09-17T21:18:28.978086+0000","last_peered":"2024-09-17T21:18:28.978086+0000","last_clean":"2024-09-17T21:18:28.978086+0000","last_became_active":"2024-09-17T21:18:01.214272+0000","last_became_peered":"2024-09-17T21:18:01.214272+0000","last_unstale":"2024-09-17T21:18:28.978086+0000","last_undegraded":"2024-09-17T21:18:28.978086+0000","last_fullsized":"2024-09-17T21:18:28.978086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:04:38.082107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099219900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"32'30","reported_seq":488,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.865769+0000","last_change":"2024-09-17T21:18:13.315662+0000","last_active":"2024-09-17T21:18:28.865769+0000","last_peered":"2024-09-17T21:18:28.865769+0000","last_clean":"2024-09-17T21:18:28.865769+0000","last_became_active":"2024-09-17T21:18:11.309133+0000","last_became_peered":"2024-09-17T21:18:11.309133+0000","last_unstale":"2024-09-17T21:18:28.865769+0000","last_undegraded":"2024-09-17T21:18:28.865769+0000","last_fullsized":"2024-09-17T21:18:28.865769+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:09:42.837944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078465700000000002,"stat_sum":{"num_bytes":20897792,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":440,"num_read_kb":9454,"num_write":38,"num_write_kb":5895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.729179+0000","last_change":"2024-09-17T21:18:03.073508+0000","last_active":"2024-09-17T21:18:28.729179+0000","last_peered":"2024-09-17T21:18:28.729179+0000","last_clean":"2024-09-17T21:18:28.729179+0000","last_became_active":"2024-09-17T21:18:01.213705+0000","last_became_peered":"2024-09-17T21:18:01.213705+0000","last_unstale":"2024-09-17T21:18:28.729179+0000","last_undegraded":"2024-09-17T21:18:28.729179+0000","last_fullsized":"2024-09-17T21:18:28.729179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:46:08.811379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006222,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"30'8","reported_seq":502,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.716790+0000","last_change":"2024-09-17T21:18:13.327202+0000","last_active":"2024-09-17T21:18:28.716790+0000","last_peered":"2024-09-17T21:18:28.716790+0000","last_clean":"2024-09-17T21:18:28.716790+0000","last_became_active":"2024-09-17T21:18:11.317127+0000","last_became_peered":"2024-09-17T21:18:11.317127+0000","last_unstale":"2024-09-17T21:18:28.716790+0000","last_undegraded":"2024-09-17T21:18:28.716790+0000","last_fullsized":"2024-09-17T21:18:28.716790+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:50:29.697720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036493720000000001,"stat_sum":{"num_bytes":15728656,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":464,"num_read_kb":9481,"num_write":11,"num_write_kb":4192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"32'22","reported_seq":135,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.967441+0000","last_change":"2024-09-17T21:18:13.312043+0000","last_active":"2024-09-17T21:18:28.967441+0000","last_peered":"2024-09-17T21:18:28.967441+0000","last_clean":"2024-09-17T21:18:28.967441+0000","last_became_active":"2024-09-17T21:18:11.321078+0000","last_became_peered":"2024-09-17T21:18:11.321078+0000","last_unstale":"2024-09-17T21:18:28.967441+0000","last_undegraded":"2024-09-17T21:18:28.967441+0000","last_fullsized":"2024-09-17T21:18:28.967441+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:00:12.336814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084988700000000002,"stat_sum":{"num_bytes":3272704,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":202,"num_write":24,"num_write_kb":1510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":48,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.716320+0000","last_change":"2024-09-17T21:18:03.070336+0000","last_active":"2024-09-17T21:18:28.716320+0000","last_peered":"2024-09-17T21:18:28.716320+0000","last_clean":"2024-09-17T21:18:28.716320+0000","last_became_active":"2024-09-17T21:18:01.049336+0000","last_became_peered":"2024-09-17T21:18:01.049336+0000","last_unstale":"2024-09-17T21:18:28.716320+0000","last_undegraded":"2024-09-17T21:18:28.716320+0000","last_fullsized":"2024-09-17T21:18:28.716320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:56:24.329470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023986049999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"30'13","reported_seq":286,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.728728+0000","last_change":"2024-09-17T21:18:13.314742+0000","last_active":"2024-09-17T21:18:28.728728+0000","last_peered":"2024-09-17T21:18:28.728728+0000","last_clean":"2024-09-17T21:18:28.728728+0000","last_became_active":"2024-09-17T21:18:11.308568+0000","last_became_peered":"2024-09-17T21:18:11.308568+0000","last_unstale":"2024-09-17T21:18:28.728728+0000","last_undegraded":"2024-09-17T21:18:28.728728+0000","last_fullsized":"2024-09-17T21:18:28.728728+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:02:57.477967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049125000000000004,"stat_sum":{"num_bytes":8762384,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":245,"num_read_kb":4993,"num_write":17,"num_write_kb":2244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.728969+0000","last_change":"2024-09-17T21:18:17.280358+0000","last_active":"2024-09-17T21:18:28.728969+0000","last_peered":"2024-09-17T21:18:28.728969+0000","last_clean":"2024-09-17T21:18:28.728969+0000","last_became_active":"2024-09-17T21:18:17.279474+0000","last_became_peered":"2024-09-17T21:18:17.279474+0000","last_unstale":"2024-09-17T21:18:28.728969+0000","last_undegraded":"2024-09-17T21:18:28.728969+0000","last_fullsized":"2024-09-17T21:18:28.728969+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:55:54.183527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"32'59","reported_seq":375,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:29.277467+0000","last_change":"2024-09-17T21:18:13.304977+0000","last_active":"2024-09-17T21:18:29.277467+0000","last_peered":"2024-09-17T21:18:29.277467+0000","last_clean":"2024-09-17T21:18:29.277467+0000","last_became_active":"2024-09-17T21:18:11.308973+0000","last_became_peered":"2024-09-17T21:18:11.308973+0000","last_unstale":"2024-09-17T21:18:29.277467+0000","last_undegraded":"2024-09-17T21:18:29.277467+0000","last_fullsized":"2024-09-17T21:18:29.277467+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:01:18.577013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066398399999999995,"stat_sum":{"num_bytes":7517232,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":490,"num_read_kb":6613,"num_write":91,"num_write_kb":4251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":46,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.977255+0000","last_change":"2024-09-17T21:18:03.059156+0000","last_active":"2024-09-17T21:18:28.977255+0000","last_peered":"2024-09-17T21:18:28.977255+0000","last_clean":"2024-09-17T21:18:28.977255+0000","last_became_active":"2024-09-17T21:18:01.049180+0000","last_became_peered":"2024-09-17T21:18:01.049180+0000","last_unstale":"2024-09-17T21:18:28.977255+0000","last_undegraded":"2024-09-17T21:18:28.977255+0000","last_fullsized":"2024-09-17T21:18:28.977255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:27.069794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077408,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"30'10","reported_seq":470,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.751443+0000","last_change":"2024-09-17T21:18:13.315294+0000","last_active":"2024-09-17T21:18:28.751443+0000","last_peered":"2024-09-17T21:18:28.751443+0000","last_clean":"2024-09-17T21:18:28.751443+0000","last_became_active":"2024-09-17T21:18:11.294154+0000","last_became_peered":"2024-09-17T21:18:11.294154+0000","last_unstale":"2024-09-17T21:18:28.751443+0000","last_undegraded":"2024-09-17T21:18:28.751443+0000","last_fullsized":"2024-09-17T21:18:28.751443+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:14:37.207468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049408099999999995,"stat_sum":{"num_bytes":32962576,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":433,"num_read_kb":15084,"num_write":24,"num_write_kb":7918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":47,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.716503+0000","last_change":"2024-09-17T21:18:03.069091+0000","last_active":"2024-09-17T21:18:28.716503+0000","last_peered":"2024-09-17T21:18:28.716503+0000","last_clean":"2024-09-17T21:18:28.716503+0000","last_became_active":"2024-09-17T21:18:01.044275+0000","last_became_peered":"2024-09-17T21:18:01.044275+0000","last_unstale":"2024-09-17T21:18:28.716503+0000","last_undegraded":"2024-09-17T21:18:28.716503+0000","last_fullsized":"2024-09-17T21:18:28.716503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:28:27.066081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001145884,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":137,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.977355+0000","last_change":"2024-09-17T21:17:57.194934+0000","last_active":"2024-09-17T21:18:28.977355+0000","last_peered":"2024-09-17T21:18:28.977355+0000","last_clean":"2024-09-17T21:18:28.977355+0000","last_became_active":"2024-09-17T21:17:57.192405+0000","last_became_peered":"2024-09-17T21:17:57.192405+0000","last_unstale":"2024-09-17T21:18:28.977355+0000","last_undegraded":"2024-09-17T21:18:28.977355+0000","last_fullsized":"2024-09-17T21:18:28.977355+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_clean_scrub_stamp":"2024-09-17T21:17:55.982339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:30.628595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"33'39","reported_seq":491,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.728760+0000","last_change":"2024-09-17T21:18:17.280477+0000","last_active":"2024-09-17T21:18:28.728760+0000","last_peered":"2024-09-17T21:18:28.728760+0000","last_clean":"2024-09-17T21:18:28.728760+0000","last_became_active":"2024-09-17T21:18:17.279644+0000","last_became_peered":"2024-09-17T21:18:17.279644+0000","last_unstale":"2024-09-17T21:18:28.728760+0000","last_undegraded":"2024-09-17T21:18:28.728760+0000","last_fullsized":"2024-09-17T21:18:28.728760+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:24:19.390633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4549648,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":874,"num_read_kb":5934,"num_write":26,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"30'141","reported_seq":425,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.967256+0000","last_change":"2024-09-17T21:18:13.313980+0000","last_active":"2024-09-17T21:18:28.967256+0000","last_peered":"2024-09-17T21:18:28.967256+0000","last_clean":"2024-09-17T21:18:28.967256+0000","last_became_active":"2024-09-17T21:18:11.304021+0000","last_became_peered":"2024-09-17T21:18:11.304021+0000","last_unstale":"2024-09-17T21:18:28.967256+0000","last_undegraded":"2024-09-17T21:18:28.967256+0000","last_fullsized":"2024-09-17T21:18:28.967256+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:57:10.415064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023749890000000001,"stat_sum":{"num_bytes":6532649,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":919,"num_read_kb":1626,"num_write":240,"num_write_kb":399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"33'19","reported_seq":217,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.977444+0000","last_change":"2024-09-17T21:18:13.307298+0000","last_active":"2024-09-17T21:18:28.977444+0000","last_peered":"2024-09-17T21:18:28.977444+0000","last_clean":"2024-09-17T21:18:28.977444+0000","last_became_active":"2024-09-17T21:18:11.294907+0000","last_became_peered":"2024-09-17T21:18:11.294907+0000","last_unstale":"2024-09-17T21:18:28.977444+0000","last_undegraded":"2024-09-17T21:18:28.977444+0000","last_fullsized":"2024-09-17T21:18:28.977444+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:03:52.610782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021951829999999999,"stat_sum":{"num_bytes":5216850,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":176,"num_read_kb":483,"num_write":18,"num_write_kb":222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"32'19","reported_seq":141,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.728804+0000","last_change":"2024-09-17T21:18:13.315320+0000","last_active":"2024-09-17T21:18:28.728804+0000","last_peered":"2024-09-17T21:18:28.728804+0000","last_clean":"2024-09-17T21:18:28.728804+0000","last_became_active":"2024-09-17T21:18:11.312997+0000","last_became_peered":"2024-09-17T21:18:11.312997+0000","last_unstale":"2024-09-17T21:18:28.728804+0000","last_undegraded":"2024-09-17T21:18:28.728804+0000","last_fullsized":"2024-09-17T21:18:28.728804+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:43:00.122842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048704800000000002,"stat_sum":{"num_bytes":6107648,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":87,"num_read_kb":873,"num_write":18,"num_write_kb":295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"32'47","reported_seq":1131,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.716649+0000","last_change":"2024-09-17T21:18:13.324310+0000","last_active":"2024-09-17T21:18:28.716649+0000","last_peered":"2024-09-17T21:18:28.716649+0000","last_clean":"2024-09-17T21:18:28.716649+0000","last_became_active":"2024-09-17T21:18:11.317909+0000","last_became_peered":"2024-09-17T21:18:11.317909+0000","last_unstale":"2024-09-17T21:18:28.716649+0000","last_undegraded":"2024-09-17T21:18:28.716649+0000","last_fullsized":"2024-09-17T21:18:28.716649+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:17:22.796546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015319789999999999,"stat_sum":{"num_bytes":18737152,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":1447,"num_read_kb":3194,"num_write":30,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"28'7","reported_seq":48,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.977969+0000","last_change":"2024-09-17T21:18:13.306351+0000","last_active":"2024-09-17T21:18:28.977969+0000","last_peered":"2024-09-17T21:18:28.977969+0000","last_clean":"2024-09-17T21:18:28.977969+0000","last_became_active":"2024-09-17T21:18:11.316999+0000","last_became_peered":"2024-09-17T21:18:11.316999+0000","last_unstale":"2024-09-17T21:18:28.977969+0000","last_undegraded":"2024-09-17T21:18:28.977969+0000","last_fullsized":"2024-09-17T21:18:28.977969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:16:52.334321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015238949999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"33'99","reported_seq":1063,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.716949+0000","last_change":"2024-09-17T21:18:13.327081+0000","last_active":"2024-09-17T21:18:28.716949+0000","last_peered":"2024-09-17T21:18:28.716949+0000","last_clean":"2024-09-17T21:18:28.716949+0000","last_became_active":"2024-09-17T21:18:11.301652+0000","last_became_peered":"2024-09-17T21:18:11.301652+0000","last_unstale":"2024-09-17T21:18:28.716949+0000","last_undegraded":"2024-09-17T21:18:28.716949+0000","last_fullsized":"2024-09-17T21:18:28.716949+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:36:17.154068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003632209,"stat_sum":{"num_bytes":16759808,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":1808,"num_read_kb":9991,"num_write":75,"num_write_kb":6220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"29'6","reported_seq":49,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.717014+0000","last_change":"2024-09-17T21:18:13.325500+0000","last_active":"2024-09-17T21:18:28.717014+0000","last_peered":"2024-09-17T21:18:28.717014+0000","last_clean":"2024-09-17T21:18:28.717014+0000","last_became_active":"2024-09-17T21:18:11.307060+0000","last_became_peered":"2024-09-17T21:18:11.307060+0000","last_unstale":"2024-09-17T21:18:28.717014+0000","last_undegraded":"2024-09-17T21:18:28.717014+0000","last_fullsized":"2024-09-17T21:18:28.717014+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:46:16.641376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002454758,"stat_sum":{"num_bytes":1242112,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":6,"num_read_kb":20,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"33'9","reported_seq":79,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.716282+0000","last_change":"2024-09-17T21:18:13.324120+0000","last_active":"2024-09-17T21:18:28.716282+0000","last_peered":"2024-09-17T21:18:28.716282+0000","last_clean":"2024-09-17T21:18:28.716282+0000","last_became_active":"2024-09-17T21:18:11.305272+0000","last_became_peered":"2024-09-17T21:18:11.305272+0000","last_unstale":"2024-09-17T21:18:28.716282+0000","last_undegraded":"2024-09-17T21:18:28.716282+0000","last_fullsized":"2024-09-17T21:18:28.716282+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:04:44.375265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001519261,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":91,"num_read_kb":77,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"32'11","reported_seq":117,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:29.094857+0000","last_change":"2024-09-17T21:18:13.305680+0000","last_active":"2024-09-17T21:18:29.094857+0000","last_peered":"2024-09-17T21:18:29.094857+0000","last_clean":"2024-09-17T21:18:29.094857+0000","last_became_active":"2024-09-17T21:18:11.299946+0000","last_became_peered":"2024-09-17T21:18:11.299946+0000","last_unstale":"2024-09-17T21:18:29.094857+0000","last_undegraded":"2024-09-17T21:18:29.094857+0000","last_fullsized":"2024-09-17T21:18:29.094857+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:57:24.362034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225884,"stat_sum":{"num_bytes":5532720,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":82,"num_read_kb":122,"num_write":12,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"33'23","reported_seq":280,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:29.394420+0000","last_change":"2024-09-17T21:18:13.305783+0000","last_active":"2024-09-17T21:18:29.394420+0000","last_peered":"2024-09-17T21:18:29.394420+0000","last_clean":"2024-09-17T21:18:29.394420+0000","last_became_active":"2024-09-17T21:18:11.319369+0000","last_became_peered":"2024-09-17T21:18:11.319369+0000","last_unstale":"2024-09-17T21:18:29.394420+0000","last_undegraded":"2024-09-17T21:18:29.394420+0000","last_fullsized":"2024-09-17T21:18:29.394420+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:23:37.717403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013045489999999999,"stat_sum":{"num_bytes":12535338,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":240,"num_read_kb":584,"num_write":26,"num_write_kb":1210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"32'78","reported_seq":1538,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T21:18:28.977415+0000","last_change":"2024-09-17T21:18:13.306520+0000","last_active":"2024-09-17T21:18:28.977415+0000","last_peered":"2024-09-17T21:18:28.977415+0000","last_clean":"2024-09-17T21:18:28.977415+0000","last_became_active":"2024-09-17T21:18:11.294953+0000","last_became_peered":"2024-09-17T21:18:11.294953+0000","last_unstale":"2024-09-17T21:18:28.977415+0000","last_undegraded":"2024-09-17T21:18:28.977415+0000","last_fullsized":"2024-09-17T21:18:28.977415+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:02:13.722119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019790290000000002,"stat_sum":{"num_bytes":1179648,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2334,"num_read_kb":4145,"num_write":46,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":226443996,"num_objects":145,"num_object_clones":32,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":26,"num_read":14640,"num_read_kb":95429,"num_write":1167,"num_write_kb":51086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":93618176,"data_stored":152456632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1024,"ondisk_log_size":1024,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51756,"kb_used_data":20264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94320084,"statfs":{"total":96636764160,"available":96583766016,"internally_reserved":0,"allocated":20750336,"data_stored":36958222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44944,"kb_used_data":13452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94326896,"statfs":{"total":96636764160,"available":96590741504,"internally_reserved":0,"allocated":13774848,"data_stored":20814409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34948,"kb_used_data":3760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336892,"statfs":{"total":96636764160,"available":96600977408,"internally_reserved":0,"allocated":3850240,"data_stored":3553244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57324,"kb_used_data":26136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94314516,"statfs":{"total":96636764160,"available":96578064384,"internally_reserved":0,"allocated":26763264,"data_stored":44476070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47044,"kb_used_data":15856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324796,"statfs":{"total":96636764160,"available":96588591104,"internally_reserved":0,"allocated":16236544,"data_stored":24342294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607563,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46336,"kb_used_data":15152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325504,"statfs":{"total":96636764160,"available":96589316096,"internally_reserved":0,"allocated":15515648,"data_stored":24262726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15298560,"data_stored":23624432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26415104,"data_stored":44348557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":15183872,"data_stored":24140503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2912256,"data_stored":2835382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":13418496,"data_stored":20681982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20389888,"data_stored":36825776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:18:32.697 INFO:tasks.thrashosds.thrasher:Setting 3.1e to [0, 4] 2024-09-17T21:18:32.697 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.1e', '0', '4'] 2024-09-17T21:18:32.697 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.1e 0 4 2024-09-17T21:18:32.761 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:32.756+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:32.788 INFO:teuthology.orchestra.run.smithi184.stdout:517 write 0x4039400 thru 0x4041893 (0x8494 bytes) 2024-09-17T21:18:32.790 INFO:teuthology.orchestra.run.smithi184.stdout:519 punch from 0x66fe00 to 0x679e00, (0xa000 bytes) 2024-09-17T21:18:32.814 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:32.814 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:32.829 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:18:32.843 INFO:teuthology.orchestra.run.smithi184.stdout:520 write 0x6313800 thru 0x6316b54 (0x3355 bytes) 2024-09-17T21:18:32.844 INFO:teuthology.orchestra.run.smithi184.stdout:521 punch from 0x1afba00 to 0x1b08a00, (0xd000 bytes) 2024-09-17T21:18:32.848 INFO:teuthology.orchestra.run.smithi184.stdout:522 read 0x413bc00 thru 0x4141d67 (0x6168 bytes) 2024-09-17T21:18:32.849 INFO:teuthology.orchestra.run.smithi184.stdout:523 read 0x4c33000 thru 0x4c3e9bc (0xb9bd bytes) 2024-09-17T21:18:32.873 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:32.870+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:32.874 INFO:teuthology.orchestra.run.smithi184.stdout:525 write 0x7790200 thru 0x77958da (0x56db bytes) 2024-09-17T21:18:32.933 INFO:teuthology.orchestra.run.smithi184.stdout:526 trunc from 0x7795a00 to 0xe201e00 2024-09-17T21:18:32.961 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:32.961 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:32.962 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:32.958+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:32.970 INFO:teuthology.orchestra.run.smithi184.stdout:528 trunc from 0xe201e00 to 0xdd3dc00 2024-09-17T21:18:32.976 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:18:32.992 INFO:teuthology.orchestra.run.smithi184.stdout:529 punch from 0xf08600 to 0xf17a00, (0xf400 bytes) 2024-09-17T21:18:32.994 INFO:teuthology.orchestra.run.smithi184.stdout:531 read 0xab14400 thru 0xab1bc94 (0x7895 bytes) 2024-09-17T21:18:32.995 INFO:teuthology.orchestra.run.smithi184.stdout:532 read 0x64f3200 thru 0x64f46d1 (0x14d2 bytes) 2024-09-17T21:18:32.995 INFO:teuthology.orchestra.run.smithi184.stdout:533 read 0x9dc8000 thru 0x9dd3d43 (0xbd44 bytes) 2024-09-17T21:18:32.996 INFO:teuthology.orchestra.run.smithi184.stdout:538 write 0xb0d4c00 thru 0xb0e080b (0xbc0c bytes) 2024-09-17T21:18:32.997 INFO:teuthology.orchestra.run.smithi184.stdout:543 punch from 0x6f00800 to 0x6f0c400, (0xbc00 bytes) 2024-09-17T21:18:32.997 INFO:teuthology.orchestra.run.smithi184.stdout:545 punch from 0xb6ab800 to 0xb6ada00, (0x2200 bytes) 2024-09-17T21:18:32.997 INFO:teuthology.orchestra.run.smithi184.stdout:547 read 0x2082e00 thru 0x2083ca0 (0xea1 bytes) 2024-09-17T21:18:32.999 INFO:teuthology.orchestra.run.smithi184.stdout:550 punch from 0xb8d3400 to 0xb8d5e00, (0x2a00 bytes) 2024-09-17T21:18:32.999 INFO:teuthology.orchestra.run.smithi184.stdout:551 read 0xbac6e00 thru 0xbac83b5 (0x15b6 bytes) 2024-09-17T21:18:33.032 INFO:teuthology.orchestra.run.smithi184.stdout:552 trunc from 0xdd3dc00 to 0x22ee000 2024-09-17T21:18:33.063 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:33.056+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:33.129 INFO:teuthology.orchestra.run.smithi184.stdout:554 read 0xb2d200 thru 0xb34d69 (0x7b6a bytes) 2024-09-17T21:18:33.132 INFO:teuthology.orchestra.run.smithi184.stdout:557 read 0x1c0d600 thru 0x1c1393c (0x633d bytes) 2024-09-17T21:18:33.133 INFO:teuthology.orchestra.run.smithi184.stdout:558 read 0x10e5800 thru 0x10ebff8 (0x67f9 bytes) 2024-09-17T21:18:33.135 INFO:teuthology.orchestra.run.smithi184.stdout:559 read 0x1c23a00 thru 0x1c32662 (0xec63 bytes) 2024-09-17T21:18:33.157 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:33.158 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:33.164 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:33.158+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:33.172 INFO:teuthology.orchestra.run.smithi184.stdout:560 write 0xa840a00 thru 0xa845307 (0x4908 bytes) 2024-09-17T21:18:33.174 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:18:33.193 INFO:teuthology.orchestra.run.smithi184.stdout:563 trunc from 0xa845400 to 0x3abde00 2024-09-17T21:18:33.225 INFO:teuthology.orchestra.run.smithi184.stdout:565 write 0xd3fa00 thru 0xd401e9 (0x7ea bytes) 2024-09-17T21:18:33.228 INFO:teuthology.orchestra.run.smithi184.stdout:568 read 0x2e87000 thru 0x2e89029 (0x202a bytes) 2024-09-17T21:18:33.251 INFO:teuthology.orchestra.run.smithi184.stdout:572 write 0x65e7c00 thru 0x65eeb9c (0x6f9d bytes) 2024-09-17T21:18:33.265 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:33.260+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:33.278 INFO:teuthology.orchestra.run.smithi184.stdout:574 write 0x9da8000 thru 0x9dacc06 (0x4c07 bytes) 2024-09-17T21:18:33.280 INFO:teuthology.orchestra.run.smithi184.stdout:576 punch from 0x26d1c00 to 0x26e0000, (0xe400 bytes) 2024-09-17T21:18:33.281 INFO:teuthology.orchestra.run.smithi184.stdout:577 write 0x8b6800 thru 0x8c0efb (0xa6fc bytes) 2024-09-17T21:18:33.281 INFO:teuthology.orchestra.run.smithi184.stdout:579 read 0x8003400 thru 0x8011b88 (0xe789 bytes) 2024-09-17T21:18:33.282 INFO:teuthology.orchestra.run.smithi184.stdout:581 punch from 0x5cffc00 to 0x5d09400, (0x9800 bytes) 2024-09-17T21:18:33.283 INFO:teuthology.orchestra.run.smithi184.stdout:583 write 0x16c1e00 thru 0x16cdcbd (0xbebe bytes) 2024-09-17T21:18:33.288 INFO:teuthology.orchestra.run.smithi184.stdout:584 write 0x14f7c00 thru 0x1501c59 (0xa05a bytes) 2024-09-17T21:18:33.289 INFO:teuthology.orchestra.run.smithi184.stdout:587 read 0x1b92200 thru 0x1b976ef (0x54f0 bytes) 2024-09-17T21:18:33.318 INFO:teuthology.orchestra.run.smithi184.stdout:589 write 0xd066000 thru 0xd069d58 (0x3d59 bytes) 2024-09-17T21:18:33.319 INFO:teuthology.orchestra.run.smithi184.stdout:590 read 0xac95600 thru 0xaca2198 (0xcb99 bytes) 2024-09-17T21:18:33.320 INFO:teuthology.orchestra.run.smithi184.stdout:591 read 0x89ad400 thru 0x89b5f64 (0x8b65 bytes) 2024-09-17T21:18:33.331 INFO:teuthology.orchestra.run.smithi184.stdout:592 trunc from 0xd069e00 to 0x77f9c00 2024-09-17T21:18:33.356 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:33.356 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:33.366 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:33.360+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:33.371 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:18:33.374 INFO:teuthology.orchestra.run.smithi184.stdout:593 read 0x5db7000 thru 0x5dbc46a (0x546b bytes) 2024-09-17T21:18:33.380 INFO:teuthology.orchestra.run.smithi184.stdout:598 write 0x6cbc400 thru 0x6cc2313 (0x5f14 bytes) 2024-09-17T21:18:33.380 INFO:teuthology.orchestra.run.smithi184.stdout:600 read 0x44fcc00 thru 0x45057b5 (0x8bb6 bytes) 2024-09-17T21:18:33.381 INFO:teuthology.orchestra.run.smithi184.stdout:601 write 0x52e9800 thru 0x52eb226 (0x1a27 bytes) 2024-09-17T21:18:33.383 INFO:teuthology.orchestra.run.smithi184.stdout:602 read 0x3e9e00 thru 0x3ec297 (0x2498 bytes) 2024-09-17T21:18:33.411 INFO:teuthology.orchestra.run.smithi184.stdout:603 write 0xd8af800 thru 0xd8b06e8 (0xee9 bytes) 2024-09-17T21:18:33.413 INFO:teuthology.orchestra.run.smithi184.stdout:604 read 0xbd19600 thru 0xbd1bb8a (0x258b bytes) 2024-09-17T21:18:33.413 INFO:teuthology.orchestra.run.smithi184.stdout:605 read 0xa501800 thru 0xa506344 (0x4b45 bytes) 2024-09-17T21:18:33.425 INFO:teuthology.orchestra.run.smithi184.stdout:606 trunc from 0xd8b0800 to 0x9179800 2024-09-17T21:18:33.459 INFO:teuthology.orchestra.run.smithi184.stdout:608 read 0x7f1a00 thru 0x7f317c (0x177d bytes) 2024-09-17T21:18:33.462 INFO:teuthology.orchestra.run.smithi184.stdout:609 write 0x751dc00 thru 0x7522684 (0x4a85 bytes) 2024-09-17T21:18:33.463 INFO:teuthology.orchestra.run.smithi184.stdout:610 punch from 0x244b000 to 0x244c800, (0x1800 bytes) 2024-09-17T21:18:33.468 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:33.460+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:33.474 INFO:teuthology.orchestra.run.smithi184.stdout:612 trunc from 0x9179800 to 0x450a800 2024-09-17T21:18:33.514 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:33.514 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:33.528 INFO:teuthology.orchestra.run.smithi184.stdout:613 write 0x8154c00 thru 0x815af1f (0x6320 bytes) 2024-09-17T21:18:33.531 INFO:teuthology.orchestra.run.smithi184.stdout:618 read 0x767e200 thru 0x7687144 (0x8f45 bytes) 2024-09-17T21:18:33.531 INFO:teuthology.orchestra.run.smithi184.stdout:619 read 0x1ce9600 thru 0x1cf614b (0xcb4c bytes) 2024-09-17T21:18:33.532 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:18:33.533 INFO:teuthology.orchestra.run.smithi184.stdout:621 write 0x1e3b000 thru 0x1e440f8 (0x90f9 bytes) 2024-09-17T21:18:33.533 INFO:teuthology.orchestra.run.smithi184.stdout:623 read 0x3d2a800 thru 0x3d2c968 (0x2169 bytes) 2024-09-17T21:18:33.534 INFO:teuthology.orchestra.run.smithi184.stdout:truncating to largest ever: 0xeb46400 2024-09-17T21:18:33.556 INFO:teuthology.orchestra.run.smithi184.stdout:631 trunc from 0x815b000 to 0xeb46400 2024-09-17T21:18:33.568 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:33.564+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:33.573 INFO:teuthology.orchestra.run.smithi184.stdout:633 punch from 0x2b5d000 to 0x2b66800, (0x9800 bytes) 2024-09-17T21:18:33.600 INFO:teuthology.orchestra.run.smithi184.stdout:634 trunc from 0xeb46400 to 0x2406000 2024-09-17T21:18:33.650 INFO:teuthology.orchestra.run.smithi184.stdout:635 write 0x1f000 thru 0x206b2 (0x16b3 bytes) 2024-09-17T21:18:33.659 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:33.660 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:33.669 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:33.664+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:33.675 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:18:33.690 INFO:teuthology.orchestra.run.smithi184.stdout:636 write 0x7e96800 thru 0x7e97360 (0xb61 bytes) 2024-09-17T21:18:33.693 INFO:teuthology.orchestra.run.smithi184.stdout:638 read 0x1991000 thru 0x1996d16 (0x5d17 bytes) 2024-09-17T21:18:33.695 INFO:teuthology.orchestra.run.smithi184.stdout:643 write 0x59b7800 thru 0x59c1f71 (0xa772 bytes) 2024-09-17T21:18:33.696 INFO:teuthology.orchestra.run.smithi184.stdout:644 write 0x254d800 thru 0x25572fb (0x9afc bytes) 2024-09-17T21:18:33.697 INFO:teuthology.orchestra.run.smithi184.stdout:645 write 0x613f600 thru 0x6143a6a (0x446b bytes) 2024-09-17T21:18:33.722 INFO:teuthology.orchestra.run.smithi184.stdout:646 trunc from 0x7e97400 to 0x10ae400 2024-09-17T21:18:33.755 INFO:teuthology.orchestra.run.smithi083.stderr:set 3.1e pg_upmap mapping to [0,4] 2024-09-17T21:18:33.756 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:18:33.781 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:33.776+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:33.796 INFO:teuthology.orchestra.run.smithi184.stdout:647 write 0x6250200 thru 0x62576f3 (0x74f4 bytes) 2024-09-17T21:18:33.799 INFO:teuthology.orchestra.run.smithi184.stdout:648 write 0x4f44800 thru 0x4f4c9c0 (0x81c1 bytes) 2024-09-17T21:18:33.807 INFO:teuthology.orchestra.run.smithi184.stdout:649 trunc from 0x6257800 to 0x311fa00 2024-09-17T21:18:33.811 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:33.811 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:33.828 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:18:33.843 INFO:teuthology.orchestra.run.smithi184.stdout:truncating to largest ever: 0xed4fe00 2024-09-17T21:18:33.871 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:33.864+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:33.873 INFO:teuthology.orchestra.run.smithi184.stdout:651 trunc from 0x311fa00 to 0xed4fe00 2024-09-17T21:18:33.890 INFO:teuthology.orchestra.run.smithi184.stdout:654 write 0x8466800 thru 0x846e1d4 (0x79d5 bytes) 2024-09-17T21:18:33.921 INFO:teuthology.orchestra.run.smithi184.stdout:655 trunc from 0xed4fe00 to 0x25c1200 2024-09-17T21:18:33.955 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:33.955 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:33.972 INFO:teuthology.orchestra.run.smithi184.stdout:657 read 0x2465600 thru 0x246d4f9 (0x7efa bytes) 2024-09-17T21:18:33.973 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:33.966+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:33.973 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:18:33.976 INFO:teuthology.orchestra.run.smithi184.stdout:660 punch from 0x24dea00 to 0x24ea000, (0xb600 bytes) 2024-09-17T21:18:33.977 INFO:teuthology.orchestra.run.smithi184.stdout:661 read 0xd6fa00 thru 0xd7ce32 (0xd433 bytes) 2024-09-17T21:18:34.022 INFO:teuthology.orchestra.run.smithi184.stdout:662 write 0xb45a200 thru 0xb45f50f (0x5310 bytes) 2024-09-17T21:18:34.025 INFO:teuthology.orchestra.run.smithi184.stdout:663 read 0x6e34e00 thru 0x6e3f5fb (0xa7fc bytes) 2024-09-17T21:18:34.026 INFO:teuthology.orchestra.run.smithi184.stdout:664 read 0x8696000 thru 0x869bc66 (0x5c67 bytes) 2024-09-17T21:18:34.027 INFO:teuthology.orchestra.run.smithi184.stdout:665 read 0xc8e400 thru 0xc9ce07 (0xea08 bytes) 2024-09-17T21:18:34.029 INFO:teuthology.orchestra.run.smithi184.stdout:667 read 0x417ca00 thru 0x418c284 (0xf885 bytes) 2024-09-17T21:18:34.030 INFO:teuthology.orchestra.run.smithi184.stdout:669 read 0x8de1400 thru 0x8df0de3 (0xf9e4 bytes) 2024-09-17T21:18:34.031 INFO:teuthology.orchestra.run.smithi184.stdout:670 write 0x1ea4400 thru 0x1eacb92 (0x8793 bytes) 2024-09-17T21:18:34.033 INFO:teuthology.orchestra.run.smithi184.stdout:671 write 0x6e89800 thru 0x6e8c028 (0x2829 bytes) 2024-09-17T21:18:34.033 INFO:teuthology.orchestra.run.smithi184.stdout:675 punch from 0x5cda00 to 0x5da000, (0xc600 bytes) 2024-09-17T21:18:34.034 INFO:teuthology.orchestra.run.smithi184.stdout:678 read 0xab13c00 thru 0xab1fbea (0xbfeb bytes) 2024-09-17T21:18:34.040 INFO:teuthology.orchestra.run.smithi184.stdout:679 read 0x2948200 thru 0x294f9d6 (0x77d7 bytes) 2024-09-17T21:18:34.041 INFO:teuthology.orchestra.run.smithi184.stdout:680 punch from 0x274400 to 0x276e00, (0x2a00 bytes) 2024-09-17T21:18:34.041 INFO:teuthology.orchestra.run.smithi184.stdout:681 read 0x24ace00 thru 0x24baa86 (0xdc87 bytes) 2024-09-17T21:18:34.042 INFO:teuthology.orchestra.run.smithi184.stdout:682 read 0x17bac00 thru 0x17c0f15 (0x6316 bytes) 2024-09-17T21:18:34.044 INFO:teuthology.orchestra.run.smithi184.stdout:685 punch from 0x6d06800 to 0x6d15e00, (0xf600 bytes) 2024-09-17T21:18:34.044 INFO:teuthology.orchestra.run.smithi184.stdout:687 punch from 0x8d3ac00 to 0x8d3f200, (0x4600 bytes) 2024-09-17T21:18:34.044 INFO:teuthology.orchestra.run.smithi184.stdout:688 write 0xa355000 thru 0xa358771 (0x3772 bytes) 2024-09-17T21:18:34.049 INFO:teuthology.orchestra.run.smithi184.stdout:689 trunc from 0xb45f600 to 0x9288c00 2024-09-17T21:18:34.073 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:34.070+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:34.094 INFO:teuthology.orchestra.run.smithi184.stdout:691 read 0x46d4400 thru 0x46db82c (0x742d bytes) 2024-09-17T21:18:34.102 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:34.102 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:34.113 INFO:teuthology.orchestra.run.smithi184.stdout:692 trunc from 0x9288c00 to 0xe5dc600 2024-09-17T21:18:34.116 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:18:34.140 INFO:teuthology.orchestra.run.smithi184.stdout:693 trunc from 0xe5dc600 to 0xd1e9000 2024-09-17T21:18:34.164 INFO:teuthology.orchestra.run.smithi184.stdout:694 read 0x7f47400 thru 0x7f533e5 (0xbfe6 bytes) 2024-09-17T21:18:34.169 INFO:teuthology.orchestra.run.smithi184.stdout:696 punch from 0x22d7200 to 0x22de400, (0x7200 bytes) 2024-09-17T21:18:34.170 INFO:teuthology.orchestra.run.smithi184.stdout:698 write 0x3031c00 thru 0x303f65f (0xda60 bytes) 2024-09-17T21:18:34.174 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:34.170+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:34.179 INFO:teuthology.orchestra.run.smithi184.stdout:699 trunc from 0xd1e9000 to 0x983f800 2024-09-17T21:18:34.216 INFO:teuthology.orchestra.run.smithi184.stdout:700 write 0x84e1e00 thru 0x84e6b05 (0x4d06 bytes) 2024-09-17T21:18:34.224 INFO:teuthology.orchestra.run.smithi184.stdout:702 trunc from 0x983f800 to 0xb3efe00 2024-09-17T21:18:34.240 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:34.241 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:34.255 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:18:34.260 INFO:teuthology.orchestra.run.smithi184.stdout:704 trunc from 0xb3efe00 to 0x82bbc00 2024-09-17T21:18:34.275 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:34.270+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:34.294 INFO:teuthology.orchestra.run.smithi184.stdout:706 punch from 0x1474a00 to 0x1476200, (0x1800 bytes) 2024-09-17T21:18:34.305 INFO:teuthology.orchestra.run.smithi184.stdout:708 trunc from 0x82bbc00 to 0xc2b9800 2024-09-17T21:18:34.326 INFO:teuthology.orchestra.run.smithi184.stdout:709 read 0x3bb3e00 thru 0x3bb7365 (0x3566 bytes) 2024-09-17T21:18:34.330 INFO:teuthology.orchestra.run.smithi184.stdout:710 punch from 0x3ca2a00 to 0x3cad600, (0xac00 bytes) 2024-09-17T21:18:34.330 INFO:teuthology.orchestra.run.smithi184.stdout:718 write 0xab06000 thru 0xab13e17 (0xde18 bytes) 2024-09-17T21:18:34.348 INFO:teuthology.orchestra.run.smithi184.stdout:719 trunc from 0xc2b9800 to 0x5463800 2024-09-17T21:18:34.376 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:34.370+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:34.379 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:34.379 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:34.390 INFO:teuthology.orchestra.run.smithi184.stdout:720 write 0x31bfa00 thru 0x31c8765 (0x8d66 bytes) 2024-09-17T21:18:34.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:18:34.422 INFO:teuthology.orchestra.run.smithi184.stdout:721 write 0x5e30c00 thru 0x5e38f15 (0x8316 bytes) 2024-09-17T21:18:34.427 INFO:teuthology.orchestra.run.smithi184.stdout:725 read 0x1c7e200 thru 0x1c88a8e (0xa88f bytes) 2024-09-17T21:18:34.477 INFO:teuthology.orchestra.run.smithi184.stdout:726 write 0xe1f4200 thru 0xe203e0c (0xfc0d bytes) 2024-09-17T21:18:34.477 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:34.470+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:34.479 INFO:teuthology.orchestra.run.smithi184.stdout:727 read 0x317c600 thru 0x3182650 (0x6051 bytes) 2024-09-17T21:18:34.482 INFO:teuthology.orchestra.run.smithi184.stdout:728 read 0xe58000 thru 0xe5d0af (0x50b0 bytes) 2024-09-17T21:18:34.487 INFO:teuthology.orchestra.run.smithi184.stdout:729 trunc from 0xe204000 to 0xcc63000 2024-09-17T21:18:34.504 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:18:34.519 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:34.519 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:34.524 INFO:teuthology.orchestra.run.smithi184.stdout:730 read 0xbc97400 thru 0xbca3af4 (0xc6f5 bytes) 2024-09-17T21:18:34.532 INFO:teuthology.orchestra.run.smithi184.stdout:732 trunc from 0xcc63000 to 0xb036800 2024-09-17T21:18:34.534 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:18:34.577 INFO:teuthology.orchestra.run.smithi184.stdout:733 read 0x3886200 thru 0x388cd78 (0x6b79 bytes) 2024-09-17T21:18:34.578 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:34.574+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:34.581 INFO:teuthology.orchestra.run.smithi184.stdout:736 clone 7 order 24 su 65536 sc 2 2024-09-17T21:18:34.658 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:18:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:18:34.675 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:18:34.675 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:18:34.678 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:34.672+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:34.691 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:18:34.692 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:18:34.707 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:34.707 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:34.707 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:34.707 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:34.707 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:34.715 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:18:34.715 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:18:34.721 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:18:34.736 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:18:34.736 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:18:34.753 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-17T21:18:34.760 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:18:34.761 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:18:34.779 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:34.774+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:34.867 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:34.867 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:18:34.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:18:34.867 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:18:34.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.867 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4447.0:387 3.2 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 16423936~46080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T21:18:34.868 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:21.597931+0000", 2024-09-17T21:18:34.868 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 13.266843228999999, 2024-09-17T21:18:34.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041500774999999997, 2024-09-17T21:18:34.868 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:34.868 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:34.868 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:34.868 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:34.868 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4447", 2024-09-17T21:18:34.868 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:34.868 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 387 2024-09-17T21:18:34.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.868 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:34.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:34.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.597931+0000", 2024-09-17T21:18:34.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:34.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.597931+0000", 2024-09-17T21:18:34.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:34.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.597963+0000", 2024-09-17T21:18:34.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1754999999999997e-05 2024-09-17T21:18:34.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:34.870 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.602945+0000", 2024-09-17T21:18:34.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049825319999999996 2024-09-17T21:18:34.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:34.870 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.602970+0000", 2024-09-17T21:18:34.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4956999999999998e-05 2024-09-17T21:18:34.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:34.871 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.602980+0000", 2024-09-17T21:18:34.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5899999999999997e-06 2024-09-17T21:18:34.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.871 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:34.871 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.603060+0000", 2024-09-17T21:18:34.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0412000000000002e-05 2024-09-17T21:18:34.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.871 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:34.871 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.603182+0000", 2024-09-17T21:18:34.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121302 2024-09-17T21:18:34.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:18:34.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.603419+0000", 2024-09-17T21:18:34.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023761499999999999 2024-09-17T21:18:34.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:34.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.611819+0000", 2024-09-17T21:18:34.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00839992 2024-09-17T21:18:34.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.873 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:34.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.639315+0000", 2024-09-17T21:18:34.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.027496181000000001 2024-09-17T21:18:34.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.873 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:34.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.639369+0000", 2024-09-17T21:18:34.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3909999999999997e-05 2024-09-17T21:18:34.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.873 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:34.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:21.639432+0000", 2024-09-17T21:18:34.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2601000000000006e-05 2024-09-17T21:18:34.874 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.874 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:34.874 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.874 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.874 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:468 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 2359296~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:34.874 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197658+0000", 2024-09-17T21:18:34.874 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.667116214, 2024-09-17T21:18:34.874 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041681556000000002, 2024-09-17T21:18:34.874 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:34.874 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:34.874 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:34.874 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:34.874 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:34.874 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:34.875 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 468 2024-09-17T21:18:34.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.875 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:34.875 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.875 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:34.875 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197658+0000", 2024-09-17T21:18:34.875 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.875 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.875 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:34.875 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197658+0000", 2024-09-17T21:18:34.875 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.875 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.876 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:34.876 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197659+0000", 2024-09-17T21:18:34.876 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.144e-06 2024-09-17T21:18:34.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.876 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:34.876 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197666+0000", 2024-09-17T21:18:34.876 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.323e-06 2024-09-17T21:18:34.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.876 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:34.876 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197683+0000", 2024-09-17T21:18:34.876 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6699999999999999e-05 2024-09-17T21:18:34.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:34.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197697+0000", 2024-09-17T21:18:34.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3713000000000001e-05 2024-09-17T21:18:34.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:34.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236385+0000", 2024-09-17T21:18:34.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.038687988 2024-09-17T21:18:34.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:34.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236571+0000", 2024-09-17T21:18:34.878 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000186102 2024-09-17T21:18:34.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.878 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:34.878 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239340+0000", 2024-09-17T21:18:34.878 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002768586 2024-09-17T21:18:34.878 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.878 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:34.878 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.878 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:491 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:34.878 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:18:34.879 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.666460335, 2024-09-17T21:18:34.879 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041839013000000001, 2024-09-17T21:18:34.879 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:34.879 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:34.879 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:34.879 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:34.879 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:34.879 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:34.879 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 491 2024-09-17T21:18:34.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.879 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:34.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.879 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:34.879 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:18:34.879 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.880 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:34.880 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:18:34.880 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.880 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:34.880 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198315+0000", 2024-09-17T21:18:34.880 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6599999999999996e-07 2024-09-17T21:18:34.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.880 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:34.880 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198317+0000", 2024-09-17T21:18:34.880 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6639999999999998e-06 2024-09-17T21:18:34.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:34.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198327+0000", 2024-09-17T21:18:34.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5629999999999998e-06 2024-09-17T21:18:34.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:34.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198337+0000", 2024-09-17T21:18:34.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0667e-05 2024-09-17T21:18:34.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:34.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239793+0000", 2024-09-17T21:18:34.882 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041455519000000003 2024-09-17T21:18:34.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.882 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:34.882 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239929+0000", 2024-09-17T21:18:34.882 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135716 2024-09-17T21:18:34.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.882 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:34.882 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240153+0000", 2024-09-17T21:18:34.882 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022421799999999999 2024-09-17T21:18:34.882 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.882 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:34.882 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.883 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:492 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:34.883 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:18:34.883 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.6664219649999996, 2024-09-17T21:18:34.883 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042947165000000002, 2024-09-17T21:18:34.883 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:34.883 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:34.883 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:34.883 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:34.883 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:34.883 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:34.883 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 492 2024-09-17T21:18:34.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.883 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:34.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.884 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:34.884 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:18:34.884 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.884 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:34.884 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:18:34.884 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.884 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:34.884 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198354+0000", 2024-09-17T21:18:34.884 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:18:34.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.885 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:34.885 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198357+0000", 2024-09-17T21:18:34.885 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6969999999999999e-06 2024-09-17T21:18:34.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.885 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:34.885 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198367+0000", 2024-09-17T21:18:34.885 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8430000000000005e-06 2024-09-17T21:18:34.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.885 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:34.885 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198374+0000", 2024-09-17T21:18:34.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9600000000000003e-06 2024-09-17T21:18:34.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.886 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:34.886 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240226+0000", 2024-09-17T21:18:34.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041852157000000001 2024-09-17T21:18:34.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.886 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:34.886 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240409+0000", 2024-09-17T21:18:34.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018244800000000001 2024-09-17T21:18:34.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.886 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:34.886 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241299+0000", 2024-09-17T21:18:34.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000890554 2024-09-17T21:18:34.887 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.887 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:34.887 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.887 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:493 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 4063232~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:34.887 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:18:34.887 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.6663848790000007, 2024-09-17T21:18:34.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043897209, 2024-09-17T21:18:34.887 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:34.887 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:34.887 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:34.887 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:34.887 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:34.887 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:34.888 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 493 2024-09-17T21:18:34.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.888 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:34.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:34.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:18:34.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:34.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:18:34.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:34.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198391+0000", 2024-09-17T21:18:34.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4699999999999999e-06 2024-09-17T21:18:34.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:34.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198396+0000", 2024-09-17T21:18:34.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:18:34.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:34.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198412+0000", 2024-09-17T21:18:34.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6507e-05 2024-09-17T21:18:34.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:34.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198425+0000", 2024-09-17T21:18:34.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.256e-05 2024-09-17T21:18:34.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:34.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241383+0000", 2024-09-17T21:18:34.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042958216 2024-09-17T21:18:34.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:34.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241525+0000", 2024-09-17T21:18:34.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014204500000000001 2024-09-17T21:18:34.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:34.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242287+0000", 2024-09-17T21:18:34.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076125800000000001 2024-09-17T21:18:34.891 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.891 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:34.891 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.892 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:496 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:34.892 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:18:34.892 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.6662944799999995, 2024-09-17T21:18:34.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043732500000000001, 2024-09-17T21:18:34.892 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:34.892 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:34.892 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:34.892 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:34.892 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:34.892 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:34.892 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 496 2024-09-17T21:18:34.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.893 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:34.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.893 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:34.893 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:18:34.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.893 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:34.893 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:18:34.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.893 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:34.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198481+0000", 2024-09-17T21:18:34.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-17T21:18:34.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:34.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198488+0000", 2024-09-17T21:18:34.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8499999999999996e-06 2024-09-17T21:18:34.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:34.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198507+0000", 2024-09-17T21:18:34.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9276999999999999e-05 2024-09-17T21:18:34.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:34.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198521+0000", 2024-09-17T21:18:34.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3376e-05 2024-09-17T21:18:34.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:34.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241339+0000", 2024-09-17T21:18:34.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042818119000000002 2024-09-17T21:18:34.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:34.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241449+0000", 2024-09-17T21:18:34.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110209 2024-09-17T21:18:34.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:34.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242212+0000", 2024-09-17T21:18:34.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076329900000000005 2024-09-17T21:18:34.896 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.896 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:34.896 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.896 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:497 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:34.896 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:18:34.896 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.6662135310000004, 2024-09-17T21:18:34.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044584278999999997, 2024-09-17T21:18:34.897 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:34.897 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:34.897 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:34.897 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:34.897 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:34.897 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:34.897 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 497 2024-09-17T21:18:34.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.897 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:34.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:34.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:18:34.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:34.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:18:34.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:34.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198562+0000", 2024-09-17T21:18:34.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-17T21:18:34.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:34.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198569+0000", 2024-09-17T21:18:34.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.494e-06 2024-09-17T21:18:34.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:34.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198588+0000", 2024-09-17T21:18:34.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8955999999999999e-05 2024-09-17T21:18:34.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:34.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198621+0000", 2024-09-17T21:18:34.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3837000000000001e-05 2024-09-17T21:18:34.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:34.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242258+0000", 2024-09-17T21:18:34.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043636900999999999 2024-09-17T21:18:34.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:34.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242343+0000", 2024-09-17T21:18:34.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5048000000000003e-05 2024-09-17T21:18:34.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:34.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243145+0000", 2024-09-17T21:18:34.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00080167300000000003 2024-09-17T21:18:34.900 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.900 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:34.900 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.901 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:503 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:34.901 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:18:34.901 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.6661290070000003, 2024-09-17T21:18:34.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044220582000000001, 2024-09-17T21:18:34.901 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:34.901 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:34.901 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:34.901 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:34.901 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:34.901 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:34.901 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 503 2024-09-17T21:18:34.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.901 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:34.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:34.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:18:34.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:34.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:18:34.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:34.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198647+0000", 2024-09-17T21:18:34.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.483e-06 2024-09-17T21:18:34.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:34.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198653+0000", 2024-09-17T21:18:34.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4529999999999999e-06 2024-09-17T21:18:34.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:34.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198672+0000", 2024-09-17T21:18:34.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9193000000000001e-05 2024-09-17T21:18:34.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:34.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198685+0000", 2024-09-17T21:18:34.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.291e-05 2024-09-17T21:18:34.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:34.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:18:34.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043672141999999997 2024-09-17T21:18:34.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:34.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242716+0000", 2024-09-17T21:18:34.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00035901800000000001 2024-09-17T21:18:34.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:34.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242866+0000", 2024-09-17T21:18:34.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000149383 2024-09-17T21:18:34.905 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.905 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:34.905 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.905 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:504 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:34.905 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:18:34.905 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.6660692150000003, 2024-09-17T21:18:34.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044611406999999999, 2024-09-17T21:18:34.905 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:34.905 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:34.905 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:34.905 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:34.905 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:34.905 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:34.906 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 504 2024-09-17T21:18:34.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.906 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:34.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:34.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:18:34.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:34.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:18:34.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:34.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198706+0000", 2024-09-17T21:18:34.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:18:34.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:34.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198712+0000", 2024-09-17T21:18:34.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5770000000000001e-06 2024-09-17T21:18:34.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:34.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198729+0000", 2024-09-17T21:18:34.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6673000000000001e-05 2024-09-17T21:18:34.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:34.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198742+0000", 2024-09-17T21:18:34.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3543000000000001e-05 2024-09-17T21:18:34.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:34.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242981+0000", 2024-09-17T21:18:34.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044238953999999997 2024-09-17T21:18:34.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:34.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243155+0000", 2024-09-17T21:18:34.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017401600000000001 2024-09-17T21:18:34.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:34.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243316+0000", 2024-09-17T21:18:34.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016137399999999999 2024-09-17T21:18:34.909 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.909 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:34.909 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.909 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:521 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:34.909 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:18:34.909 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.6655220760000002, 2024-09-17T21:18:34.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045043685999999999, 2024-09-17T21:18:34.909 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:34.910 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:34.910 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:34.910 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:34.910 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:34.910 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:34.910 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 521 2024-09-17T21:18:34.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.910 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:34.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:34.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:18:34.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:34.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:18:34.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:34.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199253+0000", 2024-09-17T21:18:34.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1400000000000001e-06 2024-09-17T21:18:34.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:34.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199258+0000", 2024-09-17T21:18:34.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1800000000000004e-06 2024-09-17T21:18:34.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:34.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199276+0000", 2024-09-17T21:18:34.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7686999999999999e-05 2024-09-17T21:18:34.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:34.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199289+0000", 2024-09-17T21:18:34.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3263e-05 2024-09-17T21:18:34.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:34.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243397+0000", 2024-09-17T21:18:34.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044107636999999998 2024-09-17T21:18:34.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:34.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243527+0000", 2024-09-17T21:18:34.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013038399999999999 2024-09-17T21:18:34.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:34.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244296+0000", 2024-09-17T21:18:34.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076839499999999995 2024-09-17T21:18:34.913 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.913 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:34.913 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.914 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:522 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:34.914 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:18:34.914 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.6654667730000003, 2024-09-17T21:18:34.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046049870999999999, 2024-09-17T21:18:34.914 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:34.914 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:34.914 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:34.914 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:34.914 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:34.914 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:34.914 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 522 2024-09-17T21:18:34.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.914 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:34.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:34.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:18:34.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:34.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:18:34.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:34.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199309+0000", 2024-09-17T21:18:34.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:18:34.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:34.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199315+0000", 2024-09-17T21:18:34.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2440000000000001e-06 2024-09-17T21:18:34.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:34.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199333+0000", 2024-09-17T21:18:34.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7886000000000001e-05 2024-09-17T21:18:34.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:34.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199379+0000", 2024-09-17T21:18:34.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6383e-05 2024-09-17T21:18:34.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:34.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244394+0000", 2024-09-17T21:18:34.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045014851000000002 2024-09-17T21:18:34.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:34.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244491+0000", 2024-09-17T21:18:34.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7245999999999997e-05 2024-09-17T21:18:34.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:34.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245357+0000", 2024-09-17T21:18:34.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00086626800000000001 2024-09-17T21:18:34.918 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.918 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:34.918 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.918 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:527 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:34.918 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:18:34.919 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.6653701069999993, 2024-09-17T21:18:34.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045812522000000001, 2024-09-17T21:18:34.919 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:34.919 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:34.919 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:34.919 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:34.919 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:34.919 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:34.919 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 527 2024-09-17T21:18:34.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.919 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:34.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:34.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:18:34.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:34.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:18:34.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:34.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199406+0000", 2024-09-17T21:18:34.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.367e-06 2024-09-17T21:18:34.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:34.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199412+0000", 2024-09-17T21:18:34.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:18:34.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:34.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199429+0000", 2024-09-17T21:18:34.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7396000000000001e-05 2024-09-17T21:18:34.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:34.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199442+0000", 2024-09-17T21:18:34.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3327e-05 2024-09-17T21:18:34.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:34.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244361+0000", 2024-09-17T21:18:34.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044918197 2024-09-17T21:18:34.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:34.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244482+0000", 2024-09-17T21:18:34.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121945 2024-09-17T21:18:34.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:34.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245217+0000", 2024-09-17T21:18:34.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00073415999999999998 2024-09-17T21:18:34.923 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.923 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:34.923 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.924 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:534 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:34.924 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:18:34.924 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.6653114010000003, 2024-09-17T21:18:34.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046292505999999997, 2024-09-17T21:18:34.924 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:34.924 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:34.924 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:34.924 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:34.924 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:34.924 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:34.924 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:18:34.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.924 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:34.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:34.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:18:34.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:34.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:18:34.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:34.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199464+0000", 2024-09-17T21:18:34.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:18:34.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:34.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199473+0000", 2024-09-17T21:18:34.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5629999999999992e-06 2024-09-17T21:18:34.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:34.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199492+0000", 2024-09-17T21:18:34.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9273999999999999e-05 2024-09-17T21:18:34.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:34.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199505+0000", 2024-09-17T21:18:34.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3443e-05 2024-09-17T21:18:34.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:34.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245357+0000", 2024-09-17T21:18:34.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045851714000000002 2024-09-17T21:18:34.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:34.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245550+0000", 2024-09-17T21:18:34.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019301600000000001 2024-09-17T21:18:34.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:34.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245755+0000", 2024-09-17T21:18:34.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205226 2024-09-17T21:18:34.928 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.928 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:34.928 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.928 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:535 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:34.928 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:18:34.928 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.6652068450000002, 2024-09-17T21:18:34.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046637492000000003, 2024-09-17T21:18:34.928 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:34.928 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:34.928 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:34.928 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:34.928 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:34.929 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:34.929 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 535 2024-09-17T21:18:34.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.929 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:34.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:34.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:18:34.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:34.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:18:34.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:34.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199568+0000", 2024-09-17T21:18:34.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.037e-06 2024-09-17T21:18:34.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:34.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199572+0000", 2024-09-17T21:18:34.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5729999999999999e-06 2024-09-17T21:18:34.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:34.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199585+0000", 2024-09-17T21:18:34.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.279e-05 2024-09-17T21:18:34.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:34.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199592+0000", 2024-09-17T21:18:34.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.61e-06 2024-09-17T21:18:34.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:34.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245873+0000", 2024-09-17T21:18:34.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046280494999999998 2024-09-17T21:18:34.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:34.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246018+0000", 2024-09-17T21:18:34.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000144697 2024-09-17T21:18:34.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:34.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246205+0000", 2024-09-17T21:18:34.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018729 2024-09-17T21:18:34.932 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.932 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:34.932 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.933 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:543 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:34.933 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:18:34.933 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.6651115989999994, 2024-09-17T21:18:34.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047564097999999999, 2024-09-17T21:18:34.933 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:34.933 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:34.933 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:34.933 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:34.933 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:34.933 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:34.933 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 543 2024-09-17T21:18:34.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.933 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:34.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:34.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:18:34.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:34.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:18:34.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:34.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:18:34.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9699999999999995e-07 2024-09-17T21:18:34.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:34.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199677+0000", 2024-09-17T21:18:34.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:18:34.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:34.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199690+0000", 2024-09-17T21:18:34.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2977e-05 2024-09-17T21:18:34.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:34.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199705+0000", 2024-09-17T21:18:34.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4656e-05 2024-09-17T21:18:34.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:34.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246307+0000", 2024-09-17T21:18:34.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046601780000000002 2024-09-17T21:18:34.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:34.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246475+0000", 2024-09-17T21:18:34.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016834200000000001 2024-09-17T21:18:34.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:34.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247227+0000", 2024-09-17T21:18:34.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00075173299999999998 2024-09-17T21:18:34.937 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.937 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:34.937 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.937 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:544 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:34.937 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:18:34.937 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.6650531900000001, 2024-09-17T21:18:34.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049283559999999997, 2024-09-17T21:18:34.937 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:34.937 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:34.937 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:34.937 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:34.938 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:34.938 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:34.938 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 544 2024-09-17T21:18:34.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.938 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:34.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:34.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:18:34.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:34.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:18:34.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:34.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199722+0000", 2024-09-17T21:18:34.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.08e-06 2024-09-17T21:18:34.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:34.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199738+0000", 2024-09-17T21:18:34.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5943999999999998e-05 2024-09-17T21:18:34.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:34.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199748+0000", 2024-09-17T21:18:34.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0193e-05 2024-09-17T21:18:34.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:34.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199756+0000", 2024-09-17T21:18:34.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4200000000000001e-06 2024-09-17T21:18:34.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:34.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247311+0000", 2024-09-17T21:18:34.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047555300000000002 2024-09-17T21:18:34.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:34.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247571+0000", 2024-09-17T21:18:34.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025996999999999999 2024-09-17T21:18:34.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:34.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249005+0000", 2024-09-17T21:18:34.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001433653 2024-09-17T21:18:34.941 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.941 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:34.941 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.941 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:549 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:34.941 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:18:34.942 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.6650075659999999, 2024-09-17T21:18:34.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048464528999999999, 2024-09-17T21:18:34.942 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:34.942 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:34.942 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:34.942 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:34.942 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:34.942 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:34.942 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 549 2024-09-17T21:18:34.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.942 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:34.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:34.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:18:34.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:34.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:18:34.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:34.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:18:34.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:18:34.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:34.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199770+0000", 2024-09-17T21:18:34.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9969999999999999e-06 2024-09-17T21:18:34.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:34.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199780+0000", 2024-09-17T21:18:34.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0003000000000001e-05 2024-09-17T21:18:34.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:34.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199793+0000", 2024-09-17T21:18:34.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2074e-05 2024-09-17T21:18:34.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:34.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247289+0000", 2024-09-17T21:18:34.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047496691000000001 2024-09-17T21:18:34.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:34.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247437+0000", 2024-09-17T21:18:34.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014807899999999999 2024-09-17T21:18:34.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:34.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248231+0000", 2024-09-17T21:18:34.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00079390899999999998 2024-09-17T21:18:34.946 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.946 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:34.946 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.946 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:558 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:34.946 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:18:34.946 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.6649670529999998, 2024-09-17T21:18:34.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049709821000000001, 2024-09-17T21:18:34.947 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:34.947 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:34.947 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:34.947 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:34.947 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:34.947 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:34.947 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 558 2024-09-17T21:18:34.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.947 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:34.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:34.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:18:34.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:34.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:18:34.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:34.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199809+0000", 2024-09-17T21:18:34.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.333e-06 2024-09-17T21:18:34.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:34.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199814+0000", 2024-09-17T21:18:34.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0930000000000002e-06 2024-09-17T21:18:34.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:34.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199831+0000", 2024-09-17T21:18:34.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7580000000000001e-05 2024-09-17T21:18:34.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:34.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199860+0000", 2024-09-17T21:18:34.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8609999999999999e-05 2024-09-17T21:18:34.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:34.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249106+0000", 2024-09-17T21:18:34.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049246545000000003 2024-09-17T21:18:34.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:34.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249274+0000", 2024-09-17T21:18:34.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016806699999999999 2024-09-17T21:18:34.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:34.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249517+0000", 2024-09-17T21:18:34.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024259300000000001 2024-09-17T21:18:34.950 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.950 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:34.951 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.951 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:559 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:34.951 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:18:34.951 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.6648476910000003, 2024-09-17T21:18:34.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049990694000000002, 2024-09-17T21:18:34.951 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:34.951 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:34.951 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:34.951 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:34.951 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:34.951 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:34.952 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 559 2024-09-17T21:18:34.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.952 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:34.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:34.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:18:34.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:34.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:18:34.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:34.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199928+0000", 2024-09-17T21:18:34.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:18:34.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:34.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199935+0000", 2024-09-17T21:18:34.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5429999999999999e-06 2024-09-17T21:18:34.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:34.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199954+0000", 2024-09-17T21:18:34.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.912e-05 2024-09-17T21:18:34.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:34.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199969+0000", 2024-09-17T21:18:34.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5305999999999999e-05 2024-09-17T21:18:34.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:34.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249604+0000", 2024-09-17T21:18:34.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049634931 2024-09-17T21:18:34.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:34.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249738+0000", 2024-09-17T21:18:34.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013411599999999999 2024-09-17T21:18:34.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:34.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249917+0000", 2024-09-17T21:18:34.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179111 2024-09-17T21:18:34.955 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.955 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:34.955 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.955 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:560 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:34.955 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:18:34.956 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.6647832749999996, 2024-09-17T21:18:34.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050173059999999998, 2024-09-17T21:18:34.956 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:34.956 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:34.956 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:34.956 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:34.956 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:34.956 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:34.956 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 560 2024-09-17T21:18:34.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.956 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:34.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:34.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:18:34.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:34.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:18:34.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:34.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:34.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199992+0000", 2024-09-17T21:18:34.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.517e-06 2024-09-17T21:18:34.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:34.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199998+0000", 2024-09-17T21:18:34.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1460000000000004e-06 2024-09-17T21:18:34.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:34.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:18:34.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7407000000000001e-05 2024-09-17T21:18:34.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:34.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200029+0000", 2024-09-17T21:18:34.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3499999999999999e-05 2024-09-17T21:18:34.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:34.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249968+0000", 2024-09-17T21:18:34.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049939799 2024-09-17T21:18:34.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:34.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250055+0000", 2024-09-17T21:18:34.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6731999999999996e-05 2024-09-17T21:18:34.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:34.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:34.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:34.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250164+0000", 2024-09-17T21:18:34.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108959 2024-09-17T21:18:34.959 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.959 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:34.959 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.959 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:34.960 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:34.960 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:34.960 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone6 from 0xb036800 (overlap 0x10ae400) to 0x0 2024-09-17T21:18:34.961 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:18:34.962 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:34.956+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:35.062 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:35.056+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:35.139 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:35.139 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:35.139 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:35.139 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:35.153 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:18:35.163 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:35.162+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:35.264 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:35.258+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:35.333 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:35.333 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:35.333 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:35.333 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:35.333 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:35.347 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:18:35.392 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:35.388+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:35.466 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:35.462+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:35.476 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:35.476 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:18:35.476 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:18:35.476 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:18:35.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.477 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:12 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:35.477 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:35.477 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.141423257, 2024-09-17T21:18:35.477 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061340989999999996, 2024-09-17T21:18:35.477 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:35.477 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:35.477 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:35.477 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:35.477 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:35.477 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:35.477 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12 2024-09-17T21:18:35.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.478 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:35.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:35.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:35.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:35.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:35.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:35.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332336+0000", 2024-09-17T21:18:35.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5519999999999999e-06 2024-09-17T21:18:35.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:35.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332342+0000", 2024-09-17T21:18:35.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7180000000000001e-06 2024-09-17T21:18:35.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:35.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332352+0000", 2024-09-17T21:18:35.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:18:35.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:35.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332359+0000", 2024-09-17T21:18:35.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3479999999999998e-06 2024-09-17T21:18:35.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:35.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332413+0000", 2024-09-17T21:18:35.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4447999999999998e-05 2024-09-17T21:18:35.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-17T21:18:35.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332417+0000", 2024-09-17T21:18:35.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.895e-06 2024-09-17T21:18:35.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.481 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:35.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334755+0000", 2024-09-17T21:18:35.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023377189999999998 2024-09-17T21:18:35.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.481 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:35.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334903+0000", 2024-09-17T21:18:35.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014778999999999999 2024-09-17T21:18:35.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.481 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:18:35.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.335187+0000", 2024-09-17T21:18:35.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000284715 2024-09-17T21:18:35.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.482 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:35.482 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.337731+0000", 2024-09-17T21:18:35.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002543741 2024-09-17T21:18:35.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.482 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:35.482 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338382+0000", 2024-09-17T21:18:35.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00065029200000000001 2024-09-17T21:18:35.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:35.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338418+0000", 2024-09-17T21:18:35.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.697e-05 2024-09-17T21:18:35.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:35.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338469+0000", 2024-09-17T21:18:35.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0154999999999997e-05 2024-09-17T21:18:35.483 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.483 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:35.483 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.483 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "pg_update_log_missing(3.1e epoch 28/27 rep_tid 81 entries 28'41 (0'0) error 3:7b23fd98:::rbd_header.1152c01721e5:head by client.4488.0:561 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-17T21:18:35.484 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:18:35.484 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.192149130000001, 2024-09-17T21:18:35.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0051146969999999996, 2024-09-17T21:18:35.484 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:35.484 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:35.484 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:18:35.484 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:35.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:35.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:18:35.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:35.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:18:35.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:35.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281610+0000", 2024-09-17T21:18:35.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.16e-07 2024-09-17T21:18:35.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:35.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281626+0000", 2024-09-17T21:18:35.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6039999999999999e-05 2024-09-17T21:18:35.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:35.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281666+0000", 2024-09-17T21:18:35.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0364000000000002e-05 2024-09-17T21:18:35.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:35.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281710+0000", 2024-09-17T21:18:35.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3596000000000001e-05 2024-09-17T21:18:35.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:35.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281992+0000", 2024-09-17T21:18:35.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028273099999999999 2024-09-17T21:18:35.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:35.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286723+0000", 2024-09-17T21:18:35.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047310499999999997 2024-09-17T21:18:35.487 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.487 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:35.487 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.487 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4488.0:562 3.1e e28/27 3:7b23fd98:::rbd_header.1152c01721e5:head v 28'42, mlcod=0'0)", 2024-09-17T21:18:35.487 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:18:35.487 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.191905341999998, 2024-09-17T21:18:35.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049275020000000003, 2024-09-17T21:18:35.487 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:35.487 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:35.487 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:35.487 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:35.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:35.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:18:35.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:35.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:18:35.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:35.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281853+0000", 2024-09-17T21:18:35.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2999999999999999e-07 2024-09-17T21:18:35.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:35.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281875+0000", 2024-09-17T21:18:35.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1386000000000002e-05 2024-09-17T21:18:35.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:35.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281892+0000", 2024-09-17T21:18:35.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7637000000000001e-05 2024-09-17T21:18:35.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:35.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281908+0000", 2024-09-17T21:18:35.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5187e-05 2024-09-17T21:18:35.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:35.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.282517+0000", 2024-09-17T21:18:35.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00060989799999999999 2024-09-17T21:18:35.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:35.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.282548+0000", 2024-09-17T21:18:35.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0910000000000001e-05 2024-09-17T21:18:35.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:35.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286729+0000", 2024-09-17T21:18:35.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0041810850000000002 2024-09-17T21:18:35.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:35.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286780+0000", 2024-09-17T21:18:35.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0569000000000003e-05 2024-09-17T21:18:35.491 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.491 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:35.491 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.491 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:609 3.d e28/23 3:b7278be9:::rbd_data.1152c6af092e.000000000000008f:head v 28'6, mlcod=28'4)", 2024-09-17T21:18:35.491 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:18:35.491 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 17.358961055000002, 2024-09-17T21:18:35.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049797280000000001, 2024-09-17T21:18:35.492 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:35.492 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:35.492 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:35.492 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:35.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:35.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:18:35.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:35.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:18:35.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:35.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114800+0000", 2024-09-17T21:18:35.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:18:35.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:35.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114915+0000", 2024-09-17T21:18:35.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115053 2024-09-17T21:18:35.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:35.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114951+0000", 2024-09-17T21:18:35.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6165999999999999e-05 2024-09-17T21:18:35.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:35.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114965+0000", 2024-09-17T21:18:35.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4364e-05 2024-09-17T21:18:35.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:35.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115097+0000", 2024-09-17T21:18:35.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013139200000000001 2024-09-17T21:18:35.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.495 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:35.495 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115151+0000", 2024-09-17T21:18:35.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4422000000000002e-05 2024-09-17T21:18:35.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.495 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:35.495 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.119633+0000", 2024-09-17T21:18:35.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044818820000000004 2024-09-17T21:18:35.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.495 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:35.495 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.119776+0000", 2024-09-17T21:18:35.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000143609 2024-09-17T21:18:35.496 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.496 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:35.496 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.496 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:782 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'63, mlcod=28'60)", 2024-09-17T21:18:35.496 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:18:35.496 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 17.258993026999999, 2024-09-17T21:18:35.496 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006167283, 2024-09-17T21:18:35.496 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:35.496 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:35.496 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:35.496 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:35.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.497 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:35.497 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:18:35.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.497 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:35.497 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:18:35.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.497 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:35.497 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214767+0000", 2024-09-17T21:18:35.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:18:35.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:35.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214779+0000", 2024-09-17T21:18:35.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2627e-05 2024-09-17T21:18:35.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:35.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214805+0000", 2024-09-17T21:18:35.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5570000000000001e-05 2024-09-17T21:18:35.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:35.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214823+0000", 2024-09-17T21:18:35.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8202999999999998e-05 2024-09-17T21:18:35.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:35.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215170+0000", 2024-09-17T21:18:35.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034720399999999999 2024-09-17T21:18:35.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:35.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215235+0000", 2024-09-17T21:18:35.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4276000000000006e-05 2024-09-17T21:18:35.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:35.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220859+0000", 2024-09-17T21:18:35.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056240500000000002 2024-09-17T21:18:35.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:35.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220932+0000", 2024-09-17T21:18:35.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3503000000000004e-05 2024-09-17T21:18:35.500 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.500 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:35.500 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.500 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:783 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'64, mlcod=28'60)", 2024-09-17T21:18:35.501 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:18:35.501 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 17.258306184999999, 2024-09-17T21:18:35.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0055240869999999996, 2024-09-17T21:18:35.501 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:35.501 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:35.501 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:35.501 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:35.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:35.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:18:35.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:35.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:18:35.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:35.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215454+0000", 2024-09-17T21:18:35.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0269999999999998e-06 2024-09-17T21:18:35.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:35.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215468+0000", 2024-09-17T21:18:35.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4816e-05 2024-09-17T21:18:35.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:35.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215501+0000", 2024-09-17T21:18:35.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2957000000000003e-05 2024-09-17T21:18:35.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:35.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215524+0000", 2024-09-17T21:18:35.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2782999999999999e-05 2024-09-17T21:18:35.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:35.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216675+0000", 2024-09-17T21:18:35.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011507869999999999 2024-09-17T21:18:35.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:35.504 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216728+0000", 2024-09-17T21:18:35.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3112999999999998e-05 2024-09-17T21:18:35.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:35.504 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220938+0000", 2024-09-17T21:18:35.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042097380000000002 2024-09-17T21:18:35.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:35.504 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220976+0000", 2024-09-17T21:18:35.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7866e-05 2024-09-17T21:18:35.505 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.505 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:35.505 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.505 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:785 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'65, mlcod=28'60)", 2024-09-17T21:18:35.505 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:18:35.505 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 17.257720136, 2024-09-17T21:18:35.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049795409999999997, 2024-09-17T21:18:35.505 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:35.505 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:35.505 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:35.505 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:35.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:35.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:18:35.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:35.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:18:35.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:35.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216039+0000", 2024-09-17T21:18:35.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.793e-06 2024-09-17T21:18:35.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.507 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:35.507 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216053+0000", 2024-09-17T21:18:35.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.345e-05 2024-09-17T21:18:35.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.507 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:35.507 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216092+0000", 2024-09-17T21:18:35.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9180000000000001e-05 2024-09-17T21:18:35.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.507 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:35.507 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216106+0000", 2024-09-17T21:18:35.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:18:35.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.508 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:35.508 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217569+0000", 2024-09-17T21:18:35.508 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001462938 2024-09-17T21:18:35.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.508 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:35.508 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217607+0000", 2024-09-17T21:18:35.508 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7267000000000001e-05 2024-09-17T21:18:35.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.508 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:35.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220980+0000", 2024-09-17T21:18:35.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003373544 2024-09-17T21:18:35.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:35.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221017+0000", 2024-09-17T21:18:35.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7116000000000002e-05 2024-09-17T21:18:35.509 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.509 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:35.509 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.510 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:786 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'66, mlcod=28'60)", 2024-09-17T21:18:35.510 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:18:35.510 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 17.257019331999999, 2024-09-17T21:18:35.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057966650000000003, 2024-09-17T21:18:35.510 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:35.510 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:35.510 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:35.510 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:35.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:35.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:18:35.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:35.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:18:35.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:35.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216740+0000", 2024-09-17T21:18:35.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5600000000000001e-06 2024-09-17T21:18:35.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.512 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:35.512 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216752+0000", 2024-09-17T21:18:35.512 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1477e-05 2024-09-17T21:18:35.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.512 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:35.512 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216779+0000", 2024-09-17T21:18:35.512 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7093e-05 2024-09-17T21:18:35.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.512 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:35.513 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216817+0000", 2024-09-17T21:18:35.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8275999999999997e-05 2024-09-17T21:18:35.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.513 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:35.513 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218387+0000", 2024-09-17T21:18:35.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015703749999999999 2024-09-17T21:18:35.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.513 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:35.513 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218432+0000", 2024-09-17T21:18:35.514 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4910000000000002e-05 2024-09-17T21:18:35.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.514 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:35.514 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222432+0000", 2024-09-17T21:18:35.514 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039997189999999997 2024-09-17T21:18:35.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.514 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:35.514 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222535+0000", 2024-09-17T21:18:35.514 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103255 2024-09-17T21:18:35.514 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.514 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:35.514 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.515 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:18:35.515 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:18:35.515 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 17.256397499999999, 2024-09-17T21:18:35.515 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:18:35.515 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:35.515 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:35.515 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:35.515 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:35.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.515 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:35.515 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:18:35.515 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.516 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:35.516 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:18:35.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.516 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:35.516 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:18:35.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:18:35.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.516 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:35.516 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:18:35.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:18:35.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:35.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:18:35.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:18:35.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:35.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:18:35.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:18:35.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:35.518 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:18:35.518 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:18:35.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.518 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:35.518 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:18:35.518 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:18:35.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.518 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:35.518 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:18:35.518 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:18:35.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.518 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:35.519 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:18:35.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:18:35.519 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.519 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:35.519 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.519 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:788 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'68, mlcod=28'60)", 2024-09-17T21:18:35.519 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:18:35.519 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 17.255763068, 2024-09-17T21:18:35.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058495320000000002, 2024-09-17T21:18:35.519 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:35.519 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:35.519 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:35.520 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:35.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.520 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:35.520 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:18:35.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.520 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:35.520 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:18:35.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.520 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:35.520 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217996+0000", 2024-09-17T21:18:35.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.376e-06 2024-09-17T21:18:35.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:35.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218006+0000", 2024-09-17T21:18:35.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.01e-05 2024-09-17T21:18:35.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:35.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218030+0000", 2024-09-17T21:18:35.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4026999999999999e-05 2024-09-17T21:18:35.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:35.522 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218042+0000", 2024-09-17T21:18:35.522 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1977000000000001e-05 2024-09-17T21:18:35.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.522 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.522 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:35.522 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221081+0000", 2024-09-17T21:18:35.522 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030385960000000002 2024-09-17T21:18:35.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.522 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.522 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:35.522 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221115+0000", 2024-09-17T21:18:35.522 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3772999999999997e-05 2024-09-17T21:18:35.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.522 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.523 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:35.523 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223749+0000", 2024-09-17T21:18:35.523 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0026340629999999999 2024-09-17T21:18:35.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.523 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.523 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:35.523 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223844+0000", 2024-09-17T21:18:35.523 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5619999999999996e-05 2024-09-17T21:18:35.523 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.523 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:35.523 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.523 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.523 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:790 3.5 e28/23 3:a14b1a37:::rbd_data.1152c6af092e.000000000000002f:head v 28'1, mlcod=0'0)", 2024-09-17T21:18:35.524 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:18:35.524 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 17.251358742000001, 2024-09-17T21:18:35.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059622240000000003, 2024-09-17T21:18:35.524 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:35.524 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:35.524 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:35.524 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:35.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:35.524 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:18:35.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:35.524 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:18:35.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.525 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.525 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:35.525 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222400+0000", 2024-09-17T21:18:35.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.02e-06 2024-09-17T21:18:35.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.525 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.525 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:35.525 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222411+0000", 2024-09-17T21:18:35.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.044e-05 2024-09-17T21:18:35.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.525 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:35.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222440+0000", 2024-09-17T21:18:35.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.932e-05 2024-09-17T21:18:35.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:35.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222458+0000", 2024-09-17T21:18:35.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8615999999999999e-05 2024-09-17T21:18:35.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:35.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222609+0000", 2024-09-17T21:18:35.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015102299999999999 2024-09-17T21:18:35.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:35.527 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222657+0000", 2024-09-17T21:18:35.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7206000000000002e-05 2024-09-17T21:18:35.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:35.527 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228278+0000", 2024-09-17T21:18:35.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056208100000000004 2024-09-17T21:18:35.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:35.527 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228361+0000", 2024-09-17T21:18:35.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3788999999999993e-05 2024-09-17T21:18:35.528 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.528 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:35.528 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.528 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:793 3.1e e28/27 3:793c7d73:::rbd_data.1152c6af092e.0000000000000032:head v 28'48, mlcod=28'46)", 2024-09-17T21:18:35.528 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.224161+0000", 2024-09-17T21:18:35.528 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 17.249597289, 2024-09-17T21:18:35.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004730723, 2024-09-17T21:18:35.528 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:35.528 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:35.528 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:35.529 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:35.529 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.529 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:35.529 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224161+0000", 2024-09-17T21:18:35.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.529 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.529 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:35.529 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224161+0000", 2024-09-17T21:18:35.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.529 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.529 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:35.530 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224163+0000", 2024-09-17T21:18:35.530 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.244e-06 2024-09-17T21:18:35.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.530 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:35.530 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224178+0000", 2024-09-17T21:18:35.530 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:18:35.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.530 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:35.530 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224210+0000", 2024-09-17T21:18:35.530 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1152999999999998e-05 2024-09-17T21:18:35.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:35.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224227+0000", 2024-09-17T21:18:35.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7863000000000002e-05 2024-09-17T21:18:35.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:35.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224347+0000", 2024-09-17T21:18:35.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119376 2024-09-17T21:18:35.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:35.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.224385+0000", 2024-09-17T21:18:35.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8408999999999998e-05 2024-09-17T21:18:35.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:35.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228842+0000", 2024-09-17T21:18:35.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044566859999999996 2024-09-17T21:18:35.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:35.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228891+0000", 2024-09-17T21:18:35.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9292000000000002e-05 2024-09-17T21:18:35.532 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.533 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:35.533 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.533 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:884 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'76, mlcod=28'70)", 2024-09-17T21:18:35.533 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:18:35.533 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 17.204773861, 2024-09-17T21:18:35.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056451990000000001, 2024-09-17T21:18:35.533 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:35.533 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:35.533 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:35.533 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:35.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:35.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:18:35.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:35.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:18:35.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:35.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268986+0000", 2024-09-17T21:18:35.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9360000000000002e-06 2024-09-17T21:18:35.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.535 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:35.535 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269001+0000", 2024-09-17T21:18:35.535 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4684e-05 2024-09-17T21:18:35.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.535 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:35.535 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269026+0000", 2024-09-17T21:18:35.535 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5193000000000001e-05 2024-09-17T21:18:35.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.535 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:35.535 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269043+0000", 2024-09-17T21:18:35.536 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7306e-05 2024-09-17T21:18:35.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.536 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:35.536 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.270024+0000", 2024-09-17T21:18:35.536 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00098120600000000009 2024-09-17T21:18:35.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.536 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:35.536 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.270085+0000", 2024-09-17T21:18:35.536 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0486000000000003e-05 2024-09-17T21:18:35.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:35.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.274557+0000", 2024-09-17T21:18:35.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044723419999999998 2024-09-17T21:18:35.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:35.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.274629+0000", 2024-09-17T21:18:35.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2045999999999994e-05 2024-09-17T21:18:35.537 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.537 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:35.537 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.537 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:18:35.538 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:18:35.538 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.606096836000001, 2024-09-17T21:18:35.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:18:35.538 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:35.538 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:35.538 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:35.538 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:35.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:35.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:18:35.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:35.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:18:35.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:35.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:18:35.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:18:35.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:35.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:18:35.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:18:35.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:35.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:18:35.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:18:35.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:35.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:18:35.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:18:35.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:35.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:18:35.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:18:35.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:35.541 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:18:35.541 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:18:35.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.541 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:35.541 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:18:35.541 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:18:35.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.541 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:35.541 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:18:35.541 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:18:35.541 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.542 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:35.542 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.542 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:18:35.542 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:18:35.542 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.604831752999999, 2024-09-17T21:18:35.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:18:35.542 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:35.542 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:35.542 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:35.542 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:35.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.542 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:35.543 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:18:35.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:35.543 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:18:35.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:35.543 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:18:35.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:18:35.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.544 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:35.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:18:35.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:18:35.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.544 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:35.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:18:35.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:18:35.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.544 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:35.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:18:35.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:18:35.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:35.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:18:35.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:18:35.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:35.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:18:35.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:18:35.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:35.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:18:35.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:18:35.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:35.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:18:35.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:18:35.546 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.546 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:35.546 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.546 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:18:35.546 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:18:35.547 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.595052343000001, 2024-09-17T21:18:35.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:18:35.547 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:35.547 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:35.547 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:35.547 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:35.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:35.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:18:35.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:35.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:18:35.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:35.548 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:18:35.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:18:35.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:35.548 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:18:35.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:18:35.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:35.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:18:35.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:18:35.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:35.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:18:35.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:18:35.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:35.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:18:35.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:18:35.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:35.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:18:35.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:18:35.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:35.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:18:35.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:18:35.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:35.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:18:35.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:18:35.551 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.551 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:35.551 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.551 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:18:35.551 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:18:35.551 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.586329940000001, 2024-09-17T21:18:35.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:18:35.551 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:35.551 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:35.551 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:35.551 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:35.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:35.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:18:35.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:35.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:18:35.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:35.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:18:35.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:18:35.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:35.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:18:35.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:18:35.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:35.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:18:35.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:18:35.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:35.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:18:35.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:18:35.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:35.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:18:35.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:18:35.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:35.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:18:35.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:18:35.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:35.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:18:35.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:18:35.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:35.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:18:35.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:18:35.555 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.555 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:35.555 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.555 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4498.0:596 3.4 e32/23 3:20f262ea:::rbd_data.1152b51cb396.000000000000000d:head v 32'17, mlcod=30'15)", 2024-09-17T21:18:35.556 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.258354+0000", 2024-09-17T21:18:35.556 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.2154040760000004, 2024-09-17T21:18:35.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0053727619999999997, 2024-09-17T21:18:35.556 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:35.556 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:35.556 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:35.556 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:35.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:35.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.258354+0000", 2024-09-17T21:18:35.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:35.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.258354+0000", 2024-09-17T21:18:35.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:35.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.258356+0000", 2024-09-17T21:18:35.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.798e-06 2024-09-17T21:18:35.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:35.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259198+0000", 2024-09-17T21:18:35.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084279199999999998 2024-09-17T21:18:35.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:35.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259229+0000", 2024-09-17T21:18:35.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0383999999999999e-05 2024-09-17T21:18:35.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:35.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259241+0000", 2024-09-17T21:18:35.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2139e-05 2024-09-17T21:18:35.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:35.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259341+0000", 2024-09-17T21:18:35.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010049 2024-09-17T21:18:35.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:35.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259380+0000", 2024-09-17T21:18:35.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9072000000000001e-05 2024-09-17T21:18:35.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:35.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.263668+0000", 2024-09-17T21:18:35.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042875389999999999 2024-09-17T21:18:35.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.560 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:35.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.263726+0000", 2024-09-17T21:18:35.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8548000000000003e-05 2024-09-17T21:18:35.560 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.560 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:35.560 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.560 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4520.0:6177 3.8 e34/27 3:1665de58:::rbd_data.11528c3b0529.0000000000000019:head v 34'43, mlcod=34'41)", 2024-09-17T21:18:35.560 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:30.509962+0000", 2024-09-17T21:18:35.560 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.9637960860000003, 2024-09-17T21:18:35.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049320850000000001, 2024-09-17T21:18:35.561 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:35.561 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:35.561 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:35.561 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:35.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:35.561 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509962+0000", 2024-09-17T21:18:35.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:35.561 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509962+0000", 2024-09-17T21:18:35.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:35.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509964+0000", 2024-09-17T21:18:35.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.356e-06 2024-09-17T21:18:35.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:35.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509978+0000", 2024-09-17T21:18:35.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3434e-05 2024-09-17T21:18:35.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:35.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510001+0000", 2024-09-17T21:18:35.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3436000000000001e-05 2024-09-17T21:18:35.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:35.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510023+0000", 2024-09-17T21:18:35.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1739999999999999e-05 2024-09-17T21:18:35.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:35.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510193+0000", 2024-09-17T21:18:35.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017040600000000001 2024-09-17T21:18:35.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:35.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510250+0000", 2024-09-17T21:18:35.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6749000000000002e-05 2024-09-17T21:18:35.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:35.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.514823+0000", 2024-09-17T21:18:35.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0045728180000000002 2024-09-17T21:18:35.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:35.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.514894+0000", 2024-09-17T21:18:35.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1145999999999999e-05 2024-09-17T21:18:35.565 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.565 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:35.565 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.565 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4520.0:6179 3.7 e34/27 3:e77b9c77:::rbd_data.11528c3b0529.0000000000000023:head v 34'39, mlcod=30'37)", 2024-09-17T21:18:35.565 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:30.510076+0000", 2024-09-17T21:18:35.565 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.9636822370000004, 2024-09-17T21:18:35.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00479934, 2024-09-17T21:18:35.565 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:35.565 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:35.565 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:35.565 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:35.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:35.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510076+0000", 2024-09-17T21:18:35.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:35.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510076+0000", 2024-09-17T21:18:35.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:35.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:35.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510076+0000", 2024-09-17T21:18:35.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1299999999999998e-07 2024-09-17T21:18:35.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.567 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:35.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510095+0000", 2024-09-17T21:18:35.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8637000000000001e-05 2024-09-17T21:18:35.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.567 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:35.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510122+0000", 2024-09-17T21:18:35.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7316000000000001e-05 2024-09-17T21:18:35.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.567 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:35.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510136+0000", 2024-09-17T21:18:35.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3824e-05 2024-09-17T21:18:35.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.568 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.568 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:35.568 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510700+0000", 2024-09-17T21:18:35.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00056388200000000003 2024-09-17T21:18:35.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.568 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.568 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:35.568 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510751+0000", 2024-09-17T21:18:35.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0416e-05 2024-09-17T21:18:35.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.568 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.568 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:35.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.514787+0000", 2024-09-17T21:18:35.569 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004036792 2024-09-17T21:18:35.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:35.569 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:35.569 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:35.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.514875+0000", 2024-09-17T21:18:35.569 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7559999999999995e-05 2024-09-17T21:18:35.569 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.569 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:35.569 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.569 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:35.569 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:35.569 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:35.571 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:18:35.571 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:35.564+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:35.613 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-17T21:18:35.672 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:35.668+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:35.713 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:35.714 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:35.714 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:18:35.714 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:35.731 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:18:35.781 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:35.778+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:35.872 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:35.873 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:35.873 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:18:35.873 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:18:35.873 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:35.873 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:35.868+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:35.887 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:18:35.974 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:35.970+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:36.070 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:18:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:18:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:18:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 39.054350350999997, 2024-09-17T21:18:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:18:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:18:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:18:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:18:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:18:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:18:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:18:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:18:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:18:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:18:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:18:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:18:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:18:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:18:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:18:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:18:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:18:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:18:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:18:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:18:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:18:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:18:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:18:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:18:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:18:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:18:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:18:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:18:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:18:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:18:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.233810+0000", 2024-09-17T21:18:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.834305915999998, 2024-09-17T21:18:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0096383549999999995, 2024-09-17T21:18:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 24 2024-09-17T21:18:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.233810+0000", 2024-09-17T21:18:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.233810+0000", 2024-09-17T21:18:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.233811+0000", 2024-09-17T21:18:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3230000000000001e-06 2024-09-17T21:18:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.233985+0000", 2024-09-17T21:18:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000173609 2024-09-17T21:18:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234013+0000", 2024-09-17T21:18:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8396000000000001e-05 2024-09-17T21:18:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234035+0000", 2024-09-17T21:18:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1756999999999999e-05 2024-09-17T21:18:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.239983+0000", 2024-09-17T21:18:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059477030000000004 2024-09-17T21:18:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.240111+0000", 2024-09-17T21:18:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012860899999999999 2024-09-17T21:18:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.240317+0000", 2024-09-17T21:18:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020541900000000001 2024-09-17T21:18:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.243187+0000", 2024-09-17T21:18:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002870531 2024-09-17T21:18:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.243295+0000", 2024-09-17T21:18:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000107606 2024-09-17T21:18:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.243381+0000", 2024-09-17T21:18:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5995999999999995e-05 2024-09-17T21:18:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.243448+0000", 2024-09-17T21:18:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7405999999999998e-05 2024-09-17T21:18:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.234074+0000", 2024-09-17T21:18:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.834041315, 2024-09-17T21:18:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011808219, 2024-09-17T21:18:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 25 2024-09-17T21:18:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234074+0000", 2024-09-17T21:18:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234074+0000", 2024-09-17T21:18:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234076+0000", 2024-09-17T21:18:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1799999999999999e-06 2024-09-17T21:18:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234094+0000", 2024-09-17T21:18:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8240000000000002e-05 2024-09-17T21:18:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234116+0000", 2024-09-17T21:18:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2296000000000001e-05 2024-09-17T21:18:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.234140+0000", 2024-09-17T21:18:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4204000000000001e-05 2024-09-17T21:18:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.241931+0000", 2024-09-17T21:18:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0077910419999999998 2024-09-17T21:18:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.242167+0000", 2024-09-17T21:18:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000236025 2024-09-17T21:18:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.242544+0000", 2024-09-17T21:18:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037610699999999998 2024-09-17T21:18:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.243799+0000", 2024-09-17T21:18:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001255611 2024-09-17T21:18:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.245610+0000", 2024-09-17T21:18:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001811099 2024-09-17T21:18:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.245758+0000", 2024-09-17T21:18:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148212 2024-09-17T21:18:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.245883+0000", 2024-09-17T21:18:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124203 2024-09-17T21:18:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.335442+0000", 2024-09-17T21:18:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.732674221000003, 2024-09-17T21:18:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0096596089999999996, 2024-09-17T21:18:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 44 2024-09-17T21:18:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335442+0000", 2024-09-17T21:18:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335442+0000", 2024-09-17T21:18:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335443+0000", 2024-09-17T21:18:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.747e-06 2024-09-17T21:18:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335562+0000", 2024-09-17T21:18:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000118656 2024-09-17T21:18:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335592+0000", 2024-09-17T21:18:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9743e-05 2024-09-17T21:18:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335609+0000", 2024-09-17T21:18:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7558999999999999e-05 2024-09-17T21:18:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.340968+0000", 2024-09-17T21:18:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0053589220000000003 2024-09-17T21:18:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.341101+0000", 2024-09-17T21:18:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013255600000000001 2024-09-17T21:18:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.341308+0000", 2024-09-17T21:18:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020729099999999999 2024-09-17T21:18:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.344815+0000", 2024-09-17T21:18:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003506594 2024-09-17T21:18:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.344960+0000", 2024-09-17T21:18:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145105 2024-09-17T21:18:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.345037+0000", 2024-09-17T21:18:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6987e-05 2024-09-17T21:18:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.345101+0000", 2024-09-17T21:18:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4449000000000006e-05 2024-09-17T21:18:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.335647+0000", 2024-09-17T21:18:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.732468683, 2024-09-17T21:18:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012049351, 2024-09-17T21:18:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 45 2024-09-17T21:18:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335647+0000", 2024-09-17T21:18:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335647+0000", 2024-09-17T21:18:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335649+0000", 2024-09-17T21:18:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.624e-06 2024-09-17T21:18:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335680+0000", 2024-09-17T21:18:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0913e-05 2024-09-17T21:18:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335701+0000", 2024-09-17T21:18:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1307000000000001e-05 2024-09-17T21:18:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335718+0000", 2024-09-17T21:18:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6586e-05 2024-09-17T21:18:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.342345+0000", 2024-09-17T21:18:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006627712 2024-09-17T21:18:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.342463+0000", 2024-09-17T21:18:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000117439 2024-09-17T21:18:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.342647+0000", 2024-09-17T21:18:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000184692 2024-09-17T21:18:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.344938+0000", 2024-09-17T21:18:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022909190000000002 2024-09-17T21:18:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.347555+0000", 2024-09-17T21:18:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026162429999999999 2024-09-17T21:18:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.347635+0000", 2024-09-17T21:18:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0253000000000006e-05 2024-09-17T21:18:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.347696+0000", 2024-09-17T21:18:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1662999999999996e-05 2024-09-17T21:18:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.335761+0000", 2024-09-17T21:18:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.732355243999997, 2024-09-17T21:18:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013102346000000001, 2024-09-17T21:18:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 46 2024-09-17T21:18:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335761+0000", 2024-09-17T21:18:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335761+0000", 2024-09-17T21:18:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335762+0000", 2024-09-17T21:18:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:18:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335787+0000", 2024-09-17T21:18:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5360000000000001e-05 2024-09-17T21:18:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335808+0000", 2024-09-17T21:18:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0333e-05 2024-09-17T21:18:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335868+0000", 2024-09-17T21:18:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0463e-05 2024-09-17T21:18:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.343865+0000", 2024-09-17T21:18:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0079966350000000002 2024-09-17T21:18:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.343989+0000", 2024-09-17T21:18:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124449 2024-09-17T21:18:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.344190+0000", 2024-09-17T21:18:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020082499999999999 2024-09-17T21:18:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.347528+0000", 2024-09-17T21:18:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003337998 2024-09-17T21:18:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.348737+0000", 2024-09-17T21:18:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0012084470000000001 2024-09-17T21:18:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.348805+0000", 2024-09-17T21:18:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8306000000000006e-05 2024-09-17T21:18:36.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.348863+0000", 2024-09-17T21:18:36.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8006e-05 2024-09-17T21:18:36.108 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.108 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.108 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.108 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:36.108 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.335903+0000", 2024-09-17T21:18:36.108 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.732212631000003, 2024-09-17T21:18:36.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013146595000000001, 2024-09-17T21:18:36.109 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.109 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.109 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:36.109 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.109 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:36.109 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:36.109 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 47 2024-09-17T21:18:36.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.109 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335903+0000", 2024-09-17T21:18:36.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335903+0000", 2024-09-17T21:18:36.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.335905+0000", 2024-09-17T21:18:36.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.556e-06 2024-09-17T21:18:36.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336023+0000", 2024-09-17T21:18:36.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000118136 2024-09-17T21:18:36.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336050+0000", 2024-09-17T21:18:36.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7472999999999999e-05 2024-09-17T21:18:36.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336066+0000", 2024-09-17T21:18:36.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5934e-05 2024-09-17T21:18:36.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.345186+0000", 2024-09-17T21:18:36.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0091198389999999994 2024-09-17T21:18:36.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.345337+0000", 2024-09-17T21:18:36.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015087600000000001 2024-09-17T21:18:36.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:36.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.345549+0000", 2024-09-17T21:18:36.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021209500000000001 2024-09-17T21:18:36.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:36.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.347760+0000", 2024-09-17T21:18:36.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022113900000000001 2024-09-17T21:18:36.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:36.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.348877+0000", 2024-09-17T21:18:36.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011163340000000001 2024-09-17T21:18:36.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:36.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.348951+0000", 2024-09-17T21:18:36.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4666000000000006e-05 2024-09-17T21:18:36.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.349050+0000", 2024-09-17T21:18:36.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8295999999999995e-05 2024-09-17T21:18:36.114 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.114 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.114 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.114 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:36.114 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.336089+0000", 2024-09-17T21:18:36.114 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.732026898999997, 2024-09-17T21:18:36.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014613607000000001, 2024-09-17T21:18:36.114 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.114 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.114 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:36.115 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.115 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:36.115 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:36.115 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 48 2024-09-17T21:18:36.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.115 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336089+0000", 2024-09-17T21:18:36.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336089+0000", 2024-09-17T21:18:36.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336090+0000", 2024-09-17T21:18:36.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.083e-06 2024-09-17T21:18:36.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336097+0000", 2024-09-17T21:18:36.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0299999999999996e-06 2024-09-17T21:18:36.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336151+0000", 2024-09-17T21:18:36.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4292999999999997e-05 2024-09-17T21:18:36.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336163+0000", 2024-09-17T21:18:36.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.136e-05 2024-09-17T21:18:36.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.346568+0000", 2024-09-17T21:18:36.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010405009999999999 2024-09-17T21:18:36.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.346687+0000", 2024-09-17T21:18:36.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011885899999999999 2024-09-17T21:18:36.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:36.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.346865+0000", 2024-09-17T21:18:36.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017869500000000001 2024-09-17T21:18:36.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:36.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.350554+0000", 2024-09-17T21:18:36.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036889879999999998 2024-09-17T21:18:36.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:36.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.350575+0000", 2024-09-17T21:18:36.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.052e-05 2024-09-17T21:18:36.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:36.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.350645+0000", 2024-09-17T21:18:36.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0506000000000006e-05 2024-09-17T21:18:36.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.350702+0000", 2024-09-17T21:18:36.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7262999999999997e-05 2024-09-17T21:18:36.120 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.120 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.120 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.120 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:36.120 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.336187+0000", 2024-09-17T21:18:36.120 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.73192873, 2024-09-17T21:18:36.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015759305000000001, 2024-09-17T21:18:36.120 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.120 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.120 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:36.120 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.120 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:36.121 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:36.121 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 49 2024-09-17T21:18:36.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.121 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336187+0000", 2024-09-17T21:18:36.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336187+0000", 2024-09-17T21:18:36.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336188+0000", 2024-09-17T21:18:36.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3230000000000001e-06 2024-09-17T21:18:36.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336198+0000", 2024-09-17T21:18:36.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7440000000000002e-06 2024-09-17T21:18:36.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336210+0000", 2024-09-17T21:18:36.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1950000000000001e-05 2024-09-17T21:18:36.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336218+0000", 2024-09-17T21:18:36.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6329999999999996e-06 2024-09-17T21:18:36.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.347858+0000", 2024-09-17T21:18:36.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011640322999999999 2024-09-17T21:18:36.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.347979+0000", 2024-09-17T21:18:36.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012050899999999999 2024-09-17T21:18:36.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:36.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.348185+0000", 2024-09-17T21:18:36.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020600499999999999 2024-09-17T21:18:36.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:36.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.350801+0000", 2024-09-17T21:18:36.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026167830000000001 2024-09-17T21:18:36.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:36.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.351816+0000", 2024-09-17T21:18:36.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001015019 2024-09-17T21:18:36.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:36.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.351886+0000", 2024-09-17T21:18:36.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9952999999999999e-05 2024-09-17T21:18:36.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.351946+0000", 2024-09-17T21:18:36.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0062999999999997e-05 2024-09-17T21:18:36.125 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.125 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.125 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.126 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:36.126 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.336258+0000", 2024-09-17T21:18:36.126 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.731857978000001, 2024-09-17T21:18:36.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016750900999999999, 2024-09-17T21:18:36.126 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.126 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.126 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:36.126 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.126 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:36.126 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:36.126 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 50 2024-09-17T21:18:36.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.127 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336258+0000", 2024-09-17T21:18:36.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336258+0000", 2024-09-17T21:18:36.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336259+0000", 2024-09-17T21:18:36.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.11e-06 2024-09-17T21:18:36.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336369+0000", 2024-09-17T21:18:36.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010995 2024-09-17T21:18:36.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336384+0000", 2024-09-17T21:18:36.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5353e-05 2024-09-17T21:18:36.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336395+0000", 2024-09-17T21:18:36.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0995999999999999e-05 2024-09-17T21:18:36.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.349183+0000", 2024-09-17T21:18:36.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012788152000000001 2024-09-17T21:18:36.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.349310+0000", 2024-09-17T21:18:36.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012668600000000001 2024-09-17T21:18:36.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:36.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.349523+0000", 2024-09-17T21:18:36.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021328499999999999 2024-09-17T21:18:36.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:36.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.352854+0000", 2024-09-17T21:18:36.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033302140000000002 2024-09-17T21:18:36.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:36.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.352877+0000", 2024-09-17T21:18:36.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3195999999999999e-05 2024-09-17T21:18:36.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:36.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.352949+0000", 2024-09-17T21:18:36.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2086000000000001e-05 2024-09-17T21:18:36.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353009+0000", 2024-09-17T21:18:36.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9873000000000001e-05 2024-09-17T21:18:36.131 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.131 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.131 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.132 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:36.132 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.336415+0000", 2024-09-17T21:18:36.132 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.731700408999998, 2024-09-17T21:18:36.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016888507000000001, 2024-09-17T21:18:36.132 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.132 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.132 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:36.132 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.132 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:36.132 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:36.132 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 51 2024-09-17T21:18:36.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.133 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336415+0000", 2024-09-17T21:18:36.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336415+0000", 2024-09-17T21:18:36.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336416+0000", 2024-09-17T21:18:36.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7000000000000003e-07 2024-09-17T21:18:36.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336423+0000", 2024-09-17T21:18:36.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0940000000000002e-06 2024-09-17T21:18:36.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336433+0000", 2024-09-17T21:18:36.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.73e-06 2024-09-17T21:18:36.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336440+0000", 2024-09-17T21:18:36.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.973e-06 2024-09-17T21:18:36.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.350896+0000", 2024-09-17T21:18:36.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014456101000000001 2024-09-17T21:18:36.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.351027+0000", 2024-09-17T21:18:36.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000131063 2024-09-17T21:18:36.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:36.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.351249+0000", 2024-09-17T21:18:36.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000221568 2024-09-17T21:18:36.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:36.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353101+0000", 2024-09-17T21:18:36.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018519020000000001 2024-09-17T21:18:36.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:36.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353150+0000", 2024-09-17T21:18:36.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9607000000000001e-05 2024-09-17T21:18:36.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:36.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353249+0000", 2024-09-17T21:18:36.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8176000000000003e-05 2024-09-17T21:18:36.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353304+0000", 2024-09-17T21:18:36.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5322999999999999e-05 2024-09-17T21:18:36.137 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.137 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.137 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.138 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:36.138 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.336482+0000", 2024-09-17T21:18:36.138 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.731633385999999, 2024-09-17T21:18:36.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017076834999999999, 2024-09-17T21:18:36.138 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.138 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.138 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:36.138 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.138 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:36.138 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:36.138 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 52 2024-09-17T21:18:36.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.138 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336482+0000", 2024-09-17T21:18:36.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336482+0000", 2024-09-17T21:18:36.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336483+0000", 2024-09-17T21:18:36.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1699999999999997e-07 2024-09-17T21:18:36.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336490+0000", 2024-09-17T21:18:36.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5030000000000002e-06 2024-09-17T21:18:36.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336501+0000", 2024-09-17T21:18:36.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0827e-05 2024-09-17T21:18:36.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.336508+0000", 2024-09-17T21:18:36.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4630000000000004e-06 2024-09-17T21:18:36.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.352032+0000", 2024-09-17T21:18:36.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015524263 2024-09-17T21:18:36.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.352175+0000", 2024-09-17T21:18:36.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000143206 2024-09-17T21:18:36.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:36.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.352363+0000", 2024-09-17T21:18:36.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000187579 2024-09-17T21:18:36.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:36.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353377+0000", 2024-09-17T21:18:36.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001013772 2024-09-17T21:18:36.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:36.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353424+0000", 2024-09-17T21:18:36.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7456000000000001e-05 2024-09-17T21:18:36.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:36.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353499+0000", 2024-09-17T21:18:36.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4518999999999996e-05 2024-09-17T21:18:36.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.353559+0000", 2024-09-17T21:18:36.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0430000000000002e-05 2024-09-17T21:18:36.143 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.143 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.143 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.143 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:36.143 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.531642+0000", 2024-09-17T21:18:36.144 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.536473288000003, 2024-09-17T21:18:36.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010095023999999999, 2024-09-17T21:18:36.144 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.144 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.144 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:36.144 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.144 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:36.144 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:36.144 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 73 2024-09-17T21:18:36.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.144 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531642+0000", 2024-09-17T21:18:36.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531642+0000", 2024-09-17T21:18:36.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531643+0000", 2024-09-17T21:18:36.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3099999999999998e-07 2024-09-17T21:18:36.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531647+0000", 2024-09-17T21:18:36.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6370000000000001e-06 2024-09-17T21:18:36.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531652+0000", 2024-09-17T21:18:36.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4790000000000004e-06 2024-09-17T21:18:36.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531656+0000", 2024-09-17T21:18:36.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8530000000000002e-06 2024-09-17T21:18:36.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.537928+0000", 2024-09-17T21:18:36.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062718560000000001 2024-09-17T21:18:36.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.537992+0000", 2024-09-17T21:18:36.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4284999999999999e-05 2024-09-17T21:18:36.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:36.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.538086+0000", 2024-09-17T21:18:36.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4094999999999999e-05 2024-09-17T21:18:36.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:36.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.540707+0000", 2024-09-17T21:18:36.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002621057 2024-09-17T21:18:36.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:36.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541673+0000", 2024-09-17T21:18:36.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00096603399999999995 2024-09-17T21:18:36.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:36.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541709+0000", 2024-09-17T21:18:36.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5790999999999997e-05 2024-09-17T21:18:36.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541738+0000", 2024-09-17T21:18:36.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8405999999999999e-05 2024-09-17T21:18:36.149 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.149 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.149 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.150 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:36.150 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.531665+0000", 2024-09-17T21:18:36.150 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.536450995000003, 2024-09-17T21:18:36.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010231192, 2024-09-17T21:18:36.150 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.150 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.150 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:36.150 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.150 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:36.150 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:36.150 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 74 2024-09-17T21:18:36.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.150 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531665+0000", 2024-09-17T21:18:36.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531665+0000", 2024-09-17T21:18:36.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531665+0000", 2024-09-17T21:18:36.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.34e-07 2024-09-17T21:18:36.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531670+0000", 2024-09-17T21:18:36.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8230000000000002e-06 2024-09-17T21:18:36.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531675+0000", 2024-09-17T21:18:36.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8810000000000004e-06 2024-09-17T21:18:36.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531680+0000", 2024-09-17T21:18:36.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2959999999999998e-06 2024-09-17T21:18:36.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.538749+0000", 2024-09-17T21:18:36.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0070683309999999997 2024-09-17T21:18:36.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.538834+0000", 2024-09-17T21:18:36.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5620999999999999e-05 2024-09-17T21:18:36.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:36.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.538934+0000", 2024-09-17T21:18:36.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9508999999999999e-05 2024-09-17T21:18:36.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:36.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541012+0000", 2024-09-17T21:18:36.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0020783099999999999 2024-09-17T21:18:36.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:36.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541781+0000", 2024-09-17T21:18:36.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00076862700000000003 2024-09-17T21:18:36.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:36.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541835+0000", 2024-09-17T21:18:36.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4283999999999997e-05 2024-09-17T21:18:36.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541896+0000", 2024-09-17T21:18:36.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1075999999999996e-05 2024-09-17T21:18:36.155 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.155 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.155 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.155 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:36.155 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.531687+0000", 2024-09-17T21:18:36.156 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.536428733000001, 2024-09-17T21:18:36.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010899883000000001, 2024-09-17T21:18:36.156 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.156 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.156 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:36.156 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.156 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:36.156 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:36.156 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 75 2024-09-17T21:18:36.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.156 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531687+0000", 2024-09-17T21:18:36.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531687+0000", 2024-09-17T21:18:36.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531687+0000", 2024-09-17T21:18:36.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8500000000000002e-07 2024-09-17T21:18:36.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531748+0000", 2024-09-17T21:18:36.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0588000000000003e-05 2024-09-17T21:18:36.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531756+0000", 2024-09-17T21:18:36.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6399999999999997e-06 2024-09-17T21:18:36.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531760+0000", 2024-09-17T21:18:36.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4870000000000004e-06 2024-09-17T21:18:36.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.539287+0000", 2024-09-17T21:18:36.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0075267379999999998 2024-09-17T21:18:36.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.539353+0000", 2024-09-17T21:18:36.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6577999999999999e-05 2024-09-17T21:18:36.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:36.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.539470+0000", 2024-09-17T21:18:36.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000116662 2024-09-17T21:18:36.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:36.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.541287+0000", 2024-09-17T21:18:36.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018171839999999999 2024-09-17T21:18:36.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:36.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.542494+0000", 2024-09-17T21:18:36.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0012064650000000001 2024-09-17T21:18:36.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:36.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.542533+0000", 2024-09-17T21:18:36.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9196999999999998e-05 2024-09-17T21:18:36.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.542587+0000", 2024-09-17T21:18:36.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3959000000000003e-05 2024-09-17T21:18:36.161 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.161 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.161 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.161 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:36.161 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.531769+0000", 2024-09-17T21:18:36.162 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.536346311999999, 2024-09-17T21:18:36.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011381054, 2024-09-17T21:18:36.162 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.162 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.162 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:36.162 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.162 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:36.162 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:36.162 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 76 2024-09-17T21:18:36.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.162 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531769+0000", 2024-09-17T21:18:36.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531769+0000", 2024-09-17T21:18:36.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531770+0000", 2024-09-17T21:18:36.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8100000000000003e-07 2024-09-17T21:18:36.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531782+0000", 2024-09-17T21:18:36.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1832e-05 2024-09-17T21:18:36.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531787+0000", 2024-09-17T21:18:36.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3870000000000003e-06 2024-09-17T21:18:36.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.531797+0000", 2024-09-17T21:18:36.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0036e-05 2024-09-17T21:18:36.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.540161+0000", 2024-09-17T21:18:36.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083641919999999995 2024-09-17T21:18:36.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.540220+0000", 2024-09-17T21:18:36.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8829000000000002e-05 2024-09-17T21:18:36.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:36.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.540312+0000", 2024-09-17T21:18:36.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1983000000000003e-05 2024-09-17T21:18:36.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:36.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.542152+0000", 2024-09-17T21:18:36.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018396840000000001 2024-09-17T21:18:36.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:36.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.543063+0000", 2024-09-17T21:18:36.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00091107600000000003 2024-09-17T21:18:36.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:36.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.543102+0000", 2024-09-17T21:18:36.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8767999999999999e-05 2024-09-17T21:18:36.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.543151+0000", 2024-09-17T21:18:36.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8785999999999997e-05 2024-09-17T21:18:36.167 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.167 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.167 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.168 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:18:36.168 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:18:36.168 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.474186070999998, 2024-09-17T21:18:36.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:18:36.168 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.168 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.168 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:36.168 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.168 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:18:36.168 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:36.168 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:18:36.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.168 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:18:36.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:18:36.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:18:36.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:18:36.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:18:36.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:18:36.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:18:36.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:18:36.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:18:36.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:18:36.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:18:36.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:18:36.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:36.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:18:36.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:18:36.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:18:36.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:18:36.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:36.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:18:36.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:18:36.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:18:36.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:18:36.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:18:36.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:18:36.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:18:36.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:18:36.173 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.173 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.173 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.174 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:18:36.174 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:18:36.174 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.474088759000001, 2024-09-17T21:18:36.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:18:36.174 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.174 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.174 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:36.174 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.174 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:36.174 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:36.174 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:18:36.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.175 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:18:36.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:18:36.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:18:36.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:18:36.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:18:36.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:18:36.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:18:36.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:18:36.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:18:36.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:18:36.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:18:36.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:18:36.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:36.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:18:36.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:18:36.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:18:36.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:18:36.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:36.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:18:36.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:18:36.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:18:36.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:18:36.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:18:36.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:18:36.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:18:36.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:18:36.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:18:36.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:18:36.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:18:36.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:18:36.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:18:36.180 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.180 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.180 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.180 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:18:36.180 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:18:36.180 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.472001958, 2024-09-17T21:18:36.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:18:36.181 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.181 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.181 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:36.181 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.181 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:18:36.181 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:36.181 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:18:36.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.181 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:18:36.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:18:36.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:18:36.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:18:36.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:18:36.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:18:36.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:18:36.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:18:36.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:18:36.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:18:36.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:18:36.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:18:36.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:18:36.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:18:36.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:18:36.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:18:36.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:18:36.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:36.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:18:36.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:18:36.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:18:36.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:18:36.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:18:36.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:18:36.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:36.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:18:36.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:18:36.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:36.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:18:36.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:18:36.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:36.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:18:36.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:18:36.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:36.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:18:36.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:18:36.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:18:36.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:18:36.188 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.188 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.188 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.188 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:575 3.15 3:ac921cb1:::rbd_data.1152b51cb396.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:36.188 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.213766+0000", 2024-09-17T21:18:36.188 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.8543493800000004, 2024-09-17T21:18:36.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015902976999999999, 2024-09-17T21:18:36.188 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.188 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.188 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:36.188 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.188 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:36.189 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:36.189 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 575 2024-09-17T21:18:36.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.189 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.213766+0000", 2024-09-17T21:18:36.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.213766+0000", 2024-09-17T21:18:36.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.213768+0000", 2024-09-17T21:18:36.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1149999999999999e-06 2024-09-17T21:18:36.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.216434+0000", 2024-09-17T21:18:36.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026669580000000001 2024-09-17T21:18:36.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.216450+0000", 2024-09-17T21:18:36.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5122e-05 2024-09-17T21:18:36.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.216458+0000", 2024-09-17T21:18:36.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7949999999999996e-06 2024-09-17T21:18:36.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.216604+0000", 2024-09-17T21:18:36.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145313 2024-09-17T21:18:36.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.216940+0000", 2024-09-17T21:18:36.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033653099999999999 2024-09-17T21:18:36.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:36.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.217594+0000", 2024-09-17T21:18:36.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00065363499999999996 2024-09-17T21:18:36.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:36.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.224088+0000", 2024-09-17T21:18:36.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0064940270000000003 2024-09-17T21:18:36.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:36.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.229575+0000", 2024-09-17T21:18:36.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0054871620000000003 2024-09-17T21:18:36.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:36.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.229635+0000", 2024-09-17T21:18:36.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9781000000000002e-05 2024-09-17T21:18:36.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.229669+0000", 2024-09-17T21:18:36.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4538e-05 2024-09-17T21:18:36.194 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.194 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.194 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.194 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.194 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.194 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:36.195 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:18:36.196 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:36.190+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:36.201 INFO:teuthology.orchestra.run.smithi184.stdout:737 read 0x7462e00 thru 0x7466c4e (0x3e4f bytes) 2024-09-17T21:18:36.203 INFO:teuthology.orchestra.run.smithi184.stdout:739 read 0x17aea00 thru 0x17b96df (0xace0 bytes) 2024-09-17T21:18:36.206 INFO:teuthology.orchestra.run.smithi184.stdout:740 write 0x4a7800 thru 0x4aa35f (0x2b60 bytes) 2024-09-17T21:18:36.209 INFO:teuthology.orchestra.run.smithi184.stdout:741 write 0xf7bc00 thru 0xf8266d (0x6a6e bytes) 2024-09-17T21:18:36.213 INFO:teuthology.orchestra.run.smithi184.stdout:742 punch from 0x81dd400 to 0x81e5000, (0x7c00 bytes) 2024-09-17T21:18:36.296 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:36.292+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:36.334 INFO:teuthology.orchestra.run.smithi184.stdout:744 read 0x9eda200 thru 0x9ee9c3a (0xfa3b bytes) 2024-09-17T21:18:36.335 INFO:teuthology.orchestra.run.smithi184.stdout:745 punch from 0x8420a00 to 0x8429000, (0x8600 bytes) 2024-09-17T21:18:36.339 INFO:teuthology.orchestra.run.smithi184.stdout:746 read 0xa3ce800 thru 0xa3d7914 (0x9115 bytes) 2024-09-17T21:18:36.340 INFO:teuthology.orchestra.run.smithi184.stdout:749 clone 8 order 24 su 65536 sc 12 2024-09-17T21:18:36.384 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:18:36.385 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:36.397 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:36.392+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:36.398 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:18:36.498 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:36.494+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:36.520 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:36.520 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:36.520 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:18:36.520 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:18:36.520 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:36.534 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:18:36.626 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:36.622+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:36.700 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:36.694+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:36.705 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:36.705 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:18:36.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:18:36.705 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:36.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.705 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:478 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 720896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:36.706 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:18:36.706 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.469248261, 2024-09-17T21:18:36.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016845300000000001, 2024-09-17T21:18:36.706 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.706 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.706 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:36.706 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.706 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:36.706 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:36.706 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 478 2024-09-17T21:18:36.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.706 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:18:36.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:18:36.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233785+0000", 2024-09-17T21:18:36.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.116e-06 2024-09-17T21:18:36.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233803+0000", 2024-09-17T21:18:36.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7932999999999999e-05 2024-09-17T21:18:36.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233822+0000", 2024-09-17T21:18:36.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9276999999999999e-05 2024-09-17T21:18:36.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233836+0000", 2024-09-17T21:18:36.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4053000000000001e-05 2024-09-17T21:18:36.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.248579+0000", 2024-09-17T21:18:36.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014743529 2024-09-17T21:18:36.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.248795+0000", 2024-09-17T21:18:36.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021535200000000001 2024-09-17T21:18:36.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250629+0000", 2024-09-17T21:18:36.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018340400000000001 2024-09-17T21:18:36.710 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.710 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.710 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.710 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:479 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 851968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:36.710 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:18:36.710 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.46915717499999998, 2024-09-17T21:18:36.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017383900000000001, 2024-09-17T21:18:36.710 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.710 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.711 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:36.711 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.711 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:36.711 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:36.711 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 479 2024-09-17T21:18:36.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.711 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:18:36.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:18:36.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233876+0000", 2024-09-17T21:18:36.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2160000000000001e-06 2024-09-17T21:18:36.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233883+0000", 2024-09-17T21:18:36.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9199999999999998e-06 2024-09-17T21:18:36.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233900+0000", 2024-09-17T21:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7034e-05 2024-09-17T21:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233913+0000", 2024-09-17T21:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159000000000001e-05 2024-09-17T21:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250712+0000", 2024-09-17T21:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016798644000000001 2024-09-17T21:18:36.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250913+0000", 2024-09-17T21:18:36.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020127900000000001 2024-09-17T21:18:36.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251259+0000", 2024-09-17T21:18:36.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00034564799999999999 2024-09-17T21:18:36.714 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.714 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.714 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.715 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:480 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:36.715 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:18:36.715 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.46906432999999997, 2024-09-17T21:18:36.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017902482000000001, 2024-09-17T21:18:36.715 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.715 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.715 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:36.715 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.715 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:36.715 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:36.715 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 480 2024-09-17T21:18:36.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.715 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:18:36.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:18:36.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233969+0000", 2024-09-17T21:18:36.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.454e-06 2024-09-17T21:18:36.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233981+0000", 2024-09-17T21:18:36.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1593000000000001e-05 2024-09-17T21:18:36.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233999+0000", 2024-09-17T21:18:36.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.806e-05 2024-09-17T21:18:36.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234012+0000", 2024-09-17T21:18:36.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:18:36.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251339+0000", 2024-09-17T21:18:36.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017326352 2024-09-17T21:18:36.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251534+0000", 2024-09-17T21:18:36.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000195842 2024-09-17T21:18:36.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251870+0000", 2024-09-17T21:18:36.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033551400000000003 2024-09-17T21:18:36.719 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.719 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.719 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.719 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:481 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1114112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:36.719 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:18:36.719 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.46899596300000002, 2024-09-17T21:18:36.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018442621999999999, 2024-09-17T21:18:36.719 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.719 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.719 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:36.720 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.720 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:36.720 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:36.720 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 481 2024-09-17T21:18:36.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.720 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:18:36.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:18:36.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234037+0000", 2024-09-17T21:18:36.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:18:36.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234043+0000", 2024-09-17T21:18:36.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.49e-06 2024-09-17T21:18:36.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234061+0000", 2024-09-17T21:18:36.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8416000000000001e-05 2024-09-17T21:18:36.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234076+0000", 2024-09-17T21:18:36.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.456e-05 2024-09-17T21:18:36.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251948+0000", 2024-09-17T21:18:36.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017872774000000001 2024-09-17T21:18:36.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252144+0000", 2024-09-17T21:18:36.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019517300000000001 2024-09-17T21:18:36.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252478+0000", 2024-09-17T21:18:36.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033482899999999998 2024-09-17T21:18:36.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.723 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.724 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:482 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:36.724 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:18:36.724 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.46893741100000003, 2024-09-17T21:18:36.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018988026000000002, 2024-09-17T21:18:36.724 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.724 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.724 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:36.724 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.724 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:36.724 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:36.724 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 482 2024-09-17T21:18:36.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.724 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:18:36.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:18:36.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234096+0000", 2024-09-17T21:18:36.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2139999999999999e-06 2024-09-17T21:18:36.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234101+0000", 2024-09-17T21:18:36.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7100000000000004e-06 2024-09-17T21:18:36.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234118+0000", 2024-09-17T21:18:36.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6486000000000001e-05 2024-09-17T21:18:36.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234131+0000", 2024-09-17T21:18:36.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2824e-05 2024-09-17T21:18:36.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252552+0000", 2024-09-17T21:18:36.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018421549999999998 2024-09-17T21:18:36.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252746+0000", 2024-09-17T21:18:36.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019424399999999999 2024-09-17T21:18:36.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253082+0000", 2024-09-17T21:18:36.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000335998 2024-09-17T21:18:36.728 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.728 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.728 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.728 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:483 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:36.728 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:18:36.728 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.46888279100000002, 2024-09-17T21:18:36.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019462882000000001, 2024-09-17T21:18:36.728 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.728 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.728 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:36.728 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:36.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:36.729 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 483 2024-09-17T21:18:36.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.729 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:18:36.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:18:36.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234150+0000", 2024-09-17T21:18:36.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.127e-06 2024-09-17T21:18:36.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234155+0000", 2024-09-17T21:18:36.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:18:36.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234171+0000", 2024-09-17T21:18:36.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6160000000000001e-05 2024-09-17T21:18:36.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234185+0000", 2024-09-17T21:18:36.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3370000000000001e-05 2024-09-17T21:18:36.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253158+0000", 2024-09-17T21:18:36.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018972968999999999 2024-09-17T21:18:36.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253349+0000", 2024-09-17T21:18:36.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000190733 2024-09-17T21:18:36.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253612+0000", 2024-09-17T21:18:36.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026337000000000002 2024-09-17T21:18:36.732 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.732 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.732 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.732 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:484 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:36.732 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:18:36.733 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.46882876499999998, 2024-09-17T21:18:36.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020029261999999999, 2024-09-17T21:18:36.733 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.733 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.733 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:36.733 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.733 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:36.733 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:36.733 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 484 2024-09-17T21:18:36.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.733 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:18:36.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:18:36.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234205+0000", 2024-09-17T21:18:36.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6470000000000001e-06 2024-09-17T21:18:36.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234210+0000", 2024-09-17T21:18:36.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:18:36.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234227+0000", 2024-09-17T21:18:36.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.73e-05 2024-09-17T21:18:36.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234241+0000", 2024-09-17T21:18:36.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3757000000000001e-05 2024-09-17T21:18:36.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253690+0000", 2024-09-17T21:18:36.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019449368000000002 2024-09-17T21:18:36.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253898+0000", 2024-09-17T21:18:36.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000207161 2024-09-17T21:18:36.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254232+0000", 2024-09-17T21:18:36.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033474299999999998 2024-09-17T21:18:36.737 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.737 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.737 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.737 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:485 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1638400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:36.737 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:18:36.737 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.46877153799999999, 2024-09-17T21:18:36.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020553914, 2024-09-17T21:18:36.737 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.737 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.737 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:36.737 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.737 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:36.738 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:36.738 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 485 2024-09-17T21:18:36.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.738 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:18:36.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:18:36.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234261+0000", 2024-09-17T21:18:36.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.136e-06 2024-09-17T21:18:36.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234266+0000", 2024-09-17T21:18:36.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0669999999999999e-06 2024-09-17T21:18:36.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234284+0000", 2024-09-17T21:18:36.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7839999999999999e-05 2024-09-17T21:18:36.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234298+0000", 2024-09-17T21:18:36.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3852999999999999e-05 2024-09-17T21:18:36.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254311+0000", 2024-09-17T21:18:36.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020012973 2024-09-17T21:18:36.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254552+0000", 2024-09-17T21:18:36.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024116400000000001 2024-09-17T21:18:36.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254814+0000", 2024-09-17T21:18:36.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000261881 2024-09-17T21:18:36.741 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.741 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.741 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.741 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:486 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:36.741 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:18:36.741 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.468715149, 2024-09-17T21:18:36.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021010240999999999, 2024-09-17T21:18:36.742 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.742 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.742 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:36.742 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.742 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:36.742 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:36.742 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 486 2024-09-17T21:18:36.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.742 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:18:36.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:18:36.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234318+0000", 2024-09-17T21:18:36.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.37e-06 2024-09-17T21:18:36.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234324+0000", 2024-09-17T21:18:36.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:18:36.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234341+0000", 2024-09-17T21:18:36.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7127000000000001e-05 2024-09-17T21:18:36.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234354+0000", 2024-09-17T21:18:36.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3210000000000001e-05 2024-09-17T21:18:36.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254896+0000", 2024-09-17T21:18:36.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020541835000000001 2024-09-17T21:18:36.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255092+0000", 2024-09-17T21:18:36.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019634700000000001 2024-09-17T21:18:36.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255327+0000", 2024-09-17T21:18:36.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023475899999999999 2024-09-17T21:18:36.745 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.745 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.746 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.746 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:487 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:36.746 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:18:36.746 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.46865895299999999, 2024-09-17T21:18:36.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021453911999999999, 2024-09-17T21:18:36.746 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.746 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.746 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:36.746 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.746 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:36.746 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:36.746 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 487 2024-09-17T21:18:36.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.747 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:18:36.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:18:36.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234374+0000", 2024-09-17T21:18:36.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3039999999999999e-06 2024-09-17T21:18:36.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234380+0000", 2024-09-17T21:18:36.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:18:36.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234397+0000", 2024-09-17T21:18:36.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7232999999999999e-05 2024-09-17T21:18:36.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234411+0000", 2024-09-17T21:18:36.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3784000000000001e-05 2024-09-17T21:18:36.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255406+0000", 2024-09-17T21:18:36.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020995204 2024-09-17T21:18:36.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255595+0000", 2024-09-17T21:18:36.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018903299999999999 2024-09-17T21:18:36.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255827+0000", 2024-09-17T21:18:36.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023190099999999999 2024-09-17T21:18:36.750 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.750 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.750 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.750 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:488 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:36.750 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:18:36.750 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.46860327600000001, 2024-09-17T21:18:36.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026887966999999999, 2024-09-17T21:18:36.750 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.750 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.751 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:36.751 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.751 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:36.751 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:36.751 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 488 2024-09-17T21:18:36.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.751 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:18:36.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:18:36.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234430+0000", 2024-09-17T21:18:36.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1829999999999999e-06 2024-09-17T21:18:36.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234435+0000", 2024-09-17T21:18:36.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9869999999999999e-06 2024-09-17T21:18:36.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234451+0000", 2024-09-17T21:18:36.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6458999999999999e-05 2024-09-17T21:18:36.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234465+0000", 2024-09-17T21:18:36.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3730000000000001e-05 2024-09-17T21:18:36.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255908+0000", 2024-09-17T21:18:36.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021443413000000001 2024-09-17T21:18:36.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256098+0000", 2024-09-17T21:18:36.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018961 2024-09-17T21:18:36.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261317+0000", 2024-09-17T21:18:36.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052185850000000004 2024-09-17T21:18:36.754 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.754 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.754 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.755 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:489 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2162688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:36.755 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:18:36.755 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.46854743700000001, 2024-09-17T21:18:36.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022632815000000001, 2024-09-17T21:18:36.755 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.755 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.755 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:36.755 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.755 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:36.755 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:36.755 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 489 2024-09-17T21:18:36.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.755 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:18:36.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:18:36.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234486+0000", 2024-09-17T21:18:36.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5099999999999999e-06 2024-09-17T21:18:36.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234491+0000", 2024-09-17T21:18:36.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.147e-06 2024-09-17T21:18:36.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234508+0000", 2024-09-17T21:18:36.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7405999999999999e-05 2024-09-17T21:18:36.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234522+0000", 2024-09-17T21:18:36.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3677e-05 2024-09-17T21:18:36.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256536+0000", 2024-09-17T21:18:36.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022013464999999999 2024-09-17T21:18:36.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256752+0000", 2024-09-17T21:18:36.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021637800000000001 2024-09-17T21:18:36.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257117+0000", 2024-09-17T21:18:36.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000365232 2024-09-17T21:18:36.759 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.759 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.759 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.759 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:490 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2293760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:36.759 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:18:36.759 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.46849264699999998, 2024-09-17T21:18:36.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023146462999999999, 2024-09-17T21:18:36.759 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.759 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.759 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:36.760 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.760 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:36.760 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:36.760 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 490 2024-09-17T21:18:36.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.760 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:18:36.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:18:36.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234541+0000", 2024-09-17T21:18:36.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4869999999999999e-06 2024-09-17T21:18:36.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234559+0000", 2024-09-17T21:18:36.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8105999999999999e-05 2024-09-17T21:18:36.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234577+0000", 2024-09-17T21:18:36.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.836e-05 2024-09-17T21:18:36.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234592+0000", 2024-09-17T21:18:36.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4766000000000001e-05 2024-09-17T21:18:36.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257187+0000", 2024-09-17T21:18:36.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022594725 2024-09-17T21:18:36.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257374+0000", 2024-09-17T21:18:36.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018767299999999999 2024-09-17T21:18:36.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257686+0000", 2024-09-17T21:18:36.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000311346 2024-09-17T21:18:36.763 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.763 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.763 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.764 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:491 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:36.764 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:18:36.764 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.46842003799999998, 2024-09-17T21:18:36.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023696872000000001, 2024-09-17T21:18:36.764 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.764 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.764 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:36.764 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.764 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:36.764 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:36.764 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 491 2024-09-17T21:18:36.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.764 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:18:36.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:18:36.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234613+0000", 2024-09-17T21:18:36.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-06 2024-09-17T21:18:36.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234618+0000", 2024-09-17T21:18:36.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3839999999999996e-06 2024-09-17T21:18:36.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234635+0000", 2024-09-17T21:18:36.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.685e-05 2024-09-17T21:18:36.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234648+0000", 2024-09-17T21:18:36.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.313e-05 2024-09-17T21:18:36.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257780+0000", 2024-09-17T21:18:36.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023131946 2024-09-17T21:18:36.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257985+0000", 2024-09-17T21:18:36.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020468400000000001 2024-09-17T21:18:36.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258309+0000", 2024-09-17T21:18:36.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032357499999999999 2024-09-17T21:18:36.768 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.768 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.768 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.768 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:492 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:36.768 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:18:36.768 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.46836445100000001, 2024-09-17T21:18:36.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024258994999999998, 2024-09-17T21:18:36.768 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.768 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.768 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:36.768 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.769 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:36.769 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:36.769 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 492 2024-09-17T21:18:36.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.769 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:18:36.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:18:36.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234669+0000", 2024-09-17T21:18:36.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.313e-06 2024-09-17T21:18:36.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234674+0000", 2024-09-17T21:18:36.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1599999999999997e-06 2024-09-17T21:18:36.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234690+0000", 2024-09-17T21:18:36.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.666e-05 2024-09-17T21:18:36.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234704+0000", 2024-09-17T21:18:36.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3043e-05 2024-09-17T21:18:36.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258397+0000", 2024-09-17T21:18:36.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023693204999999998 2024-09-17T21:18:36.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258593+0000", 2024-09-17T21:18:36.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019578999999999999 2024-09-17T21:18:36.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258926+0000", 2024-09-17T21:18:36.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033382399999999998 2024-09-17T21:18:36.772 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.772 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.772 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.772 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:493 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:36.773 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:18:36.773 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.46830798600000001, 2024-09-17T21:18:36.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024793461999999999, 2024-09-17T21:18:36.773 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.773 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.773 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:36.773 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.773 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:36.773 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:36.773 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 493 2024-09-17T21:18:36.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.773 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:18:36.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:18:36.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234725+0000", 2024-09-17T21:18:36.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.24e-06 2024-09-17T21:18:36.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234731+0000", 2024-09-17T21:18:36.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8939999999999998e-06 2024-09-17T21:18:36.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234749+0000", 2024-09-17T21:18:36.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7863000000000002e-05 2024-09-17T21:18:36.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234762+0000", 2024-09-17T21:18:36.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3322999999999999e-05 2024-09-17T21:18:36.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259022+0000", 2024-09-17T21:18:36.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024259691999999999 2024-09-17T21:18:36.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259259+0000", 2024-09-17T21:18:36.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023669199999999999 2024-09-17T21:18:36.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259517+0000", 2024-09-17T21:18:36.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025875799999999999 2024-09-17T21:18:36.776 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.776 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.777 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.777 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:494 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:36.777 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:18:36.777 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.46825028600000002, 2024-09-17T21:18:36.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025251952000000001, 2024-09-17T21:18:36.777 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.777 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.777 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:36.777 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.777 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:36.777 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:36.777 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 494 2024-09-17T21:18:36.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.778 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:18:36.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:18:36.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234783+0000", 2024-09-17T21:18:36.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2440000000000001e-06 2024-09-17T21:18:36.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234789+0000", 2024-09-17T21:18:36.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:18:36.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234806+0000", 2024-09-17T21:18:36.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7450000000000001e-05 2024-09-17T21:18:36.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234819+0000", 2024-09-17T21:18:36.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3326000000000001e-05 2024-09-17T21:18:36.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259585+0000", 2024-09-17T21:18:36.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024765579999999999 2024-09-17T21:18:36.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259787+0000", 2024-09-17T21:18:36.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202444 2024-09-17T21:18:36.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260033+0000", 2024-09-17T21:18:36.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024611499999999998 2024-09-17T21:18:36.781 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.781 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.781 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.781 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:495 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:36.781 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:18:36.781 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.46819492299999999, 2024-09-17T21:18:36.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025776342000000001, 2024-09-17T21:18:36.782 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.782 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.782 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:36.782 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.782 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:36.782 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:36.782 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 495 2024-09-17T21:18:36.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.782 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:18:36.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:18:36.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234838+0000", 2024-09-17T21:18:36.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2130000000000001e-06 2024-09-17T21:18:36.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234843+0000", 2024-09-17T21:18:36.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9899999999999997e-06 2024-09-17T21:18:36.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234860+0000", 2024-09-17T21:18:36.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6973000000000002e-05 2024-09-17T21:18:36.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234873+0000", 2024-09-17T21:18:36.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3179999999999999e-05 2024-09-17T21:18:36.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260135+0000", 2024-09-17T21:18:36.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025262118 2024-09-17T21:18:36.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260364+0000", 2024-09-17T21:18:36.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022830699999999999 2024-09-17T21:18:36.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260613+0000", 2024-09-17T21:18:36.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024956099999999998 2024-09-17T21:18:36.785 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.785 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.785 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.786 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:496 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:36.786 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:18:36.786 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.468141643, 2024-09-17T21:18:36.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026376773999999999, 2024-09-17T21:18:36.786 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.786 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.786 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:36.786 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.786 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:36.786 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:36.786 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 496 2024-09-17T21:18:36.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.786 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:18:36.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:18:36.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234891+0000", 2024-09-17T21:18:36.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1230000000000001e-06 2024-09-17T21:18:36.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234896+0000", 2024-09-17T21:18:36.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9599999999999999e-06 2024-09-17T21:18:36.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234913+0000", 2024-09-17T21:18:36.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6747000000000001e-05 2024-09-17T21:18:36.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234926+0000", 2024-09-17T21:18:36.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-05 2024-09-17T21:18:36.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260675+0000", 2024-09-17T21:18:36.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025748979000000002 2024-09-17T21:18:36.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260883+0000", 2024-09-17T21:18:36.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020848399999999999 2024-09-17T21:18:36.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261267+0000", 2024-09-17T21:18:36.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038345099999999998 2024-09-17T21:18:36.790 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.790 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.790 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.790 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:682 3.9 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2455552~12288,write 8126464~10240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:36.790 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:18:36.790 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.434204592, 2024-09-17T21:18:36.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018447261999999999, 2024-09-17T21:18:36.790 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.791 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.791 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:36.791 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.791 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:36.791 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:36.791 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 682 2024-09-17T21:18:36.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.791 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:18:36.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:18:36.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268830+0000", 2024-09-17T21:18:36.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6419999999999999e-06 2024-09-17T21:18:36.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271673+0000", 2024-09-17T21:18:36.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028432610000000001 2024-09-17T21:18:36.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271700+0000", 2024-09-17T21:18:36.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7200999999999999e-05 2024-09-17T21:18:36.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271715+0000", 2024-09-17T21:18:36.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5145e-05 2024-09-17T21:18:36.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271785+0000", 2024-09-17T21:18:36.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9519999999999998e-05 2024-09-17T21:18:36.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271918+0000", 2024-09-17T21:18:36.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013330899999999999 2024-09-17T21:18:36.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:36.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.272449+0000", 2024-09-17T21:18:36.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00053051200000000002 2024-09-17T21:18:36.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:36.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.278822+0000", 2024-09-17T21:18:36.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0063731619999999999 2024-09-17T21:18:36.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:36.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287167+0000", 2024-09-17T21:18:36.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083452149999999996 2024-09-17T21:18:36.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:36.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287213+0000", 2024-09-17T21:18:36.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5972e-05 2024-09-17T21:18:36.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287274+0000", 2024-09-17T21:18:36.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1322999999999995e-05 2024-09-17T21:18:36.796 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.796 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.796 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.796 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.796 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.796 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:36.796 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:18:36.797 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:18:36.798 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:18:36.842 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:36.838+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:36.901 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:36.896+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:36.929 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone7 from 0xb036800 (overlap 0xb036800) to 0x467c200 2024-09-17T21:18:36.956 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:18:36.956 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:18:36.972 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:18:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:18:36.991 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:36.991 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:36.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.991 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:1680 3.c 3:37eb6dd1:::rbd_data.11527824718.0000000000000007:9 [sparse-read 10944512~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T21:18:36.991 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.982984+0000", 2024-09-17T21:18:36.991 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.005429026, 2024-09-17T21:18:36.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055056519999999998, 2024-09-17T21:18:36.991 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.991 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.991 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:36.992 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.992 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:36.992 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:36.992 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1680 2024-09-17T21:18:36.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.992 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.992 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.992 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.982984+0000", 2024-09-17T21:18:36.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.982984+0000", 2024-09-17T21:18:36.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.982985+0000", 2024-09-17T21:18:36.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2100000000000001e-06 2024-09-17T21:18:36.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.982989+0000", 2024-09-17T21:18:36.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4870000000000004e-06 2024-09-17T21:18:36.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983005+0000", 2024-09-17T21:18:36.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5130000000000001e-05 2024-09-17T21:18:36.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983016+0000", 2024-09-17T21:18:36.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1294999999999999e-05 2024-09-17T21:18:36.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.988368+0000", 2024-09-17T21:18:36.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0053521690000000004 2024-09-17T21:18:36.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:36.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.988415+0000", 2024-09-17T21:18:36.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6582000000000002e-05 2024-09-17T21:18:36.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:36.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.988491+0000", 2024-09-17T21:18:36.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6426000000000003e-05 2024-09-17T21:18:36.995 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.996 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:36.996 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:36.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.996 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:1681 3.c 3:37eb6dd1:::rbd_data.11527824718.0000000000000007:9 [sparse-read 11075584~0 out=8b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T21:18:36.996 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.983031+0000", 2024-09-17T21:18:36.996 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0053819990000000002, 2024-09-17T21:18:36.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0056896830000000001, 2024-09-17T21:18:36.996 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:36.996 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:36.996 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:36.996 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:36.996 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:36.997 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:36.997 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1681 2024-09-17T21:18:36.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.997 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:36.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:36.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983031+0000", 2024-09-17T21:18:36.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:36.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983031+0000", 2024-09-17T21:18:36.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:36.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:36.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983032+0000", 2024-09-17T21:18:36.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.048e-06 2024-09-17T21:18:36.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:36.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983036+0000", 2024-09-17T21:18:36.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2270000000000004e-06 2024-09-17T21:18:36.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:36.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983051+0000", 2024-09-17T21:18:36.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4962e-05 2024-09-17T21:18:36.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:36.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983062+0000", 2024-09-17T21:18:36.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0698e-05 2024-09-17T21:18:36.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:36.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:36.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:36.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.988527+0000", 2024-09-17T21:18:36.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0054656669999999996 2024-09-17T21:18:37.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:37.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.988636+0000", 2024-09-17T21:18:37.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000108898 2024-09-17T21:18:37.000 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:37.000 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:37.000 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:37.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.000 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:1682 3.c 3:37eb6dd1:::rbd_data.11527824718.0000000000000007:9 [sparse-read 11206656~0 out=8b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T21:18:37.000 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.983077+0000", 2024-09-17T21:18:37.000 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0053359269999999999, 2024-09-17T21:18:37.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0058085569999999998, 2024-09-17T21:18:37.001 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:37.001 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:37.001 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:37.001 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:37.001 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:37.001 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:37.001 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1682 2024-09-17T21:18:37.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.001 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:37.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:37.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983077+0000", 2024-09-17T21:18:37.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:37.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:37.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983077+0000", 2024-09-17T21:18:37.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:37.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:37.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983078+0000", 2024-09-17T21:18:37.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.08e-06 2024-09-17T21:18:37.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:37.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983082+0000", 2024-09-17T21:18:37.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4259999999999996e-06 2024-09-17T21:18:37.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:37.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983097+0000", 2024-09-17T21:18:37.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4549000000000001e-05 2024-09-17T21:18:37.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:37.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983108+0000", 2024-09-17T21:18:37.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0781999999999999e-05 2024-09-17T21:18:37.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:37.004 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.988782+0000", 2024-09-17T21:18:37.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0056745179999999999 2024-09-17T21:18:37.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:37.004 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.988828+0000", 2024-09-17T21:18:37.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5915999999999999e-05 2024-09-17T21:18:37.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:37.004 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.988885+0000", 2024-09-17T21:18:37.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7286e-05 2024-09-17T21:18:37.005 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:37.005 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:37.005 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:37.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.005 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:1683 3.c 3:37eb6dd1:::rbd_data.11527824718.0000000000000007:9 [sparse-read 11337728~0 out=8b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T21:18:37.005 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.983123+0000", 2024-09-17T21:18:37.005 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0052897789999999997, 2024-09-17T21:18:37.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0060183160000000001, 2024-09-17T21:18:37.005 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:37.005 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:37.005 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:37.005 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:37.005 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:37.006 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:37.006 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1683 2024-09-17T21:18:37.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.006 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:37.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:37.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983123+0000", 2024-09-17T21:18:37.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:37.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:37.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983123+0000", 2024-09-17T21:18:37.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:37.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:37.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983124+0000", 2024-09-17T21:18:37.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.217e-06 2024-09-17T21:18:37.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:37.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983129+0000", 2024-09-17T21:18:37.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.33e-06 2024-09-17T21:18:37.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:37.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983143+0000", 2024-09-17T21:18:37.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4792e-05 2024-09-17T21:18:37.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:37.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983157+0000", 2024-09-17T21:18:37.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.323e-05 2024-09-17T21:18:37.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:37.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.988922+0000", 2024-09-17T21:18:37.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057659260000000002 2024-09-17T21:18:37.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:37.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.989026+0000", 2024-09-17T21:18:37.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103725 2024-09-17T21:18:37.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:37.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.989154+0000", 2024-09-17T21:18:37.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012769300000000001 2024-09-17T21:18:37.009 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:37.009 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:37.009 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:37.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.009 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:1684 3.c 3:37eb6dd1:::rbd_data.11527824718.0000000000000007:9 [sparse-read 11468800~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T21:18:37.010 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.983173+0000", 2024-09-17T21:18:37.010 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0052394, 2024-09-17T21:18:37.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0061872189999999999, 2024-09-17T21:18:37.010 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:37.010 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:37.010 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:37.010 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:37.010 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:37.010 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:37.010 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1684 2024-09-17T21:18:37.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.010 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:37.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:37.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983173+0000", 2024-09-17T21:18:37.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:37.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:37.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983173+0000", 2024-09-17T21:18:37.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:37.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:37.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983175+0000", 2024-09-17T21:18:37.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.218e-06 2024-09-17T21:18:37.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:37.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983179+0000", 2024-09-17T21:18:37.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3390000000000003e-06 2024-09-17T21:18:37.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:37.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983196+0000", 2024-09-17T21:18:37.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6738000000000001e-05 2024-09-17T21:18:37.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:37.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983207+0000", 2024-09-17T21:18:37.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1572e-05 2024-09-17T21:18:37.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:37.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.989184+0000", 2024-09-17T21:18:37.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059765670000000003 2024-09-17T21:18:37.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:37.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.989288+0000", 2024-09-17T21:18:37.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104112 2024-09-17T21:18:37.013 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:37.013 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:37.013 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:37.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.014 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:1685 3.c 3:37eb6dd1:::rbd_data.11527824718.0000000000000007:9 [sparse-read 11599872~0 out=8b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T21:18:37.014 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.983223+0000", 2024-09-17T21:18:37.014 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.005190025, 2024-09-17T21:18:37.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00630877, 2024-09-17T21:18:37.014 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:37.014 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:37.014 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:37.014 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:37.014 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:37.014 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:37.014 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1685 2024-09-17T21:18:37.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.014 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:37.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:37.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983223+0000", 2024-09-17T21:18:37.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:37.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:37.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983223+0000", 2024-09-17T21:18:37.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:37.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:37.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983224+0000", 2024-09-17T21:18:37.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8200000000000008e-07 2024-09-17T21:18:37.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:37.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983228+0000", 2024-09-17T21:18:37.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.16e-06 2024-09-17T21:18:37.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:37.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983242+0000", 2024-09-17T21:18:37.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4569999999999999e-05 2024-09-17T21:18:37.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:37.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983254+0000", 2024-09-17T21:18:37.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1112e-05 2024-09-17T21:18:37.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:37.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.989424+0000", 2024-09-17T21:18:37.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0061707400000000001 2024-09-17T21:18:37.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:37.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.989470+0000", 2024-09-17T21:18:37.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5757000000000003e-05 2024-09-17T21:18:37.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:37.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.989532+0000", 2024-09-17T21:18:37.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1449000000000001e-05 2024-09-17T21:18:37.018 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:37.018 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:37.018 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:37.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.018 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:1686 3.c 3:37eb6dd1:::rbd_data.11527824718.0000000000000007:9 [sparse-read 11730944~0 out=8b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T21:18:37.018 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.983270+0000", 2024-09-17T21:18:37.018 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0051430060000000003, 2024-09-17T21:18:37.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0064906920000000002, 2024-09-17T21:18:37.019 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:37.019 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:37.019 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:37.019 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:37.019 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:37.019 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:37.019 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1686 2024-09-17T21:18:37.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.019 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:37.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:37.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983270+0000", 2024-09-17T21:18:37.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:37.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:37.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983270+0000", 2024-09-17T21:18:37.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:37.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:37.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983271+0000", 2024-09-17T21:18:37.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.093e-06 2024-09-17T21:18:37.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:37.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983276+0000", 2024-09-17T21:18:37.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6469999999999997e-06 2024-09-17T21:18:37.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:37.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983290+0000", 2024-09-17T21:18:37.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4785e-05 2024-09-17T21:18:37.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:37.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983301+0000", 2024-09-17T21:18:37.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0898e-05 2024-09-17T21:18:37.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:37.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.989562+0000", 2024-09-17T21:18:37.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006261193 2024-09-17T21:18:37.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:37.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.989654+0000", 2024-09-17T21:18:37.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2042999999999999e-05 2024-09-17T21:18:37.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:37.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.989761+0000", 2024-09-17T21:18:37.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106745 2024-09-17T21:18:37.022 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:37.023 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:37.023 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:37.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.023 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:1688 3.c 3:37eb6dd1:::rbd_data.11527824718.0000000000000007:9 [sparse-read 11993088~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T21:18:37.023 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.983363+0000", 2024-09-17T21:18:37.023 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0050498180000000002, 2024-09-17T21:18:37.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0066289649999999997, 2024-09-17T21:18:37.023 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:37.023 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:37.023 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:37.023 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:37.023 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:37.023 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:37.024 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1688 2024-09-17T21:18:37.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.024 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:37.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:37.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983363+0000", 2024-09-17T21:18:37.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:37.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:37.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983363+0000", 2024-09-17T21:18:37.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:37.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:37.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983364+0000", 2024-09-17T21:18:37.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.034e-06 2024-09-17T21:18:37.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:37.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983369+0000", 2024-09-17T21:18:37.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7369999999999997e-06 2024-09-17T21:18:37.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:37.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983384+0000", 2024-09-17T21:18:37.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4866e-05 2024-09-17T21:18:37.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:37.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983395+0000", 2024-09-17T21:18:37.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1494e-05 2024-09-17T21:18:37.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:37.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.989914+0000", 2024-09-17T21:18:37.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0065186460000000003 2024-09-17T21:18:37.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:37.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.989955+0000", 2024-09-17T21:18:37.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1247e-05 2024-09-17T21:18:37.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:37.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.990010+0000", 2024-09-17T21:18:37.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4715e-05 2024-09-17T21:18:37.027 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:37.027 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:37.027 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:37.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.027 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:1689 3.c 3.8bb6d7ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T21:18:37.028 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.983410+0000", 2024-09-17T21:18:37.028 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0050025920000000001, 2024-09-17T21:18:37.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067789879999999997, 2024-09-17T21:18:37.028 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:37.028 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:37.028 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:18:37.028 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:37.028 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:37.028 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:37.028 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1689 2024-09-17T21:18:37.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.028 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:37.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:37.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983410+0000", 2024-09-17T21:18:37.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:37.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:37.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983410+0000", 2024-09-17T21:18:37.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:37.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:37.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983411+0000", 2024-09-17T21:18:37.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0130000000000001e-06 2024-09-17T21:18:37.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:37.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983415+0000", 2024-09-17T21:18:37.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2270000000000004e-06 2024-09-17T21:18:37.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:37.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983430+0000", 2024-09-17T21:18:37.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4610000000000001e-05 2024-09-17T21:18:37.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:37.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:37.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:37.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.983441+0000", 2024-09-17T21:18:37.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0977e-05 2024-09-17T21:18:37.031 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:37.031 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:37.031 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:37.031 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:37.031 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-09-17T21:18:37.031 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 9 2024-09-17T21:18:37.031 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:37.032 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:18:37.033 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:18:37.033 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:18:37.033 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:18:37.034 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:18:37.034 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:37.026+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:37.034 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:18:37.035 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:18:37.048 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:18:37.049 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:18:37.133 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:37.130+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:37.202 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:37.202 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:37.217 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:18:37.234 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:37.232+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:37.335 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:37.328+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:37.376 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:37.376 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:37.390 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:18:37.436 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:37.430+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:37.521 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:37.521 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:37.535 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:18:37.536 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:37.532+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:37.638 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:37.632+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:37.653 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:37.653 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:37.667 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:18:37.739 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:37.734+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:37.789 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-17T21:18:37.840 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:37.840 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:37.840 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:37.836+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:37.856 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:18:37.944 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:37.940+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:38.021 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:38.022 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:38.036 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:18:38.080 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:38.076+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:38.142 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:38.136+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:38.203 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:38.204 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:38.218 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:18:38.242 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:38.238+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:38.343 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:38.336+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:38.388 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:38.388 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:38.403 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:18:38.429 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-17T21:18:38.444 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:38.440+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:38.491 INFO:teuthology.orchestra.run.smithi184.stdout:751 read 0x136c200 thru 0x13799bf (0xd7c0 bytes) 2024-09-17T21:18:38.493 INFO:teuthology.orchestra.run.smithi184.stdout:752 read 0x3213a00 thru 0x321fbd1 (0xc1d2 bytes) 2024-09-17T21:18:38.495 INFO:teuthology.orchestra.run.smithi184.stdout:753 punch from 0x8d4f800 to 0x8d5d400, (0xdc00 bytes) 2024-09-17T21:18:38.544 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:38.538+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:38.585 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:38.585 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:38.600 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:18:38.621 INFO:teuthology.orchestra.run.smithi184.stdout:754 read 0x5d52600 thru 0x5d59cea (0x76eb bytes) 2024-09-17T21:18:38.623 INFO:teuthology.orchestra.run.smithi184.stdout:757 punch from 0x462fe00 to 0x4633e00, (0x4000 bytes) 2024-09-17T21:18:38.645 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:38.640+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:38.735 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:38.735 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:38.737 INFO:teuthology.orchestra.run.smithi184.stdout:759 read 0x67d3e00 thru 0x67d62ee (0x24ef bytes) 2024-09-17T21:18:38.739 INFO:teuthology.orchestra.run.smithi184.stdout:763 write 0x7822800 thru 0x7826c04 (0x4405 bytes) 2024-09-17T21:18:38.740 INFO:teuthology.orchestra.run.smithi184.stdout:764 read 0x8f2e600 thru 0x8f3c0b2 (0xdab3 bytes) 2024-09-17T21:18:38.742 INFO:teuthology.orchestra.run.smithi184.stdout:766 write 0x5559c00 thru 0x556071b (0x6b1c bytes) 2024-09-17T21:18:38.745 INFO:teuthology.orchestra.run.smithi184.stdout:767 read 0x6c1ac00 thru 0x6c1eb81 (0x3f82 bytes) 2024-09-17T21:18:38.746 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:38.740+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:38.746 INFO:teuthology.orchestra.run.smithi184.stdout:769 read 0x30cee00 thru 0x30ddc38 (0xee39 bytes) 2024-09-17T21:18:38.748 INFO:teuthology.orchestra.run.smithi184.stdout:770 read 0x898c800 thru 0x899a89e (0xe09f bytes) 2024-09-17T21:18:38.750 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:18:38.750 INFO:teuthology.orchestra.run.smithi184.stdout:775 punch from 0x6e21c00 to 0x6e29c00, (0x8000 bytes) 2024-09-17T21:18:38.781 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:18:38.782 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:18:38.782 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 2, 3, 4, 5] 2024-09-17T21:18:38.782 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-09-17T21:18:38.847 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:38.842+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:38.914 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:38.914 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:38.929 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:18:38.948 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:38.944+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:39.049 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:39.046+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:39.059 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:39.060 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:39.065 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:18:39.074 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:18:39.076 INFO:teuthology.orchestra.run.smithi184.stdout:777 read 0xa642200 thru 0xa643625 (0x1426 bytes) 2024-09-17T21:18:39.121 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-09-17T21:18:39.116+0000 7fe37768f640 -1 mon.a@0(leader).osd e41 definitely_dead 0 2024-09-17T21:18:39.124 INFO:teuthology.orchestra.run.smithi184.stdout:778 write 0xcf91a00 thru 0xcf98e42 (0x7443 bytes) 2024-09-17T21:18:39.150 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:39.144+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:39.171 INFO:teuthology.orchestra.run.smithi184.stdout:780 trunc from 0xcf99000 to 0x4fa3800 2024-09-17T21:18:39.225 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:18:39.226 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:18:39.248 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:18:39.248 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:18:39.250 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:39.244+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:39.252 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:39.252 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:39.266 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:18:39.266 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:18:39.272 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:18:39.287 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:18:39.287 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:18:39.351 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:39.346+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:39.407 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:39.407 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:18:39.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:18:39.407 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:39.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.407 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:39.407 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:18:39.407 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 42.391417519999997, 2024-09-17T21:18:39.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:18:39.408 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:39.408 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:39.408 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:39.408 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:39.408 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:39.408 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:39.408 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:18:39.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.408 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:39.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.408 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:39.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:18:39.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:39.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:18:39.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:39.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:18:39.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:18:39.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:39.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:18:39.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:18:39.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:39.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:18:39.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:18:39.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:39.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:18:39.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:18:39.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:18:39.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:18:39.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.412 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:39.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:18:39.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:18:39.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.412 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:18:39.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:18:39.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:39.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:18:39.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:18:39.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:39.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:18:39.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:18:39.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:39.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:18:39.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:18:39.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:39.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:18:39.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:18:39.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:39.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:18:39.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:18:39.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:39.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:18:39.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:18:39.415 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.415 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:39.415 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.415 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:18:39.415 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:18:39.415 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.811253239999999, 2024-09-17T21:18:39.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:18:39.415 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:18:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:18:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:18:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:18:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:18:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:18:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:18:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:18:39.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:39.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:18:39.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:18:39.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:39.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:18:39.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:18:39.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:18:39.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:18:39.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:39.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:18:39.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:18:39.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:18:39.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:18:39.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:39.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:18:39.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:18:39.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:18:39.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:18:39.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:39.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:18:39.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:18:39.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:39.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:18:39.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:18:39.421 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.421 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:39.421 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:18:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:18:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.811155928000002, 2024-09-17T21:18:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:18:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:18:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:18:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:18:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:18:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:18:39.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:39.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:18:39.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:18:39.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:39.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:18:39.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:18:39.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:39.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:18:39.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:18:39.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:18:39.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:18:39.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:39.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:18:39.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:18:39.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.426 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.426 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:18:39.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:18:39.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.426 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:39.426 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:18:39.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:18:39.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.426 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.426 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:18:39.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:18:39.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:18:39.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:18:39.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:18:39.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:18:39.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:18:39.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:39.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:18:39.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:18:39.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:39.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:18:39.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:18:39.428 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.428 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:39.428 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.429 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:18:39.429 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:18:39.429 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.809069127000001, 2024-09-17T21:18:39.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:18:39.429 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:39.429 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:39.429 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:39.429 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:39.429 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:18:39.429 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:39.429 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:18:39.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.429 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:39.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:39.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:18:39.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:39.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:18:39.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:39.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:18:39.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:18:39.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:39.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:18:39.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:18:39.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:39.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:18:39.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:18:39.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:39.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:18:39.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:18:39.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:18:39.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:18:39.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:18:39.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:18:39.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:18:39.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:18:39.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:18:39.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:39.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:18:39.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:18:39.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:18:39.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:18:39.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:39.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:18:39.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:18:39.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:39.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:18:39.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:18:39.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:39.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:18:39.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:18:39.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:39.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:18:39.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:18:39.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:39.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:18:39.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:18:39.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:39.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:18:39.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:18:39.436 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.436 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:39.436 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.436 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:282 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:39.436 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:18:39.436 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.1773638659999999, 2024-09-17T21:18:39.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023571801, 2024-09-17T21:18:39.436 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:39.436 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:39.436 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:39.437 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:39.437 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:39.437 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:39.437 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 282 2024-09-17T21:18:39.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.437 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:39.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:39.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:18:39.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:39.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:18:39.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:39.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227820+0000", 2024-09-17T21:18:39.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.11e-06 2024-09-17T21:18:39.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:39.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227825+0000", 2024-09-17T21:18:39.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8099999999999997e-06 2024-09-17T21:18:39.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:39.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227848+0000", 2024-09-17T21:18:39.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2949999999999999e-05 2024-09-17T21:18:39.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:39.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227862+0000", 2024-09-17T21:18:39.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:18:39.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.247977+0000", 2024-09-17T21:18:39.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020114968 2024-09-17T21:18:39.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:39.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251141+0000", 2024-09-17T21:18:39.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031640689999999998 2024-09-17T21:18:39.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:39.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251391+0000", 2024-09-17T21:18:39.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025018499999999999 2024-09-17T21:18:39.440 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.440 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:39.440 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.441 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:283 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:39.441 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:18:39.441 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.1773020270000001, 2024-09-17T21:18:39.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024022274999999999, 2024-09-17T21:18:39.441 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:39.441 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:39.441 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:39.441 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:39.441 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:39.441 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:39.441 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 283 2024-09-17T21:18:39.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.441 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:39.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:39.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:18:39.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:39.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:18:39.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:39.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227882+0000", 2024-09-17T21:18:39.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-06 2024-09-17T21:18:39.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:39.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227887+0000", 2024-09-17T21:18:39.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1769999999999997e-06 2024-09-17T21:18:39.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:39.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227905+0000", 2024-09-17T21:18:39.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7235999999999999e-05 2024-09-17T21:18:39.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:39.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227918+0000", 2024-09-17T21:18:39.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3387000000000001e-05 2024-09-17T21:18:39.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251469+0000", 2024-09-17T21:18:39.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023550894999999999 2024-09-17T21:18:39.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:39.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251669+0000", 2024-09-17T21:18:39.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019992499999999999 2024-09-17T21:18:39.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:39.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251903+0000", 2024-09-17T21:18:39.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000234412 2024-09-17T21:18:39.445 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.445 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:39.445 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.445 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:284 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:39.445 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:18:39.445 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.1772464739999999, 2024-09-17T21:18:39.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024450084, 2024-09-17T21:18:39.445 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:39.445 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:39.446 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:39.446 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:39.446 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:39.446 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:39.446 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 284 2024-09-17T21:18:39.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.446 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:39.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:39.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:18:39.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:39.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:18:39.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:39.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227937+0000", 2024-09-17T21:18:39.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6700000000000002e-07 2024-09-17T21:18:39.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:39.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227943+0000", 2024-09-17T21:18:39.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:18:39.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:39.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227961+0000", 2024-09-17T21:18:39.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.827e-05 2024-09-17T21:18:39.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:39.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227975+0000", 2024-09-17T21:18:39.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.394e-05 2024-09-17T21:18:39.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251977+0000", 2024-09-17T21:18:39.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024001340999999999 2024-09-17T21:18:39.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:39.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252169+0000", 2024-09-17T21:18:39.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019271199999999999 2024-09-17T21:18:39.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:39.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252387+0000", 2024-09-17T21:18:39.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021726100000000001 2024-09-17T21:18:39.449 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.449 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:39.449 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.450 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:285 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:39.450 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:18:39.450 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.1771901840000001, 2024-09-17T21:18:39.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024884791, 2024-09-17T21:18:39.450 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:39.450 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:39.450 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:39.450 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:39.450 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:39.450 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:39.450 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 285 2024-09-17T21:18:39.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.450 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:39.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:39.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:18:39.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:39.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:18:39.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:39.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227994+0000", 2024-09-17T21:18:39.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1429999999999999e-06 2024-09-17T21:18:39.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:39.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227999+0000", 2024-09-17T21:18:39.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1669999999999998e-06 2024-09-17T21:18:39.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:39.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228015+0000", 2024-09-17T21:18:39.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6056e-05 2024-09-17T21:18:39.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:39.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228028+0000", 2024-09-17T21:18:39.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3196e-05 2024-09-17T21:18:39.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252458+0000", 2024-09-17T21:18:39.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024429751999999999 2024-09-17T21:18:39.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:39.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252650+0000", 2024-09-17T21:18:39.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019187800000000001 2024-09-17T21:18:39.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:39.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252878+0000", 2024-09-17T21:18:39.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022759899999999999 2024-09-17T21:18:39.454 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.454 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:39.454 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.454 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:286 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:39.454 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:18:39.454 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.1771344250000002, 2024-09-17T21:18:39.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025299318000000001, 2024-09-17T21:18:39.455 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:39.455 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:39.455 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:39.455 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:39.455 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:39.455 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:39.455 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 286 2024-09-17T21:18:39.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.455 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:39.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:39.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:18:39.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:39.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:18:39.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:39.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228050+0000", 2024-09-17T21:18:39.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:18:39.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:39.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228055+0000", 2024-09-17T21:18:39.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4960000000000004e-06 2024-09-17T21:18:39.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:39.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228075+0000", 2024-09-17T21:18:39.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0109999999999999e-05 2024-09-17T21:18:39.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:39.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228089+0000", 2024-09-17T21:18:39.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4154e-05 2024-09-17T21:18:39.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252952+0000", 2024-09-17T21:18:39.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024862934 2024-09-17T21:18:39.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:39.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253141+0000", 2024-09-17T21:18:39.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000188782 2024-09-17T21:18:39.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:39.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253348+0000", 2024-09-17T21:18:39.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020684499999999999 2024-09-17T21:18:39.458 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.458 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:39.459 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.459 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:439 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:39.459 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241357+0000", 2024-09-17T21:18:39.459 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.163825621, 2024-09-17T21:18:39.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023114866000000001, 2024-09-17T21:18:39.459 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:39.459 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:39.459 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:39.459 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:39.459 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:39.459 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:39.459 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 439 2024-09-17T21:18:39.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.460 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:39.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:39.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241357+0000", 2024-09-17T21:18:39.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:39.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241357+0000", 2024-09-17T21:18:39.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:39.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241358+0000", 2024-09-17T21:18:39.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.113e-06 2024-09-17T21:18:39.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:39.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241364+0000", 2024-09-17T21:18:39.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3900000000000001e-06 2024-09-17T21:18:39.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:39.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241382+0000", 2024-09-17T21:18:39.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7759999999999999e-05 2024-09-17T21:18:39.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:39.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241396+0000", 2024-09-17T21:18:39.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4303e-05 2024-09-17T21:18:39.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263586+0000", 2024-09-17T21:18:39.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022189952999999998 2024-09-17T21:18:39.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:39.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263683+0000", 2024-09-17T21:18:39.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7145000000000006e-05 2024-09-17T21:18:39.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:39.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264472+0000", 2024-09-17T21:18:39.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00078920200000000002 2024-09-17T21:18:39.463 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.463 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:39.463 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.463 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:440 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:39.463 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241415+0000", 2024-09-17T21:18:39.463 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.1637676809999999, 2024-09-17T21:18:39.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022660967000000001, 2024-09-17T21:18:39.464 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:39.464 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:39.464 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:39.464 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:39.464 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:39.464 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:39.464 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 440 2024-09-17T21:18:39.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.464 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:39.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:39.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241415+0000", 2024-09-17T21:18:39.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:39.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241415+0000", 2024-09-17T21:18:39.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:39.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241416+0000", 2024-09-17T21:18:39.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.133e-06 2024-09-17T21:18:39.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:39.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241422+0000", 2024-09-17T21:18:39.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7470000000000002e-06 2024-09-17T21:18:39.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:39.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241440+0000", 2024-09-17T21:18:39.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8029999999999998e-05 2024-09-17T21:18:39.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:39.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241454+0000", 2024-09-17T21:18:39.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3626e-05 2024-09-17T21:18:39.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263842+0000", 2024-09-17T21:18:39.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022388308999999999 2024-09-17T21:18:39.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:39.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263936+0000", 2024-09-17T21:18:39.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4190000000000005e-05 2024-09-17T21:18:39.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:39.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264076+0000", 2024-09-17T21:18:39.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000139932 2024-09-17T21:18:39.467 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.468 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:39.468 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.468 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:441 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2162688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:39.468 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241478+0000", 2024-09-17T21:18:39.468 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.163705142, 2024-09-17T21:18:39.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022827729000000001, 2024-09-17T21:18:39.468 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:39.468 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:39.468 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:39.468 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:39.468 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:39.469 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:39.469 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 441 2024-09-17T21:18:39.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.469 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:39.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:39.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241478+0000", 2024-09-17T21:18:39.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:39.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241478+0000", 2024-09-17T21:18:39.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:39.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241479+0000", 2024-09-17T21:18:39.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6530000000000001e-06 2024-09-17T21:18:39.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:39.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241485+0000", 2024-09-17T21:18:39.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3839999999999996e-06 2024-09-17T21:18:39.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:39.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241503+0000", 2024-09-17T21:18:39.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8233e-05 2024-09-17T21:18:39.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:39.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241517+0000", 2024-09-17T21:18:39.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3893000000000001e-05 2024-09-17T21:18:39.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264107+0000", 2024-09-17T21:18:39.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022590328999999999 2024-09-17T21:18:39.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:39.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264195+0000", 2024-09-17T21:18:39.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7255000000000004e-05 2024-09-17T21:18:39.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:39.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264306+0000", 2024-09-17T21:18:39.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000110982 2024-09-17T21:18:39.472 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.472 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:39.472 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.473 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:442 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2293760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:39.473 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241535+0000", 2024-09-17T21:18:39.473 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.163647536, 2024-09-17T21:18:39.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023012353999999999, 2024-09-17T21:18:39.473 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:39.473 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:39.473 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:39.473 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:39.473 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:39.473 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:39.473 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 442 2024-09-17T21:18:39.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.473 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:39.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:39.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241535+0000", 2024-09-17T21:18:39.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:39.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241535+0000", 2024-09-17T21:18:39.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:39.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241537+0000", 2024-09-17T21:18:39.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1939999999999999e-06 2024-09-17T21:18:39.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:39.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241557+0000", 2024-09-17T21:18:39.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.054e-05 2024-09-17T21:18:39.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:39.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241594+0000", 2024-09-17T21:18:39.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6782999999999997e-05 2024-09-17T21:18:39.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:39.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241611+0000", 2024-09-17T21:18:39.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6833e-05 2024-09-17T21:18:39.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264340+0000", 2024-09-17T21:18:39.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022729668000000001 2024-09-17T21:18:39.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:39.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264426+0000", 2024-09-17T21:18:39.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5180999999999996e-05 2024-09-17T21:18:39.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:39.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264548+0000", 2024-09-17T21:18:39.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012215500000000001 2024-09-17T21:18:39.477 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.477 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:39.477 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.477 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:443 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:39.477 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:18:39.477 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.16354712, 2024-09-17T21:18:39.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023179569000000001, 2024-09-17T21:18:39.477 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:39.477 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:39.477 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:39.478 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:39.478 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:39.478 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:39.478 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 443 2024-09-17T21:18:39.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.478 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:39.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:39.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:18:39.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:39.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:18:39.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:39.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241637+0000", 2024-09-17T21:18:39.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.46e-06 2024-09-17T21:18:39.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:39.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241643+0000", 2024-09-17T21:18:39.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1970000000000001e-06 2024-09-17T21:18:39.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:39.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241662+0000", 2024-09-17T21:18:39.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8712999999999999e-05 2024-09-17T21:18:39.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:39.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241676+0000", 2024-09-17T21:18:39.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.393e-05 2024-09-17T21:18:39.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264596+0000", 2024-09-17T21:18:39.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022919688000000001 2024-09-17T21:18:39.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:39.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264690+0000", 2024-09-17T21:18:39.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4145000000000001e-05 2024-09-17T21:18:39.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:39.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264815+0000", 2024-09-17T21:18:39.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012543600000000001 2024-09-17T21:18:39.481 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.481 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:39.481 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.482 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:444 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:39.482 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:18:39.482 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.1634860699999998, 2024-09-17T21:18:39.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023260383999999999, 2024-09-17T21:18:39.482 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:39.482 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:39.482 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:39.482 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:39.482 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:39.482 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:39.482 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 444 2024-09-17T21:18:39.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.482 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:39.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:39.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:18:39.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:39.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:18:39.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:39.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241698+0000", 2024-09-17T21:18:39.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2270000000000001e-06 2024-09-17T21:18:39.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:39.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241704+0000", 2024-09-17T21:18:39.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9800000000000003e-06 2024-09-17T21:18:39.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:39.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241725+0000", 2024-09-17T21:18:39.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0812999999999999e-05 2024-09-17T21:18:39.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:39.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241739+0000", 2024-09-17T21:18:39.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3773e-05 2024-09-17T21:18:39.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264850+0000", 2024-09-17T21:18:39.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023111326000000001 2024-09-17T21:18:39.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:39.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264897+0000", 2024-09-17T21:18:39.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6638999999999998e-05 2024-09-17T21:18:39.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:39.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264957+0000", 2024-09-17T21:18:39.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0625999999999999e-05 2024-09-17T21:18:39.486 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.486 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:39.486 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.486 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:445 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:39.486 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:18:39.486 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.163425401, 2024-09-17T21:18:39.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023315144999999999, 2024-09-17T21:18:39.486 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:39.486 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:39.487 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:39.487 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:39.487 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:39.487 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:39.487 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 445 2024-09-17T21:18:39.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.487 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:39.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:39.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:18:39.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:39.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:18:39.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:39.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241759+0000", 2024-09-17T21:18:39.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-06 2024-09-17T21:18:39.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:39.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241764+0000", 2024-09-17T21:18:39.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5899999999999998e-06 2024-09-17T21:18:39.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:39.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241784+0000", 2024-09-17T21:18:39.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9204000000000001e-05 2024-09-17T21:18:39.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:39.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241798+0000", 2024-09-17T21:18:39.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3953e-05 2024-09-17T21:18:39.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264977+0000", 2024-09-17T21:18:39.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023179182 2024-09-17T21:18:39.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:39.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.265018+0000", 2024-09-17T21:18:39.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1171999999999998e-05 2024-09-17T21:18:39.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:39.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.265073+0000", 2024-09-17T21:18:39.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4741000000000003e-05 2024-09-17T21:18:39.490 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.490 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:39.490 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.491 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:677 3.13 3:cd0d74f9:::rbd_data.11527824718.000000000000000a:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:39.491 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.250552+0000", 2024-09-17T21:18:39.491 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.1546314120000001, 2024-09-17T21:18:39.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022610023999999999, 2024-09-17T21:18:39.491 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:39.491 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:39.491 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:39.491 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:39.491 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:39.491 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:39.491 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 677 2024-09-17T21:18:39.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.491 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:39.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:39.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250552+0000", 2024-09-17T21:18:39.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:39.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250552+0000", 2024-09-17T21:18:39.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:39.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250553+0000", 2024-09-17T21:18:39.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4699999999999999e-06 2024-09-17T21:18:39.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:39.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250571+0000", 2024-09-17T21:18:39.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8303000000000001e-05 2024-09-17T21:18:39.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:39.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250590+0000", 2024-09-17T21:18:39.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8723e-05 2024-09-17T21:18:39.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:39.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250603+0000", 2024-09-17T21:18:39.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3307e-05 2024-09-17T21:18:39.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.272971+0000", 2024-09-17T21:18:39.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022368140000000002 2024-09-17T21:18:39.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:39.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273051+0000", 2024-09-17T21:18:39.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9339999999999996e-05 2024-09-17T21:18:39.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:39.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273162+0000", 2024-09-17T21:18:39.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000110741 2024-09-17T21:18:39.495 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.495 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:39.495 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.495 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:678 3.13 3:cd0d74f9:::rbd_data.11527824718.000000000000000a:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:39.495 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.250622+0000", 2024-09-17T21:18:39.495 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.1545605330000002, 2024-09-17T21:18:39.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023048448999999999, 2024-09-17T21:18:39.495 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:39.495 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 678 2024-09-17T21:18:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250622+0000", 2024-09-17T21:18:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250622+0000", 2024-09-17T21:18:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250624+0000", 2024-09-17T21:18:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4270000000000001e-06 2024-09-17T21:18:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250629+0000", 2024-09-17T21:18:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6030000000000004e-06 2024-09-17T21:18:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250648+0000", 2024-09-17T21:18:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8337000000000001e-05 2024-09-17T21:18:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250661+0000", 2024-09-17T21:18:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3446e-05 2024-09-17T21:18:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273275+0000", 2024-09-17T21:18:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022613292 2024-09-17T21:18:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:39.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273467+0000", 2024-09-17T21:18:39.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019237900000000001 2024-09-17T21:18:39.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:39.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273671+0000", 2024-09-17T21:18:39.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020396499999999999 2024-09-17T21:18:39.499 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.499 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:39.499 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.500 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:679 3.13 3:cd0d74f9:::rbd_data.11527824718.000000000000000a:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:39.500 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:18:39.500 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.1545039799999999, 2024-09-17T21:18:39.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023284978000000001, 2024-09-17T21:18:39.500 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:39.500 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:39.500 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:39.500 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:39.500 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:39.500 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:39.500 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 679 2024-09-17T21:18:39.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.500 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:39.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:39.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:18:39.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:39.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:18:39.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:39.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250680+0000", 2024-09-17T21:18:39.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:18:39.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:39.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250685+0000", 2024-09-17T21:18:39.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1970000000000004e-06 2024-09-17T21:18:39.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:39.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250703+0000", 2024-09-17T21:18:39.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7396000000000001e-05 2024-09-17T21:18:39.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:39.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250716+0000", 2024-09-17T21:18:39.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.29e-05 2024-09-17T21:18:39.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273740+0000", 2024-09-17T21:18:39.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023024405000000001 2024-09-17T21:18:39.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:39.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273834+0000", 2024-09-17T21:18:39.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4124999999999998e-05 2024-09-17T21:18:39.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:39.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273964+0000", 2024-09-17T21:18:39.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012963499999999999 2024-09-17T21:18:39.504 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.504 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:39.504 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.504 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:680 3.13 3:cd0d74f9:::rbd_data.11527824718.000000000000000a:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:39.504 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:18:39.504 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.1544489000000002, 2024-09-17T21:18:39.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023687792999999999, 2024-09-17T21:18:39.504 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:39.505 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:39.505 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:39.505 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:39.505 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:39.505 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:39.505 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 680 2024-09-17T21:18:39.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.505 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:39.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:39.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:18:39.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:39.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:18:39.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:39.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250735+0000", 2024-09-17T21:18:39.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2669999999999999e-06 2024-09-17T21:18:39.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:39.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250740+0000", 2024-09-17T21:18:39.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8060000000000002e-06 2024-09-17T21:18:39.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:39.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250757+0000", 2024-09-17T21:18:39.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7303e-05 2024-09-17T21:18:39.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:39.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250771+0000", 2024-09-17T21:18:39.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3876999999999999e-05 2024-09-17T21:18:39.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274022+0000", 2024-09-17T21:18:39.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023251135999999999 2024-09-17T21:18:39.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:39.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274211+0000", 2024-09-17T21:18:39.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018813599999999999 2024-09-17T21:18:39.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:39.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274422+0000", 2024-09-17T21:18:39.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000211268 2024-09-17T21:18:39.508 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.508 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:39.509 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.509 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.509 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:39.509 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:39.511 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:18:39.512 INFO:teuthology.orchestra.run.smithi184.stdout:783 write 0xe101c00 thru 0xe1086b3 (0x6ab4 bytes) 2024-09-17T21:18:39.512 INFO:teuthology.orchestra.run.smithi184.stdout:784 read 0xdc70e00 thru 0xdc79b0a (0x8d0b bytes) 2024-09-17T21:18:39.512 INFO:teuthology.orchestra.run.smithi184.stdout:truncating to largest ever: 0xed7b600 2024-09-17T21:18:39.512 INFO:teuthology.orchestra.run.smithi184.stdout:786 trunc from 0xe108800 to 0xed7b600 2024-09-17T21:18:39.512 INFO:teuthology.orchestra.run.smithi184.stdout:788 write 0x5a3ac00 thru 0x5a42cd9 (0x80da bytes) 2024-09-17T21:18:39.512 INFO:teuthology.orchestra.run.smithi184.stdout:789 read 0x7e17200 thru 0x7e2492c (0xd72d bytes) 2024-09-17T21:18:39.512 INFO:teuthology.orchestra.run.smithi184.stdout:790 write 0x2c0fa00 thru 0x2c17c21 (0x8222 bytes) 2024-09-17T21:18:39.512 INFO:teuthology.orchestra.run.smithi184.stdout:793 read 0x7b69c00 thru 0x7b6d873 (0x3c74 bytes) 2024-09-17T21:18:39.512 INFO:teuthology.orchestra.run.smithi184.stdout:795 read 0x9524000 thru 0x9531b6d (0xdb6e bytes) 2024-09-17T21:18:39.512 INFO:teuthology.orchestra.run.smithi184.stdout:796 trunc from 0xed7b600 to 0x998b200 2024-09-17T21:18:39.513 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:39.504+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:39.514 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:18:39.514 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:18:39.520 INFO:teuthology.orchestra.run.smithi184.stdout:797 trunc from 0x998b200 to 0xcc0dc00 2024-09-17T21:18:39.537 INFO:teuthology.orchestra.run.smithi184.stdout:798 punch from 0x2bdaa00 to 0x2be9600, (0xec00 bytes) 2024-09-17T21:18:39.540 INFO:teuthology.orchestra.run.smithi184.stdout:799 read 0x70f8c00 thru 0x7105a41 (0xce42 bytes) 2024-09-17T21:18:39.541 INFO:teuthology.orchestra.run.smithi184.stdout:800 write 0x8f94000 thru 0x8f9d84e (0x984f bytes) 2024-09-17T21:18:39.544 INFO:teuthology.orchestra.run.smithi184.stdout:802 read 0x3874a00 thru 0x3877f5a (0x355b bytes) 2024-09-17T21:18:39.546 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:18:39.546 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:18:39.547 INFO:teuthology.orchestra.run.smithi184.stdout:803 punch from 0x33dd400 to 0x33de200, (0xe00 bytes) 2024-09-17T21:18:39.547 INFO:teuthology.orchestra.run.smithi184.stdout:804 write 0xe9800 thru 0xebed2 (0x26d3 bytes) 2024-09-17T21:18:39.563 INFO:teuthology.orchestra.run.smithi184.stdout:805 trunc from 0xcc0dc00 to 0x55af800 2024-09-17T21:18:39.612 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:39.608+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:39.615 INFO:teuthology.orchestra.run.smithi184.stdout:806 write 0x8775800 thru 0x877bf9f (0x67a0 bytes) 2024-09-17T21:18:39.619 INFO:teuthology.orchestra.run.smithi184.stdout:807 trunc from 0x877c000 to 0x7880e00 2024-09-17T21:18:39.640 INFO:teuthology.orchestra.run.smithi184.stdout:808 read 0x338000 thru 0x33c780 (0x4781 bytes) 2024-09-17T21:18:39.642 INFO:teuthology.orchestra.run.smithi184.stdout:809 write 0xe13800 thru 0xe15de8 (0x25e9 bytes) 2024-09-17T21:18:39.643 INFO:teuthology.orchestra.run.smithi184.stdout:811 write 0x4d7bc00 thru 0x4d883d6 (0xc7d7 bytes) 2024-09-17T21:18:39.669 INFO:teuthology.orchestra.run.smithi184.stdout:814 trunc from 0x7880e00 to 0x1d74e00 2024-09-17T21:18:39.695 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4615.0:3577 3.e e41/23 3:775f9571:::rbd_data.11527035abe.0000000000000008:head v 41'147, mlcod=41'145)", 2024-09-17T21:18:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:39.694746+0000", 2024-09-17T21:18:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.00046042300000000001, 2024-09-17T21:18:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00048272899999999999, 2024-09-17T21:18:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:39.694746+0000", 2024-09-17T21:18:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:39.694746+0000", 2024-09-17T21:18:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:39.694748+0000", 2024-09-17T21:18:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8470000000000001e-06 2024-09-17T21:18:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:39.694762+0000", 2024-09-17T21:18:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3818999999999999e-05 2024-09-17T21:18:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:39.694796+0000", 2024-09-17T21:18:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4007999999999997e-05 2024-09-17T21:18:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:39.694817+0000", 2024-09-17T21:18:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1452000000000001e-05 2024-09-17T21:18:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:39.694913+0000", 2024-09-17T21:18:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5544999999999994e-05 2024-09-17T21:18:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:39.694935+0000", 2024-09-17T21:18:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2493e-05 2024-09-17T21:18:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-09-17T21:18:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 1 2024-09-17T21:18:39.701 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:39.712 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:18:39.712 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:39.706+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:39.763 INFO:teuthology.orchestra.run.smithi184.stdout:815 write 0xcfac200 thru 0xcfb1dfa (0x5bfb bytes) 2024-09-17T21:18:39.767 INFO:teuthology.orchestra.run.smithi184.stdout:816 flatten 2024-09-17T21:18:39.791 INFO:teuthology.orchestra.run.smithi083.stderr:marked out osd.0. 2024-09-17T21:18:39.807 INFO:teuthology.orchestra.run.smithi184.stdout:817 write 0xaea7c00 thru 0xaead137 (0x5538 bytes) 2024-09-17T21:18:39.813 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:39.806+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:39.813 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:18:39.878 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:39.878 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:39.879 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:18:39.879 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:18:39.879 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:39.892 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:18:39.914 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:39.908+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:40.014 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:40.010+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:40.057 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:18:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:18:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:478 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 720896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:18:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.8225330980000001, 2024-09-17T21:18:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016845300000000001, 2024-09-17T21:18:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 478 2024-09-17T21:18:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:18:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:18:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233785+0000", 2024-09-17T21:18:40.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.116e-06 2024-09-17T21:18:40.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233803+0000", 2024-09-17T21:18:40.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7932999999999999e-05 2024-09-17T21:18:40.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233822+0000", 2024-09-17T21:18:40.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9276999999999999e-05 2024-09-17T21:18:40.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233836+0000", 2024-09-17T21:18:40.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4053000000000001e-05 2024-09-17T21:18:40.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.248579+0000", 2024-09-17T21:18:40.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014743529 2024-09-17T21:18:40.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.248795+0000", 2024-09-17T21:18:40.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021535200000000001 2024-09-17T21:18:40.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250629+0000", 2024-09-17T21:18:40.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018340400000000001 2024-09-17T21:18:40.062 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.062 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.062 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.062 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:479 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 851968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.062 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:18:40.062 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.8224420119999998, 2024-09-17T21:18:40.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017383900000000001, 2024-09-17T21:18:40.063 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.063 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.063 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.063 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.063 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.063 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.063 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 479 2024-09-17T21:18:40.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.063 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:18:40.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:18:40.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233876+0000", 2024-09-17T21:18:40.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2160000000000001e-06 2024-09-17T21:18:40.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233883+0000", 2024-09-17T21:18:40.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9199999999999998e-06 2024-09-17T21:18:40.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233900+0000", 2024-09-17T21:18:40.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7034e-05 2024-09-17T21:18:40.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233913+0000", 2024-09-17T21:18:40.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159000000000001e-05 2024-09-17T21:18:40.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250712+0000", 2024-09-17T21:18:40.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016798644000000001 2024-09-17T21:18:40.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250913+0000", 2024-09-17T21:18:40.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020127900000000001 2024-09-17T21:18:40.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251259+0000", 2024-09-17T21:18:40.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00034564799999999999 2024-09-17T21:18:40.066 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.066 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.066 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.067 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:480 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.067 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:18:40.067 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.822349167, 2024-09-17T21:18:40.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017902482000000001, 2024-09-17T21:18:40.067 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.067 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.067 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.067 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.067 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.067 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.067 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 480 2024-09-17T21:18:40.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.068 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:18:40.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:18:40.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233969+0000", 2024-09-17T21:18:40.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.454e-06 2024-09-17T21:18:40.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233981+0000", 2024-09-17T21:18:40.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1593000000000001e-05 2024-09-17T21:18:40.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233999+0000", 2024-09-17T21:18:40.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.806e-05 2024-09-17T21:18:40.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234012+0000", 2024-09-17T21:18:40.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:18:40.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251339+0000", 2024-09-17T21:18:40.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017326352 2024-09-17T21:18:40.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251534+0000", 2024-09-17T21:18:40.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000195842 2024-09-17T21:18:40.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251870+0000", 2024-09-17T21:18:40.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033551400000000003 2024-09-17T21:18:40.071 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.071 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.071 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.071 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:481 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1114112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.071 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:18:40.071 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.8222808000000001, 2024-09-17T21:18:40.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018442621999999999, 2024-09-17T21:18:40.072 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.072 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.072 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.072 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.072 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.072 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.072 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 481 2024-09-17T21:18:40.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.072 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:18:40.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:18:40.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234037+0000", 2024-09-17T21:18:40.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:18:40.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234043+0000", 2024-09-17T21:18:40.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.49e-06 2024-09-17T21:18:40.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234061+0000", 2024-09-17T21:18:40.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8416000000000001e-05 2024-09-17T21:18:40.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234076+0000", 2024-09-17T21:18:40.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.456e-05 2024-09-17T21:18:40.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251948+0000", 2024-09-17T21:18:40.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017872774000000001 2024-09-17T21:18:40.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252144+0000", 2024-09-17T21:18:40.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019517300000000001 2024-09-17T21:18:40.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252478+0000", 2024-09-17T21:18:40.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033482899999999998 2024-09-17T21:18:40.076 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.076 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.076 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.076 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:482 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.076 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:18:40.076 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.8222222480000001, 2024-09-17T21:18:40.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018988026000000002, 2024-09-17T21:18:40.076 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.076 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.076 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.076 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.077 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.077 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.077 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 482 2024-09-17T21:18:40.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.077 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:18:40.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:18:40.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234096+0000", 2024-09-17T21:18:40.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2139999999999999e-06 2024-09-17T21:18:40.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234101+0000", 2024-09-17T21:18:40.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7100000000000004e-06 2024-09-17T21:18:40.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234118+0000", 2024-09-17T21:18:40.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6486000000000001e-05 2024-09-17T21:18:40.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234131+0000", 2024-09-17T21:18:40.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2824e-05 2024-09-17T21:18:40.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252552+0000", 2024-09-17T21:18:40.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018421549999999998 2024-09-17T21:18:40.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252746+0000", 2024-09-17T21:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019424399999999999 2024-09-17T21:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253082+0000", 2024-09-17T21:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000335998 2024-09-17T21:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:483 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.8221676279999999, 2024-09-17T21:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019462882000000001, 2024-09-17T21:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 483 2024-09-17T21:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234150+0000", 2024-09-17T21:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.127e-06 2024-09-17T21:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234155+0000", 2024-09-17T21:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234171+0000", 2024-09-17T21:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6160000000000001e-05 2024-09-17T21:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234185+0000", 2024-09-17T21:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3370000000000001e-05 2024-09-17T21:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253158+0000", 2024-09-17T21:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018972968999999999 2024-09-17T21:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253349+0000", 2024-09-17T21:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000190733 2024-09-17T21:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253612+0000", 2024-09-17T21:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026337000000000002 2024-09-17T21:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:484 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.8221136019999999, 2024-09-17T21:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020029261999999999, 2024-09-17T21:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 484 2024-09-17T21:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234205+0000", 2024-09-17T21:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6470000000000001e-06 2024-09-17T21:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234210+0000", 2024-09-17T21:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234227+0000", 2024-09-17T21:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.73e-05 2024-09-17T21:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234241+0000", 2024-09-17T21:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3757000000000001e-05 2024-09-17T21:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253690+0000", 2024-09-17T21:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019449368000000002 2024-09-17T21:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253898+0000", 2024-09-17T21:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000207161 2024-09-17T21:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254232+0000", 2024-09-17T21:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033474299999999998 2024-09-17T21:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:485 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1638400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.8220563749999998, 2024-09-17T21:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020553914, 2024-09-17T21:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 485 2024-09-17T21:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234261+0000", 2024-09-17T21:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.136e-06 2024-09-17T21:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234266+0000", 2024-09-17T21:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0669999999999999e-06 2024-09-17T21:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234284+0000", 2024-09-17T21:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7839999999999999e-05 2024-09-17T21:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234298+0000", 2024-09-17T21:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3852999999999999e-05 2024-09-17T21:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254311+0000", 2024-09-17T21:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020012973 2024-09-17T21:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254552+0000", 2024-09-17T21:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024116400000000001 2024-09-17T21:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254814+0000", 2024-09-17T21:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000261881 2024-09-17T21:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:486 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.8219999859999998, 2024-09-17T21:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021010240999999999, 2024-09-17T21:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 486 2024-09-17T21:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234318+0000", 2024-09-17T21:18:40.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.37e-06 2024-09-17T21:18:40.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234324+0000", 2024-09-17T21:18:40.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:18:40.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234341+0000", 2024-09-17T21:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7127000000000001e-05 2024-09-17T21:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234354+0000", 2024-09-17T21:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3210000000000001e-05 2024-09-17T21:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254896+0000", 2024-09-17T21:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020541835000000001 2024-09-17T21:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255092+0000", 2024-09-17T21:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019634700000000001 2024-09-17T21:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255327+0000", 2024-09-17T21:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023475899999999999 2024-09-17T21:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:487 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.8219437899999997, 2024-09-17T21:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021453911999999999, 2024-09-17T21:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 487 2024-09-17T21:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234374+0000", 2024-09-17T21:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3039999999999999e-06 2024-09-17T21:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234380+0000", 2024-09-17T21:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234397+0000", 2024-09-17T21:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7232999999999999e-05 2024-09-17T21:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234411+0000", 2024-09-17T21:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3784000000000001e-05 2024-09-17T21:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255406+0000", 2024-09-17T21:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020995204 2024-09-17T21:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255595+0000", 2024-09-17T21:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018903299999999999 2024-09-17T21:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255827+0000", 2024-09-17T21:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023190099999999999 2024-09-17T21:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:488 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.821888113, 2024-09-17T21:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026887966999999999, 2024-09-17T21:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 488 2024-09-17T21:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234430+0000", 2024-09-17T21:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1829999999999999e-06 2024-09-17T21:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234435+0000", 2024-09-17T21:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9869999999999999e-06 2024-09-17T21:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234451+0000", 2024-09-17T21:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6458999999999999e-05 2024-09-17T21:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234465+0000", 2024-09-17T21:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3730000000000001e-05 2024-09-17T21:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255908+0000", 2024-09-17T21:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021443413000000001 2024-09-17T21:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256098+0000", 2024-09-17T21:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018961 2024-09-17T21:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261317+0000", 2024-09-17T21:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052185850000000004 2024-09-17T21:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:489 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2162688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.8218322740000001, 2024-09-17T21:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022632815000000001, 2024-09-17T21:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 489 2024-09-17T21:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234486+0000", 2024-09-17T21:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5099999999999999e-06 2024-09-17T21:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234491+0000", 2024-09-17T21:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.147e-06 2024-09-17T21:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234508+0000", 2024-09-17T21:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7405999999999999e-05 2024-09-17T21:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234522+0000", 2024-09-17T21:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3677e-05 2024-09-17T21:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256536+0000", 2024-09-17T21:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022013464999999999 2024-09-17T21:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256752+0000", 2024-09-17T21:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021637800000000001 2024-09-17T21:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257117+0000", 2024-09-17T21:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000365232 2024-09-17T21:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:490 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2293760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.8217774840000001, 2024-09-17T21:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023146462999999999, 2024-09-17T21:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 490 2024-09-17T21:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234541+0000", 2024-09-17T21:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4869999999999999e-06 2024-09-17T21:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234559+0000", 2024-09-17T21:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8105999999999999e-05 2024-09-17T21:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234577+0000", 2024-09-17T21:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.836e-05 2024-09-17T21:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234592+0000", 2024-09-17T21:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4766000000000001e-05 2024-09-17T21:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257187+0000", 2024-09-17T21:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022594725 2024-09-17T21:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257374+0000", 2024-09-17T21:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018767299999999999 2024-09-17T21:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257686+0000", 2024-09-17T21:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000311346 2024-09-17T21:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:491 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.821704875, 2024-09-17T21:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023696872000000001, 2024-09-17T21:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 491 2024-09-17T21:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234613+0000", 2024-09-17T21:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-06 2024-09-17T21:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234618+0000", 2024-09-17T21:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3839999999999996e-06 2024-09-17T21:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234635+0000", 2024-09-17T21:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.685e-05 2024-09-17T21:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234648+0000", 2024-09-17T21:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.313e-05 2024-09-17T21:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257780+0000", 2024-09-17T21:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023131946 2024-09-17T21:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257985+0000", 2024-09-17T21:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020468400000000001 2024-09-17T21:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258309+0000", 2024-09-17T21:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032357499999999999 2024-09-17T21:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:492 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.8216492880000001, 2024-09-17T21:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024258994999999998, 2024-09-17T21:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 492 2024-09-17T21:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234669+0000", 2024-09-17T21:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.313e-06 2024-09-17T21:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234674+0000", 2024-09-17T21:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1599999999999997e-06 2024-09-17T21:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234690+0000", 2024-09-17T21:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.666e-05 2024-09-17T21:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234704+0000", 2024-09-17T21:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3043e-05 2024-09-17T21:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258397+0000", 2024-09-17T21:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023693204999999998 2024-09-17T21:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258593+0000", 2024-09-17T21:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019578999999999999 2024-09-17T21:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258926+0000", 2024-09-17T21:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033382399999999998 2024-09-17T21:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:493 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.821592823, 2024-09-17T21:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024793461999999999, 2024-09-17T21:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 493 2024-09-17T21:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234725+0000", 2024-09-17T21:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.24e-06 2024-09-17T21:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234731+0000", 2024-09-17T21:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8939999999999998e-06 2024-09-17T21:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234749+0000", 2024-09-17T21:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7863000000000002e-05 2024-09-17T21:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234762+0000", 2024-09-17T21:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3322999999999999e-05 2024-09-17T21:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259022+0000", 2024-09-17T21:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024259691999999999 2024-09-17T21:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259259+0000", 2024-09-17T21:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023669199999999999 2024-09-17T21:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259517+0000", 2024-09-17T21:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025875799999999999 2024-09-17T21:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:494 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.8215351229999999, 2024-09-17T21:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025251952000000001, 2024-09-17T21:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 494 2024-09-17T21:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234783+0000", 2024-09-17T21:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2440000000000001e-06 2024-09-17T21:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234789+0000", 2024-09-17T21:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234806+0000", 2024-09-17T21:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7450000000000001e-05 2024-09-17T21:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234819+0000", 2024-09-17T21:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3326000000000001e-05 2024-09-17T21:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259585+0000", 2024-09-17T21:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024765579999999999 2024-09-17T21:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259787+0000", 2024-09-17T21:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202444 2024-09-17T21:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260033+0000", 2024-09-17T21:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024611499999999998 2024-09-17T21:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:495 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.8214797599999999, 2024-09-17T21:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025776342000000001, 2024-09-17T21:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 495 2024-09-17T21:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234838+0000", 2024-09-17T21:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2130000000000001e-06 2024-09-17T21:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234843+0000", 2024-09-17T21:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9899999999999997e-06 2024-09-17T21:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234860+0000", 2024-09-17T21:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6973000000000002e-05 2024-09-17T21:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234873+0000", 2024-09-17T21:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3179999999999999e-05 2024-09-17T21:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260135+0000", 2024-09-17T21:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025262118 2024-09-17T21:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260364+0000", 2024-09-17T21:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022830699999999999 2024-09-17T21:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260613+0000", 2024-09-17T21:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024956099999999998 2024-09-17T21:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:496 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.82142648, 2024-09-17T21:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026376773999999999, 2024-09-17T21:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 496 2024-09-17T21:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234891+0000", 2024-09-17T21:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1230000000000001e-06 2024-09-17T21:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234896+0000", 2024-09-17T21:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9599999999999999e-06 2024-09-17T21:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234913+0000", 2024-09-17T21:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6747000000000001e-05 2024-09-17T21:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234926+0000", 2024-09-17T21:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-05 2024-09-17T21:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260675+0000", 2024-09-17T21:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025748979000000002 2024-09-17T21:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260883+0000", 2024-09-17T21:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020848399999999999 2024-09-17T21:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261267+0000", 2024-09-17T21:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038345099999999998 2024-09-17T21:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:682 3.9 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2455552~12288,write 8126464~10240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.7874894289999999, 2024-09-17T21:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018447261999999999, 2024-09-17T21:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 682 2024-09-17T21:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:18:40.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268830+0000", 2024-09-17T21:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6419999999999999e-06 2024-09-17T21:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271673+0000", 2024-09-17T21:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028432610000000001 2024-09-17T21:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271700+0000", 2024-09-17T21:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7200999999999999e-05 2024-09-17T21:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271715+0000", 2024-09-17T21:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5145e-05 2024-09-17T21:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271785+0000", 2024-09-17T21:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9519999999999998e-05 2024-09-17T21:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271918+0000", 2024-09-17T21:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013330899999999999 2024-09-17T21:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.272449+0000", 2024-09-17T21:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00053051200000000002 2024-09-17T21:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.278822+0000", 2024-09-17T21:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0063731619999999999 2024-09-17T21:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287167+0000", 2024-09-17T21:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083452149999999996 2024-09-17T21:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287213+0000", 2024-09-17T21:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5972e-05 2024-09-17T21:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287274+0000", 2024-09-17T21:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1322999999999995e-05 2024-09-17T21:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.152 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.152 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.152 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.152 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.152 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:40.154 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:18:40.154 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:40.150+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:40.254 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:40.252+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:40.324 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:40.324 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:40.324 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:18:40.324 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:40.339 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:18:40.386 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:40.382+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:40.456 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:40.452+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:40.462 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:40.462 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:40.462 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:18:40.462 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:18:40.462 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:40.476 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:18:40.557 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:40.552+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:40.646 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:40.646 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:18:40.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:18:40.646 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:40.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:18:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:18:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.039222182, 2024-09-17T21:18:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:18:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:18:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:18:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:18:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:18:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:18:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:18:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:18:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:18:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:18:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:18:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:18:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:18:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:18:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:18:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:18:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:18:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:18:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:18:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:18:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:18:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.029275546000001, 2024-09-17T21:18:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:18:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:18:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:18:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:18:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:18:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:18:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:18:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:18:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:18:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:18:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:18:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:18:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:18:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:18:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:18:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:18:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:18:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:18:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:18:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:18:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:18:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.786124705999999, 2024-09-17T21:18:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:18:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:18:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:18:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:18:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:18:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:18:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:18:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:18:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:18:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:18:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:18:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:18:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:18:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:18:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:18:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:18:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:18:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:18:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:18:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:18:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:18:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:18:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:18:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:18:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:18:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:18:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:18:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:18:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:18:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.776479219999999, 2024-09-17T21:18:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:18:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:18:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:18:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:18:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:18:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:18:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:18:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:18:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:18:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:18:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:18:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:18:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:18:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:18:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:18:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:18:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:18:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:18:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:18:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:18:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:18:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:18:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:18:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:18:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:18:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:18:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:18:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:495 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 2228224~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.197359+0000", 2024-09-17T21:18:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.446053386999999, 2024-09-17T21:18:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021840167000000001, 2024-09-17T21:18:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 495 2024-09-17T21:18:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197359+0000", 2024-09-17T21:18:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197359+0000", 2024-09-17T21:18:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197360+0000", 2024-09-17T21:18:40.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0380000000000001e-06 2024-09-17T21:18:40.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197376+0000", 2024-09-17T21:18:40.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6147999999999999e-05 2024-09-17T21:18:40.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197392+0000", 2024-09-17T21:18:40.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6025999999999999e-05 2024-09-17T21:18:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197401+0000", 2024-09-17T21:18:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6640000000000004e-06 2024-09-17T21:18:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.218539+0000", 2024-09-17T21:18:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021137808000000001 2024-09-17T21:18:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.218611+0000", 2024-09-17T21:18:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2471000000000005e-05 2024-09-17T21:18:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.219199+0000", 2024-09-17T21:18:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00058801199999999995 2024-09-17T21:18:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:548 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.200281+0000", 2024-09-17T21:18:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.443131728999999, 2024-09-17T21:18:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021975320999999999, 2024-09-17T21:18:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 548 2024-09-17T21:18:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200281+0000", 2024-09-17T21:18:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200281+0000", 2024-09-17T21:18:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200281+0000", 2024-09-17T21:18:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5100000000000001e-07 2024-09-17T21:18:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200283+0000", 2024-09-17T21:18:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4699999999999999e-06 2024-09-17T21:18:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200287+0000", 2024-09-17T21:18:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3939999999999998e-06 2024-09-17T21:18:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200290+0000", 2024-09-17T21:18:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3160000000000001e-06 2024-09-17T21:18:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.221522+0000", 2024-09-17T21:18:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021231782000000001 2024-09-17T21:18:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.221624+0000", 2024-09-17T21:18:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101693 2024-09-17T21:18:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.222256+0000", 2024-09-17T21:18:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00063231500000000005 2024-09-17T21:18:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:467 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235571+0000", 2024-09-17T21:18:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.4078413300000001, 2024-09-17T21:18:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021817462999999999, 2024-09-17T21:18:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 467 2024-09-17T21:18:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235571+0000", 2024-09-17T21:18:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235571+0000", 2024-09-17T21:18:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235573+0000", 2024-09-17T21:18:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5659999999999999e-06 2024-09-17T21:18:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235579+0000", 2024-09-17T21:18:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7729999999999996e-06 2024-09-17T21:18:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235598+0000", 2024-09-17T21:18:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9106999999999998e-05 2024-09-17T21:18:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235612+0000", 2024-09-17T21:18:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4032999999999999e-05 2024-09-17T21:18:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257023+0000", 2024-09-17T21:18:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021411619 2024-09-17T21:18:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257194+0000", 2024-09-17T21:18:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017076999999999999 2024-09-17T21:18:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257389+0000", 2024-09-17T21:18:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019459499999999999 2024-09-17T21:18:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:468 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235632+0000", 2024-09-17T21:18:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.4077808010000004, 2024-09-17T21:18:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022180165000000002, 2024-09-17T21:18:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 468 2024-09-17T21:18:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235632+0000", 2024-09-17T21:18:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235632+0000", 2024-09-17T21:18:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235633+0000", 2024-09-17T21:18:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.446e-06 2024-09-17T21:18:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235639+0000", 2024-09-17T21:18:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6269999999999997e-06 2024-09-17T21:18:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235658+0000", 2024-09-17T21:18:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8686999999999999e-05 2024-09-17T21:18:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235672+0000", 2024-09-17T21:18:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.449e-05 2024-09-17T21:18:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257452+0000", 2024-09-17T21:18:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021779827000000002 2024-09-17T21:18:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257622+0000", 2024-09-17T21:18:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016976099999999999 2024-09-17T21:18:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257812+0000", 2024-09-17T21:18:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019032700000000001 2024-09-17T21:18:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:469 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235692+0000", 2024-09-17T21:18:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.4077204710000002, 2024-09-17T21:18:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022615679, 2024-09-17T21:18:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 469 2024-09-17T21:18:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235692+0000", 2024-09-17T21:18:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235692+0000", 2024-09-17T21:18:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235694+0000", 2024-09-17T21:18:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.643e-06 2024-09-17T21:18:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235700+0000", 2024-09-17T21:18:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.733e-06 2024-09-17T21:18:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235718+0000", 2024-09-17T21:18:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8029999999999998e-05 2024-09-17T21:18:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235732+0000", 2024-09-17T21:18:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4239999999999999e-05 2024-09-17T21:18:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257935+0000", 2024-09-17T21:18:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022203514000000001 2024-09-17T21:18:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258107+0000", 2024-09-17T21:18:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017220700000000001 2024-09-17T21:18:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258308+0000", 2024-09-17T21:18:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020031199999999999 2024-09-17T21:18:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:470 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235752+0000", 2024-09-17T21:18:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.4076607790000004, 2024-09-17T21:18:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023324234999999999, 2024-09-17T21:18:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 470 2024-09-17T21:18:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235752+0000", 2024-09-17T21:18:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235752+0000", 2024-09-17T21:18:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235753+0000", 2024-09-17T21:18:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.46e-06 2024-09-17T21:18:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235759+0000", 2024-09-17T21:18:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5829999999999997e-06 2024-09-17T21:18:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235777+0000", 2024-09-17T21:18:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8196999999999999e-05 2024-09-17T21:18:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235791+0000", 2024-09-17T21:18:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4406999999999999e-05 2024-09-17T21:18:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258351+0000", 2024-09-17T21:18:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022559157999999999 2024-09-17T21:18:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258491+0000", 2024-09-17T21:18:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014083300000000001 2024-09-17T21:18:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259076+0000", 2024-09-17T21:18:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00058459699999999996 2024-09-17T21:18:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:471 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235811+0000", 2024-09-17T21:18:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.4076015589999997, 2024-09-17T21:18:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023519748, 2024-09-17T21:18:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 471 2024-09-17T21:18:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235811+0000", 2024-09-17T21:18:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235811+0000", 2024-09-17T21:18:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235813+0000", 2024-09-17T21:18:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.57e-06 2024-09-17T21:18:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235818+0000", 2024-09-17T21:18:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.417e-06 2024-09-17T21:18:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235836+0000", 2024-09-17T21:18:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8023e-05 2024-09-17T21:18:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235850+0000", 2024-09-17T21:18:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3759e-05 2024-09-17T21:18:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259120+0000", 2024-09-17T21:18:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023270193000000002 2024-09-17T21:18:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259225+0000", 2024-09-17T21:18:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104614 2024-09-17T21:18:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259331+0000", 2024-09-17T21:18:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106172 2024-09-17T21:18:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:18:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.4075431869999999, 2024-09-17T21:18:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:18:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:18:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:18:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:18:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:18:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:18:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:18:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:18:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:18:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:18:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:18:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:18:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:18:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:18:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:18:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:18:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:18:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:18:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:558 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239154+0000", 2024-09-17T21:18:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.4042583049999999, 2024-09-17T21:18:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021884532000000002, 2024-09-17T21:18:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 558 2024-09-17T21:18:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239154+0000", 2024-09-17T21:18:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239154+0000", 2024-09-17T21:18:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239156+0000", 2024-09-17T21:18:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4670000000000001e-06 2024-09-17T21:18:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239162+0000", 2024-09-17T21:18:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2829999999999998e-06 2024-09-17T21:18:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239181+0000", 2024-09-17T21:18:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8960000000000001e-05 2024-09-17T21:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239196+0000", 2024-09-17T21:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4523e-05 2024-09-17T21:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258757+0000", 2024-09-17T21:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019561775 2024-09-17T21:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258880+0000", 2024-09-17T21:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000123013 2024-09-17T21:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261039+0000", 2024-09-17T21:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021585110000000001 2024-09-17T21:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:559 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2162688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239215+0000", 2024-09-17T21:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.4041971320000002, 2024-09-17T21:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022314902000000001, 2024-09-17T21:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 559 2024-09-17T21:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239215+0000", 2024-09-17T21:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239215+0000", 2024-09-17T21:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239217+0000", 2024-09-17T21:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239223+0000", 2024-09-17T21:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6099999999999997e-06 2024-09-17T21:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239241+0000", 2024-09-17T21:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8576e-05 2024-09-17T21:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239256+0000", 2024-09-17T21:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4397e-05 2024-09-17T21:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261086+0000", 2024-09-17T21:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021830374 2024-09-17T21:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261203+0000", 2024-09-17T21:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000117192 2024-09-17T21:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261530+0000", 2024-09-17T21:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032712300000000002 2024-09-17T21:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:560 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2293760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239276+0000", 2024-09-17T21:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.4041367730000003, 2024-09-17T21:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022636927000000001, 2024-09-17T21:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 560 2024-09-17T21:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239276+0000", 2024-09-17T21:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239276+0000", 2024-09-17T21:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239277+0000", 2024-09-17T21:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4699999999999999e-06 2024-09-17T21:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239283+0000", 2024-09-17T21:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5899999999999998e-06 2024-09-17T21:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239301+0000", 2024-09-17T21:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8593999999999998e-05 2024-09-17T21:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239316+0000", 2024-09-17T21:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.428e-05 2024-09-17T21:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261592+0000", 2024-09-17T21:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022276118000000001 2024-09-17T21:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261733+0000", 2024-09-17T21:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014099199999999999 2024-09-17T21:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261913+0000", 2024-09-17T21:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017988300000000001 2024-09-17T21:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:561 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239336+0000", 2024-09-17T21:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.404076946, 2024-09-17T21:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02290615, 2024-09-17T21:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239336+0000", 2024-09-17T21:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239336+0000", 2024-09-17T21:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239337+0000", 2024-09-17T21:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3969999999999999e-06 2024-09-17T21:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239343+0000", 2024-09-17T21:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.699e-06 2024-09-17T21:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239361+0000", 2024-09-17T21:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8107000000000001e-05 2024-09-17T21:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239379+0000", 2024-09-17T21:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7972999999999999e-05 2024-09-17T21:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261956+0000", 2024-09-17T21:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022577105 2024-09-17T21:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262082+0000", 2024-09-17T21:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012621600000000001 2024-09-17T21:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262242+0000", 2024-09-17T21:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000159653 2024-09-17T21:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:562 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239398+0000", 2024-09-17T21:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.4040144699999999, 2024-09-17T21:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023207746000000001, 2024-09-17T21:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239398+0000", 2024-09-17T21:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239398+0000", 2024-09-17T21:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239400+0000", 2024-09-17T21:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.55e-06 2024-09-17T21:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239427+0000", 2024-09-17T21:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.745e-05 2024-09-17T21:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239447+0000", 2024-09-17T21:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9715999999999999e-05 2024-09-17T21:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239468+0000", 2024-09-17T21:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0894e-05 2024-09-17T21:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262285+0000", 2024-09-17T21:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022817456999999999 2024-09-17T21:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262436+0000", 2024-09-17T21:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015082999999999999 2024-09-17T21:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262606+0000", 2024-09-17T21:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016984899999999999 2024-09-17T21:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:563 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239497+0000", 2024-09-17T21:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.4039160610000003, 2024-09-17T21:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023494231000000001, 2024-09-17T21:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 563 2024-09-17T21:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239497+0000", 2024-09-17T21:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239497+0000", 2024-09-17T21:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239498+0000", 2024-09-17T21:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3969999999999999e-06 2024-09-17T21:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239539+0000", 2024-09-17T21:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0669e-05 2024-09-17T21:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239559+0000", 2024-09-17T21:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0353999999999999e-05 2024-09-17T21:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239575+0000", 2024-09-17T21:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5642999999999999e-05 2024-09-17T21:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262656+0000", 2024-09-17T21:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023081005000000002 2024-09-17T21:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262769+0000", 2024-09-17T21:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113616 2024-09-17T21:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262991+0000", 2024-09-17T21:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000221547 2024-09-17T21:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:564 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239598+0000", 2024-09-17T21:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.4038147409999997, 2024-09-17T21:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023713695999999999, 2024-09-17T21:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 564 2024-09-17T21:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239598+0000", 2024-09-17T21:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239598+0000", 2024-09-17T21:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239600+0000", 2024-09-17T21:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239606+0000", 2024-09-17T21:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2029999999999997e-06 2024-09-17T21:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239623+0000", 2024-09-17T21:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7669999999999999e-05 2024-09-17T21:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239638+0000", 2024-09-17T21:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4416999999999999e-05 2024-09-17T21:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263063+0000", 2024-09-17T21:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023424932999999998 2024-09-17T21:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263182+0000", 2024-09-17T21:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011914899999999999 2024-09-17T21:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263312+0000", 2024-09-17T21:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012956799999999999 2024-09-17T21:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:565 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.4037542219999999, 2024-09-17T21:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024193392000000001, 2024-09-17T21:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 565 2024-09-17T21:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239660+0000", 2024-09-17T21:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239666+0000", 2024-09-17T21:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8699999999999997e-06 2024-09-17T21:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239688+0000", 2024-09-17T21:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.213e-05 2024-09-17T21:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239703+0000", 2024-09-17T21:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4813e-05 2024-09-17T21:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263427+0000", 2024-09-17T21:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023724630999999999 2024-09-17T21:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263614+0000", 2024-09-17T21:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018664500000000001 2024-09-17T21:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263852+0000", 2024-09-17T21:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023793999999999999 2024-09-17T21:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:40.738 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:40.739 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:18:40.740 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:40.732+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:40.793 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-17T21:18:40.818 INFO:teuthology.orchestra.run.smithi184.stdout:818 punch from 0x3da6400 to 0x3dada00, (0x7600 bytes) 2024-09-17T21:18:40.819 INFO:teuthology.orchestra.run.smithi184.stdout:821 write 0x217ea00 thru 0x218dcfa (0xf2fb bytes) 2024-09-17T21:18:40.823 INFO:teuthology.orchestra.run.smithi184.stdout:822 read 0x426b200 thru 0x4273790 (0x8591 bytes) 2024-09-17T21:18:40.824 INFO:teuthology.orchestra.run.smithi184.stdout:825 write 0x69f2000 thru 0x69f63db (0x43dc bytes) 2024-09-17T21:18:40.825 INFO:teuthology.orchestra.run.smithi184.stdout:828 punch from 0x2079a00 to 0x207aa00, (0x1000 bytes) 2024-09-17T21:18:40.826 INFO:teuthology.orchestra.run.smithi184.stdout:831 read 0x48cf800 thru 0x48df0c6 (0xf8c7 bytes) 2024-09-17T21:18:40.828 INFO:teuthology.orchestra.run.smithi184.stdout:832 read 0xab70600 thru 0xab7b7b1 (0xb1b2 bytes) 2024-09-17T21:18:40.831 INFO:teuthology.orchestra.run.smithi184.stdout:833 read 0x8170200 thru 0x8175ac1 (0x58c2 bytes) 2024-09-17T21:18:40.833 INFO:teuthology.orchestra.run.smithi184.stdout:835 write 0x74a2800 thru 0x74a9476 (0x6c77 bytes) 2024-09-17T21:18:40.834 INFO:teuthology.orchestra.run.smithi184.stdout:837 write 0x5e26c00 thru 0x5e27ed8 (0x12d9 bytes) 2024-09-17T21:18:40.836 INFO:teuthology.orchestra.run.smithi184.stdout:839 read 0x1568e00 thru 0x1572f33 (0xa134 bytes) 2024-09-17T21:18:40.837 INFO:teuthology.orchestra.run.smithi184.stdout:841 read 0xabb8a00 thru 0xabc3bec (0xb1ed bytes) 2024-09-17T21:18:40.839 INFO:teuthology.orchestra.run.smithi184.stdout:842 write 0x2c55600 thru 0x2c5ee0e (0x980f bytes) 2024-09-17T21:18:40.840 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:40.836+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:40.840 INFO:teuthology.orchestra.run.smithi184.stdout:843 read 0xb637e00 thru 0xb646e9e (0xf09f bytes) 2024-09-17T21:18:40.842 INFO:teuthology.orchestra.run.smithi184.stdout:846 read 0x9142000 thru 0x914cf15 (0xaf16 bytes) 2024-09-17T21:18:40.843 INFO:teuthology.orchestra.run.smithi184.stdout:847 punch from 0xb5c8000 to 0xb5cdc00, (0x5c00 bytes) 2024-09-17T21:18:40.843 INFO:teuthology.orchestra.run.smithi184.stdout:848 read 0xb3dec00 thru 0xb3e6fb0 (0x83b1 bytes) 2024-09-17T21:18:40.845 INFO:teuthology.orchestra.run.smithi184.stdout:849 read 0x6ba4e00 thru 0x6bb2e60 (0xe061 bytes) 2024-09-17T21:18:40.849 INFO:teuthology.orchestra.run.smithi184.stdout:851 write 0x70daa00 thru 0x70e229b (0x789c bytes) 2024-09-17T21:18:40.856 INFO:teuthology.orchestra.run.smithi184.stdout:856 trunc from 0xcfb1e00 to 0xe377000 2024-09-17T21:18:40.894 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:40.894 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:40.894 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:40.894 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:40.910 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:18:40.933 INFO:teuthology.orchestra.run.smithi184.stdout:858 trunc from 0xe377000 to 0xa74c00 2024-09-17T21:18:40.956 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:40.952+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:41.001 INFO:teuthology.orchestra.run.smithi184.stdout:859 punch from 0x2a0600 to 0x2a0c00, (0x600 bytes) 2024-09-17T21:18:41.003 INFO:teuthology.orchestra.run.smithi184.stdout:861 punch from 0x6e1800 to 0x6e1c00, (0x400 bytes) 2024-09-17T21:18:41.020 INFO:teuthology.orchestra.run.smithi184.stdout:863 write 0x202c000 thru 0x2032552 (0x6553 bytes) 2024-09-17T21:18:41.041 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:41.036+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:41.061 INFO:teuthology.orchestra.run.smithi184.stdout:865 write 0x9341400 thru 0x934d718 (0xc319 bytes) 2024-09-17T21:18:41.063 INFO:teuthology.orchestra.run.smithi184.stdout:867 punch from 0x448e00 to 0x456400, (0xd600 bytes) 2024-09-17T21:18:41.064 INFO:teuthology.orchestra.run.smithi184.stdout:870 clone 9 order 18 su 65536 sc 2 2024-09-17T21:18:41.084 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:41.085 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:41.085 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:41.085 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:41.085 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:41.099 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:18:41.144 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:41.140+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:41.243 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:41.242+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:41.270 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:41.270 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:18:41.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:18:41.270 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:18:41.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.271 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:389 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:41.271 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.194936+0000", 2024-09-17T21:18:41.271 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.07252375, 2024-09-17T21:18:41.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041176510999999999, 2024-09-17T21:18:41.271 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:41.271 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:41.271 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:41.271 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:41.271 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:41.271 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:41.271 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 389 2024-09-17T21:18:41.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.272 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:41.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:41.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194936+0000", 2024-09-17T21:18:41.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:41.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194936+0000", 2024-09-17T21:18:41.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:41.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194937+0000", 2024-09-17T21:18:41.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6270000000000001e-06 2024-09-17T21:18:41.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:41.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194944+0000", 2024-09-17T21:18:41.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5729999999999996e-06 2024-09-17T21:18:41.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:41.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194961+0000", 2024-09-17T21:18:41.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7623000000000001e-05 2024-09-17T21:18:41.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:41.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194974+0000", 2024-09-17T21:18:41.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2989999999999999e-05 2024-09-17T21:18:41.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:41.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.235848+0000", 2024-09-17T21:18:41.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.040874099999999997 2024-09-17T21:18:41.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:41.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.235971+0000", 2024-09-17T21:18:41.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012254400000000001 2024-09-17T21:18:41.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:41.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236112+0000", 2024-09-17T21:18:41.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014105399999999999 2024-09-17T21:18:41.275 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.275 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:41.275 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.275 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:406 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 2359296~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:41.275 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:18:41.275 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.072404944000001, 2024-09-17T21:18:41.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042448018999999997, 2024-09-17T21:18:41.276 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:41.276 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:41.276 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:41.276 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:41.276 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:41.276 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:41.276 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 406 2024-09-17T21:18:41.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.276 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:41.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:41.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:18:41.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:41.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:18:41.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:41.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195056+0000", 2024-09-17T21:18:41.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2729999999999999e-06 2024-09-17T21:18:41.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:41.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195060+0000", 2024-09-17T21:18:41.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5399999999999997e-06 2024-09-17T21:18:41.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:41.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195076+0000", 2024-09-17T21:18:41.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6217000000000001e-05 2024-09-17T21:18:41.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:41.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195093+0000", 2024-09-17T21:18:41.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.683e-05 2024-09-17T21:18:41.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:41.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236217+0000", 2024-09-17T21:18:41.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041124102000000003 2024-09-17T21:18:41.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:41.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236339+0000", 2024-09-17T21:18:41.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121876 2024-09-17T21:18:41.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:41.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237502+0000", 2024-09-17T21:18:41.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011631809999999999 2024-09-17T21:18:41.280 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.280 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:41.280 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.280 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:416 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2228224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:41.280 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:18:41.280 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.071906918, 2024-09-17T21:18:41.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042361990000000002, 2024-09-17T21:18:41.280 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:41.280 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:41.280 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:41.281 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:41.281 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:41.281 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:41.281 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 416 2024-09-17T21:18:41.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.281 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:41.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:41.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:18:41.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:41.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:18:41.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:41.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195554+0000", 2024-09-17T21:18:41.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9300000000000002e-06 2024-09-17T21:18:41.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:41.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195563+0000", 2024-09-17T21:18:41.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2770000000000005e-06 2024-09-17T21:18:41.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:41.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195586+0000", 2024-09-17T21:18:41.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3889000000000001e-05 2024-09-17T21:18:41.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:41.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195603+0000", 2024-09-17T21:18:41.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6084000000000001e-05 2024-09-17T21:18:41.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:41.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237563+0000", 2024-09-17T21:18:41.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041960856999999997 2024-09-17T21:18:41.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:41.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237695+0000", 2024-09-17T21:18:41.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013144400000000001 2024-09-17T21:18:41.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:41.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237914+0000", 2024-09-17T21:18:41.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000219509 2024-09-17T21:18:41.284 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.284 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:41.285 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.285 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:439 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:41.285 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:18:41.285 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.071343016, 2024-09-17T21:18:41.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043053811999999997, 2024-09-17T21:18:41.285 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:41.285 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:41.285 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:41.285 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:41.285 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:41.285 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:41.286 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 439 2024-09-17T21:18:41.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.286 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:41.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:41.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:18:41.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:41.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:18:41.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:41.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196118+0000", 2024-09-17T21:18:41.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3969999999999999e-06 2024-09-17T21:18:41.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:41.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196157+0000", 2024-09-17T21:18:41.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9826999999999997e-05 2024-09-17T21:18:41.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:41.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196179+0000", 2024-09-17T21:18:41.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1756e-05 2024-09-17T21:18:41.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:41.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196194+0000", 2024-09-17T21:18:41.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4707e-05 2024-09-17T21:18:41.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:41.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237980+0000", 2024-09-17T21:18:41.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041786219999999999 2024-09-17T21:18:41.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:41.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.238065+0000", 2024-09-17T21:18:41.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4560000000000004e-05 2024-09-17T21:18:41.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:41.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239170+0000", 2024-09-17T21:18:41.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001105345 2024-09-17T21:18:41.289 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.289 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:41.289 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.289 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:440 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:41.289 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:18:41.289 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.071240512999999, 2024-09-17T21:18:41.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043890039999999998, 2024-09-17T21:18:41.290 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:41.290 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:41.290 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:41.290 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:41.290 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:41.290 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:41.290 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 440 2024-09-17T21:18:41.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.290 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:41.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:41.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:18:41.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:41.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:18:41.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:41.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196220+0000", 2024-09-17T21:18:41.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.254e-06 2024-09-17T21:18:41.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:41.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196225+0000", 2024-09-17T21:18:41.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7829999999999998e-06 2024-09-17T21:18:41.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:41.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196241+0000", 2024-09-17T21:18:41.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6322999999999999e-05 2024-09-17T21:18:41.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:41.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196258+0000", 2024-09-17T21:18:41.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7056000000000001e-05 2024-09-17T21:18:41.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:41.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239229+0000", 2024-09-17T21:18:41.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042971058999999999 2024-09-17T21:18:41.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:41.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239403+0000", 2024-09-17T21:18:41.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017399200000000001 2024-09-17T21:18:41.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:41.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240109+0000", 2024-09-17T21:18:41.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00070557300000000003 2024-09-17T21:18:41.293 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.293 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:41.294 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.294 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:450 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:41.294 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:18:41.294 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.070879562, 2024-09-17T21:18:41.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043994521000000002, 2024-09-17T21:18:41.294 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:41.294 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:41.294 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:41.294 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:41.294 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:41.294 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:41.294 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 450 2024-09-17T21:18:41.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.295 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:41.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:41.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:18:41.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:41.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:18:41.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:41.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196581+0000", 2024-09-17T21:18:41.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4500000000000001e-06 2024-09-17T21:18:41.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:41.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196588+0000", 2024-09-17T21:18:41.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7530000000000004e-06 2024-09-17T21:18:41.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:41.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196607+0000", 2024-09-17T21:18:41.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8987e-05 2024-09-17T21:18:41.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:41.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196620+0000", 2024-09-17T21:18:41.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3589e-05 2024-09-17T21:18:41.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:41.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240225+0000", 2024-09-17T21:18:41.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043604878999999999 2024-09-17T21:18:41.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:41.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240377+0000", 2024-09-17T21:18:41.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015146699999999999 2024-09-17T21:18:41.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:41.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240574+0000", 2024-09-17T21:18:41.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019739599999999999 2024-09-17T21:18:41.298 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.298 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:41.298 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.298 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:464 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:41.298 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:18:41.299 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.070629141, 2024-09-17T21:18:41.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044651692999999999, 2024-09-17T21:18:41.299 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:41.299 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:41.299 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:41.299 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:41.299 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:41.299 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:41.299 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 464 2024-09-17T21:18:41.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.299 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:41.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:41.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:18:41.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:41.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:18:41.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:41.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196838+0000", 2024-09-17T21:18:41.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4970000000000004e-06 2024-09-17T21:18:41.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:41.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196859+0000", 2024-09-17T21:18:41.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1166e-05 2024-09-17T21:18:41.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:41.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196879+0000", 2024-09-17T21:18:41.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.995e-05 2024-09-17T21:18:41.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:41.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196896+0000", 2024-09-17T21:18:41.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6929999999999999e-05 2024-09-17T21:18:41.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:41.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240617+0000", 2024-09-17T21:18:41.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043721784999999999 2024-09-17T21:18:41.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:41.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240702+0000", 2024-09-17T21:18:41.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4060999999999994e-05 2024-09-17T21:18:41.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:41.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241482+0000", 2024-09-17T21:18:41.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000780304 2024-09-17T21:18:41.303 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.303 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:41.303 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.303 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:475 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:41.304 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:18:41.304 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.070360009, 2024-09-17T21:18:41.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044661981000000003, 2024-09-17T21:18:41.304 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:41.304 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:41.304 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:41.304 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:41.304 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:41.304 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:41.304 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 475 2024-09-17T21:18:41.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.304 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:41.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:41.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:18:41.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:41.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:18:41.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:41.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197100+0000", 2024-09-17T21:18:41.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.206e-06 2024-09-17T21:18:41.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:41.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197105+0000", 2024-09-17T21:18:41.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.527e-06 2024-09-17T21:18:41.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:41.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197124+0000", 2024-09-17T21:18:41.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:18:41.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:41.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197136+0000", 2024-09-17T21:18:41.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2857000000000001e-05 2024-09-17T21:18:41.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:41.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241539+0000", 2024-09-17T21:18:41.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044402077999999998 2024-09-17T21:18:41.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:41.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241653+0000", 2024-09-17T21:18:41.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114931 2024-09-17T21:18:41.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:41.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241761+0000", 2024-09-17T21:18:41.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107753 2024-09-17T21:18:41.308 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.308 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:41.308 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.308 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:476 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:41.308 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:18:41.308 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.070306184, 2024-09-17T21:18:41.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045038976000000001, 2024-09-17T21:18:41.308 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:41.309 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:41.309 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:41.309 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:41.309 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:41.309 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:41.309 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 476 2024-09-17T21:18:41.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.309 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:41.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.309 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:41.309 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:18:41.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:41.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:18:41.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:41.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197154+0000", 2024-09-17T21:18:41.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.127e-06 2024-09-17T21:18:41.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:41.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197159+0000", 2024-09-17T21:18:41.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4569999999999998e-06 2024-09-17T21:18:41.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:41.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197174+0000", 2024-09-17T21:18:41.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5173e-05 2024-09-17T21:18:41.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:41.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197186+0000", 2024-09-17T21:18:41.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:18:41.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:41.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241863+0000", 2024-09-17T21:18:41.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044676826000000003 2024-09-17T21:18:41.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:41.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241998+0000", 2024-09-17T21:18:41.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013476100000000001 2024-09-17T21:18:41.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:41.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242192+0000", 2024-09-17T21:18:41.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019436900000000001 2024-09-17T21:18:41.312 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.313 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:41.313 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.313 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:494 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:41.313 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:18:41.313 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.070131001, 2024-09-17T21:18:41.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046244995999999997, 2024-09-17T21:18:41.313 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:41.313 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:41.313 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:41.313 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:41.313 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:41.313 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:41.314 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 494 2024-09-17T21:18:41.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.314 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:41.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:41.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:18:41.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:41.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:18:41.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:41.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197329+0000", 2024-09-17T21:18:41.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.173e-06 2024-09-17T21:18:41.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:41.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197334+0000", 2024-09-17T21:18:41.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0499999999999999e-06 2024-09-17T21:18:41.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:41.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197352+0000", 2024-09-17T21:18:41.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7133e-05 2024-09-17T21:18:41.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:41.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197367+0000", 2024-09-17T21:18:41.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5123000000000001e-05 2024-09-17T21:18:41.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:41.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:18:41.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044989829000000002 2024-09-17T21:18:41.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:41.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242526+0000", 2024-09-17T21:18:41.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016968199999999999 2024-09-17T21:18:41.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:41.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243573+0000", 2024-09-17T21:18:41.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010470060000000001 2024-09-17T21:18:41.317 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.317 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:41.317 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.318 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:502 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:41.318 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:18:41.318 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.067897342, 2024-09-17T21:18:41.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044346086999999999, 2024-09-17T21:18:41.318 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:41.318 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:41.318 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:41.318 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:41.318 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:41.318 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:41.318 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 502 2024-09-17T21:18:41.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.318 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:41.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:41.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:18:41.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:41.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:18:41.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:41.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199563+0000", 2024-09-17T21:18:41.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2929999999999999e-06 2024-09-17T21:18:41.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:41.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199569+0000", 2024-09-17T21:18:41.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5330000000000002e-06 2024-09-17T21:18:41.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:41.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199614+0000", 2024-09-17T21:18:41.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4793000000000003e-05 2024-09-17T21:18:41.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:41.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199628+0000", 2024-09-17T21:18:41.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4037e-05 2024-09-17T21:18:41.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:41.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243665+0000", 2024-09-17T21:18:41.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044037106999999999 2024-09-17T21:18:41.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:41.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243767+0000", 2024-09-17T21:18:41.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010279499999999999 2024-09-17T21:18:41.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:41.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243908+0000", 2024-09-17T21:18:41.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140529 2024-09-17T21:18:41.322 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.322 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:41.322 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.322 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:523 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:41.322 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:18:41.322 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.067451189, 2024-09-17T21:18:41.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044789645000000003, 2024-09-17T21:18:41.322 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:41.323 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:41.323 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:41.323 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:41.323 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:41.323 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:41.323 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 523 2024-09-17T21:18:41.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.323 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:41.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:41.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:18:41.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:41.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:18:41.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:41.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:18:41.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9e-06 2024-09-17T21:18:41.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:41.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200021+0000", 2024-09-17T21:18:41.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9499999999999998e-06 2024-09-17T21:18:41.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:41.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200038+0000", 2024-09-17T21:18:41.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.73e-05 2024-09-17T21:18:41.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:41.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200051+0000", 2024-09-17T21:18:41.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.322e-05 2024-09-17T21:18:41.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:41.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243987+0000", 2024-09-17T21:18:41.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043935386999999999 2024-09-17T21:18:41.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:41.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244136+0000", 2024-09-17T21:18:41.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014868300000000001 2024-09-17T21:18:41.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:41.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244798+0000", 2024-09-17T21:18:41.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00066220500000000002 2024-09-17T21:18:41.326 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.326 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:41.327 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.327 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:524 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:41.327 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:18:41.327 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.067384972999999, 2024-09-17T21:18:41.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045737414999999997, 2024-09-17T21:18:41.327 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:41.327 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:41.327 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:41.327 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:41.327 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:41.327 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:41.327 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 524 2024-09-17T21:18:41.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.328 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:41.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:41.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:18:41.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:41.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:18:41.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:41.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200076+0000", 2024-09-17T21:18:41.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:18:41.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:41.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200081+0000", 2024-09-17T21:18:41.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7030000000000003e-06 2024-09-17T21:18:41.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:41.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200099+0000", 2024-09-17T21:18:41.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7836999999999999e-05 2024-09-17T21:18:41.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:41.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200113+0000", 2024-09-17T21:18:41.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3570000000000001e-05 2024-09-17T21:18:41.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:41.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244866+0000", 2024-09-17T21:18:41.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044753573999999997 2024-09-17T21:18:41.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:41.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244972+0000", 2024-09-17T21:18:41.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106111 2024-09-17T21:18:41.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:41.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245812+0000", 2024-09-17T21:18:41.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083923999999999999 2024-09-17T21:18:41.331 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.331 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:41.331 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.331 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:525 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3670016~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:41.331 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:18:41.332 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.067233591000001, 2024-09-17T21:18:41.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046645697, 2024-09-17T21:18:41.332 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:41.332 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:41.332 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:41.332 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:41.332 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:41.332 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:41.332 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 525 2024-09-17T21:18:41.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.332 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:41.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:41.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:18:41.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:41.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:18:41.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:41.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:18:41.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:18:41.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:41.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200234+0000", 2024-09-17T21:18:41.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:18:41.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:41.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200254+0000", 2024-09-17T21:18:41.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9959999999999999e-05 2024-09-17T21:18:41.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:41.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200293+0000", 2024-09-17T21:18:41.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9755999999999997e-05 2024-09-17T21:18:41.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:41.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245871+0000", 2024-09-17T21:18:41.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045577871999999998 2024-09-17T21:18:41.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:41.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246030+0000", 2024-09-17T21:18:41.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158975 2024-09-17T21:18:41.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:41.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246871+0000", 2024-09-17T21:18:41.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084111699999999995 2024-09-17T21:18:41.336 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.336 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:41.336 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.336 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:41.336 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:18:41.337 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.066987916, 2024-09-17T21:18:41.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:18:41.337 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:41.337 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:41.337 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:41.337 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:41.337 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:41.337 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:41.337 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:18:41.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.337 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:41.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:41.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:18:41.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:41.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:18:41.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:41.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:18:41.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:18:41.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:41.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:18:41.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:18:41.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:41.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:18:41.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:18:41.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:41.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:18:41.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:18:41.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:41.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:18:41.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:18:41.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:41.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:18:41.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:18:41.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:41.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:18:41.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:18:41.341 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.341 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:41.341 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.341 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:41.341 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:18:41.341 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.066809234000001, 2024-09-17T21:18:41.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:18:41.341 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:41.342 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:41.342 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:41.342 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:41.342 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:41.342 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:41.342 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:18:41.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.342 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:41.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:41.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:18:41.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:41.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:18:41.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:41.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:18:41.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:18:41.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:41.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:18:41.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:18:41.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:41.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:18:41.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:18:41.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:41.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:18:41.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:18:41.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:41.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:18:41.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:18:41.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:41.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:18:41.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:18:41.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:41.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:18:41.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:18:41.345 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.345 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:41.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.346 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:41.346 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:18:41.346 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.066749960999999, 2024-09-17T21:18:41.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:18:41.346 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:41.346 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:41.346 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:41.346 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:41.346 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:41.346 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:41.347 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:18:41.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.347 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:41.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:41.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:18:41.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:41.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:18:41.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:41.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:18:41.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:18:41.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:41.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:18:41.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:18:41.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:41.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:18:41.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:18:41.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:41.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:18:41.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:18:41.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:41.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:18:41.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:18:41.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:41.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:18:41.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:18:41.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:41.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:18:41.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:18:41.350 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.350 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:41.350 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.350 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:41.350 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:18:41.351 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.066607766000001, 2024-09-17T21:18:41.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:18:41.351 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:41.351 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:41.351 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:41.351 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:41.351 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:41.351 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:41.351 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:18:41.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.351 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:41.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:41.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:18:41.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:41.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:18:41.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:41.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:18:41.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:18:41.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:41.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:18:41.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:18:41.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:41.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:18:41.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:18:41.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:41.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:18:41.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:18:41.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:41.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:18:41.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:18:41.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:41.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:18:41.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:18:41.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:41.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:18:41.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:18:41.354 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.354 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:41.355 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.355 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:41.355 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:18:41.355 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.066551842999999, 2024-09-17T21:18:41.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:18:41.355 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:41.355 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:41.355 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:41.355 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:41.355 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:41.355 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:41.356 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:18:41.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.356 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:41.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:41.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:18:41.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:41.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:18:41.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:41.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:18:41.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:18:41.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:41.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:18:41.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:18:41.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:41.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:18:41.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:18:41.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:41.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:18:41.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:18:41.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:41.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:18:41.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:18:41.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:41.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:18:41.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:18:41.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:41.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:18:41.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:18:41.359 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.359 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:41.359 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.359 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:41.359 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:18:41.359 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.066498226, 2024-09-17T21:18:41.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:18:41.360 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:41.360 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:41.360 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:41.360 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:41.360 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:41.360 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:41.360 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:18:41.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.360 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:41.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:41.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:18:41.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:41.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:18:41.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:41.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:41.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:18:41.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:18:41.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:41.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:18:41.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:18:41.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:41.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:18:41.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:18:41.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:41.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:18:41.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:18:41.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:41.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:18:41.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:18:41.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:41.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:18:41.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:18:41.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:41.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:41.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:41.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:18:41.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:18:41.363 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.363 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:41.364 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.364 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:41.364 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:41.364 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:41.365 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:18:41.366 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:41.360+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:41.466 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:41.460+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:41.534 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:41.535 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:41.535 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:41.535 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:41.549 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:18:41.567 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:41.562+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:41.569 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:18:41.668 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:41.662+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:41.710 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:41.710 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:41.711 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:41.711 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:41.711 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:41.726 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:18:41.731 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:18:41.731 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:18:41.749 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:18:41.749 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:18:41.767 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:18:41.768 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:18:41.768 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:41.764+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:41.788 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:18:41.788 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:18:41.812 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:18:41.812 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:18:41.843 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:18:41.843 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:18:41.851 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:41.852 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:41.868 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:18:41.870 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:41.866+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:41.971 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:41.966+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:42.028 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:42.028 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:42.042 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:18:42.071 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:42.066+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:42.160 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:42.161 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:42.172 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:42.168+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:42.175 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:18:42.273 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:42.266+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:42.348 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:42.348 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:42.362 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:18:42.374 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:42.368+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:42.474 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:42.470+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:42.498 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-17T21:18:42.499 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:42.500 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:42.515 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:18:42.576 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:42.568+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:42.677 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:42.672+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:42.689 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:42.690 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:42.707 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:18:42.778 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:42.773+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:42.831 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:18:42.854 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:42.854 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:42.870 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:18:42.879 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:42.872+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:42.980 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:42.976+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:43.042 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:43.042 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:43.057 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:18:43.105 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:43.101+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:43.113 INFO:teuthology.orchestra.run.smithi184.stdout:873 read 0x7f80a00 thru 0x7f8d442 (0xca43 bytes) 2024-09-17T21:18:43.115 INFO:teuthology.orchestra.run.smithi184.stdout:874 write 0x4818000 thru 0x4819ea8 (0x1ea9 bytes) 2024-09-17T21:18:43.117 INFO:teuthology.orchestra.run.smithi184.stdout:875 read 0x4a3200 thru 0x4a8105 (0x4f06 bytes) 2024-09-17T21:18:43.119 INFO:teuthology.orchestra.run.smithi184.stdout:876 read 0x1fbba00 thru 0x1fc3f05 (0x8506 bytes) 2024-09-17T21:18:43.120 INFO:teuthology.orchestra.run.smithi184.stdout:878 write 0x6a25000 thru 0x6a32368 (0xd369 bytes) 2024-09-17T21:18:43.123 INFO:teuthology.orchestra.run.smithi184.stdout:879 write 0x48ab800 thru 0x48ae1ed (0x29ee bytes) 2024-09-17T21:18:43.124 INFO:teuthology.orchestra.run.smithi184.stdout:881 read 0x42c8000 thru 0x42c8a3b (0xa3c bytes) 2024-09-17T21:18:43.159 INFO:teuthology.orchestra.run.smithi184.stdout:883 trunc from 0x934d800 to 0x2f6b000 2024-09-17T21:18:43.182 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:43.176+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:43.184 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:43.184 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:43.198 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:18:43.268 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-17T21:18:43.285 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:43.281+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:43.313 INFO:teuthology.orchestra.run.smithi184.stdout:884 write 0x854200 thru 0x8612d8 (0xd0d9 bytes) 2024-09-17T21:18:43.321 INFO:teuthology.orchestra.run.smithi184.stdout:885 trunc from 0x2f6b000 to 0x52cfa00 2024-09-17T21:18:43.349 INFO:teuthology.orchestra.run.smithi184.stdout:887 punch from 0x38d8c00 to 0x38dac00, (0x2000 bytes) 2024-09-17T21:18:43.351 INFO:teuthology.orchestra.run.smithi184.stdout:888 read 0x4116400 thru 0x411a025 (0x3c26 bytes) 2024-09-17T21:18:43.351 INFO:teuthology.orchestra.run.smithi184.stdout:889 read 0x13bd400 thru 0x13bfae0 (0x26e1 bytes) 2024-09-17T21:18:43.367 INFO:teuthology.orchestra.run.smithi184.stdout:890 trunc from 0x52cfa00 to 0x1749400 2024-09-17T21:18:43.376 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:43.377 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:43.383 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:43.377+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:43.392 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:18:43.403 INFO:teuthology.orchestra.run.smithi184.stdout:893 write 0xf00400 thru 0xf097fe (0x93ff bytes) 2024-09-17T21:18:43.406 INFO:teuthology.orchestra.run.smithi184.stdout:894 read 0x1262600 thru 0x126f528 (0xcf29 bytes) 2024-09-17T21:18:43.449 INFO:teuthology.orchestra.run.smithi184.stdout:895 write 0x6cca800 thru 0x6cd9fb5 (0xf7b6 bytes) 2024-09-17T21:18:43.484 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:43.480+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:43.527 INFO:teuthology.orchestra.run.smithi184.stdout:896 write 0xc305600 thru 0xc3155d8 (0xffd9 bytes) 2024-09-17T21:18:43.574 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:43.574 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:43.578 INFO:teuthology.orchestra.run.smithi184.stdout:900 write 0xd6d3000 thru 0xd6d4fa8 (0x1fa9 bytes) 2024-09-17T21:18:43.580 INFO:teuthology.orchestra.run.smithi184.stdout:901 write 0x6d15400 thru 0x6d1a509 (0x510a bytes) 2024-09-17T21:18:43.582 INFO:teuthology.orchestra.run.smithi184.stdout:903 trunc from 0xd6d5000 to 0xca58400 2024-09-17T21:18:43.585 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:43.577+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:43.590 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:18:43.618 INFO:teuthology.orchestra.run.smithi184.stdout:905 read 0x9070200 thru 0x9070e4a (0xc4b bytes) 2024-09-17T21:18:43.619 INFO:teuthology.orchestra.run.smithi184.stdout:906 read 0x245ca00 thru 0x2468fe6 (0xc5e7 bytes) 2024-09-17T21:18:43.619 INFO:teuthology.orchestra.run.smithi184.stdout:907 write 0x3527e00 thru 0x3534b47 (0xcd48 bytes) 2024-09-17T21:18:43.653 INFO:teuthology.orchestra.run.smithi184.stdout:908 write 0xe83d400 thru 0xe84a1d0 (0xcdd1 bytes) 2024-09-17T21:18:43.670 INFO:teuthology.orchestra.run.smithi184.stdout:909 trunc from 0xe84a200 to 0x4b96600 2024-09-17T21:18:43.685 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:43.681+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:43.708 INFO:teuthology.orchestra.run.smithi184.stdout:910 read 0x1382400 thru 0x138e5ed (0xc1ee bytes) 2024-09-17T21:18:43.733 INFO:teuthology.orchestra.run.smithi184.stdout:911 trunc from 0x4b96600 to 0xe95f000 2024-09-17T21:18:43.749 INFO:teuthology.orchestra.run.smithi184.stdout:912 read 0xd224400 thru 0xd228009 (0x3c0a bytes) 2024-09-17T21:18:43.765 INFO:teuthology.orchestra.run.smithi184.stdout:914 trunc from 0xe95f000 to 0x4f88000 2024-09-17T21:18:43.765 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:43.765 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:43.783 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:18:43.790 INFO:teuthology.orchestra.run.smithi184.stdout:915 read 0xacd200 thru 0xad16e8 (0x44e9 bytes) 2024-09-17T21:18:43.793 INFO:teuthology.orchestra.run.smithi184.stdout:917 flatten 2024-09-17T21:18:43.826 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:43.821+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:43.863 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:18:43.874 INFO:teuthology.orchestra.run.smithi184.stdout:921 write 0x2d5be00 thru 0x2d61970 (0x5b71 bytes) 2024-09-17T21:18:43.876 INFO:teuthology.orchestra.run.smithi184.stdout:923 punch from 0x53c000 to 0x548800, (0xc800 bytes) 2024-09-17T21:18:43.878 INFO:teuthology.orchestra.run.smithi184.stdout:924 punch from 0x3dbd400 to 0x3dc8e00, (0xba00 bytes) 2024-09-17T21:18:43.887 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:43.881+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:43.892 INFO:teuthology.orchestra.run.smithi184.stdout:925 trunc from 0x4f88000 to 0xc0fda00 2024-09-17T21:18:43.907 INFO:teuthology.orchestra.run.smithi184.stdout:926 read 0x8e4fa00 thru 0x8e5d7b4 (0xddb5 bytes) 2024-09-17T21:18:43.908 INFO:teuthology.orchestra.run.smithi184.stdout:927 read 0x3b1c00 thru 0x3b2d4b (0x114c bytes) 2024-09-17T21:18:43.926 INFO:teuthology.orchestra.run.smithi184.stdout:930 trunc from 0xc0fda00 to 0x39efe00 2024-09-17T21:18:43.956 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:43.956 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:43.959 INFO:teuthology.orchestra.run.smithi184.stdout:931 read 0x89de00 thru 0x89f148 (0x1349 bytes) 2024-09-17T21:18:43.971 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:18:43.991 INFO:teuthology.orchestra.run.smithi184.stdout:933 trunc from 0x39efe00 to 0xed53400 2024-09-17T21:18:44.017 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:44.013+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:44.018 INFO:teuthology.orchestra.run.smithi184.stdout:934 trunc from 0xed53400 to 0x8509000 2024-09-17T21:18:44.022 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:18:44.023 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:18:44.033 INFO:teuthology.orchestra.run.smithi184.stdout:937 punch from 0x13e1a00 to 0x13ee000, (0xc600 bytes) 2024-09-17T21:18:44.040 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:18:44.040 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:18:44.055 INFO:teuthology.orchestra.run.smithi184.stdout:938 write 0x974f000 thru 0x9756072 (0x7073 bytes) 2024-09-17T21:18:44.056 INFO:teuthology.orchestra.run.smithi184.stdout:939 read 0x3c1c00 thru 0x3cfe81 (0xe282 bytes) 2024-09-17T21:18:44.057 INFO:teuthology.orchestra.run.smithi184.stdout:941 write 0x7883800 thru 0x7884bb2 (0x13b3 bytes) 2024-09-17T21:18:44.057 INFO:teuthology.orchestra.run.smithi184.stdout:944 read 0x8387e00 thru 0x83919e9 (0x9bea bytes) 2024-09-17T21:18:44.058 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:18:44.058 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:18:44.078 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:18:44.078 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:18:44.089 INFO:teuthology.orchestra.run.smithi184.stdout:945 write 0xd160600 thru 0xd16370b (0x310c bytes) 2024-09-17T21:18:44.089 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:44.084+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:44.091 INFO:teuthology.orchestra.run.smithi184.stdout:947 punch from 0xbce5e00 to 0xbcf5200, (0xf400 bytes) 2024-09-17T21:18:44.095 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:44.096 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:44.096 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:44.096 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:44.098 INFO:teuthology.orchestra.run.smithi184.stdout:948 trunc from 0xd163800 to 0xa519800 2024-09-17T21:18:44.101 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:18:44.101 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:18:44.112 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:18:44.122 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:18:44.122 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:18:44.130 INFO:teuthology.orchestra.run.smithi184.stdout:950 punch from 0x8345a00 to 0x834ec00, (0x9200 bytes) 2024-09-17T21:18:44.131 INFO:teuthology.orchestra.run.smithi184.stdout:951 punch from 0x32a1800 to 0x32a2c00, (0x1400 bytes) 2024-09-17T21:18:44.131 INFO:teuthology.orchestra.run.smithi184.stdout:954 read 0x66f6c00 thru 0x66fce7d (0x627e bytes) 2024-09-17T21:18:44.138 INFO:teuthology.orchestra.run.smithi184.stdout:955 trunc from 0xa519800 to 0xc86a200 2024-09-17T21:18:44.156 INFO:teuthology.orchestra.run.smithi184.stdout:957 read 0xbed200 thru 0xbf6847 (0x9648 bytes) 2024-09-17T21:18:44.157 INFO:teuthology.orchestra.run.smithi184.stdout:960 punch from 0x6578000 to 0x657fa00, (0x7a00 bytes) 2024-09-17T21:18:44.157 INFO:teuthology.orchestra.run.smithi184.stdout:962 punch from 0x1c93400 to 0x1c9b400, (0x8000 bytes) 2024-09-17T21:18:44.157 INFO:teuthology.orchestra.run.smithi184.stdout:963 read 0x4504200 thru 0x450c036 (0x7e37 bytes) 2024-09-17T21:18:44.179 INFO:teuthology.orchestra.run.smithi184.stdout:964 trunc from 0xc86a200 to 0x1e15800 2024-09-17T21:18:44.190 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:44.184+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:44.219 INFO:teuthology.orchestra.run.smithi184.stdout:965 read 0xbaf000 thru 0xbbdd41 (0xed42 bytes) 2024-09-17T21:18:44.220 INFO:teuthology.orchestra.run.smithi184.stdout:970 read 0x491000 thru 0x49225d (0x125e bytes) 2024-09-17T21:18:44.220 INFO:teuthology.orchestra.run.smithi184.stdout:971 punch from 0xfcdc00 to 0xfd6600, (0x8a00 bytes) 2024-09-17T21:18:44.246 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:44.247 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:44.247 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:44.247 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:44.247 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:44.253 INFO:teuthology.orchestra.run.smithi184.stdout:972 trunc from 0x1e15800 to 0xe880e00 2024-09-17T21:18:44.260 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:18:44.279 INFO:teuthology.orchestra.run.smithi184.stdout:974 read 0xb429000 thru 0xb431603 (0x8604 bytes) 2024-09-17T21:18:44.283 INFO:teuthology.orchestra.run.smithi184.stdout:977 punch from 0x3287600 to 0x328a000, (0x2a00 bytes) 2024-09-17T21:18:44.283 INFO:teuthology.orchestra.run.smithi184.stdout:978 write 0x927a000 thru 0x9280b97 (0x6b98 bytes) 2024-09-17T21:18:44.284 INFO:teuthology.orchestra.run.smithi184.stdout:979 punch from 0x3e02600 to 0x3e03400, (0xe00 bytes) 2024-09-17T21:18:44.284 INFO:teuthology.orchestra.run.smithi184.stdout:980 write 0x58ebc00 thru 0x58f1733 (0x5b34 bytes) 2024-09-17T21:18:44.285 INFO:teuthology.orchestra.run.smithi184.stdout:983 write 0x59a4000 thru 0x59a701c (0x301d bytes) 2024-09-17T21:18:44.285 INFO:teuthology.orchestra.run.smithi184.stdout:984 read 0x7409200 thru 0x74119e3 (0x87e4 bytes) 2024-09-17T21:18:44.285 INFO:teuthology.orchestra.run.smithi184.stdout:985 punch from 0x884a800 to 0x8850e00, (0x6600 bytes) 2024-09-17T21:18:44.286 INFO:teuthology.orchestra.run.smithi184.stdout:987 read 0xe55a00 thru 0xe635ab (0xdbac bytes) 2024-09-17T21:18:44.286 INFO:teuthology.orchestra.run.smithi184.stdout:993 read 0x5ae200 thru 0x5b0913 (0x2714 bytes) 2024-09-17T21:18:44.286 INFO:teuthology.orchestra.run.smithi184.stdout:994 write 0x91fe000 thru 0x9207c59 (0x9c5a bytes) 2024-09-17T21:18:44.286 INFO:teuthology.orchestra.run.smithi184.stdout:995 write 0x2d0de00 thru 0x2d0f374 (0x1575 bytes) 2024-09-17T21:18:44.288 INFO:teuthology.orchestra.run.smithi184.stdout:996 trunc from 0xe880e00 to 0xdd41a00 2024-09-17T21:18:44.291 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:44.285+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:44.330 INFO:teuthology.orchestra.run.smithi184.stdout:997 read 0x3baa600 thru 0x3baabd0 (0x5d1 bytes) 2024-09-17T21:18:44.332 INFO:teuthology.orchestra.run.smithi184.stdout:998 write 0x9c7e800 thru 0x9c8d84d (0xf04e bytes) 2024-09-17T21:18:44.333 INFO:teuthology.orchestra.run.smithi184.stdout:999 write 0x120fa00 thru 0x1213835 (0x3e36 bytes) 2024-09-17T21:18:44.333 INFO:teuthology.orchestra.run.smithi184.stdout:1000 punch from 0x8446e00 to 0x844de00, (0x7000 bytes) 2024-09-17T21:18:44.333 INFO:teuthology.orchestra.run.smithi184.stdout:1002 read 0x8015200 thru 0x8020ecd (0xbcce bytes) 2024-09-17T21:18:44.334 INFO:teuthology.orchestra.run.smithi184.stdout:1003 punch from 0x8ff4200 to 0x8ff7000, (0x2e00 bytes) 2024-09-17T21:18:44.334 INFO:teuthology.orchestra.run.smithi184.stdout:1004 read 0x4bfac00 thru 0x4c0521b (0xa61c bytes) 2024-09-17T21:18:44.334 INFO:teuthology.orchestra.run.smithi184.stdout:1006 read 0x6499c00 thru 0x64a6055 (0xc456 bytes) 2024-09-17T21:18:44.335 INFO:teuthology.orchestra.run.smithi184.stdout:1008 write 0xd963000 thru 0xd96eaca (0xbacb bytes) 2024-09-17T21:18:44.367 INFO:teuthology.orchestra.run.smithi184.stdout:1009 write 0xe745000 thru 0xe74f348 (0xa349 bytes) 2024-09-17T21:18:44.370 INFO:teuthology.orchestra.run.smithi184.stdout:1010 read 0xd9daa00 thru 0xd9df517 (0x4b18 bytes) 2024-09-17T21:18:44.370 INFO:teuthology.orchestra.run.smithi184.stdout:1013 write 0xab0a800 thru 0xab1440b (0x9c0c bytes) 2024-09-17T21:18:44.375 INFO:teuthology.orchestra.run.smithi184.stdout:1020 trunc from 0xe74f400 to 0xc0ab200 2024-09-17T21:18:44.392 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:44.385+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:44.420 INFO:teuthology.orchestra.run.smithi184.stdout:1021 clone 10 order 24 su 2097152 sc 15 2024-09-17T21:18:44.433 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:44.433 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:18:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:18:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:18:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:389 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.194936+0000", 2024-09-17T21:18:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.235989007000001, 2024-09-17T21:18:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041176510999999999, 2024-09-17T21:18:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 389 2024-09-17T21:18:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194936+0000", 2024-09-17T21:18:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194936+0000", 2024-09-17T21:18:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194937+0000", 2024-09-17T21:18:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6270000000000001e-06 2024-09-17T21:18:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194944+0000", 2024-09-17T21:18:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5729999999999996e-06 2024-09-17T21:18:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194961+0000", 2024-09-17T21:18:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7623000000000001e-05 2024-09-17T21:18:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194974+0000", 2024-09-17T21:18:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2989999999999999e-05 2024-09-17T21:18:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.235848+0000", 2024-09-17T21:18:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.040874099999999997 2024-09-17T21:18:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.235971+0000", 2024-09-17T21:18:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012254400000000001 2024-09-17T21:18:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236112+0000", 2024-09-17T21:18:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014105399999999999 2024-09-17T21:18:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:406 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 2359296~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:18:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.235870201000001, 2024-09-17T21:18:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042448018999999997, 2024-09-17T21:18:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 406 2024-09-17T21:18:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:18:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:18:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195056+0000", 2024-09-17T21:18:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2729999999999999e-06 2024-09-17T21:18:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195060+0000", 2024-09-17T21:18:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5399999999999997e-06 2024-09-17T21:18:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195076+0000", 2024-09-17T21:18:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6217000000000001e-05 2024-09-17T21:18:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:44.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195093+0000", 2024-09-17T21:18:44.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.683e-05 2024-09-17T21:18:44.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.443 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236217+0000", 2024-09-17T21:18:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041124102000000003 2024-09-17T21:18:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236339+0000", 2024-09-17T21:18:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121876 2024-09-17T21:18:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237502+0000", 2024-09-17T21:18:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011631809999999999 2024-09-17T21:18:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:416 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2228224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:18:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.235372174999998, 2024-09-17T21:18:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042361990000000002, 2024-09-17T21:18:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 416 2024-09-17T21:18:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:18:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:18:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195554+0000", 2024-09-17T21:18:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9300000000000002e-06 2024-09-17T21:18:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195563+0000", 2024-09-17T21:18:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2770000000000005e-06 2024-09-17T21:18:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195586+0000", 2024-09-17T21:18:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3889000000000001e-05 2024-09-17T21:18:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195603+0000", 2024-09-17T21:18:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6084000000000001e-05 2024-09-17T21:18:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237563+0000", 2024-09-17T21:18:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041960856999999997 2024-09-17T21:18:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237695+0000", 2024-09-17T21:18:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013144400000000001 2024-09-17T21:18:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237914+0000", 2024-09-17T21:18:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000219509 2024-09-17T21:18:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:439 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:18:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.234808272999999, 2024-09-17T21:18:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043053811999999997, 2024-09-17T21:18:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 439 2024-09-17T21:18:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:18:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:18:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196118+0000", 2024-09-17T21:18:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3969999999999999e-06 2024-09-17T21:18:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196157+0000", 2024-09-17T21:18:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9826999999999997e-05 2024-09-17T21:18:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196179+0000", 2024-09-17T21:18:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1756e-05 2024-09-17T21:18:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196194+0000", 2024-09-17T21:18:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4707e-05 2024-09-17T21:18:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237980+0000", 2024-09-17T21:18:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041786219999999999 2024-09-17T21:18:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.238065+0000", 2024-09-17T21:18:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4560000000000004e-05 2024-09-17T21:18:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239170+0000", 2024-09-17T21:18:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001105345 2024-09-17T21:18:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:440 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:18:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.234705770000001, 2024-09-17T21:18:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043890039999999998, 2024-09-17T21:18:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 440 2024-09-17T21:18:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:18:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:18:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196220+0000", 2024-09-17T21:18:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.254e-06 2024-09-17T21:18:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196225+0000", 2024-09-17T21:18:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7829999999999998e-06 2024-09-17T21:18:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196241+0000", 2024-09-17T21:18:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6322999999999999e-05 2024-09-17T21:18:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196258+0000", 2024-09-17T21:18:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7056000000000001e-05 2024-09-17T21:18:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239229+0000", 2024-09-17T21:18:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042971058999999999 2024-09-17T21:18:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239403+0000", 2024-09-17T21:18:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017399200000000001 2024-09-17T21:18:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240109+0000", 2024-09-17T21:18:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00070557300000000003 2024-09-17T21:18:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:450 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:18:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.234344819, 2024-09-17T21:18:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043994521000000002, 2024-09-17T21:18:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 450 2024-09-17T21:18:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:18:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:18:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196581+0000", 2024-09-17T21:18:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4500000000000001e-06 2024-09-17T21:18:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196588+0000", 2024-09-17T21:18:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7530000000000004e-06 2024-09-17T21:18:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196607+0000", 2024-09-17T21:18:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8987e-05 2024-09-17T21:18:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196620+0000", 2024-09-17T21:18:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3589e-05 2024-09-17T21:18:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240225+0000", 2024-09-17T21:18:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043604878999999999 2024-09-17T21:18:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240377+0000", 2024-09-17T21:18:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015146699999999999 2024-09-17T21:18:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240574+0000", 2024-09-17T21:18:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019739599999999999 2024-09-17T21:18:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:464 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:18:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.234094398, 2024-09-17T21:18:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044651692999999999, 2024-09-17T21:18:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 464 2024-09-17T21:18:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:18:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:18:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196838+0000", 2024-09-17T21:18:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4970000000000004e-06 2024-09-17T21:18:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196859+0000", 2024-09-17T21:18:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1166e-05 2024-09-17T21:18:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196879+0000", 2024-09-17T21:18:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.995e-05 2024-09-17T21:18:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196896+0000", 2024-09-17T21:18:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6929999999999999e-05 2024-09-17T21:18:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240617+0000", 2024-09-17T21:18:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043721784999999999 2024-09-17T21:18:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240702+0000", 2024-09-17T21:18:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4060999999999994e-05 2024-09-17T21:18:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241482+0000", 2024-09-17T21:18:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000780304 2024-09-17T21:18:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:475 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:18:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.233825266, 2024-09-17T21:18:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044661981000000003, 2024-09-17T21:18:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 475 2024-09-17T21:18:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:18:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:18:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197100+0000", 2024-09-17T21:18:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.206e-06 2024-09-17T21:18:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197105+0000", 2024-09-17T21:18:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.527e-06 2024-09-17T21:18:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197124+0000", 2024-09-17T21:18:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:18:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197136+0000", 2024-09-17T21:18:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2857000000000001e-05 2024-09-17T21:18:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241539+0000", 2024-09-17T21:18:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044402077999999998 2024-09-17T21:18:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241653+0000", 2024-09-17T21:18:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114931 2024-09-17T21:18:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241761+0000", 2024-09-17T21:18:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107753 2024-09-17T21:18:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:476 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:18:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.233771440999998, 2024-09-17T21:18:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045038976000000001, 2024-09-17T21:18:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 476 2024-09-17T21:18:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:18:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:18:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197154+0000", 2024-09-17T21:18:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.127e-06 2024-09-17T21:18:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197159+0000", 2024-09-17T21:18:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4569999999999998e-06 2024-09-17T21:18:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197174+0000", 2024-09-17T21:18:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5173e-05 2024-09-17T21:18:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197186+0000", 2024-09-17T21:18:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:18:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241863+0000", 2024-09-17T21:18:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044676826000000003 2024-09-17T21:18:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241998+0000", 2024-09-17T21:18:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013476100000000001 2024-09-17T21:18:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242192+0000", 2024-09-17T21:18:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019436900000000001 2024-09-17T21:18:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:494 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:18:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.233596257999999, 2024-09-17T21:18:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046244995999999997, 2024-09-17T21:18:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 494 2024-09-17T21:18:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:18:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:18:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197329+0000", 2024-09-17T21:18:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.173e-06 2024-09-17T21:18:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197334+0000", 2024-09-17T21:18:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0499999999999999e-06 2024-09-17T21:18:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197352+0000", 2024-09-17T21:18:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7133e-05 2024-09-17T21:18:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197367+0000", 2024-09-17T21:18:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5123000000000001e-05 2024-09-17T21:18:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:18:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044989829000000002 2024-09-17T21:18:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242526+0000", 2024-09-17T21:18:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016968199999999999 2024-09-17T21:18:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243573+0000", 2024-09-17T21:18:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010470060000000001 2024-09-17T21:18:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:502 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:18:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.231362599000001, 2024-09-17T21:18:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044346086999999999, 2024-09-17T21:18:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 502 2024-09-17T21:18:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:18:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:18:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199563+0000", 2024-09-17T21:18:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2929999999999999e-06 2024-09-17T21:18:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199569+0000", 2024-09-17T21:18:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5330000000000002e-06 2024-09-17T21:18:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199614+0000", 2024-09-17T21:18:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4793000000000003e-05 2024-09-17T21:18:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199628+0000", 2024-09-17T21:18:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4037e-05 2024-09-17T21:18:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243665+0000", 2024-09-17T21:18:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044037106999999999 2024-09-17T21:18:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243767+0000", 2024-09-17T21:18:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010279499999999999 2024-09-17T21:18:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243908+0000", 2024-09-17T21:18:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140529 2024-09-17T21:18:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:523 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:18:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.230916445999998, 2024-09-17T21:18:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044789645000000003, 2024-09-17T21:18:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 523 2024-09-17T21:18:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:18:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:18:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:18:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9e-06 2024-09-17T21:18:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200021+0000", 2024-09-17T21:18:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9499999999999998e-06 2024-09-17T21:18:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200038+0000", 2024-09-17T21:18:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.73e-05 2024-09-17T21:18:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200051+0000", 2024-09-17T21:18:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.322e-05 2024-09-17T21:18:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243987+0000", 2024-09-17T21:18:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043935386999999999 2024-09-17T21:18:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244136+0000", 2024-09-17T21:18:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014868300000000001 2024-09-17T21:18:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244798+0000", 2024-09-17T21:18:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00066220500000000002 2024-09-17T21:18:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:524 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:18:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.230850230000001, 2024-09-17T21:18:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045737414999999997, 2024-09-17T21:18:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 524 2024-09-17T21:18:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:18:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:18:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200076+0000", 2024-09-17T21:18:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:18:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200081+0000", 2024-09-17T21:18:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7030000000000003e-06 2024-09-17T21:18:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200099+0000", 2024-09-17T21:18:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7836999999999999e-05 2024-09-17T21:18:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200113+0000", 2024-09-17T21:18:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3570000000000001e-05 2024-09-17T21:18:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244866+0000", 2024-09-17T21:18:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044753573999999997 2024-09-17T21:18:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244972+0000", 2024-09-17T21:18:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106111 2024-09-17T21:18:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245812+0000", 2024-09-17T21:18:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083923999999999999 2024-09-17T21:18:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:525 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3670016~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:44.495 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:18:44.495 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.230698847999999, 2024-09-17T21:18:44.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046645697, 2024-09-17T21:18:44.495 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:44.495 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:44.495 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:44.495 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:44.495 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:44.495 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 525 2024-09-17T21:18:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:18:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:18:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:18:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:18:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200234+0000", 2024-09-17T21:18:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:18:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200254+0000", 2024-09-17T21:18:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9959999999999999e-05 2024-09-17T21:18:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200293+0000", 2024-09-17T21:18:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9755999999999997e-05 2024-09-17T21:18:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245871+0000", 2024-09-17T21:18:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045577871999999998 2024-09-17T21:18:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246030+0000", 2024-09-17T21:18:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158975 2024-09-17T21:18:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246871+0000", 2024-09-17T21:18:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084111699999999995 2024-09-17T21:18:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:18:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.230453173000001, 2024-09-17T21:18:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:18:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:18:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:18:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:18:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:18:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:18:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:18:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:18:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:18:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:18:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:18:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:18:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:18:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:18:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:18:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:18:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:18:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:18:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:18:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.230274490999999, 2024-09-17T21:18:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:18:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:18:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:18:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:18:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:18:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:18:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:18:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:18:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:18:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:18:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:18:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:18:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:18:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:18:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:18:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:18:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:18:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:18:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:18:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.230215218000001, 2024-09-17T21:18:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:18:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:18:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:18:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:18:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:18:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:18:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:18:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:18:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:18:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:18:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:18:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:18:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:18:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:18:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:18:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:18:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:18:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:18:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:44.513 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:18:44.513 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.230073022999999, 2024-09-17T21:18:44.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:18:44.513 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:44.513 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:44.513 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:44.513 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:44.513 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:44.513 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:44.514 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:18:44.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.514 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:44.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.514 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:44.514 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:18:44.514 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.514 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:44.514 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:18:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:18:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:18:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:18:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:18:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:18:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:18:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:18:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:18:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:18:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:18:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:18:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:18:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:18:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:18:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:18:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.230017100000001, 2024-09-17T21:18:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:18:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:18:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:18:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:18:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:18:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:18:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:18:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:18:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:18:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:18:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:18:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:18:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:18:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:18:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:18:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:18:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:18:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:18:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:18:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.229963482999999, 2024-09-17T21:18:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:18:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:18:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:18:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:18:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:18:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:18:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:18:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:18:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:18:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:18:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.525 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:44.525 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:18:44.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:18:44.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.525 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.525 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:44.525 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:18:44.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:18:44.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:18:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:18:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:18:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:18:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:44.527 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:44.528 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:18:44.528 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:44.524+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:44.629 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:44.621+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:44.701 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:44.701 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:44.701 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:44.701 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:44.715 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:18:44.760 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:44.756+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:44.821 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:18:44.860 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:18:44.861 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-17T21:18:44.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T21:18:44.863 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:44.863 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:44.863 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:44.863 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:44.863 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:44.863 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:18:44.864 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:44.857+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:44.964 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:44.961+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:45.025 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:45.025 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:18:45.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:18:45.025 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:18:45.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.025 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:468 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 2359296~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:45.025 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197658+0000", 2024-09-17T21:18:45.025 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.825086573, 2024-09-17T21:18:45.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041681556000000002, 2024-09-17T21:18:45.025 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.025 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.025 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:45.025 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:45.026 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:45.026 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:45.026 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 468 2024-09-17T21:18:45.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.026 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.026 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.026 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197658+0000", 2024-09-17T21:18:45.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.026 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.026 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197658+0000", 2024-09-17T21:18:45.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.027 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.027 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197659+0000", 2024-09-17T21:18:45.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.144e-06 2024-09-17T21:18:45.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.027 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.027 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197666+0000", 2024-09-17T21:18:45.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.323e-06 2024-09-17T21:18:45.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197683+0000", 2024-09-17T21:18:45.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6699999999999999e-05 2024-09-17T21:18:45.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197697+0000", 2024-09-17T21:18:45.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3713000000000001e-05 2024-09-17T21:18:45.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236385+0000", 2024-09-17T21:18:45.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.038687988 2024-09-17T21:18:45.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236571+0000", 2024-09-17T21:18:45.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000186102 2024-09-17T21:18:45.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239340+0000", 2024-09-17T21:18:45.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002768586 2024-09-17T21:18:45.030 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.030 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.030 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.030 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:491 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:45.030 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:18:45.030 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.824430694, 2024-09-17T21:18:45.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041839013000000001, 2024-09-17T21:18:45.030 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.030 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.030 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:45.031 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:45.031 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:45.031 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:45.031 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 491 2024-09-17T21:18:45.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.031 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:18:45.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:18:45.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.032 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198315+0000", 2024-09-17T21:18:45.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6599999999999996e-07 2024-09-17T21:18:45.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.032 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198317+0000", 2024-09-17T21:18:45.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6639999999999998e-06 2024-09-17T21:18:45.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198327+0000", 2024-09-17T21:18:45.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5629999999999998e-06 2024-09-17T21:18:45.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198337+0000", 2024-09-17T21:18:45.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0667e-05 2024-09-17T21:18:45.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.034 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239793+0000", 2024-09-17T21:18:45.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041455519000000003 2024-09-17T21:18:45.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.034 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239929+0000", 2024-09-17T21:18:45.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135716 2024-09-17T21:18:45.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.034 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240153+0000", 2024-09-17T21:18:45.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022421799999999999 2024-09-17T21:18:45.034 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.035 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.035 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.035 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:492 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:45.035 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:18:45.035 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.824392324000002, 2024-09-17T21:18:45.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042947165000000002, 2024-09-17T21:18:45.035 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.035 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.035 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:45.035 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:45.035 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:45.035 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:45.035 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 492 2024-09-17T21:18:45.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.036 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:18:45.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:18:45.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198354+0000", 2024-09-17T21:18:45.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:18:45.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198357+0000", 2024-09-17T21:18:45.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6969999999999999e-06 2024-09-17T21:18:45.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198367+0000", 2024-09-17T21:18:45.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8430000000000005e-06 2024-09-17T21:18:45.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.038 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.038 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198374+0000", 2024-09-17T21:18:45.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9600000000000003e-06 2024-09-17T21:18:45.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.038 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.038 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240226+0000", 2024-09-17T21:18:45.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041852157000000001 2024-09-17T21:18:45.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.039 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240409+0000", 2024-09-17T21:18:45.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018244800000000001 2024-09-17T21:18:45.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.039 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241299+0000", 2024-09-17T21:18:45.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000890554 2024-09-17T21:18:45.039 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.039 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.039 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.040 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:493 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 4063232~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:45.040 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:18:45.040 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.824355237999999, 2024-09-17T21:18:45.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043897209, 2024-09-17T21:18:45.040 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.040 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.040 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:45.040 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:45.040 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:45.040 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:45.040 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 493 2024-09-17T21:18:45.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.041 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:18:45.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:18:45.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198391+0000", 2024-09-17T21:18:45.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4699999999999999e-06 2024-09-17T21:18:45.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198396+0000", 2024-09-17T21:18:45.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:18:45.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198412+0000", 2024-09-17T21:18:45.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6507e-05 2024-09-17T21:18:45.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198425+0000", 2024-09-17T21:18:45.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.256e-05 2024-09-17T21:18:45.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241383+0000", 2024-09-17T21:18:45.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042958216 2024-09-17T21:18:45.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241525+0000", 2024-09-17T21:18:45.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014204500000000001 2024-09-17T21:18:45.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242287+0000", 2024-09-17T21:18:45.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076125800000000001 2024-09-17T21:18:45.044 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.044 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.044 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.045 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:496 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:45.045 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:18:45.045 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.824264839000001, 2024-09-17T21:18:45.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043732500000000001, 2024-09-17T21:18:45.045 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.045 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.045 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:45.045 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:45.045 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:45.045 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:45.045 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 496 2024-09-17T21:18:45.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.045 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:18:45.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:18:45.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198481+0000", 2024-09-17T21:18:45.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-17T21:18:45.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198488+0000", 2024-09-17T21:18:45.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8499999999999996e-06 2024-09-17T21:18:45.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198507+0000", 2024-09-17T21:18:45.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9276999999999999e-05 2024-09-17T21:18:45.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198521+0000", 2024-09-17T21:18:45.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3376e-05 2024-09-17T21:18:45.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241339+0000", 2024-09-17T21:18:45.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042818119000000002 2024-09-17T21:18:45.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241449+0000", 2024-09-17T21:18:45.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110209 2024-09-17T21:18:45.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.049 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.049 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242212+0000", 2024-09-17T21:18:45.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076329900000000005 2024-09-17T21:18:45.049 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.049 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.049 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.049 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:497 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:45.049 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:18:45.049 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.82418389, 2024-09-17T21:18:45.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044584278999999997, 2024-09-17T21:18:45.050 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.050 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.050 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:45.050 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:45.050 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:45.050 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:45.050 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 497 2024-09-17T21:18:45.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.050 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.050 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.050 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:18:45.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:18:45.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198562+0000", 2024-09-17T21:18:45.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-17T21:18:45.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198569+0000", 2024-09-17T21:18:45.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.494e-06 2024-09-17T21:18:45.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198588+0000", 2024-09-17T21:18:45.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8955999999999999e-05 2024-09-17T21:18:45.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198621+0000", 2024-09-17T21:18:45.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3837000000000001e-05 2024-09-17T21:18:45.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242258+0000", 2024-09-17T21:18:45.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043636900999999999 2024-09-17T21:18:45.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242343+0000", 2024-09-17T21:18:45.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5048000000000003e-05 2024-09-17T21:18:45.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243145+0000", 2024-09-17T21:18:45.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00080167300000000003 2024-09-17T21:18:45.054 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.054 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.054 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.054 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:503 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:45.054 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:18:45.054 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.824099365999999, 2024-09-17T21:18:45.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044220582000000001, 2024-09-17T21:18:45.054 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.054 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.054 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:45.054 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:45.054 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:45.055 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:45.055 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 503 2024-09-17T21:18:45.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.055 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.055 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.055 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:18:45.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:18:45.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198647+0000", 2024-09-17T21:18:45.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.483e-06 2024-09-17T21:18:45.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198653+0000", 2024-09-17T21:18:45.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4529999999999999e-06 2024-09-17T21:18:45.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.057 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.057 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198672+0000", 2024-09-17T21:18:45.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9193000000000001e-05 2024-09-17T21:18:45.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.057 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.057 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198685+0000", 2024-09-17T21:18:45.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.291e-05 2024-09-17T21:18:45.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:18:45.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043672141999999997 2024-09-17T21:18:45.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.060 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.060 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242716+0000", 2024-09-17T21:18:45.060 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00035901800000000001 2024-09-17T21:18:45.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.060 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.060 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.060 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242866+0000", 2024-09-17T21:18:45.060 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000149383 2024-09-17T21:18:45.060 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.060 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.060 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.061 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:504 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:45.061 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:18:45.061 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.824039574, 2024-09-17T21:18:45.061 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044611406999999999, 2024-09-17T21:18:45.061 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.061 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.061 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:45.061 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:45.061 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:45.061 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:45.062 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 504 2024-09-17T21:18:45.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.062 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.062 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.062 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.062 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:18:45.062 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.062 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.062 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.062 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:18:45.062 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198706+0000", 2024-09-17T21:18:45.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:18:45.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198712+0000", 2024-09-17T21:18:45.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5770000000000001e-06 2024-09-17T21:18:45.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198729+0000", 2024-09-17T21:18:45.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6673000000000001e-05 2024-09-17T21:18:45.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.064 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.064 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198742+0000", 2024-09-17T21:18:45.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3543000000000001e-05 2024-09-17T21:18:45.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.064 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.064 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242981+0000", 2024-09-17T21:18:45.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044238953999999997 2024-09-17T21:18:45.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.064 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.064 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243155+0000", 2024-09-17T21:18:45.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017401600000000001 2024-09-17T21:18:45.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.065 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.065 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.065 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243316+0000", 2024-09-17T21:18:45.065 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016137399999999999 2024-09-17T21:18:45.065 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.065 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.065 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.065 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.065 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:521 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:45.065 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:18:45.066 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.823492434999999, 2024-09-17T21:18:45.066 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045043685999999999, 2024-09-17T21:18:45.066 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.066 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.066 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:45.066 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:45.066 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:45.066 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:45.066 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 521 2024-09-17T21:18:45.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.066 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.066 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.066 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:18:45.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:18:45.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199253+0000", 2024-09-17T21:18:45.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1400000000000001e-06 2024-09-17T21:18:45.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.068 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.068 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199258+0000", 2024-09-17T21:18:45.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1800000000000004e-06 2024-09-17T21:18:45.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.068 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.068 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199276+0000", 2024-09-17T21:18:45.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7686999999999999e-05 2024-09-17T21:18:45.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.068 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.068 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199289+0000", 2024-09-17T21:18:45.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3263e-05 2024-09-17T21:18:45.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.069 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.069 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243397+0000", 2024-09-17T21:18:45.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044107636999999998 2024-09-17T21:18:45.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.069 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.069 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243527+0000", 2024-09-17T21:18:45.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013038399999999999 2024-09-17T21:18:45.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.069 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.069 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244296+0000", 2024-09-17T21:18:45.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076839499999999995 2024-09-17T21:18:45.070 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.070 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.070 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.070 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.070 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:522 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:45.070 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:18:45.070 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.823437131999999, 2024-09-17T21:18:45.070 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046049870999999999, 2024-09-17T21:18:45.070 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.070 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.070 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:45.070 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:45.070 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:45.071 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:45.071 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 522 2024-09-17T21:18:45.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.071 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.071 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:18:45.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.071 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:18:45.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199309+0000", 2024-09-17T21:18:45.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:18:45.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199315+0000", 2024-09-17T21:18:45.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2440000000000001e-06 2024-09-17T21:18:45.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199333+0000", 2024-09-17T21:18:45.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7886000000000001e-05 2024-09-17T21:18:45.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199379+0000", 2024-09-17T21:18:45.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6383e-05 2024-09-17T21:18:45.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244394+0000", 2024-09-17T21:18:45.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045014851000000002 2024-09-17T21:18:45.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244491+0000", 2024-09-17T21:18:45.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7245999999999997e-05 2024-09-17T21:18:45.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245357+0000", 2024-09-17T21:18:45.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00086626800000000001 2024-09-17T21:18:45.074 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.074 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.074 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.075 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:527 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:45.075 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:18:45.075 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.823340466000001, 2024-09-17T21:18:45.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045812522000000001, 2024-09-17T21:18:45.075 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.075 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.075 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:45.075 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:45.075 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:45.075 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:45.075 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 527 2024-09-17T21:18:45.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.075 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.076 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.076 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.076 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:18:45.076 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.076 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.076 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.076 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:18:45.076 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.076 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.076 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.076 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199406+0000", 2024-09-17T21:18:45.076 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.367e-06 2024-09-17T21:18:45.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199412+0000", 2024-09-17T21:18:45.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:18:45.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199429+0000", 2024-09-17T21:18:45.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7396000000000001e-05 2024-09-17T21:18:45.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199442+0000", 2024-09-17T21:18:45.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3327e-05 2024-09-17T21:18:45.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244361+0000", 2024-09-17T21:18:45.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044918197 2024-09-17T21:18:45.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244482+0000", 2024-09-17T21:18:45.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121945 2024-09-17T21:18:45.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245217+0000", 2024-09-17T21:18:45.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00073415999999999998 2024-09-17T21:18:45.079 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.079 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.079 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.079 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:534 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:45.079 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:18:45.079 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.82328176, 2024-09-17T21:18:45.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046292505999999997, 2024-09-17T21:18:45.079 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.079 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.079 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:45.079 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:45.080 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:45.080 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:45.080 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:18:45.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.080 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:18:45.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:18:45.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.081 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.081 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199464+0000", 2024-09-17T21:18:45.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:18:45.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.081 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.081 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199473+0000", 2024-09-17T21:18:45.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5629999999999992e-06 2024-09-17T21:18:45.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.081 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199492+0000", 2024-09-17T21:18:45.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9273999999999999e-05 2024-09-17T21:18:45.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.082 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199505+0000", 2024-09-17T21:18:45.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3443e-05 2024-09-17T21:18:45.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.082 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245357+0000", 2024-09-17T21:18:45.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045851714000000002 2024-09-17T21:18:45.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.083 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.083 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245550+0000", 2024-09-17T21:18:45.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019301600000000001 2024-09-17T21:18:45.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.083 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.083 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245755+0000", 2024-09-17T21:18:45.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205226 2024-09-17T21:18:45.083 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.083 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.083 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.083 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:535 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:45.083 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:18:45.084 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.823177204, 2024-09-17T21:18:45.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046637492000000003, 2024-09-17T21:18:45.084 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.084 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.084 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:45.084 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:45.084 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:45.084 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:45.084 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 535 2024-09-17T21:18:45.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.084 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.084 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:18:45.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:18:45.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199568+0000", 2024-09-17T21:18:45.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.037e-06 2024-09-17T21:18:45.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199572+0000", 2024-09-17T21:18:45.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5729999999999999e-06 2024-09-17T21:18:45.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199585+0000", 2024-09-17T21:18:45.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.279e-05 2024-09-17T21:18:45.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199592+0000", 2024-09-17T21:18:45.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.61e-06 2024-09-17T21:18:45.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.087 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.087 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245873+0000", 2024-09-17T21:18:45.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046280494999999998 2024-09-17T21:18:45.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.087 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.087 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246018+0000", 2024-09-17T21:18:45.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000144697 2024-09-17T21:18:45.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.087 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.087 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246205+0000", 2024-09-17T21:18:45.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018729 2024-09-17T21:18:45.087 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.088 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.088 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.088 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:543 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:45.088 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:18:45.088 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.823081958, 2024-09-17T21:18:45.088 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047564097999999999, 2024-09-17T21:18:45.088 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.088 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.088 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:45.088 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:45.088 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:45.088 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:45.088 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 543 2024-09-17T21:18:45.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.089 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:18:45.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:18:45.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:18:45.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9699999999999995e-07 2024-09-17T21:18:45.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199677+0000", 2024-09-17T21:18:45.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:18:45.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199690+0000", 2024-09-17T21:18:45.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2977e-05 2024-09-17T21:18:45.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.091 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199705+0000", 2024-09-17T21:18:45.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4656e-05 2024-09-17T21:18:45.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.091 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.091 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246307+0000", 2024-09-17T21:18:45.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046601780000000002 2024-09-17T21:18:45.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.091 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.091 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246475+0000", 2024-09-17T21:18:45.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016834200000000001 2024-09-17T21:18:45.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.092 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.092 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247227+0000", 2024-09-17T21:18:45.092 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00075173299999999998 2024-09-17T21:18:45.092 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.092 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.092 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.092 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.092 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:544 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:45.092 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:18:45.092 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.823023548999998, 2024-09-17T21:18:45.092 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049283559999999997, 2024-09-17T21:18:45.092 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.092 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.093 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:45.093 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:45.093 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:45.093 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:45.093 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 544 2024-09-17T21:18:45.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.093 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.093 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.093 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:18:45.093 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.093 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.093 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:18:45.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199722+0000", 2024-09-17T21:18:45.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.08e-06 2024-09-17T21:18:45.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199738+0000", 2024-09-17T21:18:45.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5943999999999998e-05 2024-09-17T21:18:45.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199748+0000", 2024-09-17T21:18:45.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0193e-05 2024-09-17T21:18:45.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199756+0000", 2024-09-17T21:18:45.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4200000000000001e-06 2024-09-17T21:18:45.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247311+0000", 2024-09-17T21:18:45.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047555300000000002 2024-09-17T21:18:45.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.096 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.096 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247571+0000", 2024-09-17T21:18:45.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025996999999999999 2024-09-17T21:18:45.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.096 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.096 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249005+0000", 2024-09-17T21:18:45.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001433653 2024-09-17T21:18:45.096 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.096 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.096 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.096 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:549 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:45.096 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:18:45.097 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.822977925, 2024-09-17T21:18:45.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048464528999999999, 2024-09-17T21:18:45.097 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.097 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.097 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:45.097 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:45.097 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:45.097 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:45.097 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 549 2024-09-17T21:18:45.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.097 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:18:45.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:18:45.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:18:45.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:18:45.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199770+0000", 2024-09-17T21:18:45.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9969999999999999e-06 2024-09-17T21:18:45.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199780+0000", 2024-09-17T21:18:45.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0003000000000001e-05 2024-09-17T21:18:45.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199793+0000", 2024-09-17T21:18:45.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2074e-05 2024-09-17T21:18:45.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247289+0000", 2024-09-17T21:18:45.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047496691000000001 2024-09-17T21:18:45.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247437+0000", 2024-09-17T21:18:45.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014807899999999999 2024-09-17T21:18:45.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248231+0000", 2024-09-17T21:18:45.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00079390899999999998 2024-09-17T21:18:45.100 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.101 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.101 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.101 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:558 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:45.101 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:18:45.101 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.822937412000002, 2024-09-17T21:18:45.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049709821000000001, 2024-09-17T21:18:45.101 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.101 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.101 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:45.101 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:45.102 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:45.102 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:45.102 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 558 2024-09-17T21:18:45.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.102 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.102 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.102 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:18:45.102 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.102 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.102 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:18:45.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199809+0000", 2024-09-17T21:18:45.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.333e-06 2024-09-17T21:18:45.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199814+0000", 2024-09-17T21:18:45.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0930000000000002e-06 2024-09-17T21:18:45.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199831+0000", 2024-09-17T21:18:45.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7580000000000001e-05 2024-09-17T21:18:45.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199860+0000", 2024-09-17T21:18:45.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8609999999999999e-05 2024-09-17T21:18:45.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249106+0000", 2024-09-17T21:18:45.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049246545000000003 2024-09-17T21:18:45.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249274+0000", 2024-09-17T21:18:45.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016806699999999999 2024-09-17T21:18:45.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249517+0000", 2024-09-17T21:18:45.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024259300000000001 2024-09-17T21:18:45.105 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.105 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.105 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.105 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:559 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:45.105 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:18:45.105 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.822818049999999, 2024-09-17T21:18:45.106 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049990694000000002, 2024-09-17T21:18:45.106 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.106 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.106 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:45.106 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:45.106 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:45.106 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:45.106 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 559 2024-09-17T21:18:45.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.106 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.106 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.106 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:18:45.106 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.107 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:18:45.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.107 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199928+0000", 2024-09-17T21:18:45.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:18:45.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.107 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199935+0000", 2024-09-17T21:18:45.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5429999999999999e-06 2024-09-17T21:18:45.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199954+0000", 2024-09-17T21:18:45.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.912e-05 2024-09-17T21:18:45.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199969+0000", 2024-09-17T21:18:45.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5305999999999999e-05 2024-09-17T21:18:45.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249604+0000", 2024-09-17T21:18:45.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049634931 2024-09-17T21:18:45.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249738+0000", 2024-09-17T21:18:45.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013411599999999999 2024-09-17T21:18:45.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249917+0000", 2024-09-17T21:18:45.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179111 2024-09-17T21:18:45.109 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.109 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.109 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.110 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:560 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:45.110 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:18:45.110 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.822753634000001, 2024-09-17T21:18:45.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050173059999999998, 2024-09-17T21:18:45.110 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.110 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.110 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:45.110 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:45.110 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:45.110 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:45.110 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 560 2024-09-17T21:18:45.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.110 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.110 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:18:45.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:18:45.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199992+0000", 2024-09-17T21:18:45.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.517e-06 2024-09-17T21:18:45.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199998+0000", 2024-09-17T21:18:45.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1460000000000004e-06 2024-09-17T21:18:45.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:18:45.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7407000000000001e-05 2024-09-17T21:18:45.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200029+0000", 2024-09-17T21:18:45.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3499999999999999e-05 2024-09-17T21:18:45.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249968+0000", 2024-09-17T21:18:45.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049939799 2024-09-17T21:18:45.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250055+0000", 2024-09-17T21:18:45.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6731999999999996e-05 2024-09-17T21:18:45.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250164+0000", 2024-09-17T21:18:45.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108959 2024-09-17T21:18:45.113 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.113 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.114 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.114 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:18:45.114 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:18:45.114 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.2129639250000004, 2024-09-17T21:18:45.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:18:45.114 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.114 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.114 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:45.114 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:45.114 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:18:45.114 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:18:45.114 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:18:45.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.115 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:18:45.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:18:45.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:18:45.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:18:45.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:18:45.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:18:45.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:18:45.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:18:45.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:18:45.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:18:45.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:18:45.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:18:45.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:18:45.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:18:45.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:18:45.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:18:45.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:18:45.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.119 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.119 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:18:45.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:18:45.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.119 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.119 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:18:45.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:18:45.119 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.119 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.119 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.119 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.120 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.120 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:45.121 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:18:45.122 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:45.117+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:45.215 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:18:45.215 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:45.215 INFO:teuthology.orchestra.run.smithi083.stdout: "epoch": 46, 2024-09-17T21:18:45.216 INFO:teuthology.orchestra.run.smithi083.stdout: "fsid": "224a7a29-e112-472e-8a13-f67345d6d123", 2024-09-17T21:18:45.216 INFO:teuthology.orchestra.run.smithi083.stdout: "created": "2024-09-17T21:17:42.597312+0000", 2024-09-17T21:18:45.216 INFO:teuthology.orchestra.run.smithi083.stdout: "modified": "2024-09-17T21:18:43.266117+0000", 2024-09-17T21:18:45.216 INFO:teuthology.orchestra.run.smithi083.stdout: "last_up_change": "2024-09-17T21:17:53.976897+0000", 2024-09-17T21:18:45.216 INFO:teuthology.orchestra.run.smithi083.stdout: "last_in_change": "2024-09-17T21:18:39.781919+0000", 2024-09-17T21:18:45.216 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T21:18:45.216 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_num": 5799936, 2024-09-17T21:18:45.216 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_set": [ 2024-09-17T21:18:45.216 INFO:teuthology.orchestra.run.smithi083.stdout: "pglog_hardlimit", 2024-09-17T21:18:45.216 INFO:teuthology.orchestra.run.smithi083.stdout: "purged_snapdirs", 2024-09-17T21:18:45.216 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_deletes", 2024-09-17T21:18:45.216 INFO:teuthology.orchestra.run.smithi083.stdout: "sortbitwise" 2024-09-17T21:18:45.216 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:18:45.217 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_version": 5, 2024-09-17T21:18:45.217 INFO:teuthology.orchestra.run.smithi083.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T21:18:45.217 INFO:teuthology.orchestra.run.smithi083.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T21:18:45.217 INFO:teuthology.orchestra.run.smithi083.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T21:18:45.217 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_snapshot": "", 2024-09-17T21:18:45.217 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_max": 3, 2024-09-17T21:18:45.217 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd": 6, 2024-09-17T21:18:45.217 INFO:teuthology.orchestra.run.smithi083.stdout: "require_min_compat_client": "luminous", 2024-09-17T21:18:45.217 INFO:teuthology.orchestra.run.smithi083.stdout: "min_compat_client": "luminous", 2024-09-17T21:18:45.217 INFO:teuthology.orchestra.run.smithi083.stdout: "require_osd_release": "squid", 2024-09-17T21:18:45.217 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_crimson": false, 2024-09-17T21:18:45.217 INFO:teuthology.orchestra.run.smithi083.stdout: "pools": [ 2024-09-17T21:18:45.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.217 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 1, 2024-09-17T21:18:45.218 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": ".mgr", 2024-09-17T21:18:45.218 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-17T21:17:55.581350+0000", 2024-09-17T21:18:45.218 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-17T21:18:45.218 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-17T21:18:45.218 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-17T21:18:45.218 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 2, 2024-09-17T21:18:45.218 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 1, 2024-09-17T21:18:45.218 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-17T21:18:45.218 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:18:45.218 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:18:45.218 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:18:45.218 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:18:45.218 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-17T21:18:45.218 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-17T21:18:45.219 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:18:45.219 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 1, 2024-09-17T21:18:45.219 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 1, 2024-09-17T21:18:45.219 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 1, 2024-09-17T21:18:45.219 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 1, 2024-09-17T21:18:45.219 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 1, 2024-09-17T21:18:45.219 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-17T21:18:45.219 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-17T21:18:45.219 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-17T21:18:45.219 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-17T21:18:45.219 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-17T21:18:45.219 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-17T21:18:45.219 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-17T21:18:45.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.219 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "17", 2024-09-17T21:18:45.220 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-17T21:18:45.220 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:18:45.220 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:18:45.220 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-17T21:18:45.220 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:18:45.220 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-17T21:18:45.220 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-17T21:18:45.220 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-17T21:18:45.220 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-17T21:18:45.220 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-17T21:18:45.220 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-17T21:18:45.220 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-17T21:18:45.220 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-17T21:18:45.220 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-17T21:18:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-17T21:18:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-17T21:18:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-17T21:18:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-17T21:18:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:18:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:18:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:18:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-17T21:18:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-17T21:18:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-17T21:18:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-17T21:18:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-17T21:18:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-17T21:18:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-17T21:18:45.222 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-17T21:18:45.222 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:18:45.222 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:18:45.222 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:18:45.222 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:18:45.222 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-17T21:18:45.222 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-17T21:18:45.222 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-17T21:18:45.222 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-17T21:18:45.222 INFO:teuthology.orchestra.run.smithi083.stdout: "options": { 2024-09-17T21:18:45.222 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_max": 32, 2024-09-17T21:18:45.222 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_min": 1 2024-09-17T21:18:45.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.223 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-17T21:18:45.223 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": {} 2024-09-17T21:18:45.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.223 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-17T21:18:45.223 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-17T21:18:45.223 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 6.059999942779541, 2024-09-17T21:18:45.223 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 6.059999942779541, 2024-09-17T21:18:45.223 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 0.33000001311302185, 2024-09-17T21:18:45.223 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2, 2024-09-17T21:18:45.223 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2, 2024-09-17T21:18:45.223 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 0.87948358058929443, 2024-09-17T21:18:45.223 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-17T21:18:45.224 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-17T21:18:45.224 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.224 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T21:18:45.224 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "rbd", 2024-09-17T21:18:45.224 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-17T21:17:59.603025+0000", 2024-09-17T21:18:45.224 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 8193, 2024-09-17T21:18:45.224 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:18:45.224 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-17T21:18:45.224 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 2, 2024-09-17T21:18:45.224 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 1, 2024-09-17T21:18:45.224 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-17T21:18:45.224 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:18:45.225 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:18:45.225 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:18:45.225 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:18:45.225 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-17T21:18:45.225 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-17T21:18:45.225 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:18:45.225 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 8, 2024-09-17T21:18:45.225 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 8, 2024-09-17T21:18:45.225 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 8, 2024-09-17T21:18:45.225 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 8, 2024-09-17T21:18:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 8, 2024-09-17T21:18:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-17T21:18:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-17T21:18:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-17T21:18:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-17T21:18:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-17T21:18:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-17T21:18:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-17T21:18:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "21", 2024-09-17T21:18:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-17T21:18:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:18:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:18:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-17T21:18:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:18:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 2, 2024-09-17T21:18:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 21, 2024-09-17T21:18:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-17T21:18:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-17T21:18:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-17T21:18:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-17T21:18:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-17T21:18:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-17T21:18:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-17T21:18:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-17T21:18:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-17T21:18:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-17T21:18:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-17T21:18:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:18:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:18:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:18:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-17T21:18:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-17T21:18:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-17T21:18:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-17T21:18:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-17T21:18:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-17T21:18:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-17T21:18:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-17T21:18:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:18:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:18:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:18:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:18:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-17T21:18:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-17T21:18:45.230 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-17T21:18:45.230 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-17T21:18:45.231 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-17T21:18:45.231 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-17T21:18:45.231 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd": {} 2024-09-17T21:18:45.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.231 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-17T21:18:45.231 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-17T21:18:45.231 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 2.2699999809265137, 2024-09-17T21:18:45.231 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 2.2699999809265137, 2024-09-17T21:18:45.231 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 0.82999998331069946, 2024-09-17T21:18:45.231 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-17T21:18:45.231 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-17T21:18:45.231 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 0.87948358058929443, 2024-09-17T21:18:45.231 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-17T21:18:45.232 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-17T21:18:45.232 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.232 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 3, 2024-09-17T21:18:45.232 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "pool_client.0", 2024-09-17T21:18:45.232 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-17T21:18:09.441137+0000", 2024-09-17T21:18:45.232 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 8193, 2024-09-17T21:18:45.232 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:18:45.232 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-17T21:18:45.232 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 2, 2024-09-17T21:18:45.232 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 1, 2024-09-17T21:18:45.232 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-17T21:18:45.232 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:18:45.233 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:18:45.233 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:18:45.233 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:18:45.233 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-17T21:18:45.233 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-17T21:18:45.233 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:18:45.233 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-17T21:18:45.233 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-17T21:18:45.233 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-17T21:18:45.233 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-17T21:18:45.233 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-17T21:18:45.233 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-17T21:18:45.233 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-17T21:18:45.234 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-17T21:18:45.234 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-17T21:18:45.234 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-17T21:18:45.234 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-17T21:18:45.234 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-17T21:18:45.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.234 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "44", 2024-09-17T21:18:45.234 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-17T21:18:45.234 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:18:45.234 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:18:45.234 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-17T21:18:45.234 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:18:45.234 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 11, 2024-09-17T21:18:45.234 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 44, 2024-09-17T21:18:45.235 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-17T21:18:45.235 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-17T21:18:45.235 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-17T21:18:45.235 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-17T21:18:45.235 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-17T21:18:45.235 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-17T21:18:45.235 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-17T21:18:45.235 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-17T21:18:45.235 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-17T21:18:45.235 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-17T21:18:45.235 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-17T21:18:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:18:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:18:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:18:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-17T21:18:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-17T21:18:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-17T21:18:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-17T21:18:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-17T21:18:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-17T21:18:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-17T21:18:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-17T21:18:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:18:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:18:45.237 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:18:45.237 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:18:45.237 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-17T21:18:45.237 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-17T21:18:45.237 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-17T21:18:45.237 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-17T21:18:45.237 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-17T21:18:45.237 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-17T21:18:45.237 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd": {} 2024-09-17T21:18:45.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.237 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-17T21:18:45.237 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-17T21:18:45.237 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.7000000476837158, 2024-09-17T21:18:45.237 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.7000000476837158, 2024-09-17T21:18:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 0.82999998331069946, 2024-09-17T21:18:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.4099999666213989, 2024-09-17T21:18:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.4099999666213989, 2024-09-17T21:18:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 0.87948358058929443, 2024-09-17T21:18:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-17T21:18:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-17T21:18:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:18:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-17T21:18:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-17T21:18:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "5bfd41c9-0cac-47c4-9a6e-8dce774c3314", 2024-09-17T21:18:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:18:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 0, 2024-09-17T21:18:45.239 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 0, 2024-09-17T21:18:45.239 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-17T21:18:45.239 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:18:45.239 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:18:45.239 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:18:45.239 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 44, 2024-09-17T21:18:45.239 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:18:45.239 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:18:45.239 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:18:45.239 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.239 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.239 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6816", 2024-09-17T21:18:45.239 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:45.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.240 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.240 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6817", 2024-09-17T21:18:45.240 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:45.240 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.240 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.240 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:18:45.240 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.240 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.240 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6818", 2024-09-17T21:18:45.240 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:45.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.241 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.241 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6819", 2024-09-17T21:18:45.241 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:45.241 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.241 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.241 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:18:45.241 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.241 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.241 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6822", 2024-09-17T21:18:45.241 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:45.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.242 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.242 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6823", 2024-09-17T21:18:45.242 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:45.242 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.242 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.242 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:18:45.242 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.242 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.242 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6820", 2024-09-17T21:18:45.242 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6821", 2024-09-17T21:18:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6817/1111036754", 2024-09-17T21:18:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6819/1111036754", 2024-09-17T21:18:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6823/1111036754", 2024-09-17T21:18:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6821/1111036754", 2024-09-17T21:18:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:18:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:18:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:18:45.244 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.244 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-17T21:18:45.244 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "78404e0d-d397-4806-8e90-5f32182643bb", 2024-09-17T21:18:45.244 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:18:45.244 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:18:45.244 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:18:45.244 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-17T21:18:45.244 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:18:45.244 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:18:45.244 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:18:45.244 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 42, 2024-09-17T21:18:45.244 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:18:45.245 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:18:45.245 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:18:45.245 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.245 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.245 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6808", 2024-09-17T21:18:45.245 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:45.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.245 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.245 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6809", 2024-09-17T21:18:45.245 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:45.245 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.246 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.246 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:18:45.246 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.246 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.246 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6810", 2024-09-17T21:18:45.246 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:45.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.246 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.246 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6811", 2024-09-17T21:18:45.246 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:18:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6814", 2024-09-17T21:18:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6815", 2024-09-17T21:18:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:45.248 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.248 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.248 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:18:45.248 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.248 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.248 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6812", 2024-09-17T21:18:45.248 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:45.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.248 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.248 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6813", 2024-09-17T21:18:45.249 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:45.249 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.249 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.249 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6809/1460363937", 2024-09-17T21:18:45.249 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6811/1460363937", 2024-09-17T21:18:45.249 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6815/1460363937", 2024-09-17T21:18:45.249 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6813/1460363937", 2024-09-17T21:18:45.249 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:18:45.249 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:18:45.249 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:18:45.249 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.250 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-17T21:18:45.250 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "286ed6e0-c121-4ebb-8cf6-37e506a397aa", 2024-09-17T21:18:45.250 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:18:45.250 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:18:45.250 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:18:45.250 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 0.2769012451171875, 2024-09-17T21:18:45.250 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:18:45.250 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:18:45.250 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 12, 2024-09-17T21:18:45.250 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 39, 2024-09-17T21:18:45.250 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:18:45.250 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:18:45.250 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:18:45.250 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.251 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.251 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6800", 2024-09-17T21:18:45.251 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:45.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.251 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.251 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6801", 2024-09-17T21:18:45.251 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:45.251 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.251 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.251 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:18:45.251 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.252 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.252 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6802", 2024-09-17T21:18:45.252 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:45.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.252 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.252 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6803", 2024-09-17T21:18:45.252 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:45.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.252 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.252 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:18:45.253 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.253 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.253 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6806", 2024-09-17T21:18:45.253 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:45.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.253 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.253 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6807", 2024-09-17T21:18:45.253 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:45.253 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.253 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.253 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:18:45.254 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.254 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.254 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6804", 2024-09-17T21:18:45.254 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:45.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.254 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.254 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6805", 2024-09-17T21:18:45.254 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:45.254 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.254 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.254 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6801/2837444034", 2024-09-17T21:18:45.255 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6803/2837444034", 2024-09-17T21:18:45.255 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6807/2837444034", 2024-09-17T21:18:45.255 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6805/2837444034", 2024-09-17T21:18:45.255 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:18:45.255 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:18:45.255 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:18:45.255 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.255 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-17T21:18:45.255 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "b0af73a1-4bca-4892-a550-2ced87f4c357", 2024-09-17T21:18:45.255 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:18:45.255 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:18:45.255 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:18:45.256 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-17T21:18:45.256 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:18:45.256 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:18:45.256 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:18:45.256 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 45, 2024-09-17T21:18:45.256 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:18:45.256 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:18:45.256 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:18:45.256 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.256 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.256 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6808", 2024-09-17T21:18:45.256 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:45.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.257 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.257 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6809", 2024-09-17T21:18:45.257 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:45.257 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.257 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.257 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:18:45.257 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.257 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.257 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6810", 2024-09-17T21:18:45.257 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:45.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6811", 2024-09-17T21:18:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:18:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6814", 2024-09-17T21:18:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.259 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6815", 2024-09-17T21:18:45.259 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:45.259 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.259 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.259 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:18:45.259 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.259 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.259 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6812", 2024-09-17T21:18:45.259 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:45.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.260 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.260 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6813", 2024-09-17T21:18:45.260 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:45.260 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.260 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.260 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.97:6809/3637204865", 2024-09-17T21:18:45.260 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.97:6811/3637204865", 2024-09-17T21:18:45.260 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.97:6815/3637204865", 2024-09-17T21:18:45.260 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.97:6813/3637204865", 2024-09-17T21:18:45.260 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:18:45.260 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:18:45.260 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:18:45.260 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.261 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-17T21:18:45.261 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "2188361c-e1a1-45fa-bc01-da5a984ffd9f", 2024-09-17T21:18:45.261 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:18:45.261 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:18:45.261 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:18:45.261 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-17T21:18:45.261 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:18:45.261 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:18:45.261 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:18:45.261 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 42, 2024-09-17T21:18:45.261 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:18:45.261 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:18:45.262 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:18:45.262 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.262 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.262 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6800", 2024-09-17T21:18:45.262 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:45.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.262 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.262 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6801", 2024-09-17T21:18:45.262 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:45.262 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.262 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.262 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:18:45.263 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.263 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.263 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6802", 2024-09-17T21:18:45.263 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:45.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.263 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.263 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6803", 2024-09-17T21:18:45.263 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:45.263 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.263 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.263 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:18:45.263 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6806", 2024-09-17T21:18:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6807", 2024-09-17T21:18:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:18:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6804", 2024-09-17T21:18:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6805", 2024-09-17T21:18:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.97:6801/2391356911", 2024-09-17T21:18:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.97:6803/2391356911", 2024-09-17T21:18:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.97:6807/2391356911", 2024-09-17T21:18:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.97:6805/2391356911", 2024-09-17T21:18:45.266 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:18:45.266 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:18:45.266 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:18:45.266 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.266 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-17T21:18:45.266 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "7feccf4d-d8e4-4d06-ac83-bbc1bc623a5c", 2024-09-17T21:18:45.266 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:18:45.266 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:18:45.266 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:18:45.266 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-17T21:18:45.266 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:18:45.267 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:18:45.267 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:18:45.267 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 42, 2024-09-17T21:18:45.267 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:18:45.267 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:18:45.267 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:18:45.267 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.267 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.267 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6816", 2024-09-17T21:18:45.267 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:45.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.267 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.267 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6817", 2024-09-17T21:18:45.268 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:45.268 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.268 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.268 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:18:45.268 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.268 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.268 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6818", 2024-09-17T21:18:45.268 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:45.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.268 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.268 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6819", 2024-09-17T21:18:45.268 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:45.269 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.269 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.269 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:18:45.269 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.269 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.269 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6822", 2024-09-17T21:18:45.269 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:45.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.269 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.269 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6823", 2024-09-17T21:18:45.269 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:45.270 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.270 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.270 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:18:45.270 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:45.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.270 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:45.270 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6820", 2024-09-17T21:18:45.270 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:45.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.270 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:45.270 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6821", 2024-09-17T21:18:45.270 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:45.271 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.271 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.271 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.97:6817/2326316381", 2024-09-17T21:18:45.271 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.97:6819/2326316381", 2024-09-17T21:18:45.271 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.97:6823/2326316381", 2024-09-17T21:18:45.271 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.97:6821/2326316381", 2024-09-17T21:18:45.271 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:18:45.271 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:18:45.271 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:18:45.271 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.271 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.271 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:18:45.271 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_xinfo": [ 2024-09-17T21:18:45.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.272 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-17T21:18:45.272 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:18:45.272 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:18:45.272 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:18:45.272 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:18:45.272 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 65536, 2024-09-17T21:18:45.272 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:51.138101+0000", 2024-09-17T21:18:45.272 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:18:45.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.272 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-17T21:18:45.272 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:18:45.272 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:18:45.272 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:18:45.272 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:18:45.273 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:18:45.273 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:50.581632+0000", 2024-09-17T21:18:45.273 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:18:45.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.273 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-17T21:18:45.273 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:18:45.273 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:18:45.273 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:18:45.273 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:18:45.273 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:18:45.273 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:50.365193+0000", 2024-09-17T21:18:45.273 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:18:45.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-17T21:18:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:18:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:18:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:18:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:18:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:18:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:50.718974+0000", 2024-09-17T21:18:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:18:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-17T21:18:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:18:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:18:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:18:45.275 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:18:45.275 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:18:45.275 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:50.501878+0000", 2024-09-17T21:18:45.275 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:18:45.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.275 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-17T21:18:45.275 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:18:45.275 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:18:45.275 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:18:45.275 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:18:45.275 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:18:45.275 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:51.150747+0000", 2024-09-17T21:18:45.275 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:18:45.276 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.276 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:18:45.276 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap": [ 2024-09-17T21:18:45.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.276 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "3.1e", 2024-09-17T21:18:45.276 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-17T21:18:45.276 INFO:teuthology.orchestra.run.smithi083.stdout: 0, 2024-09-17T21:18:45.276 INFO:teuthology.orchestra.run.smithi083.stdout: 4 2024-09-17T21:18:45.276 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.276 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.276 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:18:45.276 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_items": [], 2024-09-17T21:18:45.276 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_primaries": [], 2024-09-17T21:18:45.277 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_temp": [], 2024-09-17T21:18:45.277 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_temp": [], 2024-09-17T21:18:45.277 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklist": {}, 2024-09-17T21:18:45.277 INFO:teuthology.orchestra.run.smithi083.stdout: "range_blocklist": {}, 2024-09-17T21:18:45.277 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profiles": { 2024-09-17T21:18:45.277 INFO:teuthology.orchestra.run.smithi083.stdout: "default": { 2024-09-17T21:18:45.277 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-failure-domain": "osd", 2024-09-17T21:18:45.277 INFO:teuthology.orchestra.run.smithi083.stdout: "k": "2", 2024-09-17T21:18:45.277 INFO:teuthology.orchestra.run.smithi083.stdout: "m": "1", 2024-09-17T21:18:45.277 INFO:teuthology.orchestra.run.smithi083.stdout: "plugin": "jerasure", 2024-09-17T21:18:45.277 INFO:teuthology.orchestra.run.smithi083.stdout: "technique": "reed_sol_van" 2024-09-17T21:18:45.277 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.278 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps_queue": [], 2024-09-17T21:18:45.278 INFO:teuthology.orchestra.run.smithi083.stdout: "new_removed_snaps": [], 2024-09-17T21:18:45.278 INFO:teuthology.orchestra.run.smithi083.stdout: "new_purged_snaps": [], 2024-09-17T21:18:45.278 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_node_flags": {}, 2024-09-17T21:18:45.278 INFO:teuthology.orchestra.run.smithi083.stdout: "device_class_flags": {}, 2024-09-17T21:18:45.278 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode": { 2024-09-17T21:18:45.278 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_enabled": false, 2024-09-17T21:18:45.278 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_bucket_count": 0, 2024-09-17T21:18:45.278 INFO:teuthology.orchestra.run.smithi083.stdout: "degraded_stretch_mode": 0, 2024-09-17T21:18:45.278 INFO:teuthology.orchestra.run.smithi083.stdout: "recovering_stretch_mode": 0, 2024-09-17T21:18:45.278 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_bucket": 0 2024-09-17T21:18:45.278 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.278 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:45.280 INFO:tasks.thrashosds.thrasher:j is {'epoch': 46, 'fsid': '224a7a29-e112-472e-8a13-f67345d6d123', 'created': '2024-09-17T21:17:42.597312+0000', 'modified': '2024-09-17T21:18:43.266117+0000', 'last_up_change': '2024-09-17T21:17:53.976897+0000', 'last_in_change': '2024-09-17T21:18:39.781919+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T21:17:55.581350+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8794835805892944, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T21:17:59.603025+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.2699999809265137, 'score_stable': 2.2699999809265137, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.8794835805892944, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T21:18:09.441137+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 11, 'snap_epoch': 44, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7000000476837158, 'score_stable': 1.7000000476837158, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.409999966621399, 'raw_score_stable': 1.409999966621399, 'primary_affinity_weighted': 0.8794835805892944, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '5bfd41c9-0cac-47c4-9a6e-8dce774c3314', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6816', 'nonce': 1111036754}, {'type': 'v1', 'addr': '172.21.15.83:6817', 'nonce': 1111036754}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6818', 'nonce': 1111036754}, {'type': 'v1', 'addr': '172.21.15.83:6819', 'nonce': 1111036754}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6822', 'nonce': 1111036754}, {'type': 'v1', 'addr': '172.21.15.83:6823', 'nonce': 1111036754}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6820', 'nonce': 1111036754}, {'type': 'v1', 'addr': '172.21.15.83:6821', 'nonce': 1111036754}]}, 'public_addr': '172.21.15.83:6817/1111036754', 'cluster_addr': '172.21.15.83:6819/1111036754', 'heartbeat_back_addr': '172.21.15.83:6823/1111036754', 'heartbeat_front_addr': '172.21.15.83:6821/1111036754', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '78404e0d-d397-4806-8e90-5f32182643bb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 42, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6808', 'nonce': 1460363937}, {'type': 'v1', 'addr': '172.21.15.83:6809', 'nonce': 1460363937}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6810', 'nonce': 1460363937}, {'type': 'v1', 'addr': '172.21.15.83:6811', 'nonce': 1460363937}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6814', 'nonce': 1460363937}, {'type': 'v1', 'addr': '172.21.15.83:6815', 'nonce': 1460363937}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6812', 'nonce': 1460363937}, {'type': 'v1', 'addr': '172.21.15.83:6813', 'nonce': 1460363937}]}, 'public_addr': '172.21.15.83:6809/1460363937', 'cluster_addr': '172.21.15.83:6811/1460363937', 'heartbeat_back_addr': '172.21.15.83:6815/1460363937', 'heartbeat_front_addr': '172.21.15.83:6813/1460363937', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '286ed6e0-c121-4ebb-8cf6-37e506a397aa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.2769012451171875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6800', 'nonce': 2837444034}, {'type': 'v1', 'addr': '172.21.15.83:6801', 'nonce': 2837444034}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6802', 'nonce': 2837444034}, {'type': 'v1', 'addr': '172.21.15.83:6803', 'nonce': 2837444034}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6806', 'nonce': 2837444034}, {'type': 'v1', 'addr': '172.21.15.83:6807', 'nonce': 2837444034}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6804', 'nonce': 2837444034}, {'type': 'v1', 'addr': '172.21.15.83:6805', 'nonce': 2837444034}]}, 'public_addr': '172.21.15.83:6801/2837444034', 'cluster_addr': '172.21.15.83:6803/2837444034', 'heartbeat_back_addr': '172.21.15.83:6807/2837444034', 'heartbeat_front_addr': '172.21.15.83:6805/2837444034', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'b0af73a1-4bca-4892-a550-2ced87f4c357', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 45, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6808', 'nonce': 3637204865}, {'type': 'v1', 'addr': '172.21.15.97:6809', 'nonce': 3637204865}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6810', 'nonce': 3637204865}, {'type': 'v1', 'addr': '172.21.15.97:6811', 'nonce': 3637204865}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6814', 'nonce': 3637204865}, {'type': 'v1', 'addr': '172.21.15.97:6815', 'nonce': 3637204865}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6812', 'nonce': 3637204865}, {'type': 'v1', 'addr': '172.21.15.97:6813', 'nonce': 3637204865}]}, 'public_addr': '172.21.15.97:6809/3637204865', 'cluster_addr': '172.21.15.97:6811/3637204865', 'heartbeat_back_addr': '172.21.15.97:6815/3637204865', 'heartbeat_front_addr': '172.21.15.97:6813/3637204865', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2188361c-e1a1-45fa-bc01-da5a984ffd9f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 42, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6800', 'nonce': 2391356911}, {'type': 'v1', 'addr': '172.21.15.97:6801', 'nonce': 2391356911}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6802', 'nonce': 2391356911}, {'type': 'v1', 'addr': '172.21.15.97:6803', 'nonce': 2391356911}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6806', 'nonce': 2391356911}, {'type': 'v1', 'addr': '172.21.15.97:6807', 'nonce': 2391356911}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6804', 'nonce': 2391356911}, {'type': 'v1', 'addr': '172.21.15.97:6805', 'nonce': 2391356911}]}, 'public_addr': '172.21.15.97:6801/2391356911', 'cluster_addr': '172.21.15.97:6803/2391356911', 'heartbeat_back_addr': '172.21.15.97:6807/2391356911', 'heartbeat_front_addr': '172.21.15.97:6805/2391356911', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '7feccf4d-d8e4-4d06-ac83-bbc1bc623a5c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 42, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6816', 'nonce': 2326316381}, {'type': 'v1', 'addr': '172.21.15.97:6817', 'nonce': 2326316381}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6818', 'nonce': 2326316381}, {'type': 'v1', 'addr': '172.21.15.97:6819', 'nonce': 2326316381}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6822', 'nonce': 2326316381}, {'type': 'v1', 'addr': '172.21.15.97:6823', 'nonce': 2326316381}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6820', 'nonce': 2326316381}, {'type': 'v1', 'addr': '172.21.15.97:6821', 'nonce': 2326316381}]}, 'public_addr': '172.21.15.97:6817/2326316381', 'cluster_addr': '172.21.15.97:6819/2326316381', 'heartbeat_back_addr': '172.21.15.97:6823/2326316381', 'heartbeat_front_addr': '172.21.15.97:6821/2326316381', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-17T21:17:51.138101+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:50.581632+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:50.365193+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:50.718974+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:50.501878+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:51.150747+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1e', 'osds': [0, 4]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T21:18:45.281 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:18:45.281 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:45.282 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:45.282 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:45.282 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:45.282 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:18:45.282 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:45.272+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:45.300 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:18:45.380 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:45.376+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:45.453 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:45.453 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:45.453 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:45.454 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:45.454 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:45.472 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:18:45.481 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:45.477+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:45.582 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:45.576+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:45.631 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:18:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:18:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:18:45.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.632 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:12 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:45.632 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:45.632 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 32.295961335000001, 2024-09-17T21:18:45.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061340989999999996, 2024-09-17T21:18:45.632 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.632 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.632 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:45.632 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:45.632 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:45.632 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:45.632 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12 2024-09-17T21:18:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332336+0000", 2024-09-17T21:18:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5519999999999999e-06 2024-09-17T21:18:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332342+0000", 2024-09-17T21:18:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7180000000000001e-06 2024-09-17T21:18:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332352+0000", 2024-09-17T21:18:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:18:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332359+0000", 2024-09-17T21:18:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3479999999999998e-06 2024-09-17T21:18:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332413+0000", 2024-09-17T21:18:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4447999999999998e-05 2024-09-17T21:18:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-17T21:18:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332417+0000", 2024-09-17T21:18:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.895e-06 2024-09-17T21:18:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334755+0000", 2024-09-17T21:18:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023377189999999998 2024-09-17T21:18:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334903+0000", 2024-09-17T21:18:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014778999999999999 2024-09-17T21:18:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:18:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.335187+0000", 2024-09-17T21:18:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000284715 2024-09-17T21:18:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.337731+0000", 2024-09-17T21:18:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002543741 2024-09-17T21:18:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338382+0000", 2024-09-17T21:18:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00065029200000000001 2024-09-17T21:18:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338418+0000", 2024-09-17T21:18:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.697e-05 2024-09-17T21:18:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338469+0000", 2024-09-17T21:18:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0154999999999997e-05 2024-09-17T21:18:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "pg_update_log_missing(3.1e epoch 28/27 rep_tid 81 entries 28'41 (0'0) error 3:7b23fd98:::rbd_header.1152c01721e5:head by client.4488.0:561 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-17T21:18:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:18:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.346687207999999, 2024-09-17T21:18:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0051146969999999996, 2024-09-17T21:18:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:18:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:18:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:18:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281610+0000", 2024-09-17T21:18:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.16e-07 2024-09-17T21:18:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281626+0000", 2024-09-17T21:18:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6039999999999999e-05 2024-09-17T21:18:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281666+0000", 2024-09-17T21:18:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0364000000000002e-05 2024-09-17T21:18:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281710+0000", 2024-09-17T21:18:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3596000000000001e-05 2024-09-17T21:18:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281992+0000", 2024-09-17T21:18:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028273099999999999 2024-09-17T21:18:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286723+0000", 2024-09-17T21:18:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047310499999999997 2024-09-17T21:18:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4488.0:562 3.1e e28/27 3:7b23fd98:::rbd_header.1152c01721e5:head v 28'42, mlcod=0'0)", 2024-09-17T21:18:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:18:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.34644342, 2024-09-17T21:18:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049275020000000003, 2024-09-17T21:18:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:18:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:18:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281853+0000", 2024-09-17T21:18:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2999999999999999e-07 2024-09-17T21:18:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281875+0000", 2024-09-17T21:18:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1386000000000002e-05 2024-09-17T21:18:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281892+0000", 2024-09-17T21:18:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7637000000000001e-05 2024-09-17T21:18:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281908+0000", 2024-09-17T21:18:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5187e-05 2024-09-17T21:18:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.282517+0000", 2024-09-17T21:18:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00060989799999999999 2024-09-17T21:18:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.282548+0000", 2024-09-17T21:18:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0910000000000001e-05 2024-09-17T21:18:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286729+0000", 2024-09-17T21:18:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0041810850000000002 2024-09-17T21:18:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286780+0000", 2024-09-17T21:18:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0569000000000003e-05 2024-09-17T21:18:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:609 3.d e28/23 3:b7278be9:::rbd_data.1152c6af092e.000000000000008f:head v 28'6, mlcod=28'4)", 2024-09-17T21:18:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:18:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.513499133, 2024-09-17T21:18:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049797280000000001, 2024-09-17T21:18:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:18:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:18:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114800+0000", 2024-09-17T21:18:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:18:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114915+0000", 2024-09-17T21:18:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115053 2024-09-17T21:18:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114951+0000", 2024-09-17T21:18:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6165999999999999e-05 2024-09-17T21:18:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114965+0000", 2024-09-17T21:18:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4364e-05 2024-09-17T21:18:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115097+0000", 2024-09-17T21:18:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013139200000000001 2024-09-17T21:18:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115151+0000", 2024-09-17T21:18:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4422000000000002e-05 2024-09-17T21:18:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.119633+0000", 2024-09-17T21:18:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044818820000000004 2024-09-17T21:18:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.119776+0000", 2024-09-17T21:18:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000143609 2024-09-17T21:18:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:782 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'63, mlcod=28'60)", 2024-09-17T21:18:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:18:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.413531105000001, 2024-09-17T21:18:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006167283, 2024-09-17T21:18:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:18:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:18:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214767+0000", 2024-09-17T21:18:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:18:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214779+0000", 2024-09-17T21:18:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2627e-05 2024-09-17T21:18:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214805+0000", 2024-09-17T21:18:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5570000000000001e-05 2024-09-17T21:18:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214823+0000", 2024-09-17T21:18:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8202999999999998e-05 2024-09-17T21:18:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215170+0000", 2024-09-17T21:18:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034720399999999999 2024-09-17T21:18:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215235+0000", 2024-09-17T21:18:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4276000000000006e-05 2024-09-17T21:18:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220859+0000", 2024-09-17T21:18:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056240500000000002 2024-09-17T21:18:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220932+0000", 2024-09-17T21:18:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3503000000000004e-05 2024-09-17T21:18:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:783 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'64, mlcod=28'60)", 2024-09-17T21:18:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:18:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.412844263, 2024-09-17T21:18:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0055240869999999996, 2024-09-17T21:18:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:18:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:18:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215454+0000", 2024-09-17T21:18:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0269999999999998e-06 2024-09-17T21:18:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215468+0000", 2024-09-17T21:18:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4816e-05 2024-09-17T21:18:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215501+0000", 2024-09-17T21:18:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2957000000000003e-05 2024-09-17T21:18:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215524+0000", 2024-09-17T21:18:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2782999999999999e-05 2024-09-17T21:18:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216675+0000", 2024-09-17T21:18:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011507869999999999 2024-09-17T21:18:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216728+0000", 2024-09-17T21:18:45.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3112999999999998e-05 2024-09-17T21:18:45.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:45.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220938+0000", 2024-09-17T21:18:45.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042097380000000002 2024-09-17T21:18:45.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220976+0000", 2024-09-17T21:18:45.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7866e-05 2024-09-17T21:18:45.661 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.661 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:785 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'65, mlcod=28'60)", 2024-09-17T21:18:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:18:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.412258214000001, 2024-09-17T21:18:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049795409999999997, 2024-09-17T21:18:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:18:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:18:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216039+0000", 2024-09-17T21:18:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.793e-06 2024-09-17T21:18:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216053+0000", 2024-09-17T21:18:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.345e-05 2024-09-17T21:18:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216092+0000", 2024-09-17T21:18:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9180000000000001e-05 2024-09-17T21:18:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216106+0000", 2024-09-17T21:18:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:18:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217569+0000", 2024-09-17T21:18:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001462938 2024-09-17T21:18:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217607+0000", 2024-09-17T21:18:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7267000000000001e-05 2024-09-17T21:18:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220980+0000", 2024-09-17T21:18:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003373544 2024-09-17T21:18:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221017+0000", 2024-09-17T21:18:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7116000000000002e-05 2024-09-17T21:18:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:786 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'66, mlcod=28'60)", 2024-09-17T21:18:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:18:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.41155741, 2024-09-17T21:18:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057966650000000003, 2024-09-17T21:18:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:18:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:18:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216740+0000", 2024-09-17T21:18:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5600000000000001e-06 2024-09-17T21:18:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216752+0000", 2024-09-17T21:18:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1477e-05 2024-09-17T21:18:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216779+0000", 2024-09-17T21:18:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7093e-05 2024-09-17T21:18:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216817+0000", 2024-09-17T21:18:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8275999999999997e-05 2024-09-17T21:18:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218387+0000", 2024-09-17T21:18:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015703749999999999 2024-09-17T21:18:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218432+0000", 2024-09-17T21:18:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4910000000000002e-05 2024-09-17T21:18:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222432+0000", 2024-09-17T21:18:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039997189999999997 2024-09-17T21:18:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222535+0000", 2024-09-17T21:18:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103255 2024-09-17T21:18:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:18:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:18:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.410935578, 2024-09-17T21:18:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:18:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:18:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:18:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:18:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:18:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:18:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:18:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:18:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:18:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:18:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:18:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:18:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:18:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:18:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:18:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:18:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:18:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:18:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:18:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:788 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'68, mlcod=28'60)", 2024-09-17T21:18:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:18:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.410301145999998, 2024-09-17T21:18:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058495320000000002, 2024-09-17T21:18:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:18:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:18:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217996+0000", 2024-09-17T21:18:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.376e-06 2024-09-17T21:18:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218006+0000", 2024-09-17T21:18:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.01e-05 2024-09-17T21:18:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218030+0000", 2024-09-17T21:18:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4026999999999999e-05 2024-09-17T21:18:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218042+0000", 2024-09-17T21:18:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1977000000000001e-05 2024-09-17T21:18:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221081+0000", 2024-09-17T21:18:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030385960000000002 2024-09-17T21:18:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221115+0000", 2024-09-17T21:18:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3772999999999997e-05 2024-09-17T21:18:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223749+0000", 2024-09-17T21:18:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0026340629999999999 2024-09-17T21:18:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223844+0000", 2024-09-17T21:18:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5619999999999996e-05 2024-09-17T21:18:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:790 3.5 e28/23 3:a14b1a37:::rbd_data.1152c6af092e.000000000000002f:head v 28'1, mlcod=0'0)", 2024-09-17T21:18:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:18:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.405896819999999, 2024-09-17T21:18:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059622240000000003, 2024-09-17T21:18:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:18:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:18:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222400+0000", 2024-09-17T21:18:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.02e-06 2024-09-17T21:18:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222411+0000", 2024-09-17T21:18:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.044e-05 2024-09-17T21:18:45.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222440+0000", 2024-09-17T21:18:45.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.932e-05 2024-09-17T21:18:45.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222458+0000", 2024-09-17T21:18:45.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8615999999999999e-05 2024-09-17T21:18:45.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222609+0000", 2024-09-17T21:18:45.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015102299999999999 2024-09-17T21:18:45.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222657+0000", 2024-09-17T21:18:45.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7206000000000002e-05 2024-09-17T21:18:45.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:45.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228278+0000", 2024-09-17T21:18:45.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056208100000000004 2024-09-17T21:18:45.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228361+0000", 2024-09-17T21:18:45.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3788999999999993e-05 2024-09-17T21:18:45.685 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.685 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.685 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.685 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:884 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'76, mlcod=28'70)", 2024-09-17T21:18:45.685 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:18:45.685 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.359311939000001, 2024-09-17T21:18:45.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056451990000000001, 2024-09-17T21:18:45.685 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.685 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:18:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:18:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268986+0000", 2024-09-17T21:18:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9360000000000002e-06 2024-09-17T21:18:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269001+0000", 2024-09-17T21:18:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4684e-05 2024-09-17T21:18:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269026+0000", 2024-09-17T21:18:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5193000000000001e-05 2024-09-17T21:18:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269043+0000", 2024-09-17T21:18:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7306e-05 2024-09-17T21:18:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.270024+0000", 2024-09-17T21:18:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00098120600000000009 2024-09-17T21:18:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.270085+0000", 2024-09-17T21:18:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0486000000000003e-05 2024-09-17T21:18:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.274557+0000", 2024-09-17T21:18:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044723419999999998 2024-09-17T21:18:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.274629+0000", 2024-09-17T21:18:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2045999999999994e-05 2024-09-17T21:18:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:18:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:18:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.760634914000001, 2024-09-17T21:18:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:18:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:18:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:18:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:18:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:18:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:18:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:18:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:18:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:18:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:18:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:18:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:18:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:18:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:18:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:18:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:18:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:18:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:18:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:18:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:18:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:18:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.759369831000001, 2024-09-17T21:18:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:18:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:18:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:18:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:18:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:18:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:18:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:18:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:18:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:18:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:18:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:18:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:18:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:18:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:18:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:18:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:18:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:18:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:18:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:18:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:18:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:18:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.749590421000001, 2024-09-17T21:18:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:18:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:18:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:18:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:18:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:18:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:18:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:18:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:18:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:18:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:18:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:18:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:18:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:18:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:18:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:18:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:18:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:18:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:18:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:18:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:18:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:18:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.740868018, 2024-09-17T21:18:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:18:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:18:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:18:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:18:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:18:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:18:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:18:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:18:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:18:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:18:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:18:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:18:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:18:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:18:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:18:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:18:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:18:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:18:45.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:18:45.708 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.708 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.708 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.708 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4498.0:596 3.4 e32/23 3:20f262ea:::rbd_data.1152b51cb396.000000000000000d:head v 32'17, mlcod=30'15)", 2024-09-17T21:18:45.708 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.258354+0000", 2024-09-17T21:18:45.708 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 19.369942154, 2024-09-17T21:18:45.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0053727619999999997, 2024-09-17T21:18:45.708 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.708 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.709 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:45.709 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.258354+0000", 2024-09-17T21:18:45.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.258354+0000", 2024-09-17T21:18:45.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.258356+0000", 2024-09-17T21:18:45.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.798e-06 2024-09-17T21:18:45.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259198+0000", 2024-09-17T21:18:45.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084279199999999998 2024-09-17T21:18:45.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259229+0000", 2024-09-17T21:18:45.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0383999999999999e-05 2024-09-17T21:18:45.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259241+0000", 2024-09-17T21:18:45.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2139e-05 2024-09-17T21:18:45.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259341+0000", 2024-09-17T21:18:45.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010049 2024-09-17T21:18:45.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259380+0000", 2024-09-17T21:18:45.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9072000000000001e-05 2024-09-17T21:18:45.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:45.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.263668+0000", 2024-09-17T21:18:45.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042875389999999999 2024-09-17T21:18:45.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.263726+0000", 2024-09-17T21:18:45.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8548000000000003e-05 2024-09-17T21:18:45.712 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.712 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.712 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.713 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4520.0:6177 3.8 e34/27 3:1665de58:::rbd_data.11528c3b0529.0000000000000019:head v 34'43, mlcod=34'41)", 2024-09-17T21:18:45.713 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:30.509962+0000", 2024-09-17T21:18:45.713 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.118334164, 2024-09-17T21:18:45.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049320850000000001, 2024-09-17T21:18:45.713 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.713 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.713 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:45.713 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509962+0000", 2024-09-17T21:18:45.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509962+0000", 2024-09-17T21:18:45.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509964+0000", 2024-09-17T21:18:45.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.356e-06 2024-09-17T21:18:45.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509978+0000", 2024-09-17T21:18:45.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3434e-05 2024-09-17T21:18:45.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510001+0000", 2024-09-17T21:18:45.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3436000000000001e-05 2024-09-17T21:18:45.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510023+0000", 2024-09-17T21:18:45.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1739999999999999e-05 2024-09-17T21:18:45.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510193+0000", 2024-09-17T21:18:45.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017040600000000001 2024-09-17T21:18:45.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510250+0000", 2024-09-17T21:18:45.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6749000000000002e-05 2024-09-17T21:18:45.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:45.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.514823+0000", 2024-09-17T21:18:45.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0045728180000000002 2024-09-17T21:18:45.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.514894+0000", 2024-09-17T21:18:45.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1145999999999999e-05 2024-09-17T21:18:45.717 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.717 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.717 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.717 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4520.0:6179 3.7 e34/27 3:e77b9c77:::rbd_data.11528c3b0529.0000000000000023:head v 34'39, mlcod=30'37)", 2024-09-17T21:18:45.717 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:30.510076+0000", 2024-09-17T21:18:45.717 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.118220315, 2024-09-17T21:18:45.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00479934, 2024-09-17T21:18:45.718 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.718 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.718 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:45.718 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510076+0000", 2024-09-17T21:18:45.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510076+0000", 2024-09-17T21:18:45.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510076+0000", 2024-09-17T21:18:45.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1299999999999998e-07 2024-09-17T21:18:45.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510095+0000", 2024-09-17T21:18:45.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8637000000000001e-05 2024-09-17T21:18:45.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510122+0000", 2024-09-17T21:18:45.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7316000000000001e-05 2024-09-17T21:18:45.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.720 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510136+0000", 2024-09-17T21:18:45.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3824e-05 2024-09-17T21:18:45.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.720 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510700+0000", 2024-09-17T21:18:45.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00056388200000000003 2024-09-17T21:18:45.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510751+0000", 2024-09-17T21:18:45.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0416e-05 2024-09-17T21:18:45.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:45.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.514787+0000", 2024-09-17T21:18:45.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004036792 2024-09-17T21:18:45.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.514875+0000", 2024-09-17T21:18:45.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7559999999999995e-05 2024-09-17T21:18:45.721 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.722 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.722 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.722 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:18:45.722 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:18:45.722 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.355777454, 2024-09-17T21:18:45.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:18:45.722 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:45.722 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:45.722 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:45.722 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:45.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:45.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:18:45.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:45.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:18:45.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:45.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:45.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:18:45.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:18:45.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:45.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:18:45.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:18:45.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:45.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:18:45.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:18:45.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:45.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:18:45.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:18:45.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:45.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:18:45.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:18:45.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:45.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:18:45.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:18:45.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:45.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:18:45.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:18:45.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:45.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:45.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:45.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:18:45.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:18:45.726 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.726 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.726 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.726 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:45.726 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:45.726 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:45.728 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:18:45.729 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:18:45.730 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T21:18:45.731 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-09-17T21:18:45.546400+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":394108683,"num_objects":272,"num_object_clones":45,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":43,"num_read":39577,"num_read_kb":203500,"num_write":2693,"num_write_kb":149624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":20052489,"num_keys_recovered":53,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2273,"ondisk_log_size":2273,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":285404,"kb_used_data":125372,"kb_used_omap":7,"kb_used_meta":155832,"kb_avail":471573796,"statfs":{"total":483183820800,"available":482891567104,"internally_reserved":0,"allocated":128380928,"data_stored":185296559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7978,"internal_metadata":159572182},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-127462635,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":4661,"num_read_kb":35064,"num_write":485,"num_write_kb":3346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":20016095,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001018"},"pg_stats":[{"pgid":"3.1f","version":"46'15","reported_seq":251,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:44.298736+0000","last_change":"2024-09-17T21:18:40.931000+0000","last_active":"2024-09-17T21:18:44.298736+0000","last_peered":"2024-09-17T21:18:44.298736+0000","last_clean":"2024-09-17T21:18:44.298736+0000","last_became_active":"2024-09-17T21:18:40.810689+0000","last_became_peered":"2024-09-17T21:18:40.810689+0000","last_unstale":"2024-09-17T21:18:44.298736+0000","last_undegraded":"2024-09-17T21:18:44.298736+0000","last_fullsized":"2024-09-17T21:18:44.298736+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:15:27.955549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088525599999999996,"stat_sum":{"num_bytes":2931776,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":184,"num_read_kb":1019,"num_write":17,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":96832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"46'139","reported_seq":1386,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:44.306078+0000","last_change":"2024-09-17T21:18:43.278035+0000","last_active":"2024-09-17T21:18:44.306078+0000","last_peered":"2024-09-17T21:18:44.306078+0000","last_clean":"2024-09-17T21:18:44.306078+0000","last_became_active":"2024-09-17T21:18:43.277411+0000","last_became_peered":"2024-09-17T21:18:43.277411+0000","last_unstale":"2024-09-17T21:18:44.306078+0000","last_undegraded":"2024-09-17T21:18:44.306078+0000","last_fullsized":"2024-09-17T21:18:44.306078+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:54:18.904883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013256229999999999,"stat_sum":{"num_bytes":7076906,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":2059,"num_read_kb":2580,"num_write":146,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":127572,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"46'131","reported_seq":2090,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:44.392385+0000","last_change":"2024-09-17T21:18:13.324697+0000","last_active":"2024-09-17T21:18:44.392385+0000","last_peered":"2024-09-17T21:18:44.392385+0000","last_clean":"2024-09-17T21:18:44.392385+0000","last_became_active":"2024-09-17T21:18:11.302254+0000","last_became_peered":"2024-09-17T21:18:11.302254+0000","last_unstale":"2024-09-17T21:18:44.392385+0000","last_undegraded":"2024-09-17T21:18:44.392385+0000","last_fullsized":"2024-09-17T21:18:44.392385+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:47:36.997274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018973079999999999,"stat_sum":{"num_bytes":15739422,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":3,"num_read":2251,"num_read_kb":8805,"num_write":169,"num_write_kb":6219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"44'14","reported_seq":244,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:43.285041+0000","last_change":"2024-09-17T21:18:13.326517+0000","last_active":"2024-09-17T21:18:43.285041+0000","last_peered":"2024-09-17T21:18:43.285041+0000","last_clean":"2024-09-17T21:18:43.285041+0000","last_became_active":"2024-09-17T21:18:11.302134+0000","last_became_peered":"2024-09-17T21:18:11.302134+0000","last_unstale":"2024-09-17T21:18:43.285041+0000","last_undegraded":"2024-09-17T21:18:43.285041+0000","last_fullsized":"2024-09-17T21:18:43.285041+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:03:06.349519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034186630000000002,"stat_sum":{"num_bytes":1048576,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":183,"num_read_kb":2223,"num_write":16,"num_write_kb":1050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"46'28","reported_seq":484,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:44.348488+0000","last_change":"2024-09-17T21:18:41.001528+0000","last_active":"2024-09-17T21:18:44.348488+0000","last_peered":"2024-09-17T21:18:44.348488+0000","last_clean":"2024-09-17T21:18:44.348488+0000","last_became_active":"2024-09-17T21:18:40.827008+0000","last_became_peered":"2024-09-17T21:18:40.827008+0000","last_unstale":"2024-09-17T21:18:44.348488+0000","last_undegraded":"2024-09-17T21:18:44.348488+0000","last_fullsized":"2024-09-17T21:18:44.348488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:20:25.638516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089504999999999999,"stat_sum":{"num_bytes":4260352,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":2,"num_read":389,"num_read_kb":1617,"num_write":28,"num_write_kb":843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":615936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"46'182","reported_seq":2744,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:44.419457+0000","last_change":"2024-09-17T21:18:13.314063+0000","last_active":"2024-09-17T21:18:44.419457+0000","last_peered":"2024-09-17T21:18:44.419457+0000","last_clean":"2024-09-17T21:18:44.419457+0000","last_became_active":"2024-09-17T21:18:11.301426+0000","last_became_peered":"2024-09-17T21:18:11.301426+0000","last_unstale":"2024-09-17T21:18:44.419457+0000","last_undegraded":"2024-09-17T21:18:44.419457+0000","last_fullsized":"2024-09-17T21:18:44.419457+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:15:59.268414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027366090000000001,"stat_sum":{"num_bytes":14090808,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":3999,"num_read_kb":8208,"num_write":161,"num_write_kb":1189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"46'15","reported_seq":901,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:44.041708+0000","last_change":"2024-09-17T21:18:13.314862+0000","last_active":"2024-09-17T21:18:44.041708+0000","last_peered":"2024-09-17T21:18:44.041708+0000","last_clean":"2024-09-17T21:18:44.041708+0000","last_became_active":"2024-09-17T21:18:11.300793+0000","last_became_peered":"2024-09-17T21:18:11.300793+0000","last_unstale":"2024-09-17T21:18:44.041708+0000","last_undegraded":"2024-09-17T21:18:44.041708+0000","last_fullsized":"2024-09-17T21:18:44.041708+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:12:04.321836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047673000000000002,"stat_sum":{"num_bytes":12744704,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":825,"num_read_kb":3567,"num_write":47,"num_write_kb":1515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"46'82","reported_seq":130,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:44.227216+0000","last_change":"2024-09-17T21:18:40.830426+0000","last_active":"2024-09-17T21:18:44.227216+0000","last_peered":"2024-09-17T21:18:44.227216+0000","last_clean":"2024-09-17T21:18:44.227216+0000","last_became_active":"2024-09-17T21:18:40.801216+0000","last_became_peered":"2024-09-17T21:18:40.801216+0000","last_unstale":"2024-09-17T21:18:44.227216+0000","last_undegraded":"2024-09-17T21:18:44.227216+0000","last_fullsized":"2024-09-17T21:18:44.227216+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:00.815542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9056304,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":796,"num_read_kb":3066,"num_write":117,"num_write_kb":1005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":428592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"46'26","reported_seq":358,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:44.439882+0000","last_change":"2024-09-17T21:18:40.854226+0000","last_active":"2024-09-17T21:18:44.439882+0000","last_peered":"2024-09-17T21:18:44.439882+0000","last_clean":"2024-09-17T21:18:44.439882+0000","last_became_active":"2024-09-17T21:18:40.812620+0000","last_became_peered":"2024-09-17T21:18:40.812620+0000","last_unstale":"2024-09-17T21:18:44.439882+0000","last_undegraded":"2024-09-17T21:18:44.439882+0000","last_fullsized":"2024-09-17T21:18:44.439882+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:48:49.335251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088962999999999998,"stat_sum":{"num_bytes":6785024,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":277,"num_read_kb":2907,"num_write":36,"num_write_kb":1370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":793088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"46'91","reported_seq":300,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:44.111771+0000","last_change":"2024-09-17T21:18:13.324809+0000","last_active":"2024-09-17T21:18:44.111771+0000","last_peered":"2024-09-17T21:18:44.111771+0000","last_clean":"2024-09-17T21:18:44.111771+0000","last_became_active":"2024-09-17T21:18:11.303835+0000","last_became_peered":"2024-09-17T21:18:11.303835+0000","last_unstale":"2024-09-17T21:18:44.111771+0000","last_undegraded":"2024-09-17T21:18:44.111771+0000","last_fullsized":"2024-09-17T21:18:44.111771+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:10:20.232060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001825658,"stat_sum":{"num_bytes":3637322,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":493,"num_read_kb":2099,"num_write":152,"num_write_kb":10323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"46'41","reported_seq":1246,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:44.386134+0000","last_change":"2024-09-17T21:18:41.006288+0000","last_active":"2024-09-17T21:18:44.386134+0000","last_peered":"2024-09-17T21:18:44.386134+0000","last_clean":"2024-09-17T21:18:44.386134+0000","last_became_active":"2024-09-17T21:18:40.820709+0000","last_became_peered":"2024-09-17T21:18:40.820709+0000","last_unstale":"2024-09-17T21:18:44.386134+0000","last_undegraded":"2024-09-17T21:18:44.386134+0000","last_fullsized":"2024-09-17T21:18:44.386134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:36:23.611638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027112149999999999,"stat_sum":{"num_bytes":13129766,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":923,"num_read_kb":5184,"num_write":49,"num_write_kb":1776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1160742,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"43'44","reported_seq":706,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:43.850186+0000","last_change":"2024-09-17T21:18:13.307234+0000","last_active":"2024-09-17T21:18:43.850186+0000","last_peered":"2024-09-17T21:18:43.850186+0000","last_clean":"2024-09-17T21:18:43.850186+0000","last_became_active":"2024-09-17T21:18:11.299817+0000","last_became_peered":"2024-09-17T21:18:11.299817+0000","last_unstale":"2024-09-17T21:18:43.850186+0000","last_undegraded":"2024-09-17T21:18:43.850186+0000","last_fullsized":"2024-09-17T21:18:43.850186+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:03:03.399096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002281113,"stat_sum":{"num_bytes":10600976,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":621,"num_read_kb":4051,"num_write":53,"num_write_kb":11299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":70,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:43.274061+0000","last_change":"2024-09-17T21:18:03.059715+0000","last_active":"2024-09-17T21:18:43.274061+0000","last_peered":"2024-09-17T21:18:43.274061+0000","last_clean":"2024-09-17T21:18:43.274061+0000","last_became_active":"2024-09-17T21:18:01.213047+0000","last_became_peered":"2024-09-17T21:18:01.213047+0000","last_unstale":"2024-09-17T21:18:43.274061+0000","last_undegraded":"2024-09-17T21:18:43.274061+0000","last_fullsized":"2024-09-17T21:18:43.274061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:46:35.222373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"46'144","reported_seq":761,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:44.410308+0000","last_change":"2024-09-17T21:18:40.976309+0000","last_active":"2024-09-17T21:18:44.410308+0000","last_peered":"2024-09-17T21:18:44.410308+0000","last_clean":"2024-09-17T21:18:44.410308+0000","last_became_active":"2024-09-17T21:18:40.818354+0000","last_became_peered":"2024-09-17T21:18:40.818354+0000","last_unstale":"2024-09-17T21:18:44.410308+0000","last_undegraded":"2024-09-17T21:18:44.410308+0000","last_fullsized":"2024-09-17T21:18:44.410308+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:25:16.525893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084327200000000005,"stat_sum":{"num_bytes":21660462,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1183,"num_read_kb":20004,"num_write":247,"num_write_kb":13041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8773670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":70,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:43.281293+0000","last_change":"2024-09-17T21:18:03.073440+0000","last_active":"2024-09-17T21:18:43.281293+0000","last_peered":"2024-09-17T21:18:43.281293+0000","last_clean":"2024-09-17T21:18:43.281293+0000","last_became_active":"2024-09-17T21:18:01.048806+0000","last_became_peered":"2024-09-17T21:18:01.048806+0000","last_unstale":"2024-09-17T21:18:43.281293+0000","last_undegraded":"2024-09-17T21:18:43.281293+0000","last_fullsized":"2024-09-17T21:18:43.281293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:26.487163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094161400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"43'73","reported_seq":839,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:43.849771+0000","last_change":"2024-09-17T21:18:17.280432+0000","last_active":"2024-09-17T21:18:43.849771+0000","last_peered":"2024-09-17T21:18:43.849771+0000","last_clean":"2024-09-17T21:18:43.849771+0000","last_became_active":"2024-09-17T21:18:17.279625+0000","last_became_peered":"2024-09-17T21:18:17.279625+0000","last_unstale":"2024-09-17T21:18:43.849771+0000","last_undegraded":"2024-09-17T21:18:43.849771+0000","last_fullsized":"2024-09-17T21:18:43.849771+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:46:31.826275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21278756,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":888,"num_read_kb":4450,"num_write":98,"num_write_kb":11083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":76,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:43.273931+0000","last_change":"2024-09-17T21:18:40.821956+0000","last_active":"2024-09-17T21:18:43.273931+0000","last_peered":"2024-09-17T21:18:43.273931+0000","last_clean":"2024-09-17T21:18:43.273931+0000","last_became_active":"2024-09-17T21:18:40.820574+0000","last_became_peered":"2024-09-17T21:18:40.820574+0000","last_unstale":"2024-09-17T21:18:43.273931+0000","last_undegraded":"2024-09-17T21:18:43.273931+0000","last_fullsized":"2024-09-17T21:18:43.273931+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:35:05.837282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099219900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"46'33","reported_seq":1298,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:43.888781+0000","last_change":"2024-09-17T21:18:13.315662+0000","last_active":"2024-09-17T21:18:43.888781+0000","last_peered":"2024-09-17T21:18:43.888781+0000","last_clean":"2024-09-17T21:18:43.888781+0000","last_became_active":"2024-09-17T21:18:11.309133+0000","last_became_peered":"2024-09-17T21:18:11.309133+0000","last_unstale":"2024-09-17T21:18:43.888781+0000","last_undegraded":"2024-09-17T21:18:43.888781+0000","last_fullsized":"2024-09-17T21:18:43.888781+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:18:17.567718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078465700000000002,"stat_sum":{"num_bytes":21127168,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":1220,"num_read_kb":12574,"num_write":43,"num_write_kb":5995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":76,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:43.281077+0000","last_change":"2024-09-17T21:18:40.818736+0000","last_active":"2024-09-17T21:18:43.281077+0000","last_peered":"2024-09-17T21:18:43.281077+0000","last_clean":"2024-09-17T21:18:43.281077+0000","last_became_active":"2024-09-17T21:18:40.818309+0000","last_became_peered":"2024-09-17T21:18:40.818309+0000","last_unstale":"2024-09-17T21:18:43.281077+0000","last_undegraded":"2024-09-17T21:18:43.281077+0000","last_fullsized":"2024-09-17T21:18:43.281077+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:42:22.148483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006222,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"46'97","reported_seq":1867,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:44.432160+0000","last_change":"2024-09-17T21:18:13.327202+0000","last_active":"2024-09-17T21:18:44.432160+0000","last_peered":"2024-09-17T21:18:44.432160+0000","last_clean":"2024-09-17T21:18:44.432160+0000","last_became_active":"2024-09-17T21:18:11.317127+0000","last_became_peered":"2024-09-17T21:18:11.317127+0000","last_unstale":"2024-09-17T21:18:44.432160+0000","last_undegraded":"2024-09-17T21:18:44.432160+0000","last_fullsized":"2024-09-17T21:18:44.432160+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:51:15.047007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036493720000000001,"stat_sum":{"num_bytes":16750623,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1932,"num_read_kb":11024,"num_write":80,"num_write_kb":14243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"34'24","reported_seq":578,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:43.289946+0000","last_change":"2024-09-17T21:18:40.963953+0000","last_active":"2024-09-17T21:18:43.289946+0000","last_peered":"2024-09-17T21:18:43.289946+0000","last_clean":"2024-09-17T21:18:43.289946+0000","last_became_active":"2024-09-17T21:18:40.820910+0000","last_became_peered":"2024-09-17T21:18:40.820910+0000","last_unstale":"2024-09-17T21:18:43.289946+0000","last_undegraded":"2024-09-17T21:18:43.289946+0000","last_fullsized":"2024-09-17T21:18:43.289946+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:40:54.868628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084988700000000002,"stat_sum":{"num_bytes":3272704,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":469,"num_read_kb":2254,"num_write":26,"num_write_kb":1536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":978944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":72,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:43.284925+0000","last_change":"2024-09-17T21:18:03.070336+0000","last_active":"2024-09-17T21:18:43.284925+0000","last_peered":"2024-09-17T21:18:43.284925+0000","last_clean":"2024-09-17T21:18:43.284925+0000","last_became_active":"2024-09-17T21:18:01.049336+0000","last_became_peered":"2024-09-17T21:18:01.049336+0000","last_unstale":"2024-09-17T21:18:43.284925+0000","last_undegraded":"2024-09-17T21:18:43.284925+0000","last_fullsized":"2024-09-17T21:18:43.284925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:56:24.329470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023986049999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"46'22","reported_seq":411,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:44.438335+0000","last_change":"2024-09-17T21:18:13.314742+0000","last_active":"2024-09-17T21:18:44.438335+0000","last_peered":"2024-09-17T21:18:44.438335+0000","last_clean":"2024-09-17T21:18:44.438335+0000","last_became_active":"2024-09-17T21:18:11.308568+0000","last_became_peered":"2024-09-17T21:18:11.308568+0000","last_unstale":"2024-09-17T21:18:44.438335+0000","last_undegraded":"2024-09-17T21:18:44.438335+0000","last_fullsized":"2024-09-17T21:18:44.438335+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:38:49.051549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049125000000000004,"stat_sum":{"num_bytes":9221152,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":340,"num_read_kb":6684,"num_write":29,"num_write_kb":2842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":48,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:43.280861+0000","last_change":"2024-09-17T21:18:17.280358+0000","last_active":"2024-09-17T21:18:43.280861+0000","last_peered":"2024-09-17T21:18:43.280861+0000","last_clean":"2024-09-17T21:18:43.280861+0000","last_became_active":"2024-09-17T21:18:17.279474+0000","last_became_peered":"2024-09-17T21:18:17.279474+0000","last_unstale":"2024-09-17T21:18:43.280861+0000","last_undegraded":"2024-09-17T21:18:43.280861+0000","last_fullsized":"2024-09-17T21:18:43.280861+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:55:54.183527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"46'64","reported_seq":681,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:44.347920+0000","last_change":"2024-09-17T21:18:40.906152+0000","last_active":"2024-09-17T21:18:44.347920+0000","last_peered":"2024-09-17T21:18:44.347920+0000","last_clean":"2024-09-17T21:18:44.347920+0000","last_became_active":"2024-09-17T21:18:40.816571+0000","last_became_peered":"2024-09-17T21:18:40.816571+0000","last_unstale":"2024-09-17T21:18:44.347920+0000","last_undegraded":"2024-09-17T21:18:44.347920+0000","last_fullsized":"2024-09-17T21:18:44.347920+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:42:11.111911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066398399999999995,"stat_sum":{"num_bytes":7944240,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":755,"num_read_kb":7658,"num_write":96,"num_write_kb":4313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":4070960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":70,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:43.273507+0000","last_change":"2024-09-17T21:18:03.059156+0000","last_active":"2024-09-17T21:18:43.273507+0000","last_peered":"2024-09-17T21:18:43.273507+0000","last_clean":"2024-09-17T21:18:43.273507+0000","last_became_active":"2024-09-17T21:18:01.049180+0000","last_became_peered":"2024-09-17T21:18:01.049180+0000","last_unstale":"2024-09-17T21:18:43.273507+0000","last_undegraded":"2024-09-17T21:18:43.273507+0000","last_fullsized":"2024-09-17T21:18:43.273507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:27.069794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077408,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"46'18","reported_seq":618,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:43.511423+0000","last_change":"2024-09-17T21:18:13.315294+0000","last_active":"2024-09-17T21:18:43.511423+0000","last_peered":"2024-09-17T21:18:43.511423+0000","last_clean":"2024-09-17T21:18:43.511423+0000","last_became_active":"2024-09-17T21:18:11.294154+0000","last_became_peered":"2024-09-17T21:18:11.294154+0000","last_unstale":"2024-09-17T21:18:43.511423+0000","last_undegraded":"2024-09-17T21:18:43.511423+0000","last_fullsized":"2024-09-17T21:18:43.511423+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:02:36.888598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049408099999999995,"stat_sum":{"num_bytes":33486864,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":547,"num_read_kb":16970,"num_write":33,"num_write_kb":8516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":71,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:43.285018+0000","last_change":"2024-09-17T21:18:03.069091+0000","last_active":"2024-09-17T21:18:43.285018+0000","last_peered":"2024-09-17T21:18:43.285018+0000","last_clean":"2024-09-17T21:18:43.285018+0000","last_became_active":"2024-09-17T21:18:01.044275+0000","last_became_peered":"2024-09-17T21:18:01.044275+0000","last_unstale":"2024-09-17T21:18:43.285018+0000","last_undegraded":"2024-09-17T21:18:43.285018+0000","last_fullsized":"2024-09-17T21:18:43.285018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:28:27.066081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001145884,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":173,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:43.273542+0000","last_change":"2024-09-17T21:18:40.829967+0000","last_active":"2024-09-17T21:18:43.273542+0000","last_peered":"2024-09-17T21:18:43.273542+0000","last_clean":"2024-09-17T21:18:43.273542+0000","last_became_active":"2024-09-17T21:18:40.803358+0000","last_became_peered":"2024-09-17T21:18:40.803358+0000","last_unstale":"2024-09-17T21:18:43.273542+0000","last_undegraded":"2024-09-17T21:18:43.273542+0000","last_fullsized":"2024-09-17T21:18:43.273542+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_clean_scrub_stamp":"2024-09-17T21:17:55.982339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:57:35.748912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"46'109","reported_seq":1576,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:43.849210+0000","last_change":"2024-09-17T21:18:17.280477+0000","last_active":"2024-09-17T21:18:43.849210+0000","last_peered":"2024-09-17T21:18:43.849210+0000","last_clean":"2024-09-17T21:18:43.849210+0000","last_became_active":"2024-09-17T21:18:17.279644+0000","last_became_peered":"2024-09-17T21:18:17.279644+0000","last_unstale":"2024-09-17T21:18:43.849210+0000","last_undegraded":"2024-09-17T21:18:43.849210+0000","last_fullsized":"2024-09-17T21:18:43.849210+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:22:46.205078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6194192,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":2511,"num_read_kb":10707,"num_write":87,"num_write_kb":13816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"46'155","reported_seq":1157,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:44.302013+0000","last_change":"2024-09-17T21:18:13.313980+0000","last_active":"2024-09-17T21:18:44.302013+0000","last_peered":"2024-09-17T21:18:44.302013+0000","last_clean":"2024-09-17T21:18:44.302013+0000","last_became_active":"2024-09-17T21:18:11.304021+0000","last_became_peered":"2024-09-17T21:18:11.304021+0000","last_unstale":"2024-09-17T21:18:44.302013+0000","last_undegraded":"2024-09-17T21:18:44.302013+0000","last_fullsized":"2024-09-17T21:18:44.302013+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:16:45.940695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023749890000000001,"stat_sum":{"num_bytes":23323689,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1610,"num_read_kb":7296,"num_write":292,"num_write_kb":2628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"46'31","reported_seq":305,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:43.858924+0000","last_change":"2024-09-17T21:18:13.307298+0000","last_active":"2024-09-17T21:18:43.858924+0000","last_peered":"2024-09-17T21:18:43.858924+0000","last_clean":"2024-09-17T21:18:43.858924+0000","last_became_active":"2024-09-17T21:18:11.294907+0000","last_became_peered":"2024-09-17T21:18:11.294907+0000","last_unstale":"2024-09-17T21:18:43.858924+0000","last_undegraded":"2024-09-17T21:18:43.858924+0000","last_fullsized":"2024-09-17T21:18:43.858924+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:46:36.226997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021951829999999999,"stat_sum":{"num_bytes":6134354,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":233,"num_read_kb":1514,"num_write":32,"num_write_kb":454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"46'62","reported_seq":508,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:43.961779+0000","last_change":"2024-09-17T21:18:40.868870+0000","last_active":"2024-09-17T21:18:43.961779+0000","last_peered":"2024-09-17T21:18:43.961779+0000","last_clean":"2024-09-17T21:18:43.961779+0000","last_became_active":"2024-09-17T21:18:40.816043+0000","last_became_peered":"2024-09-17T21:18:40.816043+0000","last_unstale":"2024-09-17T21:18:43.961779+0000","last_undegraded":"2024-09-17T21:18:43.961779+0000","last_fullsized":"2024-09-17T21:18:43.961779+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:03.414113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048704800000000002,"stat_sum":{"num_bytes":9380368,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":1,"num_read":675,"num_read_kb":1846,"num_write":58,"num_write_kb":10651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":485392,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"46'77","reported_seq":2040,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:43.689255+0000","last_change":"2024-09-17T21:18:40.925472+0000","last_active":"2024-09-17T21:18:43.689255+0000","last_peered":"2024-09-17T21:18:43.689255+0000","last_clean":"2024-09-17T21:18:43.689255+0000","last_became_active":"2024-09-17T21:18:40.821090+0000","last_became_peered":"2024-09-17T21:18:40.821090+0000","last_unstale":"2024-09-17T21:18:43.689255+0000","last_undegraded":"2024-09-17T21:18:43.689255+0000","last_fullsized":"2024-09-17T21:18:43.689255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:27.762213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015319789999999999,"stat_sum":{"num_bytes":26867200,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":2052,"num_read_kb":6572,"num_write":57,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":81408,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"37'13","reported_seq":118,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:43.273820+0000","last_change":"2024-09-17T21:18:13.306351+0000","last_active":"2024-09-17T21:18:43.273820+0000","last_peered":"2024-09-17T21:18:43.273820+0000","last_clean":"2024-09-17T21:18:43.273820+0000","last_became_active":"2024-09-17T21:18:11.316999+0000","last_became_peered":"2024-09-17T21:18:11.316999+0000","last_unstale":"2024-09-17T21:18:43.273820+0000","last_undegraded":"2024-09-17T21:18:43.273820+0000","last_fullsized":"2024-09-17T21:18:43.273820+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:06:41.052708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015238949999999999,"stat_sum":{"num_bytes":491520,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":37,"num_read_kb":976,"num_write":14,"num_write_kb":711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"46'154","reported_seq":2723,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:43.849917+0000","last_change":"2024-09-17T21:18:13.327081+0000","last_active":"2024-09-17T21:18:43.849917+0000","last_peered":"2024-09-17T21:18:43.849917+0000","last_clean":"2024-09-17T21:18:43.849917+0000","last_became_active":"2024-09-17T21:18:11.301652+0000","last_became_peered":"2024-09-17T21:18:11.301652+0000","last_unstale":"2024-09-17T21:18:43.849917+0000","last_undegraded":"2024-09-17T21:18:43.849917+0000","last_fullsized":"2024-09-17T21:18:43.849917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:32:51.480109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003632209,"stat_sum":{"num_bytes":28959232,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":2,"num_read":3469,"num_read_kb":18097,"num_write":138,"num_write_kb":15635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"46'21","reported_seq":492,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:43.849486+0000","last_change":"2024-09-17T21:18:13.325500+0000","last_active":"2024-09-17T21:18:43.849486+0000","last_peered":"2024-09-17T21:18:43.849486+0000","last_clean":"2024-09-17T21:18:43.849486+0000","last_became_active":"2024-09-17T21:18:11.307060+0000","last_became_peered":"2024-09-17T21:18:11.307060+0000","last_unstale":"2024-09-17T21:18:43.849486+0000","last_undegraded":"2024-09-17T21:18:43.849486+0000","last_fullsized":"2024-09-17T21:18:43.849486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:25:56.428327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002454758,"stat_sum":{"num_bytes":2159631,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":417,"num_read_kb":1604,"num_write":17,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"46'59","reported_seq":2599,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:44.192567+0000","last_change":"2024-09-17T21:18:40.876377+0000","last_active":"2024-09-17T21:18:44.192567+0000","last_peered":"2024-09-17T21:18:44.192567+0000","last_clean":"2024-09-17T21:18:44.192567+0000","last_became_active":"2024-09-17T21:18:40.817254+0000","last_became_peered":"2024-09-17T21:18:40.817254+0000","last_unstale":"2024-09-17T21:18:44.192567+0000","last_undegraded":"2024-09-17T21:18:44.192567+0000","last_fullsized":"2024-09-17T21:18:44.192567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:25:52.284951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001519261,"stat_sum":{"num_bytes":1272906,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1510,"num_read_kb":3999,"num_write":32,"num_write_kb":731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":538698,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"46'56","reported_seq":1099,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:43.849386+0000","last_change":"2024-09-17T21:18:13.305680+0000","last_active":"2024-09-17T21:18:43.849386+0000","last_peered":"2024-09-17T21:18:43.849386+0000","last_clean":"2024-09-17T21:18:43.849386+0000","last_became_active":"2024-09-17T21:18:11.299946+0000","last_became_peered":"2024-09-17T21:18:11.299946+0000","last_unstale":"2024-09-17T21:18:43.849386+0000","last_undegraded":"2024-09-17T21:18:43.849386+0000","last_fullsized":"2024-09-17T21:18:43.849386+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:55:39.292103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225884,"stat_sum":{"num_bytes":14459988,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":1077,"num_read_kb":4376,"num_write":67,"num_write_kb":946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"46'50","reported_seq":1336,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:44.385917+0000","last_change":"2024-09-17T21:18:40.967754+0000","last_active":"2024-09-17T21:18:44.385917+0000","last_peered":"2024-09-17T21:18:44.385917+0000","last_clean":"2024-09-17T21:18:44.385917+0000","last_became_active":"2024-09-17T21:18:40.814521+0000","last_became_peered":"2024-09-17T21:18:40.814521+0000","last_unstale":"2024-09-17T21:18:44.385917+0000","last_undegraded":"2024-09-17T21:18:44.385917+0000","last_fullsized":"2024-09-17T21:18:44.385917+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:26:04.719525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013045489999999999,"stat_sum":{"num_bytes":20606543,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":1234,"num_read_kb":5529,"num_write":57,"num_write_kb":2170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1310287,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"46'128","reported_seq":3594,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:18:44.195776+0000","last_change":"2024-09-17T21:18:13.306520+0000","last_active":"2024-09-17T21:18:44.195776+0000","last_peered":"2024-09-17T21:18:44.195776+0000","last_clean":"2024-09-17T21:18:44.195776+0000","last_became_active":"2024-09-17T21:18:11.294953+0000","last_became_peered":"2024-09-17T21:18:11.294953+0000","last_unstale":"2024-09-17T21:18:44.195776+0000","last_undegraded":"2024-09-17T21:18:44.195776+0000","last_fullsized":"2024-09-17T21:18:44.195776+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:42:36.620235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019790290000000002,"stat_sum":{"num_bytes":17824768,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":4352,"num_read_kb":13985,"num_write":96,"num_write_kb":970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":393518296,"num_objects":268,"num_object_clones":45,"num_object_copies":536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":268,"num_whiteouts":43,"num_read":39511,"num_read_kb":203445,"num_write":2590,"num_write_kb":147994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":19462121,"num_keys_recovered":53,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":154095616,"data_stored":220767087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2198,"ondisk_log_size":2198,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56716,"kb_used_data":25528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94315124,"statfs":{"total":96636764160,"available":96578686976,"internally_reserved":0,"allocated":26140672,"data_stored":42196659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63484,"kb_used_data":32300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94308356,"statfs":{"total":96636764160,"available":96571756544,"internally_reserved":0,"allocated":33075200,"data_stored":45157505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39736,"kb_used_data":8548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332104,"statfs":{"total":96636764160,"available":96596074496,"internally_reserved":0,"allocated":8753152,"data_stored":8279998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65908,"kb_used_data":34724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94305932,"statfs":{"total":96636764160,"available":96569274368,"internally_reserved":0,"allocated":35557376,"data_stored":53076497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":0,"seq":55834574861,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607566,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59560,"kb_used_data":24272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94312280,"statfs":{"total":96636764160,"available":96575774720,"internally_reserved":0,"allocated":24854528,"data_stored":36585900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28000256,"data_stored":36273087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":35360768,"data_stored":53181603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24436736,"data_stored":36464486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7831552,"data_stored":7652334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":32800768,"data_stored":45171401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":25665536,"data_stored":42024176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:18:45.732 INFO:tasks.thrashosds.thrasher:Setting 3.8 to [2, 3] 2024-09-17T21:18:45.733 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.8', '2', '3'] 2024-09-17T21:18:45.733 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.8 2 3 2024-09-17T21:18:45.733 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:45.725+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:45.829 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:45.825+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:45.872 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:45.872 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:45.872 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:18:45.872 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:45.885 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:18:45.930 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:45.929+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:45.948 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-17T21:18:46.031 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:46.029+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:46.057 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:46.057 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:46.057 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:18:46.057 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:18:46.057 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:46.074 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:18:46.131 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:46.124+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:46.141 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:18:46.232 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:46.228+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:46.260 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:46.260 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:18:46.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:18:46.260 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:46.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.260 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:46.261 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:18:46.261 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.244712227000001, 2024-09-17T21:18:46.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:18:46.261 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:46.261 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:46.261 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:46.261 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:46.261 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:46.261 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:46.261 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:18:46.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.261 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:46.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:46.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:18:46.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:46.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:18:46.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:46.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:18:46.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:18:46.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:46.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:18:46.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:18:46.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:46.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:18:46.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:18:46.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:46.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:18:46.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:18:46.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:18:46.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:18:46.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:46.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:18:46.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:18:46.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:18:46.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:18:46.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:46.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:18:46.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:18:46.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:46.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:18:46.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:18:46.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:46.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:18:46.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:18:46.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:46.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:18:46.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:18:46.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:46.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:18:46.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:18:46.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:46.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:18:46.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:18:46.267 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.267 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:46.267 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.268 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:18:46.268 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:18:46.268 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 29.664547946999999, 2024-09-17T21:18:46.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:18:46.268 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:46.268 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:46.268 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:46.268 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:46.268 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:18:46.268 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:46.268 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:18:46.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.269 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:46.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:46.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:18:46.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:46.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:18:46.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:46.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:18:46.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:18:46.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:46.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:18:46.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:18:46.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:46.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:18:46.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:18:46.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:46.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:18:46.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:18:46.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:18:46.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:18:46.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:46.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:18:46.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:18:46.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:18:46.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:18:46.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:46.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:18:46.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:18:46.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:18:46.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:18:46.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:46.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:18:46.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:18:46.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:46.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:18:46.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:18:46.273 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.274 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:46.274 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.274 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:18:46.274 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:18:46.274 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 29.664450635000001, 2024-09-17T21:18:46.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:18:46.274 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:46.274 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:46.274 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:46.274 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:46.274 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:46.274 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:46.275 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:18:46.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.275 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:46.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:46.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:18:46.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:46.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:18:46.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:46.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:18:46.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:18:46.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:46.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:18:46.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:18:46.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:46.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:18:46.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:18:46.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:46.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:18:46.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:18:46.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:18:46.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:18:46.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:46.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:18:46.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:18:46.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:18:46.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:18:46.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:46.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:18:46.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:18:46.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:18:46.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:18:46.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:18:46.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:18:46.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:18:46.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:18:46.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:18:46.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:46.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:18:46.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:18:46.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:46.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:18:46.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:18:46.282 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.282 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:46.282 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.282 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:18:46.282 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:18:46.282 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 29.662363834000001, 2024-09-17T21:18:46.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:18:46.282 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:46.282 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:46.282 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:46.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:46.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:18:46.283 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:46.283 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:18:46.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.283 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:46.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:46.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:18:46.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:46.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:18:46.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:46.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:18:46.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:18:46.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:46.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:18:46.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:18:46.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:46.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:18:46.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:18:46.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:46.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:18:46.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:18:46.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:18:46.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:18:46.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:18:46.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:18:46.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:18:46.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:18:46.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:18:46.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:46.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:18:46.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:18:46.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:18:46.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:18:46.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:46.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:18:46.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:18:46.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:46.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:18:46.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:18:46.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:46.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:18:46.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:18:46.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:46.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:18:46.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:18:46.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:46.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:18:46.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:18:46.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:46.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:18:46.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:18:46.289 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.289 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:46.289 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.289 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:282 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:46.290 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:18:46.290 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.030658573, 2024-09-17T21:18:46.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023571801, 2024-09-17T21:18:46.290 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:46.290 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:46.290 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:46.290 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:46.290 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:46.290 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:46.290 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 282 2024-09-17T21:18:46.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.290 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:46.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:46.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:18:46.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:46.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:18:46.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:46.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227820+0000", 2024-09-17T21:18:46.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.11e-06 2024-09-17T21:18:46.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:46.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227825+0000", 2024-09-17T21:18:46.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8099999999999997e-06 2024-09-17T21:18:46.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:46.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227848+0000", 2024-09-17T21:18:46.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2949999999999999e-05 2024-09-17T21:18:46.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:46.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227862+0000", 2024-09-17T21:18:46.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:18:46.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.247977+0000", 2024-09-17T21:18:46.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020114968 2024-09-17T21:18:46.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:46.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251141+0000", 2024-09-17T21:18:46.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031640689999999998 2024-09-17T21:18:46.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:46.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251391+0000", 2024-09-17T21:18:46.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025018499999999999 2024-09-17T21:18:46.294 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.294 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:46.294 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.294 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:283 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:46.294 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:18:46.294 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.030596734, 2024-09-17T21:18:46.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024022274999999999, 2024-09-17T21:18:46.294 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:46.294 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:46.295 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:46.295 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:46.295 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:46.295 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:46.295 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 283 2024-09-17T21:18:46.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.295 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:46.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:46.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:18:46.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:46.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:18:46.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:46.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227882+0000", 2024-09-17T21:18:46.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-06 2024-09-17T21:18:46.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:46.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227887+0000", 2024-09-17T21:18:46.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1769999999999997e-06 2024-09-17T21:18:46.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:46.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227905+0000", 2024-09-17T21:18:46.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7235999999999999e-05 2024-09-17T21:18:46.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:46.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227918+0000", 2024-09-17T21:18:46.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3387000000000001e-05 2024-09-17T21:18:46.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251469+0000", 2024-09-17T21:18:46.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023550894999999999 2024-09-17T21:18:46.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:46.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251669+0000", 2024-09-17T21:18:46.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019992499999999999 2024-09-17T21:18:46.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:46.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251903+0000", 2024-09-17T21:18:46.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000234412 2024-09-17T21:18:46.298 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.298 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:46.298 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.299 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:284 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:46.299 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:18:46.299 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.030541181, 2024-09-17T21:18:46.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024450084, 2024-09-17T21:18:46.299 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:46.299 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:46.299 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:46.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:46.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:46.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:46.299 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 284 2024-09-17T21:18:46.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.300 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:46.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:46.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:18:46.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:46.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:18:46.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:46.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227937+0000", 2024-09-17T21:18:46.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6700000000000002e-07 2024-09-17T21:18:46.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:46.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227943+0000", 2024-09-17T21:18:46.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:18:46.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:46.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227961+0000", 2024-09-17T21:18:46.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.827e-05 2024-09-17T21:18:46.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:46.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227975+0000", 2024-09-17T21:18:46.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.394e-05 2024-09-17T21:18:46.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251977+0000", 2024-09-17T21:18:46.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024001340999999999 2024-09-17T21:18:46.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:46.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252169+0000", 2024-09-17T21:18:46.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019271199999999999 2024-09-17T21:18:46.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:46.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252387+0000", 2024-09-17T21:18:46.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021726100000000001 2024-09-17T21:18:46.303 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.303 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:46.303 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.303 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:285 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:46.303 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:18:46.303 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.030484891, 2024-09-17T21:18:46.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024884791, 2024-09-17T21:18:46.303 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:46.304 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:46.304 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:46.304 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:46.304 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:46.304 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:46.304 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 285 2024-09-17T21:18:46.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.304 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:46.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:46.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:18:46.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:46.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:18:46.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:46.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227994+0000", 2024-09-17T21:18:46.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1429999999999999e-06 2024-09-17T21:18:46.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:46.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227999+0000", 2024-09-17T21:18:46.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1669999999999998e-06 2024-09-17T21:18:46.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:46.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228015+0000", 2024-09-17T21:18:46.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6056e-05 2024-09-17T21:18:46.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:46.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228028+0000", 2024-09-17T21:18:46.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3196e-05 2024-09-17T21:18:46.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252458+0000", 2024-09-17T21:18:46.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024429751999999999 2024-09-17T21:18:46.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:46.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252650+0000", 2024-09-17T21:18:46.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019187800000000001 2024-09-17T21:18:46.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:46.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252878+0000", 2024-09-17T21:18:46.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022759899999999999 2024-09-17T21:18:46.308 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.308 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:46.308 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.308 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:286 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:46.308 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:18:46.308 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.030429132, 2024-09-17T21:18:46.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025299318000000001, 2024-09-17T21:18:46.308 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:46.308 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:46.308 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:46.308 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:46.309 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:46.309 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:46.309 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 286 2024-09-17T21:18:46.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.309 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:46.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:46.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:18:46.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:46.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:18:46.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:46.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228050+0000", 2024-09-17T21:18:46.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:18:46.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:46.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228055+0000", 2024-09-17T21:18:46.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4960000000000004e-06 2024-09-17T21:18:46.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:46.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228075+0000", 2024-09-17T21:18:46.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0109999999999999e-05 2024-09-17T21:18:46.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:46.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228089+0000", 2024-09-17T21:18:46.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4154e-05 2024-09-17T21:18:46.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252952+0000", 2024-09-17T21:18:46.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024862934 2024-09-17T21:18:46.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:46.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253141+0000", 2024-09-17T21:18:46.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000188782 2024-09-17T21:18:46.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:46.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253348+0000", 2024-09-17T21:18:46.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020684499999999999 2024-09-17T21:18:46.312 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.312 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:46.312 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.312 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:439 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:46.313 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241357+0000", 2024-09-17T21:18:46.313 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.017120328000001, 2024-09-17T21:18:46.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023114866000000001, 2024-09-17T21:18:46.313 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:46.313 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:46.313 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:46.313 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:46.313 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:46.313 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:46.313 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 439 2024-09-17T21:18:46.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.313 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:46.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:46.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241357+0000", 2024-09-17T21:18:46.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:46.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241357+0000", 2024-09-17T21:18:46.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:46.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241358+0000", 2024-09-17T21:18:46.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.113e-06 2024-09-17T21:18:46.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:46.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241364+0000", 2024-09-17T21:18:46.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3900000000000001e-06 2024-09-17T21:18:46.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:46.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241382+0000", 2024-09-17T21:18:46.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7759999999999999e-05 2024-09-17T21:18:46.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:46.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241396+0000", 2024-09-17T21:18:46.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4303e-05 2024-09-17T21:18:46.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263586+0000", 2024-09-17T21:18:46.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022189952999999998 2024-09-17T21:18:46.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:46.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263683+0000", 2024-09-17T21:18:46.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7145000000000006e-05 2024-09-17T21:18:46.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:46.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264472+0000", 2024-09-17T21:18:46.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00078920200000000002 2024-09-17T21:18:46.317 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.317 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:46.317 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.317 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:440 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:46.317 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241415+0000", 2024-09-17T21:18:46.317 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.017062387999999, 2024-09-17T21:18:46.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022660967000000001, 2024-09-17T21:18:46.317 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:46.317 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:46.317 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:46.317 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:46.317 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:46.318 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:46.318 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 440 2024-09-17T21:18:46.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.318 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:46.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:46.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241415+0000", 2024-09-17T21:18:46.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:46.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241415+0000", 2024-09-17T21:18:46.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:46.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241416+0000", 2024-09-17T21:18:46.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.133e-06 2024-09-17T21:18:46.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:46.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241422+0000", 2024-09-17T21:18:46.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7470000000000002e-06 2024-09-17T21:18:46.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:46.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241440+0000", 2024-09-17T21:18:46.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8029999999999998e-05 2024-09-17T21:18:46.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:46.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241454+0000", 2024-09-17T21:18:46.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3626e-05 2024-09-17T21:18:46.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263842+0000", 2024-09-17T21:18:46.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022388308999999999 2024-09-17T21:18:46.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:46.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263936+0000", 2024-09-17T21:18:46.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4190000000000005e-05 2024-09-17T21:18:46.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:46.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264076+0000", 2024-09-17T21:18:46.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000139932 2024-09-17T21:18:46.321 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.322 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:46.322 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.322 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:441 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2162688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:46.322 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241478+0000", 2024-09-17T21:18:46.322 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.016999848999999, 2024-09-17T21:18:46.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022827729000000001, 2024-09-17T21:18:46.322 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:46.322 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:46.322 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:46.322 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:46.323 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:46.323 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:46.323 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 441 2024-09-17T21:18:46.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.323 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:46.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:46.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241478+0000", 2024-09-17T21:18:46.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:46.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241478+0000", 2024-09-17T21:18:46.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:46.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241479+0000", 2024-09-17T21:18:46.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6530000000000001e-06 2024-09-17T21:18:46.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:46.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241485+0000", 2024-09-17T21:18:46.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3839999999999996e-06 2024-09-17T21:18:46.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:46.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241503+0000", 2024-09-17T21:18:46.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8233e-05 2024-09-17T21:18:46.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:46.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241517+0000", 2024-09-17T21:18:46.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3893000000000001e-05 2024-09-17T21:18:46.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264107+0000", 2024-09-17T21:18:46.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022590328999999999 2024-09-17T21:18:46.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:46.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264195+0000", 2024-09-17T21:18:46.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7255000000000004e-05 2024-09-17T21:18:46.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:46.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264306+0000", 2024-09-17T21:18:46.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000110982 2024-09-17T21:18:46.326 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.326 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:46.326 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.327 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:442 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2293760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:46.327 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241535+0000", 2024-09-17T21:18:46.327 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.016942243000001, 2024-09-17T21:18:46.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023012353999999999, 2024-09-17T21:18:46.327 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:46.327 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:46.327 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:46.327 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:46.327 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:46.327 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:46.327 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 442 2024-09-17T21:18:46.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.327 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:46.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:46.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241535+0000", 2024-09-17T21:18:46.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:46.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241535+0000", 2024-09-17T21:18:46.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:46.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241537+0000", 2024-09-17T21:18:46.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1939999999999999e-06 2024-09-17T21:18:46.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:46.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241557+0000", 2024-09-17T21:18:46.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.054e-05 2024-09-17T21:18:46.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:46.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241594+0000", 2024-09-17T21:18:46.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6782999999999997e-05 2024-09-17T21:18:46.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:46.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241611+0000", 2024-09-17T21:18:46.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6833e-05 2024-09-17T21:18:46.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264340+0000", 2024-09-17T21:18:46.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022729668000000001 2024-09-17T21:18:46.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:46.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264426+0000", 2024-09-17T21:18:46.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5180999999999996e-05 2024-09-17T21:18:46.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:46.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264548+0000", 2024-09-17T21:18:46.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012215500000000001 2024-09-17T21:18:46.331 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.331 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:46.331 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.331 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:443 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:46.331 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:18:46.331 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.016841827, 2024-09-17T21:18:46.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023179569000000001, 2024-09-17T21:18:46.331 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:46.331 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:46.332 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:46.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:46.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:46.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:46.332 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 443 2024-09-17T21:18:46.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.332 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:46.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:46.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:18:46.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:46.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:18:46.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:46.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241637+0000", 2024-09-17T21:18:46.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.46e-06 2024-09-17T21:18:46.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:46.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241643+0000", 2024-09-17T21:18:46.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1970000000000001e-06 2024-09-17T21:18:46.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:46.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241662+0000", 2024-09-17T21:18:46.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8712999999999999e-05 2024-09-17T21:18:46.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:46.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241676+0000", 2024-09-17T21:18:46.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.393e-05 2024-09-17T21:18:46.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264596+0000", 2024-09-17T21:18:46.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022919688000000001 2024-09-17T21:18:46.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:46.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264690+0000", 2024-09-17T21:18:46.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4145000000000001e-05 2024-09-17T21:18:46.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:46.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264815+0000", 2024-09-17T21:18:46.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012543600000000001 2024-09-17T21:18:46.335 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.335 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:46.335 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.336 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:444 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:46.336 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:18:46.336 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.016780776999999, 2024-09-17T21:18:46.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023260383999999999, 2024-09-17T21:18:46.336 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:46.336 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:46.336 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:46.336 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:46.336 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:46.336 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:46.336 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 444 2024-09-17T21:18:46.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.337 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:46.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:46.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:18:46.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:46.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:18:46.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:46.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241698+0000", 2024-09-17T21:18:46.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2270000000000001e-06 2024-09-17T21:18:46.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:46.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241704+0000", 2024-09-17T21:18:46.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9800000000000003e-06 2024-09-17T21:18:46.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:46.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241725+0000", 2024-09-17T21:18:46.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0812999999999999e-05 2024-09-17T21:18:46.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:46.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241739+0000", 2024-09-17T21:18:46.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3773e-05 2024-09-17T21:18:46.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264850+0000", 2024-09-17T21:18:46.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023111326000000001 2024-09-17T21:18:46.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:46.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264897+0000", 2024-09-17T21:18:46.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6638999999999998e-05 2024-09-17T21:18:46.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:46.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264957+0000", 2024-09-17T21:18:46.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0625999999999999e-05 2024-09-17T21:18:46.340 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.340 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:46.340 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.340 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:445 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:46.340 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:18:46.340 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.016720107999999, 2024-09-17T21:18:46.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023315144999999999, 2024-09-17T21:18:46.341 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:46.341 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:46.341 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:46.341 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:46.341 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:46.341 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:46.341 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 445 2024-09-17T21:18:46.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.341 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:46.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:46.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:18:46.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:46.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:18:46.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:46.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241759+0000", 2024-09-17T21:18:46.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-06 2024-09-17T21:18:46.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:46.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241764+0000", 2024-09-17T21:18:46.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5899999999999998e-06 2024-09-17T21:18:46.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:46.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241784+0000", 2024-09-17T21:18:46.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9204000000000001e-05 2024-09-17T21:18:46.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:46.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241798+0000", 2024-09-17T21:18:46.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3953e-05 2024-09-17T21:18:46.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264977+0000", 2024-09-17T21:18:46.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023179182 2024-09-17T21:18:46.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:46.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.265018+0000", 2024-09-17T21:18:46.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1171999999999998e-05 2024-09-17T21:18:46.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:46.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.265073+0000", 2024-09-17T21:18:46.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4741000000000003e-05 2024-09-17T21:18:46.345 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.345 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:46.345 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.345 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:677 3.13 3:cd0d74f9:::rbd_data.11527824718.000000000000000a:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:46.345 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.250552+0000", 2024-09-17T21:18:46.345 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.007926119, 2024-09-17T21:18:46.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022610023999999999, 2024-09-17T21:18:46.345 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:46.346 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:46.346 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:46.346 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:46.346 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:46.346 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:46.346 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 677 2024-09-17T21:18:46.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.346 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:46.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:46.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250552+0000", 2024-09-17T21:18:46.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:46.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250552+0000", 2024-09-17T21:18:46.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:46.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250553+0000", 2024-09-17T21:18:46.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4699999999999999e-06 2024-09-17T21:18:46.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:46.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250571+0000", 2024-09-17T21:18:46.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8303000000000001e-05 2024-09-17T21:18:46.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:46.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250590+0000", 2024-09-17T21:18:46.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8723e-05 2024-09-17T21:18:46.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:46.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250603+0000", 2024-09-17T21:18:46.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3307e-05 2024-09-17T21:18:46.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.272971+0000", 2024-09-17T21:18:46.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022368140000000002 2024-09-17T21:18:46.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:46.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273051+0000", 2024-09-17T21:18:46.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9339999999999996e-05 2024-09-17T21:18:46.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:46.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273162+0000", 2024-09-17T21:18:46.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000110741 2024-09-17T21:18:46.349 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.349 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:46.350 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.350 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:678 3.13 3:cd0d74f9:::rbd_data.11527824718.000000000000000a:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:46.350 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.250622+0000", 2024-09-17T21:18:46.350 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.00785524, 2024-09-17T21:18:46.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023048448999999999, 2024-09-17T21:18:46.350 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:46.350 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:46.350 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:46.350 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:46.350 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:46.350 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:46.350 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 678 2024-09-17T21:18:46.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.351 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:46.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:46.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250622+0000", 2024-09-17T21:18:46.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:46.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250622+0000", 2024-09-17T21:18:46.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:46.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250624+0000", 2024-09-17T21:18:46.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4270000000000001e-06 2024-09-17T21:18:46.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:46.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250629+0000", 2024-09-17T21:18:46.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6030000000000004e-06 2024-09-17T21:18:46.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:46.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250648+0000", 2024-09-17T21:18:46.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8337000000000001e-05 2024-09-17T21:18:46.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:46.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250661+0000", 2024-09-17T21:18:46.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3446e-05 2024-09-17T21:18:46.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273275+0000", 2024-09-17T21:18:46.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022613292 2024-09-17T21:18:46.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:46.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273467+0000", 2024-09-17T21:18:46.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019237900000000001 2024-09-17T21:18:46.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:46.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273671+0000", 2024-09-17T21:18:46.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020396499999999999 2024-09-17T21:18:46.354 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.354 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:46.354 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.354 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:679 3.13 3:cd0d74f9:::rbd_data.11527824718.000000000000000a:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:46.354 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:18:46.354 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.007798686999999, 2024-09-17T21:18:46.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023284978000000001, 2024-09-17T21:18:46.354 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:46.355 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:46.355 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:46.355 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:46.355 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:46.355 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:46.355 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 679 2024-09-17T21:18:46.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.355 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:46.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:46.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:18:46.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:46.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:18:46.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:46.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250680+0000", 2024-09-17T21:18:46.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:18:46.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:46.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250685+0000", 2024-09-17T21:18:46.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1970000000000004e-06 2024-09-17T21:18:46.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:46.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250703+0000", 2024-09-17T21:18:46.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7396000000000001e-05 2024-09-17T21:18:46.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:46.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250716+0000", 2024-09-17T21:18:46.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.29e-05 2024-09-17T21:18:46.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273740+0000", 2024-09-17T21:18:46.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023024405000000001 2024-09-17T21:18:46.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:46.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273834+0000", 2024-09-17T21:18:46.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4124999999999998e-05 2024-09-17T21:18:46.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:46.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273964+0000", 2024-09-17T21:18:46.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012963499999999999 2024-09-17T21:18:46.358 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.358 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:46.358 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.359 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:680 3.13 3:cd0d74f9:::rbd_data.11527824718.000000000000000a:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:46.359 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:18:46.359 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.007743607, 2024-09-17T21:18:46.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023687792999999999, 2024-09-17T21:18:46.359 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:46.359 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:46.359 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:46.359 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:46.359 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:46.359 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:46.359 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 680 2024-09-17T21:18:46.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.360 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:46.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:46.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:18:46.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:46.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:18:46.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:46.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:46.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250735+0000", 2024-09-17T21:18:46.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2669999999999999e-06 2024-09-17T21:18:46.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:46.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250740+0000", 2024-09-17T21:18:46.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8060000000000002e-06 2024-09-17T21:18:46.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:46.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250757+0000", 2024-09-17T21:18:46.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7303e-05 2024-09-17T21:18:46.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:46.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250771+0000", 2024-09-17T21:18:46.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3876999999999999e-05 2024-09-17T21:18:46.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:46.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274022+0000", 2024-09-17T21:18:46.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023251135999999999 2024-09-17T21:18:46.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:46.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274211+0000", 2024-09-17T21:18:46.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018813599999999999 2024-09-17T21:18:46.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:46.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:46.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:46.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274422+0000", 2024-09-17T21:18:46.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000211268 2024-09-17T21:18:46.363 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.363 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:46.363 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.363 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:46.363 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:46.363 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:46.364 INFO:teuthology.orchestra.run.smithi083.stderr:set 3.8 pg_upmap mapping to [2,3] 2024-09-17T21:18:46.365 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-17T21:18:46.366 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:18:46.367 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:18:46.368 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:18:46.368 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:18:46.368 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:18:46.368 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:18:46.369 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:18:46.369 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:18:46.369 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:18:46.369 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:46.360+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:46.380 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:18:46.380 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:18:46.401 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:18:46.401 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:18:46.466 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:46.465+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:46.495 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:46.495 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:46.510 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:18:46.598 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:46.597+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:46.668 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:46.660+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:46.670 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:46.670 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:46.685 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:18:46.774 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:46.773+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:46.852 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:46.852 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:46.866 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:18:46.870 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:46.864+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:46.971 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:46.969+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:47.037 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:47.038 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:47.051 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:18:47.094 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:47.093+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:47.174 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:47.172+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:47.215 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:47.215 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:47.229 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:18:47.274 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:47.273+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:47.350 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:47.351 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:47.364 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:18:47.412 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:47.408+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:47.476 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:47.472+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:47.495 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:47.495 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:47.509 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:18:47.600 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:47.596+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:47.672 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:47.672 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:47.678 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:47.677+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:47.687 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:18:47.779 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:47.777+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:47.853 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:47.853 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:47.867 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:18:47.879 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:47.877+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:47.980 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:47.976+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:47.989 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:47.989 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:48.004 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:18:48.092 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:48.088+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:48.171 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:48.171 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:48.181 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:48.176+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:48.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:18:48.282 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:48.281+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:48.354 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:48.354 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:48.368 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:18:48.369 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:18:48.413 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:48.412+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:48.418 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:18:48.500 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:48.496+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:48.536 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:48.537 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:48.552 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:18:48.595 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:18:48.595 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:18:48.596 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:48.592+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:48.613 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:18:48.614 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:18:48.630 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:18:48.631 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:18:48.651 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:18:48.651 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:18:48.671 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:18:48.672 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:18:48.686 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:48.685+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:48.692 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:18:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:18:48.709 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:48.710 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:48.723 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:18:48.808 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:48.804+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:48.887 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:48.885+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:48.889 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:48.890 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:18:48.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:18:48.890 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:18:48.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.890 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:12 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:48.890 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:48.890 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 35.555610287, 2024-09-17T21:18:48.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061340989999999996, 2024-09-17T21:18:48.890 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:48.890 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:48.890 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:48.890 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:48.891 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:48.891 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:48.891 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12 2024-09-17T21:18:48.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.891 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:48.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:48.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:48.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:48.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:48.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:48.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332336+0000", 2024-09-17T21:18:48.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5519999999999999e-06 2024-09-17T21:18:48.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:48.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332342+0000", 2024-09-17T21:18:48.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7180000000000001e-06 2024-09-17T21:18:48.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:48.893 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332352+0000", 2024-09-17T21:18:48.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:18:48.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:48.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332359+0000", 2024-09-17T21:18:48.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3479999999999998e-06 2024-09-17T21:18:48.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:48.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332413+0000", 2024-09-17T21:18:48.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4447999999999998e-05 2024-09-17T21:18:48.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-17T21:18:48.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332417+0000", 2024-09-17T21:18:48.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.895e-06 2024-09-17T21:18:48.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:48.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334755+0000", 2024-09-17T21:18:48.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023377189999999998 2024-09-17T21:18:48.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:48.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334903+0000", 2024-09-17T21:18:48.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014778999999999999 2024-09-17T21:18:48.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:18:48.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.335187+0000", 2024-09-17T21:18:48.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000284715 2024-09-17T21:18:48.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:48.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.337731+0000", 2024-09-17T21:18:48.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002543741 2024-09-17T21:18:48.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:48.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338382+0000", 2024-09-17T21:18:48.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00065029200000000001 2024-09-17T21:18:48.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:48.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338418+0000", 2024-09-17T21:18:48.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.697e-05 2024-09-17T21:18:48.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:48.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338469+0000", 2024-09-17T21:18:48.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0154999999999997e-05 2024-09-17T21:18:48.897 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.898 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:48.898 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.898 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "pg_update_log_missing(3.1e epoch 28/27 rep_tid 81 entries 28'41 (0'0) error 3:7b23fd98:::rbd_header.1152c01721e5:head by client.4488.0:561 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-17T21:18:48.898 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:18:48.898 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.606336160000001, 2024-09-17T21:18:48.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0051146969999999996, 2024-09-17T21:18:48.898 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:48.898 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:48.898 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:18:48.898 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:48.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:48.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:18:48.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:48.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:18:48.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:48.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281610+0000", 2024-09-17T21:18:48.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.16e-07 2024-09-17T21:18:48.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:48.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281626+0000", 2024-09-17T21:18:48.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6039999999999999e-05 2024-09-17T21:18:48.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:48.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281666+0000", 2024-09-17T21:18:48.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0364000000000002e-05 2024-09-17T21:18:48.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:48.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281710+0000", 2024-09-17T21:18:48.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3596000000000001e-05 2024-09-17T21:18:48.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:48.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281992+0000", 2024-09-17T21:18:48.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028273099999999999 2024-09-17T21:18:48.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:48.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286723+0000", 2024-09-17T21:18:48.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047310499999999997 2024-09-17T21:18:48.901 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.901 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:48.901 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.902 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4488.0:562 3.1e e28/27 3:7b23fd98:::rbd_header.1152c01721e5:head v 28'42, mlcod=0'0)", 2024-09-17T21:18:48.902 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:18:48.902 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.606092371999999, 2024-09-17T21:18:48.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049275020000000003, 2024-09-17T21:18:48.902 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:48.902 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:48.902 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:48.902 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:48.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:48.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:18:48.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:48.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:18:48.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:48.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281853+0000", 2024-09-17T21:18:48.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2999999999999999e-07 2024-09-17T21:18:48.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:48.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281875+0000", 2024-09-17T21:18:48.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1386000000000002e-05 2024-09-17T21:18:48.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:48.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281892+0000", 2024-09-17T21:18:48.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7637000000000001e-05 2024-09-17T21:18:48.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:48.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281908+0000", 2024-09-17T21:18:48.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5187e-05 2024-09-17T21:18:48.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:48.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.282517+0000", 2024-09-17T21:18:48.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00060989799999999999 2024-09-17T21:18:48.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:48.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.282548+0000", 2024-09-17T21:18:48.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0910000000000001e-05 2024-09-17T21:18:48.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:48.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286729+0000", 2024-09-17T21:18:48.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0041810850000000002 2024-09-17T21:18:48.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:48.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286780+0000", 2024-09-17T21:18:48.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0569000000000003e-05 2024-09-17T21:18:48.906 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.906 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:48.906 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.906 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:609 3.d e28/23 3:b7278be9:::rbd_data.1152c6af092e.000000000000008f:head v 28'6, mlcod=28'4)", 2024-09-17T21:18:48.906 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:18:48.906 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 30.773148084999999, 2024-09-17T21:18:48.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049797280000000001, 2024-09-17T21:18:48.906 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:48.906 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:48.906 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:48.906 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:48.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:48.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:18:48.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:48.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:18:48.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:48.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114800+0000", 2024-09-17T21:18:48.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:18:48.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:48.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114915+0000", 2024-09-17T21:18:48.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115053 2024-09-17T21:18:48.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:48.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114951+0000", 2024-09-17T21:18:48.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6165999999999999e-05 2024-09-17T21:18:48.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:48.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114965+0000", 2024-09-17T21:18:48.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4364e-05 2024-09-17T21:18:48.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:48.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115097+0000", 2024-09-17T21:18:48.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013139200000000001 2024-09-17T21:18:48.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:48.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115151+0000", 2024-09-17T21:18:48.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4422000000000002e-05 2024-09-17T21:18:48.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:48.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.119633+0000", 2024-09-17T21:18:48.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044818820000000004 2024-09-17T21:18:48.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:48.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.119776+0000", 2024-09-17T21:18:48.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000143609 2024-09-17T21:18:48.910 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.910 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:48.910 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.910 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:782 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'63, mlcod=28'60)", 2024-09-17T21:18:48.910 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:18:48.910 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 30.673180057, 2024-09-17T21:18:48.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006167283, 2024-09-17T21:18:48.911 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:48.911 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:48.911 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:48.911 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:48.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:48.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:18:48.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:48.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:18:48.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:48.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214767+0000", 2024-09-17T21:18:48.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:18:48.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:48.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214779+0000", 2024-09-17T21:18:48.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2627e-05 2024-09-17T21:18:48.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:48.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214805+0000", 2024-09-17T21:18:48.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5570000000000001e-05 2024-09-17T21:18:48.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:48.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214823+0000", 2024-09-17T21:18:48.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8202999999999998e-05 2024-09-17T21:18:48.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:48.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215170+0000", 2024-09-17T21:18:48.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034720399999999999 2024-09-17T21:18:48.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:48.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215235+0000", 2024-09-17T21:18:48.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4276000000000006e-05 2024-09-17T21:18:48.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:48.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220859+0000", 2024-09-17T21:18:48.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056240500000000002 2024-09-17T21:18:48.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:48.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220932+0000", 2024-09-17T21:18:48.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3503000000000004e-05 2024-09-17T21:18:48.914 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.915 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:48.915 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.915 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:783 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'64, mlcod=28'60)", 2024-09-17T21:18:48.915 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:18:48.915 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 30.672493214999999, 2024-09-17T21:18:48.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0055240869999999996, 2024-09-17T21:18:48.915 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:48.915 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:48.915 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:48.915 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:48.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:48.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:18:48.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:48.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:18:48.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:48.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215454+0000", 2024-09-17T21:18:48.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0269999999999998e-06 2024-09-17T21:18:48.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:48.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215468+0000", 2024-09-17T21:18:48.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4816e-05 2024-09-17T21:18:48.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:48.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215501+0000", 2024-09-17T21:18:48.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2957000000000003e-05 2024-09-17T21:18:48.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:48.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215524+0000", 2024-09-17T21:18:48.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2782999999999999e-05 2024-09-17T21:18:48.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:48.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216675+0000", 2024-09-17T21:18:48.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011507869999999999 2024-09-17T21:18:48.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:48.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216728+0000", 2024-09-17T21:18:48.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3112999999999998e-05 2024-09-17T21:18:48.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:48.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220938+0000", 2024-09-17T21:18:48.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042097380000000002 2024-09-17T21:18:48.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:48.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220976+0000", 2024-09-17T21:18:48.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7866e-05 2024-09-17T21:18:48.919 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.919 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:48.919 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.919 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:785 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'65, mlcod=28'60)", 2024-09-17T21:18:48.920 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:18:48.920 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 30.671907166, 2024-09-17T21:18:48.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049795409999999997, 2024-09-17T21:18:48.920 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:48.920 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:48.920 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:48.920 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:48.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:48.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:18:48.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:48.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:18:48.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:48.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216039+0000", 2024-09-17T21:18:48.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.793e-06 2024-09-17T21:18:48.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:48.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216053+0000", 2024-09-17T21:18:48.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.345e-05 2024-09-17T21:18:48.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:48.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216092+0000", 2024-09-17T21:18:48.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9180000000000001e-05 2024-09-17T21:18:48.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:48.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216106+0000", 2024-09-17T21:18:48.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:18:48.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:48.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217569+0000", 2024-09-17T21:18:48.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001462938 2024-09-17T21:18:48.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:48.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217607+0000", 2024-09-17T21:18:48.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7267000000000001e-05 2024-09-17T21:18:48.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:48.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220980+0000", 2024-09-17T21:18:48.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003373544 2024-09-17T21:18:48.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:48.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221017+0000", 2024-09-17T21:18:48.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7116000000000002e-05 2024-09-17T21:18:48.923 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.924 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:48.924 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.924 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:786 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'66, mlcod=28'60)", 2024-09-17T21:18:48.924 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:18:48.924 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 30.671206362, 2024-09-17T21:18:48.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057966650000000003, 2024-09-17T21:18:48.924 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:48.924 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:48.924 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:48.924 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:48.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:48.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:18:48.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:48.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:18:48.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:48.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216740+0000", 2024-09-17T21:18:48.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5600000000000001e-06 2024-09-17T21:18:48.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:48.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216752+0000", 2024-09-17T21:18:48.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1477e-05 2024-09-17T21:18:48.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:48.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216779+0000", 2024-09-17T21:18:48.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7093e-05 2024-09-17T21:18:48.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:48.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216817+0000", 2024-09-17T21:18:48.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8275999999999997e-05 2024-09-17T21:18:48.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:48.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218387+0000", 2024-09-17T21:18:48.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015703749999999999 2024-09-17T21:18:48.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:48.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218432+0000", 2024-09-17T21:18:48.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4910000000000002e-05 2024-09-17T21:18:48.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:48.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222432+0000", 2024-09-17T21:18:48.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039997189999999997 2024-09-17T21:18:48.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:48.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222535+0000", 2024-09-17T21:18:48.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103255 2024-09-17T21:18:48.928 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.928 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:48.928 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.929 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:18:48.929 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:18:48.929 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 30.670584529999999, 2024-09-17T21:18:48.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:18:48.929 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:48.929 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:48.929 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:48.929 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:48.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:48.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:18:48.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:48.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:18:48.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:48.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:18:48.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:18:48.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:48.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:18:48.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:18:48.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:48.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:18:48.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:18:48.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:48.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:18:48.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:18:48.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:48.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:18:48.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:18:48.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:48.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:18:48.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:18:48.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:48.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:18:48.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:18:48.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:48.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:18:48.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:18:48.934 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.934 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:48.935 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.935 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:788 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'68, mlcod=28'60)", 2024-09-17T21:18:48.935 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:18:48.935 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 30.669950098000001, 2024-09-17T21:18:48.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058495320000000002, 2024-09-17T21:18:48.935 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:48.935 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:48.935 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:48.935 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:48.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:48.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:18:48.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:48.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:18:48.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:48.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217996+0000", 2024-09-17T21:18:48.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.376e-06 2024-09-17T21:18:48.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:48.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218006+0000", 2024-09-17T21:18:48.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.01e-05 2024-09-17T21:18:48.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:48.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218030+0000", 2024-09-17T21:18:48.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4026999999999999e-05 2024-09-17T21:18:48.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:48.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218042+0000", 2024-09-17T21:18:48.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1977000000000001e-05 2024-09-17T21:18:48.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:48.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221081+0000", 2024-09-17T21:18:48.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030385960000000002 2024-09-17T21:18:48.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:48.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221115+0000", 2024-09-17T21:18:48.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3772999999999997e-05 2024-09-17T21:18:48.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:48.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223749+0000", 2024-09-17T21:18:48.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0026340629999999999 2024-09-17T21:18:48.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:48.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223844+0000", 2024-09-17T21:18:48.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5619999999999996e-05 2024-09-17T21:18:48.939 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.939 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:48.939 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.939 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:790 3.5 e28/23 3:a14b1a37:::rbd_data.1152c6af092e.000000000000002f:head v 28'1, mlcod=0'0)", 2024-09-17T21:18:48.939 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:18:48.939 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 30.665545772000002, 2024-09-17T21:18:48.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059622240000000003, 2024-09-17T21:18:48.939 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:48.939 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:48.939 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:48.940 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:48.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:48.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:18:48.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:48.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:18:48.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:48.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222400+0000", 2024-09-17T21:18:48.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.02e-06 2024-09-17T21:18:48.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:48.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222411+0000", 2024-09-17T21:18:48.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.044e-05 2024-09-17T21:18:48.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:48.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222440+0000", 2024-09-17T21:18:48.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.932e-05 2024-09-17T21:18:48.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:48.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222458+0000", 2024-09-17T21:18:48.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8615999999999999e-05 2024-09-17T21:18:48.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:48.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222609+0000", 2024-09-17T21:18:48.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015102299999999999 2024-09-17T21:18:48.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:48.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222657+0000", 2024-09-17T21:18:48.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7206000000000002e-05 2024-09-17T21:18:48.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:48.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228278+0000", 2024-09-17T21:18:48.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056208100000000004 2024-09-17T21:18:48.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:48.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228361+0000", 2024-09-17T21:18:48.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3788999999999993e-05 2024-09-17T21:18:48.943 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.944 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:48.944 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.944 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:884 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'76, mlcod=28'70)", 2024-09-17T21:18:48.944 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:18:48.944 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 30.618960891, 2024-09-17T21:18:48.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056451990000000001, 2024-09-17T21:18:48.944 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:48.944 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:48.944 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:48.944 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:48.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:48.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:18:48.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:48.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:18:48.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:48.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268986+0000", 2024-09-17T21:18:48.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9360000000000002e-06 2024-09-17T21:18:48.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:48.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269001+0000", 2024-09-17T21:18:48.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4684e-05 2024-09-17T21:18:48.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:48.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269026+0000", 2024-09-17T21:18:48.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5193000000000001e-05 2024-09-17T21:18:48.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:48.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269043+0000", 2024-09-17T21:18:48.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7306e-05 2024-09-17T21:18:48.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:48.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.270024+0000", 2024-09-17T21:18:48.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00098120600000000009 2024-09-17T21:18:48.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:48.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.270085+0000", 2024-09-17T21:18:48.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0486000000000003e-05 2024-09-17T21:18:48.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:48.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.274557+0000", 2024-09-17T21:18:48.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044723419999999998 2024-09-17T21:18:48.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:48.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.274629+0000", 2024-09-17T21:18:48.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2045999999999994e-05 2024-09-17T21:18:48.948 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.948 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:48.948 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.948 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:18:48.948 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:18:48.949 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 26.020283866, 2024-09-17T21:18:48.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:18:48.949 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:48.949 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:48.949 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:48.949 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:48.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:48.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:18:48.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:48.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:18:48.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:48.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:18:48.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:18:48.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:48.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:18:48.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:18:48.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:48.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:18:48.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:18:48.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:48.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:18:48.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:18:48.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:48.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:18:48.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:18:48.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:48.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:18:48.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:18:48.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:48.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:18:48.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:18:48.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:48.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:18:48.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:18:48.953 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.953 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:48.953 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.953 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:18:48.953 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:18:48.953 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 26.019018783, 2024-09-17T21:18:48.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:18:48.953 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:48.953 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:48.953 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:48.954 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:48.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:48.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:18:48.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:48.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:18:48.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:48.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:18:48.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:18:48.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:48.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:18:48.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:18:48.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:48.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:18:48.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:18:48.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:48.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:18:48.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:18:48.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:48.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:18:48.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:18:48.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:48.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:18:48.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:18:48.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:48.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:18:48.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:18:48.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:48.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:18:48.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:18:48.957 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.957 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:48.957 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.957 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:18:48.958 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:18:48.958 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 26.009239373, 2024-09-17T21:18:48.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:18:48.958 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:48.958 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:48.958 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:48.958 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:48.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:48.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:18:48.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:48.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:18:48.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:48.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:18:48.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:18:48.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:48.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:18:48.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:18:48.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:48.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:18:48.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:18:48.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:48.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:18:48.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:18:48.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:48.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:18:48.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:18:48.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:48.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:18:48.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:18:48.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:48.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:18:48.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:18:48.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:48.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:18:48.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:18:48.962 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.962 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:48.962 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.962 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:18:48.962 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:18:48.962 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 26.00051697, 2024-09-17T21:18:48.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:18:48.962 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:48.962 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:48.962 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:48.963 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:48.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:48.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:18:48.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:48.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:18:48.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:48.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:18:48.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:18:48.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:48.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:18:48.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:18:48.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:48.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:18:48.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:18:48.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:48.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:18:48.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:18:48.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:48.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:18:48.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:18:48.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:48.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:18:48.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:18:48.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:48.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:18:48.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:18:48.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:48.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:18:48.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:18:48.966 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.967 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:48.967 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.967 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4498.0:596 3.4 e32/23 3:20f262ea:::rbd_data.1152b51cb396.000000000000000d:head v 32'17, mlcod=30'15)", 2024-09-17T21:18:48.967 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.258354+0000", 2024-09-17T21:18:48.967 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.629591105999999, 2024-09-17T21:18:48.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0053727619999999997, 2024-09-17T21:18:48.967 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:48.967 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:48.967 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:48.967 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:48.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:48.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.258354+0000", 2024-09-17T21:18:48.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:48.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.258354+0000", 2024-09-17T21:18:48.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:48.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.258356+0000", 2024-09-17T21:18:48.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.798e-06 2024-09-17T21:18:48.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:48.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259198+0000", 2024-09-17T21:18:48.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084279199999999998 2024-09-17T21:18:48.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:48.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259229+0000", 2024-09-17T21:18:48.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0383999999999999e-05 2024-09-17T21:18:48.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:48.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259241+0000", 2024-09-17T21:18:48.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2139e-05 2024-09-17T21:18:48.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:48.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259341+0000", 2024-09-17T21:18:48.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010049 2024-09-17T21:18:48.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:48.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259380+0000", 2024-09-17T21:18:48.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9072000000000001e-05 2024-09-17T21:18:48.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:48.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.263668+0000", 2024-09-17T21:18:48.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042875389999999999 2024-09-17T21:18:48.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:48.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.263726+0000", 2024-09-17T21:18:48.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8548000000000003e-05 2024-09-17T21:18:48.971 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.971 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:48.971 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.971 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4520.0:6177 3.8 e34/27 3:1665de58:::rbd_data.11528c3b0529.0000000000000019:head v 34'43, mlcod=34'41)", 2024-09-17T21:18:48.971 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:30.509962+0000", 2024-09-17T21:18:48.971 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.377983115999999, 2024-09-17T21:18:48.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049320850000000001, 2024-09-17T21:18:48.971 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:48.971 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:48.972 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:48.972 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:48.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:48.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509962+0000", 2024-09-17T21:18:48.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:48.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509962+0000", 2024-09-17T21:18:48.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:48.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509964+0000", 2024-09-17T21:18:48.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.356e-06 2024-09-17T21:18:48.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:48.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509978+0000", 2024-09-17T21:18:48.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3434e-05 2024-09-17T21:18:48.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:48.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510001+0000", 2024-09-17T21:18:48.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3436000000000001e-05 2024-09-17T21:18:48.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:48.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510023+0000", 2024-09-17T21:18:48.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1739999999999999e-05 2024-09-17T21:18:48.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:48.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510193+0000", 2024-09-17T21:18:48.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017040600000000001 2024-09-17T21:18:48.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:48.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510250+0000", 2024-09-17T21:18:48.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6749000000000002e-05 2024-09-17T21:18:48.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:48.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.514823+0000", 2024-09-17T21:18:48.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0045728180000000002 2024-09-17T21:18:48.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:48.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.514894+0000", 2024-09-17T21:18:48.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1145999999999999e-05 2024-09-17T21:18:48.975 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.975 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:48.976 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.976 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4520.0:6179 3.7 e34/27 3:e77b9c77:::rbd_data.11528c3b0529.0000000000000023:head v 34'39, mlcod=30'37)", 2024-09-17T21:18:48.976 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:30.510076+0000", 2024-09-17T21:18:48.976 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.377869267000001, 2024-09-17T21:18:48.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00479934, 2024-09-17T21:18:48.976 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:48.976 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:48.976 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:48.976 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:48.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:48.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510076+0000", 2024-09-17T21:18:48.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:48.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510076+0000", 2024-09-17T21:18:48.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:48.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510076+0000", 2024-09-17T21:18:48.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1299999999999998e-07 2024-09-17T21:18:48.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:48.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510095+0000", 2024-09-17T21:18:48.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8637000000000001e-05 2024-09-17T21:18:48.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:48.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510122+0000", 2024-09-17T21:18:48.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7316000000000001e-05 2024-09-17T21:18:48.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:48.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510136+0000", 2024-09-17T21:18:48.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3824e-05 2024-09-17T21:18:48.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:48.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510700+0000", 2024-09-17T21:18:48.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00056388200000000003 2024-09-17T21:18:48.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:48.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510751+0000", 2024-09-17T21:18:48.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0416e-05 2024-09-17T21:18:48.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:48.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.514787+0000", 2024-09-17T21:18:48.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004036792 2024-09-17T21:18:48.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:48.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.514875+0000", 2024-09-17T21:18:48.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7559999999999995e-05 2024-09-17T21:18:48.980 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.980 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:48.980 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.980 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:18:48.980 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:18:48.981 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.615426405999999, 2024-09-17T21:18:48.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:18:48.981 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:48.981 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:48.981 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:48.981 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:48.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:48.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:18:48.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:48.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:18:48.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:48.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:48.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:18:48.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:18:48.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:48.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:18:48.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:18:48.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:48.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:18:48.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:18:48.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:48.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:18:48.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:18:48.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:48.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:18:48.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:18:48.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:48.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:18:48.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:18:48.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:48.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:18:48.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:18:48.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:48.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:48.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:48.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:18:48.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:18:48.984 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.984 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:48.984 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.984 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:48.985 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:48.985 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:48.986 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:18:48.987 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:48.985+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:49.089 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:49.088+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:49.137 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:49.137 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:49.137 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:18:49.137 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:49.153 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:18:49.202 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:49.201+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:49.288 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-17T21:18:49.290 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:49.289+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:49.293 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:49.294 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:49.294 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:18:49.294 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:18:49.294 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:49.308 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:18:49.398 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:49.397+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:49.473 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:18:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:18:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:18:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 52.456827291000003, 2024-09-17T21:18:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:18:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:18:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:18:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:18:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:18:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:18:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:18:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:18:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:18:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:18:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:18:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:18:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:18:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:18:49.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:49.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:18:49.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:18:49.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:18:49.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:18:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:18:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:18:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:18:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:18:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:18:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:18:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:18:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:18:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:18:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:18:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:18:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:18:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:18:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:18:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.876663010999998, 2024-09-17T21:18:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:18:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:18:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:18:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:18:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:18:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:18:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:18:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:18:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:18:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:18:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:18:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:18:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:18:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:18:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:18:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:18:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:18:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:18:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:18:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:18:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:18:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:18:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:18:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:18:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:18:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:18:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:18:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:18:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:18:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.876565698999997, 2024-09-17T21:18:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:18:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:49.488 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:49.488 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:49.488 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:49.488 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:18:49.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.488 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:49.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:49.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:18:49.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:49.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:18:49.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:49.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:18:49.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:18:49.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:49.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:18:49.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:18:49.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:18:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:18:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:18:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:18:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:18:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:18:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:18:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:18:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:18:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:18:49.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:49.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:18:49.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:18:49.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:18:49.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:18:49.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:18:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:18:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:18:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:18:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:18:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:18:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:18:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:18:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:18:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:18:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:18:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.874478898, 2024-09-17T21:18:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:18:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:18:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:18:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:18:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:18:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:18:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:18:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:49.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:18:49.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:18:49.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:49.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:18:49.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:18:49.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:49.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:18:49.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:18:49.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:18:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:18:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:18:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:18:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:18:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:18:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:18:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:18:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:18:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:18:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:18:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:18:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:18:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:18:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:18:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:18:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:18:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:18:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:18:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:18:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:18:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:18:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:18:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:282 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:18:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.242773636999999, 2024-09-17T21:18:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023571801, 2024-09-17T21:18:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 282 2024-09-17T21:18:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:18:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:18:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227820+0000", 2024-09-17T21:18:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.11e-06 2024-09-17T21:18:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227825+0000", 2024-09-17T21:18:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8099999999999997e-06 2024-09-17T21:18:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227848+0000", 2024-09-17T21:18:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2949999999999999e-05 2024-09-17T21:18:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227862+0000", 2024-09-17T21:18:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:18:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.247977+0000", 2024-09-17T21:18:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020114968 2024-09-17T21:18:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251141+0000", 2024-09-17T21:18:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031640689999999998 2024-09-17T21:18:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251391+0000", 2024-09-17T21:18:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025018499999999999 2024-09-17T21:18:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:283 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:18:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.242711798, 2024-09-17T21:18:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024022274999999999, 2024-09-17T21:18:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 283 2024-09-17T21:18:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:18:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:18:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227882+0000", 2024-09-17T21:18:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-06 2024-09-17T21:18:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227887+0000", 2024-09-17T21:18:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1769999999999997e-06 2024-09-17T21:18:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227905+0000", 2024-09-17T21:18:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7235999999999999e-05 2024-09-17T21:18:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227918+0000", 2024-09-17T21:18:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3387000000000001e-05 2024-09-17T21:18:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251469+0000", 2024-09-17T21:18:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023550894999999999 2024-09-17T21:18:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251669+0000", 2024-09-17T21:18:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019992499999999999 2024-09-17T21:18:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251903+0000", 2024-09-17T21:18:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000234412 2024-09-17T21:18:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:284 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:18:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.242656244999999, 2024-09-17T21:18:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024450084, 2024-09-17T21:18:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 284 2024-09-17T21:18:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:18:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:18:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227937+0000", 2024-09-17T21:18:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6700000000000002e-07 2024-09-17T21:18:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227943+0000", 2024-09-17T21:18:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:18:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227961+0000", 2024-09-17T21:18:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.827e-05 2024-09-17T21:18:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227975+0000", 2024-09-17T21:18:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.394e-05 2024-09-17T21:18:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251977+0000", 2024-09-17T21:18:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024001340999999999 2024-09-17T21:18:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252169+0000", 2024-09-17T21:18:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019271199999999999 2024-09-17T21:18:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252387+0000", 2024-09-17T21:18:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021726100000000001 2024-09-17T21:18:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.517 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:285 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:49.517 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:18:49.517 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.242599954999999, 2024-09-17T21:18:49.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024884791, 2024-09-17T21:18:49.517 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:49.517 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:49.517 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:49.517 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:49.517 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:49.517 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:49.517 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 285 2024-09-17T21:18:49.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.518 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:49.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:49.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:18:49.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:49.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:18:49.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:49.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227994+0000", 2024-09-17T21:18:49.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1429999999999999e-06 2024-09-17T21:18:49.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:49.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227999+0000", 2024-09-17T21:18:49.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1669999999999998e-06 2024-09-17T21:18:49.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:49.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228015+0000", 2024-09-17T21:18:49.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6056e-05 2024-09-17T21:18:49.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:49.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228028+0000", 2024-09-17T21:18:49.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3196e-05 2024-09-17T21:18:49.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252458+0000", 2024-09-17T21:18:49.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024429751999999999 2024-09-17T21:18:49.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:49.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252650+0000", 2024-09-17T21:18:49.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019187800000000001 2024-09-17T21:18:49.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:49.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252878+0000", 2024-09-17T21:18:49.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022759899999999999 2024-09-17T21:18:49.521 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.521 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:49.521 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.521 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:286 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:49.521 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:18:49.522 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.242544196000001, 2024-09-17T21:18:49.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025299318000000001, 2024-09-17T21:18:49.522 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:49.522 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:49.522 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:49.522 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:49.522 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:49.522 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:49.522 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 286 2024-09-17T21:18:49.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.522 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:49.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:49.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:18:49.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:49.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:18:49.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:49.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228050+0000", 2024-09-17T21:18:49.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:18:49.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:49.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228055+0000", 2024-09-17T21:18:49.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4960000000000004e-06 2024-09-17T21:18:49.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:49.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228075+0000", 2024-09-17T21:18:49.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0109999999999999e-05 2024-09-17T21:18:49.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:49.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228089+0000", 2024-09-17T21:18:49.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4154e-05 2024-09-17T21:18:49.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252952+0000", 2024-09-17T21:18:49.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024862934 2024-09-17T21:18:49.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:49.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253141+0000", 2024-09-17T21:18:49.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000188782 2024-09-17T21:18:49.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:49.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253348+0000", 2024-09-17T21:18:49.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020684499999999999 2024-09-17T21:18:49.526 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.526 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:49.526 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.526 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:439 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:49.526 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241357+0000", 2024-09-17T21:18:49.526 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.229235392, 2024-09-17T21:18:49.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023114866000000001, 2024-09-17T21:18:49.526 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:49.526 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:49.526 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:49.527 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:49.527 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:49.527 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:49.527 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 439 2024-09-17T21:18:49.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.527 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:49.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:49.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241357+0000", 2024-09-17T21:18:49.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:49.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241357+0000", 2024-09-17T21:18:49.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:49.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241358+0000", 2024-09-17T21:18:49.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.113e-06 2024-09-17T21:18:49.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:49.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241364+0000", 2024-09-17T21:18:49.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3900000000000001e-06 2024-09-17T21:18:49.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:49.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241382+0000", 2024-09-17T21:18:49.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7759999999999999e-05 2024-09-17T21:18:49.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:49.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241396+0000", 2024-09-17T21:18:49.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4303e-05 2024-09-17T21:18:49.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263586+0000", 2024-09-17T21:18:49.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022189952999999998 2024-09-17T21:18:49.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:49.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263683+0000", 2024-09-17T21:18:49.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7145000000000006e-05 2024-09-17T21:18:49.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:49.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264472+0000", 2024-09-17T21:18:49.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00078920200000000002 2024-09-17T21:18:49.530 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.530 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:49.531 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.531 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:440 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:49.531 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241415+0000", 2024-09-17T21:18:49.531 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.229177452, 2024-09-17T21:18:49.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022660967000000001, 2024-09-17T21:18:49.531 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:49.531 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:49.531 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:49.531 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:49.531 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:49.531 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:49.532 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 440 2024-09-17T21:18:49.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.532 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:49.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:49.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241415+0000", 2024-09-17T21:18:49.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:49.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241415+0000", 2024-09-17T21:18:49.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:49.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241416+0000", 2024-09-17T21:18:49.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.133e-06 2024-09-17T21:18:49.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:49.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241422+0000", 2024-09-17T21:18:49.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7470000000000002e-06 2024-09-17T21:18:49.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:49.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241440+0000", 2024-09-17T21:18:49.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8029999999999998e-05 2024-09-17T21:18:49.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:49.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241454+0000", 2024-09-17T21:18:49.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3626e-05 2024-09-17T21:18:49.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263842+0000", 2024-09-17T21:18:49.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022388308999999999 2024-09-17T21:18:49.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:49.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263936+0000", 2024-09-17T21:18:49.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4190000000000005e-05 2024-09-17T21:18:49.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:49.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264076+0000", 2024-09-17T21:18:49.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000139932 2024-09-17T21:18:49.535 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.535 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:49.535 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.536 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:441 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2162688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:49.536 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241478+0000", 2024-09-17T21:18:49.536 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.229114913, 2024-09-17T21:18:49.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022827729000000001, 2024-09-17T21:18:49.536 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:49.536 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:49.536 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:49.536 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:49.536 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:49.536 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:49.536 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 441 2024-09-17T21:18:49.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.537 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:49.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:49.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241478+0000", 2024-09-17T21:18:49.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:49.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241478+0000", 2024-09-17T21:18:49.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:49.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241479+0000", 2024-09-17T21:18:49.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6530000000000001e-06 2024-09-17T21:18:49.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:49.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241485+0000", 2024-09-17T21:18:49.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3839999999999996e-06 2024-09-17T21:18:49.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:49.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241503+0000", 2024-09-17T21:18:49.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8233e-05 2024-09-17T21:18:49.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:49.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241517+0000", 2024-09-17T21:18:49.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3893000000000001e-05 2024-09-17T21:18:49.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264107+0000", 2024-09-17T21:18:49.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022590328999999999 2024-09-17T21:18:49.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:49.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264195+0000", 2024-09-17T21:18:49.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7255000000000004e-05 2024-09-17T21:18:49.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:49.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264306+0000", 2024-09-17T21:18:49.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000110982 2024-09-17T21:18:49.540 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.540 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:49.540 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.540 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:442 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2293760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:49.540 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241535+0000", 2024-09-17T21:18:49.541 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.229057307, 2024-09-17T21:18:49.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023012353999999999, 2024-09-17T21:18:49.541 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:49.541 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:49.541 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:49.541 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:49.541 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:49.541 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:49.541 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 442 2024-09-17T21:18:49.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.541 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:49.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:49.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241535+0000", 2024-09-17T21:18:49.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:49.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241535+0000", 2024-09-17T21:18:49.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:49.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241537+0000", 2024-09-17T21:18:49.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1939999999999999e-06 2024-09-17T21:18:49.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:49.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241557+0000", 2024-09-17T21:18:49.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.054e-05 2024-09-17T21:18:49.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:49.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241594+0000", 2024-09-17T21:18:49.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6782999999999997e-05 2024-09-17T21:18:49.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:49.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241611+0000", 2024-09-17T21:18:49.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6833e-05 2024-09-17T21:18:49.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264340+0000", 2024-09-17T21:18:49.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022729668000000001 2024-09-17T21:18:49.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:49.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264426+0000", 2024-09-17T21:18:49.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5180999999999996e-05 2024-09-17T21:18:49.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:49.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264548+0000", 2024-09-17T21:18:49.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012215500000000001 2024-09-17T21:18:49.545 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.545 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:49.545 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.545 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:443 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:49.545 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:18:49.545 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.228956890999999, 2024-09-17T21:18:49.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023179569000000001, 2024-09-17T21:18:49.545 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:49.545 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:49.545 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:49.546 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:49.546 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:49.546 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:49.546 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 443 2024-09-17T21:18:49.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.546 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:49.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:49.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:18:49.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:49.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:18:49.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:49.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241637+0000", 2024-09-17T21:18:49.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.46e-06 2024-09-17T21:18:49.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:49.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241643+0000", 2024-09-17T21:18:49.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1970000000000001e-06 2024-09-17T21:18:49.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:49.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241662+0000", 2024-09-17T21:18:49.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8712999999999999e-05 2024-09-17T21:18:49.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:49.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241676+0000", 2024-09-17T21:18:49.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.393e-05 2024-09-17T21:18:49.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264596+0000", 2024-09-17T21:18:49.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022919688000000001 2024-09-17T21:18:49.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:49.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264690+0000", 2024-09-17T21:18:49.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4145000000000001e-05 2024-09-17T21:18:49.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:49.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264815+0000", 2024-09-17T21:18:49.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012543600000000001 2024-09-17T21:18:49.549 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.549 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:49.549 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.550 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:444 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:49.550 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:18:49.550 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.228895841, 2024-09-17T21:18:49.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023260383999999999, 2024-09-17T21:18:49.550 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:49.550 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:49.550 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:49.550 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:49.550 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:49.550 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:49.550 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 444 2024-09-17T21:18:49.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.550 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:49.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:49.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:18:49.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:49.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:18:49.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:49.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241698+0000", 2024-09-17T21:18:49.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2270000000000001e-06 2024-09-17T21:18:49.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:49.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241704+0000", 2024-09-17T21:18:49.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9800000000000003e-06 2024-09-17T21:18:49.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:49.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241725+0000", 2024-09-17T21:18:49.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0812999999999999e-05 2024-09-17T21:18:49.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:49.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241739+0000", 2024-09-17T21:18:49.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3773e-05 2024-09-17T21:18:49.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.553 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264850+0000", 2024-09-17T21:18:49.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023111326000000001 2024-09-17T21:18:49.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.553 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:49.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264897+0000", 2024-09-17T21:18:49.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6638999999999998e-05 2024-09-17T21:18:49.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.553 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:49.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264957+0000", 2024-09-17T21:18:49.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0625999999999999e-05 2024-09-17T21:18:49.554 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.554 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:49.554 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.554 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:445 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:49.554 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:18:49.554 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.228835172, 2024-09-17T21:18:49.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023315144999999999, 2024-09-17T21:18:49.554 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:49.554 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:49.554 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:49.554 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:49.555 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:49.555 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:49.555 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 445 2024-09-17T21:18:49.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.555 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:49.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:49.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:18:49.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:49.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:18:49.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:49.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241759+0000", 2024-09-17T21:18:49.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-06 2024-09-17T21:18:49.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:49.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241764+0000", 2024-09-17T21:18:49.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5899999999999998e-06 2024-09-17T21:18:49.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:49.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241784+0000", 2024-09-17T21:18:49.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9204000000000001e-05 2024-09-17T21:18:49.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.557 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:49.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241798+0000", 2024-09-17T21:18:49.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3953e-05 2024-09-17T21:18:49.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.557 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264977+0000", 2024-09-17T21:18:49.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023179182 2024-09-17T21:18:49.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:49.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.265018+0000", 2024-09-17T21:18:49.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1171999999999998e-05 2024-09-17T21:18:49.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:49.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.265073+0000", 2024-09-17T21:18:49.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4741000000000003e-05 2024-09-17T21:18:49.558 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.558 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:49.558 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.559 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:677 3.13 3:cd0d74f9:::rbd_data.11527824718.000000000000000a:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:49.559 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.250552+0000", 2024-09-17T21:18:49.559 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.220041182999999, 2024-09-17T21:18:49.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022610023999999999, 2024-09-17T21:18:49.559 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:49.559 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:49.559 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:49.559 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:49.559 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:49.559 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:49.559 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 677 2024-09-17T21:18:49.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.559 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:49.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:49.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250552+0000", 2024-09-17T21:18:49.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:49.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250552+0000", 2024-09-17T21:18:49.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:49.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250553+0000", 2024-09-17T21:18:49.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4699999999999999e-06 2024-09-17T21:18:49.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:49.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250571+0000", 2024-09-17T21:18:49.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8303000000000001e-05 2024-09-17T21:18:49.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:49.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250590+0000", 2024-09-17T21:18:49.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8723e-05 2024-09-17T21:18:49.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:49.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250603+0000", 2024-09-17T21:18:49.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3307e-05 2024-09-17T21:18:49.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.562 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.272971+0000", 2024-09-17T21:18:49.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022368140000000002 2024-09-17T21:18:49.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.562 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:49.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273051+0000", 2024-09-17T21:18:49.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9339999999999996e-05 2024-09-17T21:18:49.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:49.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273162+0000", 2024-09-17T21:18:49.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000110741 2024-09-17T21:18:49.563 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.563 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:49.563 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.563 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:678 3.13 3:cd0d74f9:::rbd_data.11527824718.000000000000000a:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:49.563 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.250622+0000", 2024-09-17T21:18:49.563 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.219970304, 2024-09-17T21:18:49.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023048448999999999, 2024-09-17T21:18:49.563 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:49.564 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:49.564 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:49.564 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:49.564 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:49.564 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:49.564 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 678 2024-09-17T21:18:49.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.564 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:49.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:49.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250622+0000", 2024-09-17T21:18:49.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:49.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250622+0000", 2024-09-17T21:18:49.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:49.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250624+0000", 2024-09-17T21:18:49.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4270000000000001e-06 2024-09-17T21:18:49.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:49.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250629+0000", 2024-09-17T21:18:49.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6030000000000004e-06 2024-09-17T21:18:49.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:49.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250648+0000", 2024-09-17T21:18:49.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8337000000000001e-05 2024-09-17T21:18:49.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:49.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250661+0000", 2024-09-17T21:18:49.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3446e-05 2024-09-17T21:18:49.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273275+0000", 2024-09-17T21:18:49.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022613292 2024-09-17T21:18:49.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:49.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273467+0000", 2024-09-17T21:18:49.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019237900000000001 2024-09-17T21:18:49.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:49.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273671+0000", 2024-09-17T21:18:49.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020396499999999999 2024-09-17T21:18:49.568 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.568 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:49.568 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.568 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:679 3.13 3:cd0d74f9:::rbd_data.11527824718.000000000000000a:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:49.568 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:18:49.568 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.219913751, 2024-09-17T21:18:49.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023284978000000001, 2024-09-17T21:18:49.568 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:49.568 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:49.568 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:49.568 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:49.569 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:49.569 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:49.569 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 679 2024-09-17T21:18:49.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.569 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:49.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:49.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:18:49.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:49.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:18:49.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:49.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250680+0000", 2024-09-17T21:18:49.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:18:49.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:49.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250685+0000", 2024-09-17T21:18:49.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1970000000000004e-06 2024-09-17T21:18:49.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:49.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250703+0000", 2024-09-17T21:18:49.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7396000000000001e-05 2024-09-17T21:18:49.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:49.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250716+0000", 2024-09-17T21:18:49.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.29e-05 2024-09-17T21:18:49.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273740+0000", 2024-09-17T21:18:49.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023024405000000001 2024-09-17T21:18:49.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:49.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273834+0000", 2024-09-17T21:18:49.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4124999999999998e-05 2024-09-17T21:18:49.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:49.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273964+0000", 2024-09-17T21:18:49.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012963499999999999 2024-09-17T21:18:49.572 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.572 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:49.572 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.572 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:680 3.13 3:cd0d74f9:::rbd_data.11527824718.000000000000000a:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:49.572 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:18:49.572 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.219858671000001, 2024-09-17T21:18:49.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023687792999999999, 2024-09-17T21:18:49.573 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:49.573 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:49.573 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:49.573 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:49.573 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:49.573 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:49.573 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 680 2024-09-17T21:18:49.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.573 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:49.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:49.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:18:49.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:49.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:18:49.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:49.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:49.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250735+0000", 2024-09-17T21:18:49.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2669999999999999e-06 2024-09-17T21:18:49.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:49.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250740+0000", 2024-09-17T21:18:49.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8060000000000002e-06 2024-09-17T21:18:49.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:49.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250757+0000", 2024-09-17T21:18:49.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7303e-05 2024-09-17T21:18:49.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:49.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250771+0000", 2024-09-17T21:18:49.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3876999999999999e-05 2024-09-17T21:18:49.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:49.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274022+0000", 2024-09-17T21:18:49.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023251135999999999 2024-09-17T21:18:49.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:49.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274211+0000", 2024-09-17T21:18:49.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018813599999999999 2024-09-17T21:18:49.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:49.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:49.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:49.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274422+0000", 2024-09-17T21:18:49.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000211268 2024-09-17T21:18:49.576 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.576 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:49.577 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.577 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:49.577 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:49.577 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:49.578 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:18:49.579 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:49.577+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:49.679 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:49.676+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:49.749 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:49.749 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:49.749 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:18:49.749 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:49.763 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:18:49.810 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:49.809+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:49.879 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:49.879 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:49.879 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:18:49.879 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:18:49.879 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:49.881 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:49.877+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:49.893 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:18:49.982 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:49.981+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:50.060 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:50.060 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:18:50.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:18:50.060 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:50.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.061 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:478 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 720896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.061 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:18:50.061 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.823983503999999, 2024-09-17T21:18:50.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016845300000000001, 2024-09-17T21:18:50.061 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.061 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.061 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.061 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.061 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.061 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.061 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 478 2024-09-17T21:18:50.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.062 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:18:50.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:18:50.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233785+0000", 2024-09-17T21:18:50.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.116e-06 2024-09-17T21:18:50.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233803+0000", 2024-09-17T21:18:50.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7932999999999999e-05 2024-09-17T21:18:50.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233822+0000", 2024-09-17T21:18:50.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9276999999999999e-05 2024-09-17T21:18:50.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233836+0000", 2024-09-17T21:18:50.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4053000000000001e-05 2024-09-17T21:18:50.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.248579+0000", 2024-09-17T21:18:50.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014743529 2024-09-17T21:18:50.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.248795+0000", 2024-09-17T21:18:50.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021535200000000001 2024-09-17T21:18:50.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250629+0000", 2024-09-17T21:18:50.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018340400000000001 2024-09-17T21:18:50.065 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.065 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.065 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.066 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:479 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 851968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.066 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:18:50.066 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.823892418, 2024-09-17T21:18:50.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017383900000000001, 2024-09-17T21:18:50.066 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.066 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.066 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.066 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.066 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.066 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.067 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 479 2024-09-17T21:18:50.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.067 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:18:50.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:18:50.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233876+0000", 2024-09-17T21:18:50.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2160000000000001e-06 2024-09-17T21:18:50.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233883+0000", 2024-09-17T21:18:50.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9199999999999998e-06 2024-09-17T21:18:50.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233900+0000", 2024-09-17T21:18:50.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7034e-05 2024-09-17T21:18:50.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233913+0000", 2024-09-17T21:18:50.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159000000000001e-05 2024-09-17T21:18:50.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250712+0000", 2024-09-17T21:18:50.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016798644000000001 2024-09-17T21:18:50.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250913+0000", 2024-09-17T21:18:50.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020127900000000001 2024-09-17T21:18:50.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251259+0000", 2024-09-17T21:18:50.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00034564799999999999 2024-09-17T21:18:50.071 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.071 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.071 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.072 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:480 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.072 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:18:50.072 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.823799573000001, 2024-09-17T21:18:50.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017902482000000001, 2024-09-17T21:18:50.072 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.072 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.072 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.072 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.072 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.072 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.072 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 480 2024-09-17T21:18:50.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.073 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:18:50.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:18:50.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233969+0000", 2024-09-17T21:18:50.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.454e-06 2024-09-17T21:18:50.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233981+0000", 2024-09-17T21:18:50.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1593000000000001e-05 2024-09-17T21:18:50.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233999+0000", 2024-09-17T21:18:50.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.806e-05 2024-09-17T21:18:50.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234012+0000", 2024-09-17T21:18:50.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:18:50.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251339+0000", 2024-09-17T21:18:50.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017326352 2024-09-17T21:18:50.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251534+0000", 2024-09-17T21:18:50.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000195842 2024-09-17T21:18:50.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251870+0000", 2024-09-17T21:18:50.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033551400000000003 2024-09-17T21:18:50.077 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.077 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.077 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.077 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:481 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1114112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.078 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:18:50.078 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.823731206, 2024-09-17T21:18:50.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018442621999999999, 2024-09-17T21:18:50.078 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.078 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.078 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.078 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.078 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.078 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.078 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 481 2024-09-17T21:18:50.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.079 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:18:50.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:18:50.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234037+0000", 2024-09-17T21:18:50.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:18:50.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234043+0000", 2024-09-17T21:18:50.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.49e-06 2024-09-17T21:18:50.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234061+0000", 2024-09-17T21:18:50.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8416000000000001e-05 2024-09-17T21:18:50.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234076+0000", 2024-09-17T21:18:50.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.456e-05 2024-09-17T21:18:50.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251948+0000", 2024-09-17T21:18:50.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017872774000000001 2024-09-17T21:18:50.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252144+0000", 2024-09-17T21:18:50.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019517300000000001 2024-09-17T21:18:50.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252478+0000", 2024-09-17T21:18:50.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033482899999999998 2024-09-17T21:18:50.082 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.082 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.082 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.083 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:482 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.083 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:18:50.083 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.823672653999999, 2024-09-17T21:18:50.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018988026000000002, 2024-09-17T21:18:50.083 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.083 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.083 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.083 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.083 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.083 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.083 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 482 2024-09-17T21:18:50.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.083 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:18:50.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:18:50.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234096+0000", 2024-09-17T21:18:50.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2139999999999999e-06 2024-09-17T21:18:50.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234101+0000", 2024-09-17T21:18:50.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7100000000000004e-06 2024-09-17T21:18:50.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234118+0000", 2024-09-17T21:18:50.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6486000000000001e-05 2024-09-17T21:18:50.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234131+0000", 2024-09-17T21:18:50.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2824e-05 2024-09-17T21:18:50.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252552+0000", 2024-09-17T21:18:50.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018421549999999998 2024-09-17T21:18:50.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252746+0000", 2024-09-17T21:18:50.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019424399999999999 2024-09-17T21:18:50.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253082+0000", 2024-09-17T21:18:50.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000335998 2024-09-17T21:18:50.087 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.087 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.087 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.087 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:483 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.087 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:18:50.087 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.823618034000001, 2024-09-17T21:18:50.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019462882000000001, 2024-09-17T21:18:50.088 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.088 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.088 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.088 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.088 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.088 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.088 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 483 2024-09-17T21:18:50.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.088 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:18:50.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:18:50.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234150+0000", 2024-09-17T21:18:50.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.127e-06 2024-09-17T21:18:50.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234155+0000", 2024-09-17T21:18:50.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:18:50.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234171+0000", 2024-09-17T21:18:50.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6160000000000001e-05 2024-09-17T21:18:50.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234185+0000", 2024-09-17T21:18:50.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3370000000000001e-05 2024-09-17T21:18:50.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253158+0000", 2024-09-17T21:18:50.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018972968999999999 2024-09-17T21:18:50.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253349+0000", 2024-09-17T21:18:50.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000190733 2024-09-17T21:18:50.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253612+0000", 2024-09-17T21:18:50.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026337000000000002 2024-09-17T21:18:50.092 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.092 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.092 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.092 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:484 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.092 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:18:50.093 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.823564008, 2024-09-17T21:18:50.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020029261999999999, 2024-09-17T21:18:50.093 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.093 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.093 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.093 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.093 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.093 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.093 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 484 2024-09-17T21:18:50.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.093 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:18:50.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:18:50.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234205+0000", 2024-09-17T21:18:50.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6470000000000001e-06 2024-09-17T21:18:50.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234210+0000", 2024-09-17T21:18:50.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:18:50.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234227+0000", 2024-09-17T21:18:50.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.73e-05 2024-09-17T21:18:50.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234241+0000", 2024-09-17T21:18:50.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3757000000000001e-05 2024-09-17T21:18:50.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253690+0000", 2024-09-17T21:18:50.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019449368000000002 2024-09-17T21:18:50.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253898+0000", 2024-09-17T21:18:50.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000207161 2024-09-17T21:18:50.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254232+0000", 2024-09-17T21:18:50.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033474299999999998 2024-09-17T21:18:50.097 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.097 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.097 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.097 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:485 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1638400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.097 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:18:50.097 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.823506781000001, 2024-09-17T21:18:50.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020553914, 2024-09-17T21:18:50.098 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.098 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.098 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.098 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.098 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.098 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.098 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 485 2024-09-17T21:18:50.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.098 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:18:50.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:18:50.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234261+0000", 2024-09-17T21:18:50.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.136e-06 2024-09-17T21:18:50.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234266+0000", 2024-09-17T21:18:50.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0669999999999999e-06 2024-09-17T21:18:50.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234284+0000", 2024-09-17T21:18:50.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7839999999999999e-05 2024-09-17T21:18:50.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234298+0000", 2024-09-17T21:18:50.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3852999999999999e-05 2024-09-17T21:18:50.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254311+0000", 2024-09-17T21:18:50.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020012973 2024-09-17T21:18:50.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254552+0000", 2024-09-17T21:18:50.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024116400000000001 2024-09-17T21:18:50.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254814+0000", 2024-09-17T21:18:50.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000261881 2024-09-17T21:18:50.102 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.102 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.102 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.102 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:486 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.102 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:18:50.102 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.823450392, 2024-09-17T21:18:50.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021010240999999999, 2024-09-17T21:18:50.103 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.103 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.103 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.103 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.103 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.103 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.103 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 486 2024-09-17T21:18:50.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.103 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:18:50.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:18:50.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234318+0000", 2024-09-17T21:18:50.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.37e-06 2024-09-17T21:18:50.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234324+0000", 2024-09-17T21:18:50.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:18:50.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234341+0000", 2024-09-17T21:18:50.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7127000000000001e-05 2024-09-17T21:18:50.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234354+0000", 2024-09-17T21:18:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3210000000000001e-05 2024-09-17T21:18:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254896+0000", 2024-09-17T21:18:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020541835000000001 2024-09-17T21:18:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255092+0000", 2024-09-17T21:18:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019634700000000001 2024-09-17T21:18:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255327+0000", 2024-09-17T21:18:50.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023475899999999999 2024-09-17T21:18:50.107 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.107 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.107 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.107 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:487 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.107 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:18:50.108 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.823394196000001, 2024-09-17T21:18:50.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021453911999999999, 2024-09-17T21:18:50.108 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.108 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.108 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.108 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.108 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.108 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.108 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 487 2024-09-17T21:18:50.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.108 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:18:50.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:18:50.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234374+0000", 2024-09-17T21:18:50.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3039999999999999e-06 2024-09-17T21:18:50.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234380+0000", 2024-09-17T21:18:50.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:18:50.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234397+0000", 2024-09-17T21:18:50.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7232999999999999e-05 2024-09-17T21:18:50.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234411+0000", 2024-09-17T21:18:50.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3784000000000001e-05 2024-09-17T21:18:50.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255406+0000", 2024-09-17T21:18:50.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020995204 2024-09-17T21:18:50.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255595+0000", 2024-09-17T21:18:50.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018903299999999999 2024-09-17T21:18:50.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255827+0000", 2024-09-17T21:18:50.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023190099999999999 2024-09-17T21:18:50.112 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.112 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.112 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.112 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:488 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.113 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:18:50.113 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.823338519, 2024-09-17T21:18:50.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026887966999999999, 2024-09-17T21:18:50.113 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.113 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.113 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.113 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.113 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.113 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.113 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 488 2024-09-17T21:18:50.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.113 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:18:50.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:18:50.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234430+0000", 2024-09-17T21:18:50.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1829999999999999e-06 2024-09-17T21:18:50.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234435+0000", 2024-09-17T21:18:50.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9869999999999999e-06 2024-09-17T21:18:50.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234451+0000", 2024-09-17T21:18:50.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6458999999999999e-05 2024-09-17T21:18:50.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234465+0000", 2024-09-17T21:18:50.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3730000000000001e-05 2024-09-17T21:18:50.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255908+0000", 2024-09-17T21:18:50.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021443413000000001 2024-09-17T21:18:50.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256098+0000", 2024-09-17T21:18:50.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018961 2024-09-17T21:18:50.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261317+0000", 2024-09-17T21:18:50.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052185850000000004 2024-09-17T21:18:50.117 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.117 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.117 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.117 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:489 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2162688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.117 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:18:50.118 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.82328268, 2024-09-17T21:18:50.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022632815000000001, 2024-09-17T21:18:50.118 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.118 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.118 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.118 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.118 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.118 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.118 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 489 2024-09-17T21:18:50.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.118 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:18:50.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:18:50.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234486+0000", 2024-09-17T21:18:50.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5099999999999999e-06 2024-09-17T21:18:50.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234491+0000", 2024-09-17T21:18:50.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.147e-06 2024-09-17T21:18:50.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234508+0000", 2024-09-17T21:18:50.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7405999999999999e-05 2024-09-17T21:18:50.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234522+0000", 2024-09-17T21:18:50.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3677e-05 2024-09-17T21:18:50.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256536+0000", 2024-09-17T21:18:50.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022013464999999999 2024-09-17T21:18:50.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256752+0000", 2024-09-17T21:18:50.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021637800000000001 2024-09-17T21:18:50.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257117+0000", 2024-09-17T21:18:50.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000365232 2024-09-17T21:18:50.122 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.122 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.122 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.123 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:490 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2293760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.123 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:18:50.123 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.82322789, 2024-09-17T21:18:50.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023146462999999999, 2024-09-17T21:18:50.123 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.123 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.123 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.123 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.123 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.123 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.123 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 490 2024-09-17T21:18:50.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.124 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:18:50.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:18:50.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234541+0000", 2024-09-17T21:18:50.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4869999999999999e-06 2024-09-17T21:18:50.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234559+0000", 2024-09-17T21:18:50.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8105999999999999e-05 2024-09-17T21:18:50.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234577+0000", 2024-09-17T21:18:50.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.836e-05 2024-09-17T21:18:50.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234592+0000", 2024-09-17T21:18:50.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4766000000000001e-05 2024-09-17T21:18:50.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257187+0000", 2024-09-17T21:18:50.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022594725 2024-09-17T21:18:50.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257374+0000", 2024-09-17T21:18:50.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018767299999999999 2024-09-17T21:18:50.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257686+0000", 2024-09-17T21:18:50.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000311346 2024-09-17T21:18:50.127 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.127 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.127 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.128 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:491 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.128 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:18:50.128 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.823155281, 2024-09-17T21:18:50.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023696872000000001, 2024-09-17T21:18:50.128 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.128 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.128 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.128 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.128 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.128 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.128 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 491 2024-09-17T21:18:50.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.129 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:18:50.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:18:50.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234613+0000", 2024-09-17T21:18:50.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-06 2024-09-17T21:18:50.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234618+0000", 2024-09-17T21:18:50.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3839999999999996e-06 2024-09-17T21:18:50.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234635+0000", 2024-09-17T21:18:50.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.685e-05 2024-09-17T21:18:50.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234648+0000", 2024-09-17T21:18:50.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.313e-05 2024-09-17T21:18:50.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257780+0000", 2024-09-17T21:18:50.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023131946 2024-09-17T21:18:50.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257985+0000", 2024-09-17T21:18:50.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020468400000000001 2024-09-17T21:18:50.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258309+0000", 2024-09-17T21:18:50.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032357499999999999 2024-09-17T21:18:50.132 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.132 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.132 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.132 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:492 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.132 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:18:50.132 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.823099694, 2024-09-17T21:18:50.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024258994999999998, 2024-09-17T21:18:50.132 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.132 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.133 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.133 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.133 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.133 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.133 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 492 2024-09-17T21:18:50.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.133 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:18:50.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:18:50.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234669+0000", 2024-09-17T21:18:50.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.313e-06 2024-09-17T21:18:50.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234674+0000", 2024-09-17T21:18:50.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1599999999999997e-06 2024-09-17T21:18:50.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234690+0000", 2024-09-17T21:18:50.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.666e-05 2024-09-17T21:18:50.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234704+0000", 2024-09-17T21:18:50.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3043e-05 2024-09-17T21:18:50.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258397+0000", 2024-09-17T21:18:50.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023693204999999998 2024-09-17T21:18:50.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258593+0000", 2024-09-17T21:18:50.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019578999999999999 2024-09-17T21:18:50.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258926+0000", 2024-09-17T21:18:50.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033382399999999998 2024-09-17T21:18:50.136 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.136 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.137 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.137 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:493 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.137 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:18:50.137 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.823043229, 2024-09-17T21:18:50.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024793461999999999, 2024-09-17T21:18:50.137 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.137 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.137 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.137 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.137 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.137 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.138 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 493 2024-09-17T21:18:50.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.138 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:18:50.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:18:50.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234725+0000", 2024-09-17T21:18:50.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.24e-06 2024-09-17T21:18:50.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234731+0000", 2024-09-17T21:18:50.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8939999999999998e-06 2024-09-17T21:18:50.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234749+0000", 2024-09-17T21:18:50.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7863000000000002e-05 2024-09-17T21:18:50.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234762+0000", 2024-09-17T21:18:50.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3322999999999999e-05 2024-09-17T21:18:50.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259022+0000", 2024-09-17T21:18:50.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024259691999999999 2024-09-17T21:18:50.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259259+0000", 2024-09-17T21:18:50.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023669199999999999 2024-09-17T21:18:50.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259517+0000", 2024-09-17T21:18:50.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025875799999999999 2024-09-17T21:18:50.141 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.141 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.141 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.141 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:494 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.141 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:18:50.141 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.822985529, 2024-09-17T21:18:50.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025251952000000001, 2024-09-17T21:18:50.141 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.142 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.142 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.142 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.142 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.142 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.142 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 494 2024-09-17T21:18:50.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.142 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:18:50.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:18:50.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234783+0000", 2024-09-17T21:18:50.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2440000000000001e-06 2024-09-17T21:18:50.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234789+0000", 2024-09-17T21:18:50.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:18:50.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234806+0000", 2024-09-17T21:18:50.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7450000000000001e-05 2024-09-17T21:18:50.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234819+0000", 2024-09-17T21:18:50.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3326000000000001e-05 2024-09-17T21:18:50.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259585+0000", 2024-09-17T21:18:50.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024765579999999999 2024-09-17T21:18:50.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259787+0000", 2024-09-17T21:18:50.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202444 2024-09-17T21:18:50.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260033+0000", 2024-09-17T21:18:50.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024611499999999998 2024-09-17T21:18:50.145 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.145 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.145 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.145 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:495 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.146 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:18:50.146 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.822930166000001, 2024-09-17T21:18:50.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025776342000000001, 2024-09-17T21:18:50.146 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.146 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.146 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.146 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.147 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.147 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.147 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 495 2024-09-17T21:18:50.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.147 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:18:50.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:18:50.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234838+0000", 2024-09-17T21:18:50.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2130000000000001e-06 2024-09-17T21:18:50.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234843+0000", 2024-09-17T21:18:50.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9899999999999997e-06 2024-09-17T21:18:50.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234860+0000", 2024-09-17T21:18:50.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6973000000000002e-05 2024-09-17T21:18:50.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234873+0000", 2024-09-17T21:18:50.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3179999999999999e-05 2024-09-17T21:18:50.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260135+0000", 2024-09-17T21:18:50.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025262118 2024-09-17T21:18:50.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260364+0000", 2024-09-17T21:18:50.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022830699999999999 2024-09-17T21:18:50.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260613+0000", 2024-09-17T21:18:50.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024956099999999998 2024-09-17T21:18:50.151 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.151 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.151 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.151 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:496 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.151 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:18:50.151 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.822876886, 2024-09-17T21:18:50.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026376773999999999, 2024-09-17T21:18:50.151 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.151 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.151 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.152 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.152 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.152 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.152 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 496 2024-09-17T21:18:50.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.152 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:18:50.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:18:50.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234891+0000", 2024-09-17T21:18:50.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1230000000000001e-06 2024-09-17T21:18:50.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234896+0000", 2024-09-17T21:18:50.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9599999999999999e-06 2024-09-17T21:18:50.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234913+0000", 2024-09-17T21:18:50.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6747000000000001e-05 2024-09-17T21:18:50.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234926+0000", 2024-09-17T21:18:50.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-05 2024-09-17T21:18:50.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260675+0000", 2024-09-17T21:18:50.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025748979000000002 2024-09-17T21:18:50.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260883+0000", 2024-09-17T21:18:50.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020848399999999999 2024-09-17T21:18:50.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261267+0000", 2024-09-17T21:18:50.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038345099999999998 2024-09-17T21:18:50.155 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.155 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.155 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.155 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:682 3.9 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2455552~12288,write 8126464~10240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.156 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:18:50.156 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.788939835000001, 2024-09-17T21:18:50.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018447261999999999, 2024-09-17T21:18:50.156 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.156 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.156 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:50.156 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.156 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.156 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.156 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 682 2024-09-17T21:18:50.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.156 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:18:50.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:18:50.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268830+0000", 2024-09-17T21:18:50.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6419999999999999e-06 2024-09-17T21:18:50.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271673+0000", 2024-09-17T21:18:50.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028432610000000001 2024-09-17T21:18:50.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271700+0000", 2024-09-17T21:18:50.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7200999999999999e-05 2024-09-17T21:18:50.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271715+0000", 2024-09-17T21:18:50.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5145e-05 2024-09-17T21:18:50.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271785+0000", 2024-09-17T21:18:50.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9519999999999998e-05 2024-09-17T21:18:50.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271918+0000", 2024-09-17T21:18:50.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013330899999999999 2024-09-17T21:18:50.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:50.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.272449+0000", 2024-09-17T21:18:50.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00053051200000000002 2024-09-17T21:18:50.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:50.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.278822+0000", 2024-09-17T21:18:50.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0063731619999999999 2024-09-17T21:18:50.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:50.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287167+0000", 2024-09-17T21:18:50.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083452149999999996 2024-09-17T21:18:50.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:50.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287213+0000", 2024-09-17T21:18:50.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5972e-05 2024-09-17T21:18:50.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287274+0000", 2024-09-17T21:18:50.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1322999999999995e-05 2024-09-17T21:18:50.163 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.163 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.163 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.163 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.163 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.163 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:50.165 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:18:50.165 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:50.161+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:50.266 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:50.264+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:50.337 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:50.337 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:50.337 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:18:50.337 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:50.353 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:18:50.366 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:50.364+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:50.467 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:50.465+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:50.472 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:50.472 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:50.472 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:18:50.472 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:18:50.472 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:50.487 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:18:50.569 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:50.568+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:50.647 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:50.647 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:18:50.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:18:50.647 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:50.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.647 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:18:50.647 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:18:50.647 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 29.039377100999999, 2024-09-17T21:18:50.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:18:50.648 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.648 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.648 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:50.648 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:18:50.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:18:50.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:18:50.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:18:50.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:18:50.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:18:50.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:18:50.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:18:50.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:18:50.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:18:50.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:18:50.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:18:50.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:18:50.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:18:50.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:50.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:18:50.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:18:50.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:18:50.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:18:50.652 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.652 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.652 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.652 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:18:50.653 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:18:50.653 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 29.029430465000001, 2024-09-17T21:18:50.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:18:50.653 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.653 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.653 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:50.653 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:18:50.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:18:50.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:18:50.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:18:50.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:18:50.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:18:50.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:18:50.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:18:50.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:18:50.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:18:50.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:18:50.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:18:50.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:18:50.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:18:50.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:50.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:18:50.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:18:50.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:18:50.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:18:50.657 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.657 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.657 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.657 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:18:50.657 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:18:50.658 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.786279624999999, 2024-09-17T21:18:50.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:18:50.658 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.658 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.658 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:50.658 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.658 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:18:50.658 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:50.658 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:18:50.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.658 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:18:50.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:18:50.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:18:50.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:18:50.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:18:50.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:18:50.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:18:50.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:18:50.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:18:50.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:18:50.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:18:50.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:18:50.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:18:50.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:18:50.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:50.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:18:50.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:18:50.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:50.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:18:50.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:18:50.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:50.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:18:50.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:18:50.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:50.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:18:50.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:18:50.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:18:50.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:18:50.663 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.663 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.663 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.663 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:18:50.663 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:18:50.664 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.776634138999999, 2024-09-17T21:18:50.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:18:50.664 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.664 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.664 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:50.664 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.664 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:18:50.664 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:50.665 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:18:50.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.665 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:18:50.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:18:50.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:18:50.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:18:50.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:18:50.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:18:50.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:18:50.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:18:50.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:18:50.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:18:50.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:18:50.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:18:50.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:18:50.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:18:50.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:50.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:18:50.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:18:50.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:50.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:18:50.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:18:50.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:50.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:18:50.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:18:50.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:50.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:18:50.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:18:50.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:18:50.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:18:50.670 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.670 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.670 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.670 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:495 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 2228224~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:50.670 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.197359+0000", 2024-09-17T21:18:50.670 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.446208305999999, 2024-09-17T21:18:50.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021840167000000001, 2024-09-17T21:18:50.670 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.670 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.670 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.670 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.671 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:50.671 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:50.671 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 495 2024-09-17T21:18:50.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.671 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197359+0000", 2024-09-17T21:18:50.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197359+0000", 2024-09-17T21:18:50.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197360+0000", 2024-09-17T21:18:50.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0380000000000001e-06 2024-09-17T21:18:50.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197376+0000", 2024-09-17T21:18:50.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6147999999999999e-05 2024-09-17T21:18:50.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197392+0000", 2024-09-17T21:18:50.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6025999999999999e-05 2024-09-17T21:18:50.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197401+0000", 2024-09-17T21:18:50.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6640000000000004e-06 2024-09-17T21:18:50.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.218539+0000", 2024-09-17T21:18:50.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021137808000000001 2024-09-17T21:18:50.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.218611+0000", 2024-09-17T21:18:50.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2471000000000005e-05 2024-09-17T21:18:50.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.219199+0000", 2024-09-17T21:18:50.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00058801199999999995 2024-09-17T21:18:50.674 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.674 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.674 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.674 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:548 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:50.675 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.200281+0000", 2024-09-17T21:18:50.675 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.443286648000001, 2024-09-17T21:18:50.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021975320999999999, 2024-09-17T21:18:50.675 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.675 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.675 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.675 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.675 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:50.675 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:50.675 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 548 2024-09-17T21:18:50.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.675 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200281+0000", 2024-09-17T21:18:50.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200281+0000", 2024-09-17T21:18:50.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200281+0000", 2024-09-17T21:18:50.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5100000000000001e-07 2024-09-17T21:18:50.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200283+0000", 2024-09-17T21:18:50.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4699999999999999e-06 2024-09-17T21:18:50.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200287+0000", 2024-09-17T21:18:50.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3939999999999998e-06 2024-09-17T21:18:50.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200290+0000", 2024-09-17T21:18:50.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3160000000000001e-06 2024-09-17T21:18:50.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.221522+0000", 2024-09-17T21:18:50.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021231782000000001 2024-09-17T21:18:50.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.221624+0000", 2024-09-17T21:18:50.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101693 2024-09-17T21:18:50.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.222256+0000", 2024-09-17T21:18:50.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00063231500000000005 2024-09-17T21:18:50.678 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.678 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.678 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.679 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:467 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.679 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235571+0000", 2024-09-17T21:18:50.679 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.407996249, 2024-09-17T21:18:50.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021817462999999999, 2024-09-17T21:18:50.679 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.679 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.679 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.679 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.679 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.679 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.679 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 467 2024-09-17T21:18:50.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.679 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235571+0000", 2024-09-17T21:18:50.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235571+0000", 2024-09-17T21:18:50.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235573+0000", 2024-09-17T21:18:50.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5659999999999999e-06 2024-09-17T21:18:50.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235579+0000", 2024-09-17T21:18:50.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7729999999999996e-06 2024-09-17T21:18:50.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235598+0000", 2024-09-17T21:18:50.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9106999999999998e-05 2024-09-17T21:18:50.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235612+0000", 2024-09-17T21:18:50.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4032999999999999e-05 2024-09-17T21:18:50.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257023+0000", 2024-09-17T21:18:50.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021411619 2024-09-17T21:18:50.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257194+0000", 2024-09-17T21:18:50.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017076999999999999 2024-09-17T21:18:50.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257389+0000", 2024-09-17T21:18:50.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019459499999999999 2024-09-17T21:18:50.683 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.683 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.683 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.683 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:468 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.683 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235632+0000", 2024-09-17T21:18:50.683 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.407935719999999, 2024-09-17T21:18:50.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022180165000000002, 2024-09-17T21:18:50.683 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.683 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.683 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.684 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.684 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 468 2024-09-17T21:18:50.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.684 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235632+0000", 2024-09-17T21:18:50.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235632+0000", 2024-09-17T21:18:50.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235633+0000", 2024-09-17T21:18:50.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.446e-06 2024-09-17T21:18:50.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235639+0000", 2024-09-17T21:18:50.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6269999999999997e-06 2024-09-17T21:18:50.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235658+0000", 2024-09-17T21:18:50.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8686999999999999e-05 2024-09-17T21:18:50.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235672+0000", 2024-09-17T21:18:50.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.449e-05 2024-09-17T21:18:50.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257452+0000", 2024-09-17T21:18:50.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021779827000000002 2024-09-17T21:18:50.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257622+0000", 2024-09-17T21:18:50.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016976099999999999 2024-09-17T21:18:50.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257812+0000", 2024-09-17T21:18:50.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019032700000000001 2024-09-17T21:18:50.687 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.687 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.687 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.687 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:469 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.687 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235692+0000", 2024-09-17T21:18:50.687 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.407875389999999, 2024-09-17T21:18:50.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022615679, 2024-09-17T21:18:50.688 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.688 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.688 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.688 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.688 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.688 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.688 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 469 2024-09-17T21:18:50.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.688 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235692+0000", 2024-09-17T21:18:50.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235692+0000", 2024-09-17T21:18:50.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235694+0000", 2024-09-17T21:18:50.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.643e-06 2024-09-17T21:18:50.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235700+0000", 2024-09-17T21:18:50.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.733e-06 2024-09-17T21:18:50.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235718+0000", 2024-09-17T21:18:50.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8029999999999998e-05 2024-09-17T21:18:50.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235732+0000", 2024-09-17T21:18:50.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4239999999999999e-05 2024-09-17T21:18:50.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257935+0000", 2024-09-17T21:18:50.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022203514000000001 2024-09-17T21:18:50.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258107+0000", 2024-09-17T21:18:50.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017220700000000001 2024-09-17T21:18:50.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258308+0000", 2024-09-17T21:18:50.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020031199999999999 2024-09-17T21:18:50.691 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.691 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.692 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.692 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:470 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.692 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235752+0000", 2024-09-17T21:18:50.692 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.407815698, 2024-09-17T21:18:50.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023324234999999999, 2024-09-17T21:18:50.692 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.692 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.692 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.692 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 470 2024-09-17T21:18:50.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.693 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235752+0000", 2024-09-17T21:18:50.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235752+0000", 2024-09-17T21:18:50.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235753+0000", 2024-09-17T21:18:50.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.46e-06 2024-09-17T21:18:50.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235759+0000", 2024-09-17T21:18:50.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5829999999999997e-06 2024-09-17T21:18:50.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235777+0000", 2024-09-17T21:18:50.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8196999999999999e-05 2024-09-17T21:18:50.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235791+0000", 2024-09-17T21:18:50.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4406999999999999e-05 2024-09-17T21:18:50.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258351+0000", 2024-09-17T21:18:50.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022559157999999999 2024-09-17T21:18:50.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258491+0000", 2024-09-17T21:18:50.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014083300000000001 2024-09-17T21:18:50.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259076+0000", 2024-09-17T21:18:50.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00058459699999999996 2024-09-17T21:18:50.696 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.696 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.696 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.696 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:471 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.696 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235811+0000", 2024-09-17T21:18:50.696 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.407756478, 2024-09-17T21:18:50.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023519748, 2024-09-17T21:18:50.696 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.697 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.697 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.697 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.697 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.697 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.697 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 471 2024-09-17T21:18:50.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.697 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235811+0000", 2024-09-17T21:18:50.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235811+0000", 2024-09-17T21:18:50.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235813+0000", 2024-09-17T21:18:50.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.57e-06 2024-09-17T21:18:50.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235818+0000", 2024-09-17T21:18:50.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.417e-06 2024-09-17T21:18:50.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235836+0000", 2024-09-17T21:18:50.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8023e-05 2024-09-17T21:18:50.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235850+0000", 2024-09-17T21:18:50.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3759e-05 2024-09-17T21:18:50.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259120+0000", 2024-09-17T21:18:50.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023270193000000002 2024-09-17T21:18:50.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259225+0000", 2024-09-17T21:18:50.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104614 2024-09-17T21:18:50.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259331+0000", 2024-09-17T21:18:50.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106172 2024-09-17T21:18:50.700 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.700 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.700 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.701 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.701 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:18:50.701 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.407698106, 2024-09-17T21:18:50.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:18:50.701 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.701 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.701 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.701 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.701 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.701 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.701 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:18:50.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.701 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:18:50.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:18:50.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:18:50.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:18:50.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:18:50.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:18:50.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:18:50.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:18:50.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:18:50.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:18:50.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:18:50.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:18:50.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:18:50.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:18:50.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:18:50.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:18:50.705 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.705 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.705 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.705 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:558 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.705 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239154+0000", 2024-09-17T21:18:50.705 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.404413224000001, 2024-09-17T21:18:50.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021884532000000002, 2024-09-17T21:18:50.705 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.705 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.705 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.705 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.706 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.706 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.706 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 558 2024-09-17T21:18:50.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.706 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239154+0000", 2024-09-17T21:18:50.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239154+0000", 2024-09-17T21:18:50.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239156+0000", 2024-09-17T21:18:50.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4670000000000001e-06 2024-09-17T21:18:50.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239162+0000", 2024-09-17T21:18:50.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2829999999999998e-06 2024-09-17T21:18:50.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239181+0000", 2024-09-17T21:18:50.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8960000000000001e-05 2024-09-17T21:18:50.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239196+0000", 2024-09-17T21:18:50.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4523e-05 2024-09-17T21:18:50.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258757+0000", 2024-09-17T21:18:50.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019561775 2024-09-17T21:18:50.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258880+0000", 2024-09-17T21:18:50.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000123013 2024-09-17T21:18:50.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261039+0000", 2024-09-17T21:18:50.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021585110000000001 2024-09-17T21:18:50.709 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.709 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.709 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.709 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:559 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2162688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.710 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239215+0000", 2024-09-17T21:18:50.710 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.404352051, 2024-09-17T21:18:50.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022314902000000001, 2024-09-17T21:18:50.710 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.710 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.710 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.710 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.710 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.710 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.710 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 559 2024-09-17T21:18:50.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.710 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239215+0000", 2024-09-17T21:18:50.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239215+0000", 2024-09-17T21:18:50.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239217+0000", 2024-09-17T21:18:50.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:18:50.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239223+0000", 2024-09-17T21:18:50.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6099999999999997e-06 2024-09-17T21:18:50.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239241+0000", 2024-09-17T21:18:50.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8576e-05 2024-09-17T21:18:50.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239256+0000", 2024-09-17T21:18:50.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4397e-05 2024-09-17T21:18:50.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261086+0000", 2024-09-17T21:18:50.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021830374 2024-09-17T21:18:50.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261203+0000", 2024-09-17T21:18:50.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000117192 2024-09-17T21:18:50.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261530+0000", 2024-09-17T21:18:50.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032712300000000002 2024-09-17T21:18:50.714 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.714 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.714 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.714 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:560 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2293760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.714 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239276+0000", 2024-09-17T21:18:50.714 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.404291691999999, 2024-09-17T21:18:50.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022636927000000001, 2024-09-17T21:18:50.714 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.715 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.715 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.715 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.715 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.715 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.715 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 560 2024-09-17T21:18:50.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.715 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239276+0000", 2024-09-17T21:18:50.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239276+0000", 2024-09-17T21:18:50.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239277+0000", 2024-09-17T21:18:50.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4699999999999999e-06 2024-09-17T21:18:50.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239283+0000", 2024-09-17T21:18:50.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5899999999999998e-06 2024-09-17T21:18:50.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239301+0000", 2024-09-17T21:18:50.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8593999999999998e-05 2024-09-17T21:18:50.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239316+0000", 2024-09-17T21:18:50.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.428e-05 2024-09-17T21:18:50.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261592+0000", 2024-09-17T21:18:50.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022276118000000001 2024-09-17T21:18:50.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261733+0000", 2024-09-17T21:18:50.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014099199999999999 2024-09-17T21:18:50.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261913+0000", 2024-09-17T21:18:50.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017988300000000001 2024-09-17T21:18:50.719 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.719 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.719 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.719 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:561 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.719 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239336+0000", 2024-09-17T21:18:50.719 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.404231865, 2024-09-17T21:18:50.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02290615, 2024-09-17T21:18:50.720 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.720 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.720 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.720 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.720 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.720 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.720 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:18:50.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.720 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239336+0000", 2024-09-17T21:18:50.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239336+0000", 2024-09-17T21:18:50.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239337+0000", 2024-09-17T21:18:50.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3969999999999999e-06 2024-09-17T21:18:50.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239343+0000", 2024-09-17T21:18:50.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.699e-06 2024-09-17T21:18:50.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239361+0000", 2024-09-17T21:18:50.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8107000000000001e-05 2024-09-17T21:18:50.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239379+0000", 2024-09-17T21:18:50.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7972999999999999e-05 2024-09-17T21:18:50.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261956+0000", 2024-09-17T21:18:50.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022577105 2024-09-17T21:18:50.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262082+0000", 2024-09-17T21:18:50.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012621600000000001 2024-09-17T21:18:50.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262242+0000", 2024-09-17T21:18:50.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000159653 2024-09-17T21:18:50.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.723 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.724 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:562 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.724 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239398+0000", 2024-09-17T21:18:50.724 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.404169389, 2024-09-17T21:18:50.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023207746000000001, 2024-09-17T21:18:50.724 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.724 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.724 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.724 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.724 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.724 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.724 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:18:50.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.725 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239398+0000", 2024-09-17T21:18:50.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239398+0000", 2024-09-17T21:18:50.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239400+0000", 2024-09-17T21:18:50.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.55e-06 2024-09-17T21:18:50.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239427+0000", 2024-09-17T21:18:50.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.745e-05 2024-09-17T21:18:50.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239447+0000", 2024-09-17T21:18:50.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9715999999999999e-05 2024-09-17T21:18:50.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239468+0000", 2024-09-17T21:18:50.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0894e-05 2024-09-17T21:18:50.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262285+0000", 2024-09-17T21:18:50.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022817456999999999 2024-09-17T21:18:50.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262436+0000", 2024-09-17T21:18:50.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015082999999999999 2024-09-17T21:18:50.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262606+0000", 2024-09-17T21:18:50.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016984899999999999 2024-09-17T21:18:50.728 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.728 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.728 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.728 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:563 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.728 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239497+0000", 2024-09-17T21:18:50.728 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.40407098, 2024-09-17T21:18:50.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023494231000000001, 2024-09-17T21:18:50.728 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.728 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.728 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.729 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 563 2024-09-17T21:18:50.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.729 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239497+0000", 2024-09-17T21:18:50.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239497+0000", 2024-09-17T21:18:50.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239498+0000", 2024-09-17T21:18:50.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3969999999999999e-06 2024-09-17T21:18:50.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239539+0000", 2024-09-17T21:18:50.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0669e-05 2024-09-17T21:18:50.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239559+0000", 2024-09-17T21:18:50.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0353999999999999e-05 2024-09-17T21:18:50.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239575+0000", 2024-09-17T21:18:50.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5642999999999999e-05 2024-09-17T21:18:50.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262656+0000", 2024-09-17T21:18:50.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023081005000000002 2024-09-17T21:18:50.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262769+0000", 2024-09-17T21:18:50.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113616 2024-09-17T21:18:50.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262991+0000", 2024-09-17T21:18:50.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000221547 2024-09-17T21:18:50.732 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.732 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.732 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.732 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:564 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.732 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239598+0000", 2024-09-17T21:18:50.732 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.40396966, 2024-09-17T21:18:50.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023713695999999999, 2024-09-17T21:18:50.733 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.733 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.733 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.733 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.733 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.733 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.733 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 564 2024-09-17T21:18:50.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.733 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239598+0000", 2024-09-17T21:18:50.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239598+0000", 2024-09-17T21:18:50.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239600+0000", 2024-09-17T21:18:50.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:18:50.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239606+0000", 2024-09-17T21:18:50.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2029999999999997e-06 2024-09-17T21:18:50.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239623+0000", 2024-09-17T21:18:50.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7669999999999999e-05 2024-09-17T21:18:50.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239638+0000", 2024-09-17T21:18:50.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4416999999999999e-05 2024-09-17T21:18:50.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263063+0000", 2024-09-17T21:18:50.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023424932999999998 2024-09-17T21:18:50.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263182+0000", 2024-09-17T21:18:50.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011914899999999999 2024-09-17T21:18:50.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263312+0000", 2024-09-17T21:18:50.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012956799999999999 2024-09-17T21:18:50.736 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.736 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.737 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.737 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:565 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:50.737 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:18:50.737 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.403909141, 2024-09-17T21:18:50.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024193392000000001, 2024-09-17T21:18:50.737 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:50.737 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:50.737 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:50.737 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:50.737 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:50.737 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:50.737 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 565 2024-09-17T21:18:50.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.738 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:50.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:50.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:18:50.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:50.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:18:50.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:50.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:50.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239660+0000", 2024-09-17T21:18:50.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:18:50.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:50.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239666+0000", 2024-09-17T21:18:50.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8699999999999997e-06 2024-09-17T21:18:50.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:50.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239688+0000", 2024-09-17T21:18:50.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.213e-05 2024-09-17T21:18:50.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:50.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239703+0000", 2024-09-17T21:18:50.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4813e-05 2024-09-17T21:18:50.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:50.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263427+0000", 2024-09-17T21:18:50.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023724630999999999 2024-09-17T21:18:50.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:50.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263614+0000", 2024-09-17T21:18:50.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018664500000000001 2024-09-17T21:18:50.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:50.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:50.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:50.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263852+0000", 2024-09-17T21:18:50.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023793999999999999 2024-09-17T21:18:50.741 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.741 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.741 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.742 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:50.742 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:50.742 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:50.742 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:18:50.743 INFO:teuthology.orchestra.run.smithi184.stdout:1024 trunc from 0xc0ab200 to 0x57c7600 2024-09-17T21:18:50.744 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:50.741+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:50.745 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:18:50.792 INFO:teuthology.orchestra.run.smithi184.stdout:1025 write 0xd542200 thru 0xd54f5d8 (0xd3d9 bytes) 2024-09-17T21:18:50.794 INFO:teuthology.orchestra.run.smithi184.stdout:1026 read 0xa250e00 thru 0xa254b51 (0x3d52 bytes) 2024-09-17T21:18:50.795 INFO:teuthology.orchestra.run.smithi184.stdout:1027 write 0x686e400 thru 0x687cfc7 (0xebc8 bytes) 2024-09-17T21:18:50.796 INFO:teuthology.orchestra.run.smithi184.stdout:1028 read 0xbf6b800 thru 0xbf70aaf (0x52b0 bytes) 2024-09-17T21:18:50.797 INFO:teuthology.orchestra.run.smithi184.stdout:1029 read 0x18f2e00 thru 0x18fc118 (0x9319 bytes) 2024-09-17T21:18:50.809 INFO:teuthology.orchestra.run.smithi184.stdout:1030 trunc from 0xd54f600 to 0xed3b800 2024-09-17T21:18:50.830 INFO:teuthology.orchestra.run.smithi184.stdout:1032 write 0xcd1ac00 thru 0xcd1c855 (0x1c56 bytes) 2024-09-17T21:18:50.834 INFO:teuthology.orchestra.run.smithi184.stdout:1034 punch from 0x8488200 to 0x848f200, (0x7000 bytes) 2024-09-17T21:18:50.834 INFO:teuthology.orchestra.run.smithi184.stdout:1035 read 0x1b57600 thru 0x1b57ff5 (0x9f6 bytes) 2024-09-17T21:18:50.835 INFO:teuthology.orchestra.run.smithi184.stdout:1036 read 0xde1200 thru 0xde4fee (0x3def bytes) 2024-09-17T21:18:50.839 INFO:teuthology.orchestra.run.smithi184.stdout:1038 read 0xbb38000 thru 0xbb46705 (0xe706 bytes) 2024-09-17T21:18:50.840 INFO:teuthology.orchestra.run.smithi184.stdout:1040 write 0xd483e00 thru 0xd48dd91 (0x9f92 bytes) 2024-09-17T21:18:50.844 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:50.844+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:50.857 INFO:teuthology.orchestra.run.smithi184.stdout:1041 trunc from 0xed3b800 to 0x655b800 2024-09-17T21:18:50.887 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:50.887 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:50.887 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:50.887 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:50.894 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:18:50.894 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:18:50.903 INFO:teuthology.orchestra.run.smithi184.stdout:1042 read 0x5e12600 thru 0x5e1b8c3 (0x92c4 bytes) 2024-09-17T21:18:50.904 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:18:50.906 INFO:teuthology.orchestra.run.smithi184.stdout:1043 read 0x4948200 thru 0x494d9a7 (0x57a8 bytes) 2024-09-17T21:18:50.907 INFO:teuthology.orchestra.run.smithi184.stdout:1045 read 0x1a46800 thru 0x1a483c7 (0x1bc8 bytes) 2024-09-17T21:18:50.908 INFO:teuthology.orchestra.run.smithi184.stdout:1046 clone 11 order 21 su 65536 sc 10 2024-09-17T21:18:50.914 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:18:50.915 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:18:50.932 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:18:50.932 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:18:50.946 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:50.945+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:50.953 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:18:50.954 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:18:50.976 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:18:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:18:50.994 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:18:50.995 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:18:51.028 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:51.028 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:51.042 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:18:51.047 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:51.045+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:51.148 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:51.144+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:51.204 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:51.204 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:51.218 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:18:51.260 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:51.260+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:51.309 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:18:51.349 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:51.345+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:51.367 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:18:51.368 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:18:51.368 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-17T21:18:51.368 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T21:18:51.385 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:51.385 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:51.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:18:51.454 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone10 from 0x655b800 (overlap 0x57c7600) to 0x1a55600 2024-09-17T21:18:51.488 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:51.487+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:51.551 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:51.547+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:51.565 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:51.565 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:51.580 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:18:51.668 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:51.667+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:51.694 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:18:51.694 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:51.694 INFO:teuthology.orchestra.run.smithi083.stdout: "epoch": 53, 2024-09-17T21:18:51.694 INFO:teuthology.orchestra.run.smithi083.stdout: "fsid": "224a7a29-e112-472e-8a13-f67345d6d123", 2024-09-17T21:18:51.694 INFO:teuthology.orchestra.run.smithi083.stdout: "created": "2024-09-17T21:17:42.597312+0000", 2024-09-17T21:18:51.694 INFO:teuthology.orchestra.run.smithi083.stdout: "modified": "2024-09-17T21:18:51.298603+0000", 2024-09-17T21:18:51.694 INFO:teuthology.orchestra.run.smithi083.stdout: "last_up_change": "2024-09-17T21:17:53.976897+0000", 2024-09-17T21:18:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "last_in_change": "2024-09-17T21:18:39.781919+0000", 2024-09-17T21:18:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T21:18:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_num": 5806080, 2024-09-17T21:18:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_set": [ 2024-09-17T21:18:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "nodeep-scrub", 2024-09-17T21:18:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "noscrub", 2024-09-17T21:18:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "pglog_hardlimit", 2024-09-17T21:18:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "purged_snapdirs", 2024-09-17T21:18:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_deletes", 2024-09-17T21:18:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "sortbitwise" 2024-09-17T21:18:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:18:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_version": 5, 2024-09-17T21:18:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T21:18:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T21:18:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T21:18:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_snapshot": "", 2024-09-17T21:18:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_max": 3, 2024-09-17T21:18:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd": 6, 2024-09-17T21:18:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "require_min_compat_client": "luminous", 2024-09-17T21:18:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "min_compat_client": "luminous", 2024-09-17T21:18:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "require_osd_release": "squid", 2024-09-17T21:18:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_crimson": false, 2024-09-17T21:18:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "pools": [ 2024-09-17T21:18:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 1, 2024-09-17T21:18:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": ".mgr", 2024-09-17T21:18:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-17T21:17:55.581350+0000", 2024-09-17T21:18:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-17T21:18:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-17T21:18:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-17T21:18:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 2, 2024-09-17T21:18:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 1, 2024-09-17T21:18:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-17T21:18:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:18:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:18:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:18:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:18:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-17T21:18:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-17T21:18:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:18:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 1, 2024-09-17T21:18:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 1, 2024-09-17T21:18:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 1, 2024-09-17T21:18:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 1, 2024-09-17T21:18:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 1, 2024-09-17T21:18:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-17T21:18:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-17T21:18:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-17T21:18:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-17T21:18:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-17T21:18:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-17T21:18:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-17T21:18:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "17", 2024-09-17T21:18:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-17T21:18:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:18:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:18:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-17T21:18:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:18:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-17T21:18:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-17T21:18:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-17T21:18:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-17T21:18:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-17T21:18:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-17T21:18:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-17T21:18:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-17T21:18:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-17T21:18:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-17T21:18:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-17T21:18:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-17T21:18:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-17T21:18:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:18:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:18:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:18:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-17T21:18:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-17T21:18:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-17T21:18:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-17T21:18:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-17T21:18:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-17T21:18:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-17T21:18:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-17T21:18:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:18:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:18:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:18:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:18:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-17T21:18:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-17T21:18:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-17T21:18:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-17T21:18:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "options": { 2024-09-17T21:18:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_max": 32, 2024-09-17T21:18:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_min": 1 2024-09-17T21:18:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-17T21:18:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": {} 2024-09-17T21:18:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-17T21:18:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-17T21:18:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 6.059999942779541, 2024-09-17T21:18:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 6.059999942779541, 2024-09-17T21:18:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 0.33000001311302185, 2024-09-17T21:18:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2, 2024-09-17T21:18:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2, 2024-09-17T21:18:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 0.87948358058929443, 2024-09-17T21:18:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-17T21:18:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-17T21:18:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T21:18:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "rbd", 2024-09-17T21:18:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-17T21:17:59.603025+0000", 2024-09-17T21:18:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 8193, 2024-09-17T21:18:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:18:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-17T21:18:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 2, 2024-09-17T21:18:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 1, 2024-09-17T21:18:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-17T21:18:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:18:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:18:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:18:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:18:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-17T21:18:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-17T21:18:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:18:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 8, 2024-09-17T21:18:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 8, 2024-09-17T21:18:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 8, 2024-09-17T21:18:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 8, 2024-09-17T21:18:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 8, 2024-09-17T21:18:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-17T21:18:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-17T21:18:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-17T21:18:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-17T21:18:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-17T21:18:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-17T21:18:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-17T21:18:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "21", 2024-09-17T21:18:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-17T21:18:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:18:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:18:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-17T21:18:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:18:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 2, 2024-09-17T21:18:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 21, 2024-09-17T21:18:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-17T21:18:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-17T21:18:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-17T21:18:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-17T21:18:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-17T21:18:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-17T21:18:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-17T21:18:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-17T21:18:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-17T21:18:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-17T21:18:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-17T21:18:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:18:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:18:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:18:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-17T21:18:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-17T21:18:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-17T21:18:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-17T21:18:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-17T21:18:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-17T21:18:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-17T21:18:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-17T21:18:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:18:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:18:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:18:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:18:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-17T21:18:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-17T21:18:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-17T21:18:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-17T21:18:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-17T21:18:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-17T21:18:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd": {} 2024-09-17T21:18:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-17T21:18:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-17T21:18:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 2.2699999809265137, 2024-09-17T21:18:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 2.2699999809265137, 2024-09-17T21:18:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 0.82999998331069946, 2024-09-17T21:18:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-17T21:18:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-17T21:18:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 0.87948358058929443, 2024-09-17T21:18:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-17T21:18:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-17T21:18:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 3, 2024-09-17T21:18:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "pool_client.0", 2024-09-17T21:18:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-17T21:18:09.441137+0000", 2024-09-17T21:18:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 8193, 2024-09-17T21:18:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:18:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-17T21:18:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 2, 2024-09-17T21:18:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 1, 2024-09-17T21:18:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-17T21:18:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:18:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:18:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:18:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:18:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-17T21:18:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-17T21:18:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:18:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-17T21:18:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-17T21:18:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-17T21:18:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-17T21:18:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-17T21:18:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-17T21:18:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-17T21:18:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-17T21:18:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-17T21:18:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-17T21:18:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-17T21:18:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-17T21:18:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "53", 2024-09-17T21:18:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-17T21:18:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:18:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:18:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-17T21:18:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:18:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 13, 2024-09-17T21:18:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 53, 2024-09-17T21:18:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-17T21:18:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-17T21:18:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-17T21:18:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-17T21:18:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-17T21:18:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-17T21:18:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-17T21:18:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-17T21:18:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-17T21:18:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-17T21:18:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-17T21:18:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:18:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:18:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:18:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-17T21:18:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-17T21:18:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-17T21:18:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-17T21:18:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-17T21:18:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-17T21:18:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-17T21:18:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-17T21:18:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:18:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:18:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:18:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:18:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-17T21:18:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-17T21:18:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-17T21:18:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-17T21:18:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-17T21:18:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-17T21:18:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd": {} 2024-09-17T21:18:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-17T21:18:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-17T21:18:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.8799999952316284, 2024-09-17T21:18:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.8799999952316284, 2024-09-17T21:18:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 0.82999998331069946, 2024-09-17T21:18:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.559999942779541, 2024-09-17T21:18:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.559999942779541, 2024-09-17T21:18:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 0.87948358058929443, 2024-09-17T21:18:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-17T21:18:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-17T21:18:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:18:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-17T21:18:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-17T21:18:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "5bfd41c9-0cac-47c4-9a6e-8dce774c3314", 2024-09-17T21:18:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:18:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 0, 2024-09-17T21:18:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 0, 2024-09-17T21:18:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-17T21:18:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:18:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:18:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:18:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 44, 2024-09-17T21:18:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:18:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:18:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:18:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6816", 2024-09-17T21:18:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6817", 2024-09-17T21:18:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:18:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6818", 2024-09-17T21:18:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6819", 2024-09-17T21:18:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:18:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6822", 2024-09-17T21:18:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6823", 2024-09-17T21:18:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:18:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6820", 2024-09-17T21:18:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6821", 2024-09-17T21:18:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:18:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6817/1111036754", 2024-09-17T21:18:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6819/1111036754", 2024-09-17T21:18:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6823/1111036754", 2024-09-17T21:18:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6821/1111036754", 2024-09-17T21:18:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:18:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:18:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:18:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-17T21:18:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "78404e0d-d397-4806-8e90-5f32182643bb", 2024-09-17T21:18:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:18:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:18:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:18:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-17T21:18:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:18:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:18:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:18:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 50, 2024-09-17T21:18:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:18:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:18:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:18:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6808", 2024-09-17T21:18:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6809", 2024-09-17T21:18:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:18:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6810", 2024-09-17T21:18:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6811", 2024-09-17T21:18:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:18:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6814", 2024-09-17T21:18:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6815", 2024-09-17T21:18:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:18:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6812", 2024-09-17T21:18:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6813", 2024-09-17T21:18:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:18:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6809/1460363937", 2024-09-17T21:18:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6811/1460363937", 2024-09-17T21:18:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6815/1460363937", 2024-09-17T21:18:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6813/1460363937", 2024-09-17T21:18:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:18:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:18:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:18:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-17T21:18:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "286ed6e0-c121-4ebb-8cf6-37e506a397aa", 2024-09-17T21:18:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:18:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:18:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:18:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 0.2769012451171875, 2024-09-17T21:18:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:18:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:18:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 12, 2024-09-17T21:18:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 39, 2024-09-17T21:18:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:18:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:18:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:18:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6800", 2024-09-17T21:18:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6801", 2024-09-17T21:18:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:18:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6802", 2024-09-17T21:18:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6803", 2024-09-17T21:18:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:18:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6806", 2024-09-17T21:18:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6807", 2024-09-17T21:18:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:18:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6804", 2024-09-17T21:18:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6805", 2024-09-17T21:18:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:18:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6801/2837444034", 2024-09-17T21:18:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6803/2837444034", 2024-09-17T21:18:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6807/2837444034", 2024-09-17T21:18:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6805/2837444034", 2024-09-17T21:18:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:18:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:18:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:18:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-17T21:18:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "b0af73a1-4bca-4892-a550-2ced87f4c357", 2024-09-17T21:18:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:18:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:18:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:18:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-17T21:18:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:18:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:18:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:18:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 51, 2024-09-17T21:18:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:18:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:18:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:18:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6808", 2024-09-17T21:18:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6809", 2024-09-17T21:18:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:18:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6810", 2024-09-17T21:18:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6811", 2024-09-17T21:18:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:18:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6814", 2024-09-17T21:18:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6815", 2024-09-17T21:18:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:18:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6812", 2024-09-17T21:18:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6813", 2024-09-17T21:18:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:18:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.97:6809/3637204865", 2024-09-17T21:18:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.97:6811/3637204865", 2024-09-17T21:18:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.97:6815/3637204865", 2024-09-17T21:18:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.97:6813/3637204865", 2024-09-17T21:18:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:18:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:18:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:18:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-17T21:18:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "2188361c-e1a1-45fa-bc01-da5a984ffd9f", 2024-09-17T21:18:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:18:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:18:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:18:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-17T21:18:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:18:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:18:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:18:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 42, 2024-09-17T21:18:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:18:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:18:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:18:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6800", 2024-09-17T21:18:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6801", 2024-09-17T21:18:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:18:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6802", 2024-09-17T21:18:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6803", 2024-09-17T21:18:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:18:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6806", 2024-09-17T21:18:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6807", 2024-09-17T21:18:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:18:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6804", 2024-09-17T21:18:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6805", 2024-09-17T21:18:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:18:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.97:6801/2391356911", 2024-09-17T21:18:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.97:6803/2391356911", 2024-09-17T21:18:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.97:6807/2391356911", 2024-09-17T21:18:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.97:6805/2391356911", 2024-09-17T21:18:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:18:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:18:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:18:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-17T21:18:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "7feccf4d-d8e4-4d06-ac83-bbc1bc623a5c", 2024-09-17T21:18:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:18:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:18:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:18:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-17T21:18:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:18:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:18:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:18:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 42, 2024-09-17T21:18:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:18:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:18:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:18:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6816", 2024-09-17T21:18:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6817", 2024-09-17T21:18:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:18:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6818", 2024-09-17T21:18:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6819", 2024-09-17T21:18:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:18:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6822", 2024-09-17T21:18:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6823", 2024-09-17T21:18:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:18:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:18:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:18:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6820", 2024-09-17T21:18:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:18:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6821", 2024-09-17T21:18:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:18:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.97:6817/2326316381", 2024-09-17T21:18:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.97:6819/2326316381", 2024-09-17T21:18:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.97:6823/2326316381", 2024-09-17T21:18:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.97:6821/2326316381", 2024-09-17T21:18:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:18:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:18:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:18:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:18:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_xinfo": [ 2024-09-17T21:18:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-17T21:18:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:18:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:18:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:18:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:18:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 65536, 2024-09-17T21:18:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:51.138101+0000", 2024-09-17T21:18:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:18:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-17T21:18:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:18:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:18:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:18:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:18:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:18:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:50.581632+0000", 2024-09-17T21:18:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:18:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-17T21:18:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:18:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:18:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:18:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:18:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:18:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:50.365193+0000", 2024-09-17T21:18:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:18:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-17T21:18:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:18:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:18:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:18:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:18:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:18:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:50.718974+0000", 2024-09-17T21:18:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:18:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-17T21:18:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:18:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:18:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:18:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:18:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:18:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:50.501878+0000", 2024-09-17T21:18:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:18:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-17T21:18:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:18:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:18:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:18:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:18:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:18:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:51.150747+0000", 2024-09-17T21:18:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:18:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:18:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap": [ 2024-09-17T21:18:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "3.8", 2024-09-17T21:18:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-17T21:18:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: 2, 2024-09-17T21:18:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: 3 2024-09-17T21:18:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "3.1e", 2024-09-17T21:18:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-17T21:18:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: 0, 2024-09-17T21:18:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: 4 2024-09-17T21:18:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:18:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_items": [], 2024-09-17T21:18:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_primaries": [], 2024-09-17T21:18:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_temp": [], 2024-09-17T21:18:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_temp": [], 2024-09-17T21:18:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklist": {}, 2024-09-17T21:18:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "range_blocklist": {}, 2024-09-17T21:18:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profiles": { 2024-09-17T21:18:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "default": { 2024-09-17T21:18:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-failure-domain": "osd", 2024-09-17T21:18:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "k": "2", 2024-09-17T21:18:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "m": "1", 2024-09-17T21:18:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "plugin": "jerasure", 2024-09-17T21:18:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "technique": "reed_sol_van" 2024-09-17T21:18:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps_queue": [], 2024-09-17T21:18:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "new_removed_snaps": [], 2024-09-17T21:18:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "new_purged_snaps": [], 2024-09-17T21:18:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_node_flags": {}, 2024-09-17T21:18:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "device_class_flags": {}, 2024-09-17T21:18:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode": { 2024-09-17T21:18:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_enabled": false, 2024-09-17T21:18:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_bucket_count": 0, 2024-09-17T21:18:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "degraded_stretch_mode": 0, 2024-09-17T21:18:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "recovering_stretch_mode": 0, 2024-09-17T21:18:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_bucket": 0 2024-09-17T21:18:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:51.754 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:51.755 INFO:tasks.thrashosds.thrasher:j is {'epoch': 53, 'fsid': '224a7a29-e112-472e-8a13-f67345d6d123', 'created': '2024-09-17T21:17:42.597312+0000', 'modified': '2024-09-17T21:18:51.298603+0000', 'last_up_change': '2024-09-17T21:17:53.976897+0000', 'last_in_change': '2024-09-17T21:18:39.781919+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T21:17:55.581350+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8794835805892944, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T21:17:59.603025+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.2699999809265137, 'score_stable': 2.2699999809265137, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.8794835805892944, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T21:18:09.441137+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 13, 'snap_epoch': 53, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.559999942779541, 'raw_score_stable': 1.559999942779541, 'primary_affinity_weighted': 0.8794835805892944, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '5bfd41c9-0cac-47c4-9a6e-8dce774c3314', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6816', 'nonce': 1111036754}, {'type': 'v1', 'addr': '172.21.15.83:6817', 'nonce': 1111036754}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6818', 'nonce': 1111036754}, {'type': 'v1', 'addr': '172.21.15.83:6819', 'nonce': 1111036754}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6822', 'nonce': 1111036754}, {'type': 'v1', 'addr': '172.21.15.83:6823', 'nonce': 1111036754}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6820', 'nonce': 1111036754}, {'type': 'v1', 'addr': '172.21.15.83:6821', 'nonce': 1111036754}]}, 'public_addr': '172.21.15.83:6817/1111036754', 'cluster_addr': '172.21.15.83:6819/1111036754', 'heartbeat_back_addr': '172.21.15.83:6823/1111036754', 'heartbeat_front_addr': '172.21.15.83:6821/1111036754', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '78404e0d-d397-4806-8e90-5f32182643bb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6808', 'nonce': 1460363937}, {'type': 'v1', 'addr': '172.21.15.83:6809', 'nonce': 1460363937}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6810', 'nonce': 1460363937}, {'type': 'v1', 'addr': '172.21.15.83:6811', 'nonce': 1460363937}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6814', 'nonce': 1460363937}, {'type': 'v1', 'addr': '172.21.15.83:6815', 'nonce': 1460363937}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6812', 'nonce': 1460363937}, {'type': 'v1', 'addr': '172.21.15.83:6813', 'nonce': 1460363937}]}, 'public_addr': '172.21.15.83:6809/1460363937', 'cluster_addr': '172.21.15.83:6811/1460363937', 'heartbeat_back_addr': '172.21.15.83:6815/1460363937', 'heartbeat_front_addr': '172.21.15.83:6813/1460363937', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '286ed6e0-c121-4ebb-8cf6-37e506a397aa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.2769012451171875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6800', 'nonce': 2837444034}, {'type': 'v1', 'addr': '172.21.15.83:6801', 'nonce': 2837444034}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6802', 'nonce': 2837444034}, {'type': 'v1', 'addr': '172.21.15.83:6803', 'nonce': 2837444034}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6806', 'nonce': 2837444034}, {'type': 'v1', 'addr': '172.21.15.83:6807', 'nonce': 2837444034}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6804', 'nonce': 2837444034}, {'type': 'v1', 'addr': '172.21.15.83:6805', 'nonce': 2837444034}]}, 'public_addr': '172.21.15.83:6801/2837444034', 'cluster_addr': '172.21.15.83:6803/2837444034', 'heartbeat_back_addr': '172.21.15.83:6807/2837444034', 'heartbeat_front_addr': '172.21.15.83:6805/2837444034', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'b0af73a1-4bca-4892-a550-2ced87f4c357', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6808', 'nonce': 3637204865}, {'type': 'v1', 'addr': '172.21.15.97:6809', 'nonce': 3637204865}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6810', 'nonce': 3637204865}, {'type': 'v1', 'addr': '172.21.15.97:6811', 'nonce': 3637204865}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6814', 'nonce': 3637204865}, {'type': 'v1', 'addr': '172.21.15.97:6815', 'nonce': 3637204865}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6812', 'nonce': 3637204865}, {'type': 'v1', 'addr': '172.21.15.97:6813', 'nonce': 3637204865}]}, 'public_addr': '172.21.15.97:6809/3637204865', 'cluster_addr': '172.21.15.97:6811/3637204865', 'heartbeat_back_addr': '172.21.15.97:6815/3637204865', 'heartbeat_front_addr': '172.21.15.97:6813/3637204865', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2188361c-e1a1-45fa-bc01-da5a984ffd9f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 42, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6800', 'nonce': 2391356911}, {'type': 'v1', 'addr': '172.21.15.97:6801', 'nonce': 2391356911}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6802', 'nonce': 2391356911}, {'type': 'v1', 'addr': '172.21.15.97:6803', 'nonce': 2391356911}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6806', 'nonce': 2391356911}, {'type': 'v1', 'addr': '172.21.15.97:6807', 'nonce': 2391356911}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6804', 'nonce': 2391356911}, {'type': 'v1', 'addr': '172.21.15.97:6805', 'nonce': 2391356911}]}, 'public_addr': '172.21.15.97:6801/2391356911', 'cluster_addr': '172.21.15.97:6803/2391356911', 'heartbeat_back_addr': '172.21.15.97:6807/2391356911', 'heartbeat_front_addr': '172.21.15.97:6805/2391356911', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '7feccf4d-d8e4-4d06-ac83-bbc1bc623a5c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 42, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6816', 'nonce': 2326316381}, {'type': 'v1', 'addr': '172.21.15.97:6817', 'nonce': 2326316381}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6818', 'nonce': 2326316381}, {'type': 'v1', 'addr': '172.21.15.97:6819', 'nonce': 2326316381}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6822', 'nonce': 2326316381}, {'type': 'v1', 'addr': '172.21.15.97:6823', 'nonce': 2326316381}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6820', 'nonce': 2326316381}, {'type': 'v1', 'addr': '172.21.15.97:6821', 'nonce': 2326316381}]}, 'public_addr': '172.21.15.97:6817/2326316381', 'cluster_addr': '172.21.15.97:6819/2326316381', 'heartbeat_back_addr': '172.21.15.97:6823/2326316381', 'heartbeat_front_addr': '172.21.15.97:6821/2326316381', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-17T21:17:51.138101+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:50.581632+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:50.365193+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:50.718974+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:50.501878+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:51.150747+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.8', 'osds': [2, 3]}, {'pgid': '3.1e', 'osds': [0, 4]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T21:18:51.756 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:18:51.756 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:51.756 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:51.756 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:51.751+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:51.758 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:18:51.855 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:51.853+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:51.915 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:51.916 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:51.932 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:18:51.956 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:51.953+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:51.992 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-17T21:18:52.046 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:52.046 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:52.057 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:52.053+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:52.060 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:18:52.107 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:18:52.107 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T21:18:52.127 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-09-17T21:18:51.548088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":394108964,"num_objects":276,"num_object_clones":45,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":43,"num_read":43136,"num_read_kb":222745,"num_write":2708,"num_write_kb":149637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":23291929,"num_keys_recovered":75,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2298,"ondisk_log_size":2298,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":283792,"kb_used_data":127836,"kb_used_omap":7,"kb_used_meta":155832,"kb_avail":471575408,"statfs":{"total":483183820800,"available":482893217792,"internally_reserved":0,"allocated":130904064,"data_stored":187569145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7978,"internal_metadata":159572182},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":281,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3559,"num_read_kb":19245,"num_write":15,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3239440,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001040"},"pg_stats":[{"pgid":"3.1f","version":"46'15","reported_seq":262,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.284250+0000","last_change":"2024-09-17T21:18:40.931000+0000","last_active":"2024-09-17T21:18:49.284250+0000","last_peered":"2024-09-17T21:18:49.284250+0000","last_clean":"2024-09-17T21:18:49.284250+0000","last_became_active":"2024-09-17T21:18:40.810689+0000","last_became_peered":"2024-09-17T21:18:40.810689+0000","last_unstale":"2024-09-17T21:18:49.284250+0000","last_undegraded":"2024-09-17T21:18:49.284250+0000","last_fullsized":"2024-09-17T21:18:49.284250+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:26:43.609232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088525599999999996,"stat_sum":{"num_bytes":2931776,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":184,"num_read_kb":1019,"num_write":17,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":96832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"46'139","reported_seq":1397,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.283369+0000","last_change":"2024-09-17T21:18:43.278035+0000","last_active":"2024-09-17T21:18:49.283369+0000","last_peered":"2024-09-17T21:18:49.283369+0000","last_clean":"2024-09-17T21:18:49.283369+0000","last_became_active":"2024-09-17T21:18:43.277411+0000","last_became_peered":"2024-09-17T21:18:43.277411+0000","last_unstale":"2024-09-17T21:18:49.283369+0000","last_undegraded":"2024-09-17T21:18:49.283369+0000","last_fullsized":"2024-09-17T21:18:49.283369+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:21.117143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013256229999999999,"stat_sum":{"num_bytes":7076906,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":2059,"num_read_kb":2580,"num_write":146,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":127572,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"46'131","reported_seq":2270,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.283732+0000","last_change":"2024-09-17T21:18:13.324697+0000","last_active":"2024-09-17T21:18:49.283732+0000","last_peered":"2024-09-17T21:18:49.283732+0000","last_clean":"2024-09-17T21:18:49.283732+0000","last_became_active":"2024-09-17T21:18:11.302254+0000","last_became_peered":"2024-09-17T21:18:11.302254+0000","last_unstale":"2024-09-17T21:18:49.283732+0000","last_undegraded":"2024-09-17T21:18:49.283732+0000","last_fullsized":"2024-09-17T21:18:49.283732+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:57:42.499497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018973079999999999,"stat_sum":{"num_bytes":15739422,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":3,"num_read":2420,"num_read_kb":8912,"num_write":169,"num_write_kb":6219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"47'15","reported_seq":260,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.283535+0000","last_change":"2024-09-17T21:18:13.326517+0000","last_active":"2024-09-17T21:18:49.283535+0000","last_peered":"2024-09-17T21:18:49.283535+0000","last_clean":"2024-09-17T21:18:49.283535+0000","last_became_active":"2024-09-17T21:18:11.302134+0000","last_became_peered":"2024-09-17T21:18:11.302134+0000","last_unstale":"2024-09-17T21:18:49.283535+0000","last_undegraded":"2024-09-17T21:18:49.283535+0000","last_fullsized":"2024-09-17T21:18:49.283535+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:52:28.661621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034186630000000002,"stat_sum":{"num_bytes":1048576,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":189,"num_read_kb":2229,"num_write":17,"num_write_kb":1051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"46'28","reported_seq":495,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.280896+0000","last_change":"2024-09-17T21:18:41.001528+0000","last_active":"2024-09-17T21:18:49.280896+0000","last_peered":"2024-09-17T21:18:49.280896+0000","last_clean":"2024-09-17T21:18:49.280896+0000","last_became_active":"2024-09-17T21:18:40.827008+0000","last_became_peered":"2024-09-17T21:18:40.827008+0000","last_unstale":"2024-09-17T21:18:49.280896+0000","last_undegraded":"2024-09-17T21:18:49.280896+0000","last_fullsized":"2024-09-17T21:18:49.280896+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:50:13.140996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089504999999999999,"stat_sum":{"num_bytes":4260352,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":2,"num_read":389,"num_read_kb":1617,"num_write":28,"num_write_kb":843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":615936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"46'182","reported_seq":2773,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.280896+0000","last_change":"2024-09-17T21:18:13.314063+0000","last_active":"2024-09-17T21:18:49.280896+0000","last_peered":"2024-09-17T21:18:49.280896+0000","last_clean":"2024-09-17T21:18:49.280896+0000","last_became_active":"2024-09-17T21:18:11.301426+0000","last_became_peered":"2024-09-17T21:18:11.301426+0000","last_unstale":"2024-09-17T21:18:49.280896+0000","last_undegraded":"2024-09-17T21:18:49.280896+0000","last_fullsized":"2024-09-17T21:18:49.280896+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:14:40.859232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027366090000000001,"stat_sum":{"num_bytes":14090808,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":4017,"num_read_kb":8226,"num_write":161,"num_write_kb":1189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"46'15","reported_seq":912,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.287465+0000","last_change":"2024-09-17T21:18:13.314862+0000","last_active":"2024-09-17T21:18:49.287465+0000","last_peered":"2024-09-17T21:18:49.287465+0000","last_clean":"2024-09-17T21:18:49.287465+0000","last_became_active":"2024-09-17T21:18:11.300793+0000","last_became_peered":"2024-09-17T21:18:11.300793+0000","last_unstale":"2024-09-17T21:18:49.287465+0000","last_undegraded":"2024-09-17T21:18:49.287465+0000","last_fullsized":"2024-09-17T21:18:49.287465+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:52:46.942311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047673000000000002,"stat_sum":{"num_bytes":12744704,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":825,"num_read_kb":3567,"num_write":47,"num_write_kb":1515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"46'82","reported_seq":141,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.287528+0000","last_change":"2024-09-17T21:18:40.830426+0000","last_active":"2024-09-17T21:18:49.287528+0000","last_peered":"2024-09-17T21:18:49.287528+0000","last_clean":"2024-09-17T21:18:49.287528+0000","last_became_active":"2024-09-17T21:18:40.801216+0000","last_became_peered":"2024-09-17T21:18:40.801216+0000","last_unstale":"2024-09-17T21:18:49.287528+0000","last_undegraded":"2024-09-17T21:18:49.287528+0000","last_fullsized":"2024-09-17T21:18:49.287528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:54:44.905219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9056304,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":796,"num_read_kb":3066,"num_write":117,"num_write_kb":1005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":428592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"46'26","reported_seq":426,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.283576+0000","last_change":"2024-09-17T21:18:40.854226+0000","last_active":"2024-09-17T21:18:49.283576+0000","last_peered":"2024-09-17T21:18:49.283576+0000","last_clean":"2024-09-17T21:18:49.283576+0000","last_became_active":"2024-09-17T21:18:40.812620+0000","last_became_peered":"2024-09-17T21:18:40.812620+0000","last_unstale":"2024-09-17T21:18:49.283576+0000","last_undegraded":"2024-09-17T21:18:49.283576+0000","last_fullsized":"2024-09-17T21:18:49.283576+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:00:13.987350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088962999999999998,"stat_sum":{"num_bytes":6785024,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":334,"num_read_kb":3368,"num_write":36,"num_write_kb":1370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":793088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"46'91","reported_seq":643,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.283936+0000","last_change":"2024-09-17T21:18:13.324809+0000","last_active":"2024-09-17T21:18:49.283936+0000","last_peered":"2024-09-17T21:18:49.283936+0000","last_clean":"2024-09-17T21:18:49.283936+0000","last_became_active":"2024-09-17T21:18:11.303835+0000","last_became_peered":"2024-09-17T21:18:11.303835+0000","last_unstale":"2024-09-17T21:18:49.283936+0000","last_undegraded":"2024-09-17T21:18:49.283936+0000","last_fullsized":"2024-09-17T21:18:49.283936+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:10:14.428791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001825658,"stat_sum":{"num_bytes":3637322,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":825,"num_read_kb":2995,"num_write":152,"num_write_kb":10323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"46'41","reported_seq":1259,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.284075+0000","last_change":"2024-09-17T21:18:41.006288+0000","last_active":"2024-09-17T21:18:49.284075+0000","last_peered":"2024-09-17T21:18:49.284075+0000","last_clean":"2024-09-17T21:18:49.284075+0000","last_became_active":"2024-09-17T21:18:40.820709+0000","last_became_peered":"2024-09-17T21:18:40.820709+0000","last_unstale":"2024-09-17T21:18:49.284075+0000","last_undegraded":"2024-09-17T21:18:49.284075+0000","last_fullsized":"2024-09-17T21:18:49.284075+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:32:15.777277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027112149999999999,"stat_sum":{"num_bytes":13129766,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":925,"num_read_kb":5192,"num_write":49,"num_write_kb":1776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1160742,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"43'44","reported_seq":883,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.283368+0000","last_change":"2024-09-17T21:18:13.307234+0000","last_active":"2024-09-17T21:18:49.283368+0000","last_peered":"2024-09-17T21:18:49.283368+0000","last_clean":"2024-09-17T21:18:49.283368+0000","last_became_active":"2024-09-17T21:18:11.299817+0000","last_became_peered":"2024-09-17T21:18:11.299817+0000","last_unstale":"2024-09-17T21:18:49.283368+0000","last_undegraded":"2024-09-17T21:18:49.283368+0000","last_fullsized":"2024-09-17T21:18:49.283368+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:55:12.190544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002281113,"stat_sum":{"num_bytes":10600976,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":787,"num_read_kb":4371,"num_write":53,"num_write_kb":11299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.284416+0000","last_change":"2024-09-17T21:18:03.059715+0000","last_active":"2024-09-17T21:18:49.284416+0000","last_peered":"2024-09-17T21:18:49.284416+0000","last_clean":"2024-09-17T21:18:49.284416+0000","last_became_active":"2024-09-17T21:18:01.213047+0000","last_became_peered":"2024-09-17T21:18:01.213047+0000","last_unstale":"2024-09-17T21:18:49.284416+0000","last_undegraded":"2024-09-17T21:18:49.284416+0000","last_fullsized":"2024-09-17T21:18:49.284416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:46:35.222373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"47'146","reported_seq":947,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.288106+0000","last_change":"2024-09-17T21:18:40.976309+0000","last_active":"2024-09-17T21:18:49.288106+0000","last_peered":"2024-09-17T21:18:49.288106+0000","last_clean":"2024-09-17T21:18:49.288106+0000","last_became_active":"2024-09-17T21:18:40.818354+0000","last_became_peered":"2024-09-17T21:18:40.818354+0000","last_unstale":"2024-09-17T21:18:49.288106+0000","last_undegraded":"2024-09-17T21:18:49.288106+0000","last_fullsized":"2024-09-17T21:18:49.288106+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:50.488434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084327200000000005,"stat_sum":{"num_bytes":21660462,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1360,"num_read_kb":20328,"num_write":249,"num_write_kb":13042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8773670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.288054+0000","last_change":"2024-09-17T21:18:03.073440+0000","last_active":"2024-09-17T21:18:49.288054+0000","last_peered":"2024-09-17T21:18:49.288054+0000","last_clean":"2024-09-17T21:18:49.288054+0000","last_became_active":"2024-09-17T21:18:01.048806+0000","last_became_peered":"2024-09-17T21:18:01.048806+0000","last_unstale":"2024-09-17T21:18:49.288054+0000","last_undegraded":"2024-09-17T21:18:49.288054+0000","last_fullsized":"2024-09-17T21:18:49.288054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:26.487163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094161400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"43'73","reported_seq":1188,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.288000+0000","last_change":"2024-09-17T21:18:17.280432+0000","last_active":"2024-09-17T21:18:49.288000+0000","last_peered":"2024-09-17T21:18:49.288000+0000","last_clean":"2024-09-17T21:18:49.288000+0000","last_became_active":"2024-09-17T21:18:17.279625+0000","last_became_peered":"2024-09-17T21:18:17.279625+0000","last_unstale":"2024-09-17T21:18:49.288000+0000","last_undegraded":"2024-09-17T21:18:49.288000+0000","last_fullsized":"2024-09-17T21:18:49.288000+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:44:14.653225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21278756,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":1226,"num_read_kb":15096,"num_write":98,"num_write_kb":11083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.284151+0000","last_change":"2024-09-17T21:18:40.821956+0000","last_active":"2024-09-17T21:18:49.284151+0000","last_peered":"2024-09-17T21:18:49.284151+0000","last_clean":"2024-09-17T21:18:49.284151+0000","last_became_active":"2024-09-17T21:18:40.820574+0000","last_became_peered":"2024-09-17T21:18:40.820574+0000","last_unstale":"2024-09-17T21:18:49.284151+0000","last_undegraded":"2024-09-17T21:18:49.284151+0000","last_fullsized":"2024-09-17T21:18:49.284151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:35:05.837282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099219900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"46'33","reported_seq":1309,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.287817+0000","last_change":"2024-09-17T21:18:13.315662+0000","last_active":"2024-09-17T21:18:49.287817+0000","last_peered":"2024-09-17T21:18:49.287817+0000","last_clean":"2024-09-17T21:18:49.287817+0000","last_became_active":"2024-09-17T21:18:11.309133+0000","last_became_peered":"2024-09-17T21:18:11.309133+0000","last_unstale":"2024-09-17T21:18:49.287817+0000","last_undegraded":"2024-09-17T21:18:49.287817+0000","last_fullsized":"2024-09-17T21:18:49.287817+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:27:59.732035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078465700000000002,"stat_sum":{"num_bytes":21127168,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":1220,"num_read_kb":12574,"num_write":43,"num_write_kb":5995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.287917+0000","last_change":"2024-09-17T21:18:40.818736+0000","last_active":"2024-09-17T21:18:49.287917+0000","last_peered":"2024-09-17T21:18:49.287917+0000","last_clean":"2024-09-17T21:18:49.287917+0000","last_became_active":"2024-09-17T21:18:40.818309+0000","last_became_peered":"2024-09-17T21:18:40.818309+0000","last_unstale":"2024-09-17T21:18:49.287917+0000","last_undegraded":"2024-09-17T21:18:49.287917+0000","last_fullsized":"2024-09-17T21:18:49.287917+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:42:22.148483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006222,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"47'102","reported_seq":2125,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.283814+0000","last_change":"2024-09-17T21:18:13.327202+0000","last_active":"2024-09-17T21:18:49.283814+0000","last_peered":"2024-09-17T21:18:49.283814+0000","last_clean":"2024-09-17T21:18:49.283814+0000","last_became_active":"2024-09-17T21:18:11.317127+0000","last_became_peered":"2024-09-17T21:18:11.317127+0000","last_unstale":"2024-09-17T21:18:49.283814+0000","last_undegraded":"2024-09-17T21:18:49.283814+0000","last_fullsized":"2024-09-17T21:18:49.283814+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:04:57.724148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036493720000000001,"stat_sum":{"num_bytes":16750623,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2352,"num_read_kb":11444,"num_write":84,"num_write_kb":14246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"34'24","reported_seq":589,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.281097+0000","last_change":"2024-09-17T21:18:40.963953+0000","last_active":"2024-09-17T21:18:49.281097+0000","last_peered":"2024-09-17T21:18:49.281097+0000","last_clean":"2024-09-17T21:18:49.281097+0000","last_became_active":"2024-09-17T21:18:40.820910+0000","last_became_peered":"2024-09-17T21:18:40.820910+0000","last_unstale":"2024-09-17T21:18:49.281097+0000","last_undegraded":"2024-09-17T21:18:49.281097+0000","last_fullsized":"2024-09-17T21:18:49.281097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:14.474479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084988700000000002,"stat_sum":{"num_bytes":3272704,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":469,"num_read_kb":2254,"num_write":26,"num_write_kb":1536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":978944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.283160+0000","last_change":"2024-09-17T21:18:03.070336+0000","last_active":"2024-09-17T21:18:49.283160+0000","last_peered":"2024-09-17T21:18:49.283160+0000","last_clean":"2024-09-17T21:18:49.283160+0000","last_became_active":"2024-09-17T21:18:01.049336+0000","last_became_peered":"2024-09-17T21:18:01.049336+0000","last_unstale":"2024-09-17T21:18:49.283160+0000","last_undegraded":"2024-09-17T21:18:49.283160+0000","last_fullsized":"2024-09-17T21:18:49.283160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:56:24.329470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023986049999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"46'22","reported_seq":432,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.289545+0000","last_change":"2024-09-17T21:18:13.314742+0000","last_active":"2024-09-17T21:18:49.289545+0000","last_peered":"2024-09-17T21:18:49.289545+0000","last_clean":"2024-09-17T21:18:49.289545+0000","last_became_active":"2024-09-17T21:18:11.308568+0000","last_became_peered":"2024-09-17T21:18:11.308568+0000","last_unstale":"2024-09-17T21:18:49.289545+0000","last_undegraded":"2024-09-17T21:18:49.289545+0000","last_fullsized":"2024-09-17T21:18:49.289545+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:28:11.191624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049125000000000004,"stat_sum":{"num_bytes":9221152,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":350,"num_read_kb":6940,"num_write":29,"num_write_kb":2842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":58,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.287534+0000","last_change":"2024-09-17T21:18:17.280358+0000","last_active":"2024-09-17T21:18:49.287534+0000","last_peered":"2024-09-17T21:18:49.287534+0000","last_clean":"2024-09-17T21:18:49.287534+0000","last_became_active":"2024-09-17T21:18:17.279474+0000","last_became_peered":"2024-09-17T21:18:17.279474+0000","last_unstale":"2024-09-17T21:18:49.287534+0000","last_undegraded":"2024-09-17T21:18:49.287534+0000","last_fullsized":"2024-09-17T21:18:49.287534+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:55:54.183527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"46'64","reported_seq":692,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.280727+0000","last_change":"2024-09-17T21:18:40.906152+0000","last_active":"2024-09-17T21:18:49.280727+0000","last_peered":"2024-09-17T21:18:49.280727+0000","last_clean":"2024-09-17T21:18:49.280727+0000","last_became_active":"2024-09-17T21:18:40.816571+0000","last_became_peered":"2024-09-17T21:18:40.816571+0000","last_unstale":"2024-09-17T21:18:49.280727+0000","last_undegraded":"2024-09-17T21:18:49.280727+0000","last_fullsized":"2024-09-17T21:18:49.280727+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:08:25.685226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066398399999999995,"stat_sum":{"num_bytes":7944240,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":755,"num_read_kb":7658,"num_write":96,"num_write_kb":4313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":4070960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.283208+0000","last_change":"2024-09-17T21:18:03.059156+0000","last_active":"2024-09-17T21:18:49.283208+0000","last_peered":"2024-09-17T21:18:49.283208+0000","last_clean":"2024-09-17T21:18:49.283208+0000","last_became_active":"2024-09-17T21:18:01.049180+0000","last_became_peered":"2024-09-17T21:18:01.049180+0000","last_unstale":"2024-09-17T21:18:49.283208+0000","last_undegraded":"2024-09-17T21:18:49.283208+0000","last_fullsized":"2024-09-17T21:18:49.283208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:27.069794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077408,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"46'18","reported_seq":629,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.287580+0000","last_change":"2024-09-17T21:18:13.315294+0000","last_active":"2024-09-17T21:18:49.287580+0000","last_peered":"2024-09-17T21:18:49.287580+0000","last_clean":"2024-09-17T21:18:49.287580+0000","last_became_active":"2024-09-17T21:18:11.294154+0000","last_became_peered":"2024-09-17T21:18:11.294154+0000","last_unstale":"2024-09-17T21:18:49.287580+0000","last_undegraded":"2024-09-17T21:18:49.287580+0000","last_fullsized":"2024-09-17T21:18:49.287580+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:29:01.082985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049408099999999995,"stat_sum":{"num_bytes":33486864,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":547,"num_read_kb":16970,"num_write":33,"num_write_kb":8516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.283468+0000","last_change":"2024-09-17T21:18:03.069091+0000","last_active":"2024-09-17T21:18:49.283468+0000","last_peered":"2024-09-17T21:18:49.283468+0000","last_clean":"2024-09-17T21:18:49.283468+0000","last_became_active":"2024-09-17T21:18:01.044275+0000","last_became_peered":"2024-09-17T21:18:01.044275+0000","last_unstale":"2024-09-17T21:18:49.283468+0000","last_undegraded":"2024-09-17T21:18:49.283468+0000","last_fullsized":"2024-09-17T21:18:49.283468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:28:27.066081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001145884,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":183,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.283315+0000","last_change":"2024-09-17T21:18:40.829967+0000","last_active":"2024-09-17T21:18:49.283315+0000","last_peered":"2024-09-17T21:18:49.283315+0000","last_clean":"2024-09-17T21:18:49.283315+0000","last_became_active":"2024-09-17T21:18:40.803358+0000","last_became_peered":"2024-09-17T21:18:40.803358+0000","last_unstale":"2024-09-17T21:18:49.283315+0000","last_undegraded":"2024-09-17T21:18:49.283315+0000","last_fullsized":"2024-09-17T21:18:49.283315+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_clean_scrub_stamp":"2024-09-17T21:17:55.982339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:57:35.748912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"50'115","reported_seq":1763,"reported_epoch":51,"state":"peering","last_fresh":"2024-09-17T21:18:49.277258+0000","last_change":"2024-09-17T21:18:49.500227+0000","last_active":"2024-09-17T21:18:49.500227+0000","last_peered":"2024-09-17T21:18:49.277258+0000","last_clean":"2024-09-17T21:18:46.283065+0000","last_became_active":"2024-09-17T21:18:48.868603+0000","last_became_peered":"2024-09-17T21:18:48.868603+0000","last_unstale":"2024-09-17T21:18:49.277258+0000","last_undegraded":"2024-09-17T21:18:49.277258+0000","last_fullsized":"2024-09-17T21:18:49.277258+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6194192,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":2897,"num_read_kb":11664,"num_write":87,"num_write_kb":13816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3239440,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"46'155","reported_seq":1168,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.281124+0000","last_change":"2024-09-17T21:18:13.313980+0000","last_active":"2024-09-17T21:18:49.281124+0000","last_peered":"2024-09-17T21:18:49.281124+0000","last_clean":"2024-09-17T21:18:49.281124+0000","last_became_active":"2024-09-17T21:18:11.304021+0000","last_became_peered":"2024-09-17T21:18:11.304021+0000","last_unstale":"2024-09-17T21:18:49.281124+0000","last_undegraded":"2024-09-17T21:18:49.281124+0000","last_fullsized":"2024-09-17T21:18:49.281124+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:49:42.191289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023749890000000001,"stat_sum":{"num_bytes":23323689,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1610,"num_read_kb":7296,"num_write":292,"num_write_kb":2628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"47'32","reported_seq":317,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.283898+0000","last_change":"2024-09-17T21:18:13.307298+0000","last_active":"2024-09-17T21:18:49.283898+0000","last_peered":"2024-09-17T21:18:49.283898+0000","last_clean":"2024-09-17T21:18:49.283898+0000","last_became_active":"2024-09-17T21:18:11.294907+0000","last_became_peered":"2024-09-17T21:18:11.294907+0000","last_unstale":"2024-09-17T21:18:49.283898+0000","last_undegraded":"2024-09-17T21:18:49.283898+0000","last_fullsized":"2024-09-17T21:18:49.283898+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:20:49.281104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021951829999999999,"stat_sum":{"num_bytes":6134354,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":234,"num_read_kb":1515,"num_write":33,"num_write_kb":455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"47'69","reported_seq":888,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.288378+0000","last_change":"2024-09-17T21:18:40.868870+0000","last_active":"2024-09-17T21:18:49.288378+0000","last_peered":"2024-09-17T21:18:49.288378+0000","last_clean":"2024-09-17T21:18:49.288378+0000","last_became_active":"2024-09-17T21:18:40.816043+0000","last_became_peered":"2024-09-17T21:18:40.816043+0000","last_unstale":"2024-09-17T21:18:49.288378+0000","last_undegraded":"2024-09-17T21:18:49.288378+0000","last_fullsized":"2024-09-17T21:18:49.288378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:13:48.528919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048704800000000002,"stat_sum":{"num_bytes":9380368,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":1099,"num_read_kb":4675,"num_write":62,"num_write_kb":10655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":485392,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"46'77","reported_seq":2051,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.283643+0000","last_change":"2024-09-17T21:18:40.925472+0000","last_active":"2024-09-17T21:18:49.283643+0000","last_peered":"2024-09-17T21:18:49.283643+0000","last_clean":"2024-09-17T21:18:49.283643+0000","last_became_active":"2024-09-17T21:18:40.821090+0000","last_became_peered":"2024-09-17T21:18:40.821090+0000","last_unstale":"2024-09-17T21:18:49.283643+0000","last_undegraded":"2024-09-17T21:18:49.283643+0000","last_fullsized":"2024-09-17T21:18:49.283643+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:54:59.466542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015319789999999999,"stat_sum":{"num_bytes":26867200,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":2052,"num_read_kb":6572,"num_write":57,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":81408,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"37'13","reported_seq":129,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.284042+0000","last_change":"2024-09-17T21:18:13.306351+0000","last_active":"2024-09-17T21:18:49.284042+0000","last_peered":"2024-09-17T21:18:49.284042+0000","last_clean":"2024-09-17T21:18:49.284042+0000","last_became_active":"2024-09-17T21:18:11.316999+0000","last_became_peered":"2024-09-17T21:18:11.316999+0000","last_unstale":"2024-09-17T21:18:49.284042+0000","last_undegraded":"2024-09-17T21:18:49.284042+0000","last_fullsized":"2024-09-17T21:18:49.284042+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:40:19.053823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015238949999999999,"stat_sum":{"num_bytes":491520,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":37,"num_read_kb":976,"num_write":14,"num_write_kb":711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"46'154","reported_seq":3238,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.284414+0000","last_change":"2024-09-17T21:18:13.327081+0000","last_active":"2024-09-17T21:18:49.284414+0000","last_peered":"2024-09-17T21:18:49.284414+0000","last_clean":"2024-09-17T21:18:49.284414+0000","last_became_active":"2024-09-17T21:18:11.301652+0000","last_became_peered":"2024-09-17T21:18:11.301652+0000","last_unstale":"2024-09-17T21:18:49.284414+0000","last_undegraded":"2024-09-17T21:18:49.284414+0000","last_fullsized":"2024-09-17T21:18:49.284414+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:46:41.104635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003632209,"stat_sum":{"num_bytes":28959232,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":2,"num_read":3973,"num_read_kb":19163,"num_write":138,"num_write_kb":15635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"46'21","reported_seq":855,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.283984+0000","last_change":"2024-09-17T21:18:13.325500+0000","last_active":"2024-09-17T21:18:49.283984+0000","last_peered":"2024-09-17T21:18:49.283984+0000","last_clean":"2024-09-17T21:18:49.283984+0000","last_became_active":"2024-09-17T21:18:11.307060+0000","last_became_peered":"2024-09-17T21:18:11.307060+0000","last_unstale":"2024-09-17T21:18:49.283984+0000","last_undegraded":"2024-09-17T21:18:49.283984+0000","last_fullsized":"2024-09-17T21:18:49.283984+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:09:45.690108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002454758,"stat_sum":{"num_bytes":2159631,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":771,"num_read_kb":2502,"num_write":17,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"46'59","reported_seq":2610,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.283054+0000","last_change":"2024-09-17T21:18:40.876377+0000","last_active":"2024-09-17T21:18:49.283054+0000","last_peered":"2024-09-17T21:18:49.283054+0000","last_clean":"2024-09-17T21:18:49.283054+0000","last_became_active":"2024-09-17T21:18:40.817254+0000","last_became_peered":"2024-09-17T21:18:40.817254+0000","last_unstale":"2024-09-17T21:18:49.283054+0000","last_undegraded":"2024-09-17T21:18:49.283054+0000","last_fullsized":"2024-09-17T21:18:49.283054+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:49:09.921748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001519261,"stat_sum":{"num_bytes":1272906,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1510,"num_read_kb":3999,"num_write":32,"num_write_kb":731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":538698,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"47'58","reported_seq":1291,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.283475+0000","last_change":"2024-09-17T21:18:13.305680+0000","last_active":"2024-09-17T21:18:49.283475+0000","last_peered":"2024-09-17T21:18:49.283475+0000","last_clean":"2024-09-17T21:18:49.283475+0000","last_became_active":"2024-09-17T21:18:11.299946+0000","last_became_peered":"2024-09-17T21:18:11.299946+0000","last_unstale":"2024-09-17T21:18:49.283475+0000","last_undegraded":"2024-09-17T21:18:49.283475+0000","last_fullsized":"2024-09-17T21:18:49.283475+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:11:07.816624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225884,"stat_sum":{"num_bytes":14460253,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":1261,"num_read_kb":4405,"num_write":69,"num_write_kb":948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"46'50","reported_seq":1347,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.283789+0000","last_change":"2024-09-17T21:18:40.967754+0000","last_active":"2024-09-17T21:18:49.283789+0000","last_peered":"2024-09-17T21:18:49.283789+0000","last_clean":"2024-09-17T21:18:49.283789+0000","last_became_active":"2024-09-17T21:18:40.814521+0000","last_became_peered":"2024-09-17T21:18:40.814521+0000","last_unstale":"2024-09-17T21:18:49.283789+0000","last_undegraded":"2024-09-17T21:18:49.283789+0000","last_fullsized":"2024-09-17T21:18:49.283789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:27:17.888852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013045489999999999,"stat_sum":{"num_bytes":20606543,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":1234,"num_read_kb":5529,"num_write":57,"num_write_kb":2170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1310287,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"47'129","reported_seq":3612,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T21:18:49.283303+0000","last_change":"2024-09-17T21:18:13.306520+0000","last_active":"2024-09-17T21:18:49.283303+0000","last_peered":"2024-09-17T21:18:49.283303+0000","last_clean":"2024-09-17T21:18:49.283303+0000","last_became_active":"2024-09-17T21:18:11.294953+0000","last_became_peered":"2024-09-17T21:18:11.294953+0000","last_unstale":"2024-09-17T21:18:49.283303+0000","last_undegraded":"2024-09-17T21:18:49.283303+0000","last_fullsized":"2024-09-17T21:18:49.283303+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:15:56.429660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019790290000000002,"stat_sum":{"num_bytes":17824784,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":4363,"num_read_kb":13988,"num_write":97,"num_write_kb":971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":393518577,"num_objects":272,"num_object_clones":45,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":43,"num_read":43070,"num_read_kb":222690,"num_write":2605,"num_write_kb":148007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":22701561,"num_keys_recovered":75,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":158232576,"data_stored":224839089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2223,"ondisk_log_size":2223,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56804,"kb_used_data":25612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94315036,"statfs":{"total":96636764160,"available":96578596864,"internally_reserved":0,"allocated":26226688,"data_stored":42238584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63772,"kb_used_data":32580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94308068,"statfs":{"total":96636764160,"available":96571461632,"internally_reserved":0,"allocated":33361920,"data_stored":45386039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39960,"kb_used_data":8768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331880,"statfs":{"total":96636764160,"available":96595845120,"internally_reserved":0,"allocated":8978432,"data_stored":8452328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66844,"kb_used_data":35656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94304996,"statfs":{"total":96636764160,"available":96568315904,"internally_reserved":0,"allocated":36511744,"data_stored":53986363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":55834574862,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607567,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56412,"kb_used_data":25220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94315428,"statfs":{"total":96636764160,"available":96578998272,"internally_reserved":0,"allocated":25825280,"data_stored":37505831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28000256,"data_stored":36273087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":35360768,"data_stored":53181603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":25284608,"data_stored":37296486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11108352,"data_stored":10892055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":32808960,"data_stored":45171666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":25669632,"data_stored":42024192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:18:52.129 INFO:tasks.thrashosds.thrasher:Setting 3.c to [1, 5] 2024-09-17T21:18:52.129 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.c', '1', '5'] 2024-09-17T21:18:52.129 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.c 1 5 2024-09-17T21:18:52.157 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:52.153+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:52.184 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:52.184 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:52.201 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:18:52.257 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:52.255+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:52.314 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-17T21:18:52.348 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:52.348 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:52.364 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:18:52.376 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:52.375+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:52.460 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:52.457+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:52.495 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:52.495 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:52.511 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:18:52.561 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:52.559+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:52.662 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:52.661+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:52.687 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:52.687 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:52.703 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:18:52.790 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:52.789+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:52.864 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:52.861+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:52.885 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:52.885 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:52.900 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:18:52.965 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:52.963+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:53.013 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:18:53.066 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:53.063+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:53.067 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:53.067 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:53.081 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:18:53.166 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:18:53.167 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:18:53.167 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:53.163+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:53.183 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:18:53.183 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:18:53.197 INFO:teuthology.orchestra.run.smithi184.stdout:1047 read 0x586c200 thru 0x5871849 (0x564a bytes) 2024-09-17T21:18:53.199 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:18:53.199 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:18:53.200 INFO:teuthology.orchestra.run.smithi184.stdout:1048 read 0x4ebaa00 thru 0x4ebdc44 (0x3245 bytes) 2024-09-17T21:18:53.205 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:53.205 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:53.218 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:18:53.218 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:18:53.220 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:18:53.239 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:18:53.239 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:18:53.251 INFO:teuthology.orchestra.run.smithi184.stdout:1049 write 0xb64ee00 thru 0xb6584d6 (0x96d7 bytes) 2024-09-17T21:18:53.253 INFO:teuthology.orchestra.run.smithi184.stdout:1052 trunc from 0xb658600 to 0xb35de00 2024-09-17T21:18:53.257 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:18:53.257 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:18:53.267 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:53.263+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:53.286 INFO:teuthology.orchestra.run.smithi184.stdout:1055 punch from 0x5cc6200 to 0x5cd6000, (0xfe00 bytes) 2024-09-17T21:18:53.288 INFO:teuthology.orchestra.run.smithi184.stdout:1061 write 0x236da00 thru 0x2375c5f (0x8260 bytes) 2024-09-17T21:18:53.295 INFO:teuthology.orchestra.run.smithi184.stdout:1062 read 0x685fa00 thru 0x6865177 (0x5778 bytes) 2024-09-17T21:18:53.323 INFO:teuthology.orchestra.run.smithi083.stderr:set 3.c pg_upmap mapping to [1,5] 2024-09-17T21:18:53.336 INFO:teuthology.orchestra.run.smithi184.stdout:1064 write 0xcdac600 thru 0xcdb74fc (0xaefd bytes) 2024-09-17T21:18:53.338 INFO:teuthology.orchestra.run.smithi184.stdout:1065 write 0x4fbf600 thru 0x4fc4e17 (0x5818 bytes) 2024-09-17T21:18:53.365 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:53.365 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:53.366 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:18:53.366 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:53.368 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:53.363+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:53.379 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:18:53.469 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:53.465+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:53.496 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:53.496 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:53.496 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:18:53.497 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:18:53.497 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:53.510 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:18:53.570 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:53.567+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:53.671 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:53.667+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:53.677 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:53.677 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:18:53.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:18:53.677 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:53.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.677 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:18:53.677 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:18:53.677 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.070812654999997, 2024-09-17T21:18:53.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.060866019000002, 2024-09-17T21:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.687 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:18:53.687 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:18:53.687 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 30.817715179, 2024-09-17T21:18:53.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:18:53.687 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:53.687 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:53.687 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:53.687 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:53.687 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:18:53.687 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:53.687 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:18:53.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.687 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:53.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:53.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:18:53.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:53.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:18:53.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:53.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:18:53.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:18:53.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:53.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:18:53.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:18:53.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:53.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:18:53.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:18:53.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:53.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:18:53.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:18:53.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:53.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:18:53.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:18:53.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:53.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:18:53.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:18:53.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:53.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:18:53.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:18:53.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:53.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:18:53.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:18:53.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:53.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:18:53.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:18:53.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:53.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:18:53.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:18:53.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:53.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:18:53.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:18:53.692 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.692 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:53.692 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.693 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:18:53.693 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:18:53.693 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 30.808069693, 2024-09-17T21:18:53.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:18:53.693 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:53.693 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:53.693 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:53.693 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:53.694 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:18:53.694 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:18:53.694 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:18:53.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.694 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:53.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:53.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:18:53.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:53.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:18:53.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:53.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:18:53.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:18:53.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:53.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:18:53.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:18:53.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:53.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:18:53.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:18:53.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:53.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:18:53.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:18:53.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:53.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:18:53.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:18:53.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:53.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:18:53.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:18:53.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:53.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:18:53.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:18:53.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:53.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:18:53.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:18:53.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:53.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:18:53.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:18:53.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:53.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:18:53.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:18:53.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:53.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:18:53.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:18:53.699 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.699 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:53.699 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.700 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:495 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 2228224~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:53.700 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.197359+0000", 2024-09-17T21:18:53.700 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.477643860000001, 2024-09-17T21:18:53.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021840167000000001, 2024-09-17T21:18:53.700 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:53.700 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:53.700 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:53.700 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:53.700 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:53.700 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:53.700 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 495 2024-09-17T21:18:53.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.700 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:53.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:53.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197359+0000", 2024-09-17T21:18:53.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:53.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197359+0000", 2024-09-17T21:18:53.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:53.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197360+0000", 2024-09-17T21:18:53.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0380000000000001e-06 2024-09-17T21:18:53.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:53.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197376+0000", 2024-09-17T21:18:53.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6147999999999999e-05 2024-09-17T21:18:53.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:53.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197392+0000", 2024-09-17T21:18:53.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6025999999999999e-05 2024-09-17T21:18:53.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:53.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.197401+0000", 2024-09-17T21:18:53.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6640000000000004e-06 2024-09-17T21:18:53.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:53.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.218539+0000", 2024-09-17T21:18:53.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021137808000000001 2024-09-17T21:18:53.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:53.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.218611+0000", 2024-09-17T21:18:53.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2471000000000005e-05 2024-09-17T21:18:53.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:53.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.219199+0000", 2024-09-17T21:18:53.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00058801199999999995 2024-09-17T21:18:53.704 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.704 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:53.704 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.705 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4498.0:548 3.e 3:71bb4574:::rbd_data.1152dfd5dee5.0000000000000002:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:53.705 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:26.200281+0000", 2024-09-17T21:18:53.705 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.474722201999999, 2024-09-17T21:18:53.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021975320999999999, 2024-09-17T21:18:53.705 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:53.705 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:53.705 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:53.705 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:53.705 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4498", 2024-09-17T21:18:53.705 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:53.705 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 548 2024-09-17T21:18:53.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.705 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:53.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200281+0000", 2024-09-17T21:18:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200281+0000", 2024-09-17T21:18:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200281+0000", 2024-09-17T21:18:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5100000000000001e-07 2024-09-17T21:18:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200283+0000", 2024-09-17T21:18:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4699999999999999e-06 2024-09-17T21:18:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200287+0000", 2024-09-17T21:18:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3939999999999998e-06 2024-09-17T21:18:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.200290+0000", 2024-09-17T21:18:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3160000000000001e-06 2024-09-17T21:18:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.221522+0000", 2024-09-17T21:18:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021231782000000001 2024-09-17T21:18:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.221624+0000", 2024-09-17T21:18:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101693 2024-09-17T21:18:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:26.222256+0000", 2024-09-17T21:18:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00063231500000000005 2024-09-17T21:18:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:467 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235571+0000", 2024-09-17T21:18:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.439431803000002, 2024-09-17T21:18:53.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021817462999999999, 2024-09-17T21:18:53.710 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:53.710 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:53.710 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:53.710 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:53.710 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:53.710 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:53.710 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 467 2024-09-17T21:18:53.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.710 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:53.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:53.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235571+0000", 2024-09-17T21:18:53.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:53.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235571+0000", 2024-09-17T21:18:53.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:53.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235573+0000", 2024-09-17T21:18:53.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5659999999999999e-06 2024-09-17T21:18:53.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:53.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235579+0000", 2024-09-17T21:18:53.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7729999999999996e-06 2024-09-17T21:18:53.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:53.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235598+0000", 2024-09-17T21:18:53.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9106999999999998e-05 2024-09-17T21:18:53.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:53.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235612+0000", 2024-09-17T21:18:53.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4032999999999999e-05 2024-09-17T21:18:53.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:53.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257023+0000", 2024-09-17T21:18:53.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021411619 2024-09-17T21:18:53.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:53.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257194+0000", 2024-09-17T21:18:53.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017076999999999999 2024-09-17T21:18:53.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:53.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257389+0000", 2024-09-17T21:18:53.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019459499999999999 2024-09-17T21:18:53.714 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.714 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:53.714 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.714 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:468 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:53.714 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235632+0000", 2024-09-17T21:18:53.714 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.439371273999999, 2024-09-17T21:18:53.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022180165000000002, 2024-09-17T21:18:53.715 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:53.715 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:53.715 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:53.715 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:53.715 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:53.715 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:53.715 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 468 2024-09-17T21:18:53.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.715 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:53.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:53.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235632+0000", 2024-09-17T21:18:53.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:53.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235632+0000", 2024-09-17T21:18:53.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:53.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235633+0000", 2024-09-17T21:18:53.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.446e-06 2024-09-17T21:18:53.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:53.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235639+0000", 2024-09-17T21:18:53.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6269999999999997e-06 2024-09-17T21:18:53.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:53.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235658+0000", 2024-09-17T21:18:53.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8686999999999999e-05 2024-09-17T21:18:53.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:53.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235672+0000", 2024-09-17T21:18:53.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.449e-05 2024-09-17T21:18:53.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:53.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257452+0000", 2024-09-17T21:18:53.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021779827000000002 2024-09-17T21:18:53.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:53.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257622+0000", 2024-09-17T21:18:53.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016976099999999999 2024-09-17T21:18:53.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:53.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257812+0000", 2024-09-17T21:18:53.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019032700000000001 2024-09-17T21:18:53.718 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.718 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:53.718 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.719 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:469 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:53.719 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235692+0000", 2024-09-17T21:18:53.719 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.439310943999999, 2024-09-17T21:18:53.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022615679, 2024-09-17T21:18:53.719 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:53.719 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:53.719 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:53.719 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:53.719 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:53.719 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:53.719 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 469 2024-09-17T21:18:53.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.720 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:53.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:53.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235692+0000", 2024-09-17T21:18:53.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:53.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235692+0000", 2024-09-17T21:18:53.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:53.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235694+0000", 2024-09-17T21:18:53.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.643e-06 2024-09-17T21:18:53.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:53.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235700+0000", 2024-09-17T21:18:53.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.733e-06 2024-09-17T21:18:53.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:53.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235718+0000", 2024-09-17T21:18:53.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8029999999999998e-05 2024-09-17T21:18:53.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:53.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235732+0000", 2024-09-17T21:18:53.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4239999999999999e-05 2024-09-17T21:18:53.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:53.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257935+0000", 2024-09-17T21:18:53.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022203514000000001 2024-09-17T21:18:53.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:53.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258107+0000", 2024-09-17T21:18:53.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017220700000000001 2024-09-17T21:18:53.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:53.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258308+0000", 2024-09-17T21:18:53.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020031199999999999 2024-09-17T21:18:53.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.723 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:53.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.724 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:470 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:53.724 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235752+0000", 2024-09-17T21:18:53.724 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.439251251999998, 2024-09-17T21:18:53.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023324234999999999, 2024-09-17T21:18:53.724 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:53.724 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:53.724 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:53.724 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:53.724 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:53.724 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:53.724 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 470 2024-09-17T21:18:53.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.725 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:53.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:53.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235752+0000", 2024-09-17T21:18:53.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:53.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235752+0000", 2024-09-17T21:18:53.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:53.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235753+0000", 2024-09-17T21:18:53.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.46e-06 2024-09-17T21:18:53.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:53.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235759+0000", 2024-09-17T21:18:53.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5829999999999997e-06 2024-09-17T21:18:53.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:53.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235777+0000", 2024-09-17T21:18:53.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8196999999999999e-05 2024-09-17T21:18:53.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:53.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235791+0000", 2024-09-17T21:18:53.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4406999999999999e-05 2024-09-17T21:18:53.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:53.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258351+0000", 2024-09-17T21:18:53.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022559157999999999 2024-09-17T21:18:53.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:53.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258491+0000", 2024-09-17T21:18:53.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014083300000000001 2024-09-17T21:18:53.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:53.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259076+0000", 2024-09-17T21:18:53.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00058459699999999996 2024-09-17T21:18:53.728 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.728 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:53.728 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.729 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:471 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:53.729 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235811+0000", 2024-09-17T21:18:53.729 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.439192032000001, 2024-09-17T21:18:53.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023519748, 2024-09-17T21:18:53.729 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:53.729 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:53.729 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:53.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:53.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:53.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:53.729 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 471 2024-09-17T21:18:53.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.730 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:53.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:53.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235811+0000", 2024-09-17T21:18:53.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:53.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235811+0000", 2024-09-17T21:18:53.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:53.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235813+0000", 2024-09-17T21:18:53.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.57e-06 2024-09-17T21:18:53.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:53.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235818+0000", 2024-09-17T21:18:53.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.417e-06 2024-09-17T21:18:53.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:53.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235836+0000", 2024-09-17T21:18:53.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8023e-05 2024-09-17T21:18:53.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:53.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235850+0000", 2024-09-17T21:18:53.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3759e-05 2024-09-17T21:18:53.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:53.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259120+0000", 2024-09-17T21:18:53.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023270193000000002 2024-09-17T21:18:53.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:53.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259225+0000", 2024-09-17T21:18:53.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104614 2024-09-17T21:18:53.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:53.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259331+0000", 2024-09-17T21:18:53.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106172 2024-09-17T21:18:53.733 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.733 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:53.733 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.733 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:53.733 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:18:53.733 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.43913366, 2024-09-17T21:18:53.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:18:53.734 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:53.734 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:53.734 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:53.734 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:53.734 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:53.734 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:53.734 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:18:53.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.734 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:53.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:53.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:18:53.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:53.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:18:53.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:53.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:18:53.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:18:53.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:53.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:18:53.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:18:53.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:53.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:18:53.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:18:53.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:53.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:18:53.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:18:53.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:53.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:18:53.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:18:53.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:53.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:18:53.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:18:53.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:53.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:18:53.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:18:53.737 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.737 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:53.737 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.738 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:558 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:53.738 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239154+0000", 2024-09-17T21:18:53.738 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.435848778, 2024-09-17T21:18:53.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021884532000000002, 2024-09-17T21:18:53.738 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:53.738 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:53.738 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:53.738 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:53.738 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:53.738 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:53.738 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 558 2024-09-17T21:18:53.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.739 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:53.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:53.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239154+0000", 2024-09-17T21:18:53.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:53.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239154+0000", 2024-09-17T21:18:53.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:53.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239156+0000", 2024-09-17T21:18:53.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4670000000000001e-06 2024-09-17T21:18:53.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:53.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239162+0000", 2024-09-17T21:18:53.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2829999999999998e-06 2024-09-17T21:18:53.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:53.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239181+0000", 2024-09-17T21:18:53.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8960000000000001e-05 2024-09-17T21:18:53.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:53.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239196+0000", 2024-09-17T21:18:53.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4523e-05 2024-09-17T21:18:53.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:53.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258757+0000", 2024-09-17T21:18:53.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019561775 2024-09-17T21:18:53.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:53.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258880+0000", 2024-09-17T21:18:53.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000123013 2024-09-17T21:18:53.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:53.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261039+0000", 2024-09-17T21:18:53.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021585110000000001 2024-09-17T21:18:53.742 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.742 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:53.742 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.742 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:559 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2162688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:53.742 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239215+0000", 2024-09-17T21:18:53.742 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.435787605000002, 2024-09-17T21:18:53.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022314902000000001, 2024-09-17T21:18:53.742 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:53.743 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:53.743 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:53.743 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:53.743 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:53.743 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:53.743 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 559 2024-09-17T21:18:53.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.743 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:53.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:53.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239215+0000", 2024-09-17T21:18:53.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:53.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239215+0000", 2024-09-17T21:18:53.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:53.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239217+0000", 2024-09-17T21:18:53.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:18:53.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:53.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239223+0000", 2024-09-17T21:18:53.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6099999999999997e-06 2024-09-17T21:18:53.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:53.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239241+0000", 2024-09-17T21:18:53.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8576e-05 2024-09-17T21:18:53.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:53.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239256+0000", 2024-09-17T21:18:53.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4397e-05 2024-09-17T21:18:53.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:53.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261086+0000", 2024-09-17T21:18:53.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021830374 2024-09-17T21:18:53.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:53.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261203+0000", 2024-09-17T21:18:53.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000117192 2024-09-17T21:18:53.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:53.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261530+0000", 2024-09-17T21:18:53.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032712300000000002 2024-09-17T21:18:53.746 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.746 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:53.746 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.747 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:560 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2293760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:53.747 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239276+0000", 2024-09-17T21:18:53.747 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.435727245999999, 2024-09-17T21:18:53.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022636927000000001, 2024-09-17T21:18:53.747 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:53.747 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:53.747 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:53.747 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:53.747 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:53.747 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:53.747 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 560 2024-09-17T21:18:53.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.748 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:53.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:53.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239276+0000", 2024-09-17T21:18:53.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:53.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239276+0000", 2024-09-17T21:18:53.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:53.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239277+0000", 2024-09-17T21:18:53.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4699999999999999e-06 2024-09-17T21:18:53.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:53.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239283+0000", 2024-09-17T21:18:53.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5899999999999998e-06 2024-09-17T21:18:53.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:53.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239301+0000", 2024-09-17T21:18:53.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8593999999999998e-05 2024-09-17T21:18:53.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:53.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239316+0000", 2024-09-17T21:18:53.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.428e-05 2024-09-17T21:18:53.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:53.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261592+0000", 2024-09-17T21:18:53.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022276118000000001 2024-09-17T21:18:53.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:53.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261733+0000", 2024-09-17T21:18:53.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014099199999999999 2024-09-17T21:18:53.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:53.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261913+0000", 2024-09-17T21:18:53.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017988300000000001 2024-09-17T21:18:53.751 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.751 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:53.751 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.751 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:561 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:53.751 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239336+0000", 2024-09-17T21:18:53.751 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.435667419000001, 2024-09-17T21:18:53.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02290615, 2024-09-17T21:18:53.751 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:53.752 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:53.752 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:53.752 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:53.752 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:53.752 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:53.752 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:18:53.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.752 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:53.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:53.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239336+0000", 2024-09-17T21:18:53.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:53.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239336+0000", 2024-09-17T21:18:53.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:53.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239337+0000", 2024-09-17T21:18:53.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3969999999999999e-06 2024-09-17T21:18:53.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:53.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239343+0000", 2024-09-17T21:18:53.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.699e-06 2024-09-17T21:18:53.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:53.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239361+0000", 2024-09-17T21:18:53.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8107000000000001e-05 2024-09-17T21:18:53.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:53.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239379+0000", 2024-09-17T21:18:53.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7972999999999999e-05 2024-09-17T21:18:53.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:53.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261956+0000", 2024-09-17T21:18:53.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022577105 2024-09-17T21:18:53.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:53.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262082+0000", 2024-09-17T21:18:53.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012621600000000001 2024-09-17T21:18:53.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:53.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262242+0000", 2024-09-17T21:18:53.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000159653 2024-09-17T21:18:53.755 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.755 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:53.755 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.756 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:562 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:53.756 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239398+0000", 2024-09-17T21:18:53.756 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.435604943000001, 2024-09-17T21:18:53.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023207746000000001, 2024-09-17T21:18:53.756 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:53.756 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:53.756 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:53.756 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:53.756 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:53.756 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:53.756 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:18:53.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.757 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:53.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:53.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239398+0000", 2024-09-17T21:18:53.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:53.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239398+0000", 2024-09-17T21:18:53.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:53.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239400+0000", 2024-09-17T21:18:53.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.55e-06 2024-09-17T21:18:53.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:53.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239427+0000", 2024-09-17T21:18:53.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.745e-05 2024-09-17T21:18:53.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:53.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239447+0000", 2024-09-17T21:18:53.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9715999999999999e-05 2024-09-17T21:18:53.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:53.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239468+0000", 2024-09-17T21:18:53.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0894e-05 2024-09-17T21:18:53.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:53.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262285+0000", 2024-09-17T21:18:53.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022817456999999999 2024-09-17T21:18:53.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:53.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262436+0000", 2024-09-17T21:18:53.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015082999999999999 2024-09-17T21:18:53.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:53.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262606+0000", 2024-09-17T21:18:53.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016984899999999999 2024-09-17T21:18:53.761 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.761 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:53.761 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.761 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:563 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:53.761 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239497+0000", 2024-09-17T21:18:53.761 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.435506534000002, 2024-09-17T21:18:53.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023494231000000001, 2024-09-17T21:18:53.761 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:53.761 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:53.761 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:53.761 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:53.762 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:53.762 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:53.762 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 563 2024-09-17T21:18:53.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.762 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:53.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:53.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239497+0000", 2024-09-17T21:18:53.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:53.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239497+0000", 2024-09-17T21:18:53.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:53.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239498+0000", 2024-09-17T21:18:53.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3969999999999999e-06 2024-09-17T21:18:53.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:53.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239539+0000", 2024-09-17T21:18:53.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0669e-05 2024-09-17T21:18:53.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:53.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239559+0000", 2024-09-17T21:18:53.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0353999999999999e-05 2024-09-17T21:18:53.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:53.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239575+0000", 2024-09-17T21:18:53.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5642999999999999e-05 2024-09-17T21:18:53.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:53.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262656+0000", 2024-09-17T21:18:53.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023081005000000002 2024-09-17T21:18:53.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:53.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262769+0000", 2024-09-17T21:18:53.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113616 2024-09-17T21:18:53.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:53.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.262991+0000", 2024-09-17T21:18:53.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000221547 2024-09-17T21:18:53.765 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.765 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:53.765 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.765 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:564 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:53.766 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239598+0000", 2024-09-17T21:18:53.766 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.435405213999999, 2024-09-17T21:18:53.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023713695999999999, 2024-09-17T21:18:53.766 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:53.766 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:53.766 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:53.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:53.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:53.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:53.766 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 564 2024-09-17T21:18:53.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.766 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:53.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:53.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239598+0000", 2024-09-17T21:18:53.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:53.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239598+0000", 2024-09-17T21:18:53.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:53.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239600+0000", 2024-09-17T21:18:53.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:18:53.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:53.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239606+0000", 2024-09-17T21:18:53.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2029999999999997e-06 2024-09-17T21:18:53.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:53.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239623+0000", 2024-09-17T21:18:53.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7669999999999999e-05 2024-09-17T21:18:53.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:53.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239638+0000", 2024-09-17T21:18:53.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4416999999999999e-05 2024-09-17T21:18:53.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:53.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263063+0000", 2024-09-17T21:18:53.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023424932999999998 2024-09-17T21:18:53.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:53.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263182+0000", 2024-09-17T21:18:53.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011914899999999999 2024-09-17T21:18:53.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:53.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263312+0000", 2024-09-17T21:18:53.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012956799999999999 2024-09-17T21:18:53.770 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.770 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:53.770 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.770 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:565 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:53.770 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:18:53.770 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.435344695000001, 2024-09-17T21:18:53.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024193392000000001, 2024-09-17T21:18:53.770 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:53.770 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:53.770 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:53.771 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:53.771 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:53.771 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:53.771 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 565 2024-09-17T21:18:53.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.771 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:53.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:53.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:18:53.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:53.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:18:53.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:53.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:53.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239660+0000", 2024-09-17T21:18:53.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:18:53.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:53.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239666+0000", 2024-09-17T21:18:53.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8699999999999997e-06 2024-09-17T21:18:53.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:53.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239688+0000", 2024-09-17T21:18:53.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.213e-05 2024-09-17T21:18:53.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:53.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239703+0000", 2024-09-17T21:18:53.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4813e-05 2024-09-17T21:18:53.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:53.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263427+0000", 2024-09-17T21:18:53.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023724630999999999 2024-09-17T21:18:53.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:53.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263614+0000", 2024-09-17T21:18:53.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018664500000000001 2024-09-17T21:18:53.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:53.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:53.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:53.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263852+0000", 2024-09-17T21:18:53.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023793999999999999 2024-09-17T21:18:53.774 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.774 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:53.774 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.774 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:53.774 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:53.775 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:53.776 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:53.773+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:53.776 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:18:53.877 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:53.873+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:53.910 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:53.910 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:53.910 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:53.910 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:53.926 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:18:53.978 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:53.977+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:54.048 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:54.048 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:54.049 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:54.049 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:54.049 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:54.062 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:18:54.079 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:54.077+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:54.179 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:54.175+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:54.180 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:54.180 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:18:54.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:18:54.181 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:18:54.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.181 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:389 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.181 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.194936+0000", 2024-09-17T21:18:54.181 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.983678674, 2024-09-17T21:18:54.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041176510999999999, 2024-09-17T21:18:54.181 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.181 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.181 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.181 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.181 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.182 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.182 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 389 2024-09-17T21:18:54.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.182 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.182 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.182 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194936+0000", 2024-09-17T21:18:54.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.182 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.182 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194936+0000", 2024-09-17T21:18:54.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194937+0000", 2024-09-17T21:18:54.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6270000000000001e-06 2024-09-17T21:18:54.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194944+0000", 2024-09-17T21:18:54.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5729999999999996e-06 2024-09-17T21:18:54.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194961+0000", 2024-09-17T21:18:54.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7623000000000001e-05 2024-09-17T21:18:54.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194974+0000", 2024-09-17T21:18:54.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2989999999999999e-05 2024-09-17T21:18:54.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.235848+0000", 2024-09-17T21:18:54.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.040874099999999997 2024-09-17T21:18:54.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.235971+0000", 2024-09-17T21:18:54.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012254400000000001 2024-09-17T21:18:54.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236112+0000", 2024-09-17T21:18:54.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014105399999999999 2024-09-17T21:18:54.185 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.185 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.185 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.186 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:406 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 2359296~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.186 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:18:54.186 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.983559868, 2024-09-17T21:18:54.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042448018999999997, 2024-09-17T21:18:54.186 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.186 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.186 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.186 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.186 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.186 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.186 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 406 2024-09-17T21:18:54.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.186 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:18:54.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:18:54.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195056+0000", 2024-09-17T21:18:54.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2729999999999999e-06 2024-09-17T21:18:54.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195060+0000", 2024-09-17T21:18:54.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5399999999999997e-06 2024-09-17T21:18:54.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195076+0000", 2024-09-17T21:18:54.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6217000000000001e-05 2024-09-17T21:18:54.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195093+0000", 2024-09-17T21:18:54.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.683e-05 2024-09-17T21:18:54.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236217+0000", 2024-09-17T21:18:54.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041124102000000003 2024-09-17T21:18:54.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236339+0000", 2024-09-17T21:18:54.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121876 2024-09-17T21:18:54.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.190 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.190 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237502+0000", 2024-09-17T21:18:54.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011631809999999999 2024-09-17T21:18:54.190 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.190 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.190 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.190 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:416 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2228224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.190 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:18:54.190 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.983061842000001, 2024-09-17T21:18:54.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042361990000000002, 2024-09-17T21:18:54.191 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.191 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.191 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.191 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.191 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.191 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.191 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 416 2024-09-17T21:18:54.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.191 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:18:54.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:18:54.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195554+0000", 2024-09-17T21:18:54.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9300000000000002e-06 2024-09-17T21:18:54.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195563+0000", 2024-09-17T21:18:54.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2770000000000005e-06 2024-09-17T21:18:54.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195586+0000", 2024-09-17T21:18:54.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3889000000000001e-05 2024-09-17T21:18:54.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195603+0000", 2024-09-17T21:18:54.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6084000000000001e-05 2024-09-17T21:18:54.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237563+0000", 2024-09-17T21:18:54.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041960856999999997 2024-09-17T21:18:54.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237695+0000", 2024-09-17T21:18:54.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013144400000000001 2024-09-17T21:18:54.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237914+0000", 2024-09-17T21:18:54.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000219509 2024-09-17T21:18:54.195 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.195 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.195 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.195 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:439 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.195 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:18:54.195 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.982497940000002, 2024-09-17T21:18:54.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043053811999999997, 2024-09-17T21:18:54.196 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.196 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.196 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.196 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.196 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.196 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.196 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 439 2024-09-17T21:18:54.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.196 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:18:54.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:18:54.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196118+0000", 2024-09-17T21:18:54.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3969999999999999e-06 2024-09-17T21:18:54.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196157+0000", 2024-09-17T21:18:54.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9826999999999997e-05 2024-09-17T21:18:54.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196179+0000", 2024-09-17T21:18:54.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1756e-05 2024-09-17T21:18:54.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196194+0000", 2024-09-17T21:18:54.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4707e-05 2024-09-17T21:18:54.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237980+0000", 2024-09-17T21:18:54.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041786219999999999 2024-09-17T21:18:54.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.238065+0000", 2024-09-17T21:18:54.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4560000000000004e-05 2024-09-17T21:18:54.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239170+0000", 2024-09-17T21:18:54.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001105345 2024-09-17T21:18:54.200 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.200 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.200 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.200 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:440 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.200 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:18:54.200 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.982395437000001, 2024-09-17T21:18:54.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043890039999999998, 2024-09-17T21:18:54.200 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.200 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.200 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.200 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.201 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.201 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.201 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 440 2024-09-17T21:18:54.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.201 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:18:54.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:18:54.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196220+0000", 2024-09-17T21:18:54.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.254e-06 2024-09-17T21:18:54.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196225+0000", 2024-09-17T21:18:54.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7829999999999998e-06 2024-09-17T21:18:54.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196241+0000", 2024-09-17T21:18:54.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6322999999999999e-05 2024-09-17T21:18:54.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196258+0000", 2024-09-17T21:18:54.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7056000000000001e-05 2024-09-17T21:18:54.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239229+0000", 2024-09-17T21:18:54.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042971058999999999 2024-09-17T21:18:54.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239403+0000", 2024-09-17T21:18:54.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017399200000000001 2024-09-17T21:18:54.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240109+0000", 2024-09-17T21:18:54.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00070557300000000003 2024-09-17T21:18:54.204 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.204 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.204 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.205 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:450 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.205 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:18:54.205 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.982034486, 2024-09-17T21:18:54.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043994521000000002, 2024-09-17T21:18:54.205 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.205 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.205 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.205 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.205 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.205 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.205 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 450 2024-09-17T21:18:54.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.205 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:18:54.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:18:54.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196581+0000", 2024-09-17T21:18:54.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4500000000000001e-06 2024-09-17T21:18:54.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196588+0000", 2024-09-17T21:18:54.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7530000000000004e-06 2024-09-17T21:18:54.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196607+0000", 2024-09-17T21:18:54.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8987e-05 2024-09-17T21:18:54.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196620+0000", 2024-09-17T21:18:54.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3589e-05 2024-09-17T21:18:54.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240225+0000", 2024-09-17T21:18:54.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043604878999999999 2024-09-17T21:18:54.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240377+0000", 2024-09-17T21:18:54.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015146699999999999 2024-09-17T21:18:54.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240574+0000", 2024-09-17T21:18:54.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019739599999999999 2024-09-17T21:18:54.209 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.209 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.209 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.210 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:464 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.210 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:18:54.210 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.981784064999999, 2024-09-17T21:18:54.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044651692999999999, 2024-09-17T21:18:54.210 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.210 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.210 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.210 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.210 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.210 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.210 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 464 2024-09-17T21:18:54.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.211 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:18:54.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:18:54.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196838+0000", 2024-09-17T21:18:54.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4970000000000004e-06 2024-09-17T21:18:54.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196859+0000", 2024-09-17T21:18:54.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1166e-05 2024-09-17T21:18:54.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196879+0000", 2024-09-17T21:18:54.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.995e-05 2024-09-17T21:18:54.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196896+0000", 2024-09-17T21:18:54.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6929999999999999e-05 2024-09-17T21:18:54.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240617+0000", 2024-09-17T21:18:54.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043721784999999999 2024-09-17T21:18:54.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240702+0000", 2024-09-17T21:18:54.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4060999999999994e-05 2024-09-17T21:18:54.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241482+0000", 2024-09-17T21:18:54.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000780304 2024-09-17T21:18:54.214 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.214 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.214 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.214 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:475 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.214 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:18:54.215 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.981514933, 2024-09-17T21:18:54.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044661981000000003, 2024-09-17T21:18:54.215 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.215 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.215 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.215 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.215 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.215 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.215 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 475 2024-09-17T21:18:54.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.215 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:18:54.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:18:54.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197100+0000", 2024-09-17T21:18:54.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.206e-06 2024-09-17T21:18:54.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197105+0000", 2024-09-17T21:18:54.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.527e-06 2024-09-17T21:18:54.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197124+0000", 2024-09-17T21:18:54.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:18:54.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197136+0000", 2024-09-17T21:18:54.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2857000000000001e-05 2024-09-17T21:18:54.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241539+0000", 2024-09-17T21:18:54.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044402077999999998 2024-09-17T21:18:54.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241653+0000", 2024-09-17T21:18:54.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114931 2024-09-17T21:18:54.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241761+0000", 2024-09-17T21:18:54.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107753 2024-09-17T21:18:54.219 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.219 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.219 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.219 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:476 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.219 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:18:54.219 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.981461108000001, 2024-09-17T21:18:54.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045038976000000001, 2024-09-17T21:18:54.219 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.219 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.219 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.219 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.220 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.220 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.220 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 476 2024-09-17T21:18:54.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.220 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:18:54.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:18:54.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197154+0000", 2024-09-17T21:18:54.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.127e-06 2024-09-17T21:18:54.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197159+0000", 2024-09-17T21:18:54.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4569999999999998e-06 2024-09-17T21:18:54.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197174+0000", 2024-09-17T21:18:54.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5173e-05 2024-09-17T21:18:54.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197186+0000", 2024-09-17T21:18:54.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:18:54.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241863+0000", 2024-09-17T21:18:54.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044676826000000003 2024-09-17T21:18:54.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241998+0000", 2024-09-17T21:18:54.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013476100000000001 2024-09-17T21:18:54.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242192+0000", 2024-09-17T21:18:54.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019436900000000001 2024-09-17T21:18:54.223 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.223 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.223 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.223 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:494 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.223 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:18:54.224 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.981285925000002, 2024-09-17T21:18:54.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046244995999999997, 2024-09-17T21:18:54.224 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.224 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.224 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.224 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.224 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.224 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.224 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 494 2024-09-17T21:18:54.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.224 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:18:54.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:18:54.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197329+0000", 2024-09-17T21:18:54.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.173e-06 2024-09-17T21:18:54.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197334+0000", 2024-09-17T21:18:54.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0499999999999999e-06 2024-09-17T21:18:54.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197352+0000", 2024-09-17T21:18:54.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7133e-05 2024-09-17T21:18:54.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197367+0000", 2024-09-17T21:18:54.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5123000000000001e-05 2024-09-17T21:18:54.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:18:54.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044989829000000002 2024-09-17T21:18:54.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242526+0000", 2024-09-17T21:18:54.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016968199999999999 2024-09-17T21:18:54.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243573+0000", 2024-09-17T21:18:54.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010470060000000001 2024-09-17T21:18:54.228 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.228 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.229 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.229 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:502 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.229 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:18:54.229 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.979052266, 2024-09-17T21:18:54.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044346086999999999, 2024-09-17T21:18:54.229 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.229 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.229 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.229 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.229 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.229 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.229 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 502 2024-09-17T21:18:54.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.230 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:18:54.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:18:54.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199563+0000", 2024-09-17T21:18:54.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2929999999999999e-06 2024-09-17T21:18:54.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199569+0000", 2024-09-17T21:18:54.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5330000000000002e-06 2024-09-17T21:18:54.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199614+0000", 2024-09-17T21:18:54.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4793000000000003e-05 2024-09-17T21:18:54.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199628+0000", 2024-09-17T21:18:54.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4037e-05 2024-09-17T21:18:54.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243665+0000", 2024-09-17T21:18:54.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044037106999999999 2024-09-17T21:18:54.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243767+0000", 2024-09-17T21:18:54.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010279499999999999 2024-09-17T21:18:54.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243908+0000", 2024-09-17T21:18:54.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140529 2024-09-17T21:18:54.233 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.233 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.233 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.233 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:523 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.233 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:18:54.233 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.978606113000001, 2024-09-17T21:18:54.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044789645000000003, 2024-09-17T21:18:54.234 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.234 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.234 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.234 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.234 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.234 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.234 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 523 2024-09-17T21:18:54.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.234 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:18:54.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:18:54.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:18:54.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9e-06 2024-09-17T21:18:54.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200021+0000", 2024-09-17T21:18:54.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9499999999999998e-06 2024-09-17T21:18:54.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200038+0000", 2024-09-17T21:18:54.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.73e-05 2024-09-17T21:18:54.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200051+0000", 2024-09-17T21:18:54.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.322e-05 2024-09-17T21:18:54.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243987+0000", 2024-09-17T21:18:54.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043935386999999999 2024-09-17T21:18:54.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244136+0000", 2024-09-17T21:18:54.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014868300000000001 2024-09-17T21:18:54.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244798+0000", 2024-09-17T21:18:54.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00066220500000000002 2024-09-17T21:18:54.237 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.238 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.238 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.238 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:524 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.238 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:18:54.238 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.978539897000001, 2024-09-17T21:18:54.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045737414999999997, 2024-09-17T21:18:54.238 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.238 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.238 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.238 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.238 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.238 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.239 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 524 2024-09-17T21:18:54.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.239 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:18:54.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:18:54.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200076+0000", 2024-09-17T21:18:54.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:18:54.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200081+0000", 2024-09-17T21:18:54.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7030000000000003e-06 2024-09-17T21:18:54.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200099+0000", 2024-09-17T21:18:54.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7836999999999999e-05 2024-09-17T21:18:54.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200113+0000", 2024-09-17T21:18:54.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3570000000000001e-05 2024-09-17T21:18:54.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244866+0000", 2024-09-17T21:18:54.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044753573999999997 2024-09-17T21:18:54.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244972+0000", 2024-09-17T21:18:54.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106111 2024-09-17T21:18:54.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245812+0000", 2024-09-17T21:18:54.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083923999999999999 2024-09-17T21:18:54.243 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.243 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.243 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.243 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:525 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3670016~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.243 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:18:54.243 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.978388514999999, 2024-09-17T21:18:54.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046645697, 2024-09-17T21:18:54.243 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.243 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.243 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.243 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.243 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.244 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.244 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 525 2024-09-17T21:18:54.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.244 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.244 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.244 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:18:54.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.244 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.244 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:18:54.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:18:54.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:18:54.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200234+0000", 2024-09-17T21:18:54.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:18:54.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200254+0000", 2024-09-17T21:18:54.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9959999999999999e-05 2024-09-17T21:18:54.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200293+0000", 2024-09-17T21:18:54.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9755999999999997e-05 2024-09-17T21:18:54.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245871+0000", 2024-09-17T21:18:54.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045577871999999998 2024-09-17T21:18:54.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246030+0000", 2024-09-17T21:18:54.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158975 2024-09-17T21:18:54.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246871+0000", 2024-09-17T21:18:54.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084111699999999995 2024-09-17T21:18:54.247 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.247 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.247 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.247 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.247 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:18:54.247 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.97814284, 2024-09-17T21:18:54.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:18:54.248 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.248 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.248 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.248 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.248 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.248 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.248 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:18:54.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.248 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:18:54.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:18:54.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:18:54.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:18:54.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:18:54.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:18:54.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:18:54.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:18:54.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:18:54.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:18:54.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:18:54.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:18:54.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:18:54.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:18:54.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:18:54.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:18:54.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.252 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.252 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.252 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:18:54.252 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.977964157999999, 2024-09-17T21:18:54.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:18:54.252 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.252 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.252 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.252 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.252 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.253 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.253 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:18:54.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.253 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:18:54.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:18:54.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:18:54.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:18:54.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:18:54.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:18:54.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:18:54.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:18:54.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:18:54.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:18:54.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:18:54.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:18:54.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:18:54.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:18:54.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:18:54.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:18:54.256 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.256 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.256 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.256 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.257 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:18:54.257 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.977904885000001, 2024-09-17T21:18:54.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:18:54.257 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.257 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.257 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.257 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.257 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.257 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.258 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:18:54.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.258 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:18:54.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:18:54.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:18:54.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:18:54.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:18:54.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:18:54.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:18:54.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:18:54.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:18:54.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:18:54.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:18:54.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:18:54.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:18:54.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:18:54.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:18:54.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:18:54.261 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.261 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.261 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.261 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.262 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:18:54.262 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.977762689999999, 2024-09-17T21:18:54.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:18:54.262 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.262 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.262 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.262 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.262 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.262 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.262 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:18:54.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.262 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:18:54.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:18:54.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:18:54.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:18:54.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:18:54.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:18:54.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:18:54.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:18:54.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:18:54.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:18:54.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:18:54.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:18:54.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:18:54.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:18:54.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:18:54.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:18:54.266 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.266 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.266 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.267 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.267 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:18:54.267 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.977706767000001, 2024-09-17T21:18:54.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:18:54.267 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.267 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.267 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.267 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.267 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.267 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.267 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:18:54.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.268 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:18:54.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:18:54.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:18:54.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:18:54.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:18:54.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:18:54.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:18:54.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:18:54.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:18:54.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:18:54.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:18:54.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:18:54.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:18:54.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:18:54.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:18:54.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:18:54.271 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.271 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.271 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.272 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.272 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:18:54.272 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.977653149999998, 2024-09-17T21:18:54.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:18:54.272 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.272 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.272 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.272 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.272 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.272 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.272 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:18:54.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.273 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:18:54.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:18:54.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:18:54.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:18:54.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:18:54.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:18:54.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:18:54.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:18:54.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:18:54.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:18:54.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:18:54.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:18:54.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:18:54.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:18:54.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:18:54.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:18:54.277 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.277 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.277 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.277 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.277 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.277 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:54.278 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:18:54.321 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:54.319+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:54.333 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:18:54.366 INFO:teuthology.orchestra.run.smithi184.stdout:1066 write 0xe766c00 thru 0xe770b3d (0x9f3e bytes) 2024-09-17T21:18:54.382 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:54.381+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:54.400 INFO:teuthology.orchestra.run.smithi184.stdout:1067 trunc from 0xe770c00 to 0x5854e00 2024-09-17T21:18:54.456 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:54.456 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:54.456 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:54.456 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:54.471 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:18:54.500 INFO:teuthology.orchestra.run.smithi184.stdout:1070 trunc from 0x5854e00 to 0xbf27a00 2024-09-17T21:18:54.516 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:54.515+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:54.517 INFO:teuthology.orchestra.run.smithi184.stdout:1074 read 0x22af400 thru 0x22b2b59 (0x375a bytes) 2024-09-17T21:18:54.519 INFO:teuthology.orchestra.run.smithi184.stdout:1075 read 0xb992e00 thru 0xb994688 (0x1889 bytes) 2024-09-17T21:18:54.519 INFO:teuthology.orchestra.run.smithi184.stdout:1077 read 0x4afdc00 thru 0x4b0a9ff (0xce00 bytes) 2024-09-17T21:18:54.521 INFO:teuthology.orchestra.run.smithi184.stdout:1080 write 0x9afca00 thru 0x9b05f54 (0x9555 bytes) 2024-09-17T21:18:54.522 INFO:teuthology.orchestra.run.smithi184.stdout:1081 write 0xa2cc600 thru 0xa2ce4f8 (0x1ef9 bytes) 2024-09-17T21:18:54.522 INFO:teuthology.orchestra.run.smithi184.stdout:1082 read 0x7ce4a00 thru 0x7ce543c (0xa3d bytes) 2024-09-17T21:18:54.522 INFO:teuthology.orchestra.run.smithi184.stdout:1083 punch from 0xb7c4e00 to 0xb7cdc00, (0x8e00 bytes) 2024-09-17T21:18:54.529 INFO:teuthology.orchestra.run.smithi184.stdout:1086 trunc from 0xbf27a00 to 0x9b44e00 2024-09-17T21:18:54.563 INFO:teuthology.orchestra.run.smithi184.stdout:1088 trunc from 0x9b44e00 to 0xaaab400 2024-09-17T21:18:54.579 INFO:teuthology.orchestra.run.smithi184.stdout:1089 write 0x8396600 thru 0x839ac6d (0x466e bytes) 2024-09-17T21:18:54.579 INFO:teuthology.orchestra.run.smithi184.stdout:1091 read 0x456600 thru 0x45dba1 (0x75a2 bytes) 2024-09-17T21:18:54.583 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:54.579+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:54.589 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:54.589 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:54.589 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:54.589 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:54.590 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:54.600 INFO:teuthology.orchestra.run.smithi184.stdout:1092 write 0xab0c600 thru 0xab10eac (0x48ad bytes) 2024-09-17T21:18:54.601 INFO:teuthology.orchestra.run.smithi184.stdout:1093 read 0x65e2a00 thru 0x65e9961 (0x6f62 bytes) 2024-09-17T21:18:54.601 INFO:teuthology.orchestra.run.smithi184.stdout:1094 write 0x6704200 thru 0x6711dd1 (0xdbd2 bytes) 2024-09-17T21:18:54.602 INFO:teuthology.orchestra.run.smithi184.stdout:1096 read 0x7c06200 thru 0x7c120d5 (0xbed6 bytes) 2024-09-17T21:18:54.602 INFO:teuthology.orchestra.run.smithi184.stdout:1098 write 0x4d5ec00 thru 0x4d685c0 (0x99c1 bytes) 2024-09-17T21:18:54.604 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:18:54.605 INFO:teuthology.orchestra.run.smithi184.stdout:1101 read 0x909a00 thru 0x90a51f (0xb20 bytes) 2024-09-17T21:18:54.605 INFO:teuthology.orchestra.run.smithi184.stdout:1102 write 0x7df7600 thru 0x7df7fa8 (0x9a9 bytes) 2024-09-17T21:18:54.605 INFO:teuthology.orchestra.run.smithi184.stdout:1103 read 0x3858800 thru 0x385ea2f (0x6230 bytes) 2024-09-17T21:18:54.609 INFO:teuthology.orchestra.run.smithi184.stdout:1104 trunc from 0xab11000 to 0xb953a00 2024-09-17T21:18:54.646 INFO:teuthology.orchestra.run.smithi184.stdout:1108 read 0x2b3b400 thru 0x2b4315a (0x7d5b bytes) 2024-09-17T21:18:54.649 INFO:teuthology.orchestra.run.smithi184.stdout:1109 read 0x8900e00 thru 0x890d02e (0xc22f bytes) 2024-09-17T21:18:54.650 INFO:teuthology.orchestra.run.smithi184.stdout:1110 trunc from 0xb953a00 to 0xb4ce200 2024-09-17T21:18:54.669 INFO:teuthology.orchestra.run.smithi184.stdout:1112 read 0x86f2400 thru 0x86ffbe0 (0xd7e1 bytes) 2024-09-17T21:18:54.670 INFO:teuthology.orchestra.run.smithi184.stdout:1113 read 0x2eb8400 thru 0x2ec3d72 (0xb973 bytes) 2024-09-17T21:18:54.694 INFO:teuthology.orchestra.run.smithi184.stdout:1114 write 0xe5df400 thru 0xe5ea017 (0xac18 bytes) 2024-09-17T21:18:54.696 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:54.695+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:54.696 INFO:teuthology.orchestra.run.smithi184.stdout:1115 read 0xb1ed600 thru 0xb1f3538 (0x5f39 bytes) 2024-09-17T21:18:54.697 INFO:teuthology.orchestra.run.smithi184.stdout:1117 punch from 0x763cc00 to 0x7646a00, (0x9e00 bytes) 2024-09-17T21:18:54.698 INFO:teuthology.orchestra.run.smithi184.stdout:1119 punch from 0x6f22400 to 0x6f27e00, (0x5a00 bytes) 2024-09-17T21:18:54.698 INFO:teuthology.orchestra.run.smithi184.stdout:1121 write 0xc10f400 thru 0xc11c9ce (0xd5cf bytes) 2024-09-17T21:18:54.707 INFO:teuthology.orchestra.run.smithi184.stdout:1122 trunc from 0xe5ea200 to 0xaaaf000 2024-09-17T21:18:54.744 INFO:teuthology.orchestra.run.smithi184.stdout:1123 write 0x2a59e00 thru 0x2a5b9fc (0x1bfd bytes) 2024-09-17T21:18:54.775 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:54.775 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:18:54.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:18:54.775 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:18:54.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.775 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:491 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.775 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:18:54.775 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.573206487, 2024-09-17T21:18:54.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041839013000000001, 2024-09-17T21:18:54.775 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 491 2024-09-17T21:18:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:18:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:18:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198315+0000", 2024-09-17T21:18:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6599999999999996e-07 2024-09-17T21:18:54.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198317+0000", 2024-09-17T21:18:54.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6639999999999998e-06 2024-09-17T21:18:54.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198327+0000", 2024-09-17T21:18:54.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5629999999999998e-06 2024-09-17T21:18:54.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198337+0000", 2024-09-17T21:18:54.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0667e-05 2024-09-17T21:18:54.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239793+0000", 2024-09-17T21:18:54.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041455519000000003 2024-09-17T21:18:54.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239929+0000", 2024-09-17T21:18:54.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135716 2024-09-17T21:18:54.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240153+0000", 2024-09-17T21:18:54.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022421799999999999 2024-09-17T21:18:54.780 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.780 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.780 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.780 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:492 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.780 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:18:54.780 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.573168117000002, 2024-09-17T21:18:54.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042947165000000002, 2024-09-17T21:18:54.781 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.781 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.781 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.781 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.781 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.781 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.781 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 492 2024-09-17T21:18:54.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.781 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:18:54.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:18:54.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198354+0000", 2024-09-17T21:18:54.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:18:54.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198357+0000", 2024-09-17T21:18:54.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6969999999999999e-06 2024-09-17T21:18:54.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198367+0000", 2024-09-17T21:18:54.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8430000000000005e-06 2024-09-17T21:18:54.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198374+0000", 2024-09-17T21:18:54.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9600000000000003e-06 2024-09-17T21:18:54.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240226+0000", 2024-09-17T21:18:54.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041852157000000001 2024-09-17T21:18:54.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240409+0000", 2024-09-17T21:18:54.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018244800000000001 2024-09-17T21:18:54.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241299+0000", 2024-09-17T21:18:54.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000890554 2024-09-17T21:18:54.784 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.784 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.785 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.785 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:493 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 4063232~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.785 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:18:54.785 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.573131030999999, 2024-09-17T21:18:54.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043897209, 2024-09-17T21:18:54.785 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.785 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.785 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.785 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.785 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.785 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.786 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 493 2024-09-17T21:18:54.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.786 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:18:54.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:18:54.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198391+0000", 2024-09-17T21:18:54.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4699999999999999e-06 2024-09-17T21:18:54.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198396+0000", 2024-09-17T21:18:54.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:18:54.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198412+0000", 2024-09-17T21:18:54.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6507e-05 2024-09-17T21:18:54.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198425+0000", 2024-09-17T21:18:54.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.256e-05 2024-09-17T21:18:54.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241383+0000", 2024-09-17T21:18:54.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042958216 2024-09-17T21:18:54.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241525+0000", 2024-09-17T21:18:54.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014204500000000001 2024-09-17T21:18:54.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242287+0000", 2024-09-17T21:18:54.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076125800000000001 2024-09-17T21:18:54.789 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.789 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.789 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.789 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:496 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.789 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:18:54.789 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.573040632000001, 2024-09-17T21:18:54.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043732500000000001, 2024-09-17T21:18:54.790 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.790 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.790 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.790 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.790 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.790 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.790 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 496 2024-09-17T21:18:54.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.790 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:18:54.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:18:54.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198481+0000", 2024-09-17T21:18:54.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-17T21:18:54.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198488+0000", 2024-09-17T21:18:54.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8499999999999996e-06 2024-09-17T21:18:54.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198507+0000", 2024-09-17T21:18:54.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9276999999999999e-05 2024-09-17T21:18:54.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198521+0000", 2024-09-17T21:18:54.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3376e-05 2024-09-17T21:18:54.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241339+0000", 2024-09-17T21:18:54.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042818119000000002 2024-09-17T21:18:54.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241449+0000", 2024-09-17T21:18:54.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110209 2024-09-17T21:18:54.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242212+0000", 2024-09-17T21:18:54.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076329900000000005 2024-09-17T21:18:54.793 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.794 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.794 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.794 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:497 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.794 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:18:54.794 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.572959683000001, 2024-09-17T21:18:54.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044584278999999997, 2024-09-17T21:18:54.794 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.794 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.794 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.794 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.794 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.794 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.795 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 497 2024-09-17T21:18:54.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.795 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:18:54.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:18:54.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198562+0000", 2024-09-17T21:18:54.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-17T21:18:54.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198569+0000", 2024-09-17T21:18:54.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.494e-06 2024-09-17T21:18:54.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198588+0000", 2024-09-17T21:18:54.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8955999999999999e-05 2024-09-17T21:18:54.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198621+0000", 2024-09-17T21:18:54.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3837000000000001e-05 2024-09-17T21:18:54.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242258+0000", 2024-09-17T21:18:54.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043636900999999999 2024-09-17T21:18:54.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242343+0000", 2024-09-17T21:18:54.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5048000000000003e-05 2024-09-17T21:18:54.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243145+0000", 2024-09-17T21:18:54.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00080167300000000003 2024-09-17T21:18:54.798 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.798 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.798 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.799 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:503 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.799 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:18:54.799 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.572875158999999, 2024-09-17T21:18:54.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044220582000000001, 2024-09-17T21:18:54.799 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.799 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.799 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.799 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.799 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.799 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.799 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 503 2024-09-17T21:18:54.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.799 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:18:54.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:18:54.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198647+0000", 2024-09-17T21:18:54.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.483e-06 2024-09-17T21:18:54.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198653+0000", 2024-09-17T21:18:54.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4529999999999999e-06 2024-09-17T21:18:54.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198672+0000", 2024-09-17T21:18:54.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9193000000000001e-05 2024-09-17T21:18:54.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198685+0000", 2024-09-17T21:18:54.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.291e-05 2024-09-17T21:18:54.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:18:54.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043672141999999997 2024-09-17T21:18:54.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242716+0000", 2024-09-17T21:18:54.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00035901800000000001 2024-09-17T21:18:54.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242866+0000", 2024-09-17T21:18:54.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000149383 2024-09-17T21:18:54.803 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.803 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.803 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.803 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:504 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.803 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:18:54.803 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.572815367, 2024-09-17T21:18:54.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044611406999999999, 2024-09-17T21:18:54.804 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.804 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.804 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.804 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.804 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.804 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.804 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 504 2024-09-17T21:18:54.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.804 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:18:54.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:18:54.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198706+0000", 2024-09-17T21:18:54.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:18:54.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198712+0000", 2024-09-17T21:18:54.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5770000000000001e-06 2024-09-17T21:18:54.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198729+0000", 2024-09-17T21:18:54.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6673000000000001e-05 2024-09-17T21:18:54.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198742+0000", 2024-09-17T21:18:54.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3543000000000001e-05 2024-09-17T21:18:54.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242981+0000", 2024-09-17T21:18:54.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044238953999999997 2024-09-17T21:18:54.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243155+0000", 2024-09-17T21:18:54.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017401600000000001 2024-09-17T21:18:54.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243316+0000", 2024-09-17T21:18:54.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016137399999999999 2024-09-17T21:18:54.807 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.807 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.808 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.808 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:521 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.808 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:18:54.808 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.572268227999999, 2024-09-17T21:18:54.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045043685999999999, 2024-09-17T21:18:54.808 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.808 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.808 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.808 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.808 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.808 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.808 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 521 2024-09-17T21:18:54.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.809 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:18:54.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:18:54.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199253+0000", 2024-09-17T21:18:54.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1400000000000001e-06 2024-09-17T21:18:54.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199258+0000", 2024-09-17T21:18:54.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1800000000000004e-06 2024-09-17T21:18:54.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199276+0000", 2024-09-17T21:18:54.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7686999999999999e-05 2024-09-17T21:18:54.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199289+0000", 2024-09-17T21:18:54.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3263e-05 2024-09-17T21:18:54.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243397+0000", 2024-09-17T21:18:54.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044107636999999998 2024-09-17T21:18:54.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243527+0000", 2024-09-17T21:18:54.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013038399999999999 2024-09-17T21:18:54.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244296+0000", 2024-09-17T21:18:54.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076839499999999995 2024-09-17T21:18:54.812 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.812 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.812 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.812 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:522 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.812 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:18:54.812 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.572212924999999, 2024-09-17T21:18:54.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046049870999999999, 2024-09-17T21:18:54.813 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.813 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.813 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.813 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.813 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.813 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.813 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 522 2024-09-17T21:18:54.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.813 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:18:54.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:18:54.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199309+0000", 2024-09-17T21:18:54.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:18:54.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199315+0000", 2024-09-17T21:18:54.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2440000000000001e-06 2024-09-17T21:18:54.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199333+0000", 2024-09-17T21:18:54.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7886000000000001e-05 2024-09-17T21:18:54.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199379+0000", 2024-09-17T21:18:54.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6383e-05 2024-09-17T21:18:54.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244394+0000", 2024-09-17T21:18:54.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045014851000000002 2024-09-17T21:18:54.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244491+0000", 2024-09-17T21:18:54.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7245999999999997e-05 2024-09-17T21:18:54.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245357+0000", 2024-09-17T21:18:54.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00086626800000000001 2024-09-17T21:18:54.816 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.816 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.816 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.817 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:527 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.817 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:18:54.817 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.572116259000001, 2024-09-17T21:18:54.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045812522000000001, 2024-09-17T21:18:54.817 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.817 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.817 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.817 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.817 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.817 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.817 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 527 2024-09-17T21:18:54.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.817 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:18:54.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:18:54.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199406+0000", 2024-09-17T21:18:54.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.367e-06 2024-09-17T21:18:54.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199412+0000", 2024-09-17T21:18:54.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:18:54.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199429+0000", 2024-09-17T21:18:54.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7396000000000001e-05 2024-09-17T21:18:54.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199442+0000", 2024-09-17T21:18:54.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3327e-05 2024-09-17T21:18:54.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.820 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.820 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244361+0000", 2024-09-17T21:18:54.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044918197 2024-09-17T21:18:54.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.820 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.820 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244482+0000", 2024-09-17T21:18:54.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121945 2024-09-17T21:18:54.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.820 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.820 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245217+0000", 2024-09-17T21:18:54.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00073415999999999998 2024-09-17T21:18:54.821 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.821 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.821 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.821 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.821 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:534 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.821 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:18:54.821 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.572057553, 2024-09-17T21:18:54.821 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046292505999999997, 2024-09-17T21:18:54.821 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.821 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.821 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.821 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.821 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.822 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.822 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:18:54.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.822 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.822 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.822 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:18:54.822 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.822 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.822 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:18:54.822 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199464+0000", 2024-09-17T21:18:54.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:18:54.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199473+0000", 2024-09-17T21:18:54.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5629999999999992e-06 2024-09-17T21:18:54.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199492+0000", 2024-09-17T21:18:54.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9273999999999999e-05 2024-09-17T21:18:54.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199505+0000", 2024-09-17T21:18:54.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3443e-05 2024-09-17T21:18:54.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245357+0000", 2024-09-17T21:18:54.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045851714000000002 2024-09-17T21:18:54.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245550+0000", 2024-09-17T21:18:54.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019301600000000001 2024-09-17T21:18:54.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245755+0000", 2024-09-17T21:18:54.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205226 2024-09-17T21:18:54.825 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.825 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.825 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.825 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:535 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.825 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:18:54.826 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.571952997, 2024-09-17T21:18:54.826 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046637492000000003, 2024-09-17T21:18:54.826 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.826 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.826 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.826 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.826 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.826 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.826 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 535 2024-09-17T21:18:54.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.826 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.826 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.826 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.826 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:18:54.826 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.827 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.827 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:18:54.827 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.827 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.827 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199568+0000", 2024-09-17T21:18:54.827 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.037e-06 2024-09-17T21:18:54.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.827 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.827 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199572+0000", 2024-09-17T21:18:54.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5729999999999999e-06 2024-09-17T21:18:54.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.828 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.828 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199585+0000", 2024-09-17T21:18:54.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.279e-05 2024-09-17T21:18:54.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.828 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.828 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199592+0000", 2024-09-17T21:18:54.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.61e-06 2024-09-17T21:18:54.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245873+0000", 2024-09-17T21:18:54.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046280494999999998 2024-09-17T21:18:54.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246018+0000", 2024-09-17T21:18:54.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000144697 2024-09-17T21:18:54.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246205+0000", 2024-09-17T21:18:54.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018729 2024-09-17T21:18:54.829 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:543 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:18:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.571857751, 2024-09-17T21:18:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047564097999999999, 2024-09-17T21:18:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 543 2024-09-17T21:18:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:18:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:18:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:18:54.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9699999999999995e-07 2024-09-17T21:18:54.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.832 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.832 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199677+0000", 2024-09-17T21:18:54.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:18:54.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.832 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.832 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199690+0000", 2024-09-17T21:18:54.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2977e-05 2024-09-17T21:18:54.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199705+0000", 2024-09-17T21:18:54.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4656e-05 2024-09-17T21:18:54.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246307+0000", 2024-09-17T21:18:54.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046601780000000002 2024-09-17T21:18:54.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246475+0000", 2024-09-17T21:18:54.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016834200000000001 2024-09-17T21:18:54.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247227+0000", 2024-09-17T21:18:54.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00075173299999999998 2024-09-17T21:18:54.834 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.834 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.834 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.834 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:544 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.834 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:18:54.834 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.571799341999998, 2024-09-17T21:18:54.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049283559999999997, 2024-09-17T21:18:54.834 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.835 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.835 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.835 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.835 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.835 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.835 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 544 2024-09-17T21:18:54.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.835 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.835 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.835 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:18:54.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:18:54.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199722+0000", 2024-09-17T21:18:54.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.08e-06 2024-09-17T21:18:54.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199738+0000", 2024-09-17T21:18:54.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5943999999999998e-05 2024-09-17T21:18:54.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199748+0000", 2024-09-17T21:18:54.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0193e-05 2024-09-17T21:18:54.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199756+0000", 2024-09-17T21:18:54.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4200000000000001e-06 2024-09-17T21:18:54.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247311+0000", 2024-09-17T21:18:54.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047555300000000002 2024-09-17T21:18:54.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247571+0000", 2024-09-17T21:18:54.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025996999999999999 2024-09-17T21:18:54.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249005+0000", 2024-09-17T21:18:54.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001433653 2024-09-17T21:18:54.839 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.839 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.839 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.839 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:549 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.839 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:18:54.839 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.571753718, 2024-09-17T21:18:54.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048464528999999999, 2024-09-17T21:18:54.839 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.839 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.840 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.840 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.840 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.840 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.840 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 549 2024-09-17T21:18:54.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.840 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:18:54.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.841 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:18:54.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.841 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.841 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:18:54.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:18:54.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.841 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.841 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199770+0000", 2024-09-17T21:18:54.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9969999999999999e-06 2024-09-17T21:18:54.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.842 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.842 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199780+0000", 2024-09-17T21:18:54.842 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0003000000000001e-05 2024-09-17T21:18:54.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.842 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.842 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199793+0000", 2024-09-17T21:18:54.842 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2074e-05 2024-09-17T21:18:54.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.842 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.842 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247289+0000", 2024-09-17T21:18:54.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047496691000000001 2024-09-17T21:18:54.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.843 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247437+0000", 2024-09-17T21:18:54.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014807899999999999 2024-09-17T21:18:54.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.843 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248231+0000", 2024-09-17T21:18:54.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00079390899999999998 2024-09-17T21:18:54.843 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.843 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.844 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.844 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:558 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.844 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:18:54.844 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.571713205000002, 2024-09-17T21:18:54.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049709821000000001, 2024-09-17T21:18:54.844 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.844 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.844 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.844 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.844 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.844 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.845 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 558 2024-09-17T21:18:54.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.845 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:18:54.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:18:54.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199809+0000", 2024-09-17T21:18:54.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.333e-06 2024-09-17T21:18:54.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199814+0000", 2024-09-17T21:18:54.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0930000000000002e-06 2024-09-17T21:18:54.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199831+0000", 2024-09-17T21:18:54.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7580000000000001e-05 2024-09-17T21:18:54.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.847 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.847 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.847 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199860+0000", 2024-09-17T21:18:54.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8609999999999999e-05 2024-09-17T21:18:54.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.847 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.847 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.847 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249106+0000", 2024-09-17T21:18:54.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049246545000000003 2024-09-17T21:18:54.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.847 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.847 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.848 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249274+0000", 2024-09-17T21:18:54.848 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016806699999999999 2024-09-17T21:18:54.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.848 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.848 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.848 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249517+0000", 2024-09-17T21:18:54.848 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024259300000000001 2024-09-17T21:18:54.848 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.848 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.848 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.848 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.848 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:559 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.849 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:18:54.849 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.571593842999999, 2024-09-17T21:18:54.849 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049990694000000002, 2024-09-17T21:18:54.849 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.849 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.849 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.849 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.849 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.849 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.849 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 559 2024-09-17T21:18:54.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.849 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.849 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.849 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:18:54.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:18:54.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199928+0000", 2024-09-17T21:18:54.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:18:54.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.851 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199935+0000", 2024-09-17T21:18:54.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5429999999999999e-06 2024-09-17T21:18:54.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.851 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199954+0000", 2024-09-17T21:18:54.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.912e-05 2024-09-17T21:18:54.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.852 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199969+0000", 2024-09-17T21:18:54.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5305999999999999e-05 2024-09-17T21:18:54.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.852 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.852 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249604+0000", 2024-09-17T21:18:54.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049634931 2024-09-17T21:18:54.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.852 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.852 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249738+0000", 2024-09-17T21:18:54.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013411599999999999 2024-09-17T21:18:54.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.853 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.853 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.853 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249917+0000", 2024-09-17T21:18:54.853 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179111 2024-09-17T21:18:54.853 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.853 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.853 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.853 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.856 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:560 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:54.856 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:18:54.856 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.571529427000002, 2024-09-17T21:18:54.856 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050173059999999998, 2024-09-17T21:18:54.856 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.856 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.856 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.856 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.856 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:54.856 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:54.856 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 560 2024-09-17T21:18:54.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.857 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.857 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.857 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.857 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:18:54.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.857 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.857 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.857 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:18:54.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.857 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.857 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.858 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199992+0000", 2024-09-17T21:18:54.858 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.517e-06 2024-09-17T21:18:54.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.858 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.858 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.858 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199998+0000", 2024-09-17T21:18:54.858 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1460000000000004e-06 2024-09-17T21:18:54.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.858 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.858 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.858 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:18:54.858 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7407000000000001e-05 2024-09-17T21:18:54.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.858 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.859 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.859 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200029+0000", 2024-09-17T21:18:54.859 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3499999999999999e-05 2024-09-17T21:18:54.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.859 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.859 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.859 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249968+0000", 2024-09-17T21:18:54.859 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049939799 2024-09-17T21:18:54.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.859 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.859 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.859 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250055+0000", 2024-09-17T21:18:54.859 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6731999999999996e-05 2024-09-17T21:18:54.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.859 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250164+0000", 2024-09-17T21:18:54.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108959 2024-09-17T21:18:54.860 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.860 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.860 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.860 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:18:54.860 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:18:54.860 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.961739718, 2024-09-17T21:18:54.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:18:54.860 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.861 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.861 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:54.861 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.861 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:18:54.861 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:18:54.861 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:18:54.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.861 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:18:54.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.862 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.862 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:18:54.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.862 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.862 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:18:54.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:18:54.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.862 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.863 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:18:54.863 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:18:54.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.863 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.863 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:18:54.863 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:18:54.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.863 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.863 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:18:54.863 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:18:54.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:18:54.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:18:54.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:18:54.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:18:54.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:18:54.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:18:54.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:18:54.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:18:54.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:18:54.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:18:54.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:18:54.865 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.865 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.866 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.866 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:18:54.866 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:18:54.866 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.406637932, 2024-09-17T21:18:54.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:18:54.866 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:54.866 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:54.866 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:54.866 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:54.866 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:18:54.866 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:18:54.866 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:18:54.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.867 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:54.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.867 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:54.867 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:18:54.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.867 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:54.867 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:18:54.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:54.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.867 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:54.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:18:54.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:18:54.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:54.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:18:54.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:18:54.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:54.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:18:54.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:18:54.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:54.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:18:54.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:18:54.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:18:54.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:18:54.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:18:54.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:18:54.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:18:54.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:54.870 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:18:54.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:18:54.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:54.870 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:18:54.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:18:54.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:18:54.871 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:18:54.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:18:54.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.871 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:54.871 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:18:54.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:18:54.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.871 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:54.871 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:18:54.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:18:54.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:54.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:18:54.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:18:54.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:54.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:54.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:54.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:18:54.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:18:54.872 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.872 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.872 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.872 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:54.872 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:54.872 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:54.874 INFO:teuthology.orchestra.run.smithi184.stdout:1125 write 0xd7d7400 thru 0xd7de800 (0x7401 bytes) 2024-09-17T21:18:54.874 INFO:teuthology.orchestra.run.smithi184.stdout:1126 trunc from 0xd7dea00 to 0x7b57200 2024-09-17T21:18:54.874 INFO:teuthology.orchestra.run.smithi184.stdout:1131 write 0x601200 thru 0x60f43d (0xe23e bytes) 2024-09-17T21:18:54.874 INFO:teuthology.orchestra.run.smithi184.stdout:1134 write 0x555b400 thru 0x55698cf (0xe4d0 bytes) 2024-09-17T21:18:54.875 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:18:54.875 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:54.871+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:54.893 INFO:teuthology.orchestra.run.smithi184.stdout:1136 write 0xe1eac00 thru 0xe1fa07a (0xf47b bytes) 2024-09-17T21:18:54.896 INFO:teuthology.orchestra.run.smithi184.stdout:1140 write 0xe10c600 thru 0xe1142ea (0x7ceb bytes) 2024-09-17T21:18:54.897 INFO:teuthology.orchestra.run.smithi184.stdout:1141 write 0xb136c00 thru 0xb137885 (0xc86 bytes) 2024-09-17T21:18:54.897 INFO:teuthology.orchestra.run.smithi184.stdout:1147 punch from 0x816d000 to 0x817ce00, (0xfe00 bytes) 2024-09-17T21:18:54.899 INFO:teuthology.orchestra.run.smithi184.stdout:1148 trunc from 0xe1fa200 to 0xd89a000 2024-09-17T21:18:54.956 INFO:teuthology.orchestra.run.smithi184.stdout:1150 trunc from 0xd89a000 to 0xd382a00 2024-09-17T21:18:54.974 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:54.973+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:54.977 INFO:teuthology.orchestra.run.smithi184.stdout:1151 read 0xc1e7e00 thru 0xc1f2369 (0xa56a bytes) 2024-09-17T21:18:54.978 INFO:teuthology.orchestra.run.smithi184.stdout:1154 write 0x3a9dc00 thru 0x3aa486b (0x6c6c bytes) 2024-09-17T21:18:55.008 INFO:teuthology.orchestra.run.smithi184.stdout:1155 trunc from 0xd382a00 to 0x35ca800 2024-09-17T21:18:55.051 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:55.051 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:55.051 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:55.051 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:55.067 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:18:55.076 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:55.073+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:55.110 INFO:teuthology.orchestra.run.smithi184.stdout:1159 trunc from 0x35ca800 to 0x67b4800 2024-09-17T21:18:55.132 INFO:teuthology.orchestra.run.smithi184.stdout:1161 trunc from 0x67b4800 to 0x61e3200 2024-09-17T21:18:55.165 INFO:teuthology.orchestra.run.smithi184.stdout:1162 trunc from 0x61e3200 to 0x9987400 2024-09-17T21:18:55.176 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:55.173+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:55.185 INFO:teuthology.orchestra.run.smithi184.stdout:1163 read 0x8177800 thru 0x817dce0 (0x64e1 bytes) 2024-09-17T21:18:55.191 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:55.191 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:55.191 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:55.191 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:55.191 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:55.195 INFO:teuthology.orchestra.run.smithi184.stdout:1164 trunc from 0x9987400 to 0x61c8a00 2024-09-17T21:18:55.205 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:18:55.219 INFO:teuthology.orchestra.run.smithi184.stdout:1165 read 0x337a400 thru 0x33855f8 (0xb1f9 bytes) 2024-09-17T21:18:55.226 INFO:teuthology.orchestra.run.smithi184.stdout:1167 trunc from 0x61c8a00 to 0x7edf400 2024-09-17T21:18:55.251 INFO:teuthology.orchestra.run.smithi184.stdout:1168 read 0x611fa00 thru 0x612ca83 (0xd084 bytes) 2024-09-17T21:18:55.253 INFO:teuthology.orchestra.run.smithi184.stdout:1169 punch from 0x78e8800 to 0x78ef200, (0x6a00 bytes) 2024-09-17T21:18:55.253 INFO:teuthology.orchestra.run.smithi184.stdout:1172 write 0x65a7c00 thru 0x65b2adf (0xaee0 bytes) 2024-09-17T21:18:55.253 INFO:teuthology.orchestra.run.smithi184.stdout:1174 read 0x5442600 thru 0x5443d58 (0x1759 bytes) 2024-09-17T21:18:55.254 INFO:teuthology.orchestra.run.smithi184.stdout:1176 read 0x219800 thru 0x2254ff (0xbd00 bytes) 2024-09-17T21:18:55.275 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:18:55.277 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:55.273+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:55.288 INFO:teuthology.orchestra.run.smithi184.stdout:1178 write 0x8198c00 thru 0x81a2d9f (0xa1a0 bytes) 2024-09-17T21:18:55.290 INFO:teuthology.orchestra.run.smithi184.stdout:1179 read 0x24f2600 thru 0x24ff018 (0xca19 bytes) 2024-09-17T21:18:55.297 INFO:teuthology.orchestra.run.smithi184.stdout:1180 trunc from 0x81a2e00 to 0xa356000 2024-09-17T21:18:55.329 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-17T21:18:55.331 INFO:teuthology.orchestra.run.smithi184.stdout:1182 read 0x97eec00 thru 0x97f520d (0x660e bytes) 2024-09-17T21:18:55.333 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:55.333 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:18:55.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:18:55.334 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:18:55.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.334 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:12 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:55.334 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:55.334 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.999416226999998, 2024-09-17T21:18:55.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061340989999999996, 2024-09-17T21:18:55.334 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:55.334 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:55.334 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:55.335 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:55.335 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:55.335 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:55.335 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12 2024-09-17T21:18:55.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.335 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:55.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:55.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:55.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:55.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:55.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:55.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332336+0000", 2024-09-17T21:18:55.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5519999999999999e-06 2024-09-17T21:18:55.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:55.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332342+0000", 2024-09-17T21:18:55.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7180000000000001e-06 2024-09-17T21:18:55.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:55.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332352+0000", 2024-09-17T21:18:55.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:18:55.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:55.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332359+0000", 2024-09-17T21:18:55.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3479999999999998e-06 2024-09-17T21:18:55.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:55.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332413+0000", 2024-09-17T21:18:55.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4447999999999998e-05 2024-09-17T21:18:55.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-17T21:18:55.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332417+0000", 2024-09-17T21:18:55.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.895e-06 2024-09-17T21:18:55.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:55.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334755+0000", 2024-09-17T21:18:55.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023377189999999998 2024-09-17T21:18:55.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:55.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334903+0000", 2024-09-17T21:18:55.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014778999999999999 2024-09-17T21:18:55.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:18:55.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.335187+0000", 2024-09-17T21:18:55.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000284715 2024-09-17T21:18:55.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:55.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.337731+0000", 2024-09-17T21:18:55.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002543741 2024-09-17T21:18:55.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:55.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338382+0000", 2024-09-17T21:18:55.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00065029200000000001 2024-09-17T21:18:55.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:55.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338418+0000", 2024-09-17T21:18:55.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.697e-05 2024-09-17T21:18:55.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:55.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338469+0000", 2024-09-17T21:18:55.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0154999999999997e-05 2024-09-17T21:18:55.341 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.341 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:55.341 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.341 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "pg_update_log_missing(3.1e epoch 28/27 rep_tid 81 entries 28'41 (0'0) error 3:7b23fd98:::rbd_header.1152c01721e5:head by client.4488.0:561 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-17T21:18:55.341 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:18:55.341 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 38.050142100000002, 2024-09-17T21:18:55.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0051146969999999996, 2024-09-17T21:18:55.341 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:55.342 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:55.342 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:18:55.342 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:55.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:55.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:18:55.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:55.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:18:55.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:55.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281610+0000", 2024-09-17T21:18:55.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.16e-07 2024-09-17T21:18:55.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:55.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281626+0000", 2024-09-17T21:18:55.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6039999999999999e-05 2024-09-17T21:18:55.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:55.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281666+0000", 2024-09-17T21:18:55.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0364000000000002e-05 2024-09-17T21:18:55.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:55.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281710+0000", 2024-09-17T21:18:55.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3596000000000001e-05 2024-09-17T21:18:55.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:55.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281992+0000", 2024-09-17T21:18:55.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028273099999999999 2024-09-17T21:18:55.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:55.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286723+0000", 2024-09-17T21:18:55.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047310499999999997 2024-09-17T21:18:55.345 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.345 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:55.345 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.345 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4488.0:562 3.1e e28/27 3:7b23fd98:::rbd_header.1152c01721e5:head v 28'42, mlcod=0'0)", 2024-09-17T21:18:55.345 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:18:55.345 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 38.049898312000003, 2024-09-17T21:18:55.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049275020000000003, 2024-09-17T21:18:55.345 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:55.345 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:55.345 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:55.345 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:55.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:55.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:18:55.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:55.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:18:55.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:55.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281853+0000", 2024-09-17T21:18:55.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2999999999999999e-07 2024-09-17T21:18:55.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:55.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281875+0000", 2024-09-17T21:18:55.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1386000000000002e-05 2024-09-17T21:18:55.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:55.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281892+0000", 2024-09-17T21:18:55.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7637000000000001e-05 2024-09-17T21:18:55.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:55.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281908+0000", 2024-09-17T21:18:55.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5187e-05 2024-09-17T21:18:55.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:55.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.282517+0000", 2024-09-17T21:18:55.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00060989799999999999 2024-09-17T21:18:55.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:55.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.282548+0000", 2024-09-17T21:18:55.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0910000000000001e-05 2024-09-17T21:18:55.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:55.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286729+0000", 2024-09-17T21:18:55.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0041810850000000002 2024-09-17T21:18:55.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:55.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286780+0000", 2024-09-17T21:18:55.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0569000000000003e-05 2024-09-17T21:18:55.349 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.349 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:55.349 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.349 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:609 3.d e28/23 3:b7278be9:::rbd_data.1152c6af092e.000000000000008f:head v 28'6, mlcod=28'4)", 2024-09-17T21:18:55.349 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:18:55.350 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.216954025, 2024-09-17T21:18:55.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049797280000000001, 2024-09-17T21:18:55.350 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:55.350 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:55.350 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:55.350 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:55.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:55.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:18:55.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:55.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:18:55.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:55.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114800+0000", 2024-09-17T21:18:55.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:18:55.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:55.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114915+0000", 2024-09-17T21:18:55.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115053 2024-09-17T21:18:55.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:55.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114951+0000", 2024-09-17T21:18:55.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6165999999999999e-05 2024-09-17T21:18:55.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:55.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114965+0000", 2024-09-17T21:18:55.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4364e-05 2024-09-17T21:18:55.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:55.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115097+0000", 2024-09-17T21:18:55.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013139200000000001 2024-09-17T21:18:55.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:55.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115151+0000", 2024-09-17T21:18:55.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4422000000000002e-05 2024-09-17T21:18:55.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:55.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.119633+0000", 2024-09-17T21:18:55.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044818820000000004 2024-09-17T21:18:55.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:55.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.119776+0000", 2024-09-17T21:18:55.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000143609 2024-09-17T21:18:55.354 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.354 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:55.354 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.354 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:782 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'63, mlcod=28'60)", 2024-09-17T21:18:55.354 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:18:55.354 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.116985997, 2024-09-17T21:18:55.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006167283, 2024-09-17T21:18:55.355 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:55.355 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:55.355 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:55.355 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:55.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:55.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:18:55.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:55.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:18:55.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:55.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214767+0000", 2024-09-17T21:18:55.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:18:55.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:55.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214779+0000", 2024-09-17T21:18:55.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2627e-05 2024-09-17T21:18:55.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:55.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214805+0000", 2024-09-17T21:18:55.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5570000000000001e-05 2024-09-17T21:18:55.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:55.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214823+0000", 2024-09-17T21:18:55.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8202999999999998e-05 2024-09-17T21:18:55.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:55.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215170+0000", 2024-09-17T21:18:55.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034720399999999999 2024-09-17T21:18:55.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:55.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215235+0000", 2024-09-17T21:18:55.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4276000000000006e-05 2024-09-17T21:18:55.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:55.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220859+0000", 2024-09-17T21:18:55.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056240500000000002 2024-09-17T21:18:55.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:55.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220932+0000", 2024-09-17T21:18:55.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3503000000000004e-05 2024-09-17T21:18:55.358 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.359 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:55.359 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.359 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:783 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'64, mlcod=28'60)", 2024-09-17T21:18:55.359 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:18:55.359 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.116299155, 2024-09-17T21:18:55.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0055240869999999996, 2024-09-17T21:18:55.359 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:55.359 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:55.359 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:55.359 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:55.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:55.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:18:55.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:55.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:18:55.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:55.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215454+0000", 2024-09-17T21:18:55.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0269999999999998e-06 2024-09-17T21:18:55.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:55.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215468+0000", 2024-09-17T21:18:55.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4816e-05 2024-09-17T21:18:55.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:55.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215501+0000", 2024-09-17T21:18:55.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2957000000000003e-05 2024-09-17T21:18:55.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:55.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215524+0000", 2024-09-17T21:18:55.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2782999999999999e-05 2024-09-17T21:18:55.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:55.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216675+0000", 2024-09-17T21:18:55.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011507869999999999 2024-09-17T21:18:55.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:55.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216728+0000", 2024-09-17T21:18:55.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3112999999999998e-05 2024-09-17T21:18:55.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:55.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220938+0000", 2024-09-17T21:18:55.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042097380000000002 2024-09-17T21:18:55.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:55.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220976+0000", 2024-09-17T21:18:55.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7866e-05 2024-09-17T21:18:55.363 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.363 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:55.363 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.363 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:785 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'65, mlcod=28'60)", 2024-09-17T21:18:55.364 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:18:55.364 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.115713106000001, 2024-09-17T21:18:55.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049795409999999997, 2024-09-17T21:18:55.364 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:55.364 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:55.364 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:55.364 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:55.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:55.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:18:55.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:55.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:18:55.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:55.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216039+0000", 2024-09-17T21:18:55.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.793e-06 2024-09-17T21:18:55.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:55.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216053+0000", 2024-09-17T21:18:55.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.345e-05 2024-09-17T21:18:55.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:55.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216092+0000", 2024-09-17T21:18:55.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9180000000000001e-05 2024-09-17T21:18:55.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:55.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216106+0000", 2024-09-17T21:18:55.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:18:55.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:55.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217569+0000", 2024-09-17T21:18:55.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001462938 2024-09-17T21:18:55.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:55.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217607+0000", 2024-09-17T21:18:55.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7267000000000001e-05 2024-09-17T21:18:55.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:55.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220980+0000", 2024-09-17T21:18:55.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003373544 2024-09-17T21:18:55.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:55.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221017+0000", 2024-09-17T21:18:55.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7116000000000002e-05 2024-09-17T21:18:55.368 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.368 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:55.368 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.369 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:786 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'66, mlcod=28'60)", 2024-09-17T21:18:55.369 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:18:55.369 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.115012301999997, 2024-09-17T21:18:55.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057966650000000003, 2024-09-17T21:18:55.369 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:55.369 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:55.369 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:55.369 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:55.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:55.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:18:55.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:55.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:18:55.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:55.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216740+0000", 2024-09-17T21:18:55.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5600000000000001e-06 2024-09-17T21:18:55.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:55.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216752+0000", 2024-09-17T21:18:55.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1477e-05 2024-09-17T21:18:55.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:55.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216779+0000", 2024-09-17T21:18:55.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7093e-05 2024-09-17T21:18:55.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:55.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216817+0000", 2024-09-17T21:18:55.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8275999999999997e-05 2024-09-17T21:18:55.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:55.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218387+0000", 2024-09-17T21:18:55.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015703749999999999 2024-09-17T21:18:55.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:55.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218432+0000", 2024-09-17T21:18:55.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4910000000000002e-05 2024-09-17T21:18:55.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:55.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222432+0000", 2024-09-17T21:18:55.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039997189999999997 2024-09-17T21:18:55.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:55.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222535+0000", 2024-09-17T21:18:55.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103255 2024-09-17T21:18:55.373 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.373 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:55.373 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.373 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:18:55.373 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:18:55.373 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.114390469999996, 2024-09-17T21:18:55.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:18:55.374 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:55.374 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:55.374 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:55.374 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:55.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:55.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:18:55.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:55.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:18:55.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:55.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:18:55.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:18:55.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:55.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:18:55.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:18:55.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:55.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:18:55.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:18:55.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:55.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:18:55.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:18:55.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:55.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:18:55.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:18:55.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:55.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:18:55.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:18:55.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:55.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:18:55.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:18:55.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:55.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:18:55.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:18:55.377 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.377 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:55.378 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.378 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:788 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'68, mlcod=28'60)", 2024-09-17T21:18:55.378 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:18:55.378 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.113756037999998, 2024-09-17T21:18:55.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058495320000000002, 2024-09-17T21:18:55.378 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:55.378 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:55.378 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:55.378 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:55.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:55.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:18:55.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:55.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:18:55.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:55.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217996+0000", 2024-09-17T21:18:55.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.376e-06 2024-09-17T21:18:55.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:55.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218006+0000", 2024-09-17T21:18:55.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.01e-05 2024-09-17T21:18:55.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:55.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218030+0000", 2024-09-17T21:18:55.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4026999999999999e-05 2024-09-17T21:18:55.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:55.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218042+0000", 2024-09-17T21:18:55.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1977000000000001e-05 2024-09-17T21:18:55.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:55.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221081+0000", 2024-09-17T21:18:55.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030385960000000002 2024-09-17T21:18:55.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:55.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221115+0000", 2024-09-17T21:18:55.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3772999999999997e-05 2024-09-17T21:18:55.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:55.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223749+0000", 2024-09-17T21:18:55.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0026340629999999999 2024-09-17T21:18:55.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:55.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223844+0000", 2024-09-17T21:18:55.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5619999999999996e-05 2024-09-17T21:18:55.382 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.382 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:55.382 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.382 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:790 3.5 e28/23 3:a14b1a37:::rbd_data.1152c6af092e.000000000000002f:head v 28'1, mlcod=0'0)", 2024-09-17T21:18:55.382 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:18:55.382 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.109351711999999, 2024-09-17T21:18:55.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059622240000000003, 2024-09-17T21:18:55.383 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:55.383 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:55.383 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:55.383 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:55.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:55.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:18:55.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:55.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:18:55.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:55.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222400+0000", 2024-09-17T21:18:55.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.02e-06 2024-09-17T21:18:55.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:55.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222411+0000", 2024-09-17T21:18:55.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.044e-05 2024-09-17T21:18:55.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:55.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222440+0000", 2024-09-17T21:18:55.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.932e-05 2024-09-17T21:18:55.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:55.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222458+0000", 2024-09-17T21:18:55.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8615999999999999e-05 2024-09-17T21:18:55.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:55.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222609+0000", 2024-09-17T21:18:55.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015102299999999999 2024-09-17T21:18:55.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:55.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222657+0000", 2024-09-17T21:18:55.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7206000000000002e-05 2024-09-17T21:18:55.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:55.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228278+0000", 2024-09-17T21:18:55.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056208100000000004 2024-09-17T21:18:55.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:55.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228361+0000", 2024-09-17T21:18:55.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3788999999999993e-05 2024-09-17T21:18:55.386 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.386 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:55.386 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.387 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:884 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'76, mlcod=28'70)", 2024-09-17T21:18:55.387 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:18:55.387 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.062766830999998, 2024-09-17T21:18:55.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056451990000000001, 2024-09-17T21:18:55.387 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:55.387 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:55.387 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:55.387 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:55.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:55.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:18:55.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:55.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:18:55.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:55.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268986+0000", 2024-09-17T21:18:55.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9360000000000002e-06 2024-09-17T21:18:55.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:55.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269001+0000", 2024-09-17T21:18:55.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4684e-05 2024-09-17T21:18:55.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:55.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269026+0000", 2024-09-17T21:18:55.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5193000000000001e-05 2024-09-17T21:18:55.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:55.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269043+0000", 2024-09-17T21:18:55.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7306e-05 2024-09-17T21:18:55.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:55.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.270024+0000", 2024-09-17T21:18:55.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00098120600000000009 2024-09-17T21:18:55.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:55.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.270085+0000", 2024-09-17T21:18:55.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0486000000000003e-05 2024-09-17T21:18:55.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:55.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.274557+0000", 2024-09-17T21:18:55.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044723419999999998 2024-09-17T21:18:55.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:55.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.274629+0000", 2024-09-17T21:18:55.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2045999999999994e-05 2024-09-17T21:18:55.391 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.391 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:55.391 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.391 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:18:55.392 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:18:55.392 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 32.464089805999997, 2024-09-17T21:18:55.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:18:55.392 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:55.392 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:55.392 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:55.392 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:55.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:55.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:18:55.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:55.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:18:55.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:55.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:18:55.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:18:55.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:55.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:18:55.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:18:55.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:55.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:18:55.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:18:55.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:55.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:18:55.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:18:55.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:55.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:18:55.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:18:55.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:55.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:18:55.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:18:55.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:55.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:18:55.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:18:55.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:55.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:18:55.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:18:55.396 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.396 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:55.396 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.396 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:18:55.396 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:18:55.396 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 32.462824722999997, 2024-09-17T21:18:55.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:18:55.396 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:55.396 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:55.396 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:55.396 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:55.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:55.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:18:55.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:55.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:18:55.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:55.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:18:55.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:18:55.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:55.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:18:55.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:18:55.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:55.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:18:55.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:18:55.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:55.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:18:55.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:18:55.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:55.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:18:55.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:18:55.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:55.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:18:55.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:18:55.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:55.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:18:55.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:18:55.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:55.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:18:55.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:18:55.400 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.400 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:55.400 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.400 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:18:55.400 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:18:55.400 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 32.453045312999997, 2024-09-17T21:18:55.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:18:55.401 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:55.401 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:55.401 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:55.401 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:55.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:55.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:18:55.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:55.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:18:55.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:55.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:18:55.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:18:55.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:55.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:18:55.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:18:55.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:55.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:18:55.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:18:55.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:55.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:18:55.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:18:55.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:55.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:18:55.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:18:55.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:55.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:18:55.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:18:55.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:55.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:18:55.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:18:55.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:55.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:18:55.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:18:55.404 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.404 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:55.405 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.405 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:18:55.405 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:18:55.405 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 32.444322909999997, 2024-09-17T21:18:55.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:18:55.405 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:55.405 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:55.405 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:55.405 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:55.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:55.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:18:55.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:55.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:18:55.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:55.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:18:55.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:18:55.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:55.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:18:55.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:18:55.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:55.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:18:55.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:18:55.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:55.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:18:55.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:18:55.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:55.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:18:55.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:18:55.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:55.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:18:55.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:18:55.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:55.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:18:55.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:18:55.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:55.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:18:55.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:18:55.409 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.409 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:55.409 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.409 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4498.0:596 3.4 e32/23 3:20f262ea:::rbd_data.1152b51cb396.000000000000000d:head v 32'17, mlcod=30'15)", 2024-09-17T21:18:55.410 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.258354+0000", 2024-09-17T21:18:55.410 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 29.073397046, 2024-09-17T21:18:55.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0053727619999999997, 2024-09-17T21:18:55.410 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:55.410 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:55.410 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:55.410 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:55.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:55.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.258354+0000", 2024-09-17T21:18:55.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:55.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.258354+0000", 2024-09-17T21:18:55.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:55.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.258356+0000", 2024-09-17T21:18:55.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.798e-06 2024-09-17T21:18:55.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:55.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259198+0000", 2024-09-17T21:18:55.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084279199999999998 2024-09-17T21:18:55.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:55.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259229+0000", 2024-09-17T21:18:55.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0383999999999999e-05 2024-09-17T21:18:55.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:55.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259241+0000", 2024-09-17T21:18:55.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2139e-05 2024-09-17T21:18:55.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:55.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259341+0000", 2024-09-17T21:18:55.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010049 2024-09-17T21:18:55.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.413 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:55.413 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259380+0000", 2024-09-17T21:18:55.413 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9072000000000001e-05 2024-09-17T21:18:55.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.413 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:55.413 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.263668+0000", 2024-09-17T21:18:55.413 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042875389999999999 2024-09-17T21:18:55.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.413 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:55.413 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.263726+0000", 2024-09-17T21:18:55.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8548000000000003e-05 2024-09-17T21:18:55.414 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.414 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:55.414 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.414 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4520.0:6177 3.8 e34/27 3:1665de58:::rbd_data.11528c3b0529.0000000000000019:head v 34'43, mlcod=34'41)", 2024-09-17T21:18:55.414 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:30.509962+0000", 2024-09-17T21:18:55.414 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 24.821789056, 2024-09-17T21:18:55.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049320850000000001, 2024-09-17T21:18:55.414 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:55.414 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:55.414 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:55.414 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:55.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:55.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509962+0000", 2024-09-17T21:18:55.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:55.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509962+0000", 2024-09-17T21:18:55.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:55.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509964+0000", 2024-09-17T21:18:55.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.356e-06 2024-09-17T21:18:55.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:55.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509978+0000", 2024-09-17T21:18:55.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3434e-05 2024-09-17T21:18:55.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:55.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510001+0000", 2024-09-17T21:18:55.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3436000000000001e-05 2024-09-17T21:18:55.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:55.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510023+0000", 2024-09-17T21:18:55.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1739999999999999e-05 2024-09-17T21:18:55.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:55.417 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510193+0000", 2024-09-17T21:18:55.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017040600000000001 2024-09-17T21:18:55.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:55.417 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510250+0000", 2024-09-17T21:18:55.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6749000000000002e-05 2024-09-17T21:18:55.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:55.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.514823+0000", 2024-09-17T21:18:55.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0045728180000000002 2024-09-17T21:18:55.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:55.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.514894+0000", 2024-09-17T21:18:55.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1145999999999999e-05 2024-09-17T21:18:55.418 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.418 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:55.418 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.418 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:18:55.418 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:18:55.419 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 19.059232346000002, 2024-09-17T21:18:55.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:18:55.419 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:55.419 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:55.419 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:55.419 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:55.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:55.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:18:55.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:55.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:18:55.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:55.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:18:55.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:18:55.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:55.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:18:55.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:18:55.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:55.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:18:55.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:18:55.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.421 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:55.421 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:18:55.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:18:55.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.421 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:55.421 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:18:55.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:18:55.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.421 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:55.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:18:55.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:18:55.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:55.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:18:55.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:18:55.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:55.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:18:55.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:18:55.422 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.422 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:55.423 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.423 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4863.0:165 3.5 e52/23 3:a772dfb1:::rbd_data.11523651b015.0000000000000001:head v 52'103, mlcod=47'101)", 2024-09-17T21:18:55.423 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:50.682666+0000", 2024-09-17T21:18:55.423 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.6490845400000005, 2024-09-17T21:18:55.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0055448839999999999, 2024-09-17T21:18:55.423 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:55.423 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:55.423 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:55.423 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:55.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:55.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.682666+0000", 2024-09-17T21:18:55.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:55.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.682666+0000", 2024-09-17T21:18:55.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:55.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:55.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.682667+0000", 2024-09-17T21:18:55.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.16e-07 2024-09-17T21:18:55.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:55.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.683332+0000", 2024-09-17T21:18:55.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000664476 2024-09-17T21:18:55.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:55.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.683360+0000", 2024-09-17T21:18:55.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7988e-05 2024-09-17T21:18:55.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:55.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.683376+0000", 2024-09-17T21:18:55.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6578999999999998e-05 2024-09-17T21:18:55.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.426 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:55.426 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.683522+0000", 2024-09-17T21:18:55.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014536400000000001 2024-09-17T21:18:55.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.426 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:55.426 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.683560+0000", 2024-09-17T21:18:55.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8005e-05 2024-09-17T21:18:55.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:55.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.688080+0000", 2024-09-17T21:18:55.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0045201390000000003 2024-09-17T21:18:55.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:55.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:55.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:55.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.688211+0000", 2024-09-17T21:18:55.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013141699999999999 2024-09-17T21:18:55.427 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.427 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:55.427 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.427 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:55.427 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:55.427 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:55.430 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:18:55.431 INFO:teuthology.orchestra.run.smithi184.stdout:1183 read 0x2df400 thru 0x2e6e61 (0x7a62 bytes) 2024-09-17T21:18:55.431 INFO:teuthology.orchestra.run.smithi184.stdout:1184 trunc from 0xa356000 to 0xe3a3200 2024-09-17T21:18:55.431 INFO:teuthology.orchestra.run.smithi184.stdout:1185 write 0x935ca00 thru 0x93611e1 (0x47e2 bytes) 2024-09-17T21:18:55.431 INFO:teuthology.orchestra.run.smithi184.stdout:1186 write 0x4a64800 thru 0x4a69344 (0x4b45 bytes) 2024-09-17T21:18:55.431 INFO:teuthology.orchestra.run.smithi184.stdout:1187 read 0xdf0d800 thru 0xdf18613 (0xae14 bytes) 2024-09-17T21:18:55.431 INFO:teuthology.orchestra.run.smithi184.stdout:1190 read 0xc2e8800 thru 0xc2eaefe (0x26ff bytes) 2024-09-17T21:18:55.431 INFO:teuthology.orchestra.run.smithi184.stdout:1192 read 0xb6e7600 thru 0xb6e8190 (0xb91 bytes) 2024-09-17T21:18:55.431 INFO:teuthology.orchestra.run.smithi184.stdout:1196 read 0xacdae00 thru 0xace0152 (0x5353 bytes) 2024-09-17T21:18:55.431 INFO:teuthology.orchestra.run.smithi184.stdout:1198 write 0xd71600 thru 0xd7431a (0x2d1b bytes) 2024-09-17T21:18:55.431 INFO:teuthology.orchestra.run.smithi184.stdout:1199 trunc from 0xe3a3200 to 0x9bb0600 2024-09-17T21:18:55.432 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:18:55.432 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:18:55.432 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:55.427+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:55.435 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:18:55.435 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:18:55.444 INFO:teuthology.orchestra.run.smithi184.stdout:1201 punch from 0xd5e400 to 0xd67600, (0x9200 bytes) 2024-09-17T21:18:55.448 INFO:teuthology.orchestra.run.smithi184.stdout:1202 read 0x3d2f600 thru 0x3d3a247 (0xac48 bytes) 2024-09-17T21:18:55.448 INFO:teuthology.orchestra.run.smithi184.stdout:1204 read 0x9a8aa00 thru 0x9a8fecc (0x54cd bytes) 2024-09-17T21:18:55.449 INFO:teuthology.orchestra.run.smithi184.stdout:1205 read 0x9ae5e00 thru 0x9af0738 (0xa939 bytes) 2024-09-17T21:18:55.452 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:18:55.452 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:18:55.452 INFO:teuthology.orchestra.run.smithi184.stdout:1206 read 0x78ca600 thru 0x78cfc36 (0x5637 bytes) 2024-09-17T21:18:55.452 INFO:teuthology.orchestra.run.smithi184.stdout:1209 punch from 0x4e33a00 to 0x4e43200, (0xf800 bytes) 2024-09-17T21:18:55.472 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:18:55.472 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:18:55.473 INFO:teuthology.orchestra.run.smithi184.stdout:1210 trunc from 0x9bb0600 to 0x3087200 2024-09-17T21:18:55.492 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:18:55.492 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:18:55.508 INFO:teuthology.orchestra.run.smithi184.stdout:1211 read 0x1824a00 thru 0x182d282 (0x8883 bytes) 2024-09-17T21:18:55.510 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:18:55.510 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:18:55.529 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:55.525+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:55.540 INFO:teuthology.orchestra.run.smithi184.stdout:1212 trunc from 0x3087200 to 0xdc41e00 2024-09-17T21:18:55.556 INFO:teuthology.orchestra.run.smithi184.stdout:1215 write 0x2aaa600 thru 0x2aaebbe (0x45bf bytes) 2024-09-17T21:18:55.560 INFO:teuthology.orchestra.run.smithi184.stdout:1217 read 0x5648200 thru 0x5649037 (0xe38 bytes) 2024-09-17T21:18:55.560 INFO:teuthology.orchestra.run.smithi184.stdout:1218 write 0x3d2c600 thru 0x3d379da (0xb3db bytes) 2024-09-17T21:18:55.561 INFO:teuthology.orchestra.run.smithi184.stdout:1219 read 0x73f4600 thru 0x73f528c (0xc8d bytes) 2024-09-17T21:18:55.561 INFO:teuthology.orchestra.run.smithi184.stdout:1220 read 0x4c17000 thru 0x4c1ce82 (0x5e83 bytes) 2024-09-17T21:18:55.561 INFO:teuthology.orchestra.run.smithi184.stdout:1221 write 0x32ca600 thru 0x32cf636 (0x5037 bytes) 2024-09-17T21:18:55.562 INFO:teuthology.orchestra.run.smithi184.stdout:1224 write 0xd84e800 thru 0xd8500bf (0x18c0 bytes) 2024-09-17T21:18:55.567 INFO:teuthology.orchestra.run.smithi184.stdout:1225 trunc from 0xdc41e00 to 0xb693200 2024-09-17T21:18:55.579 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:55.579 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:55.593 INFO:teuthology.orchestra.run.smithi184.stdout:1228 read 0x66c6200 thru 0x66d5346 (0xf147 bytes) 2024-09-17T21:18:55.594 INFO:teuthology.orchestra.run.smithi184.stdout:1230 read 0x5e7fc00 thru 0x5e88472 (0x8873 bytes) 2024-09-17T21:18:55.594 INFO:teuthology.orchestra.run.smithi184.stdout:1233 read 0x3366000 thru 0x3375fa9 (0xffaa bytes) 2024-09-17T21:18:55.597 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:18:55.604 INFO:teuthology.orchestra.run.smithi184.stdout:1234 trunc from 0xb693200 to 0x5c29800 2024-09-17T21:18:55.617 INFO:teuthology.orchestra.run.smithi184.stdout:1235 punch from 0x53f200 to 0x54ce00, (0xdc00 bytes) 2024-09-17T21:18:55.639 INFO:teuthology.orchestra.run.smithi184.stdout:1236 write 0x60e8e00 thru 0x60f2c76 (0x9e77 bytes) 2024-09-17T21:18:55.642 INFO:teuthology.orchestra.run.smithi184.stdout:1238 write 0x1a9c000 thru 0x1aa9f74 (0xdf75 bytes) 2024-09-17T21:18:55.642 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:55.641+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:55.672 INFO:teuthology.orchestra.run.smithi184.stdout:1239 trunc from 0x60f2e00 to 0x1a41400 2024-09-17T21:18:55.729 INFO:teuthology.orchestra.run.smithi184.stdout:1242 read 0xc85c00 thru 0xc94974 (0xed75 bytes) 2024-09-17T21:18:55.730 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:55.729+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:55.772 INFO:teuthology.orchestra.run.smithi184.stdout:1243 write 0xb40e000 thru 0xb4125c2 (0x45c3 bytes) 2024-09-17T21:18:55.774 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:55.774 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:55.780 INFO:teuthology.orchestra.run.smithi184.stdout:1245 trunc from 0xb412600 to 0xe82fa00 2024-09-17T21:18:55.789 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:18:55.804 INFO:teuthology.orchestra.run.smithi184.stdout:1246 punch from 0x6aa4600 to 0x6ab1400, (0xce00 bytes) 2024-09-17T21:18:55.804 INFO:teuthology.orchestra.run.smithi184.stdout:1248 read 0xb468800 thru 0xb46d6b3 (0x4eb4 bytes) 2024-09-17T21:18:55.805 INFO:teuthology.orchestra.run.smithi184.stdout:1249 write 0x4f0d200 thru 0x4f1c7f7 (0xf5f8 bytes) 2024-09-17T21:18:55.805 INFO:teuthology.orchestra.run.smithi184.stdout:1250 punch from 0xd2c3e00 to 0xd2ca200, (0x6400 bytes) 2024-09-17T21:18:55.806 INFO:teuthology.orchestra.run.smithi184.stdout:1251 read 0x7b2f400 thru 0x7b3db8d (0xe78e bytes) 2024-09-17T21:18:55.806 INFO:teuthology.orchestra.run.smithi184.stdout:1253 read 0x40cac00 thru 0x40d5156 (0xa557 bytes) 2024-09-17T21:18:55.807 INFO:teuthology.orchestra.run.smithi184.stdout:1255 read 0x384c200 thru 0x38513bc (0x51bd bytes) 2024-09-17T21:18:55.807 INFO:teuthology.orchestra.run.smithi184.stdout:1257 write 0x3b37200 thru 0x3b3bb1e (0x491f bytes) 2024-09-17T21:18:55.807 INFO:teuthology.orchestra.run.smithi184.stdout:1259 write 0xc0fd400 thru 0xc0fe826 (0x1427 bytes) 2024-09-17T21:18:55.831 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:55.827+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:55.835 INFO:teuthology.orchestra.run.smithi184.stdout:1260 trunc from 0xe82fa00 to 0x9f9e00 2024-09-17T21:18:55.932 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:55.929+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:55.939 INFO:teuthology.orchestra.run.smithi184.stdout:1261 write 0x9ab8200 thru 0x9ac40f3 (0xbef4 bytes) 2024-09-17T21:18:55.940 INFO:teuthology.orchestra.run.smithi184.stdout:1265 punch from 0x6f55800 to 0x6f5ce00, (0x7600 bytes) 2024-09-17T21:18:55.941 INFO:teuthology.orchestra.run.smithi184.stdout:1266 punch from 0x1827e00 to 0x1832800, (0xaa00 bytes) 2024-09-17T21:18:55.941 INFO:teuthology.orchestra.run.smithi184.stdout:1267 punch from 0x4a5ca00 to 0x4a68200, (0xb800 bytes) 2024-09-17T21:18:55.944 INFO:teuthology.orchestra.run.smithi184.stdout:1268 trunc from 0x9ac4200 to 0x8a0dc00 2024-09-17T21:18:55.981 INFO:teuthology.orchestra.run.smithi184.stdout:1270 write 0x2e7a000 thru 0x2e7f043 (0x5044 bytes) 2024-09-17T21:18:55.982 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:55.982 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:55.982 INFO:teuthology.orchestra.run.smithi184.stdout:1272 punch from 0x3ee3000 to 0x3eea400, (0x7400 bytes) 2024-09-17T21:18:55.983 INFO:teuthology.orchestra.run.smithi184.stdout:1275 read 0x7458800 thru 0x745dcf3 (0x54f4 bytes) 2024-09-17T21:18:55.999 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:18:56.003 INFO:teuthology.orchestra.run.smithi184.stdout:1277 write 0x9990200 thru 0x999ed67 (0xeb68 bytes) 2024-09-17T21:18:56.005 INFO:teuthology.orchestra.run.smithi184.stdout:1278 write 0x918c200 thru 0x919bafb (0xf8fc bytes) 2024-09-17T21:18:56.005 INFO:teuthology.orchestra.run.smithi184.stdout:1280 punch from 0x45ce00 to 0x467a00, (0xac00 bytes) 2024-09-17T21:18:56.006 INFO:teuthology.orchestra.run.smithi184.stdout:1281 write 0x5e7cc00 thru 0x5e87099 (0xa49a bytes) 2024-09-17T21:18:56.022 INFO:teuthology.orchestra.run.smithi184.stdout:1284 trunc from 0x999ee00 to 0x1b6c200 2024-09-17T21:18:56.033 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:56.031+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:56.102 INFO:teuthology.orchestra.run.smithi184.stdout:1285 write 0xc7a1a00 thru 0xc7a6439 (0x4a3a bytes) 2024-09-17T21:18:56.127 INFO:teuthology.orchestra.run.smithi184.stdout:1286 write 0xdc29400 thru 0xdc31226 (0x7e27 bytes) 2024-09-17T21:18:56.128 INFO:teuthology.orchestra.run.smithi184.stdout:1288 write 0x76e3800 thru 0x76e9e3a (0x663b bytes) 2024-09-17T21:18:56.128 INFO:teuthology.orchestra.run.smithi184.stdout:1289 write 0x928fc00 thru 0x929c9d4 (0xcdd5 bytes) 2024-09-17T21:18:56.128 INFO:teuthology.orchestra.run.smithi184.stdout:1297 read 0xbf99600 thru 0xbf9fe3b (0x683c bytes) 2024-09-17T21:18:56.128 INFO:teuthology.orchestra.run.smithi184.stdout:1298 write 0x6e77c00 thru 0x6e7e279 (0x667a bytes) 2024-09-17T21:18:56.133 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:56.129+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:56.134 INFO:teuthology.orchestra.run.smithi184.stdout:1299 trunc from 0xdc31400 to 0xa70b800 2024-09-17T21:18:56.178 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:56.178 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:56.178 INFO:teuthology.orchestra.run.smithi184.stdout:1300 punch from 0x77d8e00 to 0x77dc800, (0x3a00 bytes) 2024-09-17T21:18:56.194 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:18:56.204 INFO:teuthology.orchestra.run.smithi184.stdout:1304 write 0xe6bd200 thru 0xe6c8fe6 (0xbde7 bytes) 2024-09-17T21:18:56.204 INFO:teuthology.orchestra.run.smithi184.stdout:1305 read 0xbc14e00 thru 0xbc21b80 (0xcd81 bytes) 2024-09-17T21:18:56.205 INFO:teuthology.orchestra.run.smithi184.stdout:1307 read 0x5d6fe00 thru 0x5d7cd96 (0xcf97 bytes) 2024-09-17T21:18:56.208 INFO:teuthology.orchestra.run.smithi184.stdout:1308 trunc from 0xe6c9000 to 0xc99cc00 2024-09-17T21:18:56.234 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:56.231+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:56.236 INFO:teuthology.orchestra.run.smithi184.stdout:1309 punch from 0xc7f6a00 to 0xc802a00, (0xc000 bytes) 2024-09-17T21:18:56.238 INFO:teuthology.orchestra.run.smithi184.stdout:1311 write 0xa6f5a00 thru 0xa6f9609 (0x3c0a bytes) 2024-09-17T21:18:56.238 INFO:teuthology.orchestra.run.smithi184.stdout:1312 read 0xc114000 thru 0xc119b8e (0x5b8f bytes) 2024-09-17T21:18:56.238 INFO:teuthology.orchestra.run.smithi184.stdout:1314 write 0x350e000 thru 0x351a5c3 (0xc5c4 bytes) 2024-09-17T21:18:56.238 INFO:teuthology.orchestra.run.smithi184.stdout:1316 read 0x8435400 thru 0x843ebf7 (0x97f8 bytes) 2024-09-17T21:18:56.238 INFO:teuthology.orchestra.run.smithi184.stdout:1320 write 0xbd78000 thru 0xbd80e05 (0x8e06 bytes) 2024-09-17T21:18:56.244 INFO:teuthology.orchestra.run.smithi184.stdout:1322 trunc from 0xc99cc00 to 0x935e600 2024-09-17T21:18:56.287 INFO:teuthology.orchestra.run.smithi184.stdout:1324 read 0x20a2400 thru 0x20b1739 (0xf33a bytes) 2024-09-17T21:18:56.289 INFO:teuthology.orchestra.run.smithi184.stdout:1326 punch from 0x8e31e00 to 0x8e41c00, (0xfe00 bytes) 2024-09-17T21:18:56.289 INFO:teuthology.orchestra.run.smithi184.stdout:1333 read 0x4ecee00 thru 0x4edbd08 (0xcf09 bytes) 2024-09-17T21:18:56.294 INFO:teuthology.orchestra.run.smithi184.stdout:1334 trunc from 0x935e600 to 0x7857200 2024-09-17T21:18:56.321 INFO:teuthology.orchestra.run.smithi184.stdout:1335 punch from 0x5007c00 to 0x500c000, (0x4400 bytes) 2024-09-17T21:18:56.335 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:56.333+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:56.350 INFO:teuthology.orchestra.run.smithi184.stdout:1338 write 0xa4b7e00 thru 0xa4b802c (0x22d bytes) 2024-09-17T21:18:56.351 INFO:teuthology.orchestra.run.smithi184.stdout:1340 punch from 0x3ac5600 to 0x3ac5e00, (0x800 bytes) 2024-09-17T21:18:56.352 INFO:teuthology.orchestra.run.smithi184.stdout:1342 read 0x82cd200 thru 0x82d97bf (0xc5c0 bytes) 2024-09-17T21:18:56.352 INFO:teuthology.orchestra.run.smithi184.stdout:1343 punch from 0xa290e00 to 0xa298c00, (0x7e00 bytes) 2024-09-17T21:18:56.352 INFO:teuthology.orchestra.run.smithi184.stdout:1344 punch from 0x83da400 to 0x83ea000, (0xfc00 bytes) 2024-09-17T21:18:56.353 INFO:teuthology.orchestra.run.smithi184.stdout:1349 write 0x1b6f200 thru 0x1b7e3d4 (0xf1d5 bytes) 2024-09-17T21:18:56.353 INFO:teuthology.orchestra.run.smithi184.stdout:1350 read 0x20f6200 thru 0x20fc6e6 (0x64e7 bytes) 2024-09-17T21:18:56.365 INFO:teuthology.orchestra.run.smithi184.stdout:1351 trunc from 0xa4b8200 to 0x48d5000 2024-09-17T21:18:56.381 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:56.381 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:56.398 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:18:56.404 INFO:teuthology.orchestra.run.smithi184.stdout:1352 read 0x4758600 thru 0x475a68f (0x2090 bytes) 2024-09-17T21:18:56.407 INFO:teuthology.orchestra.run.smithi184.stdout:1357 read 0x145a00 thru 0x14b6a7 (0x5ca8 bytes) 2024-09-17T21:18:56.414 INFO:teuthology.orchestra.run.smithi184.stdout:1358 trunc from 0x48d5000 to 0x7400c00 2024-09-17T21:18:56.436 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:56.435+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:56.462 INFO:teuthology.orchestra.run.smithi184.stdout:1359 write 0xc660000 thru 0xc66779e (0x779f bytes) 2024-09-17T21:18:56.463 INFO:teuthology.orchestra.run.smithi184.stdout:1361 write 0xc56b800 thru 0xc579fb2 (0xe7b3 bytes) 2024-09-17T21:18:56.463 INFO:teuthology.orchestra.run.smithi184.stdout:1362 read 0xacb5000 thru 0xacc0d20 (0xbd21 bytes) 2024-09-17T21:18:56.464 INFO:teuthology.orchestra.run.smithi184.stdout:1364 read 0x6fb6400 thru 0x6fbd5da (0x71db bytes) 2024-09-17T21:18:56.464 INFO:teuthology.orchestra.run.smithi184.stdout:1366 clone 12 order 25 su 524288 sc 2 2024-09-17T21:18:56.537 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:56.535+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:56.564 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:56.564 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:56.578 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:18:56.639 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:56.635+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:56.700 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:56.700 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:56.714 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:18:56.739 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:56.735+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:56.837 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:56.838 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:56.840 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:56.837+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:56.852 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:18:56.942 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:56.941+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:57.015 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:57.015 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:57.029 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:18:57.076 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:57.075+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:57.143 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:57.139+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:57.159 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:57.159 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:57.174 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:18:57.260 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:57.259+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:57.337 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:57.337 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:57.345 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:57.341+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:57.353 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:18:57.432 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:18:57.446 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:57.445+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:57.521 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:18:57.521 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:18:57.527 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:18:57.529 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone11 from 0xc667800 (overlap 0x9f9e00) to 0x4fce00 2024-09-17T21:18:57.537 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:18:57.580 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:57.579+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:57.647 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:57.643+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:57.675 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:18:57.675 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:18:57.693 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:18:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:18:57.699 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:57.699 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:57.699 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:57.699 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:57.710 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:18:57.711 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:18:57.714 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:18:57.731 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:18:57.731 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:18:57.752 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:18:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:18:57.752 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:57.751+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:57.773 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:18:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:18:57.846 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:57.846 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:57.847 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:57.847 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:57.847 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:57.850 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:57.847+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:57.860 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:18:57.951 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:57.947+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:58.025 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:58.025 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:18:58.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:18:58.026 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:18:58.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.026 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:491 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:58.026 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:18:58.026 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.823716924999999, 2024-09-17T21:18:58.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041839013000000001, 2024-09-17T21:18:58.026 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.026 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.026 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:58.026 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:58.026 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:58.026 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:58.027 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 491 2024-09-17T21:18:58.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.027 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.027 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.027 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:18:58.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.027 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.027 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:18:58.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198315+0000", 2024-09-17T21:18:58.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6599999999999996e-07 2024-09-17T21:18:58.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198317+0000", 2024-09-17T21:18:58.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6639999999999998e-06 2024-09-17T21:18:58.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198327+0000", 2024-09-17T21:18:58.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5629999999999998e-06 2024-09-17T21:18:58.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198337+0000", 2024-09-17T21:18:58.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0667e-05 2024-09-17T21:18:58.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.030 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239793+0000", 2024-09-17T21:18:58.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041455519000000003 2024-09-17T21:18:58.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.030 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239929+0000", 2024-09-17T21:18:58.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135716 2024-09-17T21:18:58.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.030 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240153+0000", 2024-09-17T21:18:58.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022421799999999999 2024-09-17T21:18:58.030 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.031 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.031 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.031 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:492 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:58.031 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:18:58.031 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.823678555000001, 2024-09-17T21:18:58.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042947165000000002, 2024-09-17T21:18:58.031 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.031 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.031 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:58.031 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:58.032 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:58.032 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:58.032 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 492 2024-09-17T21:18:58.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.032 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.032 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:18:58.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:18:58.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198354+0000", 2024-09-17T21:18:58.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:18:58.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198357+0000", 2024-09-17T21:18:58.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6969999999999999e-06 2024-09-17T21:18:58.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.034 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198367+0000", 2024-09-17T21:18:58.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8430000000000005e-06 2024-09-17T21:18:58.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.034 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198374+0000", 2024-09-17T21:18:58.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9600000000000003e-06 2024-09-17T21:18:58.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240226+0000", 2024-09-17T21:18:58.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041852157000000001 2024-09-17T21:18:58.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240409+0000", 2024-09-17T21:18:58.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018244800000000001 2024-09-17T21:18:58.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241299+0000", 2024-09-17T21:18:58.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000890554 2024-09-17T21:18:58.036 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.036 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.036 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.036 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:493 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 4063232~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:58.036 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:18:58.036 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.823641469000002, 2024-09-17T21:18:58.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043897209, 2024-09-17T21:18:58.036 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.036 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.036 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:58.036 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:58.037 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:58.037 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:58.037 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 493 2024-09-17T21:18:58.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.037 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:18:58.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.038 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:18:58.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.038 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.038 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198391+0000", 2024-09-17T21:18:58.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4699999999999999e-06 2024-09-17T21:18:58.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.039 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198396+0000", 2024-09-17T21:18:58.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:18:58.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.039 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198412+0000", 2024-09-17T21:18:58.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6507e-05 2024-09-17T21:18:58.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.039 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198425+0000", 2024-09-17T21:18:58.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.256e-05 2024-09-17T21:18:58.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241383+0000", 2024-09-17T21:18:58.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042958216 2024-09-17T21:18:58.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241525+0000", 2024-09-17T21:18:58.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014204500000000001 2024-09-17T21:18:58.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242287+0000", 2024-09-17T21:18:58.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076125800000000001 2024-09-17T21:18:58.041 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.041 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.041 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.041 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:496 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:58.041 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:18:58.041 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.823551070000001, 2024-09-17T21:18:58.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043732500000000001, 2024-09-17T21:18:58.041 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.041 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.042 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:58.042 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:58.042 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:58.042 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:58.042 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 496 2024-09-17T21:18:58.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.042 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:18:58.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:18:58.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198481+0000", 2024-09-17T21:18:58.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-17T21:18:58.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198488+0000", 2024-09-17T21:18:58.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8499999999999996e-06 2024-09-17T21:18:58.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198507+0000", 2024-09-17T21:18:58.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9276999999999999e-05 2024-09-17T21:18:58.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198521+0000", 2024-09-17T21:18:58.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3376e-05 2024-09-17T21:18:58.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241339+0000", 2024-09-17T21:18:58.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042818119000000002 2024-09-17T21:18:58.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241449+0000", 2024-09-17T21:18:58.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110209 2024-09-17T21:18:58.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242212+0000", 2024-09-17T21:18:58.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076329900000000005 2024-09-17T21:18:58.046 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.046 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.046 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.046 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:497 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:58.046 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:18:58.046 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.823470121, 2024-09-17T21:18:58.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044584278999999997, 2024-09-17T21:18:58.046 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.046 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.046 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:58.046 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:58.047 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:58.047 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:58.047 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 497 2024-09-17T21:18:58.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.047 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:18:58.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:18:58.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198562+0000", 2024-09-17T21:18:58.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-17T21:18:58.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198569+0000", 2024-09-17T21:18:58.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.494e-06 2024-09-17T21:18:58.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.049 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.049 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198588+0000", 2024-09-17T21:18:58.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8955999999999999e-05 2024-09-17T21:18:58.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.050 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.050 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198621+0000", 2024-09-17T21:18:58.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3837000000000001e-05 2024-09-17T21:18:58.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242258+0000", 2024-09-17T21:18:58.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043636900999999999 2024-09-17T21:18:58.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242343+0000", 2024-09-17T21:18:58.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5048000000000003e-05 2024-09-17T21:18:58.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243145+0000", 2024-09-17T21:18:58.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00080167300000000003 2024-09-17T21:18:58.052 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.052 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.053 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.053 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:503 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:58.053 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:18:58.054 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.823385597000001, 2024-09-17T21:18:58.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044220582000000001, 2024-09-17T21:18:58.054 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.054 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.054 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:58.055 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:58.055 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:58.055 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:58.055 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 503 2024-09-17T21:18:58.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.055 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:18:58.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.057 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.057 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:18:58.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.057 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.057 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198647+0000", 2024-09-17T21:18:58.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.483e-06 2024-09-17T21:18:58.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198653+0000", 2024-09-17T21:18:58.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4529999999999999e-06 2024-09-17T21:18:58.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198672+0000", 2024-09-17T21:18:58.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9193000000000001e-05 2024-09-17T21:18:58.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198685+0000", 2024-09-17T21:18:58.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.291e-05 2024-09-17T21:18:58.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:18:58.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043672141999999997 2024-09-17T21:18:58.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.060 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.060 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.060 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242716+0000", 2024-09-17T21:18:58.060 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00035901800000000001 2024-09-17T21:18:58.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.060 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.060 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.060 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242866+0000", 2024-09-17T21:18:58.060 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000149383 2024-09-17T21:18:58.060 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.060 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.060 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.061 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:504 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:58.061 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:18:58.061 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.823325805, 2024-09-17T21:18:58.061 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044611406999999999, 2024-09-17T21:18:58.061 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.061 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.061 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:58.061 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:58.061 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:58.061 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:58.061 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 504 2024-09-17T21:18:58.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.062 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.062 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.062 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.062 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:18:58.062 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.062 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.062 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.062 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:18:58.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198706+0000", 2024-09-17T21:18:58.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:18:58.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198712+0000", 2024-09-17T21:18:58.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5770000000000001e-06 2024-09-17T21:18:58.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.064 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198729+0000", 2024-09-17T21:18:58.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6673000000000001e-05 2024-09-17T21:18:58.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.064 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.064 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198742+0000", 2024-09-17T21:18:58.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3543000000000001e-05 2024-09-17T21:18:58.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.064 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.064 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242981+0000", 2024-09-17T21:18:58.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044238953999999997 2024-09-17T21:18:58.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.065 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.065 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243155+0000", 2024-09-17T21:18:58.065 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017401600000000001 2024-09-17T21:18:58.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.065 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.065 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.065 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243316+0000", 2024-09-17T21:18:58.065 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016137399999999999 2024-09-17T21:18:58.065 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.065 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.065 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.065 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.066 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:521 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:58.066 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:18:58.066 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.822778666000001, 2024-09-17T21:18:58.066 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045043685999999999, 2024-09-17T21:18:58.066 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.066 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.066 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:58.066 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:58.066 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:58.066 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:58.066 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 521 2024-09-17T21:18:58.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.066 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:18:58.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:18:58.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199253+0000", 2024-09-17T21:18:58.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1400000000000001e-06 2024-09-17T21:18:58.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.068 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.068 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199258+0000", 2024-09-17T21:18:58.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1800000000000004e-06 2024-09-17T21:18:58.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.068 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.068 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199276+0000", 2024-09-17T21:18:58.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7686999999999999e-05 2024-09-17T21:18:58.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.069 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199289+0000", 2024-09-17T21:18:58.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3263e-05 2024-09-17T21:18:58.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.069 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.069 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243397+0000", 2024-09-17T21:18:58.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044107636999999998 2024-09-17T21:18:58.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.069 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.069 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243527+0000", 2024-09-17T21:18:58.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013038399999999999 2024-09-17T21:18:58.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.070 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.070 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.070 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244296+0000", 2024-09-17T21:18:58.070 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076839499999999995 2024-09-17T21:18:58.070 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.070 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.070 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.070 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.070 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:522 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:58.070 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:18:58.070 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.822723363000001, 2024-09-17T21:18:58.070 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046049870999999999, 2024-09-17T21:18:58.070 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.071 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.071 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:58.071 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:58.071 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:58.071 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:58.071 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 522 2024-09-17T21:18:58.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.071 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.071 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:18:58.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:18:58.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199309+0000", 2024-09-17T21:18:58.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:18:58.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199315+0000", 2024-09-17T21:18:58.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2440000000000001e-06 2024-09-17T21:18:58.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199333+0000", 2024-09-17T21:18:58.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7886000000000001e-05 2024-09-17T21:18:58.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199379+0000", 2024-09-17T21:18:58.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6383e-05 2024-09-17T21:18:58.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.075 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.075 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244394+0000", 2024-09-17T21:18:58.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045014851000000002 2024-09-17T21:18:58.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.075 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.075 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244491+0000", 2024-09-17T21:18:58.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7245999999999997e-05 2024-09-17T21:18:58.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.076 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.076 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245357+0000", 2024-09-17T21:18:58.076 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00086626800000000001 2024-09-17T21:18:58.076 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.076 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.076 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.077 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:527 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:58.077 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:18:58.077 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.822626697, 2024-09-17T21:18:58.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045812522000000001, 2024-09-17T21:18:58.077 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.077 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.078 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:58.078 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:58.078 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:58.078 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:58.078 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 527 2024-09-17T21:18:58.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.078 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:18:58.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.079 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.079 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:18:58.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.079 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.079 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199406+0000", 2024-09-17T21:18:58.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.367e-06 2024-09-17T21:18:58.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.079 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.079 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199412+0000", 2024-09-17T21:18:58.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:18:58.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199429+0000", 2024-09-17T21:18:58.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7396000000000001e-05 2024-09-17T21:18:58.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199442+0000", 2024-09-17T21:18:58.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3327e-05 2024-09-17T21:18:58.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244361+0000", 2024-09-17T21:18:58.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044918197 2024-09-17T21:18:58.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.081 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.081 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244482+0000", 2024-09-17T21:18:58.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121945 2024-09-17T21:18:58.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.081 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.081 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245217+0000", 2024-09-17T21:18:58.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00073415999999999998 2024-09-17T21:18:58.081 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.081 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.082 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.082 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:534 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:58.082 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:18:58.082 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.822567991, 2024-09-17T21:18:58.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046292505999999997, 2024-09-17T21:18:58.082 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.082 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.082 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:58.082 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:58.082 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:58.082 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:58.083 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:18:58.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.083 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.083 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.083 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:18:58.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.083 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.083 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:18:58.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.084 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199464+0000", 2024-09-17T21:18:58.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:18:58.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.084 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199473+0000", 2024-09-17T21:18:58.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5629999999999992e-06 2024-09-17T21:18:58.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.084 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199492+0000", 2024-09-17T21:18:58.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9273999999999999e-05 2024-09-17T21:18:58.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199505+0000", 2024-09-17T21:18:58.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3443e-05 2024-09-17T21:18:58.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245357+0000", 2024-09-17T21:18:58.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045851714000000002 2024-09-17T21:18:58.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245550+0000", 2024-09-17T21:18:58.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019301600000000001 2024-09-17T21:18:58.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245755+0000", 2024-09-17T21:18:58.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205226 2024-09-17T21:18:58.086 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.086 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.086 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.086 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:535 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:58.086 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:18:58.087 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.822463435, 2024-09-17T21:18:58.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046637492000000003, 2024-09-17T21:18:58.087 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.087 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.087 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:58.087 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:58.087 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:58.087 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:58.087 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 535 2024-09-17T21:18:58.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.087 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.087 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.088 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:18:58.088 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.088 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.088 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:18:58.088 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.088 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.088 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199568+0000", 2024-09-17T21:18:58.088 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.037e-06 2024-09-17T21:18:58.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199572+0000", 2024-09-17T21:18:58.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5729999999999999e-06 2024-09-17T21:18:58.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199585+0000", 2024-09-17T21:18:58.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.279e-05 2024-09-17T21:18:58.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199592+0000", 2024-09-17T21:18:58.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.61e-06 2024-09-17T21:18:58.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245873+0000", 2024-09-17T21:18:58.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046280494999999998 2024-09-17T21:18:58.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246018+0000", 2024-09-17T21:18:58.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000144697 2024-09-17T21:18:58.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.091 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.091 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246205+0000", 2024-09-17T21:18:58.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018729 2024-09-17T21:18:58.091 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.091 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.091 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.091 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:543 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:58.091 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:18:58.091 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.822368188999999, 2024-09-17T21:18:58.092 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047564097999999999, 2024-09-17T21:18:58.092 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.092 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.092 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:58.092 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:58.092 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:58.092 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:58.092 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 543 2024-09-17T21:18:58.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.092 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.092 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.093 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.093 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:18:58.093 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.093 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.093 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:18:58.093 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.093 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.093 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:18:58.093 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9699999999999995e-07 2024-09-17T21:18:58.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199677+0000", 2024-09-17T21:18:58.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:18:58.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199690+0000", 2024-09-17T21:18:58.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2977e-05 2024-09-17T21:18:58.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199705+0000", 2024-09-17T21:18:58.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4656e-05 2024-09-17T21:18:58.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246307+0000", 2024-09-17T21:18:58.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046601780000000002 2024-09-17T21:18:58.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246475+0000", 2024-09-17T21:18:58.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016834200000000001 2024-09-17T21:18:58.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.096 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.096 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247227+0000", 2024-09-17T21:18:58.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00075173299999999998 2024-09-17T21:18:58.096 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.096 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.096 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.096 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:544 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:58.096 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:18:58.096 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.822309780000001, 2024-09-17T21:18:58.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049283559999999997, 2024-09-17T21:18:58.097 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.097 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.097 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:58.097 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:58.097 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:58.097 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:58.097 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 544 2024-09-17T21:18:58.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.097 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:18:58.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:18:58.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199722+0000", 2024-09-17T21:18:58.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.08e-06 2024-09-17T21:18:58.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199738+0000", 2024-09-17T21:18:58.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5943999999999998e-05 2024-09-17T21:18:58.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199748+0000", 2024-09-17T21:18:58.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0193e-05 2024-09-17T21:18:58.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199756+0000", 2024-09-17T21:18:58.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4200000000000001e-06 2024-09-17T21:18:58.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247311+0000", 2024-09-17T21:18:58.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047555300000000002 2024-09-17T21:18:58.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247571+0000", 2024-09-17T21:18:58.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025996999999999999 2024-09-17T21:18:58.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.101 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.101 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249005+0000", 2024-09-17T21:18:58.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001433653 2024-09-17T21:18:58.101 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.101 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.101 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.101 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:549 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:58.101 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:18:58.101 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.822264155999999, 2024-09-17T21:18:58.102 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048464528999999999, 2024-09-17T21:18:58.102 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.102 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.102 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:58.102 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:58.102 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:58.102 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:58.102 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 549 2024-09-17T21:18:58.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.103 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:18:58.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:18:58.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:18:58.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:18:58.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199770+0000", 2024-09-17T21:18:58.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9969999999999999e-06 2024-09-17T21:18:58.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199780+0000", 2024-09-17T21:18:58.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0003000000000001e-05 2024-09-17T21:18:58.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199793+0000", 2024-09-17T21:18:58.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2074e-05 2024-09-17T21:18:58.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247289+0000", 2024-09-17T21:18:58.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047496691000000001 2024-09-17T21:18:58.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.106 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.106 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247437+0000", 2024-09-17T21:18:58.106 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014807899999999999 2024-09-17T21:18:58.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.106 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.106 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248231+0000", 2024-09-17T21:18:58.106 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00079390899999999998 2024-09-17T21:18:58.106 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.106 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.106 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.107 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:558 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:58.107 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:18:58.107 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.822223643000001, 2024-09-17T21:18:58.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049709821000000001, 2024-09-17T21:18:58.107 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.107 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.107 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:58.107 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:58.107 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:58.107 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:58.107 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 558 2024-09-17T21:18:58.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.107 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:18:58.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:18:58.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199809+0000", 2024-09-17T21:18:58.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.333e-06 2024-09-17T21:18:58.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199814+0000", 2024-09-17T21:18:58.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0930000000000002e-06 2024-09-17T21:18:58.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199831+0000", 2024-09-17T21:18:58.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7580000000000001e-05 2024-09-17T21:18:58.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.110 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.110 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199860+0000", 2024-09-17T21:18:58.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8609999999999999e-05 2024-09-17T21:18:58.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.110 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.110 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249106+0000", 2024-09-17T21:18:58.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049246545000000003 2024-09-17T21:18:58.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.110 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249274+0000", 2024-09-17T21:18:58.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016806699999999999 2024-09-17T21:18:58.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249517+0000", 2024-09-17T21:18:58.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024259300000000001 2024-09-17T21:18:58.111 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.111 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.111 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.111 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:559 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:58.111 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:18:58.112 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.822104281000001, 2024-09-17T21:18:58.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049990694000000002, 2024-09-17T21:18:58.112 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.112 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.112 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:58.112 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:58.112 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:58.112 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:58.112 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 559 2024-09-17T21:18:58.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.112 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:18:58.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:18:58.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199928+0000", 2024-09-17T21:18:58.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:18:58.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199935+0000", 2024-09-17T21:18:58.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5429999999999999e-06 2024-09-17T21:18:58.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199954+0000", 2024-09-17T21:18:58.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.912e-05 2024-09-17T21:18:58.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199969+0000", 2024-09-17T21:18:58.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5305999999999999e-05 2024-09-17T21:18:58.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249604+0000", 2024-09-17T21:18:58.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049634931 2024-09-17T21:18:58.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249738+0000", 2024-09-17T21:18:58.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013411599999999999 2024-09-17T21:18:58.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249917+0000", 2024-09-17T21:18:58.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179111 2024-09-17T21:18:58.116 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.116 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.116 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.116 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:560 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:18:58.116 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:18:58.117 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.822039865000001, 2024-09-17T21:18:58.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050173059999999998, 2024-09-17T21:18:58.117 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.117 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.117 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:58.118 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:58.118 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:18:58.118 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:18:58.118 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 560 2024-09-17T21:18:58.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.118 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:18:58.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.119 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.119 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:18:58.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199992+0000", 2024-09-17T21:18:58.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.517e-06 2024-09-17T21:18:58.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199998+0000", 2024-09-17T21:18:58.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1460000000000004e-06 2024-09-17T21:18:58.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:18:58.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7407000000000001e-05 2024-09-17T21:18:58.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200029+0000", 2024-09-17T21:18:58.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3499999999999999e-05 2024-09-17T21:18:58.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249968+0000", 2024-09-17T21:18:58.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049939799 2024-09-17T21:18:58.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250055+0000", 2024-09-17T21:18:58.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6731999999999996e-05 2024-09-17T21:18:58.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250164+0000", 2024-09-17T21:18:58.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108959 2024-09-17T21:18:58.123 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.123 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.123 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.123 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:18:58.123 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:18:58.123 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.212250156, 2024-09-17T21:18:58.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:18:58.123 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.123 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.123 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:18:58.123 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:58.123 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:18:58.124 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:18:58.124 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:18:58.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.124 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.124 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.124 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:18:58.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:18:58.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:18:58.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:18:58.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:18:58.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:18:58.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:18:58.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:18:58.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:18:58.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:18:58.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:18:58.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:18:58.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:18:58.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:18:58.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:18:58.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:18:58.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:18:58.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.129 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:18:58.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:18:58.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.129 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.129 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:18:58.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:18:58.129 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.129 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.129 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.129 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:18:58.130 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:18:58.130 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.6571483699999998, 2024-09-17T21:18:58.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:18:58.130 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.130 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.130 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:58.130 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:58.130 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:18:58.130 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:18:58.130 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:18:58.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.130 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:18:58.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:18:58.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:18:58.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:18:58.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:18:58.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:18:58.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:18:58.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:18:58.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.133 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.133 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:18:58.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:18:58.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.133 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.133 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:18:58.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:18:58.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:18:58.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:18:58.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:18:58.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:18:58.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:18:58.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:18:58.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:18:58.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:18:58.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:18:58.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:18:58.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:58.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:18:58.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:18:58.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:58.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:18:58.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:18:58.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:58.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:18:58.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:18:58.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:18:58.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:18:58.136 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.136 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.136 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.137 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.137 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.137 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:58.138 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-17T21:18:58.139 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:18:58.139 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:58.137+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:58.239 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:58.237+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:58.315 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:58.315 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:58.315 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:18:58.316 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:58.330 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:18:58.345 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:18:58.345 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:18:58.345 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-17T21:18:58.345 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T21:18:58.345 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T21:18:58.346 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-17T21:18:58.353 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-17T21:18:58.354 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:58.351+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:58.373 INFO:teuthology.orchestra.run.smithi184.stdout:1367 write 0x6ae9000 thru 0x6aec98e (0x398f bytes) 2024-09-17T21:18:58.418 INFO:teuthology.orchestra.run.smithi184.stdout:1368 trunc from 0xc667800 to 0x69ba600 2024-09-17T21:18:58.441 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:58.439+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:58.457 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:58.457 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:18:58.458 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:18:58.458 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:18:58.458 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:58.472 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:18:58.474 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:58.474 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T21:18:58.474 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:58.478 INFO:teuthology.orchestra.run.smithi184.stdout:1369 read 0x1c4b400 thru 0x1c55427 (0xa028 bytes) 2024-09-17T21:18:58.509 INFO:teuthology.orchestra.run.smithi184.stdout:1371 write 0x85b4800 thru 0x85bb214 (0x6a15 bytes) 2024-09-17T21:18:58.543 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:58.539+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:58.554 INFO:teuthology.orchestra.run.smithi184.stdout:1373 write 0xc6e0400 thru 0xc6e5586 (0x5187 bytes) 2024-09-17T21:18:58.556 INFO:teuthology.orchestra.run.smithi184.stdout:1376 write 0x83a8600 thru 0x83b39a5 (0xb3a6 bytes) 2024-09-17T21:18:58.558 INFO:teuthology.orchestra.run.smithi184.stdout:1377 write 0x513fa00 thru 0x514c1ea (0xc7eb bytes) 2024-09-17T21:18:58.560 INFO:teuthology.orchestra.run.smithi184.stdout:1378 read 0x1084400 thru 0x10882b4 (0x3eb5 bytes) 2024-09-17T21:18:58.561 INFO:teuthology.orchestra.run.smithi184.stdout:1382 read 0x284ca00 thru 0x285c4bd (0xfabe bytes) 2024-09-17T21:18:58.562 INFO:teuthology.orchestra.run.smithi184.stdout:1384 read 0x5f3f000 thru 0x5f46727 (0x7728 bytes) 2024-09-17T21:18:58.599 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:18:58.599 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:18:58.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:18:58.600 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:18:58.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.600 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:12 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:18:58.600 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:58.600 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 45.264218419999999, 2024-09-17T21:18:58.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061340989999999996, 2024-09-17T21:18:58.600 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.600 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.600 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:58.600 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:18:58.600 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:18:58.601 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:58.601 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12 2024-09-17T21:18:58.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.601 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.601 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:58.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:18:58.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.602 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332336+0000", 2024-09-17T21:18:58.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5519999999999999e-06 2024-09-17T21:18:58.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.602 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332342+0000", 2024-09-17T21:18:58.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7180000000000001e-06 2024-09-17T21:18:58.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332352+0000", 2024-09-17T21:18:58.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:18:58.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332359+0000", 2024-09-17T21:18:58.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3479999999999998e-06 2024-09-17T21:18:58.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332413+0000", 2024-09-17T21:18:58.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4447999999999998e-05 2024-09-17T21:18:58.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-17T21:18:58.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332417+0000", 2024-09-17T21:18:58.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.895e-06 2024-09-17T21:18:58.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.605 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334755+0000", 2024-09-17T21:18:58.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023377189999999998 2024-09-17T21:18:58.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.605 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.605 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334903+0000", 2024-09-17T21:18:58.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014778999999999999 2024-09-17T21:18:58.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.605 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:18:58.605 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.335187+0000", 2024-09-17T21:18:58.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000284715 2024-09-17T21:18:58.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:58.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.337731+0000", 2024-09-17T21:18:58.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002543741 2024-09-17T21:18:58.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:18:58.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338382+0000", 2024-09-17T21:18:58.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00065029200000000001 2024-09-17T21:18:58.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:58.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338418+0000", 2024-09-17T21:18:58.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.697e-05 2024-09-17T21:18:58.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338469+0000", 2024-09-17T21:18:58.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0154999999999997e-05 2024-09-17T21:18:58.607 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.607 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.607 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.608 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "pg_update_log_missing(3.1e epoch 28/27 rep_tid 81 entries 28'41 (0'0) error 3:7b23fd98:::rbd_header.1152c01721e5:head by client.4488.0:561 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-17T21:18:58.608 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:18:58.608 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.314944293000003, 2024-09-17T21:18:58.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0051146969999999996, 2024-09-17T21:18:58.608 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.608 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.608 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:18:58.608 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:18:58.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.609 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.609 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:18:58.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.609 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.609 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281610+0000", 2024-09-17T21:18:58.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.16e-07 2024-09-17T21:18:58.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.609 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281626+0000", 2024-09-17T21:18:58.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6039999999999999e-05 2024-09-17T21:18:58.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281666+0000", 2024-09-17T21:18:58.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0364000000000002e-05 2024-09-17T21:18:58.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281710+0000", 2024-09-17T21:18:58.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3596000000000001e-05 2024-09-17T21:18:58.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281992+0000", 2024-09-17T21:18:58.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028273099999999999 2024-09-17T21:18:58.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286723+0000", 2024-09-17T21:18:58.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047310499999999997 2024-09-17T21:18:58.611 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.611 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.612 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.612 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4488.0:562 3.1e e28/27 3:7b23fd98:::rbd_header.1152c01721e5:head v 28'42, mlcod=0'0)", 2024-09-17T21:18:58.612 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:18:58.612 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.314700504999998, 2024-09-17T21:18:58.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049275020000000003, 2024-09-17T21:18:58.612 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.612 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.612 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:58.612 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.613 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:18:58.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.613 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:18:58.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281853+0000", 2024-09-17T21:18:58.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2999999999999999e-07 2024-09-17T21:18:58.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281875+0000", 2024-09-17T21:18:58.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1386000000000002e-05 2024-09-17T21:18:58.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281892+0000", 2024-09-17T21:18:58.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7637000000000001e-05 2024-09-17T21:18:58.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281908+0000", 2024-09-17T21:18:58.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5187e-05 2024-09-17T21:18:58.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.282517+0000", 2024-09-17T21:18:58.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00060989799999999999 2024-09-17T21:18:58.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.282548+0000", 2024-09-17T21:18:58.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0910000000000001e-05 2024-09-17T21:18:58.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:58.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286729+0000", 2024-09-17T21:18:58.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0041810850000000002 2024-09-17T21:18:58.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286780+0000", 2024-09-17T21:18:58.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0569000000000003e-05 2024-09-17T21:18:58.617 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.617 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.617 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.617 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:609 3.d e28/23 3:b7278be9:::rbd_data.1152c6af092e.000000000000008f:head v 28'6, mlcod=28'4)", 2024-09-17T21:18:58.617 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:18:58.617 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.481756218000001, 2024-09-17T21:18:58.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049797280000000001, 2024-09-17T21:18:58.618 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.618 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.618 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:58.618 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:18:58.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:18:58.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114800+0000", 2024-09-17T21:18:58.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:18:58.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114915+0000", 2024-09-17T21:18:58.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115053 2024-09-17T21:18:58.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114951+0000", 2024-09-17T21:18:58.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6165999999999999e-05 2024-09-17T21:18:58.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114965+0000", 2024-09-17T21:18:58.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4364e-05 2024-09-17T21:18:58.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115097+0000", 2024-09-17T21:18:58.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013139200000000001 2024-09-17T21:18:58.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115151+0000", 2024-09-17T21:18:58.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4422000000000002e-05 2024-09-17T21:18:58.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:58.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.119633+0000", 2024-09-17T21:18:58.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044818820000000004 2024-09-17T21:18:58.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.119776+0000", 2024-09-17T21:18:58.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000143609 2024-09-17T21:18:58.622 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.622 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.622 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.622 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:782 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'63, mlcod=28'60)", 2024-09-17T21:18:58.622 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:18:58.622 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.381788190000002, 2024-09-17T21:18:58.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006167283, 2024-09-17T21:18:58.623 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.623 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.623 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:58.623 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:18:58.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:18:58.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214767+0000", 2024-09-17T21:18:58.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:18:58.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214779+0000", 2024-09-17T21:18:58.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2627e-05 2024-09-17T21:18:58.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214805+0000", 2024-09-17T21:18:58.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5570000000000001e-05 2024-09-17T21:18:58.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214823+0000", 2024-09-17T21:18:58.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8202999999999998e-05 2024-09-17T21:18:58.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215170+0000", 2024-09-17T21:18:58.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034720399999999999 2024-09-17T21:18:58.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215235+0000", 2024-09-17T21:18:58.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4276000000000006e-05 2024-09-17T21:18:58.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:58.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220859+0000", 2024-09-17T21:18:58.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056240500000000002 2024-09-17T21:18:58.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220932+0000", 2024-09-17T21:18:58.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3503000000000004e-05 2024-09-17T21:18:58.627 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.627 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.627 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.627 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:783 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'64, mlcod=28'60)", 2024-09-17T21:18:58.627 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:18:58.627 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.381101348000001, 2024-09-17T21:18:58.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0055240869999999996, 2024-09-17T21:18:58.627 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.627 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.628 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:58.628 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:18:58.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:18:58.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215454+0000", 2024-09-17T21:18:58.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0269999999999998e-06 2024-09-17T21:18:58.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215468+0000", 2024-09-17T21:18:58.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4816e-05 2024-09-17T21:18:58.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215501+0000", 2024-09-17T21:18:58.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2957000000000003e-05 2024-09-17T21:18:58.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215524+0000", 2024-09-17T21:18:58.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2782999999999999e-05 2024-09-17T21:18:58.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216675+0000", 2024-09-17T21:18:58.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011507869999999999 2024-09-17T21:18:58.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216728+0000", 2024-09-17T21:18:58.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3112999999999998e-05 2024-09-17T21:18:58.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:58.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220938+0000", 2024-09-17T21:18:58.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042097380000000002 2024-09-17T21:18:58.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220976+0000", 2024-09-17T21:18:58.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7866e-05 2024-09-17T21:18:58.632 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.632 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.632 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.632 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:785 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'65, mlcod=28'60)", 2024-09-17T21:18:58.632 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:18:58.633 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.380515299000002, 2024-09-17T21:18:58.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049795409999999997, 2024-09-17T21:18:58.633 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.633 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.633 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:58.633 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:18:58.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:18:58.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216039+0000", 2024-09-17T21:18:58.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.793e-06 2024-09-17T21:18:58.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216053+0000", 2024-09-17T21:18:58.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.345e-05 2024-09-17T21:18:58.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216092+0000", 2024-09-17T21:18:58.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9180000000000001e-05 2024-09-17T21:18:58.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216106+0000", 2024-09-17T21:18:58.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:18:58.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217569+0000", 2024-09-17T21:18:58.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001462938 2024-09-17T21:18:58.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217607+0000", 2024-09-17T21:18:58.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7267000000000001e-05 2024-09-17T21:18:58.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:58.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220980+0000", 2024-09-17T21:18:58.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003373544 2024-09-17T21:18:58.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221017+0000", 2024-09-17T21:18:58.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7116000000000002e-05 2024-09-17T21:18:58.637 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.637 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.637 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.637 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:786 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'66, mlcod=28'60)", 2024-09-17T21:18:58.637 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:18:58.637 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.379814494999998, 2024-09-17T21:18:58.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057966650000000003, 2024-09-17T21:18:58.637 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.637 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.637 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:58.638 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:18:58.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:18:58.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216740+0000", 2024-09-17T21:18:58.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5600000000000001e-06 2024-09-17T21:18:58.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216752+0000", 2024-09-17T21:18:58.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1477e-05 2024-09-17T21:18:58.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216779+0000", 2024-09-17T21:18:58.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7093e-05 2024-09-17T21:18:58.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216817+0000", 2024-09-17T21:18:58.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8275999999999997e-05 2024-09-17T21:18:58.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218387+0000", 2024-09-17T21:18:58.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015703749999999999 2024-09-17T21:18:58.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218432+0000", 2024-09-17T21:18:58.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4910000000000002e-05 2024-09-17T21:18:58.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:58.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222432+0000", 2024-09-17T21:18:58.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039997189999999997 2024-09-17T21:18:58.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222535+0000", 2024-09-17T21:18:58.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103255 2024-09-17T21:18:58.642 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.642 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.642 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.642 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:18:58.642 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:18:58.642 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.379192662999998, 2024-09-17T21:18:58.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:18:58.642 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.643 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.643 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:58.643 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:18:58.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:18:58.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:18:58.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:18:58.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:18:58.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:18:58.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:18:58.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:18:58.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:18:58.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:18:58.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:18:58.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:18:58.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:18:58.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:18:58.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:58.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:18:58.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:18:58.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:18:58.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:18:58.647 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.647 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.647 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.647 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:788 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'68, mlcod=28'60)", 2024-09-17T21:18:58.647 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:18:58.647 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.378558231, 2024-09-17T21:18:58.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058495320000000002, 2024-09-17T21:18:58.648 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.648 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.648 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:58.648 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.648 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:18:58.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.648 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:18:58.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.649 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.649 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217996+0000", 2024-09-17T21:18:58.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.376e-06 2024-09-17T21:18:58.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.649 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.649 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218006+0000", 2024-09-17T21:18:58.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.01e-05 2024-09-17T21:18:58.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.649 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.649 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218030+0000", 2024-09-17T21:18:58.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4026999999999999e-05 2024-09-17T21:18:58.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218042+0000", 2024-09-17T21:18:58.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1977000000000001e-05 2024-09-17T21:18:58.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221081+0000", 2024-09-17T21:18:58.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030385960000000002 2024-09-17T21:18:58.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221115+0000", 2024-09-17T21:18:58.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3772999999999997e-05 2024-09-17T21:18:58.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:58.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223749+0000", 2024-09-17T21:18:58.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0026340629999999999 2024-09-17T21:18:58.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223844+0000", 2024-09-17T21:18:58.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5619999999999996e-05 2024-09-17T21:18:58.651 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.651 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.652 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.652 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:790 3.5 e28/23 3:a14b1a37:::rbd_data.1152c6af092e.000000000000002f:head v 28'1, mlcod=0'0)", 2024-09-17T21:18:58.652 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:18:58.652 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.374153905, 2024-09-17T21:18:58.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059622240000000003, 2024-09-17T21:18:58.652 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.652 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.652 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:58.652 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:18:58.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.653 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:18:58.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.653 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222400+0000", 2024-09-17T21:18:58.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.02e-06 2024-09-17T21:18:58.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222411+0000", 2024-09-17T21:18:58.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.044e-05 2024-09-17T21:18:58.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222440+0000", 2024-09-17T21:18:58.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.932e-05 2024-09-17T21:18:58.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222458+0000", 2024-09-17T21:18:58.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8615999999999999e-05 2024-09-17T21:18:58.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222609+0000", 2024-09-17T21:18:58.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015102299999999999 2024-09-17T21:18:58.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222657+0000", 2024-09-17T21:18:58.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7206000000000002e-05 2024-09-17T21:18:58.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:58.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228278+0000", 2024-09-17T21:18:58.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056208100000000004 2024-09-17T21:18:58.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228361+0000", 2024-09-17T21:18:58.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3788999999999993e-05 2024-09-17T21:18:58.656 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.656 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.656 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:884 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'76, mlcod=28'70)", 2024-09-17T21:18:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:18:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.327569023999999, 2024-09-17T21:18:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056451990000000001, 2024-09-17T21:18:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:18:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:18:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268986+0000", 2024-09-17T21:18:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9360000000000002e-06 2024-09-17T21:18:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269001+0000", 2024-09-17T21:18:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4684e-05 2024-09-17T21:18:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269026+0000", 2024-09-17T21:18:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5193000000000001e-05 2024-09-17T21:18:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269043+0000", 2024-09-17T21:18:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7306e-05 2024-09-17T21:18:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.270024+0000", 2024-09-17T21:18:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00098120600000000009 2024-09-17T21:18:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.270085+0000", 2024-09-17T21:18:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0486000000000003e-05 2024-09-17T21:18:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.274557+0000", 2024-09-17T21:18:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044723419999999998 2024-09-17T21:18:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.274629+0000", 2024-09-17T21:18:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2045999999999994e-05 2024-09-17T21:18:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:18:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:18:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 35.728891998999998, 2024-09-17T21:18:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:18:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:18:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:18:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:18:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:18:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:18:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:18:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:18:58.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:18:58.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:18:58.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:18:58.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:18:58.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:18:58.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:18:58.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:18:58.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:18:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:18:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:18:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:18:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:18:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:18:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 35.727626915999998, 2024-09-17T21:18:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:18:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:18:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:18:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:18:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:18:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:18:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:18:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:18:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:18:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:18:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:18:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:18:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:18:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:18:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:18:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:18:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:18:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:18:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:18:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:18:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:18:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 35.717847505999998, 2024-09-17T21:18:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:18:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:18:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:18:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:18:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:18:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:18:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:18:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:18:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:18:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:18:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:18:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:18:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:18:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:18:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:18:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:18:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:18:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:18:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:18:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:18:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:18:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 35.709125102999998, 2024-09-17T21:18:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:18:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:18:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:18:58.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:18:58.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:18:58.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:18:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:18:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:18:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:18:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:18:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:18:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:18:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:18:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:18:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:18:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:18:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:18:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:18:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:18:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4498.0:596 3.4 e32/23 3:20f262ea:::rbd_data.1152b51cb396.000000000000000d:head v 32'17, mlcod=30'15)", 2024-09-17T21:18:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.258354+0000", 2024-09-17T21:18:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 32.338199238999998, 2024-09-17T21:18:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0053727619999999997, 2024-09-17T21:18:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.258354+0000", 2024-09-17T21:18:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.258354+0000", 2024-09-17T21:18:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.258356+0000", 2024-09-17T21:18:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.798e-06 2024-09-17T21:18:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259198+0000", 2024-09-17T21:18:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084279199999999998 2024-09-17T21:18:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259229+0000", 2024-09-17T21:18:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0383999999999999e-05 2024-09-17T21:18:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259241+0000", 2024-09-17T21:18:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2139e-05 2024-09-17T21:18:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259341+0000", 2024-09-17T21:18:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010049 2024-09-17T21:18:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259380+0000", 2024-09-17T21:18:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9072000000000001e-05 2024-09-17T21:18:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.263668+0000", 2024-09-17T21:18:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042875389999999999 2024-09-17T21:18:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.263726+0000", 2024-09-17T21:18:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8548000000000003e-05 2024-09-17T21:18:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4520.0:6177 3.8 e34/27 3:1665de58:::rbd_data.11528c3b0529.0000000000000019:head v 34'43, mlcod=34'41)", 2024-09-17T21:18:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:30.509962+0000", 2024-09-17T21:18:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.086591249000001, 2024-09-17T21:18:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049320850000000001, 2024-09-17T21:18:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509962+0000", 2024-09-17T21:18:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509962+0000", 2024-09-17T21:18:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509964+0000", 2024-09-17T21:18:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.356e-06 2024-09-17T21:18:58.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509978+0000", 2024-09-17T21:18:58.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3434e-05 2024-09-17T21:18:58.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510001+0000", 2024-09-17T21:18:58.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3436000000000001e-05 2024-09-17T21:18:58.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510023+0000", 2024-09-17T21:18:58.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1739999999999999e-05 2024-09-17T21:18:58.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510193+0000", 2024-09-17T21:18:58.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017040600000000001 2024-09-17T21:18:58.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510250+0000", 2024-09-17T21:18:58.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6749000000000002e-05 2024-09-17T21:18:58.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:58.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.514823+0000", 2024-09-17T21:18:58.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0045728180000000002 2024-09-17T21:18:58.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.514894+0000", 2024-09-17T21:18:58.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1145999999999999e-05 2024-09-17T21:18:58.691 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.691 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.691 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.692 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:18:58.692 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:18:58.692 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.324034538999999, 2024-09-17T21:18:58.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:18:58.692 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.692 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.692 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:58.693 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:18:58.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:18:58.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:18:58.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:18:58.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:18:58.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:18:58.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:18:58.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:18:58.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:18:58.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:18:58.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:18:58.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:18:58.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:18:58.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:18:58.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:58.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:18:58.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:18:58.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:18:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:18:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4863.0:165 3.5 e52/23 3:a772dfb1:::rbd_data.11523651b015.0000000000000001:head v 52'103, mlcod=47'101)", 2024-09-17T21:18:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:50.682666+0000", 2024-09-17T21:18:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 7.913886733, 2024-09-17T21:18:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0055448839999999999, 2024-09-17T21:18:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:18:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:18:58.698 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:58.698 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:18:58.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:18:58.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.682666+0000", 2024-09-17T21:18:58.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:18:58.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.682666+0000", 2024-09-17T21:18:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:18:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:18:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.682667+0000", 2024-09-17T21:18:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.16e-07 2024-09-17T21:18:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:18:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.683332+0000", 2024-09-17T21:18:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000664476 2024-09-17T21:18:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:18:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.683360+0000", 2024-09-17T21:18:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7988e-05 2024-09-17T21:18:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:18:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.683376+0000", 2024-09-17T21:18:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6578999999999998e-05 2024-09-17T21:18:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:18:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.683522+0000", 2024-09-17T21:18:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014536400000000001 2024-09-17T21:18:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:18:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.683560+0000", 2024-09-17T21:18:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8005e-05 2024-09-17T21:18:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:18:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.688080+0000", 2024-09-17T21:18:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0045201390000000003 2024-09-17T21:18:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:18:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:18:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:18:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.688211+0000", 2024-09-17T21:18:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013141699999999999 2024-09-17T21:18:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:18:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:18:58.702 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:18:58.703 INFO:teuthology.orchestra.run.smithi184.stdout:1385 trunc from 0xc6e5600 to 0x16dc00 2024-09-17T21:18:58.704 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:18:58.705 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:58.701+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:58.765 INFO:teuthology.orchestra.run.smithi184.stdout:1392 trunc from 0x16dc00 to 0xa6f200 2024-09-17T21:18:58.785 INFO:teuthology.orchestra.run.smithi184.stdout:1393 read 0x5d6a00 thru 0x5e52ca (0xe8cb bytes) 2024-09-17T21:18:58.804 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:18:58.803+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:18:58.811 INFO:teuthology.orchestra.run.smithi184.stdout:1394 write 0x2de9600 thru 0x2dec784 (0x3185 bytes) 2024-09-17T21:18:58.814 INFO:teuthology.orchestra.run.smithi184.stdout:1398 read 0x884400 thru 0x88a271 (0x5e72 bytes) 2024-09-17T21:18:58.846 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:58.846 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:58.846 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:18:58.846 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:58.854 INFO:teuthology.orchestra.run.smithi184.stdout:1399 write 0xa9f7600 thru 0xaa00e85 (0x9886 bytes) 2024-09-17T21:18:58.855 INFO:teuthology.orchestra.run.smithi184.stdout:1402 read 0x576d600 thru 0x5776ed4 (0x98d5 bytes) 2024-09-17T21:18:58.855 INFO:teuthology.orchestra.run.smithi184.stdout:1403 read 0x5cf3c00 thru 0x5cfc380 (0x8781 bytes) 2024-09-17T21:18:58.856 INFO:teuthology.orchestra.run.smithi184.stdout:1405 read 0x1340200 thru 0x134e7f7 (0xe5f8 bytes) 2024-09-17T21:18:58.858 INFO:teuthology.orchestra.run.smithi184.stdout:1407 read 0x8889400 thru 0x888bee9 (0x2aea bytes) 2024-09-17T21:18:58.858 INFO:teuthology.orchestra.run.smithi184.stdout:1408 clone 13 order 18 su 65536 sc 3 2024-09-17T21:18:58.862 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:18:58.905 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:58.903+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:58.986 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:58.987 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:58.987 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:18:58.987 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:18:58.987 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:59.000 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:18:59.050 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:59.049+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:59.107 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:18:59.105+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:18:59.127 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:59.128 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:18:59.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:18:59.128 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:59.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.128 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:18:59.128 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:18:59.128 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 62.110730689999997, 2024-09-17T21:18:59.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:18:59.128 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.128 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.128 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:59.129 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.129 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:18:59.129 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:18:59.129 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:18:59.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.129 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:18:59.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:18:59.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:18:59.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:18:59.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:18:59.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:18:59.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:18:59.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:18:59.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:18:59.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:18:59.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:18:59.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:18:59.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:59.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:18:59.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:18:59.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:18:59.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:18:59.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:18:59.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:18:59.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:18:59.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:18:59.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:18:59.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:59.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:18:59.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:18:59.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:59.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:18:59.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:18:59.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:59.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:18:59.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:18:59.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:18:59.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:18:59.134 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.135 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.135 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.135 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:18:59.135 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:18:59.135 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 42.530566409999999, 2024-09-17T21:18:59.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:18:59.135 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.135 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.135 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.135 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.135 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:18:59.135 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:59.136 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:18:59.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.136 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:18:59.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:18:59.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:18:59.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:18:59.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:18:59.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:18:59.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:18:59.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:18:59.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:18:59.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:18:59.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:18:59.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:18:59.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:59.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:18:59.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:18:59.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:18:59.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:18:59.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:59.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:18:59.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:18:59.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:18:59.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:18:59.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:18:59.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:18:59.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:18:59.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:18:59.141 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.141 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.141 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.141 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:18:59.141 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:18:59.141 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 42.530469097999998, 2024-09-17T21:18:59.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:18:59.141 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.141 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.141 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.141 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.142 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:59.142 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:59.142 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:18:59.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.142 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:18:59.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:18:59.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:18:59.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:18:59.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:18:59.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:18:59.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:18:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:18:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:18:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:18:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:18:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:18:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:18:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:18:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:18:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:18:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:18:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:18:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:18:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:18:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:18:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:18:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:18:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:18:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:18:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:18:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:18:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:18:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:18:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.148 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.148 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:18:59.148 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:18:59.148 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 42.528382297, 2024-09-17T21:18:59.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:18:59.148 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.148 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.148 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:59.148 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.148 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:18:59.148 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:18:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:18:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:18:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:18:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:18:59.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:18:59.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:18:59.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:18:59.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:18:59.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:18:59.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:18:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:18:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:18:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:18:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:18:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:18:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:18:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:18:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:18:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:18:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:18:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:18:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:18:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:18:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:18:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:18:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:18:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:18:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:18:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:18:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:18:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:18:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:18:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:18:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:18:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:282 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:18:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.896677036, 2024-09-17T21:18:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023571801, 2024-09-17T21:18:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 282 2024-09-17T21:18:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:18:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:18:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227820+0000", 2024-09-17T21:18:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.11e-06 2024-09-17T21:18:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227825+0000", 2024-09-17T21:18:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8099999999999997e-06 2024-09-17T21:18:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227848+0000", 2024-09-17T21:18:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2949999999999999e-05 2024-09-17T21:18:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227862+0000", 2024-09-17T21:18:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:18:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.247977+0000", 2024-09-17T21:18:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020114968 2024-09-17T21:18:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251141+0000", 2024-09-17T21:18:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031640689999999998 2024-09-17T21:18:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251391+0000", 2024-09-17T21:18:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025018499999999999 2024-09-17T21:18:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:283 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:18:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.896615196999999, 2024-09-17T21:18:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024022274999999999, 2024-09-17T21:18:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 283 2024-09-17T21:18:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:18:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:18:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227882+0000", 2024-09-17T21:18:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-06 2024-09-17T21:18:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227887+0000", 2024-09-17T21:18:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1769999999999997e-06 2024-09-17T21:18:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227905+0000", 2024-09-17T21:18:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7235999999999999e-05 2024-09-17T21:18:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227918+0000", 2024-09-17T21:18:59.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3387000000000001e-05 2024-09-17T21:18:59.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251469+0000", 2024-09-17T21:18:59.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023550894999999999 2024-09-17T21:18:59.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251669+0000", 2024-09-17T21:18:59.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019992499999999999 2024-09-17T21:18:59.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251903+0000", 2024-09-17T21:18:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000234412 2024-09-17T21:18:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:284 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:18:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.896559644, 2024-09-17T21:18:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024450084, 2024-09-17T21:18:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 284 2024-09-17T21:18:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:18:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:18:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227937+0000", 2024-09-17T21:18:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6700000000000002e-07 2024-09-17T21:18:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227943+0000", 2024-09-17T21:18:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:18:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227961+0000", 2024-09-17T21:18:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.827e-05 2024-09-17T21:18:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227975+0000", 2024-09-17T21:18:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.394e-05 2024-09-17T21:18:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251977+0000", 2024-09-17T21:18:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024001340999999999 2024-09-17T21:18:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252169+0000", 2024-09-17T21:18:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019271199999999999 2024-09-17T21:18:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252387+0000", 2024-09-17T21:18:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021726100000000001 2024-09-17T21:18:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:285 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:18:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.896503354, 2024-09-17T21:18:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024884791, 2024-09-17T21:18:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 285 2024-09-17T21:18:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:18:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:18:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227994+0000", 2024-09-17T21:18:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1429999999999999e-06 2024-09-17T21:18:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227999+0000", 2024-09-17T21:18:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1669999999999998e-06 2024-09-17T21:18:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228015+0000", 2024-09-17T21:18:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6056e-05 2024-09-17T21:18:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228028+0000", 2024-09-17T21:18:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3196e-05 2024-09-17T21:18:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252458+0000", 2024-09-17T21:18:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024429751999999999 2024-09-17T21:18:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252650+0000", 2024-09-17T21:18:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019187800000000001 2024-09-17T21:18:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252878+0000", 2024-09-17T21:18:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022759899999999999 2024-09-17T21:18:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:286 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:18:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.896447595000001, 2024-09-17T21:18:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025299318000000001, 2024-09-17T21:18:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 286 2024-09-17T21:18:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:18:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:18:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228050+0000", 2024-09-17T21:18:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:18:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228055+0000", 2024-09-17T21:18:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4960000000000004e-06 2024-09-17T21:18:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228075+0000", 2024-09-17T21:18:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0109999999999999e-05 2024-09-17T21:18:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228089+0000", 2024-09-17T21:18:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4154e-05 2024-09-17T21:18:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252952+0000", 2024-09-17T21:18:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024862934 2024-09-17T21:18:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253141+0000", 2024-09-17T21:18:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000188782 2024-09-17T21:18:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253348+0000", 2024-09-17T21:18:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020684499999999999 2024-09-17T21:18:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:443 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:18:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.88286029, 2024-09-17T21:18:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023179569000000001, 2024-09-17T21:18:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 443 2024-09-17T21:18:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:18:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:18:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241637+0000", 2024-09-17T21:18:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.46e-06 2024-09-17T21:18:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241643+0000", 2024-09-17T21:18:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1970000000000001e-06 2024-09-17T21:18:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241662+0000", 2024-09-17T21:18:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8712999999999999e-05 2024-09-17T21:18:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241676+0000", 2024-09-17T21:18:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.393e-05 2024-09-17T21:18:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264596+0000", 2024-09-17T21:18:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022919688000000001 2024-09-17T21:18:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264690+0000", 2024-09-17T21:18:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4145000000000001e-05 2024-09-17T21:18:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264815+0000", 2024-09-17T21:18:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012543600000000001 2024-09-17T21:18:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:444 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:18:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.882799240000001, 2024-09-17T21:18:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023260383999999999, 2024-09-17T21:18:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 444 2024-09-17T21:18:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:18:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:18:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241698+0000", 2024-09-17T21:18:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2270000000000001e-06 2024-09-17T21:18:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241704+0000", 2024-09-17T21:18:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9800000000000003e-06 2024-09-17T21:18:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241725+0000", 2024-09-17T21:18:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0812999999999999e-05 2024-09-17T21:18:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241739+0000", 2024-09-17T21:18:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3773e-05 2024-09-17T21:18:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264850+0000", 2024-09-17T21:18:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023111326000000001 2024-09-17T21:18:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264897+0000", 2024-09-17T21:18:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6638999999999998e-05 2024-09-17T21:18:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264957+0000", 2024-09-17T21:18:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0625999999999999e-05 2024-09-17T21:18:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:445 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:18:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.882738571000001, 2024-09-17T21:18:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023315144999999999, 2024-09-17T21:18:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 445 2024-09-17T21:18:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:18:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:18:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241759+0000", 2024-09-17T21:18:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-06 2024-09-17T21:18:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241764+0000", 2024-09-17T21:18:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5899999999999998e-06 2024-09-17T21:18:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241784+0000", 2024-09-17T21:18:59.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9204000000000001e-05 2024-09-17T21:18:59.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241798+0000", 2024-09-17T21:18:59.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3953e-05 2024-09-17T21:18:59.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264977+0000", 2024-09-17T21:18:59.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023179182 2024-09-17T21:18:59.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.265018+0000", 2024-09-17T21:18:59.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1171999999999998e-05 2024-09-17T21:18:59.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.265073+0000", 2024-09-17T21:18:59.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4741000000000003e-05 2024-09-17T21:18:59.193 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.194 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.194 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.194 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:679 3.13 3:cd0d74f9:::rbd_data.11527824718.000000000000000a:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.194 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:18:59.194 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.873817150000001, 2024-09-17T21:18:59.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023284978000000001, 2024-09-17T21:18:59.194 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.194 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.194 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.194 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.194 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.194 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.195 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 679 2024-09-17T21:18:59.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.195 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:18:59.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:18:59.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250680+0000", 2024-09-17T21:18:59.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:18:59.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250685+0000", 2024-09-17T21:18:59.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1970000000000004e-06 2024-09-17T21:18:59.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250703+0000", 2024-09-17T21:18:59.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7396000000000001e-05 2024-09-17T21:18:59.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250716+0000", 2024-09-17T21:18:59.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.29e-05 2024-09-17T21:18:59.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273740+0000", 2024-09-17T21:18:59.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023024405000000001 2024-09-17T21:18:59.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273834+0000", 2024-09-17T21:18:59.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4124999999999998e-05 2024-09-17T21:18:59.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273964+0000", 2024-09-17T21:18:59.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012963499999999999 2024-09-17T21:18:59.198 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.198 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.198 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.198 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:680 3.13 3:cd0d74f9:::rbd_data.11527824718.000000000000000a:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.198 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:18:59.198 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.873762070000002, 2024-09-17T21:18:59.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023687792999999999, 2024-09-17T21:18:59.199 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.199 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.199 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.199 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.199 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.199 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.199 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 680 2024-09-17T21:18:59.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.199 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:18:59.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:18:59.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250735+0000", 2024-09-17T21:18:59.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2669999999999999e-06 2024-09-17T21:18:59.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250740+0000", 2024-09-17T21:18:59.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8060000000000002e-06 2024-09-17T21:18:59.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250757+0000", 2024-09-17T21:18:59.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7303e-05 2024-09-17T21:18:59.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250771+0000", 2024-09-17T21:18:59.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3876999999999999e-05 2024-09-17T21:18:59.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274022+0000", 2024-09-17T21:18:59.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023251135999999999 2024-09-17T21:18:59.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274211+0000", 2024-09-17T21:18:59.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018813599999999999 2024-09-17T21:18:59.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274422+0000", 2024-09-17T21:18:59.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000211268 2024-09-17T21:18:59.203 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.203 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.203 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.203 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:18:59.203 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:18:59.203 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.8444835160000004, 2024-09-17T21:18:59.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:18:59.203 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.203 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.203 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.204 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.204 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:18:59.204 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:18:59.204 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:18:59.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.204 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:18:59.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:18:59.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:18:59.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:18:59.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:18:59.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:18:59.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:18:59.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:18:59.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:18:59.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:18:59.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:18:59.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:18:59.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:18:59.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:18:59.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:18:59.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:18:59.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:18:59.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:59.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:18:59.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:18:59.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:18:59.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:18:59.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:18:59.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:18:59.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:18:59.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:18:59.209 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.209 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.209 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.209 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:18:59.209 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:18:59.209 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.8353918050000004, 2024-09-17T21:18:59.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:18:59.209 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.209 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.210 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.210 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.210 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:18:59.210 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:18:59.210 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:18:59.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.210 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:18:59.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:18:59.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:18:59.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:18:59.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:18:59.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:18:59.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:18:59.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:18:59.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:18:59.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:18:59.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:18:59.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:18:59.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:59.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:18:59.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:18:59.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:18:59.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:18:59.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:18:59.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:18:59.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:18:59.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:18:59.214 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.214 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.214 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.214 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:18:59.214 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:18:59.215 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.8352197540000006, 2024-09-17T21:18:59.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:18:59.215 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.215 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.215 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.215 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.215 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:18:59.215 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:18:59.215 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:18:59.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.215 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:18:59.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:18:59.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:18:59.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:18:59.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:18:59.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:18:59.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:18:59.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:18:59.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:18:59.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:18:59.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:18:59.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:18:59.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:59.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:18:59.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:18:59.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:18:59.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:18:59.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:18:59.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:18:59.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:18:59.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:18:59.220 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.220 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.220 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.220 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:18:59.220 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:18:59.220 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.6602870070000009, 2024-09-17T21:18:59.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:18:59.220 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.220 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.220 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.220 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.220 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:59.221 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:59.221 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:18:59.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.221 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:18:59.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:18:59.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:18:59.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:18:59.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:18:59.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:18:59.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:18:59.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:18:59.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:18:59.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:18:59.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:18:59.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:18:59.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:59.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:18:59.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:18:59.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:18:59.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:18:59.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:18:59.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:18:59.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:18:59.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:18:59.225 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.225 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.225 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.225 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:18:59.225 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:18:59.225 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.6602024499999999, 2024-09-17T21:18:59.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:18:59.226 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.226 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.226 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.226 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.226 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:59.226 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:59.226 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:18:59.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.226 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:18:59.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:18:59.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:18:59.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:18:59.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:18:59.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:18:59.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:18:59.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:18:59.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:18:59.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:18:59.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:18:59.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:18:59.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:59.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:18:59.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:18:59.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:18:59.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:18:59.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:18:59.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:18:59.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:18:59.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:18:59.230 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.230 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.230 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.231 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:18:59.231 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:18:59.231 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.6601368129999994, 2024-09-17T21:18:59.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:18:59.231 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.231 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.231 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.231 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.231 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:18:59.231 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:18:59.231 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:18:59.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.231 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:18:59.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:18:59.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:18:59.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:18:59.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:18:59.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:18:59.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:18:59.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:18:59.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:18:59.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:18:59.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:18:59.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:18:59.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:18:59.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:18:59.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:18:59.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:18:59.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:18:59.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:18:59.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:18:59.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:18:59.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:18:59.236 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.236 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.236 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.236 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.236 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.236 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:59.237 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:18:59.238 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:18:59.233+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:18:59.338 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:59.337+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:59.415 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:59.416 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:59.416 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:18:59.416 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:59.430 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:18:59.440 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:59.435+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:59.541 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:59.539+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:59.551 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone12 from 0xaa01000 (overlap 0x16dc00) to 0x92400 2024-09-17T21:18:59.607 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:59.608 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:18:59.608 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:18:59.608 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:18:59.608 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:59.622 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:18:59.666 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:59.665+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:59.742 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:18:59.739+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:18:59.747 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:18:59.747 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:18:59.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:18:59.747 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:18:59.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.747 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:478 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 720896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.747 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:18:59.747 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.511138417000002, 2024-09-17T21:18:59.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016845300000000001, 2024-09-17T21:18:59.748 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.748 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.748 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.748 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.748 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.748 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.748 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 478 2024-09-17T21:18:59.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.748 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:18:59.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:18:59.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233785+0000", 2024-09-17T21:18:59.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.116e-06 2024-09-17T21:18:59.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233803+0000", 2024-09-17T21:18:59.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7932999999999999e-05 2024-09-17T21:18:59.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233822+0000", 2024-09-17T21:18:59.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9276999999999999e-05 2024-09-17T21:18:59.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233836+0000", 2024-09-17T21:18:59.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4053000000000001e-05 2024-09-17T21:18:59.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.248579+0000", 2024-09-17T21:18:59.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014743529 2024-09-17T21:18:59.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.248795+0000", 2024-09-17T21:18:59.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021535200000000001 2024-09-17T21:18:59.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250629+0000", 2024-09-17T21:18:59.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018340400000000001 2024-09-17T21:18:59.752 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.752 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.752 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.752 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:479 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 851968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.752 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:18:59.752 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.511047331, 2024-09-17T21:18:59.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017383900000000001, 2024-09-17T21:18:59.752 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.752 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.753 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.753 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.753 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.753 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.753 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 479 2024-09-17T21:18:59.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.753 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:18:59.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:18:59.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233876+0000", 2024-09-17T21:18:59.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2160000000000001e-06 2024-09-17T21:18:59.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233883+0000", 2024-09-17T21:18:59.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9199999999999998e-06 2024-09-17T21:18:59.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233900+0000", 2024-09-17T21:18:59.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7034e-05 2024-09-17T21:18:59.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233913+0000", 2024-09-17T21:18:59.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159000000000001e-05 2024-09-17T21:18:59.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250712+0000", 2024-09-17T21:18:59.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016798644000000001 2024-09-17T21:18:59.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250913+0000", 2024-09-17T21:18:59.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020127900000000001 2024-09-17T21:18:59.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251259+0000", 2024-09-17T21:18:59.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00034564799999999999 2024-09-17T21:18:59.756 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.756 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.756 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.756 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:480 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.756 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:18:59.756 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.510954485999999, 2024-09-17T21:18:59.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017902482000000001, 2024-09-17T21:18:59.757 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.757 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.757 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.757 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.757 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.757 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.757 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 480 2024-09-17T21:18:59.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.757 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:18:59.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:18:59.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233969+0000", 2024-09-17T21:18:59.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.454e-06 2024-09-17T21:18:59.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233981+0000", 2024-09-17T21:18:59.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1593000000000001e-05 2024-09-17T21:18:59.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233999+0000", 2024-09-17T21:18:59.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.806e-05 2024-09-17T21:18:59.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234012+0000", 2024-09-17T21:18:59.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:18:59.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251339+0000", 2024-09-17T21:18:59.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017326352 2024-09-17T21:18:59.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251534+0000", 2024-09-17T21:18:59.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000195842 2024-09-17T21:18:59.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251870+0000", 2024-09-17T21:18:59.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033551400000000003 2024-09-17T21:18:59.761 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.761 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.761 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.761 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:481 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1114112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.761 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:18:59.761 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.510886118999998, 2024-09-17T21:18:59.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018442621999999999, 2024-09-17T21:18:59.761 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.761 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.761 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.761 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.762 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.762 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.762 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 481 2024-09-17T21:18:59.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.762 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:18:59.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:18:59.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234037+0000", 2024-09-17T21:18:59.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:18:59.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234043+0000", 2024-09-17T21:18:59.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.49e-06 2024-09-17T21:18:59.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234061+0000", 2024-09-17T21:18:59.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8416000000000001e-05 2024-09-17T21:18:59.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234076+0000", 2024-09-17T21:18:59.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.456e-05 2024-09-17T21:18:59.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251948+0000", 2024-09-17T21:18:59.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017872774000000001 2024-09-17T21:18:59.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252144+0000", 2024-09-17T21:18:59.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019517300000000001 2024-09-17T21:18:59.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252478+0000", 2024-09-17T21:18:59.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033482899999999998 2024-09-17T21:18:59.765 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.765 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.765 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.766 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:482 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.766 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:18:59.766 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.510827567, 2024-09-17T21:18:59.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018988026000000002, 2024-09-17T21:18:59.766 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.766 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.766 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.766 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 482 2024-09-17T21:18:59.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.766 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:18:59.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:18:59.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234096+0000", 2024-09-17T21:18:59.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2139999999999999e-06 2024-09-17T21:18:59.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234101+0000", 2024-09-17T21:18:59.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7100000000000004e-06 2024-09-17T21:18:59.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234118+0000", 2024-09-17T21:18:59.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6486000000000001e-05 2024-09-17T21:18:59.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234131+0000", 2024-09-17T21:18:59.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2824e-05 2024-09-17T21:18:59.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252552+0000", 2024-09-17T21:18:59.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018421549999999998 2024-09-17T21:18:59.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252746+0000", 2024-09-17T21:18:59.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019424399999999999 2024-09-17T21:18:59.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253082+0000", 2024-09-17T21:18:59.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000335998 2024-09-17T21:18:59.770 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.770 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.770 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.770 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:483 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.770 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:18:59.770 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.510772947, 2024-09-17T21:18:59.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019462882000000001, 2024-09-17T21:18:59.770 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.770 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.770 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.770 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.771 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.771 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.771 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 483 2024-09-17T21:18:59.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.771 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:18:59.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:18:59.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234150+0000", 2024-09-17T21:18:59.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.127e-06 2024-09-17T21:18:59.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234155+0000", 2024-09-17T21:18:59.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:18:59.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234171+0000", 2024-09-17T21:18:59.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6160000000000001e-05 2024-09-17T21:18:59.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234185+0000", 2024-09-17T21:18:59.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3370000000000001e-05 2024-09-17T21:18:59.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253158+0000", 2024-09-17T21:18:59.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018972968999999999 2024-09-17T21:18:59.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253349+0000", 2024-09-17T21:18:59.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000190733 2024-09-17T21:18:59.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253612+0000", 2024-09-17T21:18:59.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026337000000000002 2024-09-17T21:18:59.774 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.774 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.774 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.774 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:484 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.774 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:18:59.774 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.510718920999999, 2024-09-17T21:18:59.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020029261999999999, 2024-09-17T21:18:59.775 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.775 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.775 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.775 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.775 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.775 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.775 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 484 2024-09-17T21:18:59.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.775 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:18:59.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:18:59.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234205+0000", 2024-09-17T21:18:59.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6470000000000001e-06 2024-09-17T21:18:59.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234210+0000", 2024-09-17T21:18:59.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:18:59.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234227+0000", 2024-09-17T21:18:59.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.73e-05 2024-09-17T21:18:59.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234241+0000", 2024-09-17T21:18:59.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3757000000000001e-05 2024-09-17T21:18:59.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253690+0000", 2024-09-17T21:18:59.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019449368000000002 2024-09-17T21:18:59.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253898+0000", 2024-09-17T21:18:59.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000207161 2024-09-17T21:18:59.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254232+0000", 2024-09-17T21:18:59.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033474299999999998 2024-09-17T21:18:59.778 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.778 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.778 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.779 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:485 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1638400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.779 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:18:59.779 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.510661693999999, 2024-09-17T21:18:59.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020553914, 2024-09-17T21:18:59.779 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.779 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.779 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.779 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.779 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.779 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.779 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 485 2024-09-17T21:18:59.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.779 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:18:59.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:18:59.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234261+0000", 2024-09-17T21:18:59.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.136e-06 2024-09-17T21:18:59.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234266+0000", 2024-09-17T21:18:59.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0669999999999999e-06 2024-09-17T21:18:59.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234284+0000", 2024-09-17T21:18:59.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7839999999999999e-05 2024-09-17T21:18:59.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234298+0000", 2024-09-17T21:18:59.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3852999999999999e-05 2024-09-17T21:18:59.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254311+0000", 2024-09-17T21:18:59.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020012973 2024-09-17T21:18:59.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254552+0000", 2024-09-17T21:18:59.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024116400000000001 2024-09-17T21:18:59.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254814+0000", 2024-09-17T21:18:59.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000261881 2024-09-17T21:18:59.783 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.783 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.783 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.783 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:486 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.783 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:18:59.783 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.510605304999999, 2024-09-17T21:18:59.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021010240999999999, 2024-09-17T21:18:59.783 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.783 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.783 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.784 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.784 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.784 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.784 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 486 2024-09-17T21:18:59.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.784 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:18:59.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:18:59.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234318+0000", 2024-09-17T21:18:59.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.37e-06 2024-09-17T21:18:59.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234324+0000", 2024-09-17T21:18:59.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:18:59.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234341+0000", 2024-09-17T21:18:59.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7127000000000001e-05 2024-09-17T21:18:59.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234354+0000", 2024-09-17T21:18:59.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3210000000000001e-05 2024-09-17T21:18:59.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254896+0000", 2024-09-17T21:18:59.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020541835000000001 2024-09-17T21:18:59.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255092+0000", 2024-09-17T21:18:59.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019634700000000001 2024-09-17T21:18:59.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255327+0000", 2024-09-17T21:18:59.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023475899999999999 2024-09-17T21:18:59.787 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.787 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.787 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.787 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:487 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.788 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:18:59.788 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.510549108999999, 2024-09-17T21:18:59.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021453911999999999, 2024-09-17T21:18:59.788 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.788 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.788 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.788 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.788 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.788 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.788 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 487 2024-09-17T21:18:59.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.788 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:18:59.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:18:59.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234374+0000", 2024-09-17T21:18:59.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3039999999999999e-06 2024-09-17T21:18:59.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234380+0000", 2024-09-17T21:18:59.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:18:59.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234397+0000", 2024-09-17T21:18:59.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7232999999999999e-05 2024-09-17T21:18:59.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234411+0000", 2024-09-17T21:18:59.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3784000000000001e-05 2024-09-17T21:18:59.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255406+0000", 2024-09-17T21:18:59.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020995204 2024-09-17T21:18:59.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255595+0000", 2024-09-17T21:18:59.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018903299999999999 2024-09-17T21:18:59.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255827+0000", 2024-09-17T21:18:59.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023190099999999999 2024-09-17T21:18:59.791 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.792 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.792 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.792 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:488 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.792 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:18:59.792 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.510493432000001, 2024-09-17T21:18:59.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026887966999999999, 2024-09-17T21:18:59.792 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.792 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.792 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.792 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.792 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.793 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.793 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 488 2024-09-17T21:18:59.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.793 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:18:59.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:18:59.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234430+0000", 2024-09-17T21:18:59.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1829999999999999e-06 2024-09-17T21:18:59.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234435+0000", 2024-09-17T21:18:59.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9869999999999999e-06 2024-09-17T21:18:59.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234451+0000", 2024-09-17T21:18:59.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6458999999999999e-05 2024-09-17T21:18:59.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234465+0000", 2024-09-17T21:18:59.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3730000000000001e-05 2024-09-17T21:18:59.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255908+0000", 2024-09-17T21:18:59.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021443413000000001 2024-09-17T21:18:59.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256098+0000", 2024-09-17T21:18:59.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018961 2024-09-17T21:18:59.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261317+0000", 2024-09-17T21:18:59.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052185850000000004 2024-09-17T21:18:59.796 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.796 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.796 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.796 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:489 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2162688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.796 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:18:59.796 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.510437592999999, 2024-09-17T21:18:59.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022632815000000001, 2024-09-17T21:18:59.797 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.797 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.797 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.797 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.797 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.797 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.797 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 489 2024-09-17T21:18:59.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.797 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:18:59.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:18:59.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234486+0000", 2024-09-17T21:18:59.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5099999999999999e-06 2024-09-17T21:18:59.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234491+0000", 2024-09-17T21:18:59.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.147e-06 2024-09-17T21:18:59.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234508+0000", 2024-09-17T21:18:59.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7405999999999999e-05 2024-09-17T21:18:59.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234522+0000", 2024-09-17T21:18:59.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3677e-05 2024-09-17T21:18:59.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256536+0000", 2024-09-17T21:18:59.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022013464999999999 2024-09-17T21:18:59.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256752+0000", 2024-09-17T21:18:59.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021637800000000001 2024-09-17T21:18:59.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257117+0000", 2024-09-17T21:18:59.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000365232 2024-09-17T21:18:59.800 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.800 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.800 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.801 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:490 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2293760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.801 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:18:59.801 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.510382802999999, 2024-09-17T21:18:59.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023146462999999999, 2024-09-17T21:18:59.801 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.801 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.801 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.801 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.801 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.801 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.801 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 490 2024-09-17T21:18:59.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.802 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:18:59.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:18:59.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234541+0000", 2024-09-17T21:18:59.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4869999999999999e-06 2024-09-17T21:18:59.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234559+0000", 2024-09-17T21:18:59.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8105999999999999e-05 2024-09-17T21:18:59.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234577+0000", 2024-09-17T21:18:59.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.836e-05 2024-09-17T21:18:59.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234592+0000", 2024-09-17T21:18:59.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4766000000000001e-05 2024-09-17T21:18:59.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257187+0000", 2024-09-17T21:18:59.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022594725 2024-09-17T21:18:59.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257374+0000", 2024-09-17T21:18:59.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018767299999999999 2024-09-17T21:18:59.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257686+0000", 2024-09-17T21:18:59.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000311346 2024-09-17T21:18:59.805 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.805 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.805 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.805 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:491 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.805 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:18:59.805 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.510310193999999, 2024-09-17T21:18:59.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023696872000000001, 2024-09-17T21:18:59.805 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.805 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.805 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.806 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.806 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.806 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.806 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 491 2024-09-17T21:18:59.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.806 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:18:59.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:18:59.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234613+0000", 2024-09-17T21:18:59.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-06 2024-09-17T21:18:59.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234618+0000", 2024-09-17T21:18:59.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3839999999999996e-06 2024-09-17T21:18:59.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234635+0000", 2024-09-17T21:18:59.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.685e-05 2024-09-17T21:18:59.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234648+0000", 2024-09-17T21:18:59.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.313e-05 2024-09-17T21:18:59.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257780+0000", 2024-09-17T21:18:59.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023131946 2024-09-17T21:18:59.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257985+0000", 2024-09-17T21:18:59.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020468400000000001 2024-09-17T21:18:59.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258309+0000", 2024-09-17T21:18:59.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032357499999999999 2024-09-17T21:18:59.809 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.809 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.809 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.809 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:492 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.810 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:18:59.810 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.510254607, 2024-09-17T21:18:59.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024258994999999998, 2024-09-17T21:18:59.810 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.810 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.810 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.810 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.810 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.810 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.810 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 492 2024-09-17T21:18:59.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.810 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:18:59.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.811 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:18:59.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.811 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234669+0000", 2024-09-17T21:18:59.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.313e-06 2024-09-17T21:18:59.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234674+0000", 2024-09-17T21:18:59.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1599999999999997e-06 2024-09-17T21:18:59.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234690+0000", 2024-09-17T21:18:59.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.666e-05 2024-09-17T21:18:59.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234704+0000", 2024-09-17T21:18:59.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3043e-05 2024-09-17T21:18:59.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258397+0000", 2024-09-17T21:18:59.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023693204999999998 2024-09-17T21:18:59.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258593+0000", 2024-09-17T21:18:59.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019578999999999999 2024-09-17T21:18:59.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258926+0000", 2024-09-17T21:18:59.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033382399999999998 2024-09-17T21:18:59.814 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.814 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.814 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.814 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:493 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.814 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:18:59.814 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.510198142, 2024-09-17T21:18:59.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024793461999999999, 2024-09-17T21:18:59.814 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.815 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.815 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.815 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.815 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.815 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.815 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 493 2024-09-17T21:18:59.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.815 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:18:59.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:18:59.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234725+0000", 2024-09-17T21:18:59.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.24e-06 2024-09-17T21:18:59.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234731+0000", 2024-09-17T21:18:59.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8939999999999998e-06 2024-09-17T21:18:59.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234749+0000", 2024-09-17T21:18:59.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7863000000000002e-05 2024-09-17T21:18:59.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234762+0000", 2024-09-17T21:18:59.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3322999999999999e-05 2024-09-17T21:18:59.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259022+0000", 2024-09-17T21:18:59.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024259691999999999 2024-09-17T21:18:59.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259259+0000", 2024-09-17T21:18:59.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023669199999999999 2024-09-17T21:18:59.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259517+0000", 2024-09-17T21:18:59.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025875799999999999 2024-09-17T21:18:59.818 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.818 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.819 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.819 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:494 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.819 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:18:59.819 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.510140442000001, 2024-09-17T21:18:59.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025251952000000001, 2024-09-17T21:18:59.819 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.819 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.819 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.819 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.820 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.820 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.820 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 494 2024-09-17T21:18:59.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.820 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:18:59.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:18:59.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234783+0000", 2024-09-17T21:18:59.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2440000000000001e-06 2024-09-17T21:18:59.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234789+0000", 2024-09-17T21:18:59.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:18:59.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234806+0000", 2024-09-17T21:18:59.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7450000000000001e-05 2024-09-17T21:18:59.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234819+0000", 2024-09-17T21:18:59.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3326000000000001e-05 2024-09-17T21:18:59.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259585+0000", 2024-09-17T21:18:59.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024765579999999999 2024-09-17T21:18:59.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259787+0000", 2024-09-17T21:18:59.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202444 2024-09-17T21:18:59.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260033+0000", 2024-09-17T21:18:59.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024611499999999998 2024-09-17T21:18:59.823 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.823 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.824 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.824 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:495 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.824 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:18:59.824 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.510085079, 2024-09-17T21:18:59.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025776342000000001, 2024-09-17T21:18:59.824 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.824 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.824 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.824 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.824 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.824 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.824 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 495 2024-09-17T21:18:59.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.825 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:18:59.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:18:59.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234838+0000", 2024-09-17T21:18:59.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2130000000000001e-06 2024-09-17T21:18:59.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234843+0000", 2024-09-17T21:18:59.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9899999999999997e-06 2024-09-17T21:18:59.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234860+0000", 2024-09-17T21:18:59.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6973000000000002e-05 2024-09-17T21:18:59.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234873+0000", 2024-09-17T21:18:59.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3179999999999999e-05 2024-09-17T21:18:59.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260135+0000", 2024-09-17T21:18:59.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025262118 2024-09-17T21:18:59.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260364+0000", 2024-09-17T21:18:59.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022830699999999999 2024-09-17T21:18:59.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260613+0000", 2024-09-17T21:18:59.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024956099999999998 2024-09-17T21:18:59.828 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.828 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.828 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.828 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:496 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.828 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:18:59.828 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.510031799, 2024-09-17T21:18:59.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026376773999999999, 2024-09-17T21:18:59.828 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.828 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.829 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:18:59.829 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.829 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.829 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.829 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 496 2024-09-17T21:18:59.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.829 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:18:59.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:18:59.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234891+0000", 2024-09-17T21:18:59.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1230000000000001e-06 2024-09-17T21:18:59.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234896+0000", 2024-09-17T21:18:59.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9599999999999999e-06 2024-09-17T21:18:59.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234913+0000", 2024-09-17T21:18:59.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6747000000000001e-05 2024-09-17T21:18:59.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234926+0000", 2024-09-17T21:18:59.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-05 2024-09-17T21:18:59.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260675+0000", 2024-09-17T21:18:59.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025748979000000002 2024-09-17T21:18:59.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260883+0000", 2024-09-17T21:18:59.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020848399999999999 2024-09-17T21:18:59.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261267+0000", 2024-09-17T21:18:59.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038345099999999998 2024-09-17T21:18:59.832 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.833 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.833 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.833 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:682 3.9 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2455552~12288,write 8126464~10240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:18:59.833 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:18:59.833 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.476094748000001, 2024-09-17T21:18:59.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018447261999999999, 2024-09-17T21:18:59.833 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:18:59.833 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:18:59.834 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:18:59.834 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:18:59.834 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:18:59.834 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:18:59.834 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 682 2024-09-17T21:18:59.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.834 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:18:59.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:18:59.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:18:59.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:18:59.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:18:59.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:18:59.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:18:59.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268830+0000", 2024-09-17T21:18:59.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6419999999999999e-06 2024-09-17T21:18:59.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:18:59.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271673+0000", 2024-09-17T21:18:59.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028432610000000001 2024-09-17T21:18:59.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:18:59.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271700+0000", 2024-09-17T21:18:59.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7200999999999999e-05 2024-09-17T21:18:59.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:18:59.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271715+0000", 2024-09-17T21:18:59.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5145e-05 2024-09-17T21:18:59.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:18:59.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271785+0000", 2024-09-17T21:18:59.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9519999999999998e-05 2024-09-17T21:18:59.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:18:59.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271918+0000", 2024-09-17T21:18:59.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013330899999999999 2024-09-17T21:18:59.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:18:59.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.272449+0000", 2024-09-17T21:18:59.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00053051200000000002 2024-09-17T21:18:59.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:18:59.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.278822+0000", 2024-09-17T21:18:59.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0063731619999999999 2024-09-17T21:18:59.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:18:59.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287167+0000", 2024-09-17T21:18:59.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083452149999999996 2024-09-17T21:18:59.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:18:59.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287213+0000", 2024-09-17T21:18:59.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5972e-05 2024-09-17T21:18:59.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:18:59.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:18:59.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:18:59.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287274+0000", 2024-09-17T21:18:59.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1322999999999995e-05 2024-09-17T21:18:59.839 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.839 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.839 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.839 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:18:59.839 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:18:59.839 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:18:59.839 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:18:59.841 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:18:59.886 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:18:59.885+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:18:59.944 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:18:59.943+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:18:59.999 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:18:59.999 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:00.017 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:00.017 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:00.024 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:00.024 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:00.024 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:19:00.024 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:00.034 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:00.034 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:00.038 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:19:00.053 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:00.053 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:00.071 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:00.071 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:00.086 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:00.085+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:00.088 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:00.089 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:00.127 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-17T21:19:00.149 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:00.147+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:00.164 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:00.164 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:00.179 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:19:00.253 INFO:teuthology.orchestra.run.smithi184.stdout:1409 trunc from 0xaa01000 to 0x67c6e00 2024-09-17T21:19:00.266 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:00.265+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:00.289 INFO:teuthology.orchestra.run.smithi184.stdout:1411 read 0x3e34000 thru 0x3e42f4a (0xef4b bytes) 2024-09-17T21:19:00.310 INFO:teuthology.orchestra.run.smithi184.stdout:1414 write 0xa07f600 thru 0xa08a0de (0xaadf bytes) 2024-09-17T21:19:00.348 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:00.343+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:00.349 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:00.349 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:00.353 INFO:teuthology.orchestra.run.smithi184.stdout:1415 trunc from 0xa08a200 to 0x31daa00 2024-09-17T21:19:00.364 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:19:00.378 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:19:00.408 INFO:teuthology.orchestra.run.smithi184.stdout:1417 punch from 0x7c8000 to 0x7d7a00, (0xfa00 bytes) 2024-09-17T21:19:00.412 INFO:teuthology.orchestra.run.smithi184.stdout:1418 trunc from 0x31daa00 to 0x3065000 2024-09-17T21:19:00.452 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:00.451+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:00.462 INFO:teuthology.orchestra.run.smithi184.stdout:1424 trunc from 0x3065000 to 0x6bf6a00 2024-09-17T21:19:00.486 INFO:teuthology.orchestra.run.smithi184.stdout:1427 read 0x6aadc00 thru 0x6aba04d (0xc44e bytes) 2024-09-17T21:19:00.488 INFO:teuthology.orchestra.run.smithi184.stdout:1430 read 0x613000 thru 0x615ef7 (0x2ef8 bytes) 2024-09-17T21:19:00.504 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:00.504 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:00.519 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:19:00.556 INFO:teuthology.orchestra.run.smithi184.stdout:1431 write 0xd1c4000 thru 0xd1ca0e4 (0x60e5 bytes) 2024-09-17T21:19:00.564 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:00.563+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:00.565 INFO:teuthology.orchestra.run.smithi184.stdout:1432 trunc from 0xd1ca200 to 0xd97ce00 2024-09-17T21:19:00.596 INFO:teuthology.orchestra.run.smithi184.stdout:1435 read 0x57d3c00 thru 0x57e12b9 (0xd6ba bytes) 2024-09-17T21:19:00.600 INFO:teuthology.orchestra.run.smithi184.stdout:1437 write 0xa26e000 thru 0xa278cc9 (0xacca bytes) 2024-09-17T21:19:00.600 INFO:teuthology.orchestra.run.smithi184.stdout:1439 write 0x1050400 thru 0x105db09 (0xd70a bytes) 2024-09-17T21:19:00.602 INFO:teuthology.orchestra.run.smithi184.stdout:1441 read 0x351e000 thru 0x351e40b (0x40c bytes) 2024-09-17T21:19:00.602 INFO:teuthology.orchestra.run.smithi184.stdout:1444 write 0x4d98800 thru 0x4da731d (0xeb1e bytes) 2024-09-17T21:19:00.602 INFO:teuthology.orchestra.run.smithi184.stdout:1449 read 0x5a0fe00 thru 0x5a1e76f (0xe970 bytes) 2024-09-17T21:19:00.603 INFO:teuthology.orchestra.run.smithi184.stdout:1450 punch from 0x7ca7a00 to 0x7cab600, (0x3c00 bytes) 2024-09-17T21:19:00.612 INFO:teuthology.orchestra.run.smithi184.stdout:1453 trunc from 0xd97ce00 to 0x9627c00 2024-09-17T21:19:00.649 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:00.647+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:00.657 INFO:teuthology.orchestra.run.smithi184.stdout:1456 write 0x3661e00 thru 0x366a136 (0x8337 bytes) 2024-09-17T21:19:00.664 INFO:teuthology.orchestra.run.smithi184.stdout:1457 trunc from 0x9627c00 to 0x7966400 2024-09-17T21:19:00.688 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:00.688 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:00.703 INFO:teuthology.orchestra.run.smithi184.stdout:1458 punch from 0x264400 to 0x274000, (0xfc00 bytes) 2024-09-17T21:19:00.704 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:19:00.719 INFO:teuthology.orchestra.run.smithi184.stdout:1460 trunc from 0x7966400 to 0x2cfc600 2024-09-17T21:19:00.750 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:00.745+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:00.801 INFO:teuthology.orchestra.run.smithi184.stdout:1465 write 0x6295600 thru 0x629fb40 (0xa541 bytes) 2024-09-17T21:19:00.803 INFO:teuthology.orchestra.run.smithi184.stdout:1467 read 0x2908a00 thru 0x291183e (0x8e3f bytes) 2024-09-17T21:19:00.803 INFO:teuthology.orchestra.run.smithi184.stdout:1468 read 0x41cac00 thru 0x41d9cbe (0xf0bf bytes) 2024-09-17T21:19:00.804 INFO:teuthology.orchestra.run.smithi184.stdout:1469 punch from 0x4197600 to 0x41a0e00, (0x9800 bytes) 2024-09-17T21:19:00.805 INFO:teuthology.orchestra.run.smithi184.stdout:1471 write 0x17e6e00 thru 0x17e874f (0x1950 bytes) 2024-09-17T21:19:00.850 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:00.847+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:00.853 INFO:teuthology.orchestra.run.smithi184.stdout:1472 write 0xcfcb800 thru 0xcfd2825 (0x7026 bytes) 2024-09-17T21:19:00.854 INFO:teuthology.orchestra.run.smithi184.stdout:1473 write 0xa031200 thru 0xa032c85 (0x1a86 bytes) 2024-09-17T21:19:00.854 INFO:teuthology.orchestra.run.smithi184.stdout:1477 write 0x65d9200 thru 0x65da30d (0x110e bytes) 2024-09-17T21:19:00.854 INFO:teuthology.orchestra.run.smithi184.stdout:1479 read 0x36e200 thru 0x37130c (0x310d bytes) 2024-09-17T21:19:00.855 INFO:teuthology.orchestra.run.smithi184.stdout:1480 write 0x7e27c00 thru 0x7e2e8f3 (0x6cf4 bytes) 2024-09-17T21:19:00.855 INFO:teuthology.orchestra.run.smithi184.stdout:1482 punch from 0x208b200 to 0x2096000, (0xae00 bytes) 2024-09-17T21:19:00.857 INFO:teuthology.orchestra.run.smithi184.stdout:1484 flatten 2024-09-17T21:19:00.870 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:00.870 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:00.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:19:00.950 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:00.949+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:01.051 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:01.047+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:01.054 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:01.054 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:01.069 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:19:01.082 INFO:teuthology.orchestra.run.smithi184.stdout:1489 read 0x87b3400 thru 0x87baed8 (0x7ad9 bytes) 2024-09-17T21:19:01.085 INFO:teuthology.orchestra.run.smithi184.stdout:1490 punch from 0xb202600 to 0xb20b600, (0x9000 bytes) 2024-09-17T21:19:01.085 INFO:teuthology.orchestra.run.smithi184.stdout:1492 read 0x9602000 thru 0x960d006 (0xb007 bytes) 2024-09-17T21:19:01.086 INFO:teuthology.orchestra.run.smithi184.stdout:1493 write 0x9c66600 thru 0x9c67617 (0x1018 bytes) 2024-09-17T21:19:01.086 INFO:teuthology.orchestra.run.smithi184.stdout:1495 write 0x6b9f200 thru 0x6ba9976 (0xa777 bytes) 2024-09-17T21:19:01.086 INFO:teuthology.orchestra.run.smithi184.stdout:1496 read 0x4d73200 thru 0x4d7c442 (0x9243 bytes) 2024-09-17T21:19:01.087 INFO:teuthology.orchestra.run.smithi184.stdout:1497 read 0x6fed800 thru 0x6ff9de7 (0xc5e8 bytes) 2024-09-17T21:19:01.087 INFO:teuthology.orchestra.run.smithi184.stdout:1499 read 0x787200 thru 0x78f0db (0x7edc bytes) 2024-09-17T21:19:01.087 INFO:teuthology.orchestra.run.smithi184.stdout:1500 punch from 0x297ec00 to 0x297f400, (0x800 bytes) 2024-09-17T21:19:01.087 INFO:teuthology.orchestra.run.smithi184.stdout:1503 write 0x8569e00 thru 0x856afb2 (0x11b3 bytes) 2024-09-17T21:19:01.088 INFO:teuthology.orchestra.run.smithi184.stdout:1504 punch from 0xc56a000 to 0xc570a00, (0x6a00 bytes) 2024-09-17T21:19:01.088 INFO:teuthology.orchestra.run.smithi184.stdout:1505 read 0x73a6c00 thru 0x73aa6e0 (0x3ae1 bytes) 2024-09-17T21:19:01.088 INFO:teuthology.orchestra.run.smithi184.stdout:1506 write 0x83f7a00 thru 0x83fadfe (0x33ff bytes) 2024-09-17T21:19:01.094 INFO:teuthology.orchestra.run.smithi184.stdout:1507 trunc from 0xcfd2a00 to 0xa3e8a00 2024-09-17T21:19:01.143 INFO:teuthology.orchestra.run.smithi184.stdout:1508 write 0x9ad4c00 thru 0x9adc0b3 (0x74b4 bytes) 2024-09-17T21:19:01.145 INFO:teuthology.orchestra.run.smithi184.stdout:1509 read 0x74e3e00 thru 0x74f13d1 (0xd5d2 bytes) 2024-09-17T21:19:01.145 INFO:teuthology.orchestra.run.smithi184.stdout:1510 write 0x1399200 thru 0x139dfe5 (0x4de6 bytes) 2024-09-17T21:19:01.145 INFO:teuthology.orchestra.run.smithi184.stdout:1512 write 0x7ebc200 thru 0x7ec2da0 (0x6ba1 bytes) 2024-09-17T21:19:01.146 INFO:teuthology.orchestra.run.smithi184.stdout:1515 write 0x62c7a00 thru 0x62d74fd (0xfafe bytes) 2024-09-17T21:19:01.148 INFO:teuthology.orchestra.run.smithi184.stdout:1516 read 0x4de0600 thru 0x4de3c14 (0x3615 bytes) 2024-09-17T21:19:01.148 INFO:teuthology.orchestra.run.smithi184.stdout:1517 read 0x615c200 thru 0x6160fab (0x4dac bytes) 2024-09-17T21:19:01.152 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:01.149+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:01.181 INFO:teuthology.orchestra.run.smithi184.stdout:1518 write 0xc1ee600 thru 0xc1fdd86 (0xf787 bytes) 2024-09-17T21:19:01.184 INFO:teuthology.orchestra.run.smithi184.stdout:1521 read 0x7675600 thru 0x7679d9b (0x479c bytes) 2024-09-17T21:19:01.184 INFO:teuthology.orchestra.run.smithi184.stdout:1525 read 0x8aa3c00 thru 0x8aa8204 (0x4605 bytes) 2024-09-17T21:19:01.184 INFO:teuthology.orchestra.run.smithi184.stdout:1530 read 0xb8a2600 thru 0xb8b1a18 (0xf419 bytes) 2024-09-17T21:19:01.185 INFO:teuthology.orchestra.run.smithi184.stdout:1533 read 0x65a1a00 thru 0x65aeeb5 (0xd4b6 bytes) 2024-09-17T21:19:01.185 INFO:teuthology.orchestra.run.smithi184.stdout:1536 read 0x8207e00 thru 0x82108bb (0x8abc bytes) 2024-09-17T21:19:01.186 INFO:teuthology.orchestra.run.smithi184.stdout:1538 write 0x1908800 thru 0x191265c (0x9e5d bytes) 2024-09-17T21:19:01.186 INFO:teuthology.orchestra.run.smithi184.stdout:1539 read 0x9a04a00 thru 0x9a06ab7 (0x20b8 bytes) 2024-09-17T21:19:01.189 INFO:teuthology.orchestra.run.smithi184.stdout:1541 trunc from 0xc1fde00 to 0xac57600 2024-09-17T21:19:01.238 INFO:teuthology.orchestra.run.smithi184.stdout:1542 write 0x141b800 thru 0x141d7d3 (0x1fd4 bytes) 2024-09-17T21:19:01.239 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:01.239 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:01.241 INFO:teuthology.orchestra.run.smithi184.stdout:1544 read 0x360d000 thru 0x36146da (0x76db bytes) 2024-09-17T21:19:01.252 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:19:01.254 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:01.253+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:01.278 INFO:teuthology.orchestra.run.smithi184.stdout:1545 write 0xd980600 thru 0xd981c58 (0x1659 bytes) 2024-09-17T21:19:01.279 INFO:teuthology.orchestra.run.smithi184.stdout:1546 write 0x915a00 thru 0x923b0e (0xe10f bytes) 2024-09-17T21:19:01.280 INFO:teuthology.orchestra.run.smithi184.stdout:1547 read 0x99a9600 thru 0x99aa38c (0xd8d bytes) 2024-09-17T21:19:01.280 INFO:teuthology.orchestra.run.smithi184.stdout:1548 write 0x2b95e00 thru 0x2b9927e (0x347f bytes) 2024-09-17T21:19:01.280 INFO:teuthology.orchestra.run.smithi184.stdout:1550 write 0xb27d200 thru 0xb2894c2 (0xc2c3 bytes) 2024-09-17T21:19:01.280 INFO:teuthology.orchestra.run.smithi184.stdout:1556 punch from 0x4c2f800 to 0x4c3c200, (0xca00 bytes) 2024-09-17T21:19:01.281 INFO:teuthology.orchestra.run.smithi184.stdout:1558 write 0x8ba00 thru 0x98589 (0xcb8a bytes) 2024-09-17T21:19:01.283 INFO:teuthology.orchestra.run.smithi184.stdout:1561 write 0x93bd000 thru 0x93be7c2 (0x17c3 bytes) 2024-09-17T21:19:01.283 INFO:teuthology.orchestra.run.smithi184.stdout:1562 read 0xc0abc00 thru 0xc0b976e (0xdb6f bytes) 2024-09-17T21:19:01.283 INFO:teuthology.orchestra.run.smithi184.stdout:1563 write 0x4b0c600 thru 0x4b19089 (0xca8a bytes) 2024-09-17T21:19:01.288 INFO:teuthology.orchestra.run.smithi184.stdout:1564 trunc from 0xd981e00 to 0xb675e00 2024-09-17T21:19:01.338 INFO:teuthology.orchestra.run.smithi184.stdout:1567 write 0x77c9600 thru 0x77d21cc (0x8bcd bytes) 2024-09-17T21:19:01.339 INFO:teuthology.orchestra.run.smithi184.stdout:1570 read 0xab95e00 thru 0xaba2c15 (0xce16 bytes) 2024-09-17T21:19:01.354 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:01.351+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:01.364 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:01.364 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:01.378 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:19:01.379 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-17T21:19:01.397 INFO:teuthology.orchestra.run.smithi184.stdout:1572 write 0xe1e8800 thru 0xe1f3173 (0xa974 bytes) 2024-09-17T21:19:01.403 INFO:teuthology.orchestra.run.smithi184.stdout:1573 read 0x5296000 thru 0x52a0470 (0xa471 bytes) 2024-09-17T21:19:01.403 INFO:teuthology.orchestra.run.smithi184.stdout:1575 read 0x883d800 thru 0x88496a9 (0xbeaa bytes) 2024-09-17T21:19:01.403 INFO:teuthology.orchestra.run.smithi184.stdout:1579 read 0x46b5600 thru 0x46c1698 (0xc099 bytes) 2024-09-17T21:19:01.404 INFO:teuthology.orchestra.run.smithi184.stdout:1580 write 0x8100000 thru 0x8104a6c (0x4a6d bytes) 2024-09-17T21:19:01.404 INFO:teuthology.orchestra.run.smithi184.stdout:1583 read 0x43d0c00 thru 0x43dc6d6 (0xbad7 bytes) 2024-09-17T21:19:01.404 INFO:teuthology.orchestra.run.smithi184.stdout:1584 write 0xc129400 thru 0xc12ff62 (0x6b63 bytes) 2024-09-17T21:19:01.405 INFO:teuthology.orchestra.run.smithi184.stdout:1587 punch from 0x94dd800 to 0x94e5a00, (0x8200 bytes) 2024-09-17T21:19:01.405 INFO:teuthology.orchestra.run.smithi184.stdout:1590 read 0xf44800 thru 0xf48d5d (0x455e bytes) 2024-09-17T21:19:01.421 INFO:teuthology.orchestra.run.smithi184.stdout:1591 trunc from 0xe1f3200 to 0x8604c00 2024-09-17T21:19:01.455 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:01.451+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:01.503 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:01.503 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:01.514 INFO:teuthology.orchestra.run.smithi184.stdout:1592 punch from 0x5b62a00 to 0x5b72800, (0xfe00 bytes) 2024-09-17T21:19:01.515 INFO:teuthology.orchestra.run.smithi184.stdout:1594 read 0x658e800 thru 0x658eda3 (0x5a4 bytes) 2024-09-17T21:19:01.515 INFO:teuthology.orchestra.run.smithi184.stdout:1596 read 0x2907600 thru 0x291356f (0xbf70 bytes) 2024-09-17T21:19:01.515 INFO:teuthology.orchestra.run.smithi184.stdout:1598 read 0x47aa00 thru 0x486c90 (0xc291 bytes) 2024-09-17T21:19:01.516 INFO:teuthology.orchestra.run.smithi184.stdout:1600 punch from 0x6fe3600 to 0x6ff2e00, (0xf800 bytes) 2024-09-17T21:19:01.516 INFO:teuthology.orchestra.run.smithi184.stdout:1601 read 0x57eee00 thru 0x57f48e2 (0x5ae3 bytes) 2024-09-17T21:19:01.517 INFO:teuthology.orchestra.run.smithi184.stdout:1605 punch from 0x4925c00 to 0x4928800, (0x2c00 bytes) 2024-09-17T21:19:01.517 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:19:01.550 INFO:teuthology.orchestra.run.smithi184.stdout:1607 write 0x9de2200 thru 0x9de3b83 (0x1984 bytes) 2024-09-17T21:19:01.552 INFO:teuthology.orchestra.run.smithi184.stdout:1608 write 0x8f87400 thru 0x8f8aa01 (0x3602 bytes) 2024-09-17T21:19:01.552 INFO:teuthology.orchestra.run.smithi184.stdout:1609 write 0x3fab600 thru 0x3fba418 (0xee19 bytes) 2024-09-17T21:19:01.560 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:01.559+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:01.592 INFO:teuthology.orchestra.run.smithi184.stdout:1610 write 0xd383600 thru 0xd385d14 (0x2715 bytes) 2024-09-17T21:19:01.617 INFO:teuthology.orchestra.run.smithi184.stdout:1612 trunc from 0xd385e00 to 0x35c1200 2024-09-17T21:19:01.656 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:01.655+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:01.681 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:01.681 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:01.697 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:19:01.771 INFO:teuthology.orchestra.run.smithi184.stdout:1613 write 0xd259000 thru 0xd2612de (0x82df bytes) 2024-09-17T21:19:01.775 INFO:teuthology.orchestra.run.smithi184.stdout:1614 read 0x4b09400 thru 0x4b1680e (0xd40f bytes) 2024-09-17T21:19:01.792 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:01.791+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:01.794 INFO:teuthology.orchestra.run.smithi184.stdout:1615 trunc from 0xd261400 to 0x590e600 2024-09-17T21:19:01.858 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:01.857+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:01.867 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:01.867 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:01.878 INFO:teuthology.orchestra.run.smithi184.stdout:1618 write 0x6247c00 thru 0x624b8f8 (0x3cf9 bytes) 2024-09-17T21:19:01.881 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:19:01.917 INFO:teuthology.orchestra.run.smithi184.stdout:1619 write 0xacfcc00 thru 0xacff2c8 (0x26c9 bytes) 2024-09-17T21:19:01.941 INFO:teuthology.orchestra.run.smithi184.stdout:1620 write 0xafde600 thru 0xafe6ecc (0x88cd bytes) 2024-09-17T21:19:01.943 INFO:teuthology.orchestra.run.smithi184.stdout:1623 read 0x5a66a00 thru 0x5a6de70 (0x7471 bytes) 2024-09-17T21:19:01.944 INFO:teuthology.orchestra.run.smithi184.stdout:1626 trunc from 0xafe7000 to 0xa8eb800 2024-09-17T21:19:01.959 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:01.957+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:01.980 INFO:teuthology.orchestra.run.smithi184.stdout:1630 read 0x73f1e00 thru 0x73fd0ea (0xb2eb bytes) 2024-09-17T21:19:01.982 INFO:teuthology.orchestra.run.smithi184.stdout:1631 punch from 0xa111000 to 0xa11a000, (0x9000 bytes) 2024-09-17T21:19:01.982 INFO:teuthology.orchestra.run.smithi184.stdout:1632 write 0x87b8c00 thru 0x87c29d9 (0x9dda bytes) 2024-09-17T21:19:01.982 INFO:teuthology.orchestra.run.smithi184.stdout:1633 punch from 0x665e000 to 0x666c400, (0xe400 bytes) 2024-09-17T21:19:01.983 INFO:teuthology.orchestra.run.smithi184.stdout:1636 read 0x7737000 thru 0x773b424 (0x4425 bytes) 2024-09-17T21:19:01.983 INFO:teuthology.orchestra.run.smithi184.stdout:1638 read 0x7611800 thru 0x76123db (0xbdc bytes) 2024-09-17T21:19:01.983 INFO:teuthology.orchestra.run.smithi184.stdout:1639 write 0x1763600 thru 0x176751a (0x3f1b bytes) 2024-09-17T21:19:01.983 INFO:teuthology.orchestra.run.smithi184.stdout:1640 read 0x9522800 thru 0x95320ab (0xf8ac bytes) 2024-09-17T21:19:01.983 INFO:teuthology.orchestra.run.smithi184.stdout:1641 write 0xe48200 thru 0xe4a059 (0x1e5a bytes) 2024-09-17T21:19:01.994 INFO:teuthology.orchestra.run.smithi184.stdout:1642 trunc from 0xa8eb800 to 0x57f4000 2024-09-17T21:19:02.029 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:02.030 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:02.043 INFO:teuthology.orchestra.run.smithi184.stdout:1645 write 0x5577400 thru 0x55857cb (0xe3cc bytes) 2024-09-17T21:19:02.045 INFO:teuthology.orchestra.run.smithi184.stdout:1647 read 0x4025400 thru 0x402ed90 (0x9991 bytes) 2024-09-17T21:19:02.046 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:19:02.060 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:02.059+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:02.084 INFO:teuthology.orchestra.run.smithi184.stdout:1651 write 0x9186800 thru 0x918a33c (0x3b3d bytes) 2024-09-17T21:19:02.085 INFO:teuthology.orchestra.run.smithi184.stdout:1654 punch from 0x77a5200 to 0x77a9200, (0x4000 bytes) 2024-09-17T21:19:02.085 INFO:teuthology.orchestra.run.smithi184.stdout:1655 read 0xbbe200 thru 0xbc41ef (0x5ff0 bytes) 2024-09-17T21:19:02.085 INFO:teuthology.orchestra.run.smithi184.stdout:1656 punch from 0x34c8c00 to 0x34d0800, (0x7c00 bytes) 2024-09-17T21:19:02.085 INFO:teuthology.orchestra.run.smithi184.stdout:1657 read 0x810ca00 thru 0x8110cb3 (0x42b4 bytes) 2024-09-17T21:19:02.094 INFO:teuthology.orchestra.run.smithi184.stdout:1658 trunc from 0x918a400 to 0x5635c00 2024-09-17T21:19:02.107 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:19:02.138 INFO:teuthology.orchestra.run.smithi184.stdout:1662 write 0x68c2c00 thru 0x68cf027 (0xc428 bytes) 2024-09-17T21:19:02.140 INFO:teuthology.orchestra.run.smithi184.stdout:1669 read 0x12baa00 thru 0x12c02c6 (0x58c7 bytes) 2024-09-17T21:19:02.161 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:02.157+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:02.165 INFO:teuthology.orchestra.run.smithi184.stdout:1673 write 0x8410a00 thru 0x841ef30 (0xe531 bytes) 2024-09-17T21:19:02.167 INFO:teuthology.orchestra.run.smithi184.stdout:1674 read 0x7c8400 thru 0x7d0480 (0x8081 bytes) 2024-09-17T21:19:02.214 INFO:teuthology.orchestra.run.smithi184.stdout:1678 trunc from 0x841f000 to 0x624c00 2024-09-17T21:19:02.219 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:02.219 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:02.236 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:19:02.262 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:02.259+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:02.263 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:02.264 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:02.281 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:02.281 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:02.297 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:02.297 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:02.312 INFO:teuthology.orchestra.run.smithi184.stdout:1679 write 0x2d6ba00 thru 0x2d6e6ab (0x2cac bytes) 2024-09-17T21:19:02.314 INFO:teuthology.orchestra.run.smithi184.stdout:1680 read 0xf30c00 thru 0xf38060 (0x7461 bytes) 2024-09-17T21:19:02.315 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:02.316 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:02.320 INFO:teuthology.orchestra.run.smithi184.stdout:1682 trunc from 0x2d6e800 to 0x5045400 2024-09-17T21:19:02.335 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:02.336 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:02.354 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:02.354 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:02.363 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:02.361+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:02.383 INFO:teuthology.orchestra.run.smithi184.stdout:1683 write 0xe49f200 thru 0xe4abf3c (0xcd3d bytes) 2024-09-17T21:19:02.385 INFO:teuthology.orchestra.run.smithi184.stdout:1684 punch from 0x3c8ce00 to 0x3c98c00, (0xbe00 bytes) 2024-09-17T21:19:02.385 INFO:teuthology.orchestra.run.smithi184.stdout:1685 clone 14 order 25 su 4194304 sc 9 2024-09-17T21:19:02.412 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:02.412 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:02.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:02.412 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:02.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.412 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:19:02.412 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:02.413 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.393600454999998, 2024-09-17T21:19:02.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:19:02.413 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.413 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.413 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:02.413 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.413 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:19:02.413 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:19:02.413 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:19:02.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.413 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:02.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:02.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:19:02.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:19:02.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:19:02.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:19:02.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:19:02.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:19:02.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:19:02.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:19:02.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:19:02.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:19:02.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:02.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:19:02.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:19:02.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:19:02.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:19:02.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:19:02.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:19:02.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:19:02.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:19:02.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:19:02.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:02.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:19:02.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:19:02.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:02.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:19:02.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:19:02.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:02.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:19:02.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:19:02.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:19:02.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:19:02.419 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.419 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.419 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.419 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:02.419 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:02.420 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.813436175, 2024-09-17T21:19:02.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:19:02.420 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.420 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.420 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.420 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.420 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:19:02.420 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:19:02.420 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:19:02.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.420 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:02.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:02.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:19:02.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:19:02.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.422 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:19:02.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:19:02.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.422 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:19:02.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:19:02.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:19:02.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:19:02.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:19:02.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:19:02.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:02.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:19:02.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:19:02.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:19:02.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:19:02.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:02.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:19:02.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:19:02.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:19:02.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:19:02.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:19:02.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:19:02.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:19:02.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:19:02.426 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.426 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.426 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.426 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:02.426 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:02.426 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.813338862999998, 2024-09-17T21:19:02.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:19:02.426 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.426 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.426 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.426 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.426 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:02.427 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:02.427 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:19:02.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.427 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:02.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:02.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:19:02.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:19:02.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:19:02.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:19:02.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:19:02.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:19:02.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:19:02.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:19:02.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:19:02.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:19:02.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:02.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:19:02.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:19:02.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:19:02.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:19:02.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:02.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:19:02.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:19:02.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:19:02.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:19:02.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:19:02.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:19:02.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:19:02.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:19:02.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:19:02.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:19:02.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:19:02.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:19:02.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:19:02.432 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.433 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.433 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.433 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:02.433 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:02.433 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.811252062000001, 2024-09-17T21:19:02.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:19:02.433 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.433 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.433 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:02.433 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.433 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:19:02.433 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:19:02.434 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:19:02.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.434 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:02.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:02.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:19:02.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:19:02.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:19:02.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:19:02.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:19:02.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:19:02.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:19:02.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:19:02.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:19:02.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:19:02.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:19:02.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:19:02.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:19:02.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:19:02.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:19:02.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:02.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:19:02.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:19:02.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:19:02.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:19:02.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:19:02.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:19:02.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:02.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:19:02.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:19:02.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:02.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:19:02.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:19:02.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:02.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:19:02.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:19:02.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:02.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:19:02.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:19:02.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:19:02.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:19:02.440 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.440 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.440 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.440 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:282 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:02.440 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:19:02.440 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.179546801000001, 2024-09-17T21:19:02.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023571801, 2024-09-17T21:19:02.440 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.440 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.440 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.440 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.440 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:02.441 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:02.441 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 282 2024-09-17T21:19:02.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.441 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:19:02.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:19:02.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227820+0000", 2024-09-17T21:19:02.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.11e-06 2024-09-17T21:19:02.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227825+0000", 2024-09-17T21:19:02.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8099999999999997e-06 2024-09-17T21:19:02.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227848+0000", 2024-09-17T21:19:02.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2949999999999999e-05 2024-09-17T21:19:02.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227862+0000", 2024-09-17T21:19:02.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:19:02.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.247977+0000", 2024-09-17T21:19:02.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020114968 2024-09-17T21:19:02.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251141+0000", 2024-09-17T21:19:02.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031640689999999998 2024-09-17T21:19:02.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251391+0000", 2024-09-17T21:19:02.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025018499999999999 2024-09-17T21:19:02.444 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.444 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.444 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.444 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:283 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:02.444 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:19:02.445 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.179484962, 2024-09-17T21:19:02.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024022274999999999, 2024-09-17T21:19:02.445 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.445 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.445 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.445 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.445 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:02.445 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:02.445 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 283 2024-09-17T21:19:02.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.445 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:19:02.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:19:02.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227882+0000", 2024-09-17T21:19:02.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-06 2024-09-17T21:19:02.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227887+0000", 2024-09-17T21:19:02.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1769999999999997e-06 2024-09-17T21:19:02.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227905+0000", 2024-09-17T21:19:02.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7235999999999999e-05 2024-09-17T21:19:02.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227918+0000", 2024-09-17T21:19:02.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3387000000000001e-05 2024-09-17T21:19:02.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251469+0000", 2024-09-17T21:19:02.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023550894999999999 2024-09-17T21:19:02.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251669+0000", 2024-09-17T21:19:02.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019992499999999999 2024-09-17T21:19:02.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251903+0000", 2024-09-17T21:19:02.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000234412 2024-09-17T21:19:02.449 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.449 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.449 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.449 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:284 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:02.449 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:19:02.449 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.179429409000001, 2024-09-17T21:19:02.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024450084, 2024-09-17T21:19:02.449 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.449 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.449 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.449 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.449 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:02.450 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:02.450 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 284 2024-09-17T21:19:02.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.450 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:19:02.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:19:02.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227937+0000", 2024-09-17T21:19:02.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6700000000000002e-07 2024-09-17T21:19:02.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227943+0000", 2024-09-17T21:19:02.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:19:02.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227961+0000", 2024-09-17T21:19:02.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.827e-05 2024-09-17T21:19:02.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227975+0000", 2024-09-17T21:19:02.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.394e-05 2024-09-17T21:19:02.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251977+0000", 2024-09-17T21:19:02.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024001340999999999 2024-09-17T21:19:02.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252169+0000", 2024-09-17T21:19:02.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019271199999999999 2024-09-17T21:19:02.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252387+0000", 2024-09-17T21:19:02.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021726100000000001 2024-09-17T21:19:02.453 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.453 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.453 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.453 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:285 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:02.453 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:19:02.453 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.179373119000001, 2024-09-17T21:19:02.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024884791, 2024-09-17T21:19:02.454 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.454 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.454 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.454 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.454 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:02.454 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:02.454 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 285 2024-09-17T21:19:02.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.454 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:19:02.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:19:02.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227994+0000", 2024-09-17T21:19:02.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1429999999999999e-06 2024-09-17T21:19:02.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227999+0000", 2024-09-17T21:19:02.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1669999999999998e-06 2024-09-17T21:19:02.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228015+0000", 2024-09-17T21:19:02.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6056e-05 2024-09-17T21:19:02.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228028+0000", 2024-09-17T21:19:02.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3196e-05 2024-09-17T21:19:02.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252458+0000", 2024-09-17T21:19:02.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024429751999999999 2024-09-17T21:19:02.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252650+0000", 2024-09-17T21:19:02.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019187800000000001 2024-09-17T21:19:02.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252878+0000", 2024-09-17T21:19:02.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022759899999999999 2024-09-17T21:19:02.457 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.457 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.457 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.458 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:286 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:02.458 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:19:02.458 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.179317359999999, 2024-09-17T21:19:02.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025299318000000001, 2024-09-17T21:19:02.458 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.458 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.458 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.458 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.458 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:02.458 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:02.458 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 286 2024-09-17T21:19:02.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.458 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:19:02.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:19:02.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228050+0000", 2024-09-17T21:19:02.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:19:02.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228055+0000", 2024-09-17T21:19:02.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4960000000000004e-06 2024-09-17T21:19:02.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228075+0000", 2024-09-17T21:19:02.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0109999999999999e-05 2024-09-17T21:19:02.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228089+0000", 2024-09-17T21:19:02.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4154e-05 2024-09-17T21:19:02.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252952+0000", 2024-09-17T21:19:02.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024862934 2024-09-17T21:19:02.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253141+0000", 2024-09-17T21:19:02.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000188782 2024-09-17T21:19:02.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253348+0000", 2024-09-17T21:19:02.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020684499999999999 2024-09-17T21:19:02.462 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.462 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.462 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.462 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:443 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:02.462 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:19:02.462 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.165730055000001, 2024-09-17T21:19:02.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023179569000000001, 2024-09-17T21:19:02.462 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.462 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.463 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.463 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.463 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:02.463 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:02.463 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 443 2024-09-17T21:19:02.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.463 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:19:02.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:19:02.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241637+0000", 2024-09-17T21:19:02.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.46e-06 2024-09-17T21:19:02.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241643+0000", 2024-09-17T21:19:02.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1970000000000001e-06 2024-09-17T21:19:02.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241662+0000", 2024-09-17T21:19:02.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8712999999999999e-05 2024-09-17T21:19:02.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241676+0000", 2024-09-17T21:19:02.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.393e-05 2024-09-17T21:19:02.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264596+0000", 2024-09-17T21:19:02.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022919688000000001 2024-09-17T21:19:02.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264690+0000", 2024-09-17T21:19:02.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4145000000000001e-05 2024-09-17T21:19:02.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264815+0000", 2024-09-17T21:19:02.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012543600000000001 2024-09-17T21:19:02.466 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.466 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.466 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.466 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:444 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:02.466 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:19:02.467 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.165669005000002, 2024-09-17T21:19:02.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023260383999999999, 2024-09-17T21:19:02.467 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.467 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.467 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.467 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.467 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:02.467 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:02.467 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 444 2024-09-17T21:19:02.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.467 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:19:02.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:19:02.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241698+0000", 2024-09-17T21:19:02.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2270000000000001e-06 2024-09-17T21:19:02.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241704+0000", 2024-09-17T21:19:02.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9800000000000003e-06 2024-09-17T21:19:02.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241725+0000", 2024-09-17T21:19:02.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0812999999999999e-05 2024-09-17T21:19:02.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241739+0000", 2024-09-17T21:19:02.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3773e-05 2024-09-17T21:19:02.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264850+0000", 2024-09-17T21:19:02.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023111326000000001 2024-09-17T21:19:02.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264897+0000", 2024-09-17T21:19:02.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6638999999999998e-05 2024-09-17T21:19:02.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264957+0000", 2024-09-17T21:19:02.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0625999999999999e-05 2024-09-17T21:19:02.470 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.470 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.471 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.471 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:445 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:02.471 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:19:02.471 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.165608335999998, 2024-09-17T21:19:02.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023315144999999999, 2024-09-17T21:19:02.471 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.471 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.471 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.471 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.471 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:02.471 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:02.471 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 445 2024-09-17T21:19:02.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.471 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:19:02.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:19:02.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241759+0000", 2024-09-17T21:19:02.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-06 2024-09-17T21:19:02.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241764+0000", 2024-09-17T21:19:02.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5899999999999998e-06 2024-09-17T21:19:02.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241784+0000", 2024-09-17T21:19:02.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9204000000000001e-05 2024-09-17T21:19:02.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241798+0000", 2024-09-17T21:19:02.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3953e-05 2024-09-17T21:19:02.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264977+0000", 2024-09-17T21:19:02.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023179182 2024-09-17T21:19:02.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.265018+0000", 2024-09-17T21:19:02.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1171999999999998e-05 2024-09-17T21:19:02.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.265073+0000", 2024-09-17T21:19:02.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4741000000000003e-05 2024-09-17T21:19:02.475 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.475 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.475 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.475 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:679 3.13 3:cd0d74f9:::rbd_data.11527824718.000000000000000a:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:02.475 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:19:02.475 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.156686915000002, 2024-09-17T21:19:02.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023284978000000001, 2024-09-17T21:19:02.475 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.475 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.476 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.476 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.476 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:02.476 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:02.476 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 679 2024-09-17T21:19:02.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.476 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:19:02.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:19:02.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250680+0000", 2024-09-17T21:19:02.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:19:02.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250685+0000", 2024-09-17T21:19:02.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1970000000000004e-06 2024-09-17T21:19:02.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250703+0000", 2024-09-17T21:19:02.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7396000000000001e-05 2024-09-17T21:19:02.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250716+0000", 2024-09-17T21:19:02.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.29e-05 2024-09-17T21:19:02.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273740+0000", 2024-09-17T21:19:02.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023024405000000001 2024-09-17T21:19:02.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273834+0000", 2024-09-17T21:19:02.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4124999999999998e-05 2024-09-17T21:19:02.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273964+0000", 2024-09-17T21:19:02.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012963499999999999 2024-09-17T21:19:02.479 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.479 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.479 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.480 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:680 3.13 3:cd0d74f9:::rbd_data.11527824718.000000000000000a:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:02.480 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:19:02.480 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.156631834999999, 2024-09-17T21:19:02.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023687792999999999, 2024-09-17T21:19:02.480 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.480 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.480 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.480 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 680 2024-09-17T21:19:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:19:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:19:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250735+0000", 2024-09-17T21:19:02.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2669999999999999e-06 2024-09-17T21:19:02.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250740+0000", 2024-09-17T21:19:02.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8060000000000002e-06 2024-09-17T21:19:02.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250757+0000", 2024-09-17T21:19:02.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7303e-05 2024-09-17T21:19:02.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250771+0000", 2024-09-17T21:19:02.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3876999999999999e-05 2024-09-17T21:19:02.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274022+0000", 2024-09-17T21:19:02.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023251135999999999 2024-09-17T21:19:02.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274211+0000", 2024-09-17T21:19:02.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018813599999999999 2024-09-17T21:19:02.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274422+0000", 2024-09-17T21:19:02.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000211268 2024-09-17T21:19:02.484 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.484 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.484 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.484 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:02.484 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:02.485 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.127353281, 2024-09-17T21:19:02.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:19:02.485 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.485 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.485 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.485 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.485 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:02.485 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:02.485 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:19:02.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.485 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:02.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:02.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:19:02.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:19:02.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:19:02.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:19:02.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:19:02.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:19:02.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:19:02.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:19:02.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:19:02.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:19:02.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:02.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:19:02.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:19:02.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:19:02.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:19:02.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:02.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:19:02.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:19:02.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:19:02.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:19:02.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:19:02.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:19:02.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:19:02.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:19:02.490 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.490 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.490 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.490 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:02.490 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:02.490 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.11826157, 2024-09-17T21:19:02.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:19:02.490 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.491 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.491 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.491 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.491 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:02.491 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:02.491 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:19:02.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.491 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:02.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:02.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:19:02.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:19:02.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:19:02.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:19:02.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:19:02.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:19:02.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:19:02.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:19:02.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:19:02.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:19:02.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:02.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:19:02.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:19:02.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:19:02.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:19:02.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:19:02.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:19:02.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:19:02.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:19:02.495 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.495 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.495 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.495 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:02.495 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:02.496 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.118089519, 2024-09-17T21:19:02.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:19:02.496 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.496 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.496 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.496 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.496 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:02.496 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:02.496 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:19:02.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.496 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:02.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:02.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:19:02.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:19:02.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:19:02.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:19:02.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:19:02.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:19:02.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:19:02.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:19:02.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:19:02.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:19:02.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:02.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:19:02.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:19:02.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:19:02.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:19:02.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:19:02.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:19:02.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:19:02.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:19:02.500 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.500 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.943156772, 2024-09-17T21:19:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:19:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:19:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:19:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:19:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:19:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:19:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:19:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:19:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:19:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:19:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:19:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:19:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:19:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:19:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:19:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:19:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:19:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:19:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:19:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:19:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.943072215000001, 2024-09-17T21:19:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:19:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:19:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:19:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:19:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:19:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:19:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:19:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:19:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:19:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:19:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:19:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:19:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:19:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:19:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:19:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:19:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:19:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:19:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:19:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.943006578, 2024-09-17T21:19:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:19:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:19:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:19:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:19:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:19:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:19:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:19:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:19:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:19:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:19:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:19:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:19:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:19:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:19:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:19:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:19:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:19:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:19:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:19:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:19:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.517 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.517 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:02.518 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:19:02.518 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:02.517+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:02.621 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:02.619+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:02.687 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:02.688 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:02.688 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:19:02.688 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:02.702 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:19:02.721 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:02.719+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:02.824 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:02.825 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:02.825 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:02.825 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:02.825 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:02.825 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:02.823+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:02.838 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:19:02.926 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:02.923+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:02.966 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:02.966 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:02.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:02.967 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:02.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.967 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:478 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 720896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:02.967 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:19:02.967 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.729456059, 2024-09-17T21:19:02.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016845300000000001, 2024-09-17T21:19:02.967 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.967 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.967 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.967 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.967 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:02.967 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:02.967 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 478 2024-09-17T21:19:02.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.968 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:19:02.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:19:02.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.969 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.969 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233785+0000", 2024-09-17T21:19:02.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.116e-06 2024-09-17T21:19:02.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.969 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.969 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233803+0000", 2024-09-17T21:19:02.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7932999999999999e-05 2024-09-17T21:19:02.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.969 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.969 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233822+0000", 2024-09-17T21:19:02.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9276999999999999e-05 2024-09-17T21:19:02.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.970 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.970 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233836+0000", 2024-09-17T21:19:02.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4053000000000001e-05 2024-09-17T21:19:02.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.970 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.970 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.248579+0000", 2024-09-17T21:19:02.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014743529 2024-09-17T21:19:02.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.248795+0000", 2024-09-17T21:19:02.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021535200000000001 2024-09-17T21:19:02.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250629+0000", 2024-09-17T21:19:02.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018340400000000001 2024-09-17T21:19:02.971 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.971 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.971 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.971 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:479 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 851968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:02.972 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:19:02.972 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.729364972999999, 2024-09-17T21:19:02.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017383900000000001, 2024-09-17T21:19:02.972 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.972 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.972 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.972 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.972 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:02.972 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:02.972 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 479 2024-09-17T21:19:02.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.972 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:19:02.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:19:02.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233876+0000", 2024-09-17T21:19:02.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2160000000000001e-06 2024-09-17T21:19:02.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233883+0000", 2024-09-17T21:19:02.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9199999999999998e-06 2024-09-17T21:19:02.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233900+0000", 2024-09-17T21:19:02.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7034e-05 2024-09-17T21:19:02.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233913+0000", 2024-09-17T21:19:02.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159000000000001e-05 2024-09-17T21:19:02.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.975 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.975 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250712+0000", 2024-09-17T21:19:02.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016798644000000001 2024-09-17T21:19:02.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.975 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.975 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250913+0000", 2024-09-17T21:19:02.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020127900000000001 2024-09-17T21:19:02.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.975 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.975 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251259+0000", 2024-09-17T21:19:02.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00034564799999999999 2024-09-17T21:19:02.975 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.976 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.976 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.976 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:480 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:02.976 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:19:02.976 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.729272128000002, 2024-09-17T21:19:02.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017902482000000001, 2024-09-17T21:19:02.976 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.976 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.976 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.976 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.976 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:02.977 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:02.977 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 480 2024-09-17T21:19:02.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.977 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:19:02.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:19:02.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233969+0000", 2024-09-17T21:19:02.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.454e-06 2024-09-17T21:19:02.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233981+0000", 2024-09-17T21:19:02.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1593000000000001e-05 2024-09-17T21:19:02.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233999+0000", 2024-09-17T21:19:02.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.806e-05 2024-09-17T21:19:02.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234012+0000", 2024-09-17T21:19:02.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:19:02.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251339+0000", 2024-09-17T21:19:02.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017326352 2024-09-17T21:19:02.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251534+0000", 2024-09-17T21:19:02.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000195842 2024-09-17T21:19:02.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251870+0000", 2024-09-17T21:19:02.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033551400000000003 2024-09-17T21:19:02.981 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.981 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.981 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.981 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:481 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1114112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:02.982 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:19:02.982 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.729203761000001, 2024-09-17T21:19:02.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018442621999999999, 2024-09-17T21:19:02.982 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.982 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.982 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.982 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.983 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:02.983 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:02.983 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 481 2024-09-17T21:19:02.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.983 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:19:02.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:19:02.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234037+0000", 2024-09-17T21:19:02.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:19:02.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234043+0000", 2024-09-17T21:19:02.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.49e-06 2024-09-17T21:19:02.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234061+0000", 2024-09-17T21:19:02.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8416000000000001e-05 2024-09-17T21:19:02.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234076+0000", 2024-09-17T21:19:02.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.456e-05 2024-09-17T21:19:02.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.986 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251948+0000", 2024-09-17T21:19:02.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017872774000000001 2024-09-17T21:19:02.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.986 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252144+0000", 2024-09-17T21:19:02.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019517300000000001 2024-09-17T21:19:02.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252478+0000", 2024-09-17T21:19:02.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033482899999999998 2024-09-17T21:19:02.987 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.987 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.987 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.987 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:482 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:02.987 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:19:02.987 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.729145208999999, 2024-09-17T21:19:02.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018988026000000002, 2024-09-17T21:19:02.987 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.988 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.988 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.988 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.988 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:02.988 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:02.988 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 482 2024-09-17T21:19:02.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.988 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:19:02.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:19:02.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234096+0000", 2024-09-17T21:19:02.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2139999999999999e-06 2024-09-17T21:19:02.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234101+0000", 2024-09-17T21:19:02.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7100000000000004e-06 2024-09-17T21:19:02.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234118+0000", 2024-09-17T21:19:02.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6486000000000001e-05 2024-09-17T21:19:02.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234131+0000", 2024-09-17T21:19:02.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2824e-05 2024-09-17T21:19:02.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252552+0000", 2024-09-17T21:19:02.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018421549999999998 2024-09-17T21:19:02.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252746+0000", 2024-09-17T21:19:02.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019424399999999999 2024-09-17T21:19:02.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.992 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.992 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253082+0000", 2024-09-17T21:19:02.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000335998 2024-09-17T21:19:02.992 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.992 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.992 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.992 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:483 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:02.992 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:19:02.993 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.729090588999998, 2024-09-17T21:19:02.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019462882000000001, 2024-09-17T21:19:02.993 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.993 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.993 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.993 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.993 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:02.993 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:02.993 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 483 2024-09-17T21:19:02.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.993 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:19:02.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:02.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:19:02.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:02.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234150+0000", 2024-09-17T21:19:02.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.127e-06 2024-09-17T21:19:02.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:02.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234155+0000", 2024-09-17T21:19:02.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:19:02.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:02.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234171+0000", 2024-09-17T21:19:02.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6160000000000001e-05 2024-09-17T21:19:02.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:02.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234185+0000", 2024-09-17T21:19:02.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3370000000000001e-05 2024-09-17T21:19:02.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:02.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253158+0000", 2024-09-17T21:19:02.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018972968999999999 2024-09-17T21:19:02.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:02.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253349+0000", 2024-09-17T21:19:02.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000190733 2024-09-17T21:19:02.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:02.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253612+0000", 2024-09-17T21:19:02.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026337000000000002 2024-09-17T21:19:02.998 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.998 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:02.998 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:02.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.998 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:484 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:02.998 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:19:02.998 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.729036563000001, 2024-09-17T21:19:02.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020029261999999999, 2024-09-17T21:19:02.998 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:02.998 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:02.999 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:02.999 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:02.999 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:02.999 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:02.999 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 484 2024-09-17T21:19:02.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:02.999 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:02.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:02.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:02.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:19:02.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:02.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:19:03.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234205+0000", 2024-09-17T21:19:03.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6470000000000001e-06 2024-09-17T21:19:03.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234210+0000", 2024-09-17T21:19:03.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:19:03.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234227+0000", 2024-09-17T21:19:03.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.73e-05 2024-09-17T21:19:03.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234241+0000", 2024-09-17T21:19:03.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3757000000000001e-05 2024-09-17T21:19:03.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253690+0000", 2024-09-17T21:19:03.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019449368000000002 2024-09-17T21:19:03.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253898+0000", 2024-09-17T21:19:03.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000207161 2024-09-17T21:19:03.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254232+0000", 2024-09-17T21:19:03.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033474299999999998 2024-09-17T21:19:03.003 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.003 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.003 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.004 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:485 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1638400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:03.004 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:19:03.004 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.728979335999998, 2024-09-17T21:19:03.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020553914, 2024-09-17T21:19:03.004 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.004 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.004 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.004 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.004 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:03.004 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:03.005 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 485 2024-09-17T21:19:03.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.005 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:19:03.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:19:03.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234261+0000", 2024-09-17T21:19:03.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.136e-06 2024-09-17T21:19:03.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234266+0000", 2024-09-17T21:19:03.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0669999999999999e-06 2024-09-17T21:19:03.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234284+0000", 2024-09-17T21:19:03.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7839999999999999e-05 2024-09-17T21:19:03.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234298+0000", 2024-09-17T21:19:03.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3852999999999999e-05 2024-09-17T21:19:03.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254311+0000", 2024-09-17T21:19:03.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020012973 2024-09-17T21:19:03.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254552+0000", 2024-09-17T21:19:03.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024116400000000001 2024-09-17T21:19:03.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254814+0000", 2024-09-17T21:19:03.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000261881 2024-09-17T21:19:03.009 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.009 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.009 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.009 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:486 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:03.009 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:19:03.009 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.728922947000001, 2024-09-17T21:19:03.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021010240999999999, 2024-09-17T21:19:03.009 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.010 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.010 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.010 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.010 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:03.010 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:03.010 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 486 2024-09-17T21:19:03.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.010 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:19:03.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:19:03.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234318+0000", 2024-09-17T21:19:03.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.37e-06 2024-09-17T21:19:03.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234324+0000", 2024-09-17T21:19:03.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:19:03.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234341+0000", 2024-09-17T21:19:03.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7127000000000001e-05 2024-09-17T21:19:03.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234354+0000", 2024-09-17T21:19:03.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3210000000000001e-05 2024-09-17T21:19:03.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254896+0000", 2024-09-17T21:19:03.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020541835000000001 2024-09-17T21:19:03.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255092+0000", 2024-09-17T21:19:03.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019634700000000001 2024-09-17T21:19:03.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255327+0000", 2024-09-17T21:19:03.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023475899999999999 2024-09-17T21:19:03.014 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.014 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.014 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.014 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:487 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:03.015 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:19:03.015 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.728866750999998, 2024-09-17T21:19:03.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021453911999999999, 2024-09-17T21:19:03.015 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.015 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.015 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.015 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.015 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:03.015 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:03.015 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 487 2024-09-17T21:19:03.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.016 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:19:03.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:19:03.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234374+0000", 2024-09-17T21:19:03.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3039999999999999e-06 2024-09-17T21:19:03.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234380+0000", 2024-09-17T21:19:03.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:19:03.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234397+0000", 2024-09-17T21:19:03.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7232999999999999e-05 2024-09-17T21:19:03.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234411+0000", 2024-09-17T21:19:03.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3784000000000001e-05 2024-09-17T21:19:03.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255406+0000", 2024-09-17T21:19:03.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020995204 2024-09-17T21:19:03.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255595+0000", 2024-09-17T21:19:03.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018903299999999999 2024-09-17T21:19:03.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255827+0000", 2024-09-17T21:19:03.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023190099999999999 2024-09-17T21:19:03.019 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.019 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.020 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.020 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:488 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:03.020 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:19:03.020 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.728811073999999, 2024-09-17T21:19:03.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026887966999999999, 2024-09-17T21:19:03.020 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.020 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.020 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.020 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.020 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:03.021 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:03.021 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 488 2024-09-17T21:19:03.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.021 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:19:03.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:19:03.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234430+0000", 2024-09-17T21:19:03.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1829999999999999e-06 2024-09-17T21:19:03.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234435+0000", 2024-09-17T21:19:03.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9869999999999999e-06 2024-09-17T21:19:03.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234451+0000", 2024-09-17T21:19:03.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6458999999999999e-05 2024-09-17T21:19:03.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234465+0000", 2024-09-17T21:19:03.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3730000000000001e-05 2024-09-17T21:19:03.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255908+0000", 2024-09-17T21:19:03.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021443413000000001 2024-09-17T21:19:03.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256098+0000", 2024-09-17T21:19:03.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018961 2024-09-17T21:19:03.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261317+0000", 2024-09-17T21:19:03.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052185850000000004 2024-09-17T21:19:03.025 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.025 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.025 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.025 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:489 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2162688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:03.025 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:19:03.025 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.728755235000001, 2024-09-17T21:19:03.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022632815000000001, 2024-09-17T21:19:03.026 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.026 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.026 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.026 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.026 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:03.026 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:03.026 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 489 2024-09-17T21:19:03.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.026 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:19:03.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:19:03.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234486+0000", 2024-09-17T21:19:03.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5099999999999999e-06 2024-09-17T21:19:03.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234491+0000", 2024-09-17T21:19:03.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.147e-06 2024-09-17T21:19:03.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234508+0000", 2024-09-17T21:19:03.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7405999999999999e-05 2024-09-17T21:19:03.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234522+0000", 2024-09-17T21:19:03.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3677e-05 2024-09-17T21:19:03.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256536+0000", 2024-09-17T21:19:03.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022013464999999999 2024-09-17T21:19:03.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256752+0000", 2024-09-17T21:19:03.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021637800000000001 2024-09-17T21:19:03.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257117+0000", 2024-09-17T21:19:03.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000365232 2024-09-17T21:19:03.030 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.030 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.030 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.030 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:490 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2293760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:03.030 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:19:03.031 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.728700445000001, 2024-09-17T21:19:03.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023146462999999999, 2024-09-17T21:19:03.031 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.031 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.031 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.031 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.031 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:03.031 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:03.031 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 490 2024-09-17T21:19:03.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.031 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:19:03.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:19:03.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234541+0000", 2024-09-17T21:19:03.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4869999999999999e-06 2024-09-17T21:19:03.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234559+0000", 2024-09-17T21:19:03.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8105999999999999e-05 2024-09-17T21:19:03.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234577+0000", 2024-09-17T21:19:03.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.836e-05 2024-09-17T21:19:03.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234592+0000", 2024-09-17T21:19:03.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4766000000000001e-05 2024-09-17T21:19:03.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257187+0000", 2024-09-17T21:19:03.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022594725 2024-09-17T21:19:03.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257374+0000", 2024-09-17T21:19:03.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018767299999999999 2024-09-17T21:19:03.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257686+0000", 2024-09-17T21:19:03.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000311346 2024-09-17T21:19:03.035 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.035 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.035 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.036 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:491 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:03.036 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:19:03.036 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.728627836000001, 2024-09-17T21:19:03.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023696872000000001, 2024-09-17T21:19:03.036 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.036 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.036 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.036 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.036 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:03.036 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:03.037 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 491 2024-09-17T21:19:03.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.037 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:19:03.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:19:03.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234613+0000", 2024-09-17T21:19:03.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-06 2024-09-17T21:19:03.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234618+0000", 2024-09-17T21:19:03.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3839999999999996e-06 2024-09-17T21:19:03.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234635+0000", 2024-09-17T21:19:03.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.685e-05 2024-09-17T21:19:03.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234648+0000", 2024-09-17T21:19:03.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.313e-05 2024-09-17T21:19:03.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257780+0000", 2024-09-17T21:19:03.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023131946 2024-09-17T21:19:03.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257985+0000", 2024-09-17T21:19:03.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020468400000000001 2024-09-17T21:19:03.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258309+0000", 2024-09-17T21:19:03.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032357499999999999 2024-09-17T21:19:03.041 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.041 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.041 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.041 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:492 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:03.041 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:19:03.041 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.728572248999999, 2024-09-17T21:19:03.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024258994999999998, 2024-09-17T21:19:03.041 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.041 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.042 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.042 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.042 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:03.042 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:03.042 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 492 2024-09-17T21:19:03.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.042 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:19:03.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:19:03.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234669+0000", 2024-09-17T21:19:03.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.313e-06 2024-09-17T21:19:03.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234674+0000", 2024-09-17T21:19:03.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1599999999999997e-06 2024-09-17T21:19:03.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234690+0000", 2024-09-17T21:19:03.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.666e-05 2024-09-17T21:19:03.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234704+0000", 2024-09-17T21:19:03.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3043e-05 2024-09-17T21:19:03.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258397+0000", 2024-09-17T21:19:03.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023693204999999998 2024-09-17T21:19:03.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258593+0000", 2024-09-17T21:19:03.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019578999999999999 2024-09-17T21:19:03.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258926+0000", 2024-09-17T21:19:03.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033382399999999998 2024-09-17T21:19:03.046 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.046 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.046 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.046 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:493 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:03.046 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:19:03.046 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.728515783999999, 2024-09-17T21:19:03.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024793461999999999, 2024-09-17T21:19:03.047 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.047 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.047 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.047 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.047 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:03.047 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:03.047 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 493 2024-09-17T21:19:03.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.047 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:19:03.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:19:03.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234725+0000", 2024-09-17T21:19:03.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.24e-06 2024-09-17T21:19:03.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234731+0000", 2024-09-17T21:19:03.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8939999999999998e-06 2024-09-17T21:19:03.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234749+0000", 2024-09-17T21:19:03.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7863000000000002e-05 2024-09-17T21:19:03.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234762+0000", 2024-09-17T21:19:03.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3322999999999999e-05 2024-09-17T21:19:03.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259022+0000", 2024-09-17T21:19:03.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024259691999999999 2024-09-17T21:19:03.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259259+0000", 2024-09-17T21:19:03.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023669199999999999 2024-09-17T21:19:03.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259517+0000", 2024-09-17T21:19:03.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025875799999999999 2024-09-17T21:19:03.051 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.051 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.051 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.051 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:494 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:03.051 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:19:03.051 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.728458084, 2024-09-17T21:19:03.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025251952000000001, 2024-09-17T21:19:03.052 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.052 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.052 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.052 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.052 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:03.052 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:03.052 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 494 2024-09-17T21:19:03.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.052 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:19:03.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:19:03.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234783+0000", 2024-09-17T21:19:03.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2440000000000001e-06 2024-09-17T21:19:03.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234789+0000", 2024-09-17T21:19:03.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:19:03.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234806+0000", 2024-09-17T21:19:03.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7450000000000001e-05 2024-09-17T21:19:03.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234819+0000", 2024-09-17T21:19:03.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3326000000000001e-05 2024-09-17T21:19:03.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259585+0000", 2024-09-17T21:19:03.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024765579999999999 2024-09-17T21:19:03.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259787+0000", 2024-09-17T21:19:03.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202444 2024-09-17T21:19:03.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260033+0000", 2024-09-17T21:19:03.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024611499999999998 2024-09-17T21:19:03.056 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.056 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.056 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.057 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:495 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:03.057 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:19:03.057 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.728402720999998, 2024-09-17T21:19:03.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025776342000000001, 2024-09-17T21:19:03.057 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.057 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.057 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.057 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.057 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:03.058 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:03.058 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 495 2024-09-17T21:19:03.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.058 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:19:03.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:19:03.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234838+0000", 2024-09-17T21:19:03.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2130000000000001e-06 2024-09-17T21:19:03.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234843+0000", 2024-09-17T21:19:03.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9899999999999997e-06 2024-09-17T21:19:03.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234860+0000", 2024-09-17T21:19:03.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6973000000000002e-05 2024-09-17T21:19:03.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234873+0000", 2024-09-17T21:19:03.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3179999999999999e-05 2024-09-17T21:19:03.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260135+0000", 2024-09-17T21:19:03.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025262118 2024-09-17T21:19:03.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260364+0000", 2024-09-17T21:19:03.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022830699999999999 2024-09-17T21:19:03.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260613+0000", 2024-09-17T21:19:03.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024956099999999998 2024-09-17T21:19:03.062 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.063 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.063 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.063 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:496 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:03.063 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:19:03.063 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.728349440999999, 2024-09-17T21:19:03.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026376773999999999, 2024-09-17T21:19:03.063 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.063 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.063 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.063 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.064 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:03.064 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:03.064 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 496 2024-09-17T21:19:03.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.064 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:19:03.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:19:03.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234891+0000", 2024-09-17T21:19:03.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1230000000000001e-06 2024-09-17T21:19:03.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234896+0000", 2024-09-17T21:19:03.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9599999999999999e-06 2024-09-17T21:19:03.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234913+0000", 2024-09-17T21:19:03.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6747000000000001e-05 2024-09-17T21:19:03.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234926+0000", 2024-09-17T21:19:03.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-05 2024-09-17T21:19:03.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260675+0000", 2024-09-17T21:19:03.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025748979000000002 2024-09-17T21:19:03.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260883+0000", 2024-09-17T21:19:03.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020848399999999999 2024-09-17T21:19:03.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261267+0000", 2024-09-17T21:19:03.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038345099999999998 2024-09-17T21:19:03.068 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.068 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.068 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.068 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:682 3.9 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2455552~12288,write 8126464~10240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:03.068 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:19:03.068 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.69441239, 2024-09-17T21:19:03.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018447261999999999, 2024-09-17T21:19:03.069 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.069 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.069 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:03.069 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.069 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:03.069 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:03.069 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 682 2024-09-17T21:19:03.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.069 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:19:03.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:19:03.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268830+0000", 2024-09-17T21:19:03.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6419999999999999e-06 2024-09-17T21:19:03.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271673+0000", 2024-09-17T21:19:03.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028432610000000001 2024-09-17T21:19:03.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271700+0000", 2024-09-17T21:19:03.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7200999999999999e-05 2024-09-17T21:19:03.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271715+0000", 2024-09-17T21:19:03.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5145e-05 2024-09-17T21:19:03.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271785+0000", 2024-09-17T21:19:03.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9519999999999998e-05 2024-09-17T21:19:03.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271918+0000", 2024-09-17T21:19:03.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013330899999999999 2024-09-17T21:19:03.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:03.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.272449+0000", 2024-09-17T21:19:03.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00053051200000000002 2024-09-17T21:19:03.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:03.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.278822+0000", 2024-09-17T21:19:03.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0063731619999999999 2024-09-17T21:19:03.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:03.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287167+0000", 2024-09-17T21:19:03.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083452149999999996 2024-09-17T21:19:03.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:03.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287213+0000", 2024-09-17T21:19:03.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5972e-05 2024-09-17T21:19:03.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287274+0000", 2024-09-17T21:19:03.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1322999999999995e-05 2024-09-17T21:19:03.075 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.075 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.075 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.075 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.075 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.075 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:03.077 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:19:03.077 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:03.075+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:03.178 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:03.175+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:03.203 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:03.203 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:03.203 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:19:03.203 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:03.217 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:19:03.278 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:03.275+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:03.337 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:03.337 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:03.337 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:03.337 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:03.337 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:03.350 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:19:03.379 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:03.375+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:03.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:19:03.480 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:03.477+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.874971909999999, 2024-09-17T21:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.865025273999997, 2024-09-17T21:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 40.621874433999999, 2024-09-17T21:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 40.612228948000002, 2024-09-17T21:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:03.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:19:03.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:19:03.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:03.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:19:03.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:19:03.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:03.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:19:03.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:19:03.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:19:03.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:19:03.504 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.504 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.504 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.504 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:03.504 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:03.504 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.243292915000001, 2024-09-17T21:19:03.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:19:03.505 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.505 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.505 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.505 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.505 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:03.505 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:03.505 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:19:03.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.505 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:03.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:03.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:19:03.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:19:03.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:19:03.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:19:03.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:19:03.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:19:03.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:19:03.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:19:03.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:19:03.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:19:03.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:19:03.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:19:03.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:19:03.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:19:03.508 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.508 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.509 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.509 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:565 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:03.509 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:19:03.509 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.23950395, 2024-09-17T21:19:03.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024193392000000001, 2024-09-17T21:19:03.509 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.509 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.509 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.509 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.509 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:03.509 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:03.509 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 565 2024-09-17T21:19:03.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.510 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:19:03.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:19:03.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239660+0000", 2024-09-17T21:19:03.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:19:03.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239666+0000", 2024-09-17T21:19:03.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8699999999999997e-06 2024-09-17T21:19:03.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239688+0000", 2024-09-17T21:19:03.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.213e-05 2024-09-17T21:19:03.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239703+0000", 2024-09-17T21:19:03.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4813e-05 2024-09-17T21:19:03.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263427+0000", 2024-09-17T21:19:03.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023724630999999999 2024-09-17T21:19:03.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263614+0000", 2024-09-17T21:19:03.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018664500000000001 2024-09-17T21:19:03.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263852+0000", 2024-09-17T21:19:03.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023793999999999999 2024-09-17T21:19:03.513 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.513 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.513 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.513 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:819 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5111808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:03.513 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:03.513 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.041720625, 2024-09-17T21:19:03.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024344811000000001, 2024-09-17T21:19:03.514 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.514 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.514 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.514 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.514 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:03.514 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:03.514 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 819 2024-09-17T21:19:03.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.514 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:03.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:03.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437443+0000", 2024-09-17T21:19:03.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1969999999999999e-06 2024-09-17T21:19:03.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437448+0000", 2024-09-17T21:19:03.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9490000000000004e-06 2024-09-17T21:19:03.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437467+0000", 2024-09-17T21:19:03.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8862999999999999e-05 2024-09-17T21:19:03.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437480+0000", 2024-09-17T21:19:03.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2833000000000001e-05 2024-09-17T21:19:03.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461552+0000", 2024-09-17T21:19:03.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024072695000000002 2024-09-17T21:19:03.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461639+0000", 2024-09-17T21:19:03.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6996000000000005e-05 2024-09-17T21:19:03.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461787+0000", 2024-09-17T21:19:03.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014727800000000001 2024-09-17T21:19:03.517 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.517 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.517 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.518 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:03.518 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:03.518 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0416083530000009, 2024-09-17T21:19:03.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:19:03.518 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.518 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.518 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.518 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.518 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:03.518 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:03.518 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:19:03.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.518 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:03.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:03.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:19:03.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:19:03.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:19:03.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:19:03.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:19:03.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:19:03.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:19:03.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:19:03.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:19:03.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:19:03.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:19:03.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:19:03.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:19:03.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:19:03.522 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.522 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.522 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.522 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:833 3.5 3:a772dfb1:::rbd_data.11523651b015.0000000000000001:d [sparse-read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:03.522 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:03.522 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.041174067, 2024-09-17T21:19:03.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024216024999999999, 2024-09-17T21:19:03.522 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.522 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.522 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.522 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.522 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:03.523 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:03.523 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 833 2024-09-17T21:19:03.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.523 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:03.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:03.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437990+0000", 2024-09-17T21:19:03.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2500000000000001e-06 2024-09-17T21:19:03.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437995+0000", 2024-09-17T21:19:03.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1240000000000004e-06 2024-09-17T21:19:03.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438012+0000", 2024-09-17T21:19:03.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7572e-05 2024-09-17T21:19:03.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438031+0000", 2024-09-17T21:19:03.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8612000000000001e-05 2024-09-17T21:19:03.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461830+0000", 2024-09-17T21:19:03.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023799135999999999 2024-09-17T21:19:03.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462005+0000", 2024-09-17T21:19:03.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000174747 2024-09-17T21:19:03.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:19:03.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019958399999999999 2024-09-17T21:19:03.526 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.526 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.526 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.526 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:03.526 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:03.526 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0411137759999995, 2024-09-17T21:19:03.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:19:03.527 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.527 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.527 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.527 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.527 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:03.527 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:03.527 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:19:03.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.527 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:03.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:03.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:19:03.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:19:03.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:19:03.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:19:03.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:19:03.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:19:03.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:19:03.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:19:03.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:19:03.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:19:03.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:19:03.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:19:03.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:19:03.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:19:03.531 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.531 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.531 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.531 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:03.531 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:03.531 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0410583950000003, 2024-09-17T21:19:03.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:19:03.531 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.531 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.531 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.531 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.531 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:03.532 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:03.532 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:19:03.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.532 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:03.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:03.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:19:03.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:19:03.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:19:03.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:19:03.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:19:03.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:19:03.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:19:03.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:19:03.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:19:03.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:19:03.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:19:03.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:19:03.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:19:03.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:19:03.535 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.535 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.535 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.535 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:03.535 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:03.535 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0410021169999997, 2024-09-17T21:19:03.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:19:03.535 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.536 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.536 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.536 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.536 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:03.536 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:03.536 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:19:03.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.536 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:03.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:03.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:19:03.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:19:03.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:19:03.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:19:03.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:19:03.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:19:03.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:19:03.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:19:03.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:19:03.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:19:03.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:19:03.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:19:03.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:19:03.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:19:03.539 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.539 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.539 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.540 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:03.540 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:03.540 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0400524779999998, 2024-09-17T21:19:03.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:19:03.540 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.540 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.540 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.540 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.540 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:03.540 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:03.540 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:19:03.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.541 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:03.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:03.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:19:03.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:19:03.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:19:03.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:19:03.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:19:03.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:19:03.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:19:03.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:19:03.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:19:03.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:19:03.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:19:03.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:19:03.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:19:03.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:19:03.544 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.544 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.544 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.544 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:03.544 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:03.544 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0399738579999998, 2024-09-17T21:19:03.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:19:03.544 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.545 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.545 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.545 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.545 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:03.545 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:03.545 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:19:03.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.545 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:03.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:03.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:19:03.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:19:03.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:19:03.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:19:03.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:19:03.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:19:03.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:19:03.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:19:03.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:19:03.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:19:03.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:19:03.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:19:03.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:19:03.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:19:03.548 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.548 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.548 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.549 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:847 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 4653056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:03.549 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439245+0000", 2024-09-17T21:19:03.549 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0399174200000001, 2024-09-17T21:19:03.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024172945000000001, 2024-09-17T21:19:03.549 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.549 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.549 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.549 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.549 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:03.549 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:03.549 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 847 2024-09-17T21:19:03.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.549 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439245+0000", 2024-09-17T21:19:03.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439245+0000", 2024-09-17T21:19:03.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439246+0000", 2024-09-17T21:19:03.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:19:03.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439252+0000", 2024-09-17T21:19:03.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3569999999999997e-06 2024-09-17T21:19:03.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439269+0000", 2024-09-17T21:19:03.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7316000000000002e-05 2024-09-17T21:19:03.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439282+0000", 2024-09-17T21:19:03.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3269e-05 2024-09-17T21:19:03.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463240+0000", 2024-09-17T21:19:03.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02395721 2024-09-17T21:19:03.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463306+0000", 2024-09-17T21:19:03.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6618000000000005e-05 2024-09-17T21:19:03.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463418+0000", 2024-09-17T21:19:03.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111672 2024-09-17T21:19:03.552 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.553 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.553 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.553 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:03.553 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:03.553 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0398609190000005, 2024-09-17T21:19:03.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:19:03.553 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.553 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.553 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.553 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.553 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:03.553 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:03.554 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:19:03.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.554 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:03.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:03.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:19:03.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:19:03.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:19:03.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:19:03.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:19:03.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:19:03.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:19:03.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:19:03.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:19:03.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:19:03.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:19:03.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:19:03.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.557 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:19:03.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:19:03.557 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.557 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.557 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.557 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:849 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5963776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:03.557 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:03.557 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0392668829999998, 2024-09-17T21:19:03.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024537514, 2024-09-17T21:19:03.558 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.558 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.558 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.558 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.558 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:03.558 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:03.558 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 849 2024-09-17T21:19:03.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.558 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:03.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:03.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439897+0000", 2024-09-17T21:19:03.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9099999999999999e-06 2024-09-17T21:19:03.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439905+0000", 2024-09-17T21:19:03.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0870000000000007e-06 2024-09-17T21:19:03.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439924+0000", 2024-09-17T21:19:03.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9042000000000002e-05 2024-09-17T21:19:03.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439941+0000", 2024-09-17T21:19:03.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.624e-05 2024-09-17T21:19:03.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464132+0000", 2024-09-17T21:19:03.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024191579000000001 2024-09-17T21:19:03.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464291+0000", 2024-09-17T21:19:03.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015904800000000001 2024-09-17T21:19:03.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464433+0000", 2024-09-17T21:19:03.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014160799999999999 2024-09-17T21:19:03.561 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.561 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.561 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.562 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:03.562 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:03.562 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0391998650000005, 2024-09-17T21:19:03.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:19:03.562 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.562 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.562 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.562 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.562 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:03.562 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:03.562 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:19:03.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.562 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:03.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:03.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:19:03.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:19:03.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:19:03.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:19:03.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:19:03.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:19:03.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:19:03.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:19:03.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:19:03.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:19:03.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:19:03.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:19:03.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:19:03.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:19:03.566 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.566 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.566 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.566 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:03.566 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:03.566 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0391350540000008, 2024-09-17T21:19:03.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:19:03.566 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.566 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.566 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.566 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.566 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:03.566 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:03.567 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:19:03.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.567 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:03.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:03.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:19:03.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:19:03.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:19:03.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:19:03.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:19:03.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:19:03.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:19:03.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:19:03.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:19:03.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:19:03.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:19:03.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:19:03.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:19:03.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:19:03.570 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.570 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.570 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.570 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:03.570 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:03.570 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0390740770000004, 2024-09-17T21:19:03.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:19:03.571 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:03.571 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:03.571 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:03.571 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:03.571 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:03.571 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:03.571 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:19:03.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.571 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:03.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:03.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:03.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:03.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:03.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:03.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:03.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:19:03.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:19:03.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:03.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:19:03.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:19:03.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:03.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:19:03.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:19:03.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:03.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:19:03.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:19:03.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:03.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:19:03.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:19:03.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:03.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:19:03.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:19:03.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:03.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:03.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:03.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:19:03.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:19:03.574 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.574 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.574 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.574 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:03.574 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:03.574 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:03.575 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:19:03.575 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:19:03.575 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T21:19:03.575 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 0.000000 2024-09-17T21:19:03.575 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 0 2024-09-17T21:19:03.577 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:19:03.624 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:03.623+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:03.681 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:03.677+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:03.692 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:19:03.692 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:03.710 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:19:03.800 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:03.799+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:03.865 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:03.866 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:03.879 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:19:03.921 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:03.919+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:03.984 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:03.983+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:04.051 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:04.051 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:04.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:389 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.194936+0000", 2024-09-17T21:19:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.852264038999998, 2024-09-17T21:19:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041176510999999999, 2024-09-17T21:19:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 389 2024-09-17T21:19:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194936+0000", 2024-09-17T21:19:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194936+0000", 2024-09-17T21:19:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194937+0000", 2024-09-17T21:19:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6270000000000001e-06 2024-09-17T21:19:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194944+0000", 2024-09-17T21:19:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5729999999999996e-06 2024-09-17T21:19:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194961+0000", 2024-09-17T21:19:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7623000000000001e-05 2024-09-17T21:19:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194974+0000", 2024-09-17T21:19:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2989999999999999e-05 2024-09-17T21:19:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.235848+0000", 2024-09-17T21:19:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.040874099999999997 2024-09-17T21:19:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.235971+0000", 2024-09-17T21:19:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012254400000000001 2024-09-17T21:19:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236112+0000", 2024-09-17T21:19:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014105399999999999 2024-09-17T21:19:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:406 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 2359296~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:19:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.852145233000002, 2024-09-17T21:19:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042448018999999997, 2024-09-17T21:19:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 406 2024-09-17T21:19:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:19:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:19:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195056+0000", 2024-09-17T21:19:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2729999999999999e-06 2024-09-17T21:19:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195060+0000", 2024-09-17T21:19:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5399999999999997e-06 2024-09-17T21:19:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195076+0000", 2024-09-17T21:19:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6217000000000001e-05 2024-09-17T21:19:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195093+0000", 2024-09-17T21:19:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.683e-05 2024-09-17T21:19:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236217+0000", 2024-09-17T21:19:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041124102000000003 2024-09-17T21:19:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236339+0000", 2024-09-17T21:19:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121876 2024-09-17T21:19:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237502+0000", 2024-09-17T21:19:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011631809999999999 2024-09-17T21:19:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:416 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2228224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:19:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.851647206999999, 2024-09-17T21:19:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042361990000000002, 2024-09-17T21:19:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 416 2024-09-17T21:19:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:19:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:19:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195554+0000", 2024-09-17T21:19:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9300000000000002e-06 2024-09-17T21:19:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195563+0000", 2024-09-17T21:19:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2770000000000005e-06 2024-09-17T21:19:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195586+0000", 2024-09-17T21:19:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3889000000000001e-05 2024-09-17T21:19:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195603+0000", 2024-09-17T21:19:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6084000000000001e-05 2024-09-17T21:19:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237563+0000", 2024-09-17T21:19:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041960856999999997 2024-09-17T21:19:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237695+0000", 2024-09-17T21:19:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013144400000000001 2024-09-17T21:19:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237914+0000", 2024-09-17T21:19:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000219509 2024-09-17T21:19:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:439 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:19:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.851083305000003, 2024-09-17T21:19:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043053811999999997, 2024-09-17T21:19:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 439 2024-09-17T21:19:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:19:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:19:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196118+0000", 2024-09-17T21:19:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3969999999999999e-06 2024-09-17T21:19:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196157+0000", 2024-09-17T21:19:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9826999999999997e-05 2024-09-17T21:19:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196179+0000", 2024-09-17T21:19:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1756e-05 2024-09-17T21:19:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196194+0000", 2024-09-17T21:19:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4707e-05 2024-09-17T21:19:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237980+0000", 2024-09-17T21:19:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041786219999999999 2024-09-17T21:19:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.238065+0000", 2024-09-17T21:19:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4560000000000004e-05 2024-09-17T21:19:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239170+0000", 2024-09-17T21:19:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001105345 2024-09-17T21:19:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:440 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:19:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.850980802000002, 2024-09-17T21:19:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043890039999999998, 2024-09-17T21:19:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 440 2024-09-17T21:19:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:19:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:19:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196220+0000", 2024-09-17T21:19:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.254e-06 2024-09-17T21:19:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196225+0000", 2024-09-17T21:19:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7829999999999998e-06 2024-09-17T21:19:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196241+0000", 2024-09-17T21:19:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6322999999999999e-05 2024-09-17T21:19:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196258+0000", 2024-09-17T21:19:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7056000000000001e-05 2024-09-17T21:19:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239229+0000", 2024-09-17T21:19:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042971058999999999 2024-09-17T21:19:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239403+0000", 2024-09-17T21:19:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017399200000000001 2024-09-17T21:19:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240109+0000", 2024-09-17T21:19:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00070557300000000003 2024-09-17T21:19:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:450 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:19:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.850619850999998, 2024-09-17T21:19:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043994521000000002, 2024-09-17T21:19:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 450 2024-09-17T21:19:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:19:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:19:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196581+0000", 2024-09-17T21:19:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4500000000000001e-06 2024-09-17T21:19:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196588+0000", 2024-09-17T21:19:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7530000000000004e-06 2024-09-17T21:19:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196607+0000", 2024-09-17T21:19:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8987e-05 2024-09-17T21:19:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196620+0000", 2024-09-17T21:19:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3589e-05 2024-09-17T21:19:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240225+0000", 2024-09-17T21:19:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043604878999999999 2024-09-17T21:19:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240377+0000", 2024-09-17T21:19:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015146699999999999 2024-09-17T21:19:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240574+0000", 2024-09-17T21:19:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019739599999999999 2024-09-17T21:19:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:464 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:19:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.850369430000001, 2024-09-17T21:19:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044651692999999999, 2024-09-17T21:19:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 464 2024-09-17T21:19:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:19:04.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:04.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:19:04.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196838+0000", 2024-09-17T21:19:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4970000000000004e-06 2024-09-17T21:19:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196859+0000", 2024-09-17T21:19:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1166e-05 2024-09-17T21:19:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196879+0000", 2024-09-17T21:19:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.995e-05 2024-09-17T21:19:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196896+0000", 2024-09-17T21:19:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6929999999999999e-05 2024-09-17T21:19:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240617+0000", 2024-09-17T21:19:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043721784999999999 2024-09-17T21:19:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240702+0000", 2024-09-17T21:19:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4060999999999994e-05 2024-09-17T21:19:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241482+0000", 2024-09-17T21:19:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000780304 2024-09-17T21:19:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:475 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:19:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.850100298000001, 2024-09-17T21:19:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044661981000000003, 2024-09-17T21:19:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 475 2024-09-17T21:19:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:19:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:19:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197100+0000", 2024-09-17T21:19:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.206e-06 2024-09-17T21:19:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197105+0000", 2024-09-17T21:19:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.527e-06 2024-09-17T21:19:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197124+0000", 2024-09-17T21:19:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:19:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197136+0000", 2024-09-17T21:19:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2857000000000001e-05 2024-09-17T21:19:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241539+0000", 2024-09-17T21:19:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044402077999999998 2024-09-17T21:19:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241653+0000", 2024-09-17T21:19:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114931 2024-09-17T21:19:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241761+0000", 2024-09-17T21:19:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107753 2024-09-17T21:19:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:476 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:19:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.850046472999999, 2024-09-17T21:19:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045038976000000001, 2024-09-17T21:19:04.096 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:04.096 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:04.096 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:04.096 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:04.096 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:04.096 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:04.096 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 476 2024-09-17T21:19:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:19:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:19:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197154+0000", 2024-09-17T21:19:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.127e-06 2024-09-17T21:19:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197159+0000", 2024-09-17T21:19:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4569999999999998e-06 2024-09-17T21:19:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:04.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197174+0000", 2024-09-17T21:19:04.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5173e-05 2024-09-17T21:19:04.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:04.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197186+0000", 2024-09-17T21:19:04.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:19:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241863+0000", 2024-09-17T21:19:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044676826000000003 2024-09-17T21:19:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241998+0000", 2024-09-17T21:19:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013476100000000001 2024-09-17T21:19:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242192+0000", 2024-09-17T21:19:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019436900000000001 2024-09-17T21:19:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:494 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.849871290000003, 2024-09-17T21:19:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046244995999999997, 2024-09-17T21:19:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 494 2024-09-17T21:19:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197329+0000", 2024-09-17T21:19:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.173e-06 2024-09-17T21:19:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197334+0000", 2024-09-17T21:19:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0499999999999999e-06 2024-09-17T21:19:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197352+0000", 2024-09-17T21:19:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7133e-05 2024-09-17T21:19:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197367+0000", 2024-09-17T21:19:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5123000000000001e-05 2024-09-17T21:19:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:19:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044989829000000002 2024-09-17T21:19:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242526+0000", 2024-09-17T21:19:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016968199999999999 2024-09-17T21:19:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243573+0000", 2024-09-17T21:19:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010470060000000001 2024-09-17T21:19:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:502 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:19:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.847637630999998, 2024-09-17T21:19:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044346086999999999, 2024-09-17T21:19:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 502 2024-09-17T21:19:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:19:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:19:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199563+0000", 2024-09-17T21:19:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2929999999999999e-06 2024-09-17T21:19:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199569+0000", 2024-09-17T21:19:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5330000000000002e-06 2024-09-17T21:19:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199614+0000", 2024-09-17T21:19:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4793000000000003e-05 2024-09-17T21:19:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199628+0000", 2024-09-17T21:19:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4037e-05 2024-09-17T21:19:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243665+0000", 2024-09-17T21:19:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044037106999999999 2024-09-17T21:19:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243767+0000", 2024-09-17T21:19:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010279499999999999 2024-09-17T21:19:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243908+0000", 2024-09-17T21:19:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140529 2024-09-17T21:19:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:523 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:19:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.847191477999999, 2024-09-17T21:19:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044789645000000003, 2024-09-17T21:19:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 523 2024-09-17T21:19:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:19:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:19:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:19:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9e-06 2024-09-17T21:19:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200021+0000", 2024-09-17T21:19:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9499999999999998e-06 2024-09-17T21:19:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200038+0000", 2024-09-17T21:19:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.73e-05 2024-09-17T21:19:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200051+0000", 2024-09-17T21:19:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.322e-05 2024-09-17T21:19:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243987+0000", 2024-09-17T21:19:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043935386999999999 2024-09-17T21:19:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244136+0000", 2024-09-17T21:19:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014868300000000001 2024-09-17T21:19:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244798+0000", 2024-09-17T21:19:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00066220500000000002 2024-09-17T21:19:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:524 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.847125261999999, 2024-09-17T21:19:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045737414999999997, 2024-09-17T21:19:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 524 2024-09-17T21:19:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200076+0000", 2024-09-17T21:19:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:19:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200081+0000", 2024-09-17T21:19:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7030000000000003e-06 2024-09-17T21:19:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200099+0000", 2024-09-17T21:19:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7836999999999999e-05 2024-09-17T21:19:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200113+0000", 2024-09-17T21:19:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3570000000000001e-05 2024-09-17T21:19:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244866+0000", 2024-09-17T21:19:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044753573999999997 2024-09-17T21:19:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244972+0000", 2024-09-17T21:19:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106111 2024-09-17T21:19:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245812+0000", 2024-09-17T21:19:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083923999999999999 2024-09-17T21:19:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:525 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3670016~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.84697388, 2024-09-17T21:19:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046645697, 2024-09-17T21:19:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 525 2024-09-17T21:19:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:19:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:19:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200234+0000", 2024-09-17T21:19:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:19:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200254+0000", 2024-09-17T21:19:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9959999999999999e-05 2024-09-17T21:19:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200293+0000", 2024-09-17T21:19:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9755999999999997e-05 2024-09-17T21:19:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245871+0000", 2024-09-17T21:19:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045577871999999998 2024-09-17T21:19:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246030+0000", 2024-09-17T21:19:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158975 2024-09-17T21:19:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246871+0000", 2024-09-17T21:19:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084111699999999995 2024-09-17T21:19:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.846728204999998, 2024-09-17T21:19:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:19:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:19:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:19:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:19:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:19:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:19:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:19:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:19:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:19:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:19:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:19:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:19:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:19:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:19:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:19:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:19:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.846549523, 2024-09-17T21:19:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:19:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:19:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:19:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:19:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:19:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:19:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:19:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:19:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:19:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:19:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:19:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:19:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:19:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:19:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:19:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:19:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.846490250000002, 2024-09-17T21:19:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:19:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:19:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:19:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:19:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:19:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:19:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:19:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:19:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:19:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:19:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:19:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:19:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:19:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:19:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:19:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:19:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.846348055, 2024-09-17T21:19:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:19:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:19:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:19:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:19:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:19:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:19:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:19:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:19:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:19:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:19:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:19:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:19:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:19:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:19:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:19:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:19:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.846292132000002, 2024-09-17T21:19:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:19:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:19:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:19:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:19:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:19:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:19:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:19:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:19:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:19:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:19:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:19:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:19:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:19:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:19:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:19:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:19:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.846238515000003, 2024-09-17T21:19:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:19:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:19:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:19:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:19:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:19:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:19:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:19:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:19:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:19:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:19:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:19:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:19:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:19:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:19:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:19:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:19:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:04.161 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:04.163 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:19:04.163 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:04.161+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:04.164 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-17T21:19:04.248 INFO:teuthology.orchestra.run.smithi184.stdout:1686 write 0xcd23c00 thru 0xcd2919a (0x559b bytes) 2024-09-17T21:19:04.250 INFO:teuthology.orchestra.run.smithi184.stdout:1687 write 0x4112200 thru 0x411c071 (0x9e72 bytes) 2024-09-17T21:19:04.251 INFO:teuthology.orchestra.run.smithi184.stdout:1690 punch from 0x881da00 to 0x8821c00, (0x4200 bytes) 2024-09-17T21:19:04.264 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:04.261+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:04.274 INFO:teuthology.orchestra.run.smithi184.stdout:1691 read 0x2049200 thru 0x204bda4 (0x2ba5 bytes) 2024-09-17T21:19:04.274 INFO:teuthology.orchestra.run.smithi184.stdout:1693 punch from 0x9b1e800 to 0x9b25c00, (0x7400 bytes) 2024-09-17T21:19:04.331 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:19:04.332 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:04.344 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:19:04.371 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:19:04.383 INFO:teuthology.orchestra.run.smithi184.stdout:1695 trunc from 0xe4ac000 to 0x2c67c00 2024-09-17T21:19:04.388 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:04.387+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:set osd.5 primary-affinity to 0 (802) 2024-09-17T21:19:04.398 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-17T21:19:04.465 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:04.461+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:04.469 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:04.469 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:04.483 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:19:04.534 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:04.534 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:04.550 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:04.550 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:04.565 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:04.563+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:04.566 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:04.566 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:04.586 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:04.586 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:04.601 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:04.601 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:04.605 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:04.606 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:04.615 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:19:04.626 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:04.626 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:04.666 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:04.665+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:04.744 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:04.744 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:04.759 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:19:04.804 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:04.803+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:04.868 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:04.865+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:04.881 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:04.881 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:04.895 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:19:04.969 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:04.969+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:05.059 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:05.059 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:05.070 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:05.069+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:05.073 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:19:05.171 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:05.169+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:05.205 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:05.205 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:05.219 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:19:05.273 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:05.271+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:05.374 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:05.371+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:05.385 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:05.385 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:05.399 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:19:05.460 INFO:teuthology.orchestra.run.smithi184.stdout:1697 punch from 0x201a600 to 0x2024400, (0x9e00 bytes) 2024-09-17T21:19:05.474 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:05.473+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:05.490 INFO:teuthology.orchestra.run.smithi184.stdout:1698 write 0xaccda00 thru 0xacdca66 (0xf067 bytes) 2024-09-17T21:19:05.495 INFO:teuthology.orchestra.run.smithi184.stdout:1699 write 0x80fbe00 thru 0x810b6cb (0xf8cc bytes) 2024-09-17T21:19:05.496 INFO:teuthology.orchestra.run.smithi184.stdout:1700 read 0x515d000 thru 0x51626c9 (0x56ca bytes) 2024-09-17T21:19:05.533 INFO:teuthology.orchestra.run.smithi184.stdout:1701 write 0xec1c200 thru 0xec2620e (0xa00f bytes) 2024-09-17T21:19:05.536 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:05.537 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:05.551 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:19:05.561 INFO:teuthology.orchestra.run.smithi184.stdout:1703 trunc from 0xec26400 to 0x4a14e00 2024-09-17T21:19:05.594 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:05.593+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:05.596 INFO:teuthology.orchestra.run.smithi184.stdout:1709 punch from 0x26c8a00 to 0x26cba00, (0x3000 bytes) 2024-09-17T21:19:05.600 INFO:teuthology.orchestra.run.smithi184.stdout:1710 trunc from 0x4a14e00 to 0x38ea400 2024-09-17T21:19:05.640 INFO:teuthology.orchestra.run.smithi184.stdout:1711 write 0xa835400 thru 0xa83bef4 (0x6af5 bytes) 2024-09-17T21:19:05.643 INFO:teuthology.orchestra.run.smithi184.stdout:1718 write 0x6a6be00 thru 0x6a7a4b0 (0xe6b1 bytes) 2024-09-17T21:19:05.643 INFO:teuthology.orchestra.run.smithi184.stdout:1719 punch from 0x12d0800 to 0x12dc600, (0xbe00 bytes) 2024-09-17T21:19:05.648 INFO:teuthology.orchestra.run.smithi184.stdout:1721 trunc from 0xa83c000 to 0xcf11a00 2024-09-17T21:19:05.662 INFO:teuthology.orchestra.run.smithi184.stdout:1723 write 0x2e0b600 thru 0x2e0bc5f (0x660 bytes) 2024-09-17T21:19:05.676 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:05.673+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:05.696 INFO:teuthology.orchestra.run.smithi184.stdout:1727 trunc from 0xcf11a00 to 0x1e08c00 2024-09-17T21:19:05.728 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:05.729 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:05.745 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:19:05.746 INFO:teuthology.orchestra.run.smithi184.stdout:1728 punch from 0x9d2200 to 0x9df400, (0xd200 bytes) 2024-09-17T21:19:05.775 INFO:teuthology.orchestra.run.smithi184.stdout:1729 trunc from 0x1e08c00 to 0xcc63200 2024-09-17T21:19:05.776 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:05.775+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:05.801 INFO:teuthology.orchestra.run.smithi184.stdout:1731 trunc from 0xcc63200 to 0xaac3400 2024-09-17T21:19:05.821 INFO:teuthology.orchestra.run.smithi184.stdout:1732 read 0x3c17800 thru 0x3c23c82 (0xc483 bytes) 2024-09-17T21:19:05.824 INFO:teuthology.orchestra.run.smithi184.stdout:1734 trunc from 0xaac3400 to 0xa911e00 2024-09-17T21:19:05.838 INFO:teuthology.orchestra.run.smithi184.stdout:1737 write 0xa3ca000 thru 0xa3d1509 (0x750a bytes) 2024-09-17T21:19:05.840 INFO:teuthology.orchestra.run.smithi184.stdout:1740 write 0x4391600 thru 0x439c09e (0xaa9f bytes) 2024-09-17T21:19:05.843 INFO:teuthology.orchestra.run.smithi184.stdout:1741 punch from 0x4576200 to 0x4582000, (0xbe00 bytes) 2024-09-17T21:19:05.844 INFO:teuthology.orchestra.run.smithi184.stdout:1742 punch from 0x41cb400 to 0x41dac00, (0xf800 bytes) 2024-09-17T21:19:05.844 INFO:teuthology.orchestra.run.smithi184.stdout:1744 read 0x9478a00 thru 0x9484ab9 (0xc0ba bytes) 2024-09-17T21:19:05.845 INFO:teuthology.orchestra.run.smithi184.stdout:1745 read 0x24ffc00 thru 0x25008f8 (0xcf9 bytes) 2024-09-17T21:19:05.846 INFO:teuthology.orchestra.run.smithi184.stdout:1747 trunc from 0xa911e00 to 0xaf6f600 2024-09-17T21:19:05.869 INFO:teuthology.orchestra.run.smithi184.stdout:1748 read 0x401c600 thru 0x4023406 (0x6e07 bytes) 2024-09-17T21:19:05.871 INFO:teuthology.orchestra.run.smithi184.stdout:1749 read 0x9571a00 thru 0x9571f28 (0x529 bytes) 2024-09-17T21:19:05.871 INFO:teuthology.orchestra.run.smithi184.stdout:1751 read 0x608400 thru 0x60fb2f (0x7730 bytes) 2024-09-17T21:19:05.872 INFO:teuthology.orchestra.run.smithi184.stdout:1753 read 0x24dfa00 thru 0x24e8c20 (0x9221 bytes) 2024-09-17T21:19:05.877 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:05.873+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:05.893 INFO:teuthology.orchestra.run.smithi184.stdout:1755 write 0xbbcb600 thru 0xbbd77e8 (0xc1e9 bytes) 2024-09-17T21:19:05.896 INFO:teuthology.orchestra.run.smithi184.stdout:1757 punch from 0x1ac1800 to 0x1ac8000, (0x6800 bytes) 2024-09-17T21:19:05.896 INFO:teuthology.orchestra.run.smithi184.stdout:1759 read 0x6af0200 thru 0x6afff74 (0xfd75 bytes) 2024-09-17T21:19:05.897 INFO:teuthology.orchestra.run.smithi184.stdout:1761 read 0x74cc800 thru 0x74dac1a (0xe41b bytes) 2024-09-17T21:19:05.916 INFO:teuthology.orchestra.run.smithi184.stdout:1762 trunc from 0xbbd7800 to 0x3615000 2024-09-17T21:19:05.922 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:05.922 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:05.939 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:19:05.950 INFO:teuthology.orchestra.run.smithi184.stdout:1764 read 0x2a5a000 thru 0x2a6747e (0xd47f bytes) 2024-09-17T21:19:05.971 INFO:teuthology.orchestra.run.smithi184.stdout:1767 write 0x647ae00 thru 0x6488185 (0xd386 bytes) 2024-09-17T21:19:05.986 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:05.985+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:06.004 INFO:teuthology.orchestra.run.smithi184.stdout:1768 write 0xe680600 thru 0xe685053 (0x4a54 bytes) 2024-09-17T21:19:06.005 INFO:teuthology.orchestra.run.smithi184.stdout:1769 read 0xdf91200 thru 0xdf9f80e (0xe60f bytes) 2024-09-17T21:19:06.006 INFO:teuthology.orchestra.run.smithi184.stdout:1770 write 0x3207000 thru 0x3211a13 (0xaa14 bytes) 2024-09-17T21:19:06.007 INFO:teuthology.orchestra.run.smithi184.stdout:1773 write 0xcf67000 thru 0xcf7154a (0xa54b bytes) 2024-09-17T21:19:06.008 INFO:teuthology.orchestra.run.smithi184.stdout:1774 read 0x514f400 thru 0x5157245 (0x7e46 bytes) 2024-09-17T21:19:06.016 INFO:teuthology.orchestra.run.smithi184.stdout:1776 trunc from 0xe685200 to 0xb157600 2024-09-17T21:19:06.060 INFO:teuthology.orchestra.run.smithi184.stdout:1779 write 0xd666c00 thru 0xd66b7e8 (0x4be9 bytes) 2024-09-17T21:19:06.062 INFO:teuthology.orchestra.run.smithi184.stdout:1780 punch from 0x16c7c00 to 0x16d0400, (0x8800 bytes) 2024-09-17T21:19:06.079 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:06.077+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:06.081 INFO:teuthology.orchestra.run.smithi184.stdout:1782 write 0xe4ff600 thru 0xe504813 (0x5214 bytes) 2024-09-17T21:19:06.084 INFO:teuthology.orchestra.run.smithi184.stdout:1786 write 0x918000 thru 0x922b6d (0xab6e bytes) 2024-09-17T21:19:06.085 INFO:teuthology.orchestra.run.smithi184.stdout:1788 read 0x4fcb200 thru 0x4fce56a (0x336b bytes) 2024-09-17T21:19:06.085 INFO:teuthology.orchestra.run.smithi184.stdout:1790 write 0x8f15c00 thru 0x8f17b16 (0x1f17 bytes) 2024-09-17T21:19:06.088 INFO:teuthology.orchestra.run.smithi184.stdout:1794 read 0xb9fa600 thru 0xb9fac0e (0x60f bytes) 2024-09-17T21:19:06.088 INFO:teuthology.orchestra.run.smithi184.stdout:1795 write 0x7daf600 thru 0x7dbb27c (0xbc7d bytes) 2024-09-17T21:19:06.088 INFO:teuthology.orchestra.run.smithi184.stdout:1797 write 0xd3e3a00 thru 0xd3f13d2 (0xd9d3 bytes) 2024-09-17T21:19:06.105 INFO:teuthology.orchestra.run.smithi184.stdout:1798 trunc from 0xe504a00 to 0x70c6200 2024-09-17T21:19:06.121 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:06.122 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:06.138 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:19:06.145 INFO:teuthology.orchestra.run.smithi184.stdout:1799 read 0x6b83a00 thru 0x6b922da (0xe8db bytes) 2024-09-17T21:19:06.146 INFO:teuthology.orchestra.run.smithi184.stdout:1800 write 0x1dd1200 thru 0x1ddbe47 (0xac48 bytes) 2024-09-17T21:19:06.148 INFO:teuthology.orchestra.run.smithi184.stdout:1801 write 0x5cd7e00 thru 0x5ce1acc (0x9ccd bytes) 2024-09-17T21:19:06.149 INFO:teuthology.orchestra.run.smithi184.stdout:1802 punch from 0x131b000 to 0x1328400, (0xd400 bytes) 2024-09-17T21:19:06.150 INFO:teuthology.orchestra.run.smithi184.stdout:1803 write 0xc10000 thru 0xc1cefc (0xcefd bytes) 2024-09-17T21:19:06.171 INFO:teuthology.orchestra.run.smithi184.stdout:1813 write 0x998f200 thru 0x999130d (0x210e bytes) 2024-09-17T21:19:06.182 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:06.181+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:06.199 INFO:teuthology.orchestra.run.smithi184.stdout:1814 write 0xe1ff000 thru 0xe20a429 (0xb42a bytes) 2024-09-17T21:19:06.201 INFO:teuthology.orchestra.run.smithi184.stdout:1815 write 0x30bd200 thru 0x30c96c2 (0xc4c3 bytes) 2024-09-17T21:19:06.202 INFO:teuthology.orchestra.run.smithi184.stdout:1816 read 0xad43200 thru 0xad5115b (0xdf5c bytes) 2024-09-17T21:19:06.228 INFO:teuthology.orchestra.run.smithi184.stdout:1820 trunc from 0xe20a600 to 0x50d6000 2024-09-17T21:19:06.266 INFO:teuthology.orchestra.run.smithi184.stdout:1821 read 0x1847000 thru 0x184fff7 (0x8ff8 bytes) 2024-09-17T21:19:06.281 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:06.279+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:06.296 INFO:teuthology.orchestra.run.smithi184.stdout:1822 write 0xa0af200 thru 0xa0b60c5 (0x6ec6 bytes) 2024-09-17T21:19:06.298 INFO:teuthology.orchestra.run.smithi184.stdout:1824 write 0x4699800 thru 0x469a655 (0xe56 bytes) 2024-09-17T21:19:06.299 INFO:teuthology.orchestra.run.smithi184.stdout:1825 write 0x3609000 thru 0x360b3e1 (0x23e2 bytes) 2024-09-17T21:19:06.300 INFO:teuthology.orchestra.run.smithi184.stdout:1826 punch from 0xb1f800 to 0xb2b200, (0xba00 bytes) 2024-09-17T21:19:06.309 INFO:teuthology.orchestra.run.smithi184.stdout:1827 trunc from 0xa0b6200 to 0xea7a400 2024-09-17T21:19:06.320 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:06.320 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:06.321 INFO:teuthology.orchestra.run.smithi184.stdout:1828 write 0xd9a1000 thru 0xd9ab2f7 (0xa2f8 bytes) 2024-09-17T21:19:06.323 INFO:teuthology.orchestra.run.smithi184.stdout:1832 punch from 0x1903c00 to 0x1909e00, (0x6200 bytes) 2024-09-17T21:19:06.324 INFO:teuthology.orchestra.run.smithi184.stdout:1834 punch from 0xad5ba00 to 0xad60c00, (0x5200 bytes) 2024-09-17T21:19:06.324 INFO:teuthology.orchestra.run.smithi184.stdout:1835 punch from 0xa201400 to 0xa20e200, (0xce00 bytes) 2024-09-17T21:19:06.325 INFO:teuthology.orchestra.run.smithi184.stdout:1839 write 0x577cc00 thru 0x57888a2 (0xbca3 bytes) 2024-09-17T21:19:06.326 INFO:teuthology.orchestra.run.smithi184.stdout:1842 write 0x6f0a200 thru 0x6f16b35 (0xc936 bytes) 2024-09-17T21:19:06.328 INFO:teuthology.orchestra.run.smithi184.stdout:1843 read 0xe133c00 thru 0xe142732 (0xeb33 bytes) 2024-09-17T21:19:06.337 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:19:06.338 INFO:teuthology.orchestra.run.smithi184.stdout:1845 trunc from 0xea7a400 to 0x92f1200 2024-09-17T21:19:06.361 INFO:teuthology.orchestra.run.smithi184.stdout:1846 punch from 0x68d0a00 to 0x68d1400, (0xa00 bytes) 2024-09-17T21:19:06.363 INFO:teuthology.orchestra.run.smithi184.stdout:1848 punch from 0x6fd7600 to 0x6fdc600, (0x5000 bytes) 2024-09-17T21:19:06.376 INFO:teuthology.orchestra.run.smithi184.stdout:1850 write 0xb15a800 thru 0xb15ce84 (0x2685 bytes) 2024-09-17T21:19:06.381 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:06.379+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:06.391 INFO:teuthology.orchestra.run.smithi184.stdout:1852 trunc from 0xb15d000 to 0x3c36c00 2024-09-17T21:19:06.416 INFO:teuthology.orchestra.run.smithi184.stdout:1854 write 0xd55600 thru 0xd6101d (0xba1e bytes) 2024-09-17T21:19:06.418 INFO:teuthology.orchestra.run.smithi184.stdout:1857 read 0x2f5f200 thru 0x2f69afe (0xa8ff bytes) 2024-09-17T21:19:06.419 INFO:teuthology.orchestra.run.smithi184.stdout:1859 read 0x2e78c00 thru 0x2e8500e (0xc40f bytes) 2024-09-17T21:19:06.420 INFO:teuthology.orchestra.run.smithi184.stdout:1860 punch from 0x2631800 to 0x2635800, (0x4000 bytes) 2024-09-17T21:19:06.420 INFO:teuthology.orchestra.run.smithi184.stdout:1861 read 0x39ddc00 thru 0x39ec6ce (0xeacf bytes) 2024-09-17T21:19:06.421 INFO:teuthology.orchestra.run.smithi184.stdout:truncating to largest ever: 0xede7800 2024-09-17T21:19:06.422 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:19:06.440 INFO:teuthology.orchestra.run.smithi184.stdout:1864 trunc from 0x3c36c00 to 0xede7800 2024-09-17T21:19:06.452 INFO:teuthology.orchestra.run.smithi184.stdout:1865 write 0xdbcf200 thru 0xdbd5b78 (0x6979 bytes) 2024-09-17T21:19:06.467 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:06.467 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:06.468 INFO:teuthology.orchestra.run.smithi184.stdout:1866 trunc from 0xede7800 to 0x58b1200 2024-09-17T21:19:06.482 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:06.481+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:06.483 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:19:06.499 INFO:teuthology.orchestra.run.smithi184.stdout:1868 read 0x3759a00 thru 0x3762feb (0x95ec bytes) 2024-09-17T21:19:06.517 INFO:teuthology.orchestra.run.smithi184.stdout:1869 write 0x82bae00 thru 0x82c3862 (0x8a63 bytes) 2024-09-17T21:19:06.541 INFO:teuthology.orchestra.run.smithi184.stdout:1870 write 0xd193000 thru 0xd19d781 (0xa782 bytes) 2024-09-17T21:19:06.544 INFO:teuthology.orchestra.run.smithi184.stdout:1872 punch from 0xa9b6000 to 0xa9bdc00, (0x7c00 bytes) 2024-09-17T21:19:06.544 INFO:teuthology.orchestra.run.smithi184.stdout:1873 write 0x67aa800 thru 0x67b8fa3 (0xe7a4 bytes) 2024-09-17T21:19:06.548 INFO:teuthology.orchestra.run.smithi184.stdout:1878 read 0x72dfc00 thru 0x72e71f2 (0x75f3 bytes) 2024-09-17T21:19:06.551 INFO:teuthology.orchestra.run.smithi184.stdout:1881 trunc from 0xd19d800 to 0xb8c7200 2024-09-17T21:19:06.570 INFO:teuthology.orchestra.run.smithi184.stdout:1883 trunc from 0xb8c7200 to 0xc6b2800 2024-09-17T21:19:06.583 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:06.581+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:06.588 INFO:teuthology.orchestra.run.smithi184.stdout:1884 trunc from 0xc6b2800 to 0xa31d400 2024-09-17T21:19:06.605 INFO:teuthology.orchestra.run.smithi184.stdout:1886 punch from 0x6c56200 to 0x6c57200, (0x1000 bytes) 2024-09-17T21:19:06.608 INFO:teuthology.orchestra.run.smithi184.stdout:1887 write 0x97f5200 thru 0x97f792e (0x272f bytes) 2024-09-17T21:19:06.609 INFO:teuthology.orchestra.run.smithi184.stdout:1888 read 0x98f2600 thru 0x98f7aa8 (0x54a9 bytes) 2024-09-17T21:19:06.610 INFO:teuthology.orchestra.run.smithi184.stdout:1889 read 0xdcfe00 thru 0xddf5b9 (0xf7ba bytes) 2024-09-17T21:19:06.610 INFO:teuthology.orchestra.run.smithi184.stdout:1890 write 0x4cb1000 thru 0x4cbb25a (0xa25b bytes) 2024-09-17T21:19:06.621 INFO:teuthology.orchestra.run.smithi184.stdout:1893 trunc from 0xa31d400 to 0x4b29200 2024-09-17T21:19:06.643 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:19:06.662 INFO:teuthology.orchestra.run.smithi184.stdout:1895 trunc from 0x4b29200 to 0xaeb8000 2024-09-17T21:19:06.680 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:06.680 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:06.683 INFO:teuthology.orchestra.run.smithi184.stdout:1896 trunc from 0xaeb8000 to 0x8c1c600 2024-09-17T21:19:06.688 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:06.687+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:06.697 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:19:06.703 INFO:teuthology.orchestra.run.smithi184.stdout:1898 read 0x7a57a00 thru 0x7a6626c (0xe86d bytes) 2024-09-17T21:19:06.706 INFO:teuthology.orchestra.run.smithi184.stdout:1900 read 0x275d600 thru 0x2762916 (0x5317 bytes) 2024-09-17T21:19:06.724 INFO:teuthology.orchestra.run.smithi184.stdout:1902 write 0x9c78e00 thru 0x9c7e277 (0x5478 bytes) 2024-09-17T21:19:06.725 INFO:teuthology.orchestra.run.smithi184.stdout:1905 read 0x2ba4e00 thru 0x2bae9ff (0x9c00 bytes) 2024-09-17T21:19:06.726 INFO:teuthology.orchestra.run.smithi184.stdout:1909 read 0x5ed1e00 thru 0x5ed7341 (0x5542 bytes) 2024-09-17T21:19:06.726 INFO:teuthology.orchestra.run.smithi184.stdout:1913 write 0x27e4000 thru 0x27f2fbf (0xefc0 bytes) 2024-09-17T21:19:06.727 INFO:teuthology.orchestra.run.smithi184.stdout:1914 read 0x57ea400 thru 0x57f9e1c (0xfa1d bytes) 2024-09-17T21:19:06.727 INFO:teuthology.orchestra.run.smithi184.stdout:1916 punch from 0x31e0800 to 0x31eda00, (0xd200 bytes) 2024-09-17T21:19:06.728 INFO:teuthology.orchestra.run.smithi184.stdout:1917 read 0x8a1ea00 thru 0x8a26dd6 (0x83d7 bytes) 2024-09-17T21:19:06.746 INFO:teuthology.orchestra.run.smithi184.stdout:1918 write 0xa5bb600 thru 0xa5bec09 (0x360a bytes) 2024-09-17T21:19:06.761 INFO:teuthology.orchestra.run.smithi184.stdout:1920 trunc from 0xa5bee00 to 0x2922c00 2024-09-17T21:19:06.783 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:06.779+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:06.785 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:06.786 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:06.803 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:06.803 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:06.816 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:06.816 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:06.816 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:19:06.817 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:06.820 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:06.820 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:06.832 INFO:teuthology.orchestra.run.smithi184.stdout:1922 write 0xeb92e00 thru 0xeba05cd (0xd7ce bytes) 2024-09-17T21:19:06.834 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:19:06.836 INFO:teuthology.orchestra.run.smithi184.stdout:1925 write 0x292fc00 thru 0x29398d7 (0x9cd8 bytes) 2024-09-17T21:19:06.846 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:06.846 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:06.859 INFO:teuthology.orchestra.run.smithi184.stdout:1927 trunc from 0xeba0600 to 0x1abe800 2024-09-17T21:19:06.867 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:06.867 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:06.884 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:06.883+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:06.888 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:06.930 INFO:teuthology.orchestra.run.smithi184.stdout:1929 read 0xe78a00 thru 0xe7d58a (0x4b8b bytes) 2024-09-17T21:19:06.960 INFO:teuthology.orchestra.run.smithi184.stdout:1930 write 0x2bd1e00 thru 0x2bd334b (0x154c bytes) 2024-09-17T21:19:06.962 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:06.962 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:06.962 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:06.962 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:06.962 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:06.976 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:19:07.003 INFO:teuthology.orchestra.run.smithi184.stdout:1933 write 0x821dc00 thru 0x82283ac (0xa7ad bytes) 2024-09-17T21:19:07.018 INFO:teuthology.orchestra.run.smithi184.stdout:1937 trunc from 0x8228400 to 0xc33d800 2024-09-17T21:19:07.021 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:07.019+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:07.046 INFO:teuthology.orchestra.run.smithi184.stdout:1939 read 0x3b0f200 thru 0x3b1d6a0 (0xe4a1 bytes) 2024-09-17T21:19:07.070 INFO:teuthology.orchestra.run.smithi184.stdout:1941 trunc from 0xc33d800 to 0x3480800 2024-09-17T21:19:07.086 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:07.084+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:07.100 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:07.100 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:07.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:07.100 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:07.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.100 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:389 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.100 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.194936+0000", 2024-09-17T21:19:07.101 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.901899139000001, 2024-09-17T21:19:07.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041176510999999999, 2024-09-17T21:19:07.101 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.101 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.101 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.101 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.101 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.101 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.101 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 389 2024-09-17T21:19:07.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.102 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.102 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.102 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194936+0000", 2024-09-17T21:19:07.102 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.102 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.102 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194936+0000", 2024-09-17T21:19:07.102 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194937+0000", 2024-09-17T21:19:07.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6270000000000001e-06 2024-09-17T21:19:07.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194944+0000", 2024-09-17T21:19:07.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5729999999999996e-06 2024-09-17T21:19:07.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194961+0000", 2024-09-17T21:19:07.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7623000000000001e-05 2024-09-17T21:19:07.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194974+0000", 2024-09-17T21:19:07.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2989999999999999e-05 2024-09-17T21:19:07.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.235848+0000", 2024-09-17T21:19:07.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.040874099999999997 2024-09-17T21:19:07.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.235971+0000", 2024-09-17T21:19:07.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012254400000000001 2024-09-17T21:19:07.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236112+0000", 2024-09-17T21:19:07.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014105399999999999 2024-09-17T21:19:07.105 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.105 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.106 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.106 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:406 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 2359296~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.106 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:19:07.106 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.901780332999998, 2024-09-17T21:19:07.106 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042448018999999997, 2024-09-17T21:19:07.106 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.106 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.106 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.106 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.106 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.107 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.107 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 406 2024-09-17T21:19:07.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.107 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.107 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:19:07.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.107 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:19:07.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195056+0000", 2024-09-17T21:19:07.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2729999999999999e-06 2024-09-17T21:19:07.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195060+0000", 2024-09-17T21:19:07.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5399999999999997e-06 2024-09-17T21:19:07.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195076+0000", 2024-09-17T21:19:07.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6217000000000001e-05 2024-09-17T21:19:07.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195093+0000", 2024-09-17T21:19:07.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.683e-05 2024-09-17T21:19:07.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.110 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.110 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236217+0000", 2024-09-17T21:19:07.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041124102000000003 2024-09-17T21:19:07.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236339+0000", 2024-09-17T21:19:07.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121876 2024-09-17T21:19:07.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237502+0000", 2024-09-17T21:19:07.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011631809999999999 2024-09-17T21:19:07.112 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.112 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.112 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.113 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:416 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2228224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.113 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:19:07.113 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.901282307000002, 2024-09-17T21:19:07.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042361990000000002, 2024-09-17T21:19:07.113 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.113 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.113 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.113 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.114 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.114 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.114 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 416 2024-09-17T21:19:07.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.114 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:19:07.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:19:07.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195554+0000", 2024-09-17T21:19:07.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9300000000000002e-06 2024-09-17T21:19:07.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195563+0000", 2024-09-17T21:19:07.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2770000000000005e-06 2024-09-17T21:19:07.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195586+0000", 2024-09-17T21:19:07.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3889000000000001e-05 2024-09-17T21:19:07.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195603+0000", 2024-09-17T21:19:07.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6084000000000001e-05 2024-09-17T21:19:07.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237563+0000", 2024-09-17T21:19:07.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041960856999999997 2024-09-17T21:19:07.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237695+0000", 2024-09-17T21:19:07.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013144400000000001 2024-09-17T21:19:07.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237914+0000", 2024-09-17T21:19:07.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000219509 2024-09-17T21:19:07.118 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.118 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.118 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.118 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:439 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.119 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:19:07.119 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.900718404999999, 2024-09-17T21:19:07.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043053811999999997, 2024-09-17T21:19:07.119 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.119 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.119 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.119 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.119 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.119 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.119 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 439 2024-09-17T21:19:07.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.120 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:19:07.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:19:07.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196118+0000", 2024-09-17T21:19:07.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3969999999999999e-06 2024-09-17T21:19:07.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196157+0000", 2024-09-17T21:19:07.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9826999999999997e-05 2024-09-17T21:19:07.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196179+0000", 2024-09-17T21:19:07.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1756e-05 2024-09-17T21:19:07.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196194+0000", 2024-09-17T21:19:07.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4707e-05 2024-09-17T21:19:07.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.123 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.123 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237980+0000", 2024-09-17T21:19:07.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041786219999999999 2024-09-17T21:19:07.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.123 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.123 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.238065+0000", 2024-09-17T21:19:07.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4560000000000004e-05 2024-09-17T21:19:07.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.123 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.124 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239170+0000", 2024-09-17T21:19:07.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001105345 2024-09-17T21:19:07.124 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.124 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.124 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.124 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:440 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.124 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:19:07.124 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.900615901999998, 2024-09-17T21:19:07.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043890039999999998, 2024-09-17T21:19:07.125 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.125 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.125 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.125 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.125 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.125 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.125 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 440 2024-09-17T21:19:07.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.125 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:19:07.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:19:07.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196220+0000", 2024-09-17T21:19:07.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.254e-06 2024-09-17T21:19:07.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196225+0000", 2024-09-17T21:19:07.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7829999999999998e-06 2024-09-17T21:19:07.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196241+0000", 2024-09-17T21:19:07.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6322999999999999e-05 2024-09-17T21:19:07.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196258+0000", 2024-09-17T21:19:07.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7056000000000001e-05 2024-09-17T21:19:07.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239229+0000", 2024-09-17T21:19:07.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042971058999999999 2024-09-17T21:19:07.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.129 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239403+0000", 2024-09-17T21:19:07.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017399200000000001 2024-09-17T21:19:07.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.129 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.129 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240109+0000", 2024-09-17T21:19:07.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00070557300000000003 2024-09-17T21:19:07.129 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.129 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.129 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.130 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:450 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.130 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:19:07.130 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.900254951000001, 2024-09-17T21:19:07.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043994521000000002, 2024-09-17T21:19:07.130 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.130 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.130 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.130 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.130 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.130 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.130 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 450 2024-09-17T21:19:07.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.131 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:19:07.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:19:07.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196581+0000", 2024-09-17T21:19:07.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4500000000000001e-06 2024-09-17T21:19:07.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196588+0000", 2024-09-17T21:19:07.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7530000000000004e-06 2024-09-17T21:19:07.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.133 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196607+0000", 2024-09-17T21:19:07.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8987e-05 2024-09-17T21:19:07.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.133 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.133 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196620+0000", 2024-09-17T21:19:07.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3589e-05 2024-09-17T21:19:07.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.133 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.133 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240225+0000", 2024-09-17T21:19:07.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043604878999999999 2024-09-17T21:19:07.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240377+0000", 2024-09-17T21:19:07.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015146699999999999 2024-09-17T21:19:07.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240574+0000", 2024-09-17T21:19:07.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019739599999999999 2024-09-17T21:19:07.134 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.134 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.135 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.135 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:464 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.135 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:19:07.135 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.900004529999997, 2024-09-17T21:19:07.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044651692999999999, 2024-09-17T21:19:07.135 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.135 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.135 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.135 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.135 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.135 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.136 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 464 2024-09-17T21:19:07.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.136 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:19:07.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:19:07.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.137 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.137 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196838+0000", 2024-09-17T21:19:07.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4970000000000004e-06 2024-09-17T21:19:07.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.137 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.137 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196859+0000", 2024-09-17T21:19:07.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1166e-05 2024-09-17T21:19:07.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.137 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.138 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196879+0000", 2024-09-17T21:19:07.138 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.995e-05 2024-09-17T21:19:07.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.138 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.138 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.138 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196896+0000", 2024-09-17T21:19:07.138 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6929999999999999e-05 2024-09-17T21:19:07.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.138 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.138 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.138 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240617+0000", 2024-09-17T21:19:07.138 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043721784999999999 2024-09-17T21:19:07.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240702+0000", 2024-09-17T21:19:07.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4060999999999994e-05 2024-09-17T21:19:07.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241482+0000", 2024-09-17T21:19:07.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000780304 2024-09-17T21:19:07.139 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.139 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.139 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.140 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:475 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.140 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:19:07.140 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.899735397999997, 2024-09-17T21:19:07.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044661981000000003, 2024-09-17T21:19:07.140 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.140 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.140 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.140 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.140 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.141 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.141 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 475 2024-09-17T21:19:07.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.141 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:19:07.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:19:07.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197100+0000", 2024-09-17T21:19:07.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.206e-06 2024-09-17T21:19:07.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197105+0000", 2024-09-17T21:19:07.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.527e-06 2024-09-17T21:19:07.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.143 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.143 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197124+0000", 2024-09-17T21:19:07.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:19:07.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.143 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.143 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197136+0000", 2024-09-17T21:19:07.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2857000000000001e-05 2024-09-17T21:19:07.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241539+0000", 2024-09-17T21:19:07.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044402077999999998 2024-09-17T21:19:07.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241653+0000", 2024-09-17T21:19:07.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114931 2024-09-17T21:19:07.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241761+0000", 2024-09-17T21:19:07.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107753 2024-09-17T21:19:07.145 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.145 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.145 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.145 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:476 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.145 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:19:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.899681573000002, 2024-09-17T21:19:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045038976000000001, 2024-09-17T21:19:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 476 2024-09-17T21:19:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:19:07.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.147 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:19:07.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.147 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197154+0000", 2024-09-17T21:19:07.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.127e-06 2024-09-17T21:19:07.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197159+0000", 2024-09-17T21:19:07.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4569999999999998e-06 2024-09-17T21:19:07.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197174+0000", 2024-09-17T21:19:07.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5173e-05 2024-09-17T21:19:07.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197186+0000", 2024-09-17T21:19:07.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:19:07.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241863+0000", 2024-09-17T21:19:07.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044676826000000003 2024-09-17T21:19:07.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241998+0000", 2024-09-17T21:19:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013476100000000001 2024-09-17T21:19:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242192+0000", 2024-09-17T21:19:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019436900000000001 2024-09-17T21:19:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:494 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.899506389999999, 2024-09-17T21:19:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046244995999999997, 2024-09-17T21:19:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 494 2024-09-17T21:19:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:07.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:07.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197329+0000", 2024-09-17T21:19:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.173e-06 2024-09-17T21:19:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197334+0000", 2024-09-17T21:19:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0499999999999999e-06 2024-09-17T21:19:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197352+0000", 2024-09-17T21:19:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7133e-05 2024-09-17T21:19:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197367+0000", 2024-09-17T21:19:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5123000000000001e-05 2024-09-17T21:19:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:19:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044989829000000002 2024-09-17T21:19:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242526+0000", 2024-09-17T21:19:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016968199999999999 2024-09-17T21:19:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243573+0000", 2024-09-17T21:19:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010470060000000001 2024-09-17T21:19:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:502 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:19:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.897272731000001, 2024-09-17T21:19:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044346086999999999, 2024-09-17T21:19:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 502 2024-09-17T21:19:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:19:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:19:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199563+0000", 2024-09-17T21:19:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2929999999999999e-06 2024-09-17T21:19:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199569+0000", 2024-09-17T21:19:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5330000000000002e-06 2024-09-17T21:19:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199614+0000", 2024-09-17T21:19:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4793000000000003e-05 2024-09-17T21:19:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199628+0000", 2024-09-17T21:19:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4037e-05 2024-09-17T21:19:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243665+0000", 2024-09-17T21:19:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044037106999999999 2024-09-17T21:19:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243767+0000", 2024-09-17T21:19:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010279499999999999 2024-09-17T21:19:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243908+0000", 2024-09-17T21:19:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140529 2024-09-17T21:19:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:523 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:19:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.896826578000002, 2024-09-17T21:19:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044789645000000003, 2024-09-17T21:19:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 523 2024-09-17T21:19:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:19:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:19:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:19:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9e-06 2024-09-17T21:19:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200021+0000", 2024-09-17T21:19:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9499999999999998e-06 2024-09-17T21:19:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200038+0000", 2024-09-17T21:19:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.73e-05 2024-09-17T21:19:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200051+0000", 2024-09-17T21:19:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.322e-05 2024-09-17T21:19:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243987+0000", 2024-09-17T21:19:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043935386999999999 2024-09-17T21:19:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244136+0000", 2024-09-17T21:19:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014868300000000001 2024-09-17T21:19:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244798+0000", 2024-09-17T21:19:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00066220500000000002 2024-09-17T21:19:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:524 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.896760362000002, 2024-09-17T21:19:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045737414999999997, 2024-09-17T21:19:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 524 2024-09-17T21:19:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200076+0000", 2024-09-17T21:19:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:19:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200081+0000", 2024-09-17T21:19:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7030000000000003e-06 2024-09-17T21:19:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200099+0000", 2024-09-17T21:19:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7836999999999999e-05 2024-09-17T21:19:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200113+0000", 2024-09-17T21:19:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3570000000000001e-05 2024-09-17T21:19:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244866+0000", 2024-09-17T21:19:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044753573999999997 2024-09-17T21:19:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244972+0000", 2024-09-17T21:19:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106111 2024-09-17T21:19:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245812+0000", 2024-09-17T21:19:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083923999999999999 2024-09-17T21:19:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:525 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3670016~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.896608980000003, 2024-09-17T21:19:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046645697, 2024-09-17T21:19:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 525 2024-09-17T21:19:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:19:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:19:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200234+0000", 2024-09-17T21:19:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:19:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200254+0000", 2024-09-17T21:19:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9959999999999999e-05 2024-09-17T21:19:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200293+0000", 2024-09-17T21:19:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9755999999999997e-05 2024-09-17T21:19:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245871+0000", 2024-09-17T21:19:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045577871999999998 2024-09-17T21:19:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246030+0000", 2024-09-17T21:19:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158975 2024-09-17T21:19:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246871+0000", 2024-09-17T21:19:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084111699999999995 2024-09-17T21:19:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.896363305000001, 2024-09-17T21:19:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:19:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:19:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:19:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:19:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:19:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:19:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:19:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:19:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:19:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:19:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:19:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:19:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:19:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:19:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:19:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:19:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.896184623000003, 2024-09-17T21:19:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:19:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:19:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:19:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:19:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:19:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:19:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:19:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:19:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:19:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:19:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:19:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:19:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:19:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:19:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:19:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:19:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.896125349999998, 2024-09-17T21:19:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:19:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:19:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:19:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:19:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:19:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:19:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:19:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:19:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:19:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:19:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:19:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:19:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:19:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:19:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:19:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:19:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.895983155000003, 2024-09-17T21:19:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:19:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:19:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:19:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:19:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:19:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:19:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:19:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:19:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:19:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:19:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:19:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:19:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:19:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:19:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:19:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:19:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.895927231999998, 2024-09-17T21:19:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:19:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:19:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:19:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:19:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:19:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:19:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:19:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:19:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:19:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:19:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:19:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:19:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:19:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:19:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:19:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:19:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.895873614999999, 2024-09-17T21:19:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:19:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:19:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:19:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:19:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:19:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:19:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:19:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:19:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:19:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:19:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:19:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:19:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:19:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:19:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:19:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:19:07.208 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.208 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.208 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.208 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.208 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.208 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:07.210 INFO:teuthology.orchestra.run.smithi184.stdout:1942 read 0x1958600 thru 0x195f149 (0x6b4a bytes) 2024-09-17T21:19:07.210 INFO:teuthology.orchestra.run.smithi184.stdout:1944 read 0xdb6e00 thru 0xdbbd6f (0x4f70 bytes) 2024-09-17T21:19:07.210 INFO:teuthology.orchestra.run.smithi184.stdout:1945 write 0x8b23c00 thru 0x8b2b2d6 (0x76d7 bytes) 2024-09-17T21:19:07.210 INFO:teuthology.orchestra.run.smithi184.stdout:1947 trunc from 0x8b2b400 to 0x3d62400 2024-09-17T21:19:07.211 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:19:07.211 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:07.204+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:07.222 INFO:teuthology.orchestra.run.smithi184.stdout:1948 read 0x243a600 thru 0x2443d93 (0x9794 bytes) 2024-09-17T21:19:07.253 INFO:teuthology.orchestra.run.smithi184.stdout:1950 write 0x52c7400 thru 0x52d0700 (0x9301 bytes) 2024-09-17T21:19:07.260 INFO:teuthology.orchestra.run.smithi184.stdout:1951 read 0xb2a600 thru 0xb32f9f (0x89a0 bytes) 2024-09-17T21:19:07.262 INFO:teuthology.orchestra.run.smithi184.stdout:1952 read 0x2721e00 thru 0x272272c (0x92d bytes) 2024-09-17T21:19:07.263 INFO:teuthology.orchestra.run.smithi184.stdout:1954 punch from 0x74e400 to 0x750800, (0x2400 bytes) 2024-09-17T21:19:07.296 INFO:teuthology.orchestra.run.smithi184.stdout:1957 write 0x8213000 thru 0x8221f4b (0xef4c bytes) 2024-09-17T21:19:07.298 INFO:teuthology.orchestra.run.smithi184.stdout:1959 read 0x5c6be00 thru 0x5c758af (0x9ab0 bytes) 2024-09-17T21:19:07.310 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:07.308+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:07.313 INFO:teuthology.orchestra.run.smithi184.stdout:1960 trunc from 0x8222000 to 0x2d46c00 2024-09-17T21:19:07.352 INFO:teuthology.orchestra.run.smithi184.stdout:1961 punch from 0xfbaa00 to 0xfc5800, (0xae00 bytes) 2024-09-17T21:19:07.371 INFO:teuthology.orchestra.run.smithi184.stdout:1962 trunc from 0x2d46c00 to 0x9d7f800 2024-09-17T21:19:07.382 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:07.382 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:07.382 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:19:07.382 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:07.398 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:19:07.425 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-17T21:19:07.425 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:07.423+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:07.431 INFO:teuthology.orchestra.run.smithi184.stdout:1963 trunc from 0x9d7f800 to 0x85d400 2024-09-17T21:19:07.511 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:07.507+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:07.524 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:07.524 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:07.524 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:07.524 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:07.524 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:07.538 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:19:07.576 INFO:teuthology.orchestra.run.smithi184.stdout:1964 write 0xc53a200 thru 0xc5487f1 (0xe5f2 bytes) 2024-09-17T21:19:07.612 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:07.608+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:07.617 INFO:teuthology.orchestra.run.smithi184.stdout:1965 write 0xe963400 thru 0xe9724b5 (0xf0b6 bytes) 2024-09-17T21:19:07.622 INFO:teuthology.orchestra.run.smithi184.stdout:1967 punch from 0x8847000 to 0x8853c00, (0xcc00 bytes) 2024-09-17T21:19:07.640 INFO:teuthology.orchestra.run.smithi184.stdout:1968 trunc from 0xe972600 to 0x59a8400 2024-09-17T21:19:07.682 INFO:teuthology.orchestra.run.smithi184.stdout:1969 trunc from 0x59a8400 to 0x3ecc600 2024-09-17T21:19:07.701 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:07.702 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:07.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:07.702 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:07.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.702 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:491 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.702 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:19:07.702 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.500057906999999, 2024-09-17T21:19:07.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041839013000000001, 2024-09-17T21:19:07.702 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.702 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.702 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.702 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.702 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.703 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.703 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 491 2024-09-17T21:19:07.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.703 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:19:07.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:19:07.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198315+0000", 2024-09-17T21:19:07.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6599999999999996e-07 2024-09-17T21:19:07.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198317+0000", 2024-09-17T21:19:07.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6639999999999998e-06 2024-09-17T21:19:07.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198327+0000", 2024-09-17T21:19:07.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5629999999999998e-06 2024-09-17T21:19:07.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198337+0000", 2024-09-17T21:19:07.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0667e-05 2024-09-17T21:19:07.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239793+0000", 2024-09-17T21:19:07.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041455519000000003 2024-09-17T21:19:07.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239929+0000", 2024-09-17T21:19:07.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135716 2024-09-17T21:19:07.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240153+0000", 2024-09-17T21:19:07.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022421799999999999 2024-09-17T21:19:07.706 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.706 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.706 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.706 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:492 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.706 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:19:07.706 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.500019537, 2024-09-17T21:19:07.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042947165000000002, 2024-09-17T21:19:07.706 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.706 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.707 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.707 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.707 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.707 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.707 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 492 2024-09-17T21:19:07.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.707 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:19:07.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:19:07.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198354+0000", 2024-09-17T21:19:07.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:19:07.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198357+0000", 2024-09-17T21:19:07.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6969999999999999e-06 2024-09-17T21:19:07.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198367+0000", 2024-09-17T21:19:07.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8430000000000005e-06 2024-09-17T21:19:07.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198374+0000", 2024-09-17T21:19:07.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9600000000000003e-06 2024-09-17T21:19:07.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240226+0000", 2024-09-17T21:19:07.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041852157000000001 2024-09-17T21:19:07.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240409+0000", 2024-09-17T21:19:07.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018244800000000001 2024-09-17T21:19:07.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241299+0000", 2024-09-17T21:19:07.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000890554 2024-09-17T21:19:07.710 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.710 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.710 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.710 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:493 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 4063232~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.710 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:19:07.710 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.499982451000001, 2024-09-17T21:19:07.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043897209, 2024-09-17T21:19:07.711 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.711 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.711 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.711 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.711 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.711 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.711 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 493 2024-09-17T21:19:07.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.711 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:19:07.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:19:07.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198391+0000", 2024-09-17T21:19:07.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4699999999999999e-06 2024-09-17T21:19:07.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198396+0000", 2024-09-17T21:19:07.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:19:07.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198412+0000", 2024-09-17T21:19:07.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6507e-05 2024-09-17T21:19:07.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198425+0000", 2024-09-17T21:19:07.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.256e-05 2024-09-17T21:19:07.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241383+0000", 2024-09-17T21:19:07.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042958216 2024-09-17T21:19:07.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241525+0000", 2024-09-17T21:19:07.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014204500000000001 2024-09-17T21:19:07.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242287+0000", 2024-09-17T21:19:07.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076125800000000001 2024-09-17T21:19:07.715 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.715 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.715 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.715 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:496 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.715 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:19:07.715 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.499892052, 2024-09-17T21:19:07.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043732500000000001, 2024-09-17T21:19:07.715 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.715 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.715 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.715 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.716 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.716 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.716 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 496 2024-09-17T21:19:07.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.716 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:19:07.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:19:07.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198481+0000", 2024-09-17T21:19:07.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-17T21:19:07.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198488+0000", 2024-09-17T21:19:07.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8499999999999996e-06 2024-09-17T21:19:07.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198507+0000", 2024-09-17T21:19:07.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9276999999999999e-05 2024-09-17T21:19:07.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198521+0000", 2024-09-17T21:19:07.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3376e-05 2024-09-17T21:19:07.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241339+0000", 2024-09-17T21:19:07.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042818119000000002 2024-09-17T21:19:07.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241449+0000", 2024-09-17T21:19:07.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110209 2024-09-17T21:19:07.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242212+0000", 2024-09-17T21:19:07.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076329900000000005 2024-09-17T21:19:07.719 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.719 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.719 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.720 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:497 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.720 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:19:07.720 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.499811102999999, 2024-09-17T21:19:07.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044584278999999997, 2024-09-17T21:19:07.720 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.720 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.720 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.720 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.720 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.720 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.720 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 497 2024-09-17T21:19:07.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.721 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:19:07.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:19:07.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198562+0000", 2024-09-17T21:19:07.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-17T21:19:07.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198569+0000", 2024-09-17T21:19:07.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.494e-06 2024-09-17T21:19:07.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198588+0000", 2024-09-17T21:19:07.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8955999999999999e-05 2024-09-17T21:19:07.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198621+0000", 2024-09-17T21:19:07.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3837000000000001e-05 2024-09-17T21:19:07.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242258+0000", 2024-09-17T21:19:07.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043636900999999999 2024-09-17T21:19:07.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242343+0000", 2024-09-17T21:19:07.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5048000000000003e-05 2024-09-17T21:19:07.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243145+0000", 2024-09-17T21:19:07.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00080167300000000003 2024-09-17T21:19:07.724 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.724 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.724 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.724 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:503 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.725 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:19:07.725 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.499726578999997, 2024-09-17T21:19:07.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044220582000000001, 2024-09-17T21:19:07.725 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.725 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.725 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.725 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.725 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.725 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.725 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 503 2024-09-17T21:19:07.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.725 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:19:07.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:19:07.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198647+0000", 2024-09-17T21:19:07.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.483e-06 2024-09-17T21:19:07.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198653+0000", 2024-09-17T21:19:07.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4529999999999999e-06 2024-09-17T21:19:07.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198672+0000", 2024-09-17T21:19:07.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9193000000000001e-05 2024-09-17T21:19:07.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198685+0000", 2024-09-17T21:19:07.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.291e-05 2024-09-17T21:19:07.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:19:07.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043672141999999997 2024-09-17T21:19:07.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242716+0000", 2024-09-17T21:19:07.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00035901800000000001 2024-09-17T21:19:07.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242866+0000", 2024-09-17T21:19:07.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000149383 2024-09-17T21:19:07.728 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.729 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.729 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.729 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:504 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.729 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:19:07.729 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.499666787000002, 2024-09-17T21:19:07.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044611406999999999, 2024-09-17T21:19:07.729 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.729 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.729 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.729 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.729 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.729 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.730 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 504 2024-09-17T21:19:07.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.730 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:19:07.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:19:07.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198706+0000", 2024-09-17T21:19:07.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:19:07.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198712+0000", 2024-09-17T21:19:07.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5770000000000001e-06 2024-09-17T21:19:07.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198729+0000", 2024-09-17T21:19:07.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6673000000000001e-05 2024-09-17T21:19:07.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198742+0000", 2024-09-17T21:19:07.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3543000000000001e-05 2024-09-17T21:19:07.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242981+0000", 2024-09-17T21:19:07.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044238953999999997 2024-09-17T21:19:07.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243155+0000", 2024-09-17T21:19:07.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017401600000000001 2024-09-17T21:19:07.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243316+0000", 2024-09-17T21:19:07.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016137399999999999 2024-09-17T21:19:07.733 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.733 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.733 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.734 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:521 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.734 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:19:07.734 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.499119647999997, 2024-09-17T21:19:07.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045043685999999999, 2024-09-17T21:19:07.734 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.734 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.734 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.734 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.734 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.734 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.734 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 521 2024-09-17T21:19:07.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.735 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:19:07.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:19:07.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199253+0000", 2024-09-17T21:19:07.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1400000000000001e-06 2024-09-17T21:19:07.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199258+0000", 2024-09-17T21:19:07.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1800000000000004e-06 2024-09-17T21:19:07.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199276+0000", 2024-09-17T21:19:07.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7686999999999999e-05 2024-09-17T21:19:07.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199289+0000", 2024-09-17T21:19:07.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3263e-05 2024-09-17T21:19:07.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243397+0000", 2024-09-17T21:19:07.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044107636999999998 2024-09-17T21:19:07.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243527+0000", 2024-09-17T21:19:07.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013038399999999999 2024-09-17T21:19:07.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244296+0000", 2024-09-17T21:19:07.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076839499999999995 2024-09-17T21:19:07.738 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.738 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.738 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.738 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:522 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.738 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:19:07.738 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.499064345000001, 2024-09-17T21:19:07.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046049870999999999, 2024-09-17T21:19:07.738 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.738 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.738 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.738 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.738 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.739 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.739 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 522 2024-09-17T21:19:07.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.739 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:19:07.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:19:07.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199309+0000", 2024-09-17T21:19:07.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:19:07.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199315+0000", 2024-09-17T21:19:07.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2440000000000001e-06 2024-09-17T21:19:07.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199333+0000", 2024-09-17T21:19:07.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7886000000000001e-05 2024-09-17T21:19:07.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199379+0000", 2024-09-17T21:19:07.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6383e-05 2024-09-17T21:19:07.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244394+0000", 2024-09-17T21:19:07.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045014851000000002 2024-09-17T21:19:07.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244491+0000", 2024-09-17T21:19:07.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7245999999999997e-05 2024-09-17T21:19:07.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.742 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245357+0000", 2024-09-17T21:19:07.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00086626800000000001 2024-09-17T21:19:07.742 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.742 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.742 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.742 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:527 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.742 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:19:07.742 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.498967679000003, 2024-09-17T21:19:07.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045812522000000001, 2024-09-17T21:19:07.742 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.743 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.743 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.743 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.743 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.743 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.743 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 527 2024-09-17T21:19:07.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.743 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:19:07.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:19:07.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199406+0000", 2024-09-17T21:19:07.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.367e-06 2024-09-17T21:19:07.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199412+0000", 2024-09-17T21:19:07.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:19:07.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199429+0000", 2024-09-17T21:19:07.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7396000000000001e-05 2024-09-17T21:19:07.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199442+0000", 2024-09-17T21:19:07.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3327e-05 2024-09-17T21:19:07.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244361+0000", 2024-09-17T21:19:07.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044918197 2024-09-17T21:19:07.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244482+0000", 2024-09-17T21:19:07.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121945 2024-09-17T21:19:07.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245217+0000", 2024-09-17T21:19:07.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00073415999999999998 2024-09-17T21:19:07.746 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.746 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.746 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.746 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:534 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.746 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:19:07.746 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.498908972999999, 2024-09-17T21:19:07.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046292505999999997, 2024-09-17T21:19:07.747 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.747 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.747 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.747 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.747 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.747 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.747 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:19:07.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.747 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.747 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.747 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:19:07.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:19:07.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199464+0000", 2024-09-17T21:19:07.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:19:07.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199473+0000", 2024-09-17T21:19:07.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5629999999999992e-06 2024-09-17T21:19:07.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199492+0000", 2024-09-17T21:19:07.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9273999999999999e-05 2024-09-17T21:19:07.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199505+0000", 2024-09-17T21:19:07.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3443e-05 2024-09-17T21:19:07.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245357+0000", 2024-09-17T21:19:07.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045851714000000002 2024-09-17T21:19:07.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245550+0000", 2024-09-17T21:19:07.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019301600000000001 2024-09-17T21:19:07.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245755+0000", 2024-09-17T21:19:07.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205226 2024-09-17T21:19:07.750 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.750 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.751 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:535 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.751 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:19:07.751 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.498804417000002, 2024-09-17T21:19:07.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046637492000000003, 2024-09-17T21:19:07.751 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.751 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.751 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.751 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.751 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.752 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.752 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 535 2024-09-17T21:19:07.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.752 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.752 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.752 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:19:07.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.752 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.752 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:19:07.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199568+0000", 2024-09-17T21:19:07.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.037e-06 2024-09-17T21:19:07.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199572+0000", 2024-09-17T21:19:07.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5729999999999999e-06 2024-09-17T21:19:07.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199585+0000", 2024-09-17T21:19:07.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.279e-05 2024-09-17T21:19:07.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199592+0000", 2024-09-17T21:19:07.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.61e-06 2024-09-17T21:19:07.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245873+0000", 2024-09-17T21:19:07.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046280494999999998 2024-09-17T21:19:07.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246018+0000", 2024-09-17T21:19:07.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000144697 2024-09-17T21:19:07.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246205+0000", 2024-09-17T21:19:07.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018729 2024-09-17T21:19:07.755 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.755 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.755 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.755 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:543 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.755 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:07.755 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.498709171000002, 2024-09-17T21:19:07.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047564097999999999, 2024-09-17T21:19:07.756 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.756 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.756 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.756 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.756 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.756 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.756 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 543 2024-09-17T21:19:07.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.756 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:07.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:07.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:07.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9699999999999995e-07 2024-09-17T21:19:07.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199677+0000", 2024-09-17T21:19:07.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:19:07.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199690+0000", 2024-09-17T21:19:07.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2977e-05 2024-09-17T21:19:07.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199705+0000", 2024-09-17T21:19:07.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4656e-05 2024-09-17T21:19:07.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246307+0000", 2024-09-17T21:19:07.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046601780000000002 2024-09-17T21:19:07.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246475+0000", 2024-09-17T21:19:07.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016834200000000001 2024-09-17T21:19:07.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247227+0000", 2024-09-17T21:19:07.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00075173299999999998 2024-09-17T21:19:07.759 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.759 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.759 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.760 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:544 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.760 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:07.760 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.498650761999997, 2024-09-17T21:19:07.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049283559999999997, 2024-09-17T21:19:07.760 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.760 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.760 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.760 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.760 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.760 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.760 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 544 2024-09-17T21:19:07.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.760 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:07.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:07.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199722+0000", 2024-09-17T21:19:07.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.08e-06 2024-09-17T21:19:07.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199738+0000", 2024-09-17T21:19:07.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5943999999999998e-05 2024-09-17T21:19:07.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199748+0000", 2024-09-17T21:19:07.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0193e-05 2024-09-17T21:19:07.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199756+0000", 2024-09-17T21:19:07.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4200000000000001e-06 2024-09-17T21:19:07.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247311+0000", 2024-09-17T21:19:07.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047555300000000002 2024-09-17T21:19:07.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247571+0000", 2024-09-17T21:19:07.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025996999999999999 2024-09-17T21:19:07.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249005+0000", 2024-09-17T21:19:07.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001433653 2024-09-17T21:19:07.764 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.764 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.764 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.764 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:549 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.764 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:07.764 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.498605138000002, 2024-09-17T21:19:07.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048464528999999999, 2024-09-17T21:19:07.764 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.764 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.764 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.764 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.765 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.765 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.765 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 549 2024-09-17T21:19:07.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.765 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:07.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:07.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:07.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:19:07.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199770+0000", 2024-09-17T21:19:07.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9969999999999999e-06 2024-09-17T21:19:07.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199780+0000", 2024-09-17T21:19:07.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0003000000000001e-05 2024-09-17T21:19:07.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199793+0000", 2024-09-17T21:19:07.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2074e-05 2024-09-17T21:19:07.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247289+0000", 2024-09-17T21:19:07.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047496691000000001 2024-09-17T21:19:07.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247437+0000", 2024-09-17T21:19:07.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014807899999999999 2024-09-17T21:19:07.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248231+0000", 2024-09-17T21:19:07.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00079390899999999998 2024-09-17T21:19:07.768 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.768 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.768 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.768 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:558 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.768 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:07.769 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.498564625, 2024-09-17T21:19:07.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049709821000000001, 2024-09-17T21:19:07.769 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.769 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.769 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.769 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.769 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.769 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.769 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 558 2024-09-17T21:19:07.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.769 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:07.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:07.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199809+0000", 2024-09-17T21:19:07.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.333e-06 2024-09-17T21:19:07.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199814+0000", 2024-09-17T21:19:07.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0930000000000002e-06 2024-09-17T21:19:07.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199831+0000", 2024-09-17T21:19:07.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7580000000000001e-05 2024-09-17T21:19:07.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199860+0000", 2024-09-17T21:19:07.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8609999999999999e-05 2024-09-17T21:19:07.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249106+0000", 2024-09-17T21:19:07.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049246545000000003 2024-09-17T21:19:07.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249274+0000", 2024-09-17T21:19:07.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016806699999999999 2024-09-17T21:19:07.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249517+0000", 2024-09-17T21:19:07.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024259300000000001 2024-09-17T21:19:07.773 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.773 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.773 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.773 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:559 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.773 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:07.773 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.498445263000001, 2024-09-17T21:19:07.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049990694000000002, 2024-09-17T21:19:07.773 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.773 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.774 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.774 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.774 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.774 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.774 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 559 2024-09-17T21:19:07.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.774 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:07.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:07.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199928+0000", 2024-09-17T21:19:07.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:19:07.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199935+0000", 2024-09-17T21:19:07.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5429999999999999e-06 2024-09-17T21:19:07.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199954+0000", 2024-09-17T21:19:07.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.912e-05 2024-09-17T21:19:07.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199969+0000", 2024-09-17T21:19:07.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5305999999999999e-05 2024-09-17T21:19:07.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249604+0000", 2024-09-17T21:19:07.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049634931 2024-09-17T21:19:07.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249738+0000", 2024-09-17T21:19:07.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013411599999999999 2024-09-17T21:19:07.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249917+0000", 2024-09-17T21:19:07.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179111 2024-09-17T21:19:07.777 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.777 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.777 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.777 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:560 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:07.778 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:07.778 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.498380847, 2024-09-17T21:19:07.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050173059999999998, 2024-09-17T21:19:07.778 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.778 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.778 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.778 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.778 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:07.778 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:07.778 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 560 2024-09-17T21:19:07.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.778 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:07.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:07.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199992+0000", 2024-09-17T21:19:07.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.517e-06 2024-09-17T21:19:07.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199998+0000", 2024-09-17T21:19:07.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1460000000000004e-06 2024-09-17T21:19:07.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:19:07.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7407000000000001e-05 2024-09-17T21:19:07.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200029+0000", 2024-09-17T21:19:07.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3499999999999999e-05 2024-09-17T21:19:07.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249968+0000", 2024-09-17T21:19:07.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049939799 2024-09-17T21:19:07.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250055+0000", 2024-09-17T21:19:07.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6731999999999996e-05 2024-09-17T21:19:07.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250164+0000", 2024-09-17T21:19:07.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108959 2024-09-17T21:19:07.781 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.781 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.781 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.782 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:19:07.782 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:07.782 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.888591137999999, 2024-09-17T21:19:07.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:19:07.782 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.782 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.782 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:07.782 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.782 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:19:07.782 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:19:07.783 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:19:07.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.783 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:07.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:07.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:19:07.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:19:07.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:19:07.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:19:07.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:19:07.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:19:07.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:19:07.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:19:07.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:19:07.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:19:07.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:07.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:19:07.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:19:07.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:19:07.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:19:07.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:19:07.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:19:07.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:19:07.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:19:07.787 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.787 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.787 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.787 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:19:07.787 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:07.787 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.333489352000001, 2024-09-17T21:19:07.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:19:07.787 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:07.787 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:07.787 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:07.787 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:07.787 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:19:07.787 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:07.788 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:19:07.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.788 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:07.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:07.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:07.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:07.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:07.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:07.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:07.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:19:07.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:19:07.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:07.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:19:07.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:19:07.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:07.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:19:07.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:19:07.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:07.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:19:07.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:19:07.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:19:07.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:19:07.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:07.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:19:07.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:19:07.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:07.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:19:07.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:19:07.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:07.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:19:07.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:19:07.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:07.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:19:07.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:19:07.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:07.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:19:07.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:19:07.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:07.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:19:07.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:19:07.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:07.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:19:07.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:19:07.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:07.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:07.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:07.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:19:07.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:19:07.793 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.793 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.793 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.793 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:07.793 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:07.793 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:07.794 INFO:teuthology.orchestra.run.smithi184.stdout:1971 write 0x19a4000 thru 0x19b0472 (0xc473 bytes) 2024-09-17T21:19:07.794 INFO:teuthology.orchestra.run.smithi184.stdout:1973 trunc from 0x3ecc600 to 0x51ce00 2024-09-17T21:19:07.795 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:19:07.796 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:07.792+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:07.797 INFO:teuthology.orchestra.run.smithi184.stdout:1974 punch from 0x85800 to 0x8ba00, (0x6200 bytes) 2024-09-17T21:19:07.829 INFO:teuthology.orchestra.run.smithi184.stdout:1980 write 0x2182a00 thru 0x2185458 (0x2a59 bytes) 2024-09-17T21:19:07.843 INFO:teuthology.orchestra.run.smithi184.stdout:1981 trunc from 0x2185600 to 0x5e94c00 2024-09-17T21:19:07.875 INFO:teuthology.orchestra.run.smithi184.stdout:1982 write 0x5c2a00 thru 0x5ce79f (0xbda0 bytes) 2024-09-17T21:19:07.879 INFO:teuthology.orchestra.run.smithi184.stdout:1984 trunc from 0x5e94c00 to 0x5948c00 2024-09-17T21:19:07.896 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:07.891+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:07.905 INFO:teuthology.orchestra.run.smithi184.stdout:1985 punch from 0x3272600 to 0x3273200, (0xc00 bytes) 2024-09-17T21:19:07.906 INFO:teuthology.orchestra.run.smithi184.stdout:1986 read 0x3e7000 thru 0x3ef690 (0x8691 bytes) 2024-09-17T21:19:07.908 INFO:teuthology.orchestra.run.smithi184.stdout:1988 read 0x3320a00 thru 0x332e456 (0xda57 bytes) 2024-09-17T21:19:07.923 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:07.923 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:07.923 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:19:07.923 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:07.937 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:19:07.939 INFO:teuthology.orchestra.run.smithi184.stdout:1997 write 0x9533000 thru 0x953bcc3 (0x8cc4 bytes) 2024-09-17T21:19:07.941 INFO:teuthology.orchestra.run.smithi184.stdout:1998 write 0x792c600 thru 0x7933e05 (0x7806 bytes) 2024-09-17T21:19:07.943 INFO:teuthology.orchestra.run.smithi184.stdout:1999 write 0x52f200 thru 0x5351a9 (0x5faa bytes) 2024-09-17T21:19:07.945 INFO:teuthology.orchestra.run.smithi184.stdout:2003 punch from 0x62f9000 to 0x6301c00, (0x8c00 bytes) 2024-09-17T21:19:07.946 INFO:teuthology.orchestra.run.smithi184.stdout:2005 write 0x19f9e00 thru 0x1a026c2 (0x88c3 bytes) 2024-09-17T21:19:07.947 INFO:teuthology.orchestra.run.smithi184.stdout:2006 write 0x2698000 thru 0x26a599c (0xd99d bytes) 2024-09-17T21:19:07.979 INFO:teuthology.orchestra.run.smithi184.stdout:2012 write 0xdcab000 thru 0xdcb7374 (0xc375 bytes) 2024-09-17T21:19:07.982 INFO:teuthology.orchestra.run.smithi184.stdout:2013 read 0x3961c00 thru 0x3968168 (0x6569 bytes) 2024-09-17T21:19:07.983 INFO:teuthology.orchestra.run.smithi184.stdout:2015 read 0x36cd000 thru 0x36d1038 (0x4039 bytes) 2024-09-17T21:19:07.983 INFO:teuthology.orchestra.run.smithi184.stdout:2019 read 0x5969a00 thru 0x5974c35 (0xb236 bytes) 2024-09-17T21:19:08.014 INFO:teuthology.orchestra.run.smithi184.stdout:2021 write 0xe569800 thru 0xe5718bb (0x80bc bytes) 2024-09-17T21:19:08.020 INFO:teuthology.orchestra.run.smithi184.stdout:2022 trunc from 0xe571a00 to 0xd9d8400 2024-09-17T21:19:08.028 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:08.027+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:08.049 INFO:teuthology.orchestra.run.smithi184.stdout:2024 read 0xa32f800 thru 0xa33f70b (0xff0c bytes) 2024-09-17T21:19:08.052 INFO:teuthology.orchestra.run.smithi184.stdout:2026 write 0x450ee00 thru 0x451c691 (0xd892 bytes) 2024-09-17T21:19:08.080 INFO:teuthology.orchestra.run.smithi184.stdout:2028 write 0xdbd1600 thru 0xdbd7eaf (0x68b0 bytes) 2024-09-17T21:19:08.085 INFO:teuthology.orchestra.run.smithi184.stdout:2032 read 0x5386200 thru 0x538cda3 (0x6ba4 bytes) 2024-09-17T21:19:08.085 INFO:teuthology.orchestra.run.smithi184.stdout:2033 punch from 0x5776c00 to 0x577ea00, (0x7e00 bytes) 2024-09-17T21:19:08.086 INFO:teuthology.orchestra.run.smithi184.stdout:2034 write 0x717600 thru 0x71b35a (0x3d5b bytes) 2024-09-17T21:19:08.089 INFO:teuthology.orchestra.run.smithi184.stdout:2038 write 0xcdbd200 thru 0xcdc5c09 (0x8a0a bytes) 2024-09-17T21:19:08.091 INFO:teuthology.orchestra.run.smithi184.stdout:2042 write 0x71ec400 thru 0x71f60f7 (0x9cf8 bytes) 2024-09-17T21:19:08.093 INFO:teuthology.orchestra.run.smithi184.stdout:2044 read 0x5782400 thru 0x578a406 (0x8007 bytes) 2024-09-17T21:19:08.096 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:08.092+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:08.110 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:08.110 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:08.110 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:08.110 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:08.110 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:08.116 INFO:teuthology.orchestra.run.smithi184.stdout:2048 trunc from 0xdbd8000 to 0xb0a200 2024-09-17T21:19:08.124 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:19:08.180 INFO:teuthology.orchestra.run.smithi184.stdout:2050 read 0x8fa200 thru 0x906987 (0xc788 bytes) 2024-09-17T21:19:08.198 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:08.192+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:08.219 INFO:teuthology.orchestra.run.smithi184.stdout:2051 write 0x7229400 thru 0x722aa27 (0x1628 bytes) 2024-09-17T21:19:08.222 INFO:teuthology.orchestra.run.smithi184.stdout:2054 read 0x3939e00 thru 0x3945ead (0xc0ae bytes) 2024-09-17T21:19:08.223 INFO:teuthology.orchestra.run.smithi184.stdout:2055 read 0x6f9c000 thru 0x6fa4152 (0x8153 bytes) 2024-09-17T21:19:08.224 INFO:teuthology.orchestra.run.smithi184.stdout:2056 read 0x18a9000 thru 0x18af2dc (0x62dd bytes) 2024-09-17T21:19:08.224 INFO:teuthology.orchestra.run.smithi184.stdout:2058 read 0x370da00 thru 0x371140b (0x3a0c bytes) 2024-09-17T21:19:08.225 INFO:teuthology.orchestra.run.smithi184.stdout:2059 read 0x6fea000 thru 0x6ff713f (0xd140 bytes) 2024-09-17T21:19:08.262 INFO:teuthology.orchestra.run.smithi184.stdout:2063 write 0xd568600 thru 0xd56e1d0 (0x5bd1 bytes) 2024-09-17T21:19:08.265 INFO:teuthology.orchestra.run.smithi184.stdout:2065 write 0x2799c00 thru 0x279d7ea (0x3beb bytes) 2024-09-17T21:19:08.268 INFO:teuthology.orchestra.run.smithi184.stdout:2066 write 0x8d3ba00 thru 0x8d3f57d (0x3b7e bytes) 2024-09-17T21:19:08.268 INFO:teuthology.orchestra.run.smithi184.stdout:2067 read 0x79dec00 thru 0x79e2456 (0x3857 bytes) 2024-09-17T21:19:08.268 INFO:teuthology.orchestra.run.smithi184.stdout:2068 read 0x9d40600 thru 0x9d42bb0 (0x25b1 bytes) 2024-09-17T21:19:08.293 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:08.293 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:08.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:08.294 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:08.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.294 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:12 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:19:08.294 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:19:08.294 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 54.957990160000001, 2024-09-17T21:19:08.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061340989999999996, 2024-09-17T21:19:08.294 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:08.294 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:08.294 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:08.294 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:08.294 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:19:08.294 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:08.294 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12 2024-09-17T21:19:08.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.295 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:08.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:08.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:19:08.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:08.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:19:08.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:08.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332336+0000", 2024-09-17T21:19:08.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5519999999999999e-06 2024-09-17T21:19:08.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:08.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332342+0000", 2024-09-17T21:19:08.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7180000000000001e-06 2024-09-17T21:19:08.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:08.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332352+0000", 2024-09-17T21:19:08.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:19:08.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332359+0000", 2024-09-17T21:19:08.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3479999999999998e-06 2024-09-17T21:19:08.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:08.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332413+0000", 2024-09-17T21:19:08.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4447999999999998e-05 2024-09-17T21:19:08.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-17T21:19:08.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332417+0000", 2024-09-17T21:19:08.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.895e-06 2024-09-17T21:19:08.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:08.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334755+0000", 2024-09-17T21:19:08.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023377189999999998 2024-09-17T21:19:08.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:08.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334903+0000", 2024-09-17T21:19:08.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014778999999999999 2024-09-17T21:19:08.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:19:08.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.335187+0000", 2024-09-17T21:19:08.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000284715 2024-09-17T21:19:08.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:08.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.337731+0000", 2024-09-17T21:19:08.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002543741 2024-09-17T21:19:08.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:08.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338382+0000", 2024-09-17T21:19:08.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00065029200000000001 2024-09-17T21:19:08.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:08.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338418+0000", 2024-09-17T21:19:08.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.697e-05 2024-09-17T21:19:08.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:08.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338469+0000", 2024-09-17T21:19:08.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0154999999999997e-05 2024-09-17T21:19:08.300 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.300 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:08.300 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.300 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "pg_update_log_missing(3.1e epoch 28/27 rep_tid 81 entries 28'41 (0'0) error 3:7b23fd98:::rbd_header.1152c01721e5:head by client.4488.0:561 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-17T21:19:08.300 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:19:08.300 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.008716032999999, 2024-09-17T21:19:08.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0051146969999999996, 2024-09-17T21:19:08.300 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:08.300 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:08.301 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:19:08.301 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:08.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:08.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:19:08.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:08.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:19:08.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:08.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281610+0000", 2024-09-17T21:19:08.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.16e-07 2024-09-17T21:19:08.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:08.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281626+0000", 2024-09-17T21:19:08.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6039999999999999e-05 2024-09-17T21:19:08.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:08.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281666+0000", 2024-09-17T21:19:08.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0364000000000002e-05 2024-09-17T21:19:08.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281710+0000", 2024-09-17T21:19:08.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3596000000000001e-05 2024-09-17T21:19:08.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:08.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281992+0000", 2024-09-17T21:19:08.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028273099999999999 2024-09-17T21:19:08.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:08.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286723+0000", 2024-09-17T21:19:08.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047310499999999997 2024-09-17T21:19:08.304 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.304 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:08.304 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.304 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4488.0:562 3.1e e28/27 3:7b23fd98:::rbd_header.1152c01721e5:head v 28'42, mlcod=0'0)", 2024-09-17T21:19:08.304 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:19:08.304 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.008472245, 2024-09-17T21:19:08.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049275020000000003, 2024-09-17T21:19:08.304 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:08.304 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:08.304 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:08.304 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:08.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:08.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:19:08.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:08.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:19:08.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:08.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281853+0000", 2024-09-17T21:19:08.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2999999999999999e-07 2024-09-17T21:19:08.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:08.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281875+0000", 2024-09-17T21:19:08.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1386000000000002e-05 2024-09-17T21:19:08.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:08.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281892+0000", 2024-09-17T21:19:08.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7637000000000001e-05 2024-09-17T21:19:08.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281908+0000", 2024-09-17T21:19:08.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5187e-05 2024-09-17T21:19:08.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:08.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.282517+0000", 2024-09-17T21:19:08.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00060989799999999999 2024-09-17T21:19:08.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:08.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.282548+0000", 2024-09-17T21:19:08.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0910000000000001e-05 2024-09-17T21:19:08.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:08.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286729+0000", 2024-09-17T21:19:08.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0041810850000000002 2024-09-17T21:19:08.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:08.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286780+0000", 2024-09-17T21:19:08.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0569000000000003e-05 2024-09-17T21:19:08.308 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.308 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:08.308 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.309 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:609 3.d e28/23 3:b7278be9:::rbd_data.1152c6af092e.000000000000008f:head v 28'6, mlcod=28'4)", 2024-09-17T21:19:08.309 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:19:08.309 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.175527958000004, 2024-09-17T21:19:08.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049797280000000001, 2024-09-17T21:19:08.309 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:08.309 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:08.309 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:08.309 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:08.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.309 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:08.309 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:19:08.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:08.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:19:08.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:08.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114800+0000", 2024-09-17T21:19:08.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:19:08.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:08.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114915+0000", 2024-09-17T21:19:08.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115053 2024-09-17T21:19:08.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:08.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114951+0000", 2024-09-17T21:19:08.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6165999999999999e-05 2024-09-17T21:19:08.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114965+0000", 2024-09-17T21:19:08.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4364e-05 2024-09-17T21:19:08.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:08.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115097+0000", 2024-09-17T21:19:08.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013139200000000001 2024-09-17T21:19:08.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:08.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115151+0000", 2024-09-17T21:19:08.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4422000000000002e-05 2024-09-17T21:19:08.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:08.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.119633+0000", 2024-09-17T21:19:08.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044818820000000004 2024-09-17T21:19:08.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:08.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.119776+0000", 2024-09-17T21:19:08.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000143609 2024-09-17T21:19:08.313 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.313 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:08.313 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.313 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:782 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'63, mlcod=28'60)", 2024-09-17T21:19:08.313 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:19:08.313 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.075559929999997, 2024-09-17T21:19:08.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006167283, 2024-09-17T21:19:08.313 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:08.313 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:08.314 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:08.314 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:08.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:08.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:19:08.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:08.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:19:08.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:08.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214767+0000", 2024-09-17T21:19:08.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:19:08.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:08.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214779+0000", 2024-09-17T21:19:08.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2627e-05 2024-09-17T21:19:08.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:08.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214805+0000", 2024-09-17T21:19:08.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5570000000000001e-05 2024-09-17T21:19:08.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214823+0000", 2024-09-17T21:19:08.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8202999999999998e-05 2024-09-17T21:19:08.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:08.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215170+0000", 2024-09-17T21:19:08.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034720399999999999 2024-09-17T21:19:08.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:08.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215235+0000", 2024-09-17T21:19:08.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4276000000000006e-05 2024-09-17T21:19:08.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:08.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220859+0000", 2024-09-17T21:19:08.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056240500000000002 2024-09-17T21:19:08.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:08.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220932+0000", 2024-09-17T21:19:08.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3503000000000004e-05 2024-09-17T21:19:08.317 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.317 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:08.317 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.318 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:783 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'64, mlcod=28'60)", 2024-09-17T21:19:08.318 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:19:08.318 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.074873087999997, 2024-09-17T21:19:08.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0055240869999999996, 2024-09-17T21:19:08.318 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:08.318 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:08.318 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:08.318 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:08.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:08.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:19:08.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:08.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:19:08.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:08.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215454+0000", 2024-09-17T21:19:08.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0269999999999998e-06 2024-09-17T21:19:08.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:08.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215468+0000", 2024-09-17T21:19:08.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4816e-05 2024-09-17T21:19:08.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:08.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215501+0000", 2024-09-17T21:19:08.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2957000000000003e-05 2024-09-17T21:19:08.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215524+0000", 2024-09-17T21:19:08.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2782999999999999e-05 2024-09-17T21:19:08.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:08.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216675+0000", 2024-09-17T21:19:08.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011507869999999999 2024-09-17T21:19:08.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:08.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216728+0000", 2024-09-17T21:19:08.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3112999999999998e-05 2024-09-17T21:19:08.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:08.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220938+0000", 2024-09-17T21:19:08.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042097380000000002 2024-09-17T21:19:08.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:08.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220976+0000", 2024-09-17T21:19:08.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7866e-05 2024-09-17T21:19:08.322 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.322 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:08.322 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.323 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:785 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'65, mlcod=28'60)", 2024-09-17T21:19:08.323 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:19:08.323 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.074287038999998, 2024-09-17T21:19:08.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049795409999999997, 2024-09-17T21:19:08.323 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:08.323 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:08.323 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:08.323 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:08.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:08.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:19:08.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:08.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:19:08.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:08.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216039+0000", 2024-09-17T21:19:08.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.793e-06 2024-09-17T21:19:08.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:08.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216053+0000", 2024-09-17T21:19:08.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.345e-05 2024-09-17T21:19:08.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:08.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216092+0000", 2024-09-17T21:19:08.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9180000000000001e-05 2024-09-17T21:19:08.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216106+0000", 2024-09-17T21:19:08.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:19:08.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:08.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217569+0000", 2024-09-17T21:19:08.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001462938 2024-09-17T21:19:08.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:08.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217607+0000", 2024-09-17T21:19:08.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7267000000000001e-05 2024-09-17T21:19:08.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:08.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220980+0000", 2024-09-17T21:19:08.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003373544 2024-09-17T21:19:08.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:08.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221017+0000", 2024-09-17T21:19:08.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7116000000000002e-05 2024-09-17T21:19:08.327 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.327 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:08.327 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.327 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:786 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'66, mlcod=28'60)", 2024-09-17T21:19:08.327 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:19:08.327 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.073586235, 2024-09-17T21:19:08.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057966650000000003, 2024-09-17T21:19:08.327 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:08.327 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:08.327 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:08.328 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:08.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:08.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:19:08.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:08.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:19:08.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:08.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216740+0000", 2024-09-17T21:19:08.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5600000000000001e-06 2024-09-17T21:19:08.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:08.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216752+0000", 2024-09-17T21:19:08.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1477e-05 2024-09-17T21:19:08.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:08.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216779+0000", 2024-09-17T21:19:08.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7093e-05 2024-09-17T21:19:08.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216817+0000", 2024-09-17T21:19:08.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8275999999999997e-05 2024-09-17T21:19:08.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:08.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218387+0000", 2024-09-17T21:19:08.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015703749999999999 2024-09-17T21:19:08.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:08.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218432+0000", 2024-09-17T21:19:08.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4910000000000002e-05 2024-09-17T21:19:08.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:08.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222432+0000", 2024-09-17T21:19:08.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039997189999999997 2024-09-17T21:19:08.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:08.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222535+0000", 2024-09-17T21:19:08.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103255 2024-09-17T21:19:08.331 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.331 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:08.331 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.332 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:19:08.332 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:08.332 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.072964403, 2024-09-17T21:19:08.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:19:08.332 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:08.332 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:08.332 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:08.332 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:08.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:08.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:08.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:08.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:08.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:08.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:19:08.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:19:08.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:08.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:19:08.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:19:08.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:08.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:19:08.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:19:08.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:19:08.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:19:08.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:08.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:19:08.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:19:08.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:08.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:19:08.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:19:08.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:08.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:19:08.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:19:08.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:08.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:19:08.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:19:08.336 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.336 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:08.336 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.336 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:788 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'68, mlcod=28'60)", 2024-09-17T21:19:08.336 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:19:08.336 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.072329971000002, 2024-09-17T21:19:08.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058495320000000002, 2024-09-17T21:19:08.337 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:08.337 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:08.337 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:08.337 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:08.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:08.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:19:08.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:08.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:19:08.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:08.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217996+0000", 2024-09-17T21:19:08.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.376e-06 2024-09-17T21:19:08.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:08.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218006+0000", 2024-09-17T21:19:08.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.01e-05 2024-09-17T21:19:08.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:08.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218030+0000", 2024-09-17T21:19:08.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4026999999999999e-05 2024-09-17T21:19:08.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218042+0000", 2024-09-17T21:19:08.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1977000000000001e-05 2024-09-17T21:19:08.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:08.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221081+0000", 2024-09-17T21:19:08.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030385960000000002 2024-09-17T21:19:08.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:08.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221115+0000", 2024-09-17T21:19:08.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3772999999999997e-05 2024-09-17T21:19:08.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:08.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223749+0000", 2024-09-17T21:19:08.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0026340629999999999 2024-09-17T21:19:08.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:08.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223844+0000", 2024-09-17T21:19:08.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5619999999999996e-05 2024-09-17T21:19:08.340 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.341 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:08.341 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.341 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:790 3.5 e28/23 3:a14b1a37:::rbd_data.1152c6af092e.000000000000002f:head v 28'1, mlcod=0'0)", 2024-09-17T21:19:08.341 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:19:08.341 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.067925645000003, 2024-09-17T21:19:08.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059622240000000003, 2024-09-17T21:19:08.341 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:08.341 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:08.341 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:08.341 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:08.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:08.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:19:08.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:08.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:19:08.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:08.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222400+0000", 2024-09-17T21:19:08.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.02e-06 2024-09-17T21:19:08.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:08.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222411+0000", 2024-09-17T21:19:08.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.044e-05 2024-09-17T21:19:08.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:08.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222440+0000", 2024-09-17T21:19:08.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.932e-05 2024-09-17T21:19:08.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222458+0000", 2024-09-17T21:19:08.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8615999999999999e-05 2024-09-17T21:19:08.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:08.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222609+0000", 2024-09-17T21:19:08.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015102299999999999 2024-09-17T21:19:08.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:08.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222657+0000", 2024-09-17T21:19:08.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7206000000000002e-05 2024-09-17T21:19:08.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:08.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228278+0000", 2024-09-17T21:19:08.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056208100000000004 2024-09-17T21:19:08.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:08.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228361+0000", 2024-09-17T21:19:08.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3788999999999993e-05 2024-09-17T21:19:08.345 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.345 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:08.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.346 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:884 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'76, mlcod=28'70)", 2024-09-17T21:19:08.346 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:19:08.346 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.021340764000001, 2024-09-17T21:19:08.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056451990000000001, 2024-09-17T21:19:08.346 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:08.346 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:08.346 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:08.346 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:08.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:08.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:19:08.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:08.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:19:08.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:08.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268986+0000", 2024-09-17T21:19:08.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9360000000000002e-06 2024-09-17T21:19:08.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:08.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269001+0000", 2024-09-17T21:19:08.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4684e-05 2024-09-17T21:19:08.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:08.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269026+0000", 2024-09-17T21:19:08.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5193000000000001e-05 2024-09-17T21:19:08.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269043+0000", 2024-09-17T21:19:08.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7306e-05 2024-09-17T21:19:08.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:08.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.270024+0000", 2024-09-17T21:19:08.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00098120600000000009 2024-09-17T21:19:08.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:08.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.270085+0000", 2024-09-17T21:19:08.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0486000000000003e-05 2024-09-17T21:19:08.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:08.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.274557+0000", 2024-09-17T21:19:08.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044723419999999998 2024-09-17T21:19:08.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:08.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.274629+0000", 2024-09-17T21:19:08.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2045999999999994e-05 2024-09-17T21:19:08.350 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.350 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:08.350 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.350 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:19:08.350 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:08.350 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 45.422663739000001, 2024-09-17T21:19:08.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:19:08.351 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:08.351 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:08.351 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:08.351 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:08.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:08.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:08.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:08.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:08.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:08.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:19:08.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:19:08.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:08.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:19:08.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:19:08.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:08.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:19:08.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:19:08.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:19:08.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:19:08.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:08.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:19:08.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:19:08.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:08.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:19:08.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:19:08.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:08.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:19:08.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:19:08.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:08.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:19:08.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:19:08.355 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.355 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:08.355 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.355 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:19:08.355 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:08.355 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 45.421398656000001, 2024-09-17T21:19:08.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:19:08.355 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:08.355 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:08.355 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:08.356 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:08.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:08.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:08.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:08.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:08.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:08.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:19:08.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:19:08.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:08.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:19:08.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:19:08.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:08.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:19:08.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:19:08.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:19:08.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:19:08.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:08.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:19:08.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:19:08.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:08.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:19:08.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:19:08.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:08.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:19:08.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:19:08.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:08.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:19:08.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:19:08.359 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.359 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:08.359 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.360 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:19:08.360 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:08.360 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 45.411619246000001, 2024-09-17T21:19:08.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:19:08.360 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:08.360 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:08.360 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:08.360 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:08.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:08.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:08.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:08.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:08.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:08.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:19:08.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:19:08.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:08.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:19:08.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:19:08.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:08.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:19:08.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:19:08.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:19:08.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:19:08.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:08.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:19:08.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:19:08.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:08.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:19:08.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:19:08.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:08.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:19:08.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:19:08.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:08.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:19:08.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:19:08.364 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.364 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:08.364 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.364 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:19:08.364 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:08.364 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 45.402896843000001, 2024-09-17T21:19:08.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:19:08.365 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:08.365 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:08.365 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:08.365 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:08.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:08.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:08.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:08.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:08.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:08.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:19:08.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:19:08.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:08.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:19:08.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:19:08.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:08.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:19:08.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:19:08.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:19:08.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:19:08.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:08.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:19:08.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:19:08.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:08.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:19:08.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:19:08.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:08.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:19:08.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:19:08.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:08.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:19:08.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:19:08.368 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.368 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:08.369 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.369 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4498.0:596 3.4 e32/23 3:20f262ea:::rbd_data.1152b51cb396.000000000000000d:head v 32'17, mlcod=30'15)", 2024-09-17T21:19:08.369 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.258354+0000", 2024-09-17T21:19:08.369 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 42.031970979, 2024-09-17T21:19:08.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0053727619999999997, 2024-09-17T21:19:08.369 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:08.369 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:08.369 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:08.369 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:08.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:08.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.258354+0000", 2024-09-17T21:19:08.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:08.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.258354+0000", 2024-09-17T21:19:08.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:08.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.258356+0000", 2024-09-17T21:19:08.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.798e-06 2024-09-17T21:19:08.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:08.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259198+0000", 2024-09-17T21:19:08.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084279199999999998 2024-09-17T21:19:08.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:08.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259229+0000", 2024-09-17T21:19:08.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0383999999999999e-05 2024-09-17T21:19:08.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259241+0000", 2024-09-17T21:19:08.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2139e-05 2024-09-17T21:19:08.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:08.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259341+0000", 2024-09-17T21:19:08.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010049 2024-09-17T21:19:08.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:08.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259380+0000", 2024-09-17T21:19:08.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9072000000000001e-05 2024-09-17T21:19:08.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:08.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.263668+0000", 2024-09-17T21:19:08.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042875389999999999 2024-09-17T21:19:08.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:08.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.263726+0000", 2024-09-17T21:19:08.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8548000000000003e-05 2024-09-17T21:19:08.373 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.373 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:08.373 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.373 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4520.0:6177 3.8 e34/27 3:1665de58:::rbd_data.11528c3b0529.0000000000000019:head v 34'43, mlcod=34'41)", 2024-09-17T21:19:08.373 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:30.509962+0000", 2024-09-17T21:19:08.373 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.780362988999997, 2024-09-17T21:19:08.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049320850000000001, 2024-09-17T21:19:08.373 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:08.374 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:08.374 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:08.374 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:08.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:08.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509962+0000", 2024-09-17T21:19:08.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:08.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509962+0000", 2024-09-17T21:19:08.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:08.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509964+0000", 2024-09-17T21:19:08.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.356e-06 2024-09-17T21:19:08.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:08.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509978+0000", 2024-09-17T21:19:08.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3434e-05 2024-09-17T21:19:08.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:08.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510001+0000", 2024-09-17T21:19:08.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3436000000000001e-05 2024-09-17T21:19:08.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510023+0000", 2024-09-17T21:19:08.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1739999999999999e-05 2024-09-17T21:19:08.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:08.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510193+0000", 2024-09-17T21:19:08.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017040600000000001 2024-09-17T21:19:08.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:08.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510250+0000", 2024-09-17T21:19:08.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6749000000000002e-05 2024-09-17T21:19:08.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:08.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.514823+0000", 2024-09-17T21:19:08.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0045728180000000002 2024-09-17T21:19:08.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:08.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.514894+0000", 2024-09-17T21:19:08.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1145999999999999e-05 2024-09-17T21:19:08.377 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.377 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:08.377 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.378 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:19:08.378 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:08.378 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 32.017806278999998, 2024-09-17T21:19:08.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:19:08.378 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:08.378 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:08.378 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:08.378 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:08.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:08.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:08.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:08.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:08.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:08.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:19:08.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:19:08.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:08.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:19:08.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:19:08.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:08.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:19:08.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:19:08.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:19:08.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:19:08.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:08.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:19:08.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:19:08.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:08.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:19:08.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:19:08.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:08.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:19:08.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:19:08.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:08.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:19:08.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:19:08.382 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.382 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:08.382 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.382 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4863.0:165 3.5 e52/23 3:a772dfb1:::rbd_data.11523651b015.0000000000000001:head v 52'103, mlcod=47'101)", 2024-09-17T21:19:08.382 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:50.682666+0000", 2024-09-17T21:19:08.383 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 17.607658473000001, 2024-09-17T21:19:08.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0055448839999999999, 2024-09-17T21:19:08.383 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:08.383 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:08.383 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:08.383 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:08.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:08.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.682666+0000", 2024-09-17T21:19:08.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:08.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.682666+0000", 2024-09-17T21:19:08.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:08.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:08.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.682667+0000", 2024-09-17T21:19:08.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.16e-07 2024-09-17T21:19:08.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:08.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.683332+0000", 2024-09-17T21:19:08.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000664476 2024-09-17T21:19:08.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:08.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.683360+0000", 2024-09-17T21:19:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7988e-05 2024-09-17T21:19:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.683376+0000", 2024-09-17T21:19:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6578999999999998e-05 2024-09-17T21:19:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.683522+0000", 2024-09-17T21:19:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014536400000000001 2024-09-17T21:19:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:08.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.683560+0000", 2024-09-17T21:19:08.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8005e-05 2024-09-17T21:19:08.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:08.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.688080+0000", 2024-09-17T21:19:08.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0045201390000000003 2024-09-17T21:19:08.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:08.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:08.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:08.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.688211+0000", 2024-09-17T21:19:08.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013141699999999999 2024-09-17T21:19:08.386 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.386 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:08.387 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.387 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:08.387 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:08.387 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:08.388 INFO:teuthology.orchestra.run.smithi184.stdout:2071 trunc from 0xd56e200 to 0x3cc800 2024-09-17T21:19:08.389 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:19:08.390 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:08.384+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:08.390 INFO:teuthology.orchestra.run.smithi184.stdout:2076 write 0x1d72a00 thru 0x1d74f9a (0x259b bytes) 2024-09-17T21:19:08.390 INFO:teuthology.orchestra.run.smithi184.stdout:2077 read 0x448600 thru 0x4499bc (0x13bd bytes) 2024-09-17T21:19:08.431 INFO:teuthology.orchestra.run.smithi184.stdout:2078 write 0x977b200 thru 0x978a0a8 (0xeea9 bytes) 2024-09-17T21:19:08.434 INFO:teuthology.orchestra.run.smithi184.stdout:2080 write 0x7a49600 thru 0x7a514af (0x7eb0 bytes) 2024-09-17T21:19:08.434 INFO:teuthology.orchestra.run.smithi184.stdout:2081 read 0x379a600 thru 0x37a9ddc (0xf7dd bytes) 2024-09-17T21:19:08.434 INFO:teuthology.orchestra.run.smithi184.stdout:2083 read 0x2a1b800 thru 0x2a23b44 (0x8345 bytes) 2024-09-17T21:19:08.434 INFO:teuthology.orchestra.run.smithi184.stdout:skipping zero length punch hole 2024-09-17T21:19:08.435 INFO:teuthology.orchestra.run.smithi184.stdout:2085 read 0x643ca00 thru 0x644761f (0xac20 bytes) 2024-09-17T21:19:08.436 INFO:teuthology.orchestra.run.smithi184.stdout:2086 punch from 0x8479400 to 0x8482400, (0x9000 bytes) 2024-09-17T21:19:08.437 INFO:teuthology.orchestra.run.smithi184.stdout:2088 punch from 0x5129c00 to 0x5136e00, (0xd200 bytes) 2024-09-17T21:19:08.437 INFO:teuthology.orchestra.run.smithi184.stdout:2090 read 0x6bd7000 thru 0x6bddfda (0x6fdb bytes) 2024-09-17T21:19:08.437 INFO:teuthology.orchestra.run.smithi184.stdout:2094 write 0x6732c00 thru 0x6742b55 (0xff56 bytes) 2024-09-17T21:19:08.439 INFO:teuthology.orchestra.run.smithi184.stdout:2096 write 0x61b6e00 thru 0x61b774e (0x94f bytes) 2024-09-17T21:19:08.465 INFO:teuthology.orchestra.run.smithi184.stdout:2097 write 0xecb6e00 thru 0xecbfc2d (0x8e2e bytes) 2024-09-17T21:19:08.467 INFO:teuthology.orchestra.run.smithi184.stdout:2098 read 0xc68f200 thru 0xc6999a4 (0xa7a5 bytes) 2024-09-17T21:19:08.467 INFO:teuthology.orchestra.run.smithi184.stdout:2099 punch from 0x6b47a00 to 0x6b53e00, (0xc400 bytes) 2024-09-17T21:19:08.489 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:08.484+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:08.501 INFO:teuthology.orchestra.run.smithi184.stdout:2100 trunc from 0xecbfe00 to 0x53c400 2024-09-17T21:19:08.535 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:08.535 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:08.535 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:19:08.535 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:08.550 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:19:08.594 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:08.592+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:08.604 INFO:teuthology.orchestra.run.smithi184.stdout:2101 write 0xb1aee00 thru 0xb1b6721 (0x7922 bytes) 2024-09-17T21:19:08.607 INFO:teuthology.orchestra.run.smithi184.stdout:2104 punch from 0x9ec2400 to 0x9ecda00, (0xb600 bytes) 2024-09-17T21:19:08.607 INFO:teuthology.orchestra.run.smithi184.stdout:2105 read 0xd1d800 thru 0xd25897 (0x8098 bytes) 2024-09-17T21:19:08.617 INFO:teuthology.orchestra.run.smithi184.stdout:2109 trunc from 0xb1b6800 to 0x539e400 2024-09-17T21:19:08.641 INFO:teuthology.orchestra.run.smithi184.stdout:2111 write 0x3881000 thru 0x388fd43 (0xed44 bytes) 2024-09-17T21:19:08.642 INFO:teuthology.orchestra.run.smithi184.stdout:2113 read 0x2473a00 thru 0x247e210 (0xa811 bytes) 2024-09-17T21:19:08.642 INFO:teuthology.orchestra.run.smithi184.stdout:2114 punch from 0x20a5600 to 0x20a7e00, (0x2800 bytes) 2024-09-17T21:19:08.643 INFO:teuthology.orchestra.run.smithi184.stdout:2115 punch from 0x14cd200 to 0x14d9e00, (0xcc00 bytes) 2024-09-17T21:19:08.643 INFO:teuthology.orchestra.run.smithi184.stdout:2118 write 0x4200200 thru 0x420541c (0x521d bytes) 2024-09-17T21:19:08.643 INFO:teuthology.orchestra.run.smithi184.stdout:2119 punch from 0x1c6a600 to 0x1c76a00, (0xc400 bytes) 2024-09-17T21:19:08.644 INFO:teuthology.orchestra.run.smithi184.stdout:2121 read 0x6dba00 thru 0x6dd0e9 (0x16ea bytes) 2024-09-17T21:19:08.644 INFO:teuthology.orchestra.run.smithi184.stdout:2122 write 0x2476200 thru 0x2479e2c (0x3c2d bytes) 2024-09-17T21:19:08.645 INFO:teuthology.orchestra.run.smithi184.stdout:2123 punch from 0x1372c00 to 0x1379000, (0x6400 bytes) 2024-09-17T21:19:08.675 INFO:teuthology.orchestra.run.smithi184.stdout:2124 write 0x6b92600 thru 0x6b9d486 (0xae87 bytes) 2024-09-17T21:19:08.691 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:08.688+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:08.697 INFO:teuthology.orchestra.run.smithi184.stdout:2126 write 0x739dc00 thru 0x73a1032 (0x3433 bytes) 2024-09-17T21:19:08.699 INFO:teuthology.orchestra.run.smithi184.stdout:2127 read 0x7285800 thru 0x7286726 (0xf27 bytes) 2024-09-17T21:19:08.700 INFO:teuthology.orchestra.run.smithi184.stdout:2128 punch from 0x2d9be00 to 0x2daa600, (0xe800 bytes) 2024-09-17T21:19:08.707 INFO:teuthology.orchestra.run.smithi184.stdout:2129 trunc from 0x73a1200 to 0xa6f8200 2024-09-17T21:19:08.722 INFO:teuthology.orchestra.run.smithi184.stdout:2131 trunc from 0xa6f8200 to 0xb3abc00 2024-09-17T21:19:08.734 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:08.735 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:08.735 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:08.735 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:08.735 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:08.736 INFO:teuthology.orchestra.run.smithi184.stdout:2132 read 0x3275800 thru 0x327e4c9 (0x8cca bytes) 2024-09-17T21:19:08.737 INFO:teuthology.orchestra.run.smithi184.stdout:2134 read 0x2216000 thru 0x222156c (0xb56d bytes) 2024-09-17T21:19:08.738 INFO:teuthology.orchestra.run.smithi184.stdout:2137 write 0x46ce000 thru 0x46d78ec (0x98ed bytes) 2024-09-17T21:19:08.739 INFO:teuthology.orchestra.run.smithi184.stdout:2142 read 0x202200 thru 0x20648f (0x4290 bytes) 2024-09-17T21:19:08.739 INFO:teuthology.orchestra.run.smithi184.stdout:2145 write 0x704e00 thru 0x70ce21 (0x8022 bytes) 2024-09-17T21:19:08.747 INFO:teuthology.orchestra.run.smithi184.stdout:2146 trunc from 0xb3abc00 to 0x7476a00 2024-09-17T21:19:08.750 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:19:08.794 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:08.792+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:08.893 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:08.891+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:08.907 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:19:08.918 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:08.918 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:08.918 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:08.918 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:08.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.918 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:19:08.918 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:08.918 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 71.900571595000002, 2024-09-17T21:19:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:19:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:19:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:19:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:19:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:19:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:19:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:19:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:19:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:19:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:19:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:19:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:19:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:19:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:19:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:19:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:19:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:19:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:19:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:19:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:19:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:19:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:19:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:19:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:19:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:19:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:19:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:19:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:19:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:19:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:19:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:19:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 52.320407314999997, 2024-09-17T21:19:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:19:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:19:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:19:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:19:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:19:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:19:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:19:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:19:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:19:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:19:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:19:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:19:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:19:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:19:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:19:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:19:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:19:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:19:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:19:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:19:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:19:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:19:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:19:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:19:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:19:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:19:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 52.320310003000003, 2024-09-17T21:19:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:19:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:19:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:19:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:19:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:19:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:19:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:19:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:19:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:19:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:19:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:19:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:19:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:19:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:19:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:19:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:19:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:19:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:19:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:19:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:19:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:19:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:19:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:19:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:19:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:19:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:19:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:19:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:19:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:19:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 52.318223201999999, 2024-09-17T21:19:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:19:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:19:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:19:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:19:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:19:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:19:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:19:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:19:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:19:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:19:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:19:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:19:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:19:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:19:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:19:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:19:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:19:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:19:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:19:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:19:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:19:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:19:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:19:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:19:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:19:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:19:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:19:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:19:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:19:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:19:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:19:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:19:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:19:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:19:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:19:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:282 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:19:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.686517940999998, 2024-09-17T21:19:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023571801, 2024-09-17T21:19:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 282 2024-09-17T21:19:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:19:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:19:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227820+0000", 2024-09-17T21:19:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.11e-06 2024-09-17T21:19:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227825+0000", 2024-09-17T21:19:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8099999999999997e-06 2024-09-17T21:19:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227848+0000", 2024-09-17T21:19:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2949999999999999e-05 2024-09-17T21:19:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227862+0000", 2024-09-17T21:19:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:19:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.247977+0000", 2024-09-17T21:19:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020114968 2024-09-17T21:19:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251141+0000", 2024-09-17T21:19:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031640689999999998 2024-09-17T21:19:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251391+0000", 2024-09-17T21:19:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025018499999999999 2024-09-17T21:19:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:283 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:19:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.686456102000001, 2024-09-17T21:19:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024022274999999999, 2024-09-17T21:19:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 283 2024-09-17T21:19:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:19:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:19:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227882+0000", 2024-09-17T21:19:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-06 2024-09-17T21:19:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227887+0000", 2024-09-17T21:19:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1769999999999997e-06 2024-09-17T21:19:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227905+0000", 2024-09-17T21:19:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7235999999999999e-05 2024-09-17T21:19:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227918+0000", 2024-09-17T21:19:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3387000000000001e-05 2024-09-17T21:19:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251469+0000", 2024-09-17T21:19:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023550894999999999 2024-09-17T21:19:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251669+0000", 2024-09-17T21:19:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019992499999999999 2024-09-17T21:19:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251903+0000", 2024-09-17T21:19:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000234412 2024-09-17T21:19:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:284 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:19:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.686400548999998, 2024-09-17T21:19:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024450084, 2024-09-17T21:19:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 284 2024-09-17T21:19:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:19:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:19:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227937+0000", 2024-09-17T21:19:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6700000000000002e-07 2024-09-17T21:19:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227943+0000", 2024-09-17T21:19:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:19:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227961+0000", 2024-09-17T21:19:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.827e-05 2024-09-17T21:19:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227975+0000", 2024-09-17T21:19:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.394e-05 2024-09-17T21:19:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251977+0000", 2024-09-17T21:19:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024001340999999999 2024-09-17T21:19:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252169+0000", 2024-09-17T21:19:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019271199999999999 2024-09-17T21:19:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252387+0000", 2024-09-17T21:19:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021726100000000001 2024-09-17T21:19:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:285 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:19:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.686344259000002, 2024-09-17T21:19:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024884791, 2024-09-17T21:19:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 285 2024-09-17T21:19:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:19:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:19:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227994+0000", 2024-09-17T21:19:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1429999999999999e-06 2024-09-17T21:19:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227999+0000", 2024-09-17T21:19:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1669999999999998e-06 2024-09-17T21:19:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228015+0000", 2024-09-17T21:19:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6056e-05 2024-09-17T21:19:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228028+0000", 2024-09-17T21:19:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3196e-05 2024-09-17T21:19:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252458+0000", 2024-09-17T21:19:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024429751999999999 2024-09-17T21:19:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252650+0000", 2024-09-17T21:19:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019187800000000001 2024-09-17T21:19:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252878+0000", 2024-09-17T21:19:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022759899999999999 2024-09-17T21:19:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:286 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:19:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.686288500000003, 2024-09-17T21:19:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025299318000000001, 2024-09-17T21:19:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 286 2024-09-17T21:19:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:19:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:19:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228050+0000", 2024-09-17T21:19:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:19:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228055+0000", 2024-09-17T21:19:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4960000000000004e-06 2024-09-17T21:19:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228075+0000", 2024-09-17T21:19:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0109999999999999e-05 2024-09-17T21:19:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228089+0000", 2024-09-17T21:19:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4154e-05 2024-09-17T21:19:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252952+0000", 2024-09-17T21:19:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024862934 2024-09-17T21:19:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253141+0000", 2024-09-17T21:19:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000188782 2024-09-17T21:19:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253348+0000", 2024-09-17T21:19:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020684499999999999 2024-09-17T21:19:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:443 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:19:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.672701195000002, 2024-09-17T21:19:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023179569000000001, 2024-09-17T21:19:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 443 2024-09-17T21:19:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:19:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:19:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241637+0000", 2024-09-17T21:19:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.46e-06 2024-09-17T21:19:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241643+0000", 2024-09-17T21:19:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1970000000000001e-06 2024-09-17T21:19:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241662+0000", 2024-09-17T21:19:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8712999999999999e-05 2024-09-17T21:19:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241676+0000", 2024-09-17T21:19:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.393e-05 2024-09-17T21:19:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264596+0000", 2024-09-17T21:19:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022919688000000001 2024-09-17T21:19:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264690+0000", 2024-09-17T21:19:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4145000000000001e-05 2024-09-17T21:19:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264815+0000", 2024-09-17T21:19:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012543600000000001 2024-09-17T21:19:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:444 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:19:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.672640145000003, 2024-09-17T21:19:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023260383999999999, 2024-09-17T21:19:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 444 2024-09-17T21:19:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:19:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:19:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241698+0000", 2024-09-17T21:19:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2270000000000001e-06 2024-09-17T21:19:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241704+0000", 2024-09-17T21:19:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9800000000000003e-06 2024-09-17T21:19:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241725+0000", 2024-09-17T21:19:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0812999999999999e-05 2024-09-17T21:19:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241739+0000", 2024-09-17T21:19:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3773e-05 2024-09-17T21:19:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264850+0000", 2024-09-17T21:19:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023111326000000001 2024-09-17T21:19:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264897+0000", 2024-09-17T21:19:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6638999999999998e-05 2024-09-17T21:19:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264957+0000", 2024-09-17T21:19:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0625999999999999e-05 2024-09-17T21:19:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:445 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:19:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.672579476000003, 2024-09-17T21:19:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023315144999999999, 2024-09-17T21:19:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 445 2024-09-17T21:19:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:19:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:19:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241759+0000", 2024-09-17T21:19:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-06 2024-09-17T21:19:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241764+0000", 2024-09-17T21:19:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5899999999999998e-06 2024-09-17T21:19:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241784+0000", 2024-09-17T21:19:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9204000000000001e-05 2024-09-17T21:19:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241798+0000", 2024-09-17T21:19:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3953e-05 2024-09-17T21:19:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264977+0000", 2024-09-17T21:19:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023179182 2024-09-17T21:19:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.265018+0000", 2024-09-17T21:19:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1171999999999998e-05 2024-09-17T21:19:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.265073+0000", 2024-09-17T21:19:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4741000000000003e-05 2024-09-17T21:19:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:679 3.13 3:cd0d74f9:::rbd_data.11527824718.000000000000000a:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:19:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.663658054999999, 2024-09-17T21:19:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023284978000000001, 2024-09-17T21:19:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 679 2024-09-17T21:19:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.984 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:08.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:08.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:19:08.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:08.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:19:08.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:08.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250680+0000", 2024-09-17T21:19:08.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:19:08.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:08.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250685+0000", 2024-09-17T21:19:08.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1970000000000004e-06 2024-09-17T21:19:08.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:08.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250703+0000", 2024-09-17T21:19:08.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7396000000000001e-05 2024-09-17T21:19:08.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.986 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250716+0000", 2024-09-17T21:19:08.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.29e-05 2024-09-17T21:19:08.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.986 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273740+0000", 2024-09-17T21:19:08.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023024405000000001 2024-09-17T21:19:08.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:08.986 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273834+0000", 2024-09-17T21:19:08.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4124999999999998e-05 2024-09-17T21:19:08.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:08.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273964+0000", 2024-09-17T21:19:08.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012963499999999999 2024-09-17T21:19:08.987 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.987 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:08.987 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.987 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:680 3.13 3:cd0d74f9:::rbd_data.11527824718.000000000000000a:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:08.987 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:19:08.987 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.663602975000003, 2024-09-17T21:19:08.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023687792999999999, 2024-09-17T21:19:08.988 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:08.988 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:08.988 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:08.988 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:08.988 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:08.988 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:08.988 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 680 2024-09-17T21:19:08.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.988 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:08.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:08.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:19:08.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:08.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:19:08.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:08.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250735+0000", 2024-09-17T21:19:08.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2669999999999999e-06 2024-09-17T21:19:08.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:08.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250740+0000", 2024-09-17T21:19:08.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8060000000000002e-06 2024-09-17T21:19:08.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:08.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250757+0000", 2024-09-17T21:19:08.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7303e-05 2024-09-17T21:19:08.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250771+0000", 2024-09-17T21:19:08.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3876999999999999e-05 2024-09-17T21:19:08.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274022+0000", 2024-09-17T21:19:08.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023251135999999999 2024-09-17T21:19:08.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:08.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274211+0000", 2024-09-17T21:19:08.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018813599999999999 2024-09-17T21:19:08.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:08.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274422+0000", 2024-09-17T21:19:08.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000211268 2024-09-17T21:19:08.992 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.992 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:08.992 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.992 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:08.992 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:08.992 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.634324420999999, 2024-09-17T21:19:08.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:19:08.992 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:08.993 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:08.993 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:08.993 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:08.993 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:08.993 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:08.993 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:19:08.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.993 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:08.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:08.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:08.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:08.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:08.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:08.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:08.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:19:08.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:19:08.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:08.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:19:08.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:19:08.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:08.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:19:08.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:19:08.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:08.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:19:08.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:19:08.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:19:08.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:19:08.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:08.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:19:08.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:19:08.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:19:08.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:19:08.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:08.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:19:08.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:19:08.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:08.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:19:08.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:19:08.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:08.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:19:08.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:19:08.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:08.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:19:08.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:19:08.998 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.998 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:08.998 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:08.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:08.998 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:08.999 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:08.999 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.625232709999999, 2024-09-17T21:19:08.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:19:08.999 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:08.999 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:08.999 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:08.999 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:08.999 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:08.999 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:08.999 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:19:08.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:08.999 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:08.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:09.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:09.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:09.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:09.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:09.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:09.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:09.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:19:09.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:19:09.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:09.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:19:09.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:19:09.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:09.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:19:09.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:19:09.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:09.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:19:09.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:19:09.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:09.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:19:09.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:19:09.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:09.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:19:09.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:19:09.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:09.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:19:09.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:19:09.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:09.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:19:09.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:19:09.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:09.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:19:09.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:19:09.003 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:09.004 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:09.004 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:09.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.004 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:09.004 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:09.004 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.625060658999999, 2024-09-17T21:19:09.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:19:09.004 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:09.004 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:09.004 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:09.004 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:09.005 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:09.005 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:09.005 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:19:09.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.005 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:09.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:09.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:09.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:09.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:09.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:09.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:09.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:09.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:19:09.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:19:09.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:09.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:19:09.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:19:09.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:09.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:19:09.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:19:09.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:09.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:19:09.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:19:09.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:09.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:19:09.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:19:09.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:09.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:19:09.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:19:09.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:09.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:19:09.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:19:09.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:09.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:19:09.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:19:09.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:09.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:19:09.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:19:09.009 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:09.009 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:09.009 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:09.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.009 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:09.009 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:09.009 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.450127911999999, 2024-09-17T21:19:09.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:19:09.010 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:09.010 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:09.010 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:09.010 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:09.010 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:09.010 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:09.010 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:19:09.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.010 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:09.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:09.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:09.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:09.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:09.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:09.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:09.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:09.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:19:09.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:19:09.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:09.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:19:09.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:19:09.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:09.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:19:09.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:19:09.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:09.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:19:09.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:19:09.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:09.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:19:09.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:19:09.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:09.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:19:09.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:19:09.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:09.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:19:09.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:19:09.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:09.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:19:09.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:19:09.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:09.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:19:09.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:19:09.014 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:09.014 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:09.014 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:09.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.015 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:09.015 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:09.015 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.450043354999998, 2024-09-17T21:19:09.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:19:09.015 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:09.015 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:09.015 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:09.015 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:09.015 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:09.015 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:09.015 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:19:09.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.016 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:09.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:09.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:09.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:09.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:09.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:09.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:09.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:09.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:09.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:19:09.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:09.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:19:09.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:19:09.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:09.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:19:09.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:19:09.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:09.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:19:09.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:19:09.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:09.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:19:09.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:19:09.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:09.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:19:09.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:19:09.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:09.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:19:09.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:19:09.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:09.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:19:09.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:19:09.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:09.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:19:09.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:19:09.020 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:09.021 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:09.021 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:09.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.021 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:09.021 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:09.021 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.449977718, 2024-09-17T21:19:09.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:19:09.021 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:09.021 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:09.021 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:09.021 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:09.022 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:09.022 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:09.022 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:19:09.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.022 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:09.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:09.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:09.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:09.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:09.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:09.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:09.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:09.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:19:09.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:19:09.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:09.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:19:09.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:19:09.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:09.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:19:09.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:19:09.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:09.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:19:09.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:19:09.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:09.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:19:09.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:19:09.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:09.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:19:09.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:19:09.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:09.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:19:09.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:19:09.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:09.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:19:09.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:19:09.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:09.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:09.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:09.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:19:09.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:19:09.026 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:09.026 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:09.026 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:09.026 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:09.026 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:09.026 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:09.027 INFO:teuthology.orchestra.run.smithi184.stdout:2149 punch from 0x6e6e000 to 0x6e77000, (0x9000 bytes) 2024-09-17T21:19:09.028 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:19:09.029 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:09.024+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:09.056 INFO:teuthology.orchestra.run.smithi184.stdout:2150 write 0xc784400 thru 0xc78938a (0x4f8b bytes) 2024-09-17T21:19:09.059 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:09.059 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:09.062 INFO:teuthology.orchestra.run.smithi184.stdout:2155 trunc from 0xc789400 to 0xbb43200 2024-09-17T21:19:09.076 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:09.076 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:09.091 INFO:teuthology.orchestra.run.smithi184.stdout:2158 read 0x3f6a600 thru 0x3f6e940 (0x4341 bytes) 2024-09-17T21:19:09.092 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:09.094 INFO:teuthology.orchestra.run.smithi184.stdout:2159 read 0x565ca00 thru 0x5662be3 (0x61e4 bytes) 2024-09-17T21:19:09.095 INFO:teuthology.orchestra.run.smithi184.stdout:2160 read 0x7c74600 thru 0x7c7591a (0x131b bytes) 2024-09-17T21:19:09.095 INFO:teuthology.orchestra.run.smithi184.stdout:2162 read 0x524ce00 thru 0x52575f0 (0xa7f1 bytes) 2024-09-17T21:19:09.110 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:09.110 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:09.115 INFO:teuthology.orchestra.run.smithi184.stdout:2163 write 0xd052400 thru 0xd057041 (0x4c42 bytes) 2024-09-17T21:19:09.127 INFO:teuthology.orchestra.run.smithi184.stdout:2164 trunc from 0xd057200 to 0x99af000 2024-09-17T21:19:09.128 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:09.124+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:09.129 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:09.129 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:09.147 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:09.148 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:09.157 INFO:teuthology.orchestra.run.smithi184.stdout:2165 trunc from 0x99af000 to 0xca31800 2024-09-17T21:19:09.170 INFO:teuthology.orchestra.run.smithi184.stdout:2167 read 0x2500000 thru 0x250593f (0x5940 bytes) 2024-09-17T21:19:09.173 INFO:teuthology.orchestra.run.smithi184.stdout:2170 clone 15 order 19 su 262144 sc 2 2024-09-17T21:19:09.214 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:09.214 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:09.228 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:19:09.229 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:09.227+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:09.330 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:09.327+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:09.395 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:09.396 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:09.409 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:19:09.421 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:19:09.421 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-17T21:19:09.445 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:19:09.469 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:09.467+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:09.531 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:09.528+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:09.543 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:09.543 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:09.559 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:19:09.568 INFO:teuthology.orchestra.run.smithi184.stdout:leaving image image_client.0-clone14 intact 2024-09-17T21:19:09.646 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:09.644+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:09.721 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:09.721 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:09.733 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:09.731+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:09.735 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:19:09.834 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:09.832+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:09.906 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:09.906 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:09.921 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:19:09.962 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:09.960+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:10.036 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:10.035+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:10.088 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:10.088 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:10.103 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:19:10.118 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-17T21:19:10.137 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:10.132+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:10.234 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:10.234 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:10.245 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:10.236+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:10.248 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:19:10.339 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:10.335+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:10.371 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:10.371 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:10.385 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:19:10.440 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-17T21:19:10.504 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:10.436+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:10.504 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T21:19:10.505 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-17T21:19:10.541 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:10.536+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:10.558 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:10.558 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:10.575 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:19:10.642 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:10.640+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:10.743 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:10.744 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:10.744 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:10.740+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:10.759 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:19:10.844 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:10.843+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:10.880 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:10.880 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:10.896 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:19:10.945 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:10.940+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:11.046 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:11.043+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:11.058 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:11.059 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:11.073 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:19:11.147 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:11.144+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:11.165 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:19:11.248 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:11.244+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:11.251 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:11.251 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:11.265 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:19:11.313 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:11.314 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:11.331 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:11.332 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:11.349 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:11.344+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:11.351 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:11.352 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:11.373 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:11.373 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:11.392 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:11.392 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:11.392 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:11.392 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:11.392 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:11.395 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:11.395 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:11.405 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:19:11.419 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:11.419 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:11.450 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.2 to 1 (10000) 2024-09-17T21:19:11.450 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:11.447+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:11.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-17T21:19:11.527 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:12 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:19:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:19:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 58.192813266999998, 2024-09-17T21:19:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061340989999999996, 2024-09-17T21:19:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:11.529 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:11.529 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:11.529 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:19:11.529 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:11.529 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12 2024-09-17T21:19:11.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.529 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:11.529 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.529 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:11.529 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:19:11.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.530 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:11.530 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:19:11.530 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.530 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:11.530 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332336+0000", 2024-09-17T21:19:11.530 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5519999999999999e-06 2024-09-17T21:19:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332342+0000", 2024-09-17T21:19:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7180000000000001e-06 2024-09-17T21:19:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332352+0000", 2024-09-17T21:19:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:19:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332359+0000", 2024-09-17T21:19:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3479999999999998e-06 2024-09-17T21:19:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332413+0000", 2024-09-17T21:19:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4447999999999998e-05 2024-09-17T21:19:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-17T21:19:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332417+0000", 2024-09-17T21:19:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.895e-06 2024-09-17T21:19:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334755+0000", 2024-09-17T21:19:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023377189999999998 2024-09-17T21:19:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334903+0000", 2024-09-17T21:19:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014778999999999999 2024-09-17T21:19:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:19:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.335187+0000", 2024-09-17T21:19:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000284715 2024-09-17T21:19:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.337731+0000", 2024-09-17T21:19:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002543741 2024-09-17T21:19:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338382+0000", 2024-09-17T21:19:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00065029200000000001 2024-09-17T21:19:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338418+0000", 2024-09-17T21:19:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.697e-05 2024-09-17T21:19:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338469+0000", 2024-09-17T21:19:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0154999999999997e-05 2024-09-17T21:19:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "pg_update_log_missing(3.1e epoch 28/27 rep_tid 81 entries 28'41 (0'0) error 3:7b23fd98:::rbd_header.1152c01721e5:head by client.4488.0:561 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-17T21:19:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:19:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 54.243539140000003, 2024-09-17T21:19:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0051146969999999996, 2024-09-17T21:19:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:19:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:19:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281609+0000", 2024-09-17T21:19:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281610+0000", 2024-09-17T21:19:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.16e-07 2024-09-17T21:19:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281626+0000", 2024-09-17T21:19:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6039999999999999e-05 2024-09-17T21:19:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281666+0000", 2024-09-17T21:19:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0364000000000002e-05 2024-09-17T21:19:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281710+0000", 2024-09-17T21:19:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3596000000000001e-05 2024-09-17T21:19:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281992+0000", 2024-09-17T21:19:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028273099999999999 2024-09-17T21:19:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286723+0000", 2024-09-17T21:19:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047310499999999997 2024-09-17T21:19:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4488.0:562 3.1e e28/27 3:7b23fd98:::rbd_header.1152c01721e5:head v 28'42, mlcod=0'0)", 2024-09-17T21:19:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:19:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 54.243295351999997, 2024-09-17T21:19:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049275020000000003, 2024-09-17T21:19:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:19:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281852+0000", 2024-09-17T21:19:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281853+0000", 2024-09-17T21:19:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2999999999999999e-07 2024-09-17T21:19:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281875+0000", 2024-09-17T21:19:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1386000000000002e-05 2024-09-17T21:19:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281892+0000", 2024-09-17T21:19:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7637000000000001e-05 2024-09-17T21:19:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.281908+0000", 2024-09-17T21:19:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5187e-05 2024-09-17T21:19:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.282517+0000", 2024-09-17T21:19:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00060989799999999999 2024-09-17T21:19:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.282548+0000", 2024-09-17T21:19:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0910000000000001e-05 2024-09-17T21:19:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286729+0000", 2024-09-17T21:19:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0041810850000000002 2024-09-17T21:19:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:17.286780+0000", 2024-09-17T21:19:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0569000000000003e-05 2024-09-17T21:19:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:609 3.d e28/23 3:b7278be9:::rbd_data.1152c6af092e.000000000000008f:head v 28'6, mlcod=28'4)", 2024-09-17T21:19:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:19:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 53.410351065, 2024-09-17T21:19:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049797280000000001, 2024-09-17T21:19:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:19:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:11.548 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114797+0000", 2024-09-17T21:19:11.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:11.548 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114800+0000", 2024-09-17T21:19:11.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:19:11.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:11.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114915+0000", 2024-09-17T21:19:11.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115053 2024-09-17T21:19:11.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:11.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114951+0000", 2024-09-17T21:19:11.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6165999999999999e-05 2024-09-17T21:19:11.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:11.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.114965+0000", 2024-09-17T21:19:11.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4364e-05 2024-09-17T21:19:11.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:11.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115097+0000", 2024-09-17T21:19:11.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013139200000000001 2024-09-17T21:19:11.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:11.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.115151+0000", 2024-09-17T21:19:11.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4422000000000002e-05 2024-09-17T21:19:11.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:11.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.119633+0000", 2024-09-17T21:19:11.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044818820000000004 2024-09-17T21:19:11.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:11.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.119776+0000", 2024-09-17T21:19:11.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000143609 2024-09-17T21:19:11.551 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.551 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:11.551 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.552 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:782 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'63, mlcod=28'60)", 2024-09-17T21:19:11.552 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:19:11.552 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 53.310383037000001, 2024-09-17T21:19:11.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006167283, 2024-09-17T21:19:11.552 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:11.552 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:11.552 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:11.552 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:11.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:11.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:19:11.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:11.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:19:11.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:11.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214767+0000", 2024-09-17T21:19:11.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:19:11.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:11.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214779+0000", 2024-09-17T21:19:11.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2627e-05 2024-09-17T21:19:11.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:11.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214805+0000", 2024-09-17T21:19:11.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5570000000000001e-05 2024-09-17T21:19:11.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:11.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214823+0000", 2024-09-17T21:19:11.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8202999999999998e-05 2024-09-17T21:19:11.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:11.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215170+0000", 2024-09-17T21:19:11.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034720399999999999 2024-09-17T21:19:11.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:11.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215235+0000", 2024-09-17T21:19:11.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4276000000000006e-05 2024-09-17T21:19:11.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:11.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220859+0000", 2024-09-17T21:19:11.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056240500000000002 2024-09-17T21:19:11.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:11.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220932+0000", 2024-09-17T21:19:11.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3503000000000004e-05 2024-09-17T21:19:11.557 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.557 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:11.557 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.557 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:783 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'64, mlcod=28'60)", 2024-09-17T21:19:11.557 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:19:11.557 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 53.309696195000001, 2024-09-17T21:19:11.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0055240869999999996, 2024-09-17T21:19:11.557 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:11.557 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:11.558 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:11.558 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:11.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:11.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:19:11.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:11.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215452+0000", 2024-09-17T21:19:11.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:11.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215454+0000", 2024-09-17T21:19:11.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0269999999999998e-06 2024-09-17T21:19:11.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:11.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215468+0000", 2024-09-17T21:19:11.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4816e-05 2024-09-17T21:19:11.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:11.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215501+0000", 2024-09-17T21:19:11.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2957000000000003e-05 2024-09-17T21:19:11.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.560 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:11.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215524+0000", 2024-09-17T21:19:11.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2782999999999999e-05 2024-09-17T21:19:11.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.560 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:11.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216675+0000", 2024-09-17T21:19:11.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011507869999999999 2024-09-17T21:19:11.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:11.561 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216728+0000", 2024-09-17T21:19:11.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3112999999999998e-05 2024-09-17T21:19:11.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:11.561 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220938+0000", 2024-09-17T21:19:11.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042097380000000002 2024-09-17T21:19:11.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:11.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220976+0000", 2024-09-17T21:19:11.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7866e-05 2024-09-17T21:19:11.562 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.562 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:11.562 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.562 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:785 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'65, mlcod=28'60)", 2024-09-17T21:19:11.562 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:19:11.562 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 53.309110146000002, 2024-09-17T21:19:11.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049795409999999997, 2024-09-17T21:19:11.563 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:11.563 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:11.563 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:11.563 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:11.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:11.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:19:11.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:11.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216038+0000", 2024-09-17T21:19:11.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:11.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216039+0000", 2024-09-17T21:19:11.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.793e-06 2024-09-17T21:19:11.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:11.565 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216053+0000", 2024-09-17T21:19:11.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.345e-05 2024-09-17T21:19:11.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.565 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:11.565 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216092+0000", 2024-09-17T21:19:11.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9180000000000001e-05 2024-09-17T21:19:11.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.565 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:11.565 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216106+0000", 2024-09-17T21:19:11.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:19:11.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:11.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217569+0000", 2024-09-17T21:19:11.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001462938 2024-09-17T21:19:11.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:11.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217607+0000", 2024-09-17T21:19:11.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7267000000000001e-05 2024-09-17T21:19:11.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.567 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:11.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220980+0000", 2024-09-17T21:19:11.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003373544 2024-09-17T21:19:11.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.567 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:11.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221017+0000", 2024-09-17T21:19:11.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7116000000000002e-05 2024-09-17T21:19:11.567 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.567 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:11.567 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.568 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.568 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:786 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'66, mlcod=28'60)", 2024-09-17T21:19:11.568 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:19:11.568 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 53.308409341999997, 2024-09-17T21:19:11.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057966650000000003, 2024-09-17T21:19:11.568 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:11.568 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:11.568 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:11.568 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:11.568 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.568 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:11.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:19:11.569 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.569 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.569 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:11.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216738+0000", 2024-09-17T21:19:11.569 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.569 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.569 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:11.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216740+0000", 2024-09-17T21:19:11.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5600000000000001e-06 2024-09-17T21:19:11.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.570 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:11.570 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216752+0000", 2024-09-17T21:19:11.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1477e-05 2024-09-17T21:19:11.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.570 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:11.570 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216779+0000", 2024-09-17T21:19:11.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7093e-05 2024-09-17T21:19:11.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.571 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.571 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:11.571 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.216817+0000", 2024-09-17T21:19:11.571 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8275999999999997e-05 2024-09-17T21:19:11.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.571 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.571 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:11.571 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218387+0000", 2024-09-17T21:19:11.571 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015703749999999999 2024-09-17T21:19:11.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.572 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:11.572 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218432+0000", 2024-09-17T21:19:11.572 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4910000000000002e-05 2024-09-17T21:19:11.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.572 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:11.572 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222432+0000", 2024-09-17T21:19:11.572 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039997189999999997 2024-09-17T21:19:11.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.573 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:11.573 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222535+0000", 2024-09-17T21:19:11.573 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103255 2024-09-17T21:19:11.573 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.573 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:11.573 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.573 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.573 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:19:11.573 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:11.573 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 53.307787509999997, 2024-09-17T21:19:11.573 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:19:11.574 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:11.574 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:11.574 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:11.574 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:11.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.574 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:11.574 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:11.574 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.574 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:11.575 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:11.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.575 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.575 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:11.575 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:19:11.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:19:11.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.575 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.575 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:11.575 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:19:11.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:19:11.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.576 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.576 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:11.576 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:19:11.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:19:11.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.576 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.576 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:11.576 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:19:11.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:19:11.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.577 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:11.577 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:19:11.577 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:19:11.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.577 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:11.577 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:19:11.577 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:19:11.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.578 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:11.578 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:19:11.578 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:19:11.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.578 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.578 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:11.578 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:19:11.578 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:19:11.578 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.578 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:11.578 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.579 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:788 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'68, mlcod=28'60)", 2024-09-17T21:19:11.579 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:19:11.579 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 53.307153077999999, 2024-09-17T21:19:11.579 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058495320000000002, 2024-09-17T21:19:11.579 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:11.579 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:11.579 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:11.579 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:11.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.579 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:11.579 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:19:11.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.580 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.580 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:11.580 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217995+0000", 2024-09-17T21:19:11.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.580 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.580 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:11.580 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217996+0000", 2024-09-17T21:19:11.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.376e-06 2024-09-17T21:19:11.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.581 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:11.581 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218006+0000", 2024-09-17T21:19:11.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.01e-05 2024-09-17T21:19:11.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.581 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:11.581 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218030+0000", 2024-09-17T21:19:11.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4026999999999999e-05 2024-09-17T21:19:11.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.582 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:11.582 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.218042+0000", 2024-09-17T21:19:11.582 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1977000000000001e-05 2024-09-17T21:19:11.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.582 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.582 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:11.582 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221081+0000", 2024-09-17T21:19:11.582 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030385960000000002 2024-09-17T21:19:11.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.582 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.582 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:11.582 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.221115+0000", 2024-09-17T21:19:11.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3772999999999997e-05 2024-09-17T21:19:11.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.583 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:11.583 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223749+0000", 2024-09-17T21:19:11.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0026340629999999999 2024-09-17T21:19:11.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.583 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:11.583 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223844+0000", 2024-09-17T21:19:11.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5619999999999996e-05 2024-09-17T21:19:11.584 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.584 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:11.584 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.584 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.584 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:790 3.5 e28/23 3:a14b1a37:::rbd_data.1152c6af092e.000000000000002f:head v 28'1, mlcod=0'0)", 2024-09-17T21:19:11.584 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:19:11.584 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 53.302748751999999, 2024-09-17T21:19:11.584 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059622240000000003, 2024-09-17T21:19:11.584 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:11.584 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:11.584 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:11.585 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:11.585 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.585 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:11.585 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:19:11.585 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.585 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.585 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:11.585 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:19:11.585 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.586 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.586 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:11.586 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222400+0000", 2024-09-17T21:19:11.586 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.02e-06 2024-09-17T21:19:11.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.586 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.586 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:11.586 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222411+0000", 2024-09-17T21:19:11.586 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.044e-05 2024-09-17T21:19:11.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.586 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.586 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:11.587 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222440+0000", 2024-09-17T21:19:11.587 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.932e-05 2024-09-17T21:19:11.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.587 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:11.587 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222458+0000", 2024-09-17T21:19:11.587 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8615999999999999e-05 2024-09-17T21:19:11.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.587 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:11.587 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222609+0000", 2024-09-17T21:19:11.587 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015102299999999999 2024-09-17T21:19:11.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.588 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:11.588 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222657+0000", 2024-09-17T21:19:11.588 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7206000000000002e-05 2024-09-17T21:19:11.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.588 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:11.588 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228278+0000", 2024-09-17T21:19:11.588 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056208100000000004 2024-09-17T21:19:11.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.589 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:11.589 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228361+0000", 2024-09-17T21:19:11.589 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3788999999999993e-05 2024-09-17T21:19:11.589 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.589 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:11.589 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.589 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.589 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:884 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'76, mlcod=28'70)", 2024-09-17T21:19:11.589 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:19:11.589 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 53.256163870999998, 2024-09-17T21:19:11.590 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056451990000000001, 2024-09-17T21:19:11.590 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:11.590 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:11.590 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:11.590 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:11.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:11.590 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:19:11.590 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:11.591 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268984+0000", 2024-09-17T21:19:11.591 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.591 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:11.591 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.268986+0000", 2024-09-17T21:19:11.591 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9360000000000002e-06 2024-09-17T21:19:11.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.591 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:11.591 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269001+0000", 2024-09-17T21:19:11.591 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4684e-05 2024-09-17T21:19:11.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.592 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:11.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269026+0000", 2024-09-17T21:19:11.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5193000000000001e-05 2024-09-17T21:19:11.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.592 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:11.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.269043+0000", 2024-09-17T21:19:11.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7306e-05 2024-09-17T21:19:11.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.593 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:11.593 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.270024+0000", 2024-09-17T21:19:11.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00098120600000000009 2024-09-17T21:19:11.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.593 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:11.593 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.270085+0000", 2024-09-17T21:19:11.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0486000000000003e-05 2024-09-17T21:19:11.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.594 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:11.594 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.274557+0000", 2024-09-17T21:19:11.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044723419999999998 2024-09-17T21:19:11.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.594 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:11.594 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.274629+0000", 2024-09-17T21:19:11.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2045999999999994e-05 2024-09-17T21:19:11.594 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.594 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:11.594 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.595 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:19:11.595 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:11.595 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 48.657486845999998, 2024-09-17T21:19:11.595 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:19:11.595 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:11.595 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:11.595 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:11.595 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:11.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.595 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:11.595 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:11.595 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.596 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:11.596 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:11.596 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.596 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:11.596 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:19:11.596 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:19:11.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:11.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:19:11.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:19:11.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:11.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:19:11.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:19:11.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:11.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:19:11.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:19:11.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:11.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:19:11.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:19:11.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:11.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:19:11.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:19:11.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:11.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:19:11.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:19:11.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:11.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:19:11.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:19:11.599 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.599 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:11.599 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.600 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:19:11.600 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:11.600 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 48.656221762999998, 2024-09-17T21:19:11.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:19:11.600 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:11.600 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:11.600 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:11.600 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:11.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:11.601 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:11.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:11.601 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:11.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:11.601 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:19:11.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:19:11.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.602 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:11.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:19:11.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:19:11.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.602 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:11.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:19:11.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:19:11.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:11.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:19:11.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:19:11.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:11.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:19:11.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:19:11.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:11.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:19:11.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:19:11.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:11.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:19:11.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:19:11.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:11.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:19:11.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:19:11.605 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.605 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:11.605 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.605 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:19:11.605 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:11.605 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 48.646442352999998, 2024-09-17T21:19:11.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:19:11.605 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:11.605 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:11.606 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:11.606 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:11.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:11.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:11.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:11.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:11.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:11.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:19:11.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:19:11.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:11.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:19:11.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:19:11.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:11.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:19:11.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:19:11.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:11.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:19:11.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:19:11.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:11.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:19:11.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:19:11.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.609 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:11.609 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:19:11.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:19:11.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.609 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:11.609 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:19:11.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:19:11.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:11.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:19:11.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:19:11.610 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.610 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:11.610 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.610 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:19:11.610 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:11.610 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 48.637719949999997, 2024-09-17T21:19:11.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:19:11.611 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:11.611 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:11.611 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:11.611 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:11.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:11.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:11.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:11.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:11.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:11.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:19:11.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:19:11.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:11.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:19:11.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:19:11.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:11.613 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:19:11.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:19:11.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:11.613 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:19:11.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:19:11.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:11.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:19:11.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:19:11.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:11.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:19:11.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:19:11.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:11.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:19:11.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:19:11.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:11.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:19:11.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:19:11.615 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.615 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:11.615 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.616 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4498.0:596 3.4 e32/23 3:20f262ea:::rbd_data.1152b51cb396.000000000000000d:head v 32'17, mlcod=30'15)", 2024-09-17T21:19:11.616 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.258354+0000", 2024-09-17T21:19:11.616 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 45.266794085999997, 2024-09-17T21:19:11.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0053727619999999997, 2024-09-17T21:19:11.616 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:11.616 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:11.616 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:11.616 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:11.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:11.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.258354+0000", 2024-09-17T21:19:11.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:11.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.258354+0000", 2024-09-17T21:19:11.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:11.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.258356+0000", 2024-09-17T21:19:11.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.798e-06 2024-09-17T21:19:11.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:11.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259198+0000", 2024-09-17T21:19:11.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084279199999999998 2024-09-17T21:19:11.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:11.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259229+0000", 2024-09-17T21:19:11.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0383999999999999e-05 2024-09-17T21:19:11.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:11.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259241+0000", 2024-09-17T21:19:11.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2139e-05 2024-09-17T21:19:11.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:11.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259341+0000", 2024-09-17T21:19:11.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010049 2024-09-17T21:19:11.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:11.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.259380+0000", 2024-09-17T21:19:11.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9072000000000001e-05 2024-09-17T21:19:11.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:11.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.263668+0000", 2024-09-17T21:19:11.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042875389999999999 2024-09-17T21:19:11.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:11.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.263726+0000", 2024-09-17T21:19:11.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8548000000000003e-05 2024-09-17T21:19:11.621 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.621 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:11.621 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.621 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4520.0:6177 3.8 e34/27 3:1665de58:::rbd_data.11528c3b0529.0000000000000019:head v 34'43, mlcod=34'41)", 2024-09-17T21:19:11.621 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:30.509962+0000", 2024-09-17T21:19:11.621 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.015186096000001, 2024-09-17T21:19:11.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049320850000000001, 2024-09-17T21:19:11.622 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:11.622 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:11.622 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:11.622 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:11.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:11.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509962+0000", 2024-09-17T21:19:11.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:11.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509962+0000", 2024-09-17T21:19:11.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:11.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509964+0000", 2024-09-17T21:19:11.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.356e-06 2024-09-17T21:19:11.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:11.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.509978+0000", 2024-09-17T21:19:11.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3434e-05 2024-09-17T21:19:11.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:11.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510001+0000", 2024-09-17T21:19:11.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3436000000000001e-05 2024-09-17T21:19:11.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:11.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510023+0000", 2024-09-17T21:19:11.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1739999999999999e-05 2024-09-17T21:19:11.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:11.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510193+0000", 2024-09-17T21:19:11.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017040600000000001 2024-09-17T21:19:11.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:11.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.510250+0000", 2024-09-17T21:19:11.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6749000000000002e-05 2024-09-17T21:19:11.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:11.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.514823+0000", 2024-09-17T21:19:11.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0045728180000000002 2024-09-17T21:19:11.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:11.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:30.514894+0000", 2024-09-17T21:19:11.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1145999999999999e-05 2024-09-17T21:19:11.626 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.626 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:11.626 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.626 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:19:11.627 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:11.627 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 35.252629386000002, 2024-09-17T21:19:11.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:19:11.627 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:11.627 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:11.627 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:11.627 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:11.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:11.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:11.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:11.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:11.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:11.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:19:11.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:19:11.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:11.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:19:11.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:19:11.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:11.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:19:11.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:19:11.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:11.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:19:11.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:19:11.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:11.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:19:11.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:19:11.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:11.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:19:11.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:19:11.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:11.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:19:11.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:19:11.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:11.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:19:11.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:19:11.631 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.631 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:11.631 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.632 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4863.0:165 3.5 e52/23 3:a772dfb1:::rbd_data.11523651b015.0000000000000001:head v 52'103, mlcod=47'101)", 2024-09-17T21:19:11.632 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:50.682666+0000", 2024-09-17T21:19:11.632 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 20.842481580000001, 2024-09-17T21:19:11.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0055448839999999999, 2024-09-17T21:19:11.632 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:11.632 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:11.632 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:11.632 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:11.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:11.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.682666+0000", 2024-09-17T21:19:11.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:11.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.682666+0000", 2024-09-17T21:19:11.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:11.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:11.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.682667+0000", 2024-09-17T21:19:11.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.16e-07 2024-09-17T21:19:11.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:11.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.683332+0000", 2024-09-17T21:19:11.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000664476 2024-09-17T21:19:11.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:11.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.683360+0000", 2024-09-17T21:19:11.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7988e-05 2024-09-17T21:19:11.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:11.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.683376+0000", 2024-09-17T21:19:11.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6578999999999998e-05 2024-09-17T21:19:11.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:11.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.683522+0000", 2024-09-17T21:19:11.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014536400000000001 2024-09-17T21:19:11.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:11.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.683560+0000", 2024-09-17T21:19:11.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8005e-05 2024-09-17T21:19:11.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:11.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.688080+0000", 2024-09-17T21:19:11.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0045201390000000003 2024-09-17T21:19:11.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:11.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:11.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:11.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:50.688211+0000", 2024-09-17T21:19:11.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013141699999999999 2024-09-17T21:19:11.637 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.637 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:11.637 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.637 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:11.637 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:11.637 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:11.639 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:19:11.639 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:11.635+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:11.740 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:11.736+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:11.769 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:11.769 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:19:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:11.783 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:19:11.807 INFO:teuthology.orchestra.run.smithi184.stdout:2172 read 0x39e6e00 thru 0x39f34d5 (0xc6d6 bytes) 2024-09-17T21:19:11.870 INFO:teuthology.orchestra.run.smithi184.stdout:2174 trunc from 0xca31800 to 0x15b7200 2024-09-17T21:19:11.871 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:11.868+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:11.920 INFO:teuthology.orchestra.run.smithi184.stdout:2175 trunc from 0x15b7200 to 0x7f56400 2024-09-17T21:19:11.943 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:11.939+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:11.945 INFO:teuthology.orchestra.run.smithi184.stdout:2176 write 0x605e800 thru 0x605e88c (0x8d bytes) 2024-09-17T21:19:11.946 INFO:teuthology.orchestra.run.smithi184.stdout:2177 read 0x7412c00 thru 0x7419c16 (0x7017 bytes) 2024-09-17T21:19:11.959 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:11.960 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:11.960 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:11.960 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:11.960 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:11.976 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:19:11.977 INFO:teuthology.orchestra.run.smithi184.stdout:2178 write 0x9fd0e00 thru 0x9fe04d9 (0xf6da bytes) 2024-09-17T21:19:11.980 INFO:teuthology.orchestra.run.smithi184.stdout:2180 punch from 0xc7d400 to 0xc88400, (0xb000 bytes) 2024-09-17T21:19:12.007 INFO:teuthology.orchestra.run.smithi184.stdout:2181 write 0xc296000 thru 0xc296b31 (0xb32 bytes) 2024-09-17T21:19:12.009 INFO:teuthology.orchestra.run.smithi184.stdout:2182 read 0x547ba00 thru 0x547bc9d (0x29e bytes) 2024-09-17T21:19:12.009 INFO:teuthology.orchestra.run.smithi184.stdout:2184 write 0x7651000 thru 0x765dab6 (0xcab7 bytes) 2024-09-17T21:19:12.009 INFO:teuthology.orchestra.run.smithi184.stdout:2185 read 0x5234000 thru 0x5234343 (0x344 bytes) 2024-09-17T21:19:12.010 INFO:teuthology.orchestra.run.smithi184.stdout:2188 read 0xbecfc00 thru 0xbed38fc (0x3cfd bytes) 2024-09-17T21:19:12.010 INFO:teuthology.orchestra.run.smithi184.stdout:2189 read 0x884000 thru 0x891b7a (0xdb7b bytes) 2024-09-17T21:19:12.011 INFO:teuthology.orchestra.run.smithi184.stdout:2191 write 0x73b3200 thru 0x73bf0e2 (0xbee3 bytes) 2024-09-17T21:19:12.011 INFO:teuthology.orchestra.run.smithi184.stdout:2194 read 0x91e7000 thru 0x91f3f7f (0xcf80 bytes) 2024-09-17T21:19:12.011 INFO:teuthology.orchestra.run.smithi184.stdout:2196 punch from 0xabfa400 to 0xac04800, (0xa400 bytes) 2024-09-17T21:19:12.012 INFO:teuthology.orchestra.run.smithi184.stdout:2198 read 0xa38e000 thru 0xa3916b9 (0x36ba bytes) 2024-09-17T21:19:12.012 INFO:teuthology.orchestra.run.smithi184.stdout:2200 write 0x9978e00 thru 0x997c741 (0x3942 bytes) 2024-09-17T21:19:12.012 INFO:teuthology.orchestra.run.smithi184.stdout:2202 write 0x19ae200 thru 0x19bd0f5 (0xeef6 bytes) 2024-09-17T21:19:12.013 INFO:teuthology.orchestra.run.smithi184.stdout:2203 read 0xb7e1e00 thru 0xb7e7a67 (0x5c68 bytes) 2024-09-17T21:19:12.013 INFO:teuthology.orchestra.run.smithi184.stdout:2204 write 0x75e5e00 thru 0x75f30cf (0xd2d0 bytes) 2024-09-17T21:19:12.013 INFO:teuthology.orchestra.run.smithi184.stdout:2206 read 0x1f21400 thru 0x1f2fbf5 (0xe7f6 bytes) 2024-09-17T21:19:12.020 INFO:teuthology.orchestra.run.smithi184.stdout:2209 trunc from 0xc296c00 to 0x8d0ae00 2024-09-17T21:19:12.054 INFO:teuthology.orchestra.run.smithi184.stdout:2210 read 0x7f98600 thru 0x7f9dec2 (0x58c3 bytes) 2024-09-17T21:19:12.062 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:12.060+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:12.064 INFO:teuthology.orchestra.run.smithi184.stdout:2212 trunc from 0x8d0ae00 to 0x4c8cc00 2024-09-17T21:19:12.094 INFO:teuthology.orchestra.run.smithi184.stdout:2214 write 0x4640800 thru 0x4641343 (0xb44 bytes) 2024-09-17T21:19:12.097 INFO:teuthology.orchestra.run.smithi184.stdout:2215 write 0x35c1e00 thru 0x35cacff (0x8f00 bytes) 2024-09-17T21:19:12.097 INFO:teuthology.orchestra.run.smithi184.stdout:2220 read 0x1b8d000 thru 0x1b92e86 (0x5e87 bytes) 2024-09-17T21:19:12.098 INFO:teuthology.orchestra.run.smithi184.stdout:2221 write 0x416d600 thru 0x41757fc (0x81fd bytes) 2024-09-17T21:19:12.098 INFO:teuthology.orchestra.run.smithi184.stdout:2223 read 0x3f35c00 thru 0x3f36079 (0x47a bytes) 2024-09-17T21:19:12.098 INFO:teuthology.orchestra.run.smithi184.stdout:2224 punch from 0x2283c00 to 0x228d800, (0x9c00 bytes) 2024-09-17T21:19:12.098 INFO:teuthology.orchestra.run.smithi184.stdout:2228 read 0x3a64a00 thru 0x3a6fbe8 (0xb1e9 bytes) 2024-09-17T21:19:12.099 INFO:teuthology.orchestra.run.smithi184.stdout:2229 read 0x2057000 thru 0x2058d2d (0x1d2e bytes) 2024-09-17T21:19:12.141 INFO:teuthology.orchestra.run.smithi184.stdout:2230 write 0xed3f200 thru 0xed4a2f9 (0xb0fa bytes) 2024-09-17T21:19:12.144 INFO:teuthology.orchestra.run.smithi184.stdout:2231 write 0x85e1000 thru 0x85f07cc (0xf7cd bytes) 2024-09-17T21:19:12.144 INFO:teuthology.orchestra.run.smithi184.stdout:2232 punch from 0x9e4b600 to 0x9e58400, (0xce00 bytes) 2024-09-17T21:19:12.144 INFO:teuthology.orchestra.run.smithi184.stdout:2234 read 0x246d200 thru 0x247cf2b (0xfd2c bytes) 2024-09-17T21:19:12.145 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:12.140+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:12.145 INFO:teuthology.orchestra.run.smithi184.stdout:2236 write 0xe1d1c00 thru 0xe1dcf38 (0xb339 bytes) 2024-09-17T21:19:12.145 INFO:teuthology.orchestra.run.smithi184.stdout:2237 write 0x9829000 thru 0x9836104 (0xd105 bytes) 2024-09-17T21:19:12.145 INFO:teuthology.orchestra.run.smithi184.stdout:2239 read 0x5c5800 thru 0x5c9f4e (0x474f bytes) 2024-09-17T21:19:12.146 INFO:teuthology.orchestra.run.smithi184.stdout:2241 write 0x5240000 thru 0x5240ef9 (0xefa bytes) 2024-09-17T21:19:12.146 INFO:teuthology.orchestra.run.smithi184.stdout:2243 write 0x3d10a00 thru 0x3d1d1e8 (0xc7e9 bytes) 2024-09-17T21:19:12.147 INFO:teuthology.orchestra.run.smithi184.stdout:2245 trunc from 0xed4a400 to 0xe0f1000 2024-09-17T21:19:12.161 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:12.161 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:12.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:12.161 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:12.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.161 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:19:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 75.144386166000004, 2024-09-17T21:19:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:19:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:19:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:19:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:19:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:19:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:19:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:19:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:19:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:19:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:19:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:19:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:19:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:19:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:19:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:19:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:19:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:19:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:19:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:19:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:19:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:19:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:19:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:19:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:19:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:19:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:19:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:19:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:19:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:19:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:19:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:19:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.564221885999999, 2024-09-17T21:19:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:19:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:19:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:19:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:19:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:19:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:19:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:19:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:19:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:19:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:19:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:19:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:19:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:19:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:19:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:19:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:19:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:19:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:19:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:19:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:19:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:19:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:19:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:19:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:19:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:19:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:19:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.564124573999997, 2024-09-17T21:19:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:19:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:19:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:19:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:19:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:19:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:19:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:19:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:19:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:19:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:19:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:19:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:19:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:19:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:19:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:19:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:19:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:19:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:19:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:19:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:19:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:19:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:19:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:19:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:19:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:19:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:19:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:19:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:19:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:19:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.562037773, 2024-09-17T21:19:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:19:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:19:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:19:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:19:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:19:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:19:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:19:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:19:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:19:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:19:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:19:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:19:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:19:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:19:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:19:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:19:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:19:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:19:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:19:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:19:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:19:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:19:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:19:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:19:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:19:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:19:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:19:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:19:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:19:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:19:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:19:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:19:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:19:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:19:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:19:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:282 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:19:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.930332512, 2024-09-17T21:19:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023571801, 2024-09-17T21:19:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 282 2024-09-17T21:19:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:19:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:19:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227820+0000", 2024-09-17T21:19:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.11e-06 2024-09-17T21:19:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227825+0000", 2024-09-17T21:19:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8099999999999997e-06 2024-09-17T21:19:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227848+0000", 2024-09-17T21:19:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2949999999999999e-05 2024-09-17T21:19:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227862+0000", 2024-09-17T21:19:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:19:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.247977+0000", 2024-09-17T21:19:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020114968 2024-09-17T21:19:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251141+0000", 2024-09-17T21:19:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031640689999999998 2024-09-17T21:19:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251391+0000", 2024-09-17T21:19:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025018499999999999 2024-09-17T21:19:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:283 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:19:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.930270673000003, 2024-09-17T21:19:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024022274999999999, 2024-09-17T21:19:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 283 2024-09-17T21:19:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:19:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:19:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227882+0000", 2024-09-17T21:19:12.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-06 2024-09-17T21:19:12.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227887+0000", 2024-09-17T21:19:12.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1769999999999997e-06 2024-09-17T21:19:12.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227905+0000", 2024-09-17T21:19:12.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7235999999999999e-05 2024-09-17T21:19:12.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227918+0000", 2024-09-17T21:19:12.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3387000000000001e-05 2024-09-17T21:19:12.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251469+0000", 2024-09-17T21:19:12.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023550894999999999 2024-09-17T21:19:12.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251669+0000", 2024-09-17T21:19:12.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019992499999999999 2024-09-17T21:19:12.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251903+0000", 2024-09-17T21:19:12.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000234412 2024-09-17T21:19:12.197 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.197 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.197 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.198 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:284 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.198 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:19:12.198 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.93021512, 2024-09-17T21:19:12.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024450084, 2024-09-17T21:19:12.198 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.198 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.198 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.198 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.198 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.198 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.198 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 284 2024-09-17T21:19:12.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.198 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:19:12.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:19:12.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227937+0000", 2024-09-17T21:19:12.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6700000000000002e-07 2024-09-17T21:19:12.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227943+0000", 2024-09-17T21:19:12.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:19:12.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227961+0000", 2024-09-17T21:19:12.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.827e-05 2024-09-17T21:19:12.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227975+0000", 2024-09-17T21:19:12.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.394e-05 2024-09-17T21:19:12.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251977+0000", 2024-09-17T21:19:12.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024001340999999999 2024-09-17T21:19:12.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252169+0000", 2024-09-17T21:19:12.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019271199999999999 2024-09-17T21:19:12.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252387+0000", 2024-09-17T21:19:12.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021726100000000001 2024-09-17T21:19:12.202 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.202 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.202 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.202 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:285 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.202 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:19:12.202 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.930158830000003, 2024-09-17T21:19:12.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024884791, 2024-09-17T21:19:12.202 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.202 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.202 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.203 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.203 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.203 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.203 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 285 2024-09-17T21:19:12.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.203 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:19:12.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:19:12.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227994+0000", 2024-09-17T21:19:12.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1429999999999999e-06 2024-09-17T21:19:12.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227999+0000", 2024-09-17T21:19:12.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1669999999999998e-06 2024-09-17T21:19:12.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228015+0000", 2024-09-17T21:19:12.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6056e-05 2024-09-17T21:19:12.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228028+0000", 2024-09-17T21:19:12.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3196e-05 2024-09-17T21:19:12.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252458+0000", 2024-09-17T21:19:12.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024429751999999999 2024-09-17T21:19:12.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252650+0000", 2024-09-17T21:19:12.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019187800000000001 2024-09-17T21:19:12.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252878+0000", 2024-09-17T21:19:12.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022759899999999999 2024-09-17T21:19:12.206 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.206 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.206 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.207 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:286 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.207 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:19:12.207 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.930103070999998, 2024-09-17T21:19:12.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025299318000000001, 2024-09-17T21:19:12.207 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.207 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.207 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.207 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.207 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.207 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.207 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 286 2024-09-17T21:19:12.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.207 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:19:12.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:19:12.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228050+0000", 2024-09-17T21:19:12.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:19:12.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228055+0000", 2024-09-17T21:19:12.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4960000000000004e-06 2024-09-17T21:19:12.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228075+0000", 2024-09-17T21:19:12.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0109999999999999e-05 2024-09-17T21:19:12.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228089+0000", 2024-09-17T21:19:12.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4154e-05 2024-09-17T21:19:12.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252952+0000", 2024-09-17T21:19:12.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024862934 2024-09-17T21:19:12.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253141+0000", 2024-09-17T21:19:12.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000188782 2024-09-17T21:19:12.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253348+0000", 2024-09-17T21:19:12.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020684499999999999 2024-09-17T21:19:12.211 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.211 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.211 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.211 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:443 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.211 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:19:12.211 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.916515766000003, 2024-09-17T21:19:12.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023179569000000001, 2024-09-17T21:19:12.211 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.212 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.212 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.212 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.212 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.212 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.212 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 443 2024-09-17T21:19:12.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.212 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:19:12.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:19:12.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241637+0000", 2024-09-17T21:19:12.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.46e-06 2024-09-17T21:19:12.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241643+0000", 2024-09-17T21:19:12.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1970000000000001e-06 2024-09-17T21:19:12.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241662+0000", 2024-09-17T21:19:12.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8712999999999999e-05 2024-09-17T21:19:12.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241676+0000", 2024-09-17T21:19:12.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.393e-05 2024-09-17T21:19:12.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264596+0000", 2024-09-17T21:19:12.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022919688000000001 2024-09-17T21:19:12.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264690+0000", 2024-09-17T21:19:12.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4145000000000001e-05 2024-09-17T21:19:12.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264815+0000", 2024-09-17T21:19:12.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012543600000000001 2024-09-17T21:19:12.215 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.216 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.216 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.216 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:444 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.216 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:19:12.216 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.916454715999997, 2024-09-17T21:19:12.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023260383999999999, 2024-09-17T21:19:12.216 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.216 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.216 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.216 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.216 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.216 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.217 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 444 2024-09-17T21:19:12.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.217 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:19:12.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:19:12.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241698+0000", 2024-09-17T21:19:12.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2270000000000001e-06 2024-09-17T21:19:12.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241704+0000", 2024-09-17T21:19:12.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9800000000000003e-06 2024-09-17T21:19:12.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241725+0000", 2024-09-17T21:19:12.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0812999999999999e-05 2024-09-17T21:19:12.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241739+0000", 2024-09-17T21:19:12.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3773e-05 2024-09-17T21:19:12.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264850+0000", 2024-09-17T21:19:12.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023111326000000001 2024-09-17T21:19:12.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264897+0000", 2024-09-17T21:19:12.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6638999999999998e-05 2024-09-17T21:19:12.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264957+0000", 2024-09-17T21:19:12.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0625999999999999e-05 2024-09-17T21:19:12.220 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.220 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.220 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.220 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:445 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.221 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:19:12.221 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.916394046999997, 2024-09-17T21:19:12.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023315144999999999, 2024-09-17T21:19:12.221 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.221 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.221 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.221 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.221 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.221 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.221 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 445 2024-09-17T21:19:12.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.221 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:19:12.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:19:12.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241759+0000", 2024-09-17T21:19:12.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-06 2024-09-17T21:19:12.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241764+0000", 2024-09-17T21:19:12.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5899999999999998e-06 2024-09-17T21:19:12.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241784+0000", 2024-09-17T21:19:12.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9204000000000001e-05 2024-09-17T21:19:12.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241798+0000", 2024-09-17T21:19:12.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3953e-05 2024-09-17T21:19:12.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264977+0000", 2024-09-17T21:19:12.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023179182 2024-09-17T21:19:12.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.265018+0000", 2024-09-17T21:19:12.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1171999999999998e-05 2024-09-17T21:19:12.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.265073+0000", 2024-09-17T21:19:12.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4741000000000003e-05 2024-09-17T21:19:12.225 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.225 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.225 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.225 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:679 3.13 3:cd0d74f9:::rbd_data.11527824718.000000000000000a:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.225 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:19:12.225 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.907472626000001, 2024-09-17T21:19:12.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023284978000000001, 2024-09-17T21:19:12.225 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.226 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.226 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.226 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.226 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.226 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.226 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 679 2024-09-17T21:19:12.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.226 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:19:12.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:19:12.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250680+0000", 2024-09-17T21:19:12.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:19:12.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250685+0000", 2024-09-17T21:19:12.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1970000000000004e-06 2024-09-17T21:19:12.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250703+0000", 2024-09-17T21:19:12.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7396000000000001e-05 2024-09-17T21:19:12.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250716+0000", 2024-09-17T21:19:12.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.29e-05 2024-09-17T21:19:12.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273740+0000", 2024-09-17T21:19:12.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023024405000000001 2024-09-17T21:19:12.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273834+0000", 2024-09-17T21:19:12.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4124999999999998e-05 2024-09-17T21:19:12.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273964+0000", 2024-09-17T21:19:12.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012963499999999999 2024-09-17T21:19:12.229 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.229 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.230 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.230 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:680 3.13 3:cd0d74f9:::rbd_data.11527824718.000000000000000a:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.230 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:19:12.230 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.907417545999998, 2024-09-17T21:19:12.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023687792999999999, 2024-09-17T21:19:12.230 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.230 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.230 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.230 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.230 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.230 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.231 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 680 2024-09-17T21:19:12.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.231 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:19:12.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:19:12.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250735+0000", 2024-09-17T21:19:12.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2669999999999999e-06 2024-09-17T21:19:12.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250740+0000", 2024-09-17T21:19:12.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8060000000000002e-06 2024-09-17T21:19:12.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250757+0000", 2024-09-17T21:19:12.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7303e-05 2024-09-17T21:19:12.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250771+0000", 2024-09-17T21:19:12.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3876999999999999e-05 2024-09-17T21:19:12.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274022+0000", 2024-09-17T21:19:12.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023251135999999999 2024-09-17T21:19:12.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274211+0000", 2024-09-17T21:19:12.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018813599999999999 2024-09-17T21:19:12.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274422+0000", 2024-09-17T21:19:12.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000211268 2024-09-17T21:19:12.234 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.234 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.234 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.234 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:12.234 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:12.235 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.878138992, 2024-09-17T21:19:12.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:19:12.235 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.235 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.235 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.235 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.235 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:12.235 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:12.235 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:19:12.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.235 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:12.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:12.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:19:12.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:19:12.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:19:12.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:19:12.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:19:12.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:19:12.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:19:12.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:19:12.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:19:12.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:19:12.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:12.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:19:12.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:19:12.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:19:12.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:19:12.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:12.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:19:12.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:19:12.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:19:12.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:19:12.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:19:12.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:19:12.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:19:12.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:19:12.240 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.240 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.240 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.241 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:12.241 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:12.241 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.869047281, 2024-09-17T21:19:12.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:19:12.241 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.241 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.241 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.241 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.241 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:12.241 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:12.241 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:19:12.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.241 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:12.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:12.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:19:12.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:19:12.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:19:12.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:19:12.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:19:12.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:19:12.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:19:12.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:19:12.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:19:12.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:19:12.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:12.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:19:12.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:19:12.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:19:12.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:19:12.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:19:12.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:19:12.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:19:12.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:19:12.245 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.246 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.246 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.246 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:12.246 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:12.246 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.86887523, 2024-09-17T21:19:12.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:19:12.246 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.246 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.246 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.246 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.246 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:12.247 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:12.247 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:19:12.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.247 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:12.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:12.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:19:12.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:19:12.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:19:12.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:19:12.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:19:12.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:19:12.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:19:12.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:19:12.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:19:12.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:19:12.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:12.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:19:12.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:19:12.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:19:12.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:19:12.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:19:12.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:19:12.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:19:12.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:19:12.251 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.251 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.251 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.251 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:12.251 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:12.251 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.693942483000001, 2024-09-17T21:19:12.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:19:12.252 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.252 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.252 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.252 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.252 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:12.252 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:12.252 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:19:12.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.252 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:12.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:12.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:19:12.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:19:12.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:19:12.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:19:12.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:19:12.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:19:12.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:19:12.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:19:12.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:19:12.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:19:12.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:12.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:19:12.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:19:12.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:19:12.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:19:12.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:19:12.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:19:12.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:19:12.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:19:12.256 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.256 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.257 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.257 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:12.257 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:12.257 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.693857926, 2024-09-17T21:19:12.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:19:12.257 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.257 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.257 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.257 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.257 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:12.257 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:12.257 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:19:12.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.258 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:12.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:12.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:12.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:19:12.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:19:12.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:19:12.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:19:12.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:19:12.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:19:12.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:19:12.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:19:12.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:19:12.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:12.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:19:12.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:19:12.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:19:12.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:19:12.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:19:12.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:19:12.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:19:12.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:19:12.262 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.262 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.262 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.262 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:12.262 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:12.262 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.693792289000001, 2024-09-17T21:19:12.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:19:12.262 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.262 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.263 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.263 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.263 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:12.263 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:12.263 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:19:12.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.263 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:12.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:12.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:19:12.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:19:12.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:19:12.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:19:12.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:19:12.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:19:12.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:19:12.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:19:12.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:19:12.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:19:12.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:12.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:19:12.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:19:12.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:19:12.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:19:12.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:19:12.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:19:12.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:19:12.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:19:12.267 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.267 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.267 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.267 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.267 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.267 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:12.270 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:19:12.270 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:12.264+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:12.270 INFO:teuthology.orchestra.run.smithi184.stdout:2246 punch from 0xa269600 to 0xa272e00, (0x9800 bytes) 2024-09-17T21:19:12.271 INFO:teuthology.orchestra.run.smithi184.stdout:2248 read 0xb1db800 thru 0xb1ddba0 (0x23a1 bytes) 2024-09-17T21:19:12.271 INFO:teuthology.orchestra.run.smithi184.stdout:2250 punch from 0xc630200 to 0xc63e800, (0xe600 bytes) 2024-09-17T21:19:12.271 INFO:teuthology.orchestra.run.smithi184.stdout:2251 read 0x1d8e200 thru 0x1d9707d (0x8e7e bytes) 2024-09-17T21:19:12.271 INFO:teuthology.orchestra.run.smithi184.stdout:2252 read 0xabc8e00 thru 0xabc991c (0xb1d bytes) 2024-09-17T21:19:12.271 INFO:teuthology.orchestra.run.smithi184.stdout:2253 read 0x7165a00 thru 0x716d4b9 (0x7aba bytes) 2024-09-17T21:19:12.271 INFO:teuthology.orchestra.run.smithi184.stdout:2254 write 0xe260400 thru 0xe26a40b (0xa00c bytes) 2024-09-17T21:19:12.271 INFO:teuthology.orchestra.run.smithi184.stdout:2256 punch from 0xe122800 to 0xe12fe00, (0xd600 bytes) 2024-09-17T21:19:12.271 INFO:teuthology.orchestra.run.smithi184.stdout:2257 punch from 0x5c1a400 to 0x5c21600, (0x7200 bytes) 2024-09-17T21:19:12.271 INFO:teuthology.orchestra.run.smithi184.stdout:2258 trunc from 0xe26a600 to 0xe9fb800 2024-09-17T21:19:12.271 INFO:teuthology.orchestra.run.smithi184.stdout:2259 write 0x929e00 thru 0x938585 (0xe786 bytes) 2024-09-17T21:19:12.271 INFO:teuthology.orchestra.run.smithi184.stdout:2260 trunc from 0xe9fb800 to 0x8fcea00 2024-09-17T21:19:12.285 INFO:teuthology.orchestra.run.smithi184.stdout:2263 read 0x8178400 thru 0x817b624 (0x3225 bytes) 2024-09-17T21:19:12.286 INFO:teuthology.orchestra.run.smithi184.stdout:2264 read 0x3f17600 thru 0x3f195c7 (0x1fc8 bytes) 2024-09-17T21:19:12.287 INFO:teuthology.orchestra.run.smithi184.stdout:2265 read 0x4038400 thru 0x40464c5 (0xe0c6 bytes) 2024-09-17T21:19:12.287 INFO:teuthology.orchestra.run.smithi184.stdout:2269 read 0x7c69600 thru 0x7c6f92a (0x632b bytes) 2024-09-17T21:19:12.287 INFO:teuthology.orchestra.run.smithi184.stdout:2273 punch from 0x6385400 to 0x6391200, (0xbe00 bytes) 2024-09-17T21:19:12.287 INFO:teuthology.orchestra.run.smithi184.stdout:2274 read 0x8cc1a00 thru 0x8ccb70d (0x9d0e bytes) 2024-09-17T21:19:12.287 INFO:teuthology.orchestra.run.smithi184.stdout:2277 punch from 0x1220400 to 0x122be00, (0xba00 bytes) 2024-09-17T21:19:12.288 INFO:teuthology.orchestra.run.smithi184.stdout:2278 write 0x3715c00 thru 0x371c716 (0x6b17 bytes) 2024-09-17T21:19:12.288 INFO:teuthology.orchestra.run.smithi184.stdout:2279 write 0x73ca600 thru 0x73d8414 (0xde15 bytes) 2024-09-17T21:19:12.296 INFO:teuthology.orchestra.run.smithi184.stdout:2282 trunc from 0x8fcea00 to 0x5087a00 2024-09-17T21:19:12.334 INFO:teuthology.orchestra.run.smithi184.stdout:2285 punch from 0x1fb0600 to 0x1fb5800, (0x5200 bytes) 2024-09-17T21:19:12.346 INFO:teuthology.orchestra.run.smithi184.stdout:2286 trunc from 0x5087a00 to 0xbbb800 2024-09-17T21:19:12.369 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:12.364+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:12.399 INFO:teuthology.orchestra.run.smithi184.stdout:2287 trunc from 0xbbb800 to 0x582b400 2024-09-17T21:19:12.415 INFO:teuthology.orchestra.run.smithi184.stdout:2288 read 0x3066e00 thru 0x3070c0a (0x9e0b bytes) 2024-09-17T21:19:12.449 INFO:teuthology.orchestra.run.smithi184.stdout:2289 write 0xc739000 thru 0xc73eac6 (0x5ac7 bytes) 2024-09-17T21:19:12.452 INFO:teuthology.orchestra.run.smithi184.stdout:2291 trunc from 0xc73ec00 to 0xdbc4a00 2024-09-17T21:19:12.458 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:12.458 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:12.459 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:19:12.459 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:12.464 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T21:19:12.470 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:12.467+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:12.477 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:19:12.486 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-17T21:19:12.505 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:19:12.571 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:12.568+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:12.638 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:12.638 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:12.652 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:19:12.671 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:12.667+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:12.730 INFO:teuthology.orchestra.run.smithi184.stdout:2293 read 0x4cd0200 thru 0x4cd3d1f (0x3b20 bytes) 2024-09-17T21:19:12.734 INFO:teuthology.orchestra.run.smithi184.stdout:2294 trunc from 0xdbc4a00 to 0xc479800 2024-09-17T21:19:12.758 INFO:teuthology.orchestra.run.smithi184.stdout:2295 punch from 0x1d64400 to 0x1d72600, (0xe200 bytes) 2024-09-17T21:19:12.761 INFO:teuthology.orchestra.run.smithi184.stdout:2297 read 0xe20a00 thru 0xe29f40 (0x9541 bytes) 2024-09-17T21:19:12.761 INFO:teuthology.orchestra.run.smithi184.stdout:2299 read 0x81d5000 thru 0x81dbd72 (0x6d73 bytes) 2024-09-17T21:19:12.772 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:12.768+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:12.788 INFO:teuthology.orchestra.run.smithi184.stdout:2302 trunc from 0xc479800 to 0x14b2600 2024-09-17T21:19:12.817 INFO:teuthology.orchestra.run.smithi184.stdout:2303 read 0xe3e600 thru 0xe48f92 (0xa993 bytes) 2024-09-17T21:19:12.820 INFO:teuthology.orchestra.run.smithi184.stdout:2305 write 0x76800 thru 0x7de97 (0x7698 bytes) 2024-09-17T21:19:12.836 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:12.836 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:12.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:12.837 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:12.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.837 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:478 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 720896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.837 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:19:12.837 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.598973833000002, 2024-09-17T21:19:12.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016845300000000001, 2024-09-17T21:19:12.837 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.837 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.837 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.837 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.837 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.837 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.837 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 478 2024-09-17T21:19:12.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.838 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:19:12.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:19:12.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233785+0000", 2024-09-17T21:19:12.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.116e-06 2024-09-17T21:19:12.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233803+0000", 2024-09-17T21:19:12.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7932999999999999e-05 2024-09-17T21:19:12.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233822+0000", 2024-09-17T21:19:12.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9276999999999999e-05 2024-09-17T21:19:12.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233836+0000", 2024-09-17T21:19:12.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4053000000000001e-05 2024-09-17T21:19:12.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.248579+0000", 2024-09-17T21:19:12.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014743529 2024-09-17T21:19:12.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.248795+0000", 2024-09-17T21:19:12.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021535200000000001 2024-09-17T21:19:12.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250629+0000", 2024-09-17T21:19:12.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018340400000000001 2024-09-17T21:19:12.841 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.841 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.841 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.842 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:479 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 851968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.842 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:19:12.842 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.598882746999998, 2024-09-17T21:19:12.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017383900000000001, 2024-09-17T21:19:12.842 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.842 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.842 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.842 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.842 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.842 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.842 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 479 2024-09-17T21:19:12.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.843 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:19:12.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:19:12.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233876+0000", 2024-09-17T21:19:12.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2160000000000001e-06 2024-09-17T21:19:12.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233883+0000", 2024-09-17T21:19:12.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9199999999999998e-06 2024-09-17T21:19:12.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233900+0000", 2024-09-17T21:19:12.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7034e-05 2024-09-17T21:19:12.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233913+0000", 2024-09-17T21:19:12.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159000000000001e-05 2024-09-17T21:19:12.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250712+0000", 2024-09-17T21:19:12.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016798644000000001 2024-09-17T21:19:12.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250913+0000", 2024-09-17T21:19:12.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020127900000000001 2024-09-17T21:19:12.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251259+0000", 2024-09-17T21:19:12.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00034564799999999999 2024-09-17T21:19:12.846 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.846 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.846 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.847 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:480 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.847 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:19:12.847 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.598789902, 2024-09-17T21:19:12.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017902482000000001, 2024-09-17T21:19:12.847 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.847 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.847 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.847 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.847 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.847 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.847 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 480 2024-09-17T21:19:12.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.847 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:19:12.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:19:12.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233969+0000", 2024-09-17T21:19:12.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.454e-06 2024-09-17T21:19:12.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233981+0000", 2024-09-17T21:19:12.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1593000000000001e-05 2024-09-17T21:19:12.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233999+0000", 2024-09-17T21:19:12.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.806e-05 2024-09-17T21:19:12.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234012+0000", 2024-09-17T21:19:12.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:19:12.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251339+0000", 2024-09-17T21:19:12.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017326352 2024-09-17T21:19:12.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251534+0000", 2024-09-17T21:19:12.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000195842 2024-09-17T21:19:12.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251870+0000", 2024-09-17T21:19:12.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033551400000000003 2024-09-17T21:19:12.851 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.851 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.851 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.851 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:481 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1114112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.851 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:19:12.851 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.598721535000003, 2024-09-17T21:19:12.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018442621999999999, 2024-09-17T21:19:12.852 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.852 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.852 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.852 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.852 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.852 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.852 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 481 2024-09-17T21:19:12.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.852 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:19:12.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:19:12.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234037+0000", 2024-09-17T21:19:12.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:19:12.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234043+0000", 2024-09-17T21:19:12.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.49e-06 2024-09-17T21:19:12.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234061+0000", 2024-09-17T21:19:12.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8416000000000001e-05 2024-09-17T21:19:12.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234076+0000", 2024-09-17T21:19:12.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.456e-05 2024-09-17T21:19:12.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251948+0000", 2024-09-17T21:19:12.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017872774000000001 2024-09-17T21:19:12.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252144+0000", 2024-09-17T21:19:12.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019517300000000001 2024-09-17T21:19:12.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252478+0000", 2024-09-17T21:19:12.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033482899999999998 2024-09-17T21:19:12.856 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.856 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.856 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.856 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:482 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.856 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:19:12.856 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.598662982999997, 2024-09-17T21:19:12.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018988026000000002, 2024-09-17T21:19:12.857 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.857 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.857 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.857 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.857 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.857 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.857 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 482 2024-09-17T21:19:12.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.857 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:19:12.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:19:12.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234096+0000", 2024-09-17T21:19:12.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2139999999999999e-06 2024-09-17T21:19:12.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234101+0000", 2024-09-17T21:19:12.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7100000000000004e-06 2024-09-17T21:19:12.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234118+0000", 2024-09-17T21:19:12.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6486000000000001e-05 2024-09-17T21:19:12.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234131+0000", 2024-09-17T21:19:12.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2824e-05 2024-09-17T21:19:12.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252552+0000", 2024-09-17T21:19:12.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018421549999999998 2024-09-17T21:19:12.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252746+0000", 2024-09-17T21:19:12.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019424399999999999 2024-09-17T21:19:12.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253082+0000", 2024-09-17T21:19:12.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000335998 2024-09-17T21:19:12.861 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.861 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.861 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.861 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:483 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.861 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:19:12.861 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.598608362999997, 2024-09-17T21:19:12.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019462882000000001, 2024-09-17T21:19:12.861 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.861 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 483 2024-09-17T21:19:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:19:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:19:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234150+0000", 2024-09-17T21:19:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.127e-06 2024-09-17T21:19:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234155+0000", 2024-09-17T21:19:12.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:19:12.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234171+0000", 2024-09-17T21:19:12.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6160000000000001e-05 2024-09-17T21:19:12.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234185+0000", 2024-09-17T21:19:12.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3370000000000001e-05 2024-09-17T21:19:12.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253158+0000", 2024-09-17T21:19:12.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018972968999999999 2024-09-17T21:19:12.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253349+0000", 2024-09-17T21:19:12.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000190733 2024-09-17T21:19:12.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253612+0000", 2024-09-17T21:19:12.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026337000000000002 2024-09-17T21:19:12.866 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.866 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.866 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.866 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:484 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.866 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:19:12.866 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.598554337000003, 2024-09-17T21:19:12.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020029261999999999, 2024-09-17T21:19:12.866 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.866 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.866 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.867 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.867 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.867 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.867 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 484 2024-09-17T21:19:12.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.867 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:19:12.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:19:12.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234205+0000", 2024-09-17T21:19:12.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6470000000000001e-06 2024-09-17T21:19:12.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234210+0000", 2024-09-17T21:19:12.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:19:12.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.869 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.869 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234227+0000", 2024-09-17T21:19:12.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.73e-05 2024-09-17T21:19:12.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.869 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.869 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234241+0000", 2024-09-17T21:19:12.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3757000000000001e-05 2024-09-17T21:19:12.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253690+0000", 2024-09-17T21:19:12.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019449368000000002 2024-09-17T21:19:12.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253898+0000", 2024-09-17T21:19:12.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000207161 2024-09-17T21:19:12.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254232+0000", 2024-09-17T21:19:12.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033474299999999998 2024-09-17T21:19:12.871 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.871 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.871 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.871 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:485 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1638400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.871 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:19:12.871 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.598497109999997, 2024-09-17T21:19:12.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020553914, 2024-09-17T21:19:12.872 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.872 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.872 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.872 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.872 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.872 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.872 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 485 2024-09-17T21:19:12.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.872 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:19:12.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:19:12.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234261+0000", 2024-09-17T21:19:12.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.136e-06 2024-09-17T21:19:12.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.874 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.874 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234266+0000", 2024-09-17T21:19:12.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0669999999999999e-06 2024-09-17T21:19:12.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.874 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.874 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234284+0000", 2024-09-17T21:19:12.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7839999999999999e-05 2024-09-17T21:19:12.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234298+0000", 2024-09-17T21:19:12.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3852999999999999e-05 2024-09-17T21:19:12.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254311+0000", 2024-09-17T21:19:12.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020012973 2024-09-17T21:19:12.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254552+0000", 2024-09-17T21:19:12.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024116400000000001 2024-09-17T21:19:12.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254814+0000", 2024-09-17T21:19:12.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000261881 2024-09-17T21:19:12.876 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.876 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.876 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.876 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:486 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.876 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:19:12.877 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.598440721000003, 2024-09-17T21:19:12.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021010240999999999, 2024-09-17T21:19:12.877 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.877 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.877 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.877 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.877 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.877 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.877 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 486 2024-09-17T21:19:12.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.877 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:19:12.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:19:12.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234318+0000", 2024-09-17T21:19:12.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.37e-06 2024-09-17T21:19:12.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234324+0000", 2024-09-17T21:19:12.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:19:12.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234341+0000", 2024-09-17T21:19:12.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7127000000000001e-05 2024-09-17T21:19:12.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.880 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234354+0000", 2024-09-17T21:19:12.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3210000000000001e-05 2024-09-17T21:19:12.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.880 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254896+0000", 2024-09-17T21:19:12.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020541835000000001 2024-09-17T21:19:12.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.880 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255092+0000", 2024-09-17T21:19:12.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019634700000000001 2024-09-17T21:19:12.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255327+0000", 2024-09-17T21:19:12.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023475899999999999 2024-09-17T21:19:12.881 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.881 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.881 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.881 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:487 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.881 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:19:12.881 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.598384525, 2024-09-17T21:19:12.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021453911999999999, 2024-09-17T21:19:12.882 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.882 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.882 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.882 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.882 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.882 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.882 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 487 2024-09-17T21:19:12.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.882 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:19:12.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:19:12.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234374+0000", 2024-09-17T21:19:12.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3039999999999999e-06 2024-09-17T21:19:12.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234380+0000", 2024-09-17T21:19:12.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:19:12.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234397+0000", 2024-09-17T21:19:12.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7232999999999999e-05 2024-09-17T21:19:12.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234411+0000", 2024-09-17T21:19:12.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3784000000000001e-05 2024-09-17T21:19:12.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255406+0000", 2024-09-17T21:19:12.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020995204 2024-09-17T21:19:12.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.886 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255595+0000", 2024-09-17T21:19:12.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018903299999999999 2024-09-17T21:19:12.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.886 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255827+0000", 2024-09-17T21:19:12.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023190099999999999 2024-09-17T21:19:12.886 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.886 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.886 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.887 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:488 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.887 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:19:12.887 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.598328848000001, 2024-09-17T21:19:12.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026887966999999999, 2024-09-17T21:19:12.887 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.887 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.887 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.887 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.887 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.887 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.887 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 488 2024-09-17T21:19:12.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.888 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.888 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:19:12.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.888 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:19:12.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234430+0000", 2024-09-17T21:19:12.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1829999999999999e-06 2024-09-17T21:19:12.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234435+0000", 2024-09-17T21:19:12.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9869999999999999e-06 2024-09-17T21:19:12.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234451+0000", 2024-09-17T21:19:12.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6458999999999999e-05 2024-09-17T21:19:12.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234465+0000", 2024-09-17T21:19:12.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3730000000000001e-05 2024-09-17T21:19:12.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255908+0000", 2024-09-17T21:19:12.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021443413000000001 2024-09-17T21:19:12.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256098+0000", 2024-09-17T21:19:12.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018961 2024-09-17T21:19:12.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261317+0000", 2024-09-17T21:19:12.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052185850000000004 2024-09-17T21:19:12.891 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.891 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.891 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.891 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:489 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2162688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.892 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:19:12.892 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.598273009000003, 2024-09-17T21:19:12.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022632815000000001, 2024-09-17T21:19:12.892 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.892 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.892 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.892 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.892 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.892 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.892 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 489 2024-09-17T21:19:12.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.893 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.893 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:19:12.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.893 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:19:12.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234486+0000", 2024-09-17T21:19:12.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5099999999999999e-06 2024-09-17T21:19:12.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234491+0000", 2024-09-17T21:19:12.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.147e-06 2024-09-17T21:19:12.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234508+0000", 2024-09-17T21:19:12.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7405999999999999e-05 2024-09-17T21:19:12.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234522+0000", 2024-09-17T21:19:12.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3677e-05 2024-09-17T21:19:12.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256536+0000", 2024-09-17T21:19:12.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022013464999999999 2024-09-17T21:19:12.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256752+0000", 2024-09-17T21:19:12.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021637800000000001 2024-09-17T21:19:12.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257117+0000", 2024-09-17T21:19:12.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000365232 2024-09-17T21:19:12.896 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.896 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.896 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.897 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:490 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2293760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.897 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:19:12.897 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.598218219000003, 2024-09-17T21:19:12.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023146462999999999, 2024-09-17T21:19:12.897 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.897 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.897 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.897 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.897 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.897 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.898 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 490 2024-09-17T21:19:12.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.898 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:19:12.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:19:12.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234541+0000", 2024-09-17T21:19:12.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4869999999999999e-06 2024-09-17T21:19:12.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234559+0000", 2024-09-17T21:19:12.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8105999999999999e-05 2024-09-17T21:19:12.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234577+0000", 2024-09-17T21:19:12.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.836e-05 2024-09-17T21:19:12.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234592+0000", 2024-09-17T21:19:12.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4766000000000001e-05 2024-09-17T21:19:12.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257187+0000", 2024-09-17T21:19:12.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022594725 2024-09-17T21:19:12.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257374+0000", 2024-09-17T21:19:12.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018767299999999999 2024-09-17T21:19:12.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257686+0000", 2024-09-17T21:19:12.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000311346 2024-09-17T21:19:12.901 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.901 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.901 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.902 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:491 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.902 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:19:12.902 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.598145610000003, 2024-09-17T21:19:12.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023696872000000001, 2024-09-17T21:19:12.902 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.902 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.902 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.902 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.902 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.902 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.902 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 491 2024-09-17T21:19:12.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.903 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:19:12.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:19:12.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234613+0000", 2024-09-17T21:19:12.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-06 2024-09-17T21:19:12.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234618+0000", 2024-09-17T21:19:12.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3839999999999996e-06 2024-09-17T21:19:12.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234635+0000", 2024-09-17T21:19:12.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.685e-05 2024-09-17T21:19:12.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234648+0000", 2024-09-17T21:19:12.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.313e-05 2024-09-17T21:19:12.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257780+0000", 2024-09-17T21:19:12.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023131946 2024-09-17T21:19:12.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257985+0000", 2024-09-17T21:19:12.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020468400000000001 2024-09-17T21:19:12.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258309+0000", 2024-09-17T21:19:12.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032357499999999999 2024-09-17T21:19:12.907 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.907 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.907 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.907 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:492 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.907 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:19:12.907 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.598090022999997, 2024-09-17T21:19:12.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024258994999999998, 2024-09-17T21:19:12.907 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.907 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.907 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.907 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.908 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.908 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.908 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 492 2024-09-17T21:19:12.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.908 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:19:12.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.909 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.909 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:19:12.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.909 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.909 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234669+0000", 2024-09-17T21:19:12.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.313e-06 2024-09-17T21:19:12.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.909 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.909 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234674+0000", 2024-09-17T21:19:12.910 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1599999999999997e-06 2024-09-17T21:19:12.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.910 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.910 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234690+0000", 2024-09-17T21:19:12.910 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.666e-05 2024-09-17T21:19:12.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.910 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.910 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234704+0000", 2024-09-17T21:19:12.910 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3043e-05 2024-09-17T21:19:12.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.911 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258397+0000", 2024-09-17T21:19:12.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023693204999999998 2024-09-17T21:19:12.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.911 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258593+0000", 2024-09-17T21:19:12.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019578999999999999 2024-09-17T21:19:12.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258926+0000", 2024-09-17T21:19:12.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033382399999999998 2024-09-17T21:19:12.912 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.912 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.912 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.912 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:493 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.912 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:19:12.912 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.598033557999997, 2024-09-17T21:19:12.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024793461999999999, 2024-09-17T21:19:12.913 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.913 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.913 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.913 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.913 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.913 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.913 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 493 2024-09-17T21:19:12.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.913 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.913 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.914 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:19:12.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.914 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.914 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.914 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:19:12.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.914 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.914 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.914 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234725+0000", 2024-09-17T21:19:12.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.24e-06 2024-09-17T21:19:12.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.915 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.915 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234731+0000", 2024-09-17T21:19:12.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8939999999999998e-06 2024-09-17T21:19:12.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.915 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.915 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234749+0000", 2024-09-17T21:19:12.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7863000000000002e-05 2024-09-17T21:19:12.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234762+0000", 2024-09-17T21:19:12.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3322999999999999e-05 2024-09-17T21:19:12.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259022+0000", 2024-09-17T21:19:12.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024259691999999999 2024-09-17T21:19:12.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259259+0000", 2024-09-17T21:19:12.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023669199999999999 2024-09-17T21:19:12.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.917 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259517+0000", 2024-09-17T21:19:12.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025875799999999999 2024-09-17T21:19:12.917 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.917 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.917 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.918 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:494 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.918 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:19:12.918 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.597975857999998, 2024-09-17T21:19:12.918 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025251952000000001, 2024-09-17T21:19:12.918 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.918 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.918 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.918 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.918 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.918 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.918 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 494 2024-09-17T21:19:12.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.919 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.919 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:19:12.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.919 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:19:12.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234783+0000", 2024-09-17T21:19:12.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2440000000000001e-06 2024-09-17T21:19:12.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234789+0000", 2024-09-17T21:19:12.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:19:12.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234806+0000", 2024-09-17T21:19:12.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7450000000000001e-05 2024-09-17T21:19:12.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234819+0000", 2024-09-17T21:19:12.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3326000000000001e-05 2024-09-17T21:19:12.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259585+0000", 2024-09-17T21:19:12.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024765579999999999 2024-09-17T21:19:12.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259787+0000", 2024-09-17T21:19:12.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202444 2024-09-17T21:19:12.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260033+0000", 2024-09-17T21:19:12.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024611499999999998 2024-09-17T21:19:12.922 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.922 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.922 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.922 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:495 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.923 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:19:12.923 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.597920494999997, 2024-09-17T21:19:12.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025776342000000001, 2024-09-17T21:19:12.923 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.923 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.923 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.923 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.923 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.923 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.923 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 495 2024-09-17T21:19:12.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.923 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:19:12.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:19:12.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234838+0000", 2024-09-17T21:19:12.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2130000000000001e-06 2024-09-17T21:19:12.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.925 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234843+0000", 2024-09-17T21:19:12.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9899999999999997e-06 2024-09-17T21:19:12.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.925 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234860+0000", 2024-09-17T21:19:12.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6973000000000002e-05 2024-09-17T21:19:12.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234873+0000", 2024-09-17T21:19:12.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3179999999999999e-05 2024-09-17T21:19:12.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260135+0000", 2024-09-17T21:19:12.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025262118 2024-09-17T21:19:12.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260364+0000", 2024-09-17T21:19:12.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022830699999999999 2024-09-17T21:19:12.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260613+0000", 2024-09-17T21:19:12.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024956099999999998 2024-09-17T21:19:12.927 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.927 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.927 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.928 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:496 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.928 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:19:12.928 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.597867215000001, 2024-09-17T21:19:12.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026376773999999999, 2024-09-17T21:19:12.928 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.928 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.928 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:12.928 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.928 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.928 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.928 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 496 2024-09-17T21:19:12.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.929 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:19:12.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:19:12.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.930 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234891+0000", 2024-09-17T21:19:12.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1230000000000001e-06 2024-09-17T21:19:12.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.930 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.930 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234896+0000", 2024-09-17T21:19:12.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9599999999999999e-06 2024-09-17T21:19:12.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.930 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.930 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234913+0000", 2024-09-17T21:19:12.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6747000000000001e-05 2024-09-17T21:19:12.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234926+0000", 2024-09-17T21:19:12.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-05 2024-09-17T21:19:12.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260675+0000", 2024-09-17T21:19:12.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025748979000000002 2024-09-17T21:19:12.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260883+0000", 2024-09-17T21:19:12.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020848399999999999 2024-09-17T21:19:12.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261267+0000", 2024-09-17T21:19:12.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038345099999999998 2024-09-17T21:19:12.932 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.932 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.932 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.933 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:682 3.9 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2455552~12288,write 8126464~10240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:12.933 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:19:12.933 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.563930163999999, 2024-09-17T21:19:12.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018447261999999999, 2024-09-17T21:19:12.933 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:12.933 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:12.933 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:12.933 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:12.933 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:12.933 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:12.933 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 682 2024-09-17T21:19:12.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.934 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:12.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:12.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:19:12.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:12.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:19:12.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:12.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:12.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268830+0000", 2024-09-17T21:19:12.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6419999999999999e-06 2024-09-17T21:19:12.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:12.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271673+0000", 2024-09-17T21:19:12.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028432610000000001 2024-09-17T21:19:12.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:12.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271700+0000", 2024-09-17T21:19:12.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7200999999999999e-05 2024-09-17T21:19:12.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:12.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271715+0000", 2024-09-17T21:19:12.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5145e-05 2024-09-17T21:19:12.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:12.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271785+0000", 2024-09-17T21:19:12.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9519999999999998e-05 2024-09-17T21:19:12.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.937 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:12.937 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271918+0000", 2024-09-17T21:19:12.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013330899999999999 2024-09-17T21:19:12.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.937 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:12.937 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.272449+0000", 2024-09-17T21:19:12.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00053051200000000002 2024-09-17T21:19:12.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.937 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:12.937 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.278822+0000", 2024-09-17T21:19:12.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0063731619999999999 2024-09-17T21:19:12.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:12.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287167+0000", 2024-09-17T21:19:12.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083452149999999996 2024-09-17T21:19:12.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:12.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287213+0000", 2024-09-17T21:19:12.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5972e-05 2024-09-17T21:19:12.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:12.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:12.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:12.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287274+0000", 2024-09-17T21:19:12.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1322999999999995e-05 2024-09-17T21:19:12.939 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.939 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.939 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.939 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:12.939 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:12.939 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:12.941 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:19:12.942 INFO:teuthology.orchestra.run.smithi184.stdout:2306 write 0x9783e00 thru 0x9787109 (0x330a bytes) 2024-09-17T21:19:12.942 INFO:teuthology.orchestra.run.smithi184.stdout:2307 write 0x25b4a00 thru 0x25bb26a (0x686b bytes) 2024-09-17T21:19:12.942 INFO:teuthology.orchestra.run.smithi184.stdout:2308 read 0x8e61800 thru 0x8e717fc (0xfffd bytes) 2024-09-17T21:19:12.943 INFO:teuthology.orchestra.run.smithi184.stdout:2310 read 0x37a2800 thru 0x37b23b4 (0xfbb5 bytes) 2024-09-17T21:19:12.943 INFO:teuthology.orchestra.run.smithi184.stdout:2311 read 0x8977a00 thru 0x89878cc (0xfecd bytes) 2024-09-17T21:19:12.943 INFO:teuthology.orchestra.run.smithi184.stdout:2312 read 0x22c7000 thru 0x22d5189 (0xe18a bytes) 2024-09-17T21:19:12.943 INFO:teuthology.orchestra.run.smithi184.stdout:2314 write 0x6ab2e00 thru 0x6abc0be (0x92bf bytes) 2024-09-17T21:19:12.943 INFO:teuthology.orchestra.run.smithi184.stdout:2315 write 0x6c1d000 thru 0x6c2c835 (0xf836 bytes) 2024-09-17T21:19:12.943 INFO:teuthology.orchestra.run.smithi184.stdout:2319 write 0x2eece00 thru 0x2eef973 (0x2b74 bytes) 2024-09-17T21:19:12.943 INFO:teuthology.orchestra.run.smithi184.stdout:2323 read 0x6d64200 thru 0x6d6f055 (0xae56 bytes) 2024-09-17T21:19:12.943 INFO:teuthology.orchestra.run.smithi184.stdout:2325 punch from 0x4184800 to 0x418c600, (0x7e00 bytes) 2024-09-17T21:19:12.943 INFO:teuthology.orchestra.run.smithi184.stdout:2326 write 0xc0c8400 thru 0xc0cb490 (0x3091 bytes) 2024-09-17T21:19:12.943 INFO:teuthology.orchestra.run.smithi184.stdout:2327 punch from 0x2627600 to 0x2628000, (0xa00 bytes) 2024-09-17T21:19:12.943 INFO:teuthology.orchestra.run.smithi184.stdout:2329 write 0x2487000 thru 0x249427b (0xd27c bytes) 2024-09-17T21:19:12.943 INFO:teuthology.orchestra.run.smithi184.stdout:2331 trunc from 0xc0cb600 to 0xd9f6e00 2024-09-17T21:19:12.943 INFO:teuthology.orchestra.run.smithi184.stdout:2333 trunc from 0xd9f6e00 to 0xab01e00 2024-09-17T21:19:12.944 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:12.939+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:12.964 INFO:teuthology.orchestra.run.smithi184.stdout:2334 write 0x4b48c00 thru 0x4b56c78 (0xe079 bytes) 2024-09-17T21:19:12.974 INFO:teuthology.orchestra.run.smithi184.stdout:2335 trunc from 0xab01e00 to 0x6c56400 2024-09-17T21:19:13.031 INFO:teuthology.orchestra.run.smithi184.stdout:2336 trunc from 0x6c56400 to 0xd00fc00 2024-09-17T21:19:13.041 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:13.039+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:13.050 INFO:teuthology.orchestra.run.smithi184.stdout:2337 write 0xf60400 thru 0xf64cc0 (0x48c1 bytes) 2024-09-17T21:19:13.066 INFO:teuthology.orchestra.run.smithi184.stdout:2338 trunc from 0xd00fc00 to 0x78f8e00 2024-09-17T21:19:13.089 INFO:teuthology.orchestra.run.smithi184.stdout:2339 write 0x725e200 thru 0x726aa68 (0xc869 bytes) 2024-09-17T21:19:13.124 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:13.124 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:13.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.125 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4871.0:1563 3.5 3:a1656b6c:::rbd_header.115265216c1c:head [call rbd.parent_get in=13b,call rbd.parent_overlap_get in=29b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:19:13.125 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:13.123102+0000", 2024-09-17T21:19:13.125 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.00024210400000000001, 2024-09-17T21:19:13.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00030610899999999999, 2024-09-17T21:19:13.125 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:13.125 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:13.125 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:13.125 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:13.125 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4871", 2024-09-17T21:19:13.125 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:19:13.126 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1563 2024-09-17T21:19:13.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.126 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:13.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:13.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:13.123102+0000", 2024-09-17T21:19:13.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:13.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:13.123102+0000", 2024-09-17T21:19:13.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:13.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:13.123104+0000", 2024-09-17T21:19:13.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.843e-06 2024-09-17T21:19:13.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:13.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:13.123114+0000", 2024-09-17T21:19:13.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8770000000000005e-06 2024-09-17T21:19:13.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:13.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:13.123143+0000", 2024-09-17T21:19:13.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9371999999999999e-05 2024-09-17T21:19:13.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:13.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:13.123166+0000", 2024-09-17T21:19:13.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2920000000000001e-05 2024-09-17T21:19:13.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:13.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:13.123236+0000", 2024-09-17T21:19:13.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0517000000000003e-05 2024-09-17T21:19:13.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:13.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:13.123303+0000", 2024-09-17T21:19:13.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6705999999999995e-05 2024-09-17T21:19:13.129 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.129 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:13.129 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.130 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4871.0:1564 3.5 3.36d6a685 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:19:13.130 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:13.123379+0000", 2024-09-17T21:19:13.130 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4294967295.9999647, 2024-09-17T21:19:13.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020903100000000001, 2024-09-17T21:19:13.130 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:13.130 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:13.130 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "reached pg", 2024-09-17T21:19:13.130 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:13.130 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4871", 2024-09-17T21:19:13.130 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:19:13.131 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1564 2024-09-17T21:19:13.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.131 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:13.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:13.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:13.123379+0000", 2024-09-17T21:19:13.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:13.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:13.123379+0000", 2024-09-17T21:19:13.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:13.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:13.123381+0000", 2024-09-17T21:19:13.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9829999999999999e-06 2024-09-17T21:19:13.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:13.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:13.123391+0000", 2024-09-17T21:19:13.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8800000000000003e-06 2024-09-17T21:19:13.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:13.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:13.123420+0000", 2024-09-17T21:19:13.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8368999999999999e-05 2024-09-17T21:19:13.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:13.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:13.123440+0000", 2024-09-17T21:19:13.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0066000000000001e-05 2024-09-17T21:19:13.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:13.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:13.123587+0000", 2024-09-17T21:19:13.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014707000000000001 2024-09-17T21:19:13.134 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.134 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:13.134 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.134 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.134 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-09-17T21:19:13.134 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 2 2024-09-17T21:19:13.134 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:13.134 INFO:teuthology.orchestra.run.smithi184.stdout:2340 write 0xbc47600 thru 0xbc4aac8 (0x34c9 bytes) 2024-09-17T21:19:13.135 INFO:teuthology.orchestra.run.smithi184.stdout:2342 write 0x3ee8200 thru 0x3ef78d0 (0xf6d1 bytes) 2024-09-17T21:19:13.142 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:19:13.142 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:13.136+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:13.165 INFO:teuthology.orchestra.run.smithi184.stdout:2344 write 0xdc21000 thru 0xdc21229 (0x22a bytes) 2024-09-17T21:19:13.166 INFO:teuthology.orchestra.run.smithi184.stdout:2347 read 0x8345200 thru 0x8349acc (0x48cd bytes) 2024-09-17T21:19:13.168 INFO:teuthology.orchestra.run.smithi184.stdout:2349 trunc from 0xdc21400 to 0xd4f1c00 2024-09-17T21:19:13.199 INFO:teuthology.orchestra.run.smithi184.stdout:2350 punch from 0xabfe00 to 0xac0e00, (0x1000 bytes) 2024-09-17T21:19:13.201 INFO:teuthology.orchestra.run.smithi184.stdout:2351 write 0x1025400 thru 0x102cef1 (0x7af2 bytes) 2024-09-17T21:19:13.201 INFO:teuthology.orchestra.run.smithi184.stdout:2352 punch from 0xbcfc800 to 0xbd06600, (0x9e00 bytes) 2024-09-17T21:19:13.201 INFO:teuthology.orchestra.run.smithi184.stdout:2353 trunc from 0xd4f1c00 to 0xd747000 2024-09-17T21:19:13.229 INFO:teuthology.orchestra.run.smithi184.stdout:2356 trunc from 0xd747000 to 0xe47f400 2024-09-17T21:19:13.242 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:13.239+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:13.243 INFO:teuthology.orchestra.run.smithi184.stdout:2357 write 0x6113600 thru 0x611acc7 (0x76c8 bytes) 2024-09-17T21:19:13.244 INFO:teuthology.orchestra.run.smithi184.stdout:2358 write 0xdd04a00 thru 0xdd0a592 (0x5b93 bytes) 2024-09-17T21:19:13.244 INFO:teuthology.orchestra.run.smithi184.stdout:2359 read 0x7a0ea00 thru 0x7a1d244 (0xe845 bytes) 2024-09-17T21:19:13.245 INFO:teuthology.orchestra.run.smithi184.stdout:2362 read 0x58f200 thru 0x598d32 (0x9b33 bytes) 2024-09-17T21:19:13.245 INFO:teuthology.orchestra.run.smithi184.stdout:2363 punch from 0x85b6c00 to 0x85bc200, (0x5600 bytes) 2024-09-17T21:19:13.246 INFO:teuthology.orchestra.run.smithi184.stdout:2366 write 0x9070400 thru 0x90728a9 (0x24aa bytes) 2024-09-17T21:19:13.246 INFO:teuthology.orchestra.run.smithi184.stdout:2367 punch from 0x9e0c400 to 0x9e0ea00, (0x2600 bytes) 2024-09-17T21:19:13.247 INFO:teuthology.orchestra.run.smithi184.stdout:2369 punch from 0xcf71400 to 0xcf7d800, (0xc400 bytes) 2024-09-17T21:19:13.247 INFO:teuthology.orchestra.run.smithi184.stdout:2374 read 0xba49400 thru 0xba49f7f (0xb80 bytes) 2024-09-17T21:19:13.247 INFO:teuthology.orchestra.run.smithi184.stdout:2375 write 0xdca2800 thru 0xdca5a6b (0x326c bytes) 2024-09-17T21:19:13.247 INFO:teuthology.orchestra.run.smithi184.stdout:2376 write 0x71eaa00 thru 0x71f55e7 (0xabe8 bytes) 2024-09-17T21:19:13.247 INFO:teuthology.orchestra.run.smithi184.stdout:2382 read 0x252b600 thru 0x25338bc (0x82bd bytes) 2024-09-17T21:19:13.250 INFO:teuthology.orchestra.run.smithi184.stdout:2386 trunc from 0xe47f400 to 0xd38f600 2024-09-17T21:19:13.297 INFO:teuthology.orchestra.run.smithi184.stdout:2387 read 0x8c2f800 thru 0x8c39144 (0x9945 bytes) 2024-09-17T21:19:13.298 INFO:teuthology.orchestra.run.smithi184.stdout:2389 write 0xd117c00 thru 0xd1276d6 (0xfad7 bytes) 2024-09-17T21:19:13.304 INFO:teuthology.orchestra.run.smithi184.stdout:2390 trunc from 0xd38f600 to 0xb2cfa00 2024-09-17T21:19:13.321 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:13.321 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:13.321 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:13.321 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:13.321 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:13.337 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:19:13.355 INFO:teuthology.orchestra.run.smithi184.stdout:2392 write 0xb832000 thru 0xb83976b (0x776c bytes) 2024-09-17T21:19:13.356 INFO:teuthology.orchestra.run.smithi184.stdout:2393 read 0x7e18a00 thru 0x7e23e26 (0xb427 bytes) 2024-09-17T21:19:13.356 INFO:teuthology.orchestra.run.smithi184.stdout:2394 write 0x9040200 thru 0x904e89b (0xe69c bytes) 2024-09-17T21:19:13.358 INFO:teuthology.orchestra.run.smithi184.stdout:2397 read 0x7dbce00 thru 0x7dcc387 (0xf588 bytes) 2024-09-17T21:19:13.376 INFO:teuthology.orchestra.run.smithi184.stdout:2400 trunc from 0xb839800 to 0x20fcc00 2024-09-17T21:19:13.386 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:13.384+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:13.438 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:19:13.444 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:13.440+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:13.449 INFO:teuthology.orchestra.run.smithi184.stdout:2401 write 0xa443000 thru 0xa44abed (0x7bee bytes) 2024-09-17T21:19:13.450 INFO:teuthology.orchestra.run.smithi184.stdout:2402 write 0xb58e00 thru 0xb61972 (0x8b73 bytes) 2024-09-17T21:19:13.451 INFO:teuthology.orchestra.run.smithi184.stdout:2403 read 0x84c6800 thru 0x84d5ab1 (0xf2b2 bytes) 2024-09-17T21:19:13.452 INFO:teuthology.orchestra.run.smithi184.stdout:2405 read 0x3c73e00 thru 0x3c80bc3 (0xcdc4 bytes) 2024-09-17T21:19:13.452 INFO:teuthology.orchestra.run.smithi184.stdout:2410 read 0x83e8000 thru 0x83ecdef (0x4df0 bytes) 2024-09-17T21:19:13.452 INFO:teuthology.orchestra.run.smithi184.stdout:2412 read 0x5419000 thru 0x5426e5a (0xde5b bytes) 2024-09-17T21:19:13.459 INFO:teuthology.orchestra.run.smithi184.stdout:2414 trunc from 0xa44ac00 to 0x6165600 2024-09-17T21:19:13.469 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:13.469 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:13.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:13.470 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:13.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.470 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:19:13.470 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:13.470 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 51.861582744000003, 2024-09-17T21:19:13.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:19:13.470 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:13.470 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:13.470 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:13.470 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:13.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:13.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:13.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:13.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:13.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:13.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:19:13.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:19:13.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:13.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:19:13.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:19:13.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:13.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:19:13.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:19:13.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:13.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:19:13.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:19:13.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:13.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:19:13.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:19:13.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:13.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:19:13.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:19:13.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:13.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:19:13.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:19:13.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:13.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:19:13.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:19:13.474 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.474 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:13.474 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.475 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:19:13.475 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:13.475 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 51.851636108000001, 2024-09-17T21:19:13.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:19:13.475 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:13.475 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:13.475 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:13.475 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:13.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:13.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:13.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:13.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:13.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:13.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:19:13.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:19:13.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:13.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:19:13.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:19:13.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:13.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:19:13.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:19:13.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:13.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:19:13.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:19:13.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:13.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:19:13.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:19:13.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:13.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:19:13.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:19:13.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:13.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:19:13.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:19:13.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:13.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:19:13.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:19:13.479 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.479 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:13.479 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.479 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:19:13.479 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:13.479 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 50.608485268000003, 2024-09-17T21:19:13.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:19:13.479 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:13.479 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:13.479 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:13.480 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:13.480 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:19:13.480 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:19:13.480 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:19:13.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.480 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:13.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:13.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:13.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:13.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:13.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:13.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:19:13.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:19:13.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:13.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:19:13.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:19:13.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:13.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:19:13.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:19:13.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:13.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:19:13.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:19:13.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:13.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:19:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:19:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:19:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:19:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:19:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:19:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:19:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:19:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:19:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:19:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:19:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:19:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:19:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:19:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:19:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 50.598839781999999, 2024-09-17T21:19:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:19:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:19:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:19:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:19:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:19:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:19:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:19:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:19:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:19:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:19:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:19:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:19:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:19:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:19:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:19:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:19:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:19:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:19:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:19:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:19:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:19:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:19:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:19:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:19:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:19:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:19:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 37.229903749000002, 2024-09-17T21:19:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:19:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:19:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:19:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:19:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:19:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:19:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:19:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:19:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:19:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:19:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:19:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:19:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:19:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:19:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:19:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:19:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:565 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:19:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 37.226114784000004, 2024-09-17T21:19:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024193392000000001, 2024-09-17T21:19:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 565 2024-09-17T21:19:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:19:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:19:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239660+0000", 2024-09-17T21:19:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:19:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239666+0000", 2024-09-17T21:19:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8699999999999997e-06 2024-09-17T21:19:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239688+0000", 2024-09-17T21:19:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.213e-05 2024-09-17T21:19:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239703+0000", 2024-09-17T21:19:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4813e-05 2024-09-17T21:19:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:13.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263427+0000", 2024-09-17T21:19:13.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023724630999999999 2024-09-17T21:19:13.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:13.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263614+0000", 2024-09-17T21:19:13.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018664500000000001 2024-09-17T21:19:13.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263852+0000", 2024-09-17T21:19:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023793999999999999 2024-09-17T21:19:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:819 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5111808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.028331459, 2024-09-17T21:19:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024344811000000001, 2024-09-17T21:19:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 819 2024-09-17T21:19:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437443+0000", 2024-09-17T21:19:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1969999999999999e-06 2024-09-17T21:19:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:13.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437448+0000", 2024-09-17T21:19:13.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9490000000000004e-06 2024-09-17T21:19:13.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:13.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437467+0000", 2024-09-17T21:19:13.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8862999999999999e-05 2024-09-17T21:19:13.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437480+0000", 2024-09-17T21:19:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2833000000000001e-05 2024-09-17T21:19:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461552+0000", 2024-09-17T21:19:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024072695000000002 2024-09-17T21:19:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461639+0000", 2024-09-17T21:19:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6996000000000005e-05 2024-09-17T21:19:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461787+0000", 2024-09-17T21:19:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014727800000000001 2024-09-17T21:19:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.028219187000001, 2024-09-17T21:19:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:19:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:19:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:19:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:19:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:19:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:19:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:19:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:19:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:19:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:19:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:19:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:19:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:19:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:19:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:19:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:19:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:833 3.5 3:a772dfb1:::rbd_data.11523651b015.0000000000000001:d [sparse-read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.027784901, 2024-09-17T21:19:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024216024999999999, 2024-09-17T21:19:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 833 2024-09-17T21:19:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437990+0000", 2024-09-17T21:19:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2500000000000001e-06 2024-09-17T21:19:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437995+0000", 2024-09-17T21:19:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1240000000000004e-06 2024-09-17T21:19:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438012+0000", 2024-09-17T21:19:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7572e-05 2024-09-17T21:19:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438031+0000", 2024-09-17T21:19:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8612000000000001e-05 2024-09-17T21:19:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461830+0000", 2024-09-17T21:19:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023799135999999999 2024-09-17T21:19:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462005+0000", 2024-09-17T21:19:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000174747 2024-09-17T21:19:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:19:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019958399999999999 2024-09-17T21:19:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.02772461, 2024-09-17T21:19:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:19:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:19:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:19:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:19:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:19:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:19:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:19:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:19:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:19:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:19:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:19:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:19:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:19:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:19:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:19:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:19:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.027669229000001, 2024-09-17T21:19:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:19:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:19:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:19:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:19:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:19:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:19:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:19:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:19:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:19:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:19:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:19:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:19:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:19:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:19:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:19:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:19:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.027612950999998, 2024-09-17T21:19:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:19:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:19:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:19:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:19:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:19:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:19:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:19:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:19:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:19:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:19:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:19:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:19:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:19:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:19:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:19:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:19:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.026663312, 2024-09-17T21:19:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:19:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:19:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:19:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:19:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:19:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:19:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:19:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:19:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:19:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:19:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:19:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:19:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:19:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:19:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:19:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:19:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.026584692, 2024-09-17T21:19:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:19:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:19:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:19:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:19:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:19:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:19:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:19:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:19:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:19:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:19:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:19:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:19:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:19:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:19:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:19:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:19:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:847 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 4653056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439245+0000", 2024-09-17T21:19:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.026528253999999, 2024-09-17T21:19:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024172945000000001, 2024-09-17T21:19:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 847 2024-09-17T21:19:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439245+0000", 2024-09-17T21:19:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439245+0000", 2024-09-17T21:19:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439246+0000", 2024-09-17T21:19:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:19:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439252+0000", 2024-09-17T21:19:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3569999999999997e-06 2024-09-17T21:19:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439269+0000", 2024-09-17T21:19:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7316000000000002e-05 2024-09-17T21:19:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439282+0000", 2024-09-17T21:19:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3269e-05 2024-09-17T21:19:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463240+0000", 2024-09-17T21:19:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02395721 2024-09-17T21:19:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463306+0000", 2024-09-17T21:19:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6618000000000005e-05 2024-09-17T21:19:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463418+0000", 2024-09-17T21:19:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111672 2024-09-17T21:19:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.026471752999999, 2024-09-17T21:19:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:19:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:19:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:19:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:19:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:19:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:19:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:19:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:19:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:19:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:19:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:19:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:19:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:19:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:19:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:19:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:19:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:849 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5963776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.025877717, 2024-09-17T21:19:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024537514, 2024-09-17T21:19:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 849 2024-09-17T21:19:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439897+0000", 2024-09-17T21:19:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9099999999999999e-06 2024-09-17T21:19:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439905+0000", 2024-09-17T21:19:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0870000000000007e-06 2024-09-17T21:19:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439924+0000", 2024-09-17T21:19:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9042000000000002e-05 2024-09-17T21:19:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439941+0000", 2024-09-17T21:19:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.624e-05 2024-09-17T21:19:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464132+0000", 2024-09-17T21:19:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024191579000000001 2024-09-17T21:19:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464291+0000", 2024-09-17T21:19:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015904800000000001 2024-09-17T21:19:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464433+0000", 2024-09-17T21:19:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014160799999999999 2024-09-17T21:19:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.025810699000001, 2024-09-17T21:19:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:19:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:19:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:19:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:19:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:19:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:19:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:19:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:19:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:19:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:19:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:19:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:19:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:19:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:19:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:19:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:19:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.025745887999999, 2024-09-17T21:19:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:19:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:19:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:19:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:19:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:19:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:19:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:19:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:19:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:19:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:19:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:19:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:19:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:19:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:19:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:19:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:19:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.025684910999999, 2024-09-17T21:19:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:19:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:19:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:19:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:19:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:19:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:19:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:19:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:19:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:19:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:19:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:19:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:19:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:19:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:19:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:13.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:19:13.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:19:13.566 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.566 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:13.566 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.566 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:13.566 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:13.566 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:13.567 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-17T21:19:13.569 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T21:19:13.569 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:19:13.570 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-17T21:19:13.571 INFO:teuthology.orchestra.run.smithi184.stdout:2418 write 0xb317000 thru 0xb31f48b (0x848c bytes) 2024-09-17T21:19:13.571 INFO:teuthology.orchestra.run.smithi184.stdout:2421 punch from 0x3aaec00 to 0x3abb800, (0xcc00 bytes) 2024-09-17T21:19:13.571 INFO:teuthology.orchestra.run.smithi184.stdout:2424 read 0x46e6800 thru 0x46f2d07 (0xc508 bytes) 2024-09-17T21:19:13.571 INFO:teuthology.orchestra.run.smithi184.stdout:2425 punch from 0xaf0cc00 to 0xaf12200, (0x5600 bytes) 2024-09-17T21:19:13.571 INFO:teuthology.orchestra.run.smithi184.stdout:2426 punch from 0x469d000 to 0x46ab200, (0xe200 bytes) 2024-09-17T21:19:13.571 INFO:teuthology.orchestra.run.smithi184.stdout:2427 read 0x71ef400 thru 0x71f037e (0xf7f bytes) 2024-09-17T21:19:13.571 INFO:teuthology.orchestra.run.smithi184.stdout:2428 punch from 0x43e7a00 to 0x43e8600, (0xc00 bytes) 2024-09-17T21:19:13.571 INFO:teuthology.orchestra.run.smithi184.stdout:2433 punch from 0xa96b200 to 0xa977e00, (0xcc00 bytes) 2024-09-17T21:19:13.572 INFO:teuthology.orchestra.run.smithi184.stdout:2436 write 0x6c02c00 thru 0x6c0a5d6 (0x79d7 bytes) 2024-09-17T21:19:13.572 INFO:teuthology.orchestra.run.smithi184.stdout:2437 flatten 2024-09-17T21:19:13.572 INFO:teuthology.orchestra.run.smithi184.stdout:2439 write 0x52d7400 thru 0x52db888 (0x4489 bytes) 2024-09-17T21:19:13.572 INFO:teuthology.orchestra.run.smithi184.stdout:2440 punch from 0x2a23200 to 0x2a27e00, (0x4c00 bytes) 2024-09-17T21:19:13.572 INFO:teuthology.orchestra.run.smithi184.stdout:2441 trunc from 0xb31f600 to 0xc11c400 2024-09-17T21:19:13.572 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:13.563+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:13.581 INFO:teuthology.orchestra.run.smithi184.stdout:2442 punch from 0xb795a00 to 0xb79c800, (0x6e00 bytes) 2024-09-17T21:19:13.582 INFO:teuthology.orchestra.run.smithi184.stdout:2443 read 0x87cc400 thru 0x87d709a (0xac9b bytes) 2024-09-17T21:19:13.582 INFO:teuthology.orchestra.run.smithi184.stdout:2446 read 0xbd5ee00 thru 0xbd6cabf (0xdcc0 bytes) 2024-09-17T21:19:13.583 INFO:teuthology.orchestra.run.smithi184.stdout:2448 write 0x410d800 thru 0x411135b (0x3b5c bytes) 2024-09-17T21:19:13.583 INFO:teuthology.orchestra.run.smithi184.stdout:2451 read 0x3533c00 thru 0x3538034 (0x4435 bytes) 2024-09-17T21:19:13.583 INFO:teuthology.orchestra.run.smithi184.stdout:2452 write 0x6aad800 thru 0x6ab532b (0x7b2c bytes) 2024-09-17T21:19:13.593 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:13.594 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:13.603 INFO:teuthology.orchestra.run.smithi184.stdout:2455 trunc from 0xc11c400 to 0x4bf4200 2024-09-17T21:19:13.612 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:13.612 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:13.630 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:13.630 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:13.645 INFO:teuthology.orchestra.run.smithi184.stdout:2457 write 0x4b30200 thru 0x4b36902 (0x6703 bytes) 2024-09-17T21:19:13.647 INFO:teuthology.orchestra.run.smithi184.stdout:2458 read 0x31d2e00 thru 0x31e210e (0xf30f bytes) 2024-09-17T21:19:13.647 INFO:teuthology.orchestra.run.smithi184.stdout:2462 write 0x2b3e800 thru 0x2b43245 (0x4a46 bytes) 2024-09-17T21:19:13.649 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:13.649 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:13.653 INFO:teuthology.orchestra.run.smithi184.stdout:2463 trunc from 0x4bf4200 to 0x6f80000 2024-09-17T21:19:13.668 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:13.664+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:13.669 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:13.670 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:13.684 INFO:teuthology.orchestra.run.smithi184.stdout:2464 write 0x4052c00 thru 0x405e6ad (0xbaae bytes) 2024-09-17T21:19:13.689 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:13.689 INFO:teuthology.orchestra.run.smithi184.stdout:2466 read 0x10b0e00 thru 0x10b808e (0x728f bytes) 2024-09-17T21:19:13.690 INFO:teuthology.orchestra.run.smithi184.stdout:2470 punch from 0x1cbc800 to 0x1cc3c00, (0x7400 bytes) 2024-09-17T21:19:13.690 INFO:teuthology.orchestra.run.smithi184.stdout:2471 read 0x4308e00 thru 0x430b3ce (0x25cf bytes) 2024-09-17T21:19:13.719 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:13.719 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:13.723 INFO:teuthology.orchestra.run.smithi184.stdout:2472 write 0x9ab4e00 thru 0x9ac1a32 (0xcc33 bytes) 2024-09-17T21:19:13.724 INFO:teuthology.orchestra.run.smithi184.stdout:2473 read 0x34d5200 thru 0x34e2c9b (0xda9c bytes) 2024-09-17T21:19:13.724 INFO:teuthology.orchestra.run.smithi184.stdout:2474 read 0x24a2a00 thru 0x24a449c (0x1a9d bytes) 2024-09-17T21:19:13.724 INFO:teuthology.orchestra.run.smithi184.stdout:skipping zero length punch hole 2024-09-17T21:19:13.725 INFO:teuthology.orchestra.run.smithi184.stdout:2477 read 0x7c3d000 thru 0x7c40734 (0x3735 bytes) 2024-09-17T21:19:13.725 INFO:teuthology.orchestra.run.smithi184.stdout:2478 write 0x78e9200 thru 0x78f0e1e (0x7c1f bytes) 2024-09-17T21:19:13.725 INFO:teuthology.orchestra.run.smithi184.stdout:2480 read 0x628f200 thru 0x629a46b (0xb26c bytes) 2024-09-17T21:19:13.737 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:19:13.741 INFO:teuthology.orchestra.run.smithi184.stdout:2481 trunc from 0x9ac1c00 to 0x18d0c00 2024-09-17T21:19:13.769 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:13.764+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:13.870 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:13.868+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:13.877 INFO:teuthology.orchestra.run.smithi184.stdout:2483 write 0xd7f9000 thru 0xd8050dc (0xc0dd bytes) 2024-09-17T21:19:13.897 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:13.897 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:13.911 INFO:teuthology.orchestra.run.smithi184.stdout:2487 write 0xddadc00 thru 0xddbb54a (0xd94b bytes) 2024-09-17T21:19:13.912 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:19:13.915 INFO:teuthology.orchestra.run.smithi184.stdout:2489 write 0x55b800 thru 0x55e83a (0x303b bytes) 2024-09-17T21:19:13.915 INFO:teuthology.orchestra.run.smithi184.stdout:2492 read 0xdc1c00 thru 0xdd0631 (0xea32 bytes) 2024-09-17T21:19:13.915 INFO:teuthology.orchestra.run.smithi184.stdout:2493 read 0xa150400 thru 0xa15e8a6 (0xe4a7 bytes) 2024-09-17T21:19:13.915 INFO:teuthology.orchestra.run.smithi184.stdout:2494 write 0x7cb1a00 thru 0x7cba5aa (0x8bab bytes) 2024-09-17T21:19:13.916 INFO:teuthology.orchestra.run.smithi184.stdout:2501 write 0x4258c00 thru 0x425970e (0xb0f bytes) 2024-09-17T21:19:13.916 INFO:teuthology.orchestra.run.smithi184.stdout:2502 write 0x2774e00 thru 0x2777a49 (0x2c4a bytes) 2024-09-17T21:19:13.955 INFO:teuthology.orchestra.run.smithi184.stdout:2504 write 0xddf7e00 thru 0xde041f1 (0xc3f2 bytes) 2024-09-17T21:19:13.957 INFO:teuthology.orchestra.run.smithi184.stdout:2509 read 0x14d3000 thru 0x14d8088 (0x5089 bytes) 2024-09-17T21:19:13.957 INFO:teuthology.orchestra.run.smithi184.stdout:2512 punch from 0xcd62e00 to 0xcd6c000, (0x9200 bytes) 2024-09-17T21:19:13.966 INFO:teuthology.orchestra.run.smithi184.stdout:2513 trunc from 0xde04200 to 0xa25c200 2024-09-17T21:19:13.971 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:13.968+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:14.027 INFO:teuthology.orchestra.run.smithi184.stdout:2514 write 0xc7a5400 thru 0xc7b0416 (0xb017 bytes) 2024-09-17T21:19:14.029 INFO:teuthology.orchestra.run.smithi184.stdout:2516 read 0x1250a00 thru 0x1257162 (0x6763 bytes) 2024-09-17T21:19:14.029 INFO:teuthology.orchestra.run.smithi184.stdout:2519 write 0x952ba00 thru 0x9530b2d (0x512e bytes) 2024-09-17T21:19:14.033 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:14.033 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:14.043 INFO:teuthology.orchestra.run.smithi184.stdout:2520 trunc from 0xc7b0600 to 0x4fba200 2024-09-17T21:19:14.047 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:19:14.092 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:14.091+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:14.126 INFO:teuthology.orchestra.run.smithi184.stdout:2521 write 0xd072c00 thru 0xd076432 (0x3833 bytes) 2024-09-17T21:19:14.128 INFO:teuthology.orchestra.run.smithi184.stdout:2522 read 0x8552a00 thru 0x855ac7d (0x827e bytes) 2024-09-17T21:19:14.128 INFO:teuthology.orchestra.run.smithi184.stdout:2525 write 0x899f800 thru 0x89a68ef (0x70f0 bytes) 2024-09-17T21:19:14.128 INFO:teuthology.orchestra.run.smithi184.stdout:2527 write 0x9619200 thru 0x961c8fa (0x36fb bytes) 2024-09-17T21:19:14.128 INFO:teuthology.orchestra.run.smithi184.stdout:2529 write 0x5f19a00 thru 0x5f251bf (0xb7c0 bytes) 2024-09-17T21:19:14.129 INFO:teuthology.orchestra.run.smithi184.stdout:2530 punch from 0x5558e00 to 0x555f200, (0x6400 bytes) 2024-09-17T21:19:14.164 INFO:teuthology.orchestra.run.smithi184.stdout:2531 write 0xe7f7200 thru 0xe800f25 (0x9d26 bytes) 2024-09-17T21:19:14.166 INFO:teuthology.orchestra.run.smithi184.stdout:2533 write 0xd41a600 thru 0xd41ceac (0x28ad bytes) 2024-09-17T21:19:14.166 INFO:teuthology.orchestra.run.smithi184.stdout:2534 read 0x9273400 thru 0x9279ec9 (0x6aca bytes) 2024-09-17T21:19:14.166 INFO:teuthology.orchestra.run.smithi184.stdout:2535 punch from 0x3530000 to 0x3539800, (0x9800 bytes) 2024-09-17T21:19:14.167 INFO:teuthology.orchestra.run.smithi184.stdout:2536 write 0x68d1200 thru 0x68dd44e (0xc24f bytes) 2024-09-17T21:19:14.167 INFO:teuthology.orchestra.run.smithi184.stdout:2537 read 0x3f3ec00 thru 0x3f4b47b (0xc87c bytes) 2024-09-17T21:19:14.167 INFO:teuthology.orchestra.run.smithi184.stdout:2543 punch from 0x3c5f600 to 0x3c6b600, (0xc000 bytes) 2024-09-17T21:19:14.168 INFO:teuthology.orchestra.run.smithi184.stdout:2544 punch from 0x900800 to 0x907e00, (0x7600 bytes) 2024-09-17T21:19:14.168 INFO:teuthology.orchestra.run.smithi184.stdout:2546 write 0xe703400 thru 0xe7069f3 (0x35f4 bytes) 2024-09-17T21:19:14.169 INFO:teuthology.orchestra.run.smithi184.stdout:2547 punch from 0xd939000 to 0xd940600, (0x7600 bytes) 2024-09-17T21:19:14.169 INFO:teuthology.orchestra.run.smithi184.stdout:2548 punch from 0xaabf000 to 0xaac3600, (0x4600 bytes) 2024-09-17T21:19:14.169 INFO:teuthology.orchestra.run.smithi184.stdout:2550 read 0xaba8a00 thru 0xabadab7 (0x50b8 bytes) 2024-09-17T21:19:14.172 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:14.168+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:14.173 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:14.173 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:14.174 INFO:teuthology.orchestra.run.smithi184.stdout:2551 trunc from 0xe801000 to 0xbceb600 2024-09-17T21:19:14.190 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:19:14.234 INFO:teuthology.orchestra.run.smithi184.stdout:2556 write 0x6e70a00 thru 0x6e78e0a (0x840b bytes) 2024-09-17T21:19:14.243 INFO:teuthology.orchestra.run.smithi184.stdout:2557 trunc from 0xbceb600 to 0x86dc000 2024-09-17T21:19:14.280 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:14.279+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:14.284 INFO:teuthology.orchestra.run.smithi184.stdout:2558 read 0x78cb000 thru 0x78cce36 (0x1e37 bytes) 2024-09-17T21:19:14.287 INFO:teuthology.orchestra.run.smithi184.stdout:2559 read 0x5e56c00 thru 0x5e5c6d9 (0x5ada bytes) 2024-09-17T21:19:14.287 INFO:teuthology.orchestra.run.smithi184.stdout:2560 write 0x768ea00 thru 0x7699464 (0xaa65 bytes) 2024-09-17T21:19:14.301 INFO:teuthology.orchestra.run.smithi184.stdout:2561 trunc from 0x86dc000 to 0x10d4e00 2024-09-17T21:19:14.359 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:14.359 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:14.373 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:19:14.374 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:14.368+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:14.405 INFO:teuthology.orchestra.run.smithi184.stdout:2563 write 0x8db9e00 thru 0x8dba6ce (0x8cf bytes) 2024-09-17T21:19:14.415 INFO:teuthology.orchestra.run.smithi184.stdout:2564 trunc from 0x8dba800 to 0x5b19e00 2024-09-17T21:19:14.453 INFO:teuthology.orchestra.run.smithi184.stdout:2565 trunc from 0x5b19e00 to 0x6f53000 2024-09-17T21:19:14.477 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:14.472+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:14.478 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T21:19:14.501 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T21:19:14.502 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:19:14.517 INFO:teuthology.orchestra.run.smithi184.stdout:2566 write 0xdb02200 thru 0xdb0a781 (0x8582 bytes) 2024-09-17T21:19:14.520 INFO:teuthology.orchestra.run.smithi184.stdout:2567 read 0xd03fe00 thru 0xd040301 (0x502 bytes) 2024-09-17T21:19:14.520 INFO:teuthology.orchestra.run.smithi184.stdout:2569 punch from 0xaa95800 to 0xaa9ae00, (0x5600 bytes) 2024-09-17T21:19:14.520 INFO:teuthology.orchestra.run.smithi184.stdout:2572 write 0x695a00 thru 0x6998e0 (0x3ee1 bytes) 2024-09-17T21:19:14.529 INFO:teuthology.orchestra.run.smithi184.stdout:2573 trunc from 0xdb0a800 to 0x905f600 2024-09-17T21:19:14.553 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:14.553 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:14.567 INFO:teuthology.orchestra.run.smithi184.stdout:2574 read 0x8be8800 thru 0x8bef1fd (0x69fe bytes) 2024-09-17T21:19:14.569 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:19:14.570 INFO:teuthology.orchestra.run.smithi184.stdout:2575 write 0x3f11e00 thru 0x3f1ebf1 (0xcdf2 bytes) 2024-09-17T21:19:14.570 INFO:teuthology.orchestra.run.smithi184.stdout:2576 read 0x3dca800 thru 0x3dcba53 (0x1254 bytes) 2024-09-17T21:19:14.575 INFO:teuthology.orchestra.run.smithi184.stdout:2577 trunc from 0x905f600 to 0x69d4000 2024-09-17T21:19:14.613 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:14.611+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:14.637 INFO:teuthology.orchestra.run.smithi184.stdout:2578 write 0xbbff000 thru 0xbc0c60e (0xd60f bytes) 2024-09-17T21:19:14.639 INFO:teuthology.orchestra.run.smithi184.stdout:2579 write 0x64e800 thru 0x654307 (0x5b08 bytes) 2024-09-17T21:19:14.639 INFO:teuthology.orchestra.run.smithi184.stdout:2581 write 0x41c0e00 thru 0x41c299d (0x1b9e bytes) 2024-09-17T21:19:14.639 INFO:teuthology.orchestra.run.smithi184.stdout:2582 punch from 0x3f1600 to 0x3fcc00, (0xb600 bytes) 2024-09-17T21:19:14.646 INFO:teuthology.orchestra.run.smithi184.stdout:2585 trunc from 0xbc0c800 to 0x7f64a00 2024-09-17T21:19:14.678 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:14.676+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:14.683 INFO:teuthology.orchestra.run.smithi184.stdout:2586 read 0x25c6c00 thru 0x25c8c52 (0x2053 bytes) 2024-09-17T21:19:14.685 INFO:teuthology.orchestra.run.smithi184.stdout:2587 read 0x2a88800 thru 0x2a8f84b (0x704c bytes) 2024-09-17T21:19:14.685 INFO:teuthology.orchestra.run.smithi184.stdout:2590 punch from 0x13ff200 to 0x1400c00, (0x1a00 bytes) 2024-09-17T21:19:14.685 INFO:teuthology.orchestra.run.smithi184.stdout:2592 punch from 0x71cfe00 to 0x71d3200, (0x3400 bytes) 2024-09-17T21:19:14.695 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:14.695 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:14.710 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:19:14.719 INFO:teuthology.orchestra.run.smithi184.stdout:2594 write 0xe0ee800 thru 0xe0f144a (0x2c4b bytes) 2024-09-17T21:19:14.723 INFO:teuthology.orchestra.run.smithi184.stdout:2595 write 0xd9f1800 thru 0xd9f48c5 (0x30c6 bytes) 2024-09-17T21:19:14.723 INFO:teuthology.orchestra.run.smithi184.stdout:2597 read 0xdc1400 thru 0xdcfc36 (0xe837 bytes) 2024-09-17T21:19:14.747 INFO:teuthology.orchestra.run.smithi184.stdout:2600 trunc from 0xe0f1600 to 0x4b22e00 2024-09-17T21:19:14.796 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:14.795+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:14.811 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:19:14.812 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T21:19:14.814 INFO:teuthology.orchestra.run.smithi184.stdout:2601 write 0x47a0400 thru 0x47a2079 (0x1c7a bytes) 2024-09-17T21:19:14.816 INFO:teuthology.orchestra.run.smithi184.stdout:2602 punch from 0x3e1aa00 to 0x3e27c00, (0xd200 bytes) 2024-09-17T21:19:14.816 INFO:teuthology.orchestra.run.smithi184.stdout:2603 punch from 0x49f8600 to 0x4a00800, (0x8200 bytes) 2024-09-17T21:19:14.817 INFO:teuthology.orchestra.run.smithi184.stdout:2605 write 0x3681000 thru 0x3690655 (0xf656 bytes) 2024-09-17T21:19:14.817 INFO:teuthology.orchestra.run.smithi184.stdout:2607 read 0xded200 thru 0xdfbff1 (0xedf2 bytes) 2024-09-17T21:19:14.818 INFO:teuthology.orchestra.run.smithi184.stdout:2608 read 0x26bb200 thru 0x26bd316 (0x2117 bytes) 2024-09-17T21:19:14.818 INFO:teuthology.orchestra.run.smithi184.stdout:2612 read 0x2779200 thru 0x2786b8e (0xd98f bytes) 2024-09-17T21:19:14.818 INFO:teuthology.orchestra.run.smithi184.stdout:2614 read 0x2138400 thru 0x2142293 (0x9e94 bytes) 2024-09-17T21:19:14.832 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-17T21:19:14.479952+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":679048584,"num_objects":389,"num_object_clones":83,"num_object_copies":778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":389,"num_whiteouts":56,"num_read":70119,"num_read_kb":310126,"num_write":4988,"num_write_kb":164923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":23376921,"num_keys_recovered":87,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4502,"ondisk_log_size":4502,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":319284,"kb_used_data":138396,"kb_used_omap":7,"kb_used_meta":180792,"kb_avail":471539916,"statfs":{"total":483183820800,"available":482856873984,"internally_reserved":0,"allocated":141717504,"data_stored":202958667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7978,"internal_metadata":185131222},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":20,"num_write":6,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.039196"},"pg_stats":[{"pgid":"3.1f","version":"61'71","reported_seq":757,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447704+0000","last_change":"2024-09-17T21:18:40.931000+0000","last_active":"2024-09-17T21:19:09.447704+0000","last_peered":"2024-09-17T21:19:09.447704+0000","last_clean":"2024-09-17T21:19:09.447704+0000","last_became_active":"2024-09-17T21:18:40.810689+0000","last_became_peered":"2024-09-17T21:18:40.810689+0000","last_unstale":"2024-09-17T21:19:09.447704+0000","last_undegraded":"2024-09-17T21:19:09.447704+0000","last_fullsized":"2024-09-17T21:19:09.447704+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:31:56.719040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088525599999999996,"stat_sum":{"num_bytes":4213859,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":712,"num_read_kb":5893,"num_write":89,"num_write_kb":875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":96832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"61'155","reported_seq":1603,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.448122+0000","last_change":"2024-09-17T21:18:43.278035+0000","last_active":"2024-09-17T21:19:09.448122+0000","last_peered":"2024-09-17T21:19:09.448122+0000","last_clean":"2024-09-17T21:19:09.448122+0000","last_became_active":"2024-09-17T21:18:43.277411+0000","last_became_peered":"2024-09-17T21:18:43.277411+0000","last_unstale":"2024-09-17T21:19:09.448122+0000","last_undegraded":"2024-09-17T21:19:09.448122+0000","last_fullsized":"2024-09-17T21:19:09.448122+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:31.348355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013256229999999999,"stat_sum":{"num_bytes":7273514,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":2215,"num_read_kb":6212,"num_write":164,"num_write_kb":629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":127572,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"65'196","reported_seq":2610,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.440094+0000","last_change":"2024-09-17T21:19:05.417322+0000","last_active":"2024-09-17T21:19:09.440094+0000","last_peered":"2024-09-17T21:19:09.440094+0000","last_clean":"2024-09-17T21:19:09.440094+0000","last_became_active":"2024-09-17T21:19:05.415967+0000","last_became_peered":"2024-09-17T21:19:05.415967+0000","last_unstale":"2024-09-17T21:19:09.440094+0000","last_undegraded":"2024-09-17T21:19:09.440094+0000","last_fullsized":"2024-09-17T21:19:09.440094+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:37.433255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018973079999999999,"stat_sum":{"num_bytes":23801924,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":2700,"num_read_kb":9461,"num_write":214,"num_write_kb":6907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"62'40","reported_seq":402,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.475561+0000","last_change":"2024-09-17T21:19:05.416924+0000","last_active":"2024-09-17T21:19:09.475561+0000","last_peered":"2024-09-17T21:19:09.475561+0000","last_clean":"2024-09-17T21:19:09.475561+0000","last_became_active":"2024-09-17T21:19:05.416249+0000","last_became_peered":"2024-09-17T21:19:05.416249+0000","last_unstale":"2024-09-17T21:19:09.475561+0000","last_undegraded":"2024-09-17T21:19:09.475561+0000","last_fullsized":"2024-09-17T21:19:09.475561+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:32:09.962627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034186630000000002,"stat_sum":{"num_bytes":3104256,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":276,"num_read_kb":2317,"num_write":42,"num_write_kb":1295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"66'234","reported_seq":2959,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.482178+0000","last_change":"2024-09-17T21:18:41.001528+0000","last_active":"2024-09-17T21:19:09.482178+0000","last_peered":"2024-09-17T21:19:09.482178+0000","last_clean":"2024-09-17T21:19:09.482178+0000","last_became_active":"2024-09-17T21:18:40.827008+0000","last_became_peered":"2024-09-17T21:18:40.827008+0000","last_unstale":"2024-09-17T21:19:09.482178+0000","last_undegraded":"2024-09-17T21:19:09.482178+0000","last_fullsized":"2024-09-17T21:19:09.482178+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:13:56.566421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089504999999999999,"stat_sum":{"num_bytes":10551808,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":2,"num_read":4965,"num_read_kb":5766,"num_write":141,"num_write_kb":1065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":615936,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"61'197","reported_seq":3519,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.481609+0000","last_change":"2024-09-17T21:18:13.314063+0000","last_active":"2024-09-17T21:19:09.481609+0000","last_peered":"2024-09-17T21:19:09.481609+0000","last_clean":"2024-09-17T21:19:09.481609+0000","last_became_active":"2024-09-17T21:18:11.301426+0000","last_became_peered":"2024-09-17T21:18:11.301426+0000","last_unstale":"2024-09-17T21:19:09.481609+0000","last_undegraded":"2024-09-17T21:19:09.481609+0000","last_fullsized":"2024-09-17T21:19:09.481609+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:26.981484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027366090000000001,"stat_sum":{"num_bytes":14090824,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":3,"num_read":4723,"num_read_kb":8928,"num_write":176,"num_write_kb":1273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"65'90","reported_seq":2166,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.443643+0000","last_change":"2024-09-17T21:18:13.314862+0000","last_active":"2024-09-17T21:19:09.443643+0000","last_peered":"2024-09-17T21:19:09.443643+0000","last_clean":"2024-09-17T21:19:09.443643+0000","last_became_active":"2024-09-17T21:18:11.300793+0000","last_became_peered":"2024-09-17T21:18:11.300793+0000","last_unstale":"2024-09-17T21:19:09.443643+0000","last_undegraded":"2024-09-17T21:19:09.443643+0000","last_fullsized":"2024-09-17T21:19:09.443643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:15:33.881019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047673000000000002,"stat_sum":{"num_bytes":23540736,"num_objects":18,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":2010,"num_read_kb":9716,"num_write":107,"num_write_kb":2412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"61'98","reported_seq":271,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.443669+0000","last_change":"2024-09-17T21:18:40.830426+0000","last_active":"2024-09-17T21:19:09.443669+0000","last_peered":"2024-09-17T21:19:09.443669+0000","last_clean":"2024-09-17T21:19:09.443669+0000","last_became_active":"2024-09-17T21:18:40.801216+0000","last_became_peered":"2024-09-17T21:18:40.801216+0000","last_unstale":"2024-09-17T21:19:09.443669+0000","last_undegraded":"2024-09-17T21:19:09.443669+0000","last_fullsized":"2024-09-17T21:19:09.443669+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:06:56.249183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10079792,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":881,"num_read_kb":3803,"num_write":135,"num_write_kb":1237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":428592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"68'74","reported_seq":1055,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T21:19:13.478359+0000","last_change":"2024-09-17T21:19:05.419944+0000","last_active":"2024-09-17T21:19:13.478359+0000","last_peered":"2024-09-17T21:19:13.478359+0000","last_clean":"2024-09-17T21:19:13.478359+0000","last_became_active":"2024-09-17T21:19:05.418983+0000","last_became_peered":"2024-09-17T21:19:05.418983+0000","last_unstale":"2024-09-17T21:19:13.478359+0000","last_undegraded":"2024-09-17T21:19:13.478359+0000","last_fullsized":"2024-09-17T21:19:13.478359+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:58:01.226365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088962999999999998,"stat_sum":{"num_bytes":28542976,"num_objects":13,"num_object_clones":5,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":2,"num_read":891,"num_read_kb":5068,"num_write":80,"num_write_kb":1949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":793088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"61'104","reported_seq":1031,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447899+0000","last_change":"2024-09-17T21:19:05.411424+0000","last_active":"2024-09-17T21:19:09.447899+0000","last_peered":"2024-09-17T21:19:09.447899+0000","last_clean":"2024-09-17T21:19:09.447899+0000","last_became_active":"2024-09-17T21:19:05.410746+0000","last_became_peered":"2024-09-17T21:19:05.410746+0000","last_unstale":"2024-09-17T21:19:09.447899+0000","last_undegraded":"2024-09-17T21:19:09.447899+0000","last_fullsized":"2024-09-17T21:19:09.447899+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:43:56.625042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001825658,"stat_sum":{"num_bytes":13074506,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1169,"num_read_kb":5431,"num_write":170,"num_write_kb":10737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"66'478","reported_seq":3496,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.448157+0000","last_change":"2024-09-17T21:18:41.006288+0000","last_active":"2024-09-17T21:19:09.448157+0000","last_peered":"2024-09-17T21:19:09.448157+0000","last_clean":"2024-09-17T21:19:09.448157+0000","last_became_active":"2024-09-17T21:18:40.820709+0000","last_became_peered":"2024-09-17T21:18:40.820709+0000","last_unstale":"2024-09-17T21:19:09.448157+0000","last_undegraded":"2024-09-17T21:19:09.448157+0000","last_fullsized":"2024-09-17T21:19:09.448157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:24:07.745065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027112149999999999,"stat_sum":{"num_bytes":53657712,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":2,"num_read":4722,"num_read_kb":19835,"num_write":755,"num_write_kb":3384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1160742,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"65'98","reported_seq":1293,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447821+0000","last_change":"2024-09-17T21:18:13.307234+0000","last_active":"2024-09-17T21:19:09.447821+0000","last_peered":"2024-09-17T21:19:09.447821+0000","last_clean":"2024-09-17T21:19:09.447821+0000","last_became_active":"2024-09-17T21:18:11.299817+0000","last_became_peered":"2024-09-17T21:18:11.299817+0000","last_unstale":"2024-09-17T21:19:09.447821+0000","last_undegraded":"2024-09-17T21:19:09.447821+0000","last_fullsized":"2024-09-17T21:19:09.447821+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:44:24.083134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002281113,"stat_sum":{"num_bytes":33669911,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":1,"num_read":1146,"num_read_kb":5053,"num_write":91,"num_write_kb":11916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447912+0000","last_change":"2024-09-17T21:18:03.059715+0000","last_active":"2024-09-17T21:19:09.447912+0000","last_peered":"2024-09-17T21:19:09.447912+0000","last_clean":"2024-09-17T21:19:09.447912+0000","last_became_active":"2024-09-17T21:18:01.213047+0000","last_became_peered":"2024-09-17T21:18:01.213047+0000","last_unstale":"2024-09-17T21:19:09.447912+0000","last_undegraded":"2024-09-17T21:19:09.447912+0000","last_fullsized":"2024-09-17T21:19:09.447912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:46:35.222373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"61'162","reported_seq":1201,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.444282+0000","last_change":"2024-09-17T21:18:40.976309+0000","last_active":"2024-09-17T21:19:09.444282+0000","last_peered":"2024-09-17T21:19:09.444282+0000","last_clean":"2024-09-17T21:19:09.444282+0000","last_became_active":"2024-09-17T21:18:40.818354+0000","last_became_peered":"2024-09-17T21:18:40.818354+0000","last_unstale":"2024-09-17T21:19:09.444282+0000","last_undegraded":"2024-09-17T21:19:09.444282+0000","last_fullsized":"2024-09-17T21:19:09.444282+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:23:40.750668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084327200000000005,"stat_sum":{"num_bytes":21660498,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":1574,"num_read_kb":20376,"num_write":262,"num_write_kb":13149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8773670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":110,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.444391+0000","last_change":"2024-09-17T21:18:03.073440+0000","last_active":"2024-09-17T21:19:09.444391+0000","last_peered":"2024-09-17T21:19:09.444391+0000","last_clean":"2024-09-17T21:19:09.444391+0000","last_became_active":"2024-09-17T21:18:01.048806+0000","last_became_peered":"2024-09-17T21:18:01.048806+0000","last_unstale":"2024-09-17T21:19:09.444391+0000","last_undegraded":"2024-09-17T21:19:09.444391+0000","last_fullsized":"2024-09-17T21:19:09.444391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:26.487163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094161400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"61'95","reported_seq":1360,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.444596+0000","last_change":"2024-09-17T21:18:17.280432+0000","last_active":"2024-09-17T21:19:09.444596+0000","last_peered":"2024-09-17T21:19:09.444596+0000","last_clean":"2024-09-17T21:19:09.444596+0000","last_became_active":"2024-09-17T21:18:17.279625+0000","last_became_peered":"2024-09-17T21:18:17.279625+0000","last_unstale":"2024-09-17T21:19:09.444596+0000","last_undegraded":"2024-09-17T21:19:09.444596+0000","last_fullsized":"2024-09-17T21:19:09.444596+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:13:46.386831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22065247,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":1353,"num_read_kb":17234,"num_write":119,"num_write_kb":11269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":116,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447895+0000","last_change":"2024-09-17T21:18:40.821956+0000","last_active":"2024-09-17T21:19:09.447895+0000","last_peered":"2024-09-17T21:19:09.447895+0000","last_clean":"2024-09-17T21:19:09.447895+0000","last_became_active":"2024-09-17T21:18:40.820574+0000","last_became_peered":"2024-09-17T21:18:40.820574+0000","last_unstale":"2024-09-17T21:19:09.447895+0000","last_undegraded":"2024-09-17T21:19:09.447895+0000","last_fullsized":"2024-09-17T21:19:09.447895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:35:05.837282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099219900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"61'54","reported_seq":1427,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.444150+0000","last_change":"2024-09-17T21:18:13.315662+0000","last_active":"2024-09-17T21:19:09.444150+0000","last_peered":"2024-09-17T21:19:09.444150+0000","last_clean":"2024-09-17T21:19:09.444150+0000","last_became_active":"2024-09-17T21:18:11.309133+0000","last_became_peered":"2024-09-17T21:18:11.309133+0000","last_unstale":"2024-09-17T21:19:09.444150+0000","last_undegraded":"2024-09-17T21:19:09.444150+0000","last_fullsized":"2024-09-17T21:19:09.444150+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:42:12.633242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078465700000000002,"stat_sum":{"num_bytes":23857152,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":1291,"num_read_kb":12784,"num_write":62,"num_write_kb":6160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":116,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.444255+0000","last_change":"2024-09-17T21:18:40.818736+0000","last_active":"2024-09-17T21:19:09.444255+0000","last_peered":"2024-09-17T21:19:09.444255+0000","last_clean":"2024-09-17T21:19:09.444255+0000","last_became_active":"2024-09-17T21:18:40.818309+0000","last_became_peered":"2024-09-17T21:18:40.818309+0000","last_unstale":"2024-09-17T21:19:09.444255+0000","last_undegraded":"2024-09-17T21:19:09.444255+0000","last_fullsized":"2024-09-17T21:19:09.444255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:42:22.148483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006222,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"61'299","reported_seq":4815,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.450892+0000","last_change":"2024-09-17T21:18:13.327202+0000","last_active":"2024-09-17T21:19:09.450892+0000","last_peered":"2024-09-17T21:19:09.450892+0000","last_clean":"2024-09-17T21:19:09.450892+0000","last_became_active":"2024-09-17T21:18:11.317127+0000","last_became_peered":"2024-09-17T21:18:11.317127+0000","last_unstale":"2024-09-17T21:19:09.450892+0000","last_undegraded":"2024-09-17T21:19:09.450892+0000","last_fullsized":"2024-09-17T21:19:09.450892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:43:54.053747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036493720000000001,"stat_sum":{"num_bytes":57645087,"num_objects":19,"num_object_clones":6,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":1,"num_read":6702,"num_read_kb":18001,"num_write":207,"num_write_kb":15267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"61'54","reported_seq":671,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.440305+0000","last_change":"2024-09-17T21:18:40.963953+0000","last_active":"2024-09-17T21:19:09.440305+0000","last_peered":"2024-09-17T21:19:09.440305+0000","last_clean":"2024-09-17T21:19:09.440305+0000","last_became_active":"2024-09-17T21:18:40.820910+0000","last_became_peered":"2024-09-17T21:18:40.820910+0000","last_unstale":"2024-09-17T21:19:09.440305+0000","last_undegraded":"2024-09-17T21:19:09.440305+0000","last_fullsized":"2024-09-17T21:19:09.440305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:16:23.135778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084988700000000002,"stat_sum":{"num_bytes":5107712,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":492,"num_read_kb":2316,"num_write":55,"num_write_kb":1813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":978944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":32,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.444243+0000","last_change":"2024-09-17T21:19:05.415157+0000","last_active":"2024-09-17T21:19:09.444243+0000","last_peered":"2024-09-17T21:19:09.444243+0000","last_clean":"2024-09-17T21:19:09.444243+0000","last_became_active":"2024-09-17T21:19:05.414343+0000","last_became_peered":"2024-09-17T21:19:05.414343+0000","last_unstale":"2024-09-17T21:19:09.444243+0000","last_undegraded":"2024-09-17T21:19:09.444243+0000","last_fullsized":"2024-09-17T21:19:09.444243+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:27:06.755871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"61'32","reported_seq":493,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.443911+0000","last_change":"2024-09-17T21:18:13.314742+0000","last_active":"2024-09-17T21:19:09.443911+0000","last_peered":"2024-09-17T21:19:09.443911+0000","last_clean":"2024-09-17T21:19:09.443911+0000","last_became_active":"2024-09-17T21:18:11.308568+0000","last_became_peered":"2024-09-17T21:18:11.308568+0000","last_unstale":"2024-09-17T21:19:09.443911+0000","last_undegraded":"2024-09-17T21:19:09.443911+0000","last_fullsized":"2024-09-17T21:19:09.443911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:01:26.289237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049125000000000004,"stat_sum":{"num_bytes":9221152,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":368,"num_read_kb":7190,"num_write":39,"num_write_kb":2902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":88,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.443631+0000","last_change":"2024-09-17T21:18:17.280358+0000","last_active":"2024-09-17T21:19:09.443631+0000","last_peered":"2024-09-17T21:19:09.443631+0000","last_clean":"2024-09-17T21:19:09.443631+0000","last_became_active":"2024-09-17T21:18:17.279474+0000","last_became_peered":"2024-09-17T21:18:17.279474+0000","last_unstale":"2024-09-17T21:19:09.443631+0000","last_undegraded":"2024-09-17T21:19:09.443631+0000","last_fullsized":"2024-09-17T21:19:09.443631+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:55:54.183527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"61'68","reported_seq":748,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.439826+0000","last_change":"2024-09-17T21:18:40.906152+0000","last_active":"2024-09-17T21:19:09.439826+0000","last_peered":"2024-09-17T21:19:09.439826+0000","last_clean":"2024-09-17T21:19:09.439826+0000","last_became_active":"2024-09-17T21:18:40.816571+0000","last_became_peered":"2024-09-17T21:18:40.816571+0000","last_unstale":"2024-09-17T21:19:09.439826+0000","last_undegraded":"2024-09-17T21:19:09.439826+0000","last_fullsized":"2024-09-17T21:19:09.439826+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:11:26.159680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066398399999999995,"stat_sum":{"num_bytes":7944240,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":772,"num_read_kb":7750,"num_write":100,"num_write_kb":4353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":4070960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":110,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447467+0000","last_change":"2024-09-17T21:18:03.059156+0000","last_active":"2024-09-17T21:19:09.447467+0000","last_peered":"2024-09-17T21:19:09.447467+0000","last_clean":"2024-09-17T21:19:09.447467+0000","last_became_active":"2024-09-17T21:18:01.049180+0000","last_became_peered":"2024-09-17T21:18:01.049180+0000","last_unstale":"2024-09-17T21:19:09.447467+0000","last_undegraded":"2024-09-17T21:19:09.447467+0000","last_fullsized":"2024-09-17T21:19:09.447467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:27.069794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077408,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"62'217","reported_seq":1008,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.444202+0000","last_change":"2024-09-17T21:18:13.315294+0000","last_active":"2024-09-17T21:19:09.444202+0000","last_peered":"2024-09-17T21:19:09.444202+0000","last_clean":"2024-09-17T21:19:09.444202+0000","last_became_active":"2024-09-17T21:18:11.294154+0000","last_became_peered":"2024-09-17T21:18:11.294154+0000","last_unstale":"2024-09-17T21:19:09.444202+0000","last_undegraded":"2024-09-17T21:19:09.444202+0000","last_fullsized":"2024-09-17T21:19:09.444202+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:16:11.502838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049408099999999995,"stat_sum":{"num_bytes":34535719,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1672,"num_read_kb":20106,"num_write":377,"num_write_kb":9147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":32,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T21:19:13.478301+0000","last_change":"2024-09-17T21:19:05.419660+0000","last_active":"2024-09-17T21:19:13.478301+0000","last_peered":"2024-09-17T21:19:13.478301+0000","last_clean":"2024-09-17T21:19:13.478301+0000","last_became_active":"2024-09-17T21:19:05.419056+0000","last_became_peered":"2024-09-17T21:19:05.419056+0000","last_unstale":"2024-09-17T21:19:13.478301+0000","last_undegraded":"2024-09-17T21:19:13.478301+0000","last_fullsized":"2024-09-17T21:19:13.478301+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:32:04.987802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":213,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447628+0000","last_change":"2024-09-17T21:18:40.829967+0000","last_active":"2024-09-17T21:19:09.447628+0000","last_peered":"2024-09-17T21:19:09.447628+0000","last_clean":"2024-09-17T21:19:09.447628+0000","last_became_active":"2024-09-17T21:18:40.803358+0000","last_became_peered":"2024-09-17T21:18:40.803358+0000","last_unstale":"2024-09-17T21:19:09.447628+0000","last_undegraded":"2024-09-17T21:19:09.447628+0000","last_fullsized":"2024-09-17T21:19:09.447628+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_clean_scrub_stamp":"2024-09-17T21:17:55.982339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:57:35.748912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"65'190","reported_seq":3539,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447542+0000","last_change":"2024-09-17T21:18:50.296660+0000","last_active":"2024-09-17T21:19:09.447542+0000","last_peered":"2024-09-17T21:19:09.447542+0000","last_clean":"2024-09-17T21:19:09.447542+0000","last_became_active":"2024-09-17T21:18:50.295377+0000","last_became_peered":"2024-09-17T21:18:50.295377+0000","last_unstale":"2024-09-17T21:19:09.447542+0000","last_undegraded":"2024-09-17T21:19:09.447542+0000","last_fullsized":"2024-09-17T21:19:09.447542+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:12:40.845223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22410256,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":4607,"num_read_kb":29652,"num_write":141,"num_write_kb":14993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3239440,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"61'174","reported_seq":1245,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.440059+0000","last_change":"2024-09-17T21:18:13.313980+0000","last_active":"2024-09-17T21:19:09.440059+0000","last_peered":"2024-09-17T21:19:09.440059+0000","last_clean":"2024-09-17T21:19:09.440059+0000","last_became_active":"2024-09-17T21:18:11.304021+0000","last_became_peered":"2024-09-17T21:18:11.304021+0000","last_unstale":"2024-09-17T21:19:09.440059+0000","last_undegraded":"2024-09-17T21:19:09.440059+0000","last_fullsized":"2024-09-17T21:19:09.440059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:48:29.828438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023749890000000001,"stat_sum":{"num_bytes":23323689,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1633,"num_read_kb":7526,"num_write":311,"num_write_kb":2811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"62'65","reported_seq":651,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447581+0000","last_change":"2024-09-17T21:18:13.307298+0000","last_active":"2024-09-17T21:19:09.447581+0000","last_peered":"2024-09-17T21:19:09.447581+0000","last_clean":"2024-09-17T21:19:09.447581+0000","last_became_active":"2024-09-17T21:18:11.294907+0000","last_became_peered":"2024-09-17T21:18:11.294907+0000","last_unstale":"2024-09-17T21:19:09.447581+0000","last_undegraded":"2024-09-17T21:19:09.447581+0000","last_fullsized":"2024-09-17T21:19:09.447581+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:52:34.160289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021951829999999999,"stat_sum":{"num_bytes":34581090,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":2,"num_read":522,"num_read_kb":5302,"num_write":64,"num_write_kb":648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"65'151","reported_seq":1523,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.444525+0000","last_change":"2024-09-17T21:18:40.868870+0000","last_active":"2024-09-17T21:19:09.444525+0000","last_peered":"2024-09-17T21:19:09.444525+0000","last_clean":"2024-09-17T21:19:09.444525+0000","last_became_active":"2024-09-17T21:18:40.816043+0000","last_became_peered":"2024-09-17T21:18:40.816043+0000","last_unstale":"2024-09-17T21:19:09.444525+0000","last_undegraded":"2024-09-17T21:19:09.444525+0000","last_fullsized":"2024-09-17T21:19:09.444525+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:29:59.626472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048704800000000002,"stat_sum":{"num_bytes":26157599,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":1,"num_read":2244,"num_read_kb":6312,"num_write":128,"num_write_kb":11534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":485392,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"61'103","reported_seq":98,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.443984+0000","last_change":"2024-09-17T21:18:54.392198+0000","last_active":"2024-09-17T21:19:09.443984+0000","last_peered":"2024-09-17T21:19:09.443984+0000","last_clean":"2024-09-17T21:19:09.443984+0000","last_became_active":"2024-09-17T21:18:54.343340+0000","last_became_peered":"2024-09-17T21:18:54.343340+0000","last_unstale":"2024-09-17T21:19:09.443984+0000","last_undegraded":"2024-09-17T21:19:09.443984+0000","last_fullsized":"2024-09-17T21:19:09.443984+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:58:42.535973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015319789999999999,"stat_sum":{"num_bytes":26867200,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":2060,"num_read_kb":6588,"num_write":83,"num_write_kb":758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":166400,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"61'22","reported_seq":174,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447924+0000","last_change":"2024-09-17T21:18:13.306351+0000","last_active":"2024-09-17T21:19:09.447924+0000","last_peered":"2024-09-17T21:19:09.447924+0000","last_clean":"2024-09-17T21:19:09.447924+0000","last_became_active":"2024-09-17T21:18:11.316999+0000","last_became_peered":"2024-09-17T21:18:11.316999+0000","last_unstale":"2024-09-17T21:19:09.447924+0000","last_undegraded":"2024-09-17T21:19:09.447924+0000","last_fullsized":"2024-09-17T21:19:09.447924+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:13.482512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015238949999999999,"stat_sum":{"num_bytes":723968,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":1011,"num_write":23,"num_write_kb":818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"62'264","reported_seq":5890,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.499227+0000","last_change":"2024-09-17T21:19:05.416474+0000","last_active":"2024-09-17T21:19:09.499227+0000","last_peered":"2024-09-17T21:19:09.499227+0000","last_clean":"2024-09-17T21:19:09.499227+0000","last_became_active":"2024-09-17T21:19:05.415977+0000","last_became_peered":"2024-09-17T21:19:05.415977+0000","last_unstale":"2024-09-17T21:19:09.499227+0000","last_undegraded":"2024-09-17T21:19:09.499227+0000","last_fullsized":"2024-09-17T21:19:09.499227+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:11:58.398387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003632209,"stat_sum":{"num_bytes":29204992,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":6436,"num_read_kb":21545,"num_write":197,"num_write_kb":15776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"62'62","reported_seq":1348,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447940+0000","last_change":"2024-09-17T21:19:05.410271+0000","last_active":"2024-09-17T21:19:09.447940+0000","last_peered":"2024-09-17T21:19:09.447940+0000","last_clean":"2024-09-17T21:19:09.447940+0000","last_became_active":"2024-09-17T21:19:05.409905+0000","last_became_peered":"2024-09-17T21:19:05.409905+0000","last_unstale":"2024-09-17T21:19:09.447940+0000","last_undegraded":"2024-09-17T21:19:09.447940+0000","last_fullsized":"2024-09-17T21:19:09.447940+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:33:06.031026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002454758,"stat_sum":{"num_bytes":2159631,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":1661,"num_read_kb":3227,"num_write":43,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"61'77","reported_seq":2681,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.443563+0000","last_change":"2024-09-17T21:19:05.414774+0000","last_active":"2024-09-17T21:19:09.443563+0000","last_peered":"2024-09-17T21:19:09.443563+0000","last_clean":"2024-09-17T21:19:09.443563+0000","last_became_active":"2024-09-17T21:19:05.413373+0000","last_became_peered":"2024-09-17T21:19:05.413373+0000","last_unstale":"2024-09-17T21:19:09.443563+0000","last_undegraded":"2024-09-17T21:19:09.443563+0000","last_fullsized":"2024-09-17T21:19:09.443563+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:24:21.294368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001519261,"stat_sum":{"num_bytes":1272906,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1526,"num_read_kb":4271,"num_write":50,"num_write_kb":919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":538698,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"65'196","reported_seq":2008,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447522+0000","last_change":"2024-09-17T21:18:13.305680+0000","last_active":"2024-09-17T21:19:09.447522+0000","last_peered":"2024-09-17T21:19:09.447522+0000","last_clean":"2024-09-17T21:19:09.447522+0000","last_became_active":"2024-09-17T21:18:11.299946+0000","last_became_peered":"2024-09-17T21:18:11.299946+0000","last_unstale":"2024-09-17T21:19:09.447522+0000","last_undegraded":"2024-09-17T21:19:09.447522+0000","last_fullsized":"2024-09-17T21:19:09.447522+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:52.766042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225884,"stat_sum":{"num_bytes":33520477,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":2,"num_read":2072,"num_read_kb":8553,"num_write":217,"num_write_kb":2540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"61'73","reported_seq":1424,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447774+0000","last_change":"2024-09-17T21:18:40.967754+0000","last_active":"2024-09-17T21:19:09.447774+0000","last_peered":"2024-09-17T21:19:09.447774+0000","last_clean":"2024-09-17T21:19:09.447774+0000","last_became_active":"2024-09-17T21:18:40.814521+0000","last_became_peered":"2024-09-17T21:18:40.814521+0000","last_unstale":"2024-09-17T21:19:09.447774+0000","last_undegraded":"2024-09-17T21:19:09.447774+0000","last_fullsized":"2024-09-17T21:19:09.447774+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:48:50.619550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013045489999999999,"stat_sum":{"num_bytes":20655695,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":2,"num_read":1256,"num_read_kb":5704,"num_write":81,"num_write_kb":2386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1310287,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"66'234","reported_seq":4670,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.484210+0000","last_change":"2024-09-17T21:18:13.306520+0000","last_active":"2024-09-17T21:19:09.484210+0000","last_peered":"2024-09-17T21:19:09.484210+0000","last_clean":"2024-09-17T21:19:09.484210+0000","last_became_active":"2024-09-17T21:18:11.294953+0000","last_became_peered":"2024-09-17T21:18:11.294953+0000","last_unstale":"2024-09-17T21:19:09.484210+0000","last_undegraded":"2024-09-17T21:19:09.484210+0000","last_fullsized":"2024-09-17T21:19:09.484210+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:33:39.961492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019790290000000002,"stat_sum":{"num_bytes":49942069,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":2,"num_read":5364,"num_read_kb":17140,"num_write":162,"num_write_kb":2255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":678458197,"num_objects":385,"num_object_clones":83,"num_object_copies":770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":385,"num_whiteouts":56,"num_read":70053,"num_read_kb":310071,"num_write":4885,"num_write_kb":163293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":22786553,"num_keys_recovered":87,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":165421056,"data_stored":237171957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4427,"ondisk_log_size":4427,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62816,"kb_used_data":27472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94309024,"statfs":{"total":96636764160,"available":96572440576,"internally_reserved":0,"allocated":28131328,"data_stored":45197087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74384,"kb_used_data":34880,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94297456,"statfs":{"total":96636764160,"available":96560594944,"internally_reserved":0,"allocated":35717120,"data_stored":48655979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":40434116},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50404,"kb_used_data":15060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94321436,"statfs":{"total":96636764160,"available":96585150464,"internally_reserved":0,"allocated":15421440,"data_stored":16373642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68924,"kb_used_data":33576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94302916,"statfs":{"total":96636764160,"available":96566185984,"internally_reserved":0,"allocated":34381824,"data_stored":52123434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":0,"seq":55834574866,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607572,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62756,"kb_used_data":27408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94309084,"statfs":{"total":96636764160,"available":96572502016,"internally_reserved":0,"allocated":28065792,"data_stored":40608525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28000256,"data_stored":36273087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33046528,"data_stored":51242024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27869184,"data_stored":40856656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":14131200,"data_stored":15529152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":34979840,"data_stored":48364956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":27394048,"data_stored":44906082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:19:14.834 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:19:14.857 INFO:teuthology.orchestra.run.smithi184.stdout:2615 write 0x9d78000 thru 0x9d86e09 (0xee0a bytes) 2024-09-17T21:19:14.870 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:14.870 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:14.872 INFO:teuthology.orchestra.run.smithi184.stdout:2616 trunc from 0x9d87000 to 0x1555a00 2024-09-17T21:19:14.887 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:19:14.887 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:14.883+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:14.916 INFO:teuthology.orchestra.run.smithi184.stdout:2617 punch from 0x1a3800 to 0x1ac600, (0x8e00 bytes) 2024-09-17T21:19:14.918 INFO:teuthology.orchestra.run.smithi184.stdout:2619 read 0x4c5600 thru 0x4cc5ac (0x6fad bytes) 2024-09-17T21:19:14.918 INFO:teuthology.orchestra.run.smithi184.stdout:2620 read 0xdac000 thru 0xdace64 (0xe65 bytes) 2024-09-17T21:19:14.918 INFO:teuthology.orchestra.run.smithi184.stdout:2621 read 0x1200a00 thru 0x120539b (0x499c bytes) 2024-09-17T21:19:14.975 INFO:teuthology.orchestra.run.smithi184.stdout:2622 write 0xed2ba00 thru 0xed38371 (0xc972 bytes) 2024-09-17T21:19:14.977 INFO:teuthology.orchestra.run.smithi184.stdout:2625 punch from 0xd32ea00 to 0xd334c00, (0x6200 bytes) 2024-09-17T21:19:14.977 INFO:teuthology.orchestra.run.smithi184.stdout:2628 read 0x37bce00 thru 0x37c6226 (0x9427 bytes) 2024-09-17T21:19:14.978 INFO:teuthology.orchestra.run.smithi184.stdout:2631 read 0xc015000 thru 0xc015061 (0x62 bytes) 2024-09-17T21:19:14.978 INFO:teuthology.orchestra.run.smithi184.stdout:2632 write 0x3a1400 thru 0x3a8ed8 (0x7ad9 bytes) 2024-09-17T21:19:14.978 INFO:teuthology.orchestra.run.smithi184.stdout:2633 write 0x52fba00 thru 0x52fbf5f (0x560 bytes) 2024-09-17T21:19:14.978 INFO:teuthology.orchestra.run.smithi184.stdout:2634 read 0xba76a00 thru 0xba7a7a9 (0x3daa bytes) 2024-09-17T21:19:14.978 INFO:teuthology.orchestra.run.smithi184.stdout:2635 write 0x3818e00 thru 0x381c7b6 (0x39b7 bytes) 2024-09-17T21:19:14.978 INFO:teuthology.orchestra.run.smithi184.stdout:2636 read 0x2cf5000 thru 0x2d0387f (0xe880 bytes) 2024-09-17T21:19:14.980 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:14.979+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:14.997 INFO:teuthology.orchestra.run.smithi184.stdout:2637 trunc from 0xed38400 to 0x4f16600 2024-09-17T21:19:15.040 INFO:teuthology.orchestra.run.smithi184.stdout:2640 read 0x3094a00 thru 0x309f9c2 (0xafc3 bytes) 2024-09-17T21:19:15.042 INFO:teuthology.orchestra.run.smithi184.stdout:2644 read 0x326c600 thru 0x32795c2 (0xcfc3 bytes) 2024-09-17T21:19:15.045 INFO:teuthology.orchestra.run.smithi184.stdout:2649 trunc from 0x4f16600 to 0x600b200 2024-09-17T21:19:15.048 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:15.049 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:15.062 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:19:15.069 INFO:teuthology.orchestra.run.smithi184.stdout:2651 read 0x590dc00 thru 0x591d998 (0xfd99 bytes) 2024-09-17T21:19:15.071 INFO:teuthology.orchestra.run.smithi184.stdout:2652 read 0x2416400 thru 0x241ab46 (0x4747 bytes) 2024-09-17T21:19:15.071 INFO:teuthology.orchestra.run.smithi184.stdout:2653 write 0x2908000 thru 0x290d613 (0x5614 bytes) 2024-09-17T21:19:15.104 INFO:teuthology.orchestra.run.smithi184.stdout:2654 write 0xc09ba00 thru 0xc09f620 (0x3c21 bytes) 2024-09-17T21:19:15.105 INFO:teuthology.orchestra.run.smithi184.stdout:2655 punch from 0x7663e00 to 0x766b600, (0x7800 bytes) 2024-09-17T21:19:15.106 INFO:teuthology.orchestra.run.smithi184.stdout:2656 punch from 0xad13a00 to 0xad1ae00, (0x7400 bytes) 2024-09-17T21:19:15.109 INFO:teuthology.orchestra.run.smithi184.stdout:2658 trunc from 0xc09f800 to 0xa8ab400 2024-09-17T21:19:15.109 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:15.107+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:15.135 INFO:teuthology.orchestra.run.smithi184.stdout:2661 read 0x244800 thru 0x24a9a7 (0x61a8 bytes) 2024-09-17T21:19:15.140 INFO:teuthology.orchestra.run.smithi184.stdout:2662 trunc from 0xa8ab400 to 0xc354000 2024-09-17T21:19:15.156 INFO:teuthology.orchestra.run.smithi184.stdout:2663 read 0x5b9e400 thru 0x5ba2616 (0x4217 bytes) 2024-09-17T21:19:15.157 INFO:teuthology.orchestra.run.smithi184.stdout:2664 punch from 0x33dfc00 to 0x33ee200, (0xe600 bytes) 2024-09-17T21:19:15.157 INFO:teuthology.orchestra.run.smithi184.stdout:2666 read 0x37b4800 thru 0x37c0c89 (0xc48a bytes) 2024-09-17T21:19:15.171 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:19:15.172 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T21:19:15.181 INFO:teuthology.orchestra.run.smithi184.stdout:2667 trunc from 0xc354000 to 0x3325400 2024-09-17T21:19:15.182 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:15.180+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:15.188 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-17T21:19:14.479952+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":679048584,"num_objects":389,"num_object_clones":83,"num_object_copies":778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":389,"num_whiteouts":56,"num_read":70119,"num_read_kb":310126,"num_write":4988,"num_write_kb":164923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":23376921,"num_keys_recovered":87,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4502,"ondisk_log_size":4502,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":319284,"kb_used_data":138396,"kb_used_omap":7,"kb_used_meta":180792,"kb_avail":471539916,"statfs":{"total":483183820800,"available":482856873984,"internally_reserved":0,"allocated":141717504,"data_stored":202958667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7978,"internal_metadata":185131222},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":20,"num_write":6,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.039196"},"pg_stats":[{"pgid":"3.1f","version":"61'71","reported_seq":757,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447704+0000","last_change":"2024-09-17T21:18:40.931000+0000","last_active":"2024-09-17T21:19:09.447704+0000","last_peered":"2024-09-17T21:19:09.447704+0000","last_clean":"2024-09-17T21:19:09.447704+0000","last_became_active":"2024-09-17T21:18:40.810689+0000","last_became_peered":"2024-09-17T21:18:40.810689+0000","last_unstale":"2024-09-17T21:19:09.447704+0000","last_undegraded":"2024-09-17T21:19:09.447704+0000","last_fullsized":"2024-09-17T21:19:09.447704+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:31:56.719040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088525599999999996,"stat_sum":{"num_bytes":4213859,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":712,"num_read_kb":5893,"num_write":89,"num_write_kb":875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":96832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"61'155","reported_seq":1603,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.448122+0000","last_change":"2024-09-17T21:18:43.278035+0000","last_active":"2024-09-17T21:19:09.448122+0000","last_peered":"2024-09-17T21:19:09.448122+0000","last_clean":"2024-09-17T21:19:09.448122+0000","last_became_active":"2024-09-17T21:18:43.277411+0000","last_became_peered":"2024-09-17T21:18:43.277411+0000","last_unstale":"2024-09-17T21:19:09.448122+0000","last_undegraded":"2024-09-17T21:19:09.448122+0000","last_fullsized":"2024-09-17T21:19:09.448122+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:31.348355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013256229999999999,"stat_sum":{"num_bytes":7273514,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":2215,"num_read_kb":6212,"num_write":164,"num_write_kb":629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":127572,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"65'196","reported_seq":2610,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.440094+0000","last_change":"2024-09-17T21:19:05.417322+0000","last_active":"2024-09-17T21:19:09.440094+0000","last_peered":"2024-09-17T21:19:09.440094+0000","last_clean":"2024-09-17T21:19:09.440094+0000","last_became_active":"2024-09-17T21:19:05.415967+0000","last_became_peered":"2024-09-17T21:19:05.415967+0000","last_unstale":"2024-09-17T21:19:09.440094+0000","last_undegraded":"2024-09-17T21:19:09.440094+0000","last_fullsized":"2024-09-17T21:19:09.440094+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:37.433255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018973079999999999,"stat_sum":{"num_bytes":23801924,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":2700,"num_read_kb":9461,"num_write":214,"num_write_kb":6907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"62'40","reported_seq":402,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.475561+0000","last_change":"2024-09-17T21:19:05.416924+0000","last_active":"2024-09-17T21:19:09.475561+0000","last_peered":"2024-09-17T21:19:09.475561+0000","last_clean":"2024-09-17T21:19:09.475561+0000","last_became_active":"2024-09-17T21:19:05.416249+0000","last_became_peered":"2024-09-17T21:19:05.416249+0000","last_unstale":"2024-09-17T21:19:09.475561+0000","last_undegraded":"2024-09-17T21:19:09.475561+0000","last_fullsized":"2024-09-17T21:19:09.475561+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:32:09.962627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034186630000000002,"stat_sum":{"num_bytes":3104256,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":276,"num_read_kb":2317,"num_write":42,"num_write_kb":1295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"66'234","reported_seq":2959,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.482178+0000","last_change":"2024-09-17T21:18:41.001528+0000","last_active":"2024-09-17T21:19:09.482178+0000","last_peered":"2024-09-17T21:19:09.482178+0000","last_clean":"2024-09-17T21:19:09.482178+0000","last_became_active":"2024-09-17T21:18:40.827008+0000","last_became_peered":"2024-09-17T21:18:40.827008+0000","last_unstale":"2024-09-17T21:19:09.482178+0000","last_undegraded":"2024-09-17T21:19:09.482178+0000","last_fullsized":"2024-09-17T21:19:09.482178+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:13:56.566421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089504999999999999,"stat_sum":{"num_bytes":10551808,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":2,"num_read":4965,"num_read_kb":5766,"num_write":141,"num_write_kb":1065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":615936,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"61'197","reported_seq":3519,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.481609+0000","last_change":"2024-09-17T21:18:13.314063+0000","last_active":"2024-09-17T21:19:09.481609+0000","last_peered":"2024-09-17T21:19:09.481609+0000","last_clean":"2024-09-17T21:19:09.481609+0000","last_became_active":"2024-09-17T21:18:11.301426+0000","last_became_peered":"2024-09-17T21:18:11.301426+0000","last_unstale":"2024-09-17T21:19:09.481609+0000","last_undegraded":"2024-09-17T21:19:09.481609+0000","last_fullsized":"2024-09-17T21:19:09.481609+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:26.981484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027366090000000001,"stat_sum":{"num_bytes":14090824,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":3,"num_read":4723,"num_read_kb":8928,"num_write":176,"num_write_kb":1273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"65'90","reported_seq":2166,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.443643+0000","last_change":"2024-09-17T21:18:13.314862+0000","last_active":"2024-09-17T21:19:09.443643+0000","last_peered":"2024-09-17T21:19:09.443643+0000","last_clean":"2024-09-17T21:19:09.443643+0000","last_became_active":"2024-09-17T21:18:11.300793+0000","last_became_peered":"2024-09-17T21:18:11.300793+0000","last_unstale":"2024-09-17T21:19:09.443643+0000","last_undegraded":"2024-09-17T21:19:09.443643+0000","last_fullsized":"2024-09-17T21:19:09.443643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:15:33.881019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047673000000000002,"stat_sum":{"num_bytes":23540736,"num_objects":18,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":2010,"num_read_kb":9716,"num_write":107,"num_write_kb":2412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"61'98","reported_seq":271,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.443669+0000","last_change":"2024-09-17T21:18:40.830426+0000","last_active":"2024-09-17T21:19:09.443669+0000","last_peered":"2024-09-17T21:19:09.443669+0000","last_clean":"2024-09-17T21:19:09.443669+0000","last_became_active":"2024-09-17T21:18:40.801216+0000","last_became_peered":"2024-09-17T21:18:40.801216+0000","last_unstale":"2024-09-17T21:19:09.443669+0000","last_undegraded":"2024-09-17T21:19:09.443669+0000","last_fullsized":"2024-09-17T21:19:09.443669+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:06:56.249183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10079792,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":881,"num_read_kb":3803,"num_write":135,"num_write_kb":1237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":428592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"68'74","reported_seq":1055,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T21:19:13.478359+0000","last_change":"2024-09-17T21:19:05.419944+0000","last_active":"2024-09-17T21:19:13.478359+0000","last_peered":"2024-09-17T21:19:13.478359+0000","last_clean":"2024-09-17T21:19:13.478359+0000","last_became_active":"2024-09-17T21:19:05.418983+0000","last_became_peered":"2024-09-17T21:19:05.418983+0000","last_unstale":"2024-09-17T21:19:13.478359+0000","last_undegraded":"2024-09-17T21:19:13.478359+0000","last_fullsized":"2024-09-17T21:19:13.478359+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:58:01.226365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088962999999999998,"stat_sum":{"num_bytes":28542976,"num_objects":13,"num_object_clones":5,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":2,"num_read":891,"num_read_kb":5068,"num_write":80,"num_write_kb":1949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":793088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"61'104","reported_seq":1031,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447899+0000","last_change":"2024-09-17T21:19:05.411424+0000","last_active":"2024-09-17T21:19:09.447899+0000","last_peered":"2024-09-17T21:19:09.447899+0000","last_clean":"2024-09-17T21:19:09.447899+0000","last_became_active":"2024-09-17T21:19:05.410746+0000","last_became_peered":"2024-09-17T21:19:05.410746+0000","last_unstale":"2024-09-17T21:19:09.447899+0000","last_undegraded":"2024-09-17T21:19:09.447899+0000","last_fullsized":"2024-09-17T21:19:09.447899+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:43:56.625042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001825658,"stat_sum":{"num_bytes":13074506,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1169,"num_read_kb":5431,"num_write":170,"num_write_kb":10737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"66'478","reported_seq":3496,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.448157+0000","last_change":"2024-09-17T21:18:41.006288+0000","last_active":"2024-09-17T21:19:09.448157+0000","last_peered":"2024-09-17T21:19:09.448157+0000","last_clean":"2024-09-17T21:19:09.448157+0000","last_became_active":"2024-09-17T21:18:40.820709+0000","last_became_peered":"2024-09-17T21:18:40.820709+0000","last_unstale":"2024-09-17T21:19:09.448157+0000","last_undegraded":"2024-09-17T21:19:09.448157+0000","last_fullsized":"2024-09-17T21:19:09.448157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:24:07.745065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027112149999999999,"stat_sum":{"num_bytes":53657712,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":2,"num_read":4722,"num_read_kb":19835,"num_write":755,"num_write_kb":3384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1160742,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"65'98","reported_seq":1293,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447821+0000","last_change":"2024-09-17T21:18:13.307234+0000","last_active":"2024-09-17T21:19:09.447821+0000","last_peered":"2024-09-17T21:19:09.447821+0000","last_clean":"2024-09-17T21:19:09.447821+0000","last_became_active":"2024-09-17T21:18:11.299817+0000","last_became_peered":"2024-09-17T21:18:11.299817+0000","last_unstale":"2024-09-17T21:19:09.447821+0000","last_undegraded":"2024-09-17T21:19:09.447821+0000","last_fullsized":"2024-09-17T21:19:09.447821+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:44:24.083134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002281113,"stat_sum":{"num_bytes":33669911,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":1,"num_read":1146,"num_read_kb":5053,"num_write":91,"num_write_kb":11916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447912+0000","last_change":"2024-09-17T21:18:03.059715+0000","last_active":"2024-09-17T21:19:09.447912+0000","last_peered":"2024-09-17T21:19:09.447912+0000","last_clean":"2024-09-17T21:19:09.447912+0000","last_became_active":"2024-09-17T21:18:01.213047+0000","last_became_peered":"2024-09-17T21:18:01.213047+0000","last_unstale":"2024-09-17T21:19:09.447912+0000","last_undegraded":"2024-09-17T21:19:09.447912+0000","last_fullsized":"2024-09-17T21:19:09.447912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:46:35.222373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"61'162","reported_seq":1201,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.444282+0000","last_change":"2024-09-17T21:18:40.976309+0000","last_active":"2024-09-17T21:19:09.444282+0000","last_peered":"2024-09-17T21:19:09.444282+0000","last_clean":"2024-09-17T21:19:09.444282+0000","last_became_active":"2024-09-17T21:18:40.818354+0000","last_became_peered":"2024-09-17T21:18:40.818354+0000","last_unstale":"2024-09-17T21:19:09.444282+0000","last_undegraded":"2024-09-17T21:19:09.444282+0000","last_fullsized":"2024-09-17T21:19:09.444282+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:23:40.750668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084327200000000005,"stat_sum":{"num_bytes":21660498,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":1574,"num_read_kb":20376,"num_write":262,"num_write_kb":13149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8773670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":110,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.444391+0000","last_change":"2024-09-17T21:18:03.073440+0000","last_active":"2024-09-17T21:19:09.444391+0000","last_peered":"2024-09-17T21:19:09.444391+0000","last_clean":"2024-09-17T21:19:09.444391+0000","last_became_active":"2024-09-17T21:18:01.048806+0000","last_became_peered":"2024-09-17T21:18:01.048806+0000","last_unstale":"2024-09-17T21:19:09.444391+0000","last_undegraded":"2024-09-17T21:19:09.444391+0000","last_fullsized":"2024-09-17T21:19:09.444391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:26.487163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094161400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"61'95","reported_seq":1360,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.444596+0000","last_change":"2024-09-17T21:18:17.280432+0000","last_active":"2024-09-17T21:19:09.444596+0000","last_peered":"2024-09-17T21:19:09.444596+0000","last_clean":"2024-09-17T21:19:09.444596+0000","last_became_active":"2024-09-17T21:18:17.279625+0000","last_became_peered":"2024-09-17T21:18:17.279625+0000","last_unstale":"2024-09-17T21:19:09.444596+0000","last_undegraded":"2024-09-17T21:19:09.444596+0000","last_fullsized":"2024-09-17T21:19:09.444596+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:13:46.386831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22065247,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":1353,"num_read_kb":17234,"num_write":119,"num_write_kb":11269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":116,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447895+0000","last_change":"2024-09-17T21:18:40.821956+0000","last_active":"2024-09-17T21:19:09.447895+0000","last_peered":"2024-09-17T21:19:09.447895+0000","last_clean":"2024-09-17T21:19:09.447895+0000","last_became_active":"2024-09-17T21:18:40.820574+0000","last_became_peered":"2024-09-17T21:18:40.820574+0000","last_unstale":"2024-09-17T21:19:09.447895+0000","last_undegraded":"2024-09-17T21:19:09.447895+0000","last_fullsized":"2024-09-17T21:19:09.447895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:35:05.837282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099219900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"61'54","reported_seq":1427,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.444150+0000","last_change":"2024-09-17T21:18:13.315662+0000","last_active":"2024-09-17T21:19:09.444150+0000","last_peered":"2024-09-17T21:19:09.444150+0000","last_clean":"2024-09-17T21:19:09.444150+0000","last_became_active":"2024-09-17T21:18:11.309133+0000","last_became_peered":"2024-09-17T21:18:11.309133+0000","last_unstale":"2024-09-17T21:19:09.444150+0000","last_undegraded":"2024-09-17T21:19:09.444150+0000","last_fullsized":"2024-09-17T21:19:09.444150+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:42:12.633242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078465700000000002,"stat_sum":{"num_bytes":23857152,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":1291,"num_read_kb":12784,"num_write":62,"num_write_kb":6160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":116,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.444255+0000","last_change":"2024-09-17T21:18:40.818736+0000","last_active":"2024-09-17T21:19:09.444255+0000","last_peered":"2024-09-17T21:19:09.444255+0000","last_clean":"2024-09-17T21:19:09.444255+0000","last_became_active":"2024-09-17T21:18:40.818309+0000","last_became_peered":"2024-09-17T21:18:40.818309+0000","last_unstale":"2024-09-17T21:19:09.444255+0000","last_undegraded":"2024-09-17T21:19:09.444255+0000","last_fullsized":"2024-09-17T21:19:09.444255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:42:22.148483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006222,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"61'299","reported_seq":4815,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.450892+0000","last_change":"2024-09-17T21:18:13.327202+0000","last_active":"2024-09-17T21:19:09.450892+0000","last_peered":"2024-09-17T21:19:09.450892+0000","last_clean":"2024-09-17T21:19:09.450892+0000","last_became_active":"2024-09-17T21:18:11.317127+0000","last_became_peered":"2024-09-17T21:18:11.317127+0000","last_unstale":"2024-09-17T21:19:09.450892+0000","last_undegraded":"2024-09-17T21:19:09.450892+0000","last_fullsized":"2024-09-17T21:19:09.450892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:43:54.053747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036493720000000001,"stat_sum":{"num_bytes":57645087,"num_objects":19,"num_object_clones":6,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":1,"num_read":6702,"num_read_kb":18001,"num_write":207,"num_write_kb":15267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"61'54","reported_seq":671,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.440305+0000","last_change":"2024-09-17T21:18:40.963953+0000","last_active":"2024-09-17T21:19:09.440305+0000","last_peered":"2024-09-17T21:19:09.440305+0000","last_clean":"2024-09-17T21:19:09.440305+0000","last_became_active":"2024-09-17T21:18:40.820910+0000","last_became_peered":"2024-09-17T21:18:40.820910+0000","last_unstale":"2024-09-17T21:19:09.440305+0000","last_undegraded":"2024-09-17T21:19:09.440305+0000","last_fullsized":"2024-09-17T21:19:09.440305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:16:23.135778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084988700000000002,"stat_sum":{"num_bytes":5107712,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":492,"num_read_kb":2316,"num_write":55,"num_write_kb":1813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":978944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":32,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.444243+0000","last_change":"2024-09-17T21:19:05.415157+0000","last_active":"2024-09-17T21:19:09.444243+0000","last_peered":"2024-09-17T21:19:09.444243+0000","last_clean":"2024-09-17T21:19:09.444243+0000","last_became_active":"2024-09-17T21:19:05.414343+0000","last_became_peered":"2024-09-17T21:19:05.414343+0000","last_unstale":"2024-09-17T21:19:09.444243+0000","last_undegraded":"2024-09-17T21:19:09.444243+0000","last_fullsized":"2024-09-17T21:19:09.444243+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:27:06.755871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"61'32","reported_seq":493,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.443911+0000","last_change":"2024-09-17T21:18:13.314742+0000","last_active":"2024-09-17T21:19:09.443911+0000","last_peered":"2024-09-17T21:19:09.443911+0000","last_clean":"2024-09-17T21:19:09.443911+0000","last_became_active":"2024-09-17T21:18:11.308568+0000","last_became_peered":"2024-09-17T21:18:11.308568+0000","last_unstale":"2024-09-17T21:19:09.443911+0000","last_undegraded":"2024-09-17T21:19:09.443911+0000","last_fullsized":"2024-09-17T21:19:09.443911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:01:26.289237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049125000000000004,"stat_sum":{"num_bytes":9221152,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":368,"num_read_kb":7190,"num_write":39,"num_write_kb":2902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":88,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.443631+0000","last_change":"2024-09-17T21:18:17.280358+0000","last_active":"2024-09-17T21:19:09.443631+0000","last_peered":"2024-09-17T21:19:09.443631+0000","last_clean":"2024-09-17T21:19:09.443631+0000","last_became_active":"2024-09-17T21:18:17.279474+0000","last_became_peered":"2024-09-17T21:18:17.279474+0000","last_unstale":"2024-09-17T21:19:09.443631+0000","last_undegraded":"2024-09-17T21:19:09.443631+0000","last_fullsized":"2024-09-17T21:19:09.443631+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:55:54.183527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"61'68","reported_seq":748,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.439826+0000","last_change":"2024-09-17T21:18:40.906152+0000","last_active":"2024-09-17T21:19:09.439826+0000","last_peered":"2024-09-17T21:19:09.439826+0000","last_clean":"2024-09-17T21:19:09.439826+0000","last_became_active":"2024-09-17T21:18:40.816571+0000","last_became_peered":"2024-09-17T21:18:40.816571+0000","last_unstale":"2024-09-17T21:19:09.439826+0000","last_undegraded":"2024-09-17T21:19:09.439826+0000","last_fullsized":"2024-09-17T21:19:09.439826+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:11:26.159680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066398399999999995,"stat_sum":{"num_bytes":7944240,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":772,"num_read_kb":7750,"num_write":100,"num_write_kb":4353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":4070960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":110,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447467+0000","last_change":"2024-09-17T21:18:03.059156+0000","last_active":"2024-09-17T21:19:09.447467+0000","last_peered":"2024-09-17T21:19:09.447467+0000","last_clean":"2024-09-17T21:19:09.447467+0000","last_became_active":"2024-09-17T21:18:01.049180+0000","last_became_peered":"2024-09-17T21:18:01.049180+0000","last_unstale":"2024-09-17T21:19:09.447467+0000","last_undegraded":"2024-09-17T21:19:09.447467+0000","last_fullsized":"2024-09-17T21:19:09.447467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:27.069794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077408,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"62'217","reported_seq":1008,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.444202+0000","last_change":"2024-09-17T21:18:13.315294+0000","last_active":"2024-09-17T21:19:09.444202+0000","last_peered":"2024-09-17T21:19:09.444202+0000","last_clean":"2024-09-17T21:19:09.444202+0000","last_became_active":"2024-09-17T21:18:11.294154+0000","last_became_peered":"2024-09-17T21:18:11.294154+0000","last_unstale":"2024-09-17T21:19:09.444202+0000","last_undegraded":"2024-09-17T21:19:09.444202+0000","last_fullsized":"2024-09-17T21:19:09.444202+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:16:11.502838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049408099999999995,"stat_sum":{"num_bytes":34535719,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1672,"num_read_kb":20106,"num_write":377,"num_write_kb":9147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":32,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T21:19:13.478301+0000","last_change":"2024-09-17T21:19:05.419660+0000","last_active":"2024-09-17T21:19:13.478301+0000","last_peered":"2024-09-17T21:19:13.478301+0000","last_clean":"2024-09-17T21:19:13.478301+0000","last_became_active":"2024-09-17T21:19:05.419056+0000","last_became_peered":"2024-09-17T21:19:05.419056+0000","last_unstale":"2024-09-17T21:19:13.478301+0000","last_undegraded":"2024-09-17T21:19:13.478301+0000","last_fullsized":"2024-09-17T21:19:13.478301+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:32:04.987802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":213,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447628+0000","last_change":"2024-09-17T21:18:40.829967+0000","last_active":"2024-09-17T21:19:09.447628+0000","last_peered":"2024-09-17T21:19:09.447628+0000","last_clean":"2024-09-17T21:19:09.447628+0000","last_became_active":"2024-09-17T21:18:40.803358+0000","last_became_peered":"2024-09-17T21:18:40.803358+0000","last_unstale":"2024-09-17T21:19:09.447628+0000","last_undegraded":"2024-09-17T21:19:09.447628+0000","last_fullsized":"2024-09-17T21:19:09.447628+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_clean_scrub_stamp":"2024-09-17T21:17:55.982339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:57:35.748912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"65'190","reported_seq":3539,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447542+0000","last_change":"2024-09-17T21:18:50.296660+0000","last_active":"2024-09-17T21:19:09.447542+0000","last_peered":"2024-09-17T21:19:09.447542+0000","last_clean":"2024-09-17T21:19:09.447542+0000","last_became_active":"2024-09-17T21:18:50.295377+0000","last_became_peered":"2024-09-17T21:18:50.295377+0000","last_unstale":"2024-09-17T21:19:09.447542+0000","last_undegraded":"2024-09-17T21:19:09.447542+0000","last_fullsized":"2024-09-17T21:19:09.447542+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:12:40.845223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22410256,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":4607,"num_read_kb":29652,"num_write":141,"num_write_kb":14993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3239440,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"61'174","reported_seq":1245,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.440059+0000","last_change":"2024-09-17T21:18:13.313980+0000","last_active":"2024-09-17T21:19:09.440059+0000","last_peered":"2024-09-17T21:19:09.440059+0000","last_clean":"2024-09-17T21:19:09.440059+0000","last_became_active":"2024-09-17T21:18:11.304021+0000","last_became_peered":"2024-09-17T21:18:11.304021+0000","last_unstale":"2024-09-17T21:19:09.440059+0000","last_undegraded":"2024-09-17T21:19:09.440059+0000","last_fullsized":"2024-09-17T21:19:09.440059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:48:29.828438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023749890000000001,"stat_sum":{"num_bytes":23323689,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1633,"num_read_kb":7526,"num_write":311,"num_write_kb":2811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"62'65","reported_seq":651,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447581+0000","last_change":"2024-09-17T21:18:13.307298+0000","last_active":"2024-09-17T21:19:09.447581+0000","last_peered":"2024-09-17T21:19:09.447581+0000","last_clean":"2024-09-17T21:19:09.447581+0000","last_became_active":"2024-09-17T21:18:11.294907+0000","last_became_peered":"2024-09-17T21:18:11.294907+0000","last_unstale":"2024-09-17T21:19:09.447581+0000","last_undegraded":"2024-09-17T21:19:09.447581+0000","last_fullsized":"2024-09-17T21:19:09.447581+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:52:34.160289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021951829999999999,"stat_sum":{"num_bytes":34581090,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":2,"num_read":522,"num_read_kb":5302,"num_write":64,"num_write_kb":648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"65'151","reported_seq":1523,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.444525+0000","last_change":"2024-09-17T21:18:40.868870+0000","last_active":"2024-09-17T21:19:09.444525+0000","last_peered":"2024-09-17T21:19:09.444525+0000","last_clean":"2024-09-17T21:19:09.444525+0000","last_became_active":"2024-09-17T21:18:40.816043+0000","last_became_peered":"2024-09-17T21:18:40.816043+0000","last_unstale":"2024-09-17T21:19:09.444525+0000","last_undegraded":"2024-09-17T21:19:09.444525+0000","last_fullsized":"2024-09-17T21:19:09.444525+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:29:59.626472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048704800000000002,"stat_sum":{"num_bytes":26157599,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":1,"num_read":2244,"num_read_kb":6312,"num_write":128,"num_write_kb":11534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":485392,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"61'103","reported_seq":98,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.443984+0000","last_change":"2024-09-17T21:18:54.392198+0000","last_active":"2024-09-17T21:19:09.443984+0000","last_peered":"2024-09-17T21:19:09.443984+0000","last_clean":"2024-09-17T21:19:09.443984+0000","last_became_active":"2024-09-17T21:18:54.343340+0000","last_became_peered":"2024-09-17T21:18:54.343340+0000","last_unstale":"2024-09-17T21:19:09.443984+0000","last_undegraded":"2024-09-17T21:19:09.443984+0000","last_fullsized":"2024-09-17T21:19:09.443984+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:58:42.535973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015319789999999999,"stat_sum":{"num_bytes":26867200,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":2060,"num_read_kb":6588,"num_write":83,"num_write_kb":758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":166400,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"61'22","reported_seq":174,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447924+0000","last_change":"2024-09-17T21:18:13.306351+0000","last_active":"2024-09-17T21:19:09.447924+0000","last_peered":"2024-09-17T21:19:09.447924+0000","last_clean":"2024-09-17T21:19:09.447924+0000","last_became_active":"2024-09-17T21:18:11.316999+0000","last_became_peered":"2024-09-17T21:18:11.316999+0000","last_unstale":"2024-09-17T21:19:09.447924+0000","last_undegraded":"2024-09-17T21:19:09.447924+0000","last_fullsized":"2024-09-17T21:19:09.447924+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:13.482512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015238949999999999,"stat_sum":{"num_bytes":723968,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":1011,"num_write":23,"num_write_kb":818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"62'264","reported_seq":5890,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.499227+0000","last_change":"2024-09-17T21:19:05.416474+0000","last_active":"2024-09-17T21:19:09.499227+0000","last_peered":"2024-09-17T21:19:09.499227+0000","last_clean":"2024-09-17T21:19:09.499227+0000","last_became_active":"2024-09-17T21:19:05.415977+0000","last_became_peered":"2024-09-17T21:19:05.415977+0000","last_unstale":"2024-09-17T21:19:09.499227+0000","last_undegraded":"2024-09-17T21:19:09.499227+0000","last_fullsized":"2024-09-17T21:19:09.499227+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:11:58.398387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003632209,"stat_sum":{"num_bytes":29204992,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":6436,"num_read_kb":21545,"num_write":197,"num_write_kb":15776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"62'62","reported_seq":1348,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447940+0000","last_change":"2024-09-17T21:19:05.410271+0000","last_active":"2024-09-17T21:19:09.447940+0000","last_peered":"2024-09-17T21:19:09.447940+0000","last_clean":"2024-09-17T21:19:09.447940+0000","last_became_active":"2024-09-17T21:19:05.409905+0000","last_became_peered":"2024-09-17T21:19:05.409905+0000","last_unstale":"2024-09-17T21:19:09.447940+0000","last_undegraded":"2024-09-17T21:19:09.447940+0000","last_fullsized":"2024-09-17T21:19:09.447940+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:33:06.031026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002454758,"stat_sum":{"num_bytes":2159631,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":1661,"num_read_kb":3227,"num_write":43,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"61'77","reported_seq":2681,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.443563+0000","last_change":"2024-09-17T21:19:05.414774+0000","last_active":"2024-09-17T21:19:09.443563+0000","last_peered":"2024-09-17T21:19:09.443563+0000","last_clean":"2024-09-17T21:19:09.443563+0000","last_became_active":"2024-09-17T21:19:05.413373+0000","last_became_peered":"2024-09-17T21:19:05.413373+0000","last_unstale":"2024-09-17T21:19:09.443563+0000","last_undegraded":"2024-09-17T21:19:09.443563+0000","last_fullsized":"2024-09-17T21:19:09.443563+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:24:21.294368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001519261,"stat_sum":{"num_bytes":1272906,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1526,"num_read_kb":4271,"num_write":50,"num_write_kb":919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":538698,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"65'196","reported_seq":2008,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447522+0000","last_change":"2024-09-17T21:18:13.305680+0000","last_active":"2024-09-17T21:19:09.447522+0000","last_peered":"2024-09-17T21:19:09.447522+0000","last_clean":"2024-09-17T21:19:09.447522+0000","last_became_active":"2024-09-17T21:18:11.299946+0000","last_became_peered":"2024-09-17T21:18:11.299946+0000","last_unstale":"2024-09-17T21:19:09.447522+0000","last_undegraded":"2024-09-17T21:19:09.447522+0000","last_fullsized":"2024-09-17T21:19:09.447522+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:52.766042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225884,"stat_sum":{"num_bytes":33520477,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":2,"num_read":2072,"num_read_kb":8553,"num_write":217,"num_write_kb":2540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"61'73","reported_seq":1424,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.447774+0000","last_change":"2024-09-17T21:18:40.967754+0000","last_active":"2024-09-17T21:19:09.447774+0000","last_peered":"2024-09-17T21:19:09.447774+0000","last_clean":"2024-09-17T21:19:09.447774+0000","last_became_active":"2024-09-17T21:18:40.814521+0000","last_became_peered":"2024-09-17T21:18:40.814521+0000","last_unstale":"2024-09-17T21:19:09.447774+0000","last_undegraded":"2024-09-17T21:19:09.447774+0000","last_fullsized":"2024-09-17T21:19:09.447774+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:48:50.619550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013045489999999999,"stat_sum":{"num_bytes":20655695,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":2,"num_read":1256,"num_read_kb":5704,"num_write":81,"num_write_kb":2386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1310287,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"66'234","reported_seq":4670,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:19:09.484210+0000","last_change":"2024-09-17T21:18:13.306520+0000","last_active":"2024-09-17T21:19:09.484210+0000","last_peered":"2024-09-17T21:19:09.484210+0000","last_clean":"2024-09-17T21:19:09.484210+0000","last_became_active":"2024-09-17T21:18:11.294953+0000","last_became_peered":"2024-09-17T21:18:11.294953+0000","last_unstale":"2024-09-17T21:19:09.484210+0000","last_undegraded":"2024-09-17T21:19:09.484210+0000","last_fullsized":"2024-09-17T21:19:09.484210+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:33:39.961492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019790290000000002,"stat_sum":{"num_bytes":49942069,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":2,"num_read":5364,"num_read_kb":17140,"num_write":162,"num_write_kb":2255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":678458197,"num_objects":385,"num_object_clones":83,"num_object_copies":770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":385,"num_whiteouts":56,"num_read":70053,"num_read_kb":310071,"num_write":4885,"num_write_kb":163293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":22786553,"num_keys_recovered":87,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":165421056,"data_stored":237171957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4427,"ondisk_log_size":4427,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62816,"kb_used_data":27472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94309024,"statfs":{"total":96636764160,"available":96572440576,"internally_reserved":0,"allocated":28131328,"data_stored":45197087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74384,"kb_used_data":34880,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94297456,"statfs":{"total":96636764160,"available":96560594944,"internally_reserved":0,"allocated":35717120,"data_stored":48655979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":40434116},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50404,"kb_used_data":15060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94321436,"statfs":{"total":96636764160,"available":96585150464,"internally_reserved":0,"allocated":15421440,"data_stored":16373642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68924,"kb_used_data":33576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94302916,"statfs":{"total":96636764160,"available":96566185984,"internally_reserved":0,"allocated":34381824,"data_stored":52123434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":0,"seq":55834574866,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607572,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62756,"kb_used_data":27408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94309084,"statfs":{"total":96636764160,"available":96572502016,"internally_reserved":0,"allocated":28065792,"data_stored":40608525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28000256,"data_stored":36273087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33046528,"data_stored":51242024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27869184,"data_stored":40856656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":14131200,"data_stored":15529152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":34979840,"data_stored":48364956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":27394048,"data_stored":44906082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:19:15.190 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-17T21:19:15.190 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:19:15.190 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-09-17T21:19:15.190 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-17T21:19:15.204 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:15.204 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:15.219 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:19:15.261 INFO:teuthology.orchestra.run.smithi184.stdout:2669 write 0xe192000 thru 0xe1a0b84 (0xeb85 bytes) 2024-09-17T21:19:15.277 INFO:teuthology.orchestra.run.smithi184.stdout:2672 trunc from 0xe1a0c00 to 0x831ac00 2024-09-17T21:19:15.306 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:15.304+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:15.307 INFO:teuthology.orchestra.run.smithi184.stdout:2673 read 0x5494200 thru 0x549d997 (0x9798 bytes) 2024-09-17T21:19:15.310 INFO:teuthology.orchestra.run.smithi184.stdout:2675 write 0x304c000 thru 0x3055e12 (0x9e13 bytes) 2024-09-17T21:19:15.310 INFO:teuthology.orchestra.run.smithi184.stdout:2678 write 0x5610a00 thru 0x5611927 (0xf28 bytes) 2024-09-17T21:19:15.311 INFO:teuthology.orchestra.run.smithi184.stdout:2680 read 0x5ff800 thru 0x60e50e (0xed0f bytes) 2024-09-17T21:19:15.312 INFO:teuthology.orchestra.run.smithi184.stdout:2681 read 0x156fa00 thru 0x15734b4 (0x3ab5 bytes) 2024-09-17T21:19:15.312 INFO:teuthology.orchestra.run.smithi184.stdout:2682 write 0x4e1ba00 thru 0x4e2adbe (0xf3bf bytes) 2024-09-17T21:19:15.347 INFO:teuthology.orchestra.run.smithi184.stdout:2683 write 0xe0b2800 thru 0xe0bc676 (0x9e77 bytes) 2024-09-17T21:19:15.348 INFO:teuthology.orchestra.run.smithi184.stdout:2685 write 0x1d41a00 thru 0x1d47fe2 (0x65e3 bytes) 2024-09-17T21:19:15.349 INFO:teuthology.orchestra.run.smithi184.stdout:2687 read 0x1cb8800 thru 0x1cc0253 (0x7a54 bytes) 2024-09-17T21:19:15.349 INFO:teuthology.orchestra.run.smithi184.stdout:2688 write 0xcf5b400 thru 0xcf633ab (0x7fac bytes) 2024-09-17T21:19:15.349 INFO:teuthology.orchestra.run.smithi184.stdout:2689 read 0x754d400 thru 0x754f854 (0x2455 bytes) 2024-09-17T21:19:15.376 INFO:teuthology.orchestra.run.smithi184.stdout:2691 write 0xe5f7400 thru 0xe5f8526 (0x1127 bytes) 2024-09-17T21:19:15.383 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:15.380+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:15.383 INFO:teuthology.orchestra.run.smithi184.stdout:2692 trunc from 0xe5f8600 to 0xca80400 2024-09-17T21:19:15.396 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:15.396 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:15.412 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:19:15.414 INFO:teuthology.orchestra.run.smithi184.stdout:2693 write 0x5a76200 thru 0x5a78ca9 (0x2aaa bytes) 2024-09-17T21:19:15.442 INFO:teuthology.orchestra.run.smithi184.stdout:2694 trunc from 0xca80400 to 0x1b3d600 2024-09-17T21:19:15.477 INFO:teuthology.orchestra.run.smithi184.stdout:2695 punch from 0xa57e00 to 0xa62e00, (0xb000 bytes) 2024-09-17T21:19:15.484 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:15.479+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:15.501 INFO:teuthology.orchestra.run.smithi184.stdout:2696 write 0x5a07c00 thru 0x5a0f5aa (0x79ab bytes) 2024-09-17T21:19:15.529 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-09-17T21:19:15.527+0000 7fe37768f640 -1 mon.a@0(leader).osd e71 definitely_dead 0 2024-09-17T21:19:15.531 INFO:teuthology.orchestra.run.smithi184.stdout:2697 write 0x834fc00 thru 0x835d3fd (0xd7fe bytes) 2024-09-17T21:19:15.534 INFO:teuthology.orchestra.run.smithi184.stdout:2699 write 0x55ef000 thru 0x55fce96 (0xde97 bytes) 2024-09-17T21:19:15.535 INFO:teuthology.orchestra.run.smithi184.stdout:2700 read 0x2ea5e00 thru 0x2ea9880 (0x3a81 bytes) 2024-09-17T21:19:15.565 INFO:teuthology.orchestra.run.smithi184.stdout:2701 write 0xa937800 thru 0xa93bf74 (0x4775 bytes) 2024-09-17T21:19:15.568 INFO:teuthology.orchestra.run.smithi184.stdout:2702 write 0x6770000 thru 0x6775e7e (0x5e7f bytes) 2024-09-17T21:19:15.569 INFO:teuthology.orchestra.run.smithi184.stdout:2703 read 0x6b8b600 thru 0x6b9803a (0xca3b bytes) 2024-09-17T21:19:15.573 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:19:15.592 INFO:teuthology.orchestra.run.smithi184.stdout:2704 write 0xb79f000 thru 0xb7ace47 (0xde48 bytes) 2024-09-17T21:19:15.594 INFO:teuthology.orchestra.run.smithi184.stdout:2705 punch from 0x8400400 to 0x8406800, (0x6400 bytes) 2024-09-17T21:19:15.595 INFO:teuthology.orchestra.run.smithi184.stdout:2706 read 0x89f8c00 thru 0x8a03ac9 (0xaeca bytes) 2024-09-17T21:19:15.595 INFO:teuthology.orchestra.run.smithi184.stdout:2707 write 0xa848600 thru 0xa8491f0 (0xbf1 bytes) 2024-09-17T21:19:15.596 INFO:teuthology.orchestra.run.smithi184.stdout:2708 trunc from 0xb7ad000 to 0xb522800 2024-09-17T21:19:15.601 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:15.601 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:15.618 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:19:15.621 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:15.619+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:15.624 INFO:teuthology.orchestra.run.smithi184.stdout:2709 punch from 0x7302c00 to 0x7304400, (0x1800 bytes) 2024-09-17T21:19:15.626 INFO:teuthology.orchestra.run.smithi184.stdout:2714 write 0xacc7c00 thru 0xaccda13 (0x5e14 bytes) 2024-09-17T21:19:15.626 INFO:teuthology.orchestra.run.smithi184.stdout:2716 write 0xae4a400 thru 0xae5638b (0xbf8c bytes) 2024-09-17T21:19:15.632 INFO:teuthology.orchestra.run.smithi184.stdout:2718 trunc from 0xb522800 to 0x963fa00 2024-09-17T21:19:15.675 INFO:teuthology.orchestra.run.smithi184.stdout:2720 write 0x9cb3600 thru 0x9cbeea0 (0xb8a1 bytes) 2024-09-17T21:19:15.675 INFO:teuthology.orchestra.run.smithi184.stdout:2721 read 0x343a000 thru 0x3444dc2 (0xadc3 bytes) 2024-09-17T21:19:15.676 INFO:teuthology.orchestra.run.smithi184.stdout:2722 punch from 0x5203600 to 0x5208400, (0x4e00 bytes) 2024-09-17T21:19:15.677 INFO:teuthology.orchestra.run.smithi184.stdout:2723 write 0x1a7b000 thru 0x1a88d1c (0xdd1d bytes) 2024-09-17T21:19:15.677 INFO:teuthology.orchestra.run.smithi184.stdout:2726 read 0x6f5d800 thru 0x6f63492 (0x5c93 bytes) 2024-09-17T21:19:15.677 INFO:teuthology.orchestra.run.smithi184.stdout:2727 punch from 0x3c4000 to 0x3cc000, (0x8000 bytes) 2024-09-17T21:19:15.685 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:15.683+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:15.701 INFO:teuthology.orchestra.run.smithi184.stdout:2729 write 0xbf27400 thru 0xbf27ca0 (0x8a1 bytes) 2024-09-17T21:19:15.710 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:19:15.722 INFO:teuthology.orchestra.run.smithi184.stdout:2730 trunc from 0xbf27e00 to 0x2025400 2024-09-17T21:19:15.750 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:15.750 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:15.760 INFO:teuthology.orchestra.run.smithi184.stdout:2732 punch from 0x12e1800 to 0x12ee400, (0xcc00 bytes) 2024-09-17T21:19:15.766 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:19:15.786 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:15.783+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:15.791 INFO:teuthology.orchestra.run.smithi184.stdout:2733 write 0x7ec7a00 thru 0x7ecddbe (0x63bf bytes) 2024-09-17T21:19:15.808 INFO:teuthology.orchestra.run.smithi184.stdout:2734 trunc from 0x7ecde00 to 0x1fd8e00 2024-09-17T21:19:15.845 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:15.845 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:15.852 INFO:teuthology.orchestra.run.smithi184.stdout:2735 write 0x322ac00 thru 0x322ebea (0x3feb bytes) 2024-09-17T21:19:15.853 INFO:teuthology.orchestra.run.smithi184.stdout:2737 read 0x3aa000 thru 0x3b2112 (0x8113 bytes) 2024-09-17T21:19:15.862 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:15.863 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:15.880 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:15.880 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:15.887 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:15.884+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:15.895 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:15.895 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:15.901 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:15.901 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:15.905 INFO:teuthology.orchestra.run.smithi184.stdout:2738 write 0xc99e200 thru 0xc9a0660 (0x2461 bytes) 2024-09-17T21:19:15.907 INFO:teuthology.orchestra.run.smithi184.stdout:2739 write 0xc0e4200 thru 0xc0e626f (0x2070 bytes) 2024-09-17T21:19:15.907 INFO:teuthology.orchestra.run.smithi184.stdout:2741 read 0x7d5bc00 thru 0x7d62ec9 (0x72ca bytes) 2024-09-17T21:19:15.911 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:19:15.922 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:15.922 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:15.939 INFO:teuthology.orchestra.run.smithi184.stdout:2743 write 0xea30800 thru 0xea363fa (0x5bfb bytes) 2024-09-17T21:19:15.939 INFO:teuthology.orchestra.run.smithi184.stdout:2745 read 0x3dd8000 thru 0x3ddd073 (0x5074 bytes) 2024-09-17T21:19:15.940 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:15.940 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:15.942 INFO:teuthology.orchestra.run.smithi184.stdout:2746 trunc from 0xea36400 to 0xe13b200 2024-09-17T21:19:15.969 INFO:teuthology.orchestra.run.smithi184.stdout:2748 write 0xb367600 thru 0xb368943 (0x1344 bytes) 2024-09-17T21:19:15.970 INFO:teuthology.orchestra.run.smithi184.stdout:2749 read 0x63b9000 thru 0x63ba669 (0x166a bytes) 2024-09-17T21:19:15.970 INFO:teuthology.orchestra.run.smithi184.stdout:2750 read 0x8058a00 thru 0x8062bff (0xa200 bytes) 2024-09-17T21:19:15.970 INFO:teuthology.orchestra.run.smithi184.stdout:2751 punch from 0x369a00 to 0x36d600, (0x3c00 bytes) 2024-09-17T21:19:15.998 INFO:teuthology.orchestra.run.smithi184.stdout:2752 trunc from 0xe13b200 to 0x7e2e00 2024-09-17T21:19:15.998 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:15.996+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:16.027 INFO:teuthology.orchestra.run.smithi184.stdout:2753 punch from 0x60c200 to 0x60e400, (0x2200 bytes) 2024-09-17T21:19:16.065 INFO:teuthology.orchestra.run.smithi184.stdout:2754 write 0xcaa1a00 thru 0xcaa86e8 (0x6ce9 bytes) 2024-09-17T21:19:16.066 INFO:teuthology.orchestra.run.smithi184.stdout:2760 write 0x1264400 thru 0x12644ff (0x100 bytes) 2024-09-17T21:19:16.085 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:16.085 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:16.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:16.085 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:16.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.085 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:478 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 720896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:16.085 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:19:16.086 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 39.848014851000002, 2024-09-17T21:19:16.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016845300000000001, 2024-09-17T21:19:16.086 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.086 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.086 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.086 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.086 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:16.086 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:16.086 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 478 2024-09-17T21:19:16.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.086 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:19:16.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:19:16.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233785+0000", 2024-09-17T21:19:16.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.116e-06 2024-09-17T21:19:16.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233803+0000", 2024-09-17T21:19:16.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7932999999999999e-05 2024-09-17T21:19:16.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233822+0000", 2024-09-17T21:19:16.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9276999999999999e-05 2024-09-17T21:19:16.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233836+0000", 2024-09-17T21:19:16.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4053000000000001e-05 2024-09-17T21:19:16.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.248579+0000", 2024-09-17T21:19:16.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014743529 2024-09-17T21:19:16.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.248795+0000", 2024-09-17T21:19:16.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021535200000000001 2024-09-17T21:19:16.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250629+0000", 2024-09-17T21:19:16.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018340400000000001 2024-09-17T21:19:16.090 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.090 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.090 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.090 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:479 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 851968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:16.090 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:19:16.090 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 39.847923764999997, 2024-09-17T21:19:16.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017383900000000001, 2024-09-17T21:19:16.090 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.090 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.090 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.090 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.090 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:16.091 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:16.091 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 479 2024-09-17T21:19:16.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.091 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:19:16.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:19:16.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233876+0000", 2024-09-17T21:19:16.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2160000000000001e-06 2024-09-17T21:19:16.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233883+0000", 2024-09-17T21:19:16.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9199999999999998e-06 2024-09-17T21:19:16.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233900+0000", 2024-09-17T21:19:16.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7034e-05 2024-09-17T21:19:16.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233913+0000", 2024-09-17T21:19:16.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159000000000001e-05 2024-09-17T21:19:16.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250712+0000", 2024-09-17T21:19:16.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016798644000000001 2024-09-17T21:19:16.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250913+0000", 2024-09-17T21:19:16.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020127900000000001 2024-09-17T21:19:16.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251259+0000", 2024-09-17T21:19:16.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00034564799999999999 2024-09-17T21:19:16.094 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.094 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.094 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.094 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:480 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:16.094 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:19:16.095 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 39.84783092, 2024-09-17T21:19:16.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017902482000000001, 2024-09-17T21:19:16.095 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.095 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.095 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.095 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.095 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:16.095 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:16.095 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 480 2024-09-17T21:19:16.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.095 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:19:16.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:19:16.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233969+0000", 2024-09-17T21:19:16.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.454e-06 2024-09-17T21:19:16.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233981+0000", 2024-09-17T21:19:16.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1593000000000001e-05 2024-09-17T21:19:16.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233999+0000", 2024-09-17T21:19:16.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.806e-05 2024-09-17T21:19:16.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234012+0000", 2024-09-17T21:19:16.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:19:16.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251339+0000", 2024-09-17T21:19:16.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017326352 2024-09-17T21:19:16.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251534+0000", 2024-09-17T21:19:16.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000195842 2024-09-17T21:19:16.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251870+0000", 2024-09-17T21:19:16.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033551400000000003 2024-09-17T21:19:16.099 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.099 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.099 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.099 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:481 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1114112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:16.099 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:19:16.099 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 39.847762553000003, 2024-09-17T21:19:16.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018442621999999999, 2024-09-17T21:19:16.099 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.099 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.099 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.099 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.099 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:16.100 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:16.100 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 481 2024-09-17T21:19:16.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.100 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:19:16.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:19:16.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234037+0000", 2024-09-17T21:19:16.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:19:16.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234043+0000", 2024-09-17T21:19:16.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.49e-06 2024-09-17T21:19:16.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234061+0000", 2024-09-17T21:19:16.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8416000000000001e-05 2024-09-17T21:19:16.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234076+0000", 2024-09-17T21:19:16.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.456e-05 2024-09-17T21:19:16.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251948+0000", 2024-09-17T21:19:16.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017872774000000001 2024-09-17T21:19:16.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252144+0000", 2024-09-17T21:19:16.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019517300000000001 2024-09-17T21:19:16.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252478+0000", 2024-09-17T21:19:16.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033482899999999998 2024-09-17T21:19:16.103 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.103 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.103 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.103 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:482 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:16.104 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:19:16.104 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 39.847704000999997, 2024-09-17T21:19:16.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018988026000000002, 2024-09-17T21:19:16.104 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.104 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.104 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.104 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.104 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:16.104 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:16.104 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 482 2024-09-17T21:19:16.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.104 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:19:16.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:19:16.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234096+0000", 2024-09-17T21:19:16.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2139999999999999e-06 2024-09-17T21:19:16.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234101+0000", 2024-09-17T21:19:16.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7100000000000004e-06 2024-09-17T21:19:16.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234118+0000", 2024-09-17T21:19:16.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6486000000000001e-05 2024-09-17T21:19:16.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234131+0000", 2024-09-17T21:19:16.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2824e-05 2024-09-17T21:19:16.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252552+0000", 2024-09-17T21:19:16.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018421549999999998 2024-09-17T21:19:16.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252746+0000", 2024-09-17T21:19:16.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019424399999999999 2024-09-17T21:19:16.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253082+0000", 2024-09-17T21:19:16.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000335998 2024-09-17T21:19:16.108 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.108 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.108 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.108 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:483 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:16.108 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:19:16.108 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 39.847649380999997, 2024-09-17T21:19:16.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019462882000000001, 2024-09-17T21:19:16.108 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.108 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.108 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.109 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.109 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:16.109 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:16.109 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 483 2024-09-17T21:19:16.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.109 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:19:16.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:19:16.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234150+0000", 2024-09-17T21:19:16.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.127e-06 2024-09-17T21:19:16.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234155+0000", 2024-09-17T21:19:16.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:19:16.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234171+0000", 2024-09-17T21:19:16.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6160000000000001e-05 2024-09-17T21:19:16.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234185+0000", 2024-09-17T21:19:16.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3370000000000001e-05 2024-09-17T21:19:16.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253158+0000", 2024-09-17T21:19:16.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018972968999999999 2024-09-17T21:19:16.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253349+0000", 2024-09-17T21:19:16.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000190733 2024-09-17T21:19:16.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253612+0000", 2024-09-17T21:19:16.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026337000000000002 2024-09-17T21:19:16.114 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.114 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.115 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.115 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:484 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:16.115 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:19:16.115 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 39.847595355000003, 2024-09-17T21:19:16.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020029261999999999, 2024-09-17T21:19:16.115 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.115 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.115 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.115 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.115 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:16.115 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:16.115 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 484 2024-09-17T21:19:16.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.116 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:19:16.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:19:16.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234205+0000", 2024-09-17T21:19:16.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6470000000000001e-06 2024-09-17T21:19:16.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234210+0000", 2024-09-17T21:19:16.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:19:16.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234227+0000", 2024-09-17T21:19:16.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.73e-05 2024-09-17T21:19:16.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234241+0000", 2024-09-17T21:19:16.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3757000000000001e-05 2024-09-17T21:19:16.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253690+0000", 2024-09-17T21:19:16.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019449368000000002 2024-09-17T21:19:16.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253898+0000", 2024-09-17T21:19:16.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000207161 2024-09-17T21:19:16.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254232+0000", 2024-09-17T21:19:16.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033474299999999998 2024-09-17T21:19:16.119 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.119 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.119 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.119 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:485 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1638400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:16.119 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:19:16.119 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 39.847538127999997, 2024-09-17T21:19:16.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020553914, 2024-09-17T21:19:16.119 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.120 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.120 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.120 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.120 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:16.120 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:16.120 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 485 2024-09-17T21:19:16.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.120 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:19:16.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:19:16.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234261+0000", 2024-09-17T21:19:16.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.136e-06 2024-09-17T21:19:16.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234266+0000", 2024-09-17T21:19:16.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0669999999999999e-06 2024-09-17T21:19:16.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234284+0000", 2024-09-17T21:19:16.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7839999999999999e-05 2024-09-17T21:19:16.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234298+0000", 2024-09-17T21:19:16.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3852999999999999e-05 2024-09-17T21:19:16.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254311+0000", 2024-09-17T21:19:16.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020012973 2024-09-17T21:19:16.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254552+0000", 2024-09-17T21:19:16.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024116400000000001 2024-09-17T21:19:16.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254814+0000", 2024-09-17T21:19:16.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000261881 2024-09-17T21:19:16.123 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.123 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.123 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.124 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:486 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:16.124 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:19:16.124 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 39.847481739000003, 2024-09-17T21:19:16.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021010240999999999, 2024-09-17T21:19:16.124 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.124 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.124 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.124 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.124 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:16.124 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:16.124 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 486 2024-09-17T21:19:16.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.125 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:19:16.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:19:16.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234318+0000", 2024-09-17T21:19:16.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.37e-06 2024-09-17T21:19:16.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234324+0000", 2024-09-17T21:19:16.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:19:16.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234341+0000", 2024-09-17T21:19:16.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7127000000000001e-05 2024-09-17T21:19:16.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234354+0000", 2024-09-17T21:19:16.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3210000000000001e-05 2024-09-17T21:19:16.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254896+0000", 2024-09-17T21:19:16.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020541835000000001 2024-09-17T21:19:16.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255092+0000", 2024-09-17T21:19:16.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019634700000000001 2024-09-17T21:19:16.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255327+0000", 2024-09-17T21:19:16.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023475899999999999 2024-09-17T21:19:16.128 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.128 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.128 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.128 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:487 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:16.128 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:19:16.128 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 39.847425543, 2024-09-17T21:19:16.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021453911999999999, 2024-09-17T21:19:16.128 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.128 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.129 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.129 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.129 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:16.129 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:16.129 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 487 2024-09-17T21:19:16.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.129 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:19:16.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:19:16.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234374+0000", 2024-09-17T21:19:16.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3039999999999999e-06 2024-09-17T21:19:16.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234380+0000", 2024-09-17T21:19:16.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:19:16.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234397+0000", 2024-09-17T21:19:16.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7232999999999999e-05 2024-09-17T21:19:16.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234411+0000", 2024-09-17T21:19:16.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3784000000000001e-05 2024-09-17T21:19:16.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255406+0000", 2024-09-17T21:19:16.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020995204 2024-09-17T21:19:16.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255595+0000", 2024-09-17T21:19:16.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018903299999999999 2024-09-17T21:19:16.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255827+0000", 2024-09-17T21:19:16.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023190099999999999 2024-09-17T21:19:16.132 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.132 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.132 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.133 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:488 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:16.133 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:19:16.133 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 39.847369866000001, 2024-09-17T21:19:16.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026887966999999999, 2024-09-17T21:19:16.133 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.133 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.133 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.133 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.133 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:16.133 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:16.133 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 488 2024-09-17T21:19:16.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.133 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:19:16.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:19:16.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234430+0000", 2024-09-17T21:19:16.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1829999999999999e-06 2024-09-17T21:19:16.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234435+0000", 2024-09-17T21:19:16.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9869999999999999e-06 2024-09-17T21:19:16.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234451+0000", 2024-09-17T21:19:16.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6458999999999999e-05 2024-09-17T21:19:16.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234465+0000", 2024-09-17T21:19:16.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3730000000000001e-05 2024-09-17T21:19:16.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255908+0000", 2024-09-17T21:19:16.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021443413000000001 2024-09-17T21:19:16.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256098+0000", 2024-09-17T21:19:16.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018961 2024-09-17T21:19:16.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261317+0000", 2024-09-17T21:19:16.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052185850000000004 2024-09-17T21:19:16.137 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.137 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.137 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.137 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:489 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2162688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:16.137 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:19:16.137 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 39.847314027000003, 2024-09-17T21:19:16.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022632815000000001, 2024-09-17T21:19:16.137 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.137 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.137 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.137 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.138 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:16.138 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:16.138 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 489 2024-09-17T21:19:16.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.138 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:19:16.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:19:16.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234486+0000", 2024-09-17T21:19:16.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5099999999999999e-06 2024-09-17T21:19:16.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234491+0000", 2024-09-17T21:19:16.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.147e-06 2024-09-17T21:19:16.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234508+0000", 2024-09-17T21:19:16.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7405999999999999e-05 2024-09-17T21:19:16.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234522+0000", 2024-09-17T21:19:16.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3677e-05 2024-09-17T21:19:16.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256536+0000", 2024-09-17T21:19:16.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022013464999999999 2024-09-17T21:19:16.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256752+0000", 2024-09-17T21:19:16.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021637800000000001 2024-09-17T21:19:16.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257117+0000", 2024-09-17T21:19:16.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000365232 2024-09-17T21:19:16.141 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.141 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.141 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.142 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:490 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2293760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:16.142 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:19:16.142 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 39.847259237000003, 2024-09-17T21:19:16.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023146462999999999, 2024-09-17T21:19:16.142 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.142 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.142 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.142 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.142 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:16.142 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:16.142 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 490 2024-09-17T21:19:16.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.142 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:19:16.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:19:16.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234541+0000", 2024-09-17T21:19:16.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4869999999999999e-06 2024-09-17T21:19:16.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234559+0000", 2024-09-17T21:19:16.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8105999999999999e-05 2024-09-17T21:19:16.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234577+0000", 2024-09-17T21:19:16.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.836e-05 2024-09-17T21:19:16.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234592+0000", 2024-09-17T21:19:16.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4766000000000001e-05 2024-09-17T21:19:16.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257187+0000", 2024-09-17T21:19:16.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022594725 2024-09-17T21:19:16.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257374+0000", 2024-09-17T21:19:16.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018767299999999999 2024-09-17T21:19:16.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257686+0000", 2024-09-17T21:19:16.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000311346 2024-09-17T21:19:16.146 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.146 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.146 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.146 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:491 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:16.146 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:19:16.146 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 39.847186628000003, 2024-09-17T21:19:16.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023696872000000001, 2024-09-17T21:19:16.146 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.146 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.146 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.146 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.147 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:16.147 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:16.147 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 491 2024-09-17T21:19:16.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.147 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:19:16.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:19:16.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234613+0000", 2024-09-17T21:19:16.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-06 2024-09-17T21:19:16.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234618+0000", 2024-09-17T21:19:16.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3839999999999996e-06 2024-09-17T21:19:16.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234635+0000", 2024-09-17T21:19:16.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.685e-05 2024-09-17T21:19:16.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234648+0000", 2024-09-17T21:19:16.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.313e-05 2024-09-17T21:19:16.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257780+0000", 2024-09-17T21:19:16.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023131946 2024-09-17T21:19:16.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257985+0000", 2024-09-17T21:19:16.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020468400000000001 2024-09-17T21:19:16.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258309+0000", 2024-09-17T21:19:16.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032357499999999999 2024-09-17T21:19:16.150 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.150 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.150 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.150 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:492 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:16.150 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:19:16.151 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 39.847131040999997, 2024-09-17T21:19:16.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024258994999999998, 2024-09-17T21:19:16.151 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.151 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.151 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.151 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.151 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:16.151 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:16.151 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 492 2024-09-17T21:19:16.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.151 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:19:16.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:19:16.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234669+0000", 2024-09-17T21:19:16.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.313e-06 2024-09-17T21:19:16.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234674+0000", 2024-09-17T21:19:16.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1599999999999997e-06 2024-09-17T21:19:16.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234690+0000", 2024-09-17T21:19:16.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.666e-05 2024-09-17T21:19:16.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234704+0000", 2024-09-17T21:19:16.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3043e-05 2024-09-17T21:19:16.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258397+0000", 2024-09-17T21:19:16.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023693204999999998 2024-09-17T21:19:16.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258593+0000", 2024-09-17T21:19:16.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019578999999999999 2024-09-17T21:19:16.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258926+0000", 2024-09-17T21:19:16.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033382399999999998 2024-09-17T21:19:16.154 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.155 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.155 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.155 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:493 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:16.155 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:19:16.155 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 39.847074575999997, 2024-09-17T21:19:16.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024793461999999999, 2024-09-17T21:19:16.155 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.155 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.155 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.155 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.155 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:16.155 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:16.156 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 493 2024-09-17T21:19:16.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.156 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:19:16.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:19:16.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234725+0000", 2024-09-17T21:19:16.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.24e-06 2024-09-17T21:19:16.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234731+0000", 2024-09-17T21:19:16.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8939999999999998e-06 2024-09-17T21:19:16.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234749+0000", 2024-09-17T21:19:16.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7863000000000002e-05 2024-09-17T21:19:16.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234762+0000", 2024-09-17T21:19:16.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3322999999999999e-05 2024-09-17T21:19:16.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259022+0000", 2024-09-17T21:19:16.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024259691999999999 2024-09-17T21:19:16.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259259+0000", 2024-09-17T21:19:16.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023669199999999999 2024-09-17T21:19:16.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259517+0000", 2024-09-17T21:19:16.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025875799999999999 2024-09-17T21:19:16.159 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.159 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.159 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.159 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:494 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:16.159 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:19:16.159 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 39.847016875999998, 2024-09-17T21:19:16.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025251952000000001, 2024-09-17T21:19:16.160 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.160 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.160 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.160 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.160 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:16.160 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:16.160 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 494 2024-09-17T21:19:16.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.160 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:19:16.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:19:16.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234783+0000", 2024-09-17T21:19:16.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2440000000000001e-06 2024-09-17T21:19:16.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234789+0000", 2024-09-17T21:19:16.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:19:16.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234806+0000", 2024-09-17T21:19:16.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7450000000000001e-05 2024-09-17T21:19:16.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234819+0000", 2024-09-17T21:19:16.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3326000000000001e-05 2024-09-17T21:19:16.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259585+0000", 2024-09-17T21:19:16.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024765579999999999 2024-09-17T21:19:16.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259787+0000", 2024-09-17T21:19:16.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202444 2024-09-17T21:19:16.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260033+0000", 2024-09-17T21:19:16.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024611499999999998 2024-09-17T21:19:16.164 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.164 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.164 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.164 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:495 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:16.164 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:19:16.164 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 39.846961512999997, 2024-09-17T21:19:16.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025776342000000001, 2024-09-17T21:19:16.164 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.164 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.164 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.164 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.164 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:16.164 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:16.165 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 495 2024-09-17T21:19:16.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.165 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:19:16.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:19:16.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234838+0000", 2024-09-17T21:19:16.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2130000000000001e-06 2024-09-17T21:19:16.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234843+0000", 2024-09-17T21:19:16.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9899999999999997e-06 2024-09-17T21:19:16.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234860+0000", 2024-09-17T21:19:16.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6973000000000002e-05 2024-09-17T21:19:16.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234873+0000", 2024-09-17T21:19:16.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3179999999999999e-05 2024-09-17T21:19:16.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260135+0000", 2024-09-17T21:19:16.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025262118 2024-09-17T21:19:16.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260364+0000", 2024-09-17T21:19:16.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022830699999999999 2024-09-17T21:19:16.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260613+0000", 2024-09-17T21:19:16.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024956099999999998 2024-09-17T21:19:16.168 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.168 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.168 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.168 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:496 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:16.168 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:19:16.168 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 39.846908233000001, 2024-09-17T21:19:16.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026376773999999999, 2024-09-17T21:19:16.169 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.169 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.169 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.169 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.169 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:16.169 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:16.169 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 496 2024-09-17T21:19:16.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.169 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:19:16.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:19:16.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234891+0000", 2024-09-17T21:19:16.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1230000000000001e-06 2024-09-17T21:19:16.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234896+0000", 2024-09-17T21:19:16.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9599999999999999e-06 2024-09-17T21:19:16.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234913+0000", 2024-09-17T21:19:16.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6747000000000001e-05 2024-09-17T21:19:16.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234926+0000", 2024-09-17T21:19:16.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-05 2024-09-17T21:19:16.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260675+0000", 2024-09-17T21:19:16.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025748979000000002 2024-09-17T21:19:16.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260883+0000", 2024-09-17T21:19:16.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020848399999999999 2024-09-17T21:19:16.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261267+0000", 2024-09-17T21:19:16.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038345099999999998 2024-09-17T21:19:16.172 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.172 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.173 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.173 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:682 3.9 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2455552~12288,write 8126464~10240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:16.173 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:19:16.173 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 39.812971181999998, 2024-09-17T21:19:16.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018447261999999999, 2024-09-17T21:19:16.173 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.173 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.173 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:16.173 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.173 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:16.173 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:16.173 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 682 2024-09-17T21:19:16.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.174 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:19:16.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:19:16.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268830+0000", 2024-09-17T21:19:16.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6419999999999999e-06 2024-09-17T21:19:16.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271673+0000", 2024-09-17T21:19:16.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028432610000000001 2024-09-17T21:19:16.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271700+0000", 2024-09-17T21:19:16.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7200999999999999e-05 2024-09-17T21:19:16.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271715+0000", 2024-09-17T21:19:16.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5145e-05 2024-09-17T21:19:16.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271785+0000", 2024-09-17T21:19:16.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9519999999999998e-05 2024-09-17T21:19:16.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271918+0000", 2024-09-17T21:19:16.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013330899999999999 2024-09-17T21:19:16.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:16.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.272449+0000", 2024-09-17T21:19:16.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00053051200000000002 2024-09-17T21:19:16.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:16.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.278822+0000", 2024-09-17T21:19:16.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0063731619999999999 2024-09-17T21:19:16.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:16.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287167+0000", 2024-09-17T21:19:16.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083452149999999996 2024-09-17T21:19:16.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:16.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287213+0000", 2024-09-17T21:19:16.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5972e-05 2024-09-17T21:19:16.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287274+0000", 2024-09-17T21:19:16.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1322999999999995e-05 2024-09-17T21:19:16.178 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.178 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.178 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.178 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.178 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.178 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:16.180 INFO:teuthology.orchestra.run.smithi184.stdout:2761 trunc from 0xcaa8800 to 0x1dd5a00 2024-09-17T21:19:16.180 INFO:teuthology.orchestra.run.smithi184.stdout:2764 read 0xc2bc00 thru 0xc32449 (0x684a bytes) 2024-09-17T21:19:16.180 INFO:teuthology.orchestra.run.smithi184.stdout:2768 read 0xe59400 thru 0xe673da (0xdfdb bytes) 2024-09-17T21:19:16.181 INFO:teuthology.orchestra.run.smithi184.stdout:2770 write 0x4fa3600 thru 0x4fa747c (0x3e7d bytes) 2024-09-17T21:19:16.181 INFO:teuthology.orchestra.run.smithi184.stdout:2771 write 0x5ab0200 thru 0x5abc8d4 (0xc6d5 bytes) 2024-09-17T21:19:16.181 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:19:16.181 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:16.175+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:16.224 INFO:teuthology.orchestra.run.smithi184.stdout:2772 write 0xda88e00 thru 0xda973f5 (0xe5f6 bytes) 2024-09-17T21:19:16.227 INFO:teuthology.orchestra.run.smithi184.stdout:2774 read 0xab13c00 thru 0xab21553 (0xd954 bytes) 2024-09-17T21:19:16.227 INFO:teuthology.orchestra.run.smithi184.stdout:2775 read 0x1e45e00 thru 0x1e49265 (0x3466 bytes) 2024-09-17T21:19:16.227 INFO:teuthology.orchestra.run.smithi184.stdout:2776 punch from 0x9e8200 to 0x9f7a00, (0xf800 bytes) 2024-09-17T21:19:16.228 INFO:teuthology.orchestra.run.smithi184.stdout:2780 read 0xb348c00 thru 0xb350dac (0x81ad bytes) 2024-09-17T21:19:16.228 INFO:teuthology.orchestra.run.smithi184.stdout:2781 read 0x7380800 thru 0x738f4f4 (0xecf5 bytes) 2024-09-17T21:19:16.228 INFO:teuthology.orchestra.run.smithi184.stdout:2782 write 0x78a9800 thru 0x78ae93e (0x513f bytes) 2024-09-17T21:19:16.255 INFO:teuthology.orchestra.run.smithi184.stdout:2783 write 0xdfa1400 thru 0xdfa7d72 (0x6973 bytes) 2024-09-17T21:19:16.256 INFO:teuthology.orchestra.run.smithi184.stdout:2786 write 0x870be00 thru 0x870d596 (0x1797 bytes) 2024-09-17T21:19:16.256 INFO:teuthology.orchestra.run.smithi184.stdout:2787 write 0xae9600 thru 0xaecd96 (0x3797 bytes) 2024-09-17T21:19:16.256 INFO:teuthology.orchestra.run.smithi184.stdout:2788 punch from 0xc29aa00 to 0xc2a4000, (0x9600 bytes) 2024-09-17T21:19:16.257 INFO:teuthology.orchestra.run.smithi184.stdout:2789 punch from 0x8fd3000 to 0x8fd5000, (0x2000 bytes) 2024-09-17T21:19:16.257 INFO:teuthology.orchestra.run.smithi184.stdout:2790 punch from 0x55c2600 to 0x55c3200, (0xc00 bytes) 2024-09-17T21:19:16.257 INFO:teuthology.orchestra.run.smithi184.stdout:2791 write 0x5a97200 thru 0x5a9e00f (0x6e10 bytes) 2024-09-17T21:19:16.259 INFO:teuthology.orchestra.run.smithi184.stdout:2792 read 0xb355600 thru 0xb35d4cb (0x7ecc bytes) 2024-09-17T21:19:16.259 INFO:teuthology.orchestra.run.smithi184.stdout:2793 write 0x55f6200 thru 0x55fc34e (0x614f bytes) 2024-09-17T21:19:16.259 INFO:teuthology.orchestra.run.smithi184.stdout:2794 punch from 0x2f75e00 to 0x2f82600, (0xc800 bytes) 2024-09-17T21:19:16.260 INFO:teuthology.orchestra.run.smithi184.stdout:2796 read 0x8bd200 thru 0x8bf1b2 (0x1fb3 bytes) 2024-09-17T21:19:16.260 INFO:teuthology.orchestra.run.smithi184.stdout:2797 read 0x2234800 thru 0x22376c1 (0x2ec2 bytes) 2024-09-17T21:19:16.261 INFO:teuthology.orchestra.run.smithi184.stdout:2798 read 0x7ca8a00 thru 0x7cae721 (0x5d22 bytes) 2024-09-17T21:19:16.261 INFO:teuthology.orchestra.run.smithi184.stdout:2799 write 0x4d09c00 thru 0x4d139b0 (0x9db1 bytes) 2024-09-17T21:19:16.261 INFO:teuthology.orchestra.run.smithi184.stdout:2800 read 0x7e59400 thru 0x7e5aeb0 (0x1ab1 bytes) 2024-09-17T21:19:16.261 INFO:teuthology.orchestra.run.smithi184.stdout:2801 read 0x919de00 thru 0x91a5c93 (0x7e94 bytes) 2024-09-17T21:19:16.261 INFO:teuthology.orchestra.run.smithi184.stdout:2802 write 0x700a00 thru 0x70b78b (0xad8c bytes) 2024-09-17T21:19:16.262 INFO:teuthology.orchestra.run.smithi184.stdout:2803 punch from 0x468b600 to 0x4699600, (0xe000 bytes) 2024-09-17T21:19:16.263 INFO:teuthology.orchestra.run.smithi184.stdout:2804 read 0x9f5da00 thru 0x9f65e03 (0x8404 bytes) 2024-09-17T21:19:16.280 INFO:teuthology.orchestra.run.smithi184.stdout:2805 trunc from 0xdfa7e00 to 0x4ed6400 2024-09-17T21:19:16.280 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:16.276+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:16.307 INFO:teuthology.orchestra.run.smithi184.stdout:2806 read 0x2da8200 thru 0x2db1404 (0x9205 bytes) 2024-09-17T21:19:16.309 INFO:teuthology.orchestra.run.smithi184.stdout:2809 read 0x3bdc000 thru 0x3be9727 (0xd728 bytes) 2024-09-17T21:19:16.309 INFO:teuthology.orchestra.run.smithi184.stdout:2810 write 0x355200 thru 0x358677 (0x3478 bytes) 2024-09-17T21:19:16.311 INFO:teuthology.orchestra.run.smithi184.stdout:2815 write 0x342f200 thru 0x3430e88 (0x1c89 bytes) 2024-09-17T21:19:16.311 INFO:teuthology.orchestra.run.smithi184.stdout:2816 read 0xb55a00 thru 0xb641a0 (0xe7a1 bytes) 2024-09-17T21:19:16.311 INFO:teuthology.orchestra.run.smithi184.stdout:2818 read 0x1de6a00 thru 0x1de77fc (0xdfd bytes) 2024-09-17T21:19:16.311 INFO:teuthology.orchestra.run.smithi184.stdout:2819 read 0x3e02400 thru 0x3e12129 (0xfd2a bytes) 2024-09-17T21:19:16.341 INFO:teuthology.orchestra.run.smithi184.stdout:2820 write 0xa9e0800 thru 0xa9e8d43 (0x8544 bytes) 2024-09-17T21:19:16.342 INFO:teuthology.orchestra.run.smithi184.stdout:2822 read 0x4ef5e00 thru 0x4efd832 (0x7a33 bytes) 2024-09-17T21:19:16.353 INFO:teuthology.orchestra.run.smithi184.stdout:2824 trunc from 0xa9e8e00 to 0x3c96000 2024-09-17T21:19:16.373 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4871.0:5931 3.5 3:a1656b6c:::rbd_header.115265216c1c:head [call rbd.set_size in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-17T21:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:16.372436+0000", 2024-09-17T21:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.00028437500000000001, 2024-09-17T21:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031580200000000001, 2024-09-17T21:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4871", 2024-09-17T21:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 5931 2024-09-17T21:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:16.372436+0000", 2024-09-17T21:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:16.372436+0000", 2024-09-17T21:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:16.372438+0000", 2024-09-17T21:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:16.372445+0000", 2024-09-17T21:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2649999999999999e-06 2024-09-17T21:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:16.372462+0000", 2024-09-17T21:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7496e-05 2024-09-17T21:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:16.372471+0000", 2024-09-17T21:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9730000000000003e-06 2024-09-17T21:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:16.372522+0000", 2024-09-17T21:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1201e-05 2024-09-17T21:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:16.372573+0000", 2024-09-17T21:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0188000000000001e-05 2024-09-17T21:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:16.372712+0000", 2024-09-17T21:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013923599999999999 2024-09-17T21:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-09-17T21:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 1 2024-09-17T21:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:16.382 INFO:teuthology.orchestra.run.smithi184.stdout:2827 punch from 0x2b76200 to 0x2b84a00, (0xe800 bytes) 2024-09-17T21:19:16.382 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:16.376+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:16.385 INFO:teuthology.orchestra.run.smithi184.stdout:2828 punch from 0x3986600 to 0x3988200, (0x1c00 bytes) 2024-09-17T21:19:16.385 INFO:teuthology.orchestra.run.smithi184.stdout:2830 read 0x393f000 thru 0x393f7fe (0x7ff bytes) 2024-09-17T21:19:16.385 INFO:teuthology.orchestra.run.smithi184.stdout:2831 read 0x634400 thru 0x643344 (0xef45 bytes) 2024-09-17T21:19:16.388 INFO:teuthology.orchestra.run.smithi184.stdout:2832 write 0x3c0d000 thru 0x3c0f2e5 (0x22e6 bytes) 2024-09-17T21:19:16.388 INFO:teuthology.orchestra.run.smithi184.stdout:2834 read 0x1ccb000 thru 0x1ccfaa3 (0x4aa4 bytes) 2024-09-17T21:19:16.388 INFO:teuthology.orchestra.run.smithi184.stdout:2838 read 0x2749000 thru 0x2749f66 (0xf67 bytes) 2024-09-17T21:19:16.391 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:19:16.391 INFO:teuthology.orchestra.run.smithi184.stdout:2839 trunc from 0x3c96000 to 0x544ae00 2024-09-17T21:19:16.413 INFO:teuthology.orchestra.run.smithi184.stdout:2840 read 0x60800 thru 0x69387 (0x8b88 bytes) 2024-09-17T21:19:16.446 INFO:teuthology.orchestra.run.smithi184.stdout:2841 write 0xe5e0600 thru 0xe5f026d (0xfc6e bytes) 2024-09-17T21:19:16.448 INFO:teuthology.orchestra.run.smithi184.stdout:2843 write 0xb83d600 thru 0xb84d1e9 (0xfbea bytes) 2024-09-17T21:19:16.460 INFO:teuthology.orchestra.run.smithi184.stdout:2845 trunc from 0xe5f0400 to 0x6e84800 2024-09-17T21:19:16.482 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:16.480+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:16.489 INFO:teuthology.orchestra.run.smithi083.stderr:marked in osd.0. 2024-09-17T21:19:16.493 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-17T21:19:16.500 INFO:teuthology.orchestra.run.smithi184.stdout:2850 read 0x3b98000 thru 0x3b9d08f (0x5090 bytes) 2024-09-17T21:19:16.526 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-09-17T21:19:16.565 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:16.565 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:16.565 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:16.565 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:16.565 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:16.579 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:19:16.622 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:16.620+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:16.684 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:16.680+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:16.746 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:16.746 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:19:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.138835823000001, 2024-09-17T21:19:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:19:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:19:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:19:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:19:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:19:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:19:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:19:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:19:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:19:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:19:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:19:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:19:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:19:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:19:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:19:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:19:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:19:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:19:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.128889186999999, 2024-09-17T21:19:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:19:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:19:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:19:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:19:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:19:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:19:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:19:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:19:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:19:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:19:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:19:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:19:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:19:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:19:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:19:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:19:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:19:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:19:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 53.885738347, 2024-09-17T21:19:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:19:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:19:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:19:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:19:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:19:16.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:19:16.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:19:16.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:19:16.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:19:16.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:19:16.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:19:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:19:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:19:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:19:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:19:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:19:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:19:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:19:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:19:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:19:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:19:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:19:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:19:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:19:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:19:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:19:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:19:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 53.876092860999997, 2024-09-17T21:19:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:19:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:19:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:19:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:19:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:19:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:19:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:19:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:19:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:19:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:19:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:19:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:19:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:19:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:19:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:19:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:19:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:19:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:19:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:19:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:19:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:19:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:19:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:19:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:19:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:19:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:19:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 40.507156827999999, 2024-09-17T21:19:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:19:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:19:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:19:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:19:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:19:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:19:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:19:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:19:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:19:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:19:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:19:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:19:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:19:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:19:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:19:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:19:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:565 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:19:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 40.503367863000001, 2024-09-17T21:19:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024193392000000001, 2024-09-17T21:19:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 565 2024-09-17T21:19:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:19:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:19:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239660+0000", 2024-09-17T21:19:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:19:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239666+0000", 2024-09-17T21:19:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8699999999999997e-06 2024-09-17T21:19:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239688+0000", 2024-09-17T21:19:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.213e-05 2024-09-17T21:19:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239703+0000", 2024-09-17T21:19:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4813e-05 2024-09-17T21:19:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263427+0000", 2024-09-17T21:19:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023724630999999999 2024-09-17T21:19:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263614+0000", 2024-09-17T21:19:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018664500000000001 2024-09-17T21:19:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263852+0000", 2024-09-17T21:19:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023793999999999999 2024-09-17T21:19:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:819 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5111808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.305584538000002, 2024-09-17T21:19:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024344811000000001, 2024-09-17T21:19:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 819 2024-09-17T21:19:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437443+0000", 2024-09-17T21:19:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1969999999999999e-06 2024-09-17T21:19:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437448+0000", 2024-09-17T21:19:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9490000000000004e-06 2024-09-17T21:19:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437467+0000", 2024-09-17T21:19:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8862999999999999e-05 2024-09-17T21:19:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437480+0000", 2024-09-17T21:19:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2833000000000001e-05 2024-09-17T21:19:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461552+0000", 2024-09-17T21:19:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024072695000000002 2024-09-17T21:19:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461639+0000", 2024-09-17T21:19:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6996000000000005e-05 2024-09-17T21:19:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461787+0000", 2024-09-17T21:19:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014727800000000001 2024-09-17T21:19:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.305472265999999, 2024-09-17T21:19:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:19:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:19:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:19:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:19:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:19:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:19:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:19:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:19:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:19:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:19:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:19:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:19:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:19:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:19:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:19:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:19:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:833 3.5 3:a772dfb1:::rbd_data.11523651b015.0000000000000001:d [sparse-read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.305037980000002, 2024-09-17T21:19:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024216024999999999, 2024-09-17T21:19:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 833 2024-09-17T21:19:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437990+0000", 2024-09-17T21:19:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2500000000000001e-06 2024-09-17T21:19:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437995+0000", 2024-09-17T21:19:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1240000000000004e-06 2024-09-17T21:19:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438012+0000", 2024-09-17T21:19:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7572e-05 2024-09-17T21:19:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438031+0000", 2024-09-17T21:19:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8612000000000001e-05 2024-09-17T21:19:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461830+0000", 2024-09-17T21:19:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023799135999999999 2024-09-17T21:19:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462005+0000", 2024-09-17T21:19:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000174747 2024-09-17T21:19:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:19:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019958399999999999 2024-09-17T21:19:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.304977689000001, 2024-09-17T21:19:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:19:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:19:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:19:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:19:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:19:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:19:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:19:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:19:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:19:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:19:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:19:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:19:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:19:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:19:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:19:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:19:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.304922307999998, 2024-09-17T21:19:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:19:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:19:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:19:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:19:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:19:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:19:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:19:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:19:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:19:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:19:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:19:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:19:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:19:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:19:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:19:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:19:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.304866029999999, 2024-09-17T21:19:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:19:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:19:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:19:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:19:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:19:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:19:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:19:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:19:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:19:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:19:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:19:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:19:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:19:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:19:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:19:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:19:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.303916391000001, 2024-09-17T21:19:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:19:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:19:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:19:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:19:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:19:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:19:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:19:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:19:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:19:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:19:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:19:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:19:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:19:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:19:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:19:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:19:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.303837771000001, 2024-09-17T21:19:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:19:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:19:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:19:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:19:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:19:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:19:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:19:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:19:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:19:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:19:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:19:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:19:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:19:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:19:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:19:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:19:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:847 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 4653056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439245+0000", 2024-09-17T21:19:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.303781333, 2024-09-17T21:19:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024172945000000001, 2024-09-17T21:19:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 847 2024-09-17T21:19:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439245+0000", 2024-09-17T21:19:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439245+0000", 2024-09-17T21:19:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439246+0000", 2024-09-17T21:19:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:19:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439252+0000", 2024-09-17T21:19:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3569999999999997e-06 2024-09-17T21:19:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439269+0000", 2024-09-17T21:19:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7316000000000002e-05 2024-09-17T21:19:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439282+0000", 2024-09-17T21:19:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3269e-05 2024-09-17T21:19:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463240+0000", 2024-09-17T21:19:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02395721 2024-09-17T21:19:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463306+0000", 2024-09-17T21:19:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6618000000000005e-05 2024-09-17T21:19:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463418+0000", 2024-09-17T21:19:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111672 2024-09-17T21:19:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.303724832, 2024-09-17T21:19:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:19:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:19:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:19:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:19:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:19:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:19:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:19:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:19:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:19:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:19:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:19:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:19:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:19:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:19:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:19:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:19:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:849 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5963776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.303130796000001, 2024-09-17T21:19:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024537514, 2024-09-17T21:19:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 849 2024-09-17T21:19:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439897+0000", 2024-09-17T21:19:16.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9099999999999999e-06 2024-09-17T21:19:16.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439905+0000", 2024-09-17T21:19:16.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0870000000000007e-06 2024-09-17T21:19:16.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439924+0000", 2024-09-17T21:19:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9042000000000002e-05 2024-09-17T21:19:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439941+0000", 2024-09-17T21:19:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.624e-05 2024-09-17T21:19:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464132+0000", 2024-09-17T21:19:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024191579000000001 2024-09-17T21:19:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464291+0000", 2024-09-17T21:19:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015904800000000001 2024-09-17T21:19:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464433+0000", 2024-09-17T21:19:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014160799999999999 2024-09-17T21:19:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.303063777999999, 2024-09-17T21:19:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:19:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:19:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:19:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:19:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:19:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:19:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:19:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:19:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:19:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:19:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:19:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:19:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:19:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:19:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:19:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:19:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.302998967000001, 2024-09-17T21:19:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:19:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:19:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:19:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:19:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:19:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:19:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:19:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:19:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:19:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:19:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:19:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:19:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:19:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:19:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:19:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:19:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.30293799, 2024-09-17T21:19:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:19:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:19:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:19:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:19:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:19:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:19:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:19:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:19:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:19:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:19:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:19:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:19:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:19:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:19:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:19:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:19:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:16.841 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:16.842 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:19:16.843 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:16.839+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:16.943 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:16.940+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:16.975 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:16.975 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:16.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:16.975 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:19:16.975 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:19:16.975 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.46376649399999997, 2024-09-17T21:19:16.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.46380883299999998, 2024-09-17T21:19:16.975 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:16.975 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:16.975 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for peered", 2024-09-17T21:19:16.976 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:16.976 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:19:16.976 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:19:16.976 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:19:16.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:16.976 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:16.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:16.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:16.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:19:16.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:16.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:16.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:16.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:16.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:19:16.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:16.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:16.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:16.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:16.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:19:16.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:19:16.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:16.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:16.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:16.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:19:16.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:19:16.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:16.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:16.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:16.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:19:16.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:19:16.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:16.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:16.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:16.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:19:16.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:19:16.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:16.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:16.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:16.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:19:16.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:19:16.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:16.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:16.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:16.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:19:16.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:19:16.979 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:16.979 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:16.979 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:16.979 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:16.979 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:19:16.979 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 1 2024-09-17T21:19:16.979 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:16.989 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:19:17.045 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:17.043+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:17.109 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:17.109 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:17.109 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:17.109 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:17.110 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:17.123 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:19:17.146 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:17.140+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:17.247 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:17.243+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:17.302 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:17.302 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:17.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:17.302 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:17.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.302 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:389 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.302 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.194936+0000", 2024-09-17T21:19:17.302 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.104716211000003, 2024-09-17T21:19:17.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041176510999999999, 2024-09-17T21:19:17.302 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 389 2024-09-17T21:19:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194936+0000", 2024-09-17T21:19:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194936+0000", 2024-09-17T21:19:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194937+0000", 2024-09-17T21:19:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6270000000000001e-06 2024-09-17T21:19:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194944+0000", 2024-09-17T21:19:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5729999999999996e-06 2024-09-17T21:19:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194961+0000", 2024-09-17T21:19:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7623000000000001e-05 2024-09-17T21:19:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.194974+0000", 2024-09-17T21:19:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2989999999999999e-05 2024-09-17T21:19:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.235848+0000", 2024-09-17T21:19:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.040874099999999997 2024-09-17T21:19:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.235971+0000", 2024-09-17T21:19:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012254400000000001 2024-09-17T21:19:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236112+0000", 2024-09-17T21:19:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014105399999999999 2024-09-17T21:19:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:406 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 2359296~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:19:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.104597405, 2024-09-17T21:19:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042448018999999997, 2024-09-17T21:19:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 406 2024-09-17T21:19:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:19:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:19:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195056+0000", 2024-09-17T21:19:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2729999999999999e-06 2024-09-17T21:19:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195060+0000", 2024-09-17T21:19:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5399999999999997e-06 2024-09-17T21:19:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195076+0000", 2024-09-17T21:19:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6217000000000001e-05 2024-09-17T21:19:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195093+0000", 2024-09-17T21:19:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.683e-05 2024-09-17T21:19:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236217+0000", 2024-09-17T21:19:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041124102000000003 2024-09-17T21:19:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236339+0000", 2024-09-17T21:19:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121876 2024-09-17T21:19:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237502+0000", 2024-09-17T21:19:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011631809999999999 2024-09-17T21:19:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:416 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2228224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:19:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.104099378999997, 2024-09-17T21:19:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042361990000000002, 2024-09-17T21:19:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 416 2024-09-17T21:19:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:19:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:19:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195554+0000", 2024-09-17T21:19:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9300000000000002e-06 2024-09-17T21:19:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195563+0000", 2024-09-17T21:19:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2770000000000005e-06 2024-09-17T21:19:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195586+0000", 2024-09-17T21:19:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3889000000000001e-05 2024-09-17T21:19:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195603+0000", 2024-09-17T21:19:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6084000000000001e-05 2024-09-17T21:19:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237563+0000", 2024-09-17T21:19:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041960856999999997 2024-09-17T21:19:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237695+0000", 2024-09-17T21:19:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013144400000000001 2024-09-17T21:19:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237914+0000", 2024-09-17T21:19:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000219509 2024-09-17T21:19:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:439 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:19:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.103535477000001, 2024-09-17T21:19:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043053811999999997, 2024-09-17T21:19:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 439 2024-09-17T21:19:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:19:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:19:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196118+0000", 2024-09-17T21:19:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3969999999999999e-06 2024-09-17T21:19:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196157+0000", 2024-09-17T21:19:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9826999999999997e-05 2024-09-17T21:19:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196179+0000", 2024-09-17T21:19:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1756e-05 2024-09-17T21:19:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196194+0000", 2024-09-17T21:19:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4707e-05 2024-09-17T21:19:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237980+0000", 2024-09-17T21:19:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041786219999999999 2024-09-17T21:19:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.238065+0000", 2024-09-17T21:19:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4560000000000004e-05 2024-09-17T21:19:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239170+0000", 2024-09-17T21:19:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001105345 2024-09-17T21:19:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:440 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:19:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.103432974, 2024-09-17T21:19:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043890039999999998, 2024-09-17T21:19:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 440 2024-09-17T21:19:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:19:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:19:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196220+0000", 2024-09-17T21:19:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.254e-06 2024-09-17T21:19:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196225+0000", 2024-09-17T21:19:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7829999999999998e-06 2024-09-17T21:19:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196241+0000", 2024-09-17T21:19:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6322999999999999e-05 2024-09-17T21:19:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196258+0000", 2024-09-17T21:19:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7056000000000001e-05 2024-09-17T21:19:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239229+0000", 2024-09-17T21:19:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042971058999999999 2024-09-17T21:19:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239403+0000", 2024-09-17T21:19:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017399200000000001 2024-09-17T21:19:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240109+0000", 2024-09-17T21:19:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00070557300000000003 2024-09-17T21:19:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:450 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:19:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.103072023000003, 2024-09-17T21:19:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043994521000000002, 2024-09-17T21:19:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 450 2024-09-17T21:19:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:19:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:19:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196581+0000", 2024-09-17T21:19:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4500000000000001e-06 2024-09-17T21:19:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196588+0000", 2024-09-17T21:19:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7530000000000004e-06 2024-09-17T21:19:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196607+0000", 2024-09-17T21:19:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8987e-05 2024-09-17T21:19:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196620+0000", 2024-09-17T21:19:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3589e-05 2024-09-17T21:19:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240225+0000", 2024-09-17T21:19:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043604878999999999 2024-09-17T21:19:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240377+0000", 2024-09-17T21:19:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015146699999999999 2024-09-17T21:19:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240574+0000", 2024-09-17T21:19:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019739599999999999 2024-09-17T21:19:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:464 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:19:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.102821601999999, 2024-09-17T21:19:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044651692999999999, 2024-09-17T21:19:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.331 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 464 2024-09-17T21:19:17.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.331 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:19:17.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:19:17.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196838+0000", 2024-09-17T21:19:17.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4970000000000004e-06 2024-09-17T21:19:17.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196859+0000", 2024-09-17T21:19:17.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1166e-05 2024-09-17T21:19:17.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196879+0000", 2024-09-17T21:19:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.995e-05 2024-09-17T21:19:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196896+0000", 2024-09-17T21:19:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6929999999999999e-05 2024-09-17T21:19:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240617+0000", 2024-09-17T21:19:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043721784999999999 2024-09-17T21:19:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240702+0000", 2024-09-17T21:19:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4060999999999994e-05 2024-09-17T21:19:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241482+0000", 2024-09-17T21:19:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000780304 2024-09-17T21:19:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:475 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:19:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.102552469999999, 2024-09-17T21:19:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044661981000000003, 2024-09-17T21:19:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 475 2024-09-17T21:19:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:19:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:19:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197100+0000", 2024-09-17T21:19:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.206e-06 2024-09-17T21:19:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197105+0000", 2024-09-17T21:19:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.527e-06 2024-09-17T21:19:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197124+0000", 2024-09-17T21:19:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:19:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197136+0000", 2024-09-17T21:19:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2857000000000001e-05 2024-09-17T21:19:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241539+0000", 2024-09-17T21:19:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044402077999999998 2024-09-17T21:19:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241653+0000", 2024-09-17T21:19:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114931 2024-09-17T21:19:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241761+0000", 2024-09-17T21:19:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107753 2024-09-17T21:19:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:476 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:19:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.102498644999997, 2024-09-17T21:19:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045038976000000001, 2024-09-17T21:19:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 476 2024-09-17T21:19:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:19:17.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:19:17.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197154+0000", 2024-09-17T21:19:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.127e-06 2024-09-17T21:19:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197159+0000", 2024-09-17T21:19:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4569999999999998e-06 2024-09-17T21:19:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197174+0000", 2024-09-17T21:19:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5173e-05 2024-09-17T21:19:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197186+0000", 2024-09-17T21:19:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:19:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241863+0000", 2024-09-17T21:19:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044676826000000003 2024-09-17T21:19:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241998+0000", 2024-09-17T21:19:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013476100000000001 2024-09-17T21:19:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242192+0000", 2024-09-17T21:19:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019436900000000001 2024-09-17T21:19:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:494 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.102323462000001, 2024-09-17T21:19:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046244995999999997, 2024-09-17T21:19:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 494 2024-09-17T21:19:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197329+0000", 2024-09-17T21:19:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.173e-06 2024-09-17T21:19:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197334+0000", 2024-09-17T21:19:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0499999999999999e-06 2024-09-17T21:19:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197352+0000", 2024-09-17T21:19:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7133e-05 2024-09-17T21:19:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197367+0000", 2024-09-17T21:19:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5123000000000001e-05 2024-09-17T21:19:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:19:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044989829000000002 2024-09-17T21:19:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242526+0000", 2024-09-17T21:19:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016968199999999999 2024-09-17T21:19:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243573+0000", 2024-09-17T21:19:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010470060000000001 2024-09-17T21:19:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:502 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:19:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.100089803000003, 2024-09-17T21:19:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044346086999999999, 2024-09-17T21:19:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 502 2024-09-17T21:19:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:19:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:19:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199563+0000", 2024-09-17T21:19:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2929999999999999e-06 2024-09-17T21:19:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199569+0000", 2024-09-17T21:19:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5330000000000002e-06 2024-09-17T21:19:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199614+0000", 2024-09-17T21:19:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4793000000000003e-05 2024-09-17T21:19:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199628+0000", 2024-09-17T21:19:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4037e-05 2024-09-17T21:19:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243665+0000", 2024-09-17T21:19:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044037106999999999 2024-09-17T21:19:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243767+0000", 2024-09-17T21:19:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010279499999999999 2024-09-17T21:19:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243908+0000", 2024-09-17T21:19:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140529 2024-09-17T21:19:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:523 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:19:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.099643649999997, 2024-09-17T21:19:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044789645000000003, 2024-09-17T21:19:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 523 2024-09-17T21:19:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:19:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:19:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:19:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9e-06 2024-09-17T21:19:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200021+0000", 2024-09-17T21:19:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9499999999999998e-06 2024-09-17T21:19:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200038+0000", 2024-09-17T21:19:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.73e-05 2024-09-17T21:19:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200051+0000", 2024-09-17T21:19:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.322e-05 2024-09-17T21:19:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243987+0000", 2024-09-17T21:19:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043935386999999999 2024-09-17T21:19:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244136+0000", 2024-09-17T21:19:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014868300000000001 2024-09-17T21:19:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244798+0000", 2024-09-17T21:19:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00066220500000000002 2024-09-17T21:19:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:524 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.099577433999997, 2024-09-17T21:19:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045737414999999997, 2024-09-17T21:19:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 524 2024-09-17T21:19:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200076+0000", 2024-09-17T21:19:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:19:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200081+0000", 2024-09-17T21:19:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7030000000000003e-06 2024-09-17T21:19:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200099+0000", 2024-09-17T21:19:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7836999999999999e-05 2024-09-17T21:19:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200113+0000", 2024-09-17T21:19:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3570000000000001e-05 2024-09-17T21:19:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244866+0000", 2024-09-17T21:19:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044753573999999997 2024-09-17T21:19:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244972+0000", 2024-09-17T21:19:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106111 2024-09-17T21:19:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245812+0000", 2024-09-17T21:19:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083923999999999999 2024-09-17T21:19:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:525 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3670016~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.099426051999998, 2024-09-17T21:19:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046645697, 2024-09-17T21:19:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 525 2024-09-17T21:19:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:19:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:19:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200234+0000", 2024-09-17T21:19:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:19:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200254+0000", 2024-09-17T21:19:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9959999999999999e-05 2024-09-17T21:19:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200293+0000", 2024-09-17T21:19:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9755999999999997e-05 2024-09-17T21:19:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245871+0000", 2024-09-17T21:19:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045577871999999998 2024-09-17T21:19:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246030+0000", 2024-09-17T21:19:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158975 2024-09-17T21:19:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246871+0000", 2024-09-17T21:19:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084111699999999995 2024-09-17T21:19:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.099180377000003, 2024-09-17T21:19:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:19:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:19:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:19:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:19:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:19:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:19:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:19:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:19:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:19:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:19:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:19:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:19:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:19:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:19:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:19:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:19:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.099001694999998, 2024-09-17T21:19:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:19:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:19:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:19:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:19:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:19:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:19:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:19:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:19:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:19:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:19:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:19:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:19:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:19:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:19:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:19:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:19:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.098942422, 2024-09-17T21:19:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:19:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:19:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:19:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:19:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:19:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:19:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:19:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:19:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:19:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:19:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:19:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:19:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:19:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:19:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:19:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:19:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.098800226999998, 2024-09-17T21:19:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:19:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:19:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:19:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:19:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:19:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:19:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:19:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:19:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:19:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:19:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:19:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:19:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:19:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:19:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:19:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:19:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.098744304, 2024-09-17T21:19:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:19:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:19:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:19:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:19:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:19:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:19:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:19:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:19:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:19:17.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:19:17.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:19:17.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:19:17.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:19:17.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:19:17.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:19:17.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:19:17.391 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.391 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.391 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.391 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.391 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:17.391 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.098690687000001, 2024-09-17T21:19:17.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:19:17.391 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.392 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.392 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.392 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.392 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.392 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.392 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:19:17.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.392 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:17.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:17.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:19:17.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:19:17.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:19:17.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:19:17.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:19:17.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:19:17.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:19:17.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:19:17.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:19:17.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:19:17.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:19:17.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:19:17.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:19:17.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:19:17.395 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.395 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.395 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.395 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.396 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.396 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:17.397 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:19:17.398 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:17.392+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:17.498 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:17.492+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:17.580 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:17.580 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:17.580 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:19:17.580 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:17.598 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:19:17.599 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:17.595+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:17.700 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:17.699+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:17.718 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:17.719 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:17.719 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:17.719 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:17.719 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:17.733 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:19:17.801 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:17.800+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:17.901 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:17.901 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:17.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:17.901 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:17.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.901 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:491 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.901 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:19:17.901 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.699415602000002, 2024-09-17T21:19:17.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041839013000000001, 2024-09-17T21:19:17.902 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.902 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.902 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.902 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.902 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.902 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.902 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 491 2024-09-17T21:19:17.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.902 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:19:17.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:19:17.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198315+0000", 2024-09-17T21:19:17.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6599999999999996e-07 2024-09-17T21:19:17.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198317+0000", 2024-09-17T21:19:17.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6639999999999998e-06 2024-09-17T21:19:17.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198327+0000", 2024-09-17T21:19:17.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5629999999999998e-06 2024-09-17T21:19:17.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198337+0000", 2024-09-17T21:19:17.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0667e-05 2024-09-17T21:19:17.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239793+0000", 2024-09-17T21:19:17.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041455519000000003 2024-09-17T21:19:17.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239929+0000", 2024-09-17T21:19:17.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135716 2024-09-17T21:19:17.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240153+0000", 2024-09-17T21:19:17.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022421799999999999 2024-09-17T21:19:17.905 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.905 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.906 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.906 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:492 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.906 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:19:17.906 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.699377232000003, 2024-09-17T21:19:17.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042947165000000002, 2024-09-17T21:19:17.906 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.906 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.906 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.906 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.906 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.906 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.907 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 492 2024-09-17T21:19:17.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.907 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:19:17.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:19:17.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198354+0000", 2024-09-17T21:19:17.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:19:17.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198357+0000", 2024-09-17T21:19:17.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6969999999999999e-06 2024-09-17T21:19:17.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198367+0000", 2024-09-17T21:19:17.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8430000000000005e-06 2024-09-17T21:19:17.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198374+0000", 2024-09-17T21:19:17.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9600000000000003e-06 2024-09-17T21:19:17.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240226+0000", 2024-09-17T21:19:17.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041852157000000001 2024-09-17T21:19:17.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240409+0000", 2024-09-17T21:19:17.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018244800000000001 2024-09-17T21:19:17.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241299+0000", 2024-09-17T21:19:17.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000890554 2024-09-17T21:19:17.910 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.910 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.910 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.910 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:493 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 4063232~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.910 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:19:17.910 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.699340145999997, 2024-09-17T21:19:17.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043897209, 2024-09-17T21:19:17.911 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.911 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.911 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.911 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.911 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.911 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.911 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 493 2024-09-17T21:19:17.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.911 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:19:17.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:19:17.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198391+0000", 2024-09-17T21:19:17.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4699999999999999e-06 2024-09-17T21:19:17.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198396+0000", 2024-09-17T21:19:17.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:19:17.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198412+0000", 2024-09-17T21:19:17.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6507e-05 2024-09-17T21:19:17.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198425+0000", 2024-09-17T21:19:17.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.256e-05 2024-09-17T21:19:17.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241383+0000", 2024-09-17T21:19:17.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042958216 2024-09-17T21:19:17.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241525+0000", 2024-09-17T21:19:17.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014204500000000001 2024-09-17T21:19:17.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242287+0000", 2024-09-17T21:19:17.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076125800000000001 2024-09-17T21:19:17.915 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.915 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.915 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.915 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:496 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.915 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:19:17.915 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.699249747000003, 2024-09-17T21:19:17.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043732500000000001, 2024-09-17T21:19:17.915 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.915 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.915 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.915 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.916 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.916 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.916 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 496 2024-09-17T21:19:17.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.916 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:19:17.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:19:17.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198481+0000", 2024-09-17T21:19:17.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-17T21:19:17.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198488+0000", 2024-09-17T21:19:17.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8499999999999996e-06 2024-09-17T21:19:17.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198507+0000", 2024-09-17T21:19:17.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9276999999999999e-05 2024-09-17T21:19:17.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198521+0000", 2024-09-17T21:19:17.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3376e-05 2024-09-17T21:19:17.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241339+0000", 2024-09-17T21:19:17.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042818119000000002 2024-09-17T21:19:17.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241449+0000", 2024-09-17T21:19:17.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110209 2024-09-17T21:19:17.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242212+0000", 2024-09-17T21:19:17.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076329900000000005 2024-09-17T21:19:17.919 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.919 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.919 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.920 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:497 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.920 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:19:17.920 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.699168798000002, 2024-09-17T21:19:17.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044584278999999997, 2024-09-17T21:19:17.920 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.920 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.920 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.920 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.920 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.920 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.920 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 497 2024-09-17T21:19:17.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.920 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:19:17.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:19:17.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198562+0000", 2024-09-17T21:19:17.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-17T21:19:17.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198569+0000", 2024-09-17T21:19:17.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.494e-06 2024-09-17T21:19:17.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198588+0000", 2024-09-17T21:19:17.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8955999999999999e-05 2024-09-17T21:19:17.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198621+0000", 2024-09-17T21:19:17.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3837000000000001e-05 2024-09-17T21:19:17.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242258+0000", 2024-09-17T21:19:17.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043636900999999999 2024-09-17T21:19:17.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242343+0000", 2024-09-17T21:19:17.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5048000000000003e-05 2024-09-17T21:19:17.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243145+0000", 2024-09-17T21:19:17.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00080167300000000003 2024-09-17T21:19:17.924 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.924 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.924 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.924 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:503 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.924 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:19:17.924 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.699084274000001, 2024-09-17T21:19:17.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044220582000000001, 2024-09-17T21:19:17.924 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.924 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.924 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.924 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.925 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.925 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.925 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 503 2024-09-17T21:19:17.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.925 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:19:17.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:19:17.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198647+0000", 2024-09-17T21:19:17.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.483e-06 2024-09-17T21:19:17.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198653+0000", 2024-09-17T21:19:17.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4529999999999999e-06 2024-09-17T21:19:17.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198672+0000", 2024-09-17T21:19:17.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9193000000000001e-05 2024-09-17T21:19:17.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198685+0000", 2024-09-17T21:19:17.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.291e-05 2024-09-17T21:19:17.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:19:17.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043672141999999997 2024-09-17T21:19:17.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242716+0000", 2024-09-17T21:19:17.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00035901800000000001 2024-09-17T21:19:17.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242866+0000", 2024-09-17T21:19:17.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000149383 2024-09-17T21:19:17.928 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.928 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.928 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.929 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:504 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.929 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:19:17.929 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.699024481999999, 2024-09-17T21:19:17.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044611406999999999, 2024-09-17T21:19:17.929 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.929 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.929 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.929 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.929 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.929 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.929 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 504 2024-09-17T21:19:17.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.929 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:19:17.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:19:17.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198706+0000", 2024-09-17T21:19:17.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:19:17.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198712+0000", 2024-09-17T21:19:17.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5770000000000001e-06 2024-09-17T21:19:17.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198729+0000", 2024-09-17T21:19:17.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6673000000000001e-05 2024-09-17T21:19:17.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198742+0000", 2024-09-17T21:19:17.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3543000000000001e-05 2024-09-17T21:19:17.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242981+0000", 2024-09-17T21:19:17.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044238953999999997 2024-09-17T21:19:17.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243155+0000", 2024-09-17T21:19:17.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017401600000000001 2024-09-17T21:19:17.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243316+0000", 2024-09-17T21:19:17.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016137399999999999 2024-09-17T21:19:17.933 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.933 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.933 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.933 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:521 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.933 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:19:17.933 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.698477343, 2024-09-17T21:19:17.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045043685999999999, 2024-09-17T21:19:17.933 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.933 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.934 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.934 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.934 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.934 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.934 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 521 2024-09-17T21:19:17.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.934 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:19:17.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:19:17.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199253+0000", 2024-09-17T21:19:17.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1400000000000001e-06 2024-09-17T21:19:17.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199258+0000", 2024-09-17T21:19:17.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1800000000000004e-06 2024-09-17T21:19:17.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199276+0000", 2024-09-17T21:19:17.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7686999999999999e-05 2024-09-17T21:19:17.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199289+0000", 2024-09-17T21:19:17.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3263e-05 2024-09-17T21:19:17.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243397+0000", 2024-09-17T21:19:17.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044107636999999998 2024-09-17T21:19:17.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243527+0000", 2024-09-17T21:19:17.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013038399999999999 2024-09-17T21:19:17.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244296+0000", 2024-09-17T21:19:17.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076839499999999995 2024-09-17T21:19:17.938 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.938 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.938 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.938 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:522 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.938 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:19:17.938 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.698422039999997, 2024-09-17T21:19:17.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046049870999999999, 2024-09-17T21:19:17.938 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.938 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.938 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.938 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.939 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.939 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.939 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 522 2024-09-17T21:19:17.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.939 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:19:17.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:19:17.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199309+0000", 2024-09-17T21:19:17.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:19:17.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199315+0000", 2024-09-17T21:19:17.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2440000000000001e-06 2024-09-17T21:19:17.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199333+0000", 2024-09-17T21:19:17.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7886000000000001e-05 2024-09-17T21:19:17.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199379+0000", 2024-09-17T21:19:17.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6383e-05 2024-09-17T21:19:17.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244394+0000", 2024-09-17T21:19:17.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045014851000000002 2024-09-17T21:19:17.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244491+0000", 2024-09-17T21:19:17.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7245999999999997e-05 2024-09-17T21:19:17.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245357+0000", 2024-09-17T21:19:17.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00086626800000000001 2024-09-17T21:19:17.942 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.942 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.942 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.942 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:527 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.942 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:19:17.943 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.698325373999999, 2024-09-17T21:19:17.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045812522000000001, 2024-09-17T21:19:17.943 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.943 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.943 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.943 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.943 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.943 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.943 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 527 2024-09-17T21:19:17.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.943 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:19:17.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:19:17.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199406+0000", 2024-09-17T21:19:17.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.367e-06 2024-09-17T21:19:17.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199412+0000", 2024-09-17T21:19:17.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:19:17.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199429+0000", 2024-09-17T21:19:17.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7396000000000001e-05 2024-09-17T21:19:17.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199442+0000", 2024-09-17T21:19:17.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3327e-05 2024-09-17T21:19:17.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244361+0000", 2024-09-17T21:19:17.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044918197 2024-09-17T21:19:17.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244482+0000", 2024-09-17T21:19:17.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121945 2024-09-17T21:19:17.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245217+0000", 2024-09-17T21:19:17.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00073415999999999998 2024-09-17T21:19:17.947 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.947 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.947 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.947 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:534 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.947 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:19:17.947 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.698266668000002, 2024-09-17T21:19:17.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046292505999999997, 2024-09-17T21:19:17.947 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.947 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.947 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.948 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.948 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.948 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.948 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:19:17.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.948 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:19:17.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:19:17.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199464+0000", 2024-09-17T21:19:17.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:19:17.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199473+0000", 2024-09-17T21:19:17.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5629999999999992e-06 2024-09-17T21:19:17.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199492+0000", 2024-09-17T21:19:17.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9273999999999999e-05 2024-09-17T21:19:17.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199505+0000", 2024-09-17T21:19:17.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3443e-05 2024-09-17T21:19:17.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245357+0000", 2024-09-17T21:19:17.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045851714000000002 2024-09-17T21:19:17.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245550+0000", 2024-09-17T21:19:17.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019301600000000001 2024-09-17T21:19:17.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245755+0000", 2024-09-17T21:19:17.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205226 2024-09-17T21:19:17.951 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.952 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.952 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.952 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:535 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.952 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:19:17.952 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.698162111999999, 2024-09-17T21:19:17.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046637492000000003, 2024-09-17T21:19:17.952 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.952 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.952 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.952 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.953 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.953 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.953 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 535 2024-09-17T21:19:17.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.953 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:19:17.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:19:17.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199568+0000", 2024-09-17T21:19:17.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.037e-06 2024-09-17T21:19:17.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199572+0000", 2024-09-17T21:19:17.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5729999999999999e-06 2024-09-17T21:19:17.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199585+0000", 2024-09-17T21:19:17.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.279e-05 2024-09-17T21:19:17.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199592+0000", 2024-09-17T21:19:17.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.61e-06 2024-09-17T21:19:17.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245873+0000", 2024-09-17T21:19:17.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046280494999999998 2024-09-17T21:19:17.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246018+0000", 2024-09-17T21:19:17.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000144697 2024-09-17T21:19:17.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246205+0000", 2024-09-17T21:19:17.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018729 2024-09-17T21:19:17.956 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.956 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.956 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.957 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:543 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.957 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:17.957 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.698066865999998, 2024-09-17T21:19:17.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047564097999999999, 2024-09-17T21:19:17.957 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.957 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.957 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.957 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.957 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.957 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.957 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 543 2024-09-17T21:19:17.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.957 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:17.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:17.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:17.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9699999999999995e-07 2024-09-17T21:19:17.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199677+0000", 2024-09-17T21:19:17.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:19:17.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199690+0000", 2024-09-17T21:19:17.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2977e-05 2024-09-17T21:19:17.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199705+0000", 2024-09-17T21:19:17.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4656e-05 2024-09-17T21:19:17.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246307+0000", 2024-09-17T21:19:17.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046601780000000002 2024-09-17T21:19:17.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246475+0000", 2024-09-17T21:19:17.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016834200000000001 2024-09-17T21:19:17.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247227+0000", 2024-09-17T21:19:17.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00075173299999999998 2024-09-17T21:19:17.961 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.961 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.961 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.961 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:544 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.961 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:17.961 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.698008457, 2024-09-17T21:19:17.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049283559999999997, 2024-09-17T21:19:17.962 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.962 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.962 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.962 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.962 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.962 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.962 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 544 2024-09-17T21:19:17.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.962 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:17.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:17.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199722+0000", 2024-09-17T21:19:17.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.08e-06 2024-09-17T21:19:17.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199738+0000", 2024-09-17T21:19:17.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5943999999999998e-05 2024-09-17T21:19:17.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199748+0000", 2024-09-17T21:19:17.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0193e-05 2024-09-17T21:19:17.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199756+0000", 2024-09-17T21:19:17.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4200000000000001e-06 2024-09-17T21:19:17.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247311+0000", 2024-09-17T21:19:17.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047555300000000002 2024-09-17T21:19:17.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247571+0000", 2024-09-17T21:19:17.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025996999999999999 2024-09-17T21:19:17.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249005+0000", 2024-09-17T21:19:17.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001433653 2024-09-17T21:19:17.966 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.966 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.966 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.966 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:549 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.966 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:17.966 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.697962832999998, 2024-09-17T21:19:17.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048464528999999999, 2024-09-17T21:19:17.966 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.966 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.966 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.967 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.967 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.967 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.967 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 549 2024-09-17T21:19:17.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.967 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:17.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:17.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:17.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:19:17.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199770+0000", 2024-09-17T21:19:17.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9969999999999999e-06 2024-09-17T21:19:17.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199780+0000", 2024-09-17T21:19:17.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0003000000000001e-05 2024-09-17T21:19:17.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199793+0000", 2024-09-17T21:19:17.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2074e-05 2024-09-17T21:19:17.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247289+0000", 2024-09-17T21:19:17.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047496691000000001 2024-09-17T21:19:17.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247437+0000", 2024-09-17T21:19:17.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014807899999999999 2024-09-17T21:19:17.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248231+0000", 2024-09-17T21:19:17.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00079390899999999998 2024-09-17T21:19:17.970 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.970 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.970 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.971 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:558 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.971 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:17.971 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.697922320000004, 2024-09-17T21:19:17.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049709821000000001, 2024-09-17T21:19:17.971 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.971 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.971 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.971 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.971 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.971 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.971 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 558 2024-09-17T21:19:17.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.972 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:17.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:17.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199809+0000", 2024-09-17T21:19:17.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.333e-06 2024-09-17T21:19:17.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199814+0000", 2024-09-17T21:19:17.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0930000000000002e-06 2024-09-17T21:19:17.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199831+0000", 2024-09-17T21:19:17.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7580000000000001e-05 2024-09-17T21:19:17.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199860+0000", 2024-09-17T21:19:17.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8609999999999999e-05 2024-09-17T21:19:17.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249106+0000", 2024-09-17T21:19:17.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049246545000000003 2024-09-17T21:19:17.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249274+0000", 2024-09-17T21:19:17.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016806699999999999 2024-09-17T21:19:17.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249517+0000", 2024-09-17T21:19:17.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024259300000000001 2024-09-17T21:19:17.975 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.975 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.975 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.975 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:559 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.975 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:17.975 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.697802957999997, 2024-09-17T21:19:17.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049990694000000002, 2024-09-17T21:19:17.976 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.976 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.976 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.976 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.976 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.976 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.976 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 559 2024-09-17T21:19:17.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.976 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:17.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:17.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199928+0000", 2024-09-17T21:19:17.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:19:17.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199935+0000", 2024-09-17T21:19:17.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5429999999999999e-06 2024-09-17T21:19:17.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199954+0000", 2024-09-17T21:19:17.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.912e-05 2024-09-17T21:19:17.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199969+0000", 2024-09-17T21:19:17.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5305999999999999e-05 2024-09-17T21:19:17.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249604+0000", 2024-09-17T21:19:17.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049634931 2024-09-17T21:19:17.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249738+0000", 2024-09-17T21:19:17.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013411599999999999 2024-09-17T21:19:17.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249917+0000", 2024-09-17T21:19:17.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179111 2024-09-17T21:19:17.979 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.979 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.979 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.980 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:560 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:17.980 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:17.980 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.697738542000003, 2024-09-17T21:19:17.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050173059999999998, 2024-09-17T21:19:17.980 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.980 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.980 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.980 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.980 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:17.980 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:17.980 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 560 2024-09-17T21:19:17.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.981 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:17.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:17.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199992+0000", 2024-09-17T21:19:17.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.517e-06 2024-09-17T21:19:17.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199998+0000", 2024-09-17T21:19:17.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1460000000000004e-06 2024-09-17T21:19:17.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:19:17.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7407000000000001e-05 2024-09-17T21:19:17.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200029+0000", 2024-09-17T21:19:17.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3499999999999999e-05 2024-09-17T21:19:17.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249968+0000", 2024-09-17T21:19:17.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049939799 2024-09-17T21:19:17.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250055+0000", 2024-09-17T21:19:17.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6731999999999996e-05 2024-09-17T21:19:17.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250164+0000", 2024-09-17T21:19:17.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108959 2024-09-17T21:19:17.984 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.984 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.984 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.984 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:19:17.984 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:17.984 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 38.087948832999999, 2024-09-17T21:19:17.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:19:17.984 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.984 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.984 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:17.985 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.985 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:19:17.985 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:19:17.985 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:19:17.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.985 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:17.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:17.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:19:17.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:19:17.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:19:17.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:19:17.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:19:17.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:19:17.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:19:17.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:19:17.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:19:17.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:19:17.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:17.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:19:17.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:19:17.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:19:17.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:19:17.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:19:17.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:19:17.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:19:17.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:19:17.989 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.989 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.989 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.989 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:19:17.989 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:17.989 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 24.532847047000001, 2024-09-17T21:19:17.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:19:17.989 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:17.989 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:17.990 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:17.990 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:17.990 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:19:17.990 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:17.990 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:19:17.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.990 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:17.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:17.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:17.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:17.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:17.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:17.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:17.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:19:17.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:19:17.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:17.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:19:17.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:19:17.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:17.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:19:17.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:19:17.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:17.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:19:17.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:19:17.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:19:17.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:19:17.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:17.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:19:17.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:19:17.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:17.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:19:17.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:19:17.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:17.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:19:17.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:19:17.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.994 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:17.994 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:19:17.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:19:17.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.994 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:17.994 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:19:17.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:19:17.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.994 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:17.994 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:19:17.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:19:17.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:17.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:19:17.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:19:17.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:17.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:17.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:17.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:19:17.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:19:17.995 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.995 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.995 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.995 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:17.995 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:17.996 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:17.996 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:19:17.997 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:17.992+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:17.998 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:19:18.098 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:18.096+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:18.163 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:18.164 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:18.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:18.164 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T21:19:18.164 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:17.280167+0000", 2024-09-17T21:19:18.164 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.88270096200000003, 2024-09-17T21:19:18.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88272800699999998, 2024-09-17T21:19:18.164 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:18.164 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:18.164 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for peered", 2024-09-17T21:19:18.164 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:18.164 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:19:18.165 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:19:18.165 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:19:18.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:18.165 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:18.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:18.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:18.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.280167+0000", 2024-09-17T21:19:18.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:18.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:18.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:18.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:18.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.280167+0000", 2024-09-17T21:19:18.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:18.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:18.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:18.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:18.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.280169+0000", 2024-09-17T21:19:18.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.815e-06 2024-09-17T21:19:18.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:18.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:18.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:18.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.280176+0000", 2024-09-17T21:19:18.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0899999999999999e-06 2024-09-17T21:19:18.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:18.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:18.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:18.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.280191+0000", 2024-09-17T21:19:18.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4885999999999999e-05 2024-09-17T21:19:18.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:18.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:18.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:18.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.280202+0000", 2024-09-17T21:19:18.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1239e-05 2024-09-17T21:19:18.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:18.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:18.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:18.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.280278+0000", 2024-09-17T21:19:18.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6359000000000005e-05 2024-09-17T21:19:18.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:18.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:18.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-17T21:19:18.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.280286+0000", 2024-09-17T21:19:18.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6890000000000004e-06 2024-09-17T21:19:18.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:18.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:18.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:18.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.282549+0000", 2024-09-17T21:19:18.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022627670000000002 2024-09-17T21:19:18.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:18.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:18.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:18.169 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.282558+0000", 2024-09-17T21:19:18.169 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6010000000000001e-06 2024-09-17T21:19:18.169 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:18.169 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:18.169 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:18.169 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:18.169 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:19:18.169 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 1 2024-09-17T21:19:18.169 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:18.179 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:19:18.183 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:18.184 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:18.198 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:18.196+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:18.201 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:18.201 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:18.219 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:18.219 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:18.240 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:18.240 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:18.261 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:18.261 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:18.289 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:18.289 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:18.299 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:18.296+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:18.306 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:18.306 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:18.324 INFO:teuthology.orchestra.run.smithi184.stdout:2853 write 0xd4a5200 thru 0xd4a662d (0x142e bytes) 2024-09-17T21:19:18.325 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:19:18.327 INFO:teuthology.orchestra.run.smithi184.stdout:2855 write 0x6653400 thru 0x6662425 (0xf026 bytes) 2024-09-17T21:19:18.328 INFO:teuthology.orchestra.run.smithi184.stdout:2856 read 0xa92000 thru 0xa9ceec (0xaeed bytes) 2024-09-17T21:19:18.328 INFO:teuthology.orchestra.run.smithi184.stdout:2858 punch from 0x1ad6200 to 0x1ad9e00, (0x3c00 bytes) 2024-09-17T21:19:18.335 INFO:teuthology.orchestra.run.smithi184.stdout:2861 trunc from 0xd4a6800 to 0xa58da00 2024-09-17T21:19:18.373 INFO:teuthology.orchestra.run.smithi184.stdout:2862 write 0x4091600 thru 0x4095d62 (0x4763 bytes) 2024-09-17T21:19:18.376 INFO:teuthology.orchestra.run.smithi184.stdout:2864 read 0x1a8ec00 thru 0x1a9a881 (0xbc82 bytes) 2024-09-17T21:19:18.377 INFO:teuthology.orchestra.run.smithi184.stdout:2870 read 0x7ac5400 thru 0x7acd1d2 (0x7dd3 bytes) 2024-09-17T21:19:18.399 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:18.396+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:18.415 INFO:teuthology.orchestra.run.smithi184.stdout:2872 write 0xd5e2600 thru 0xd5e6d58 (0x4759 bytes) 2024-09-17T21:19:18.418 INFO:teuthology.orchestra.run.smithi184.stdout:2873 write 0xd217c00 thru 0xd22728e (0xf68f bytes) 2024-09-17T21:19:18.418 INFO:teuthology.orchestra.run.smithi184.stdout:2876 punch from 0x573600 to 0x57d200, (0x9c00 bytes) 2024-09-17T21:19:18.419 INFO:teuthology.orchestra.run.smithi184.stdout:2877 punch from 0x1a60a00 to 0x1a62a00, (0x2000 bytes) 2024-09-17T21:19:18.419 INFO:teuthology.orchestra.run.smithi184.stdout:2878 write 0x39a8e00 thru 0x39b537c (0xc57d bytes) 2024-09-17T21:19:18.421 INFO:teuthology.orchestra.run.smithi184.stdout:2879 punch from 0x2a53600 to 0x2a60e00, (0xd800 bytes) 2024-09-17T21:19:18.421 INFO:teuthology.orchestra.run.smithi184.stdout:2880 write 0x61e7000 thru 0x61f5290 (0xe291 bytes) 2024-09-17T21:19:18.421 INFO:teuthology.orchestra.run.smithi184.stdout:2885 read 0xbe6e600 thru 0xbe762fc (0x7cfd bytes) 2024-09-17T21:19:18.421 INFO:teuthology.orchestra.run.smithi184.stdout:2886 read 0x36be000 thru 0x36cd67c (0xf67d bytes) 2024-09-17T21:19:18.424 INFO:teuthology.orchestra.run.smithi184.stdout:2887 trunc from 0xd5e6e00 to 0xed77800 2024-09-17T21:19:18.455 INFO:teuthology.orchestra.run.smithi184.stdout:2888 read 0x2180a00 thru 0x218b506 (0xab07 bytes) 2024-09-17T21:19:18.462 INFO:teuthology.orchestra.run.smithi184.stdout:2892 trunc from 0xed77800 to 0xd1c3800 2024-09-17T21:19:18.492 INFO:teuthology.orchestra.run.smithi184.stdout:2893 read 0x7d3be00 thru 0x7d481a8 (0xc3a9 bytes) 2024-09-17T21:19:18.495 INFO:teuthology.orchestra.run.smithi184.stdout:2895 read 0x7ddd400 thru 0x7ddf117 (0x1d18 bytes) 2024-09-17T21:19:18.497 INFO:teuthology.orchestra.run.smithi184.stdout:2896 trunc from 0xd1c3800 to 0xbc43c00 2024-09-17T21:19:18.498 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:18.498 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:18.500 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:18.495+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:18.515 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:19:18.520 INFO:teuthology.orchestra.run.smithi184.stdout:2897 punch from 0x94c3e00 to 0x94d1000, (0xd200 bytes) 2024-09-17T21:19:18.522 INFO:teuthology.orchestra.run.smithi184.stdout:2901 write 0xac02800 thru 0xac055a5 (0x2da6 bytes) 2024-09-17T21:19:18.525 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:19:18.547 INFO:teuthology.orchestra.run.smithi184.stdout:2903 trunc from 0xbc43c00 to 0x27d1600 2024-09-17T21:19:18.601 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:18.599+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:18.606 INFO:teuthology.orchestra.run.smithi184.stdout:2904 write 0x2779200 thru 0x277aa4c (0x184d bytes) 2024-09-17T21:19:18.611 INFO:teuthology.orchestra.run.smithi184.stdout:2910 trunc from 0x27d1600 to 0x3b95800 2024-09-17T21:19:18.661 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:18.662 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:18.662 INFO:teuthology.orchestra.run.smithi184.stdout:2912 write 0x7e55400 thru 0x7e573ce (0x1fcf bytes) 2024-09-17T21:19:18.665 INFO:teuthology.orchestra.run.smithi184.stdout:2914 write 0x53ea200 thru 0x53ee12d (0x3f2e bytes) 2024-09-17T21:19:18.665 INFO:teuthology.orchestra.run.smithi184.stdout:2915 write 0x6c0ca00 thru 0x6c19411 (0xca12 bytes) 2024-09-17T21:19:18.677 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:19:18.695 INFO:teuthology.orchestra.run.smithi184.stdout:2916 write 0x957de00 thru 0x9584c38 (0x6e39 bytes) 2024-09-17T21:19:18.710 INFO:teuthology.orchestra.run.smithi184.stdout:2920 trunc from 0x9584e00 to 0x1642a00 2024-09-17T21:19:18.722 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:18.720+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:18.740 INFO:teuthology.orchestra.run.smithi184.stdout:2921 punch from 0x99aa00 to 0x99b000, (0x600 bytes) 2024-09-17T21:19:18.802 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:18.796+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:18.809 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:18.809 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:18.824 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:19:18.902 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:18.899+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:18.976 INFO:teuthology.orchestra.run.smithi184.stdout:2923 write 0x362c600 thru 0x363ae85 (0xe886 bytes) 2024-09-17T21:19:18.978 INFO:teuthology.orchestra.run.smithi184.stdout:2924 read 0x2cc6800 thru 0x2cd1264 (0xaa65 bytes) 2024-09-17T21:19:18.978 INFO:teuthology.orchestra.run.smithi184.stdout:2930 punch from 0xaa1800 to 0xaabe00, (0xa600 bytes) 2024-09-17T21:19:18.993 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:18.993 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:19.004 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:18.999+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:19.010 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:19:19.014 INFO:teuthology.orchestra.run.smithi184.stdout:2931 write 0xa197c00 thru 0xa1995a3 (0x19a4 bytes) 2024-09-17T21:19:19.015 INFO:teuthology.orchestra.run.smithi184.stdout:2935 read 0x4c06e00 thru 0x4c129b1 (0xbbb2 bytes) 2024-09-17T21:19:19.015 INFO:teuthology.orchestra.run.smithi184.stdout:2940 read 0x49a00 thru 0x4c2e9 (0x28ea bytes) 2024-09-17T21:19:19.040 INFO:teuthology.orchestra.run.smithi184.stdout:2941 write 0xa190e00 thru 0xa19f067 (0xe268 bytes) 2024-09-17T21:19:19.045 INFO:teuthology.orchestra.run.smithi184.stdout:2942 write 0xc3a200 thru 0xc3e5a8 (0x43a9 bytes) 2024-09-17T21:19:19.045 INFO:teuthology.orchestra.run.smithi184.stdout:2943 read 0x22b6c00 thru 0x22bf542 (0x8943 bytes) 2024-09-17T21:19:19.045 INFO:teuthology.orchestra.run.smithi184.stdout:2944 write 0x6010600 thru 0x6010fe2 (0x9e3 bytes) 2024-09-17T21:19:19.057 INFO:teuthology.orchestra.run.smithi184.stdout:2945 trunc from 0xa19f200 to 0x46abe00 2024-09-17T21:19:19.095 INFO:teuthology.orchestra.run.smithi184.stdout:2946 punch from 0x1961600 to 0x1964600, (0x3000 bytes) 2024-09-17T21:19:19.104 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:19.100+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:19.122 INFO:teuthology.orchestra.run.smithi184.stdout:2948 write 0x7325800 thru 0x732d014 (0x7815 bytes) 2024-09-17T21:19:19.124 INFO:teuthology.orchestra.run.smithi184.stdout:2949 read 0x397dc00 thru 0x397ff4a (0x234b bytes) 2024-09-17T21:19:19.135 INFO:teuthology.orchestra.run.smithi184.stdout:2952 trunc from 0x732d200 to 0x26d1000 2024-09-17T21:19:19.143 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:19.143 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:19.159 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:19:19.181 INFO:teuthology.orchestra.run.smithi184.stdout:2954 trunc from 0x26d1000 to 0x4ebf600 2024-09-17T21:19:19.206 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:19.203+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:19.227 INFO:teuthology.orchestra.run.smithi184.stdout:2956 write 0x9356000 thru 0x936390f (0xd910 bytes) 2024-09-17T21:19:19.229 INFO:teuthology.orchestra.run.smithi184.stdout:2957 write 0x8839800 thru 0x883e161 (0x4962 bytes) 2024-09-17T21:19:19.229 INFO:teuthology.orchestra.run.smithi184.stdout:2958 punch from 0x2984e00 to 0x2993c00, (0xee00 bytes) 2024-09-17T21:19:19.229 INFO:teuthology.orchestra.run.smithi184.stdout:2960 read 0x38a400 thru 0x3979bb (0xd5bc bytes) 2024-09-17T21:19:19.230 INFO:teuthology.orchestra.run.smithi184.stdout:2962 read 0x298de00 thru 0x299a962 (0xcb63 bytes) 2024-09-17T21:19:19.250 INFO:teuthology.orchestra.run.smithi184.stdout:2963 trunc from 0x9363a00 to 0xb37fc00 2024-09-17T21:19:19.268 INFO:teuthology.orchestra.run.smithi184.stdout:2966 write 0x8735a00 thru 0x873f76d (0x9d6e bytes) 2024-09-17T21:19:19.268 INFO:teuthology.orchestra.run.smithi184.stdout:2970 read 0x3d14800 thru 0x3d1ac48 (0x6449 bytes) 2024-09-17T21:19:19.277 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-17T21:19:19.307 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:19.303+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:19.343 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:19.344 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:19.357 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:19:19.408 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:19.403+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:19.479 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:19.479 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:19.495 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:19:19.509 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:19.507+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:19.610 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:19.607+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:19.615 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:19.616 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:19.630 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:19:19.711 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:19.707+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:19.751 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:19.751 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:19.766 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:19:19.812 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:19.807+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:19.890 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:19.890 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:19.905 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:19:19.912 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:19.908+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:20.013 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:20.011+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:20.038 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:20.038 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:20.052 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:19:20.114 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:20.112+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:20.215 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:20.211+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:20.220 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:20.220 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:20.234 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:19:20.314 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:19:20.316 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:20.312+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:20.326 INFO:teuthology.orchestra.run.smithi184.stdout:2972 write 0xd84ee00 thru 0xd853de8 (0x4fe9 bytes) 2024-09-17T21:19:20.328 INFO:teuthology.orchestra.run.smithi184.stdout:2976 punch from 0x6e93c00 to 0x6ea0200, (0xc600 bytes) 2024-09-17T21:19:20.328 INFO:teuthology.orchestra.run.smithi184.stdout:2978 write 0x60dd400 thru 0x60e0ce4 (0x38e5 bytes) 2024-09-17T21:19:20.329 INFO:teuthology.orchestra.run.smithi184.stdout:2979 punch from 0x4b8e800 to 0x4b95600, (0x6e00 bytes) 2024-09-17T21:19:20.329 INFO:teuthology.orchestra.run.smithi184.stdout:2982 punch from 0x1664000 to 0x1669000, (0x5000 bytes) 2024-09-17T21:19:20.330 INFO:teuthology.orchestra.run.smithi184.stdout:2983 punch from 0xaa5dc00 to 0xaa62e00, (0x5200 bytes) 2024-09-17T21:19:20.330 INFO:teuthology.orchestra.run.smithi184.stdout:2984 read 0x49d9400 thru 0x49de355 (0x4f56 bytes) 2024-09-17T21:19:20.330 INFO:teuthology.orchestra.run.smithi184.stdout:2985 punch from 0xa516000 to 0xa51fa00, (0x9a00 bytes) 2024-09-17T21:19:20.330 INFO:teuthology.orchestra.run.smithi184.stdout:2986 write 0x3872a00 thru 0x387941b (0x6a1c bytes) 2024-09-17T21:19:20.330 INFO:teuthology.orchestra.run.smithi184.stdout:2987 write 0xbf73800 thru 0xbf824ac (0xecad bytes) 2024-09-17T21:19:20.331 INFO:teuthology.orchestra.run.smithi184.stdout:2989 punch from 0x70e5200 to 0x70f2600, (0xd400 bytes) 2024-09-17T21:19:20.331 INFO:teuthology.orchestra.run.smithi184.stdout:2990 punch from 0x4846c00 to 0x484a800, (0x3c00 bytes) 2024-09-17T21:19:20.331 INFO:teuthology.orchestra.run.smithi184.stdout:2993 write 0x306c400 thru 0x3071a4f (0x5650 bytes) 2024-09-17T21:19:20.331 INFO:teuthology.orchestra.run.smithi184.stdout:2995 punch from 0x6af0e00 to 0x6af6a00, (0x5c00 bytes) 2024-09-17T21:19:20.332 INFO:teuthology.orchestra.run.smithi184.stdout:3002 read 0x5245c00 thru 0x5248ce1 (0x30e2 bytes) 2024-09-17T21:19:20.332 INFO:teuthology.orchestra.run.smithi184.stdout:3004 write 0x3467e00 thru 0x346c4d9 (0x46da bytes) 2024-09-17T21:19:20.346 INFO:teuthology.orchestra.run.smithi184.stdout:3005 trunc from 0xd853e00 to 0x528e200 2024-09-17T21:19:20.413 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:20.413 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:20.416 INFO:teuthology.orchestra.run.smithi184.stdout:3007 trunc from 0x528e200 to 0x1aa00 2024-09-17T21:19:20.417 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:20.415+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:20.430 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:19:20.474 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:20.484 INFO:teuthology.orchestra.run.smithi184.stdout:3010 read 0xc200 thru 0x16cee (0xaaef bytes) 2024-09-17T21:19:20.494 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:20.495 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:20.506 INFO:teuthology.orchestra.run.smithi184.stdout:3011 trunc from 0x1aa00 to 0x6a7ae00 2024-09-17T21:19:20.515 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:20.515 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:20.518 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:20.512+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:20.536 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:20.536 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:20.551 INFO:teuthology.orchestra.run.smithi184.stdout:3014 write 0x6ad1a00 thru 0x6ad8865 (0x6e66 bytes) 2024-09-17T21:19:20.557 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:20.557 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:20.557 INFO:teuthology.orchestra.run.smithi184.stdout:3016 trunc from 0x6ad8a00 to 0x517fe00 2024-09-17T21:19:20.580 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:20.580 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:20.594 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:20.594 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:20.595 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:19:20.595 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:20.609 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:19:20.619 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:20.616+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:20.621 INFO:teuthology.orchestra.run.smithi184.stdout:3018 write 0x8812a00 thru 0x8820dc5 (0xe3c6 bytes) 2024-09-17T21:19:20.634 INFO:teuthology.orchestra.run.smithi184.stdout:3019 trunc from 0x8820e00 to 0xe550000 2024-09-17T21:19:20.660 INFO:teuthology.orchestra.run.smithi184.stdout:3020 read 0x3653000 thru 0x365dbed (0xabee bytes) 2024-09-17T21:19:20.662 INFO:teuthology.orchestra.run.smithi184.stdout:3021 read 0x8803600 thru 0x8806b6c (0x356d bytes) 2024-09-17T21:19:20.691 INFO:teuthology.orchestra.run.smithi184.stdout:3029 write 0xeb61000 thru 0xeb6673b (0x573c bytes) 2024-09-17T21:19:20.694 INFO:teuthology.orchestra.run.smithi184.stdout:3033 read 0xb759800 thru 0xb7657d0 (0xbfd1 bytes) 2024-09-17T21:19:20.694 INFO:teuthology.orchestra.run.smithi184.stdout:3035 read 0xd2e1200 thru 0xd2e95b5 (0x83b6 bytes) 2024-09-17T21:19:20.694 INFO:teuthology.orchestra.run.smithi184.stdout:3038 punch from 0xc49be00 to 0xc4a8c00, (0xce00 bytes) 2024-09-17T21:19:20.694 INFO:teuthology.orchestra.run.smithi184.stdout:3041 write 0x7826a00 thru 0x782ddfe (0x73ff bytes) 2024-09-17T21:19:20.695 INFO:teuthology.orchestra.run.smithi184.stdout:3043 read 0x4172400 thru 0x417bf62 (0x9b63 bytes) 2024-09-17T21:19:20.695 INFO:teuthology.orchestra.run.smithi184.stdout:3044 read 0x6cf5400 thru 0x6cfbb62 (0x6763 bytes) 2024-09-17T21:19:20.695 INFO:teuthology.orchestra.run.smithi184.stdout:3045 punch from 0x6d66200 to 0x6d72c00, (0xca00 bytes) 2024-09-17T21:19:20.695 INFO:teuthology.orchestra.run.smithi184.stdout:3046 write 0xbc95800 thru 0xbc98ba2 (0x33a3 bytes) 2024-09-17T21:19:20.695 INFO:teuthology.orchestra.run.smithi184.stdout:3048 read 0x84ee600 thru 0x84f65af (0x7fb0 bytes) 2024-09-17T21:19:20.719 INFO:teuthology.orchestra.run.smithi184.stdout:3050 trunc from 0xeb66800 to 0x1e7c600 2024-09-17T21:19:20.720 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:20.716+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:20.774 INFO:teuthology.orchestra.run.smithi184.stdout:3053 clone 16 order 19 su 131072 sc 15 2024-09-17T21:19:20.774 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:20.775 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:20.775 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:20.775 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:20.775 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:20.788 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:19:20.832 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:20.831+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:20.908 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:20.908 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:20.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:20.908 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:20.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.909 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:491 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:20.909 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:19:20.909 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 54.707800120999998, 2024-09-17T21:19:20.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041839013000000001, 2024-09-17T21:19:20.909 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:20.909 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:20.909 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:20.909 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:20.909 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:20.909 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:20.909 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 491 2024-09-17T21:19:20.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.910 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:20.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:20.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:19:20.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:20.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:19:20.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:20.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198315+0000", 2024-09-17T21:19:20.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6599999999999996e-07 2024-09-17T21:19:20.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:20.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198317+0000", 2024-09-17T21:19:20.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6639999999999998e-06 2024-09-17T21:19:20.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:20.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198327+0000", 2024-09-17T21:19:20.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5629999999999998e-06 2024-09-17T21:19:20.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:20.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198337+0000", 2024-09-17T21:19:20.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0667e-05 2024-09-17T21:19:20.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:20.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239793+0000", 2024-09-17T21:19:20.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041455519000000003 2024-09-17T21:19:20.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:20.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239929+0000", 2024-09-17T21:19:20.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135716 2024-09-17T21:19:20.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:20.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240153+0000", 2024-09-17T21:19:20.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022421799999999999 2024-09-17T21:19:20.913 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.913 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:20.913 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.913 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:492 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:20.913 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:19:20.914 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 54.707761751, 2024-09-17T21:19:20.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042947165000000002, 2024-09-17T21:19:20.914 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:20.914 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:20.914 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:20.914 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:20.914 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:20.914 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:20.914 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 492 2024-09-17T21:19:20.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.914 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:20.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:20.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:19:20.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:20.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:19:20.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:20.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198354+0000", 2024-09-17T21:19:20.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:19:20.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:20.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198357+0000", 2024-09-17T21:19:20.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6969999999999999e-06 2024-09-17T21:19:20.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:20.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198367+0000", 2024-09-17T21:19:20.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8430000000000005e-06 2024-09-17T21:19:20.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:20.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198374+0000", 2024-09-17T21:19:20.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9600000000000003e-06 2024-09-17T21:19:20.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:20.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240226+0000", 2024-09-17T21:19:20.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041852157000000001 2024-09-17T21:19:20.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:20.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240409+0000", 2024-09-17T21:19:20.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018244800000000001 2024-09-17T21:19:20.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:20.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241299+0000", 2024-09-17T21:19:20.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000890554 2024-09-17T21:19:20.918 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.918 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:20.918 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.918 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:493 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 4063232~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:20.918 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:19:20.918 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 54.707724665000001, 2024-09-17T21:19:20.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043897209, 2024-09-17T21:19:20.918 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:20.919 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:20.919 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:20.919 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:20.919 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:20.919 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:20.919 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 493 2024-09-17T21:19:20.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.919 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:20.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:20.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:19:20.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:20.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:19:20.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:20.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198391+0000", 2024-09-17T21:19:20.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4699999999999999e-06 2024-09-17T21:19:20.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:20.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198396+0000", 2024-09-17T21:19:20.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:19:20.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:20.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198412+0000", 2024-09-17T21:19:20.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6507e-05 2024-09-17T21:19:20.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:20.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198425+0000", 2024-09-17T21:19:20.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.256e-05 2024-09-17T21:19:20.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:20.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241383+0000", 2024-09-17T21:19:20.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042958216 2024-09-17T21:19:20.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:20.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241525+0000", 2024-09-17T21:19:20.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014204500000000001 2024-09-17T21:19:20.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:20.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242287+0000", 2024-09-17T21:19:20.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076125800000000001 2024-09-17T21:19:20.922 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.923 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:20.923 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.923 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:496 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:20.923 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:19:20.923 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 54.707634265999999, 2024-09-17T21:19:20.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043732500000000001, 2024-09-17T21:19:20.923 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:20.923 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:20.923 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:20.923 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:20.923 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:20.924 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:20.924 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 496 2024-09-17T21:19:20.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.924 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:20.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:20.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:19:20.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:20.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:19:20.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:20.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198481+0000", 2024-09-17T21:19:20.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-17T21:19:20.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:20.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198488+0000", 2024-09-17T21:19:20.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8499999999999996e-06 2024-09-17T21:19:20.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:20.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198507+0000", 2024-09-17T21:19:20.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9276999999999999e-05 2024-09-17T21:19:20.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:20.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198521+0000", 2024-09-17T21:19:20.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3376e-05 2024-09-17T21:19:20.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:20.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241339+0000", 2024-09-17T21:19:20.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042818119000000002 2024-09-17T21:19:20.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:20.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241449+0000", 2024-09-17T21:19:20.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110209 2024-09-17T21:19:20.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:20.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242212+0000", 2024-09-17T21:19:20.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076329900000000005 2024-09-17T21:19:20.927 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.927 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:20.927 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.928 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:497 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:20.928 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:19:20.928 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 54.707553316999999, 2024-09-17T21:19:20.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044584278999999997, 2024-09-17T21:19:20.928 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:20.928 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:20.928 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:20.928 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:20.928 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:20.928 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:20.929 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 497 2024-09-17T21:19:20.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.929 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:20.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:20.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:19:20.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:20.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:19:20.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:20.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198562+0000", 2024-09-17T21:19:20.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-17T21:19:20.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:20.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198569+0000", 2024-09-17T21:19:20.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.494e-06 2024-09-17T21:19:20.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:20.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198588+0000", 2024-09-17T21:19:20.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8955999999999999e-05 2024-09-17T21:19:20.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:20.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198621+0000", 2024-09-17T21:19:20.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3837000000000001e-05 2024-09-17T21:19:20.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:20.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242258+0000", 2024-09-17T21:19:20.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043636900999999999 2024-09-17T21:19:20.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:20.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242343+0000", 2024-09-17T21:19:20.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5048000000000003e-05 2024-09-17T21:19:20.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:20.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243145+0000", 2024-09-17T21:19:20.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00080167300000000003 2024-09-17T21:19:20.932 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.932 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:20.932 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.933 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:503 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:20.933 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:19:20.933 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 54.707468792999997, 2024-09-17T21:19:20.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044220582000000001, 2024-09-17T21:19:20.933 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:20.933 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:20.933 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:20.933 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:20.933 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:20.933 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:20.933 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 503 2024-09-17T21:19:20.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.933 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:20.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:20.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:19:20.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:20.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:19:20.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:20.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198647+0000", 2024-09-17T21:19:20.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.483e-06 2024-09-17T21:19:20.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:20.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198653+0000", 2024-09-17T21:19:20.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4529999999999999e-06 2024-09-17T21:19:20.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:20.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198672+0000", 2024-09-17T21:19:20.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9193000000000001e-05 2024-09-17T21:19:20.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:20.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198685+0000", 2024-09-17T21:19:20.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.291e-05 2024-09-17T21:19:20.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:20.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:19:20.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043672141999999997 2024-09-17T21:19:20.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:20.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242716+0000", 2024-09-17T21:19:20.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00035901800000000001 2024-09-17T21:19:20.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:20.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242866+0000", 2024-09-17T21:19:20.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000149383 2024-09-17T21:19:20.937 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.937 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:20.937 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.937 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:504 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:20.937 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:19:20.937 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 54.707409001000002, 2024-09-17T21:19:20.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044611406999999999, 2024-09-17T21:19:20.937 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:20.938 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:20.938 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:20.938 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:20.938 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:20.938 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:20.938 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 504 2024-09-17T21:19:20.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.938 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:20.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:20.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:19:20.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:20.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:19:20.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:20.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198706+0000", 2024-09-17T21:19:20.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:19:20.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:20.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198712+0000", 2024-09-17T21:19:20.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5770000000000001e-06 2024-09-17T21:19:20.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:20.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198729+0000", 2024-09-17T21:19:20.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6673000000000001e-05 2024-09-17T21:19:20.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:20.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198742+0000", 2024-09-17T21:19:20.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3543000000000001e-05 2024-09-17T21:19:20.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:20.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242981+0000", 2024-09-17T21:19:20.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044238953999999997 2024-09-17T21:19:20.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:20.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243155+0000", 2024-09-17T21:19:20.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017401600000000001 2024-09-17T21:19:20.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:20.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243316+0000", 2024-09-17T21:19:20.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016137399999999999 2024-09-17T21:19:20.942 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.942 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:20.942 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.942 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:521 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:20.942 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:19:20.942 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 54.706861861999997, 2024-09-17T21:19:20.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045043685999999999, 2024-09-17T21:19:20.942 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:20.942 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:20.942 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:20.943 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:20.943 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:20.943 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:20.943 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 521 2024-09-17T21:19:20.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.943 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:20.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:20.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:19:20.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:20.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:19:20.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:20.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199253+0000", 2024-09-17T21:19:20.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1400000000000001e-06 2024-09-17T21:19:20.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:20.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199258+0000", 2024-09-17T21:19:20.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1800000000000004e-06 2024-09-17T21:19:20.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:20.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199276+0000", 2024-09-17T21:19:20.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7686999999999999e-05 2024-09-17T21:19:20.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:20.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199289+0000", 2024-09-17T21:19:20.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3263e-05 2024-09-17T21:19:20.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:20.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243397+0000", 2024-09-17T21:19:20.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044107636999999998 2024-09-17T21:19:20.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:20.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243527+0000", 2024-09-17T21:19:20.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013038399999999999 2024-09-17T21:19:20.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:20.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244296+0000", 2024-09-17T21:19:20.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076839499999999995 2024-09-17T21:19:20.947 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.947 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:20.947 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.947 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:522 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:20.947 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:19:20.947 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 54.706806559, 2024-09-17T21:19:20.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046049870999999999, 2024-09-17T21:19:20.947 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:20.947 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:20.947 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:20.948 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:20.948 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:20.948 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:20.948 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 522 2024-09-17T21:19:20.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.948 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:20.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:20.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:19:20.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:20.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:19:20.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:20.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199309+0000", 2024-09-17T21:19:20.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:19:20.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:20.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199315+0000", 2024-09-17T21:19:20.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2440000000000001e-06 2024-09-17T21:19:20.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:20.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199333+0000", 2024-09-17T21:19:20.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7886000000000001e-05 2024-09-17T21:19:20.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:20.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199379+0000", 2024-09-17T21:19:20.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6383e-05 2024-09-17T21:19:20.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:20.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244394+0000", 2024-09-17T21:19:20.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045014851000000002 2024-09-17T21:19:20.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:20.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244491+0000", 2024-09-17T21:19:20.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7245999999999997e-05 2024-09-17T21:19:20.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:20.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245357+0000", 2024-09-17T21:19:20.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00086626800000000001 2024-09-17T21:19:20.951 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.951 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:20.951 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.952 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:527 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:20.952 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:19:20.952 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 54.706709893000003, 2024-09-17T21:19:20.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045812522000000001, 2024-09-17T21:19:20.952 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:20.952 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:20.952 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:20.952 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:20.952 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:20.952 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:20.952 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 527 2024-09-17T21:19:20.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.953 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:20.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:20.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:19:20.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:20.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:19:20.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:20.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199406+0000", 2024-09-17T21:19:20.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.367e-06 2024-09-17T21:19:20.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:20.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199412+0000", 2024-09-17T21:19:20.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:19:20.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:20.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199429+0000", 2024-09-17T21:19:20.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7396000000000001e-05 2024-09-17T21:19:20.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:20.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199442+0000", 2024-09-17T21:19:20.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3327e-05 2024-09-17T21:19:20.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:20.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244361+0000", 2024-09-17T21:19:20.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044918197 2024-09-17T21:19:20.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:20.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244482+0000", 2024-09-17T21:19:20.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121945 2024-09-17T21:19:20.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:20.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245217+0000", 2024-09-17T21:19:20.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00073415999999999998 2024-09-17T21:19:20.956 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.956 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:20.956 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.956 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:534 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:20.956 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:19:20.957 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 54.706651186999999, 2024-09-17T21:19:20.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046292505999999997, 2024-09-17T21:19:20.957 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:20.957 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:20.957 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:20.957 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:20.957 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:20.957 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:20.957 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:19:20.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.957 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:20.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:20.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:19:20.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:20.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:19:20.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:20.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199464+0000", 2024-09-17T21:19:20.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:19:20.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:20.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199473+0000", 2024-09-17T21:19:20.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5629999999999992e-06 2024-09-17T21:19:20.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:20.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199492+0000", 2024-09-17T21:19:20.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9273999999999999e-05 2024-09-17T21:19:20.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:20.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199505+0000", 2024-09-17T21:19:20.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3443e-05 2024-09-17T21:19:20.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:20.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245357+0000", 2024-09-17T21:19:20.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045851714000000002 2024-09-17T21:19:20.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:20.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245550+0000", 2024-09-17T21:19:20.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019301600000000001 2024-09-17T21:19:20.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:20.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245755+0000", 2024-09-17T21:19:20.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205226 2024-09-17T21:19:20.961 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.961 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:20.961 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.961 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:535 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:20.961 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:19:20.961 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 54.706546631000002, 2024-09-17T21:19:20.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046637492000000003, 2024-09-17T21:19:20.961 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:20.962 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:20.962 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:20.962 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:20.962 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:20.962 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:20.962 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 535 2024-09-17T21:19:20.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.962 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:20.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:20.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:19:20.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:20.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:19:20.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:20.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199568+0000", 2024-09-17T21:19:20.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.037e-06 2024-09-17T21:19:20.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:20.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199572+0000", 2024-09-17T21:19:20.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5729999999999999e-06 2024-09-17T21:19:20.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:20.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199585+0000", 2024-09-17T21:19:20.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.279e-05 2024-09-17T21:19:20.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:20.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199592+0000", 2024-09-17T21:19:20.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.61e-06 2024-09-17T21:19:20.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:20.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245873+0000", 2024-09-17T21:19:20.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046280494999999998 2024-09-17T21:19:20.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:20.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246018+0000", 2024-09-17T21:19:20.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000144697 2024-09-17T21:19:20.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:20.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246205+0000", 2024-09-17T21:19:20.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018729 2024-09-17T21:19:20.965 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.966 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:20.966 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.966 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:543 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:20.966 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:20.966 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 54.706451385000001, 2024-09-17T21:19:20.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047564097999999999, 2024-09-17T21:19:20.966 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:20.966 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:20.966 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:20.966 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:20.966 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:20.967 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:20.967 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 543 2024-09-17T21:19:20.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.967 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:20.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:20.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:20.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:20.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:20.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:20.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:20.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9699999999999995e-07 2024-09-17T21:19:20.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:20.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199677+0000", 2024-09-17T21:19:20.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:19:20.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:20.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199690+0000", 2024-09-17T21:19:20.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2977e-05 2024-09-17T21:19:20.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:20.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199705+0000", 2024-09-17T21:19:20.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4656e-05 2024-09-17T21:19:20.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:20.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246307+0000", 2024-09-17T21:19:20.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046601780000000002 2024-09-17T21:19:20.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:20.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246475+0000", 2024-09-17T21:19:20.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016834200000000001 2024-09-17T21:19:20.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:20.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247227+0000", 2024-09-17T21:19:20.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00075173299999999998 2024-09-17T21:19:20.970 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.970 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:20.970 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.971 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:544 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:20.971 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:20.971 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 54.706392975999997, 2024-09-17T21:19:20.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049283559999999997, 2024-09-17T21:19:20.971 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:20.971 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:20.971 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:20.971 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:20.971 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:20.971 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:20.971 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 544 2024-09-17T21:19:20.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.972 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:20.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:20.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:20.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:20.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:20.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:20.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199722+0000", 2024-09-17T21:19:20.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.08e-06 2024-09-17T21:19:20.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:20.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199738+0000", 2024-09-17T21:19:20.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5943999999999998e-05 2024-09-17T21:19:20.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:20.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199748+0000", 2024-09-17T21:19:20.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0193e-05 2024-09-17T21:19:20.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:20.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199756+0000", 2024-09-17T21:19:20.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4200000000000001e-06 2024-09-17T21:19:20.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:20.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247311+0000", 2024-09-17T21:19:20.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047555300000000002 2024-09-17T21:19:20.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:20.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247571+0000", 2024-09-17T21:19:20.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025996999999999999 2024-09-17T21:19:20.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:20.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249005+0000", 2024-09-17T21:19:20.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001433653 2024-09-17T21:19:20.975 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.975 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:20.975 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.975 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:549 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:20.975 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:20.975 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 54.706347352000002, 2024-09-17T21:19:20.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048464528999999999, 2024-09-17T21:19:20.976 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:20.976 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:20.976 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:20.976 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:20.976 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:20.976 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:20.976 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 549 2024-09-17T21:19:20.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.976 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:20.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:20.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:20.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:20.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:20.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:20.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:20.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:19:20.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:20.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199770+0000", 2024-09-17T21:19:20.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9969999999999999e-06 2024-09-17T21:19:20.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:20.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199780+0000", 2024-09-17T21:19:20.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0003000000000001e-05 2024-09-17T21:19:20.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:20.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199793+0000", 2024-09-17T21:19:20.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2074e-05 2024-09-17T21:19:20.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:20.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247289+0000", 2024-09-17T21:19:20.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047496691000000001 2024-09-17T21:19:20.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:20.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247437+0000", 2024-09-17T21:19:20.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014807899999999999 2024-09-17T21:19:20.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:20.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248231+0000", 2024-09-17T21:19:20.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00079390899999999998 2024-09-17T21:19:20.980 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.980 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:20.980 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.980 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:558 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:20.980 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:20.980 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 54.706306839, 2024-09-17T21:19:20.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049709821000000001, 2024-09-17T21:19:20.980 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:20.980 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:20.980 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:20.981 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:20.981 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:20.981 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:20.981 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 558 2024-09-17T21:19:20.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.981 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:20.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:20.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:20.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:20.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:20.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:20.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199809+0000", 2024-09-17T21:19:20.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.333e-06 2024-09-17T21:19:20.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:20.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199814+0000", 2024-09-17T21:19:20.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0930000000000002e-06 2024-09-17T21:19:20.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:20.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199831+0000", 2024-09-17T21:19:20.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7580000000000001e-05 2024-09-17T21:19:20.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:20.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199860+0000", 2024-09-17T21:19:20.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8609999999999999e-05 2024-09-17T21:19:20.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:20.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249106+0000", 2024-09-17T21:19:20.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049246545000000003 2024-09-17T21:19:20.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:20.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249274+0000", 2024-09-17T21:19:20.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016806699999999999 2024-09-17T21:19:20.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:20.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249517+0000", 2024-09-17T21:19:20.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024259300000000001 2024-09-17T21:19:20.984 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.984 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:20.984 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.985 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:559 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:20.985 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:20.985 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 54.706187477, 2024-09-17T21:19:20.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049990694000000002, 2024-09-17T21:19:20.985 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:20.985 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:20.985 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:20.985 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:20.985 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:20.985 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:20.985 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 559 2024-09-17T21:19:20.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.985 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:20.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:20.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:20.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:20.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:20.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:20.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199928+0000", 2024-09-17T21:19:20.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:19:20.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:20.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199935+0000", 2024-09-17T21:19:20.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5429999999999999e-06 2024-09-17T21:19:20.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:20.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199954+0000", 2024-09-17T21:19:20.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.912e-05 2024-09-17T21:19:20.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:20.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199969+0000", 2024-09-17T21:19:20.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5305999999999999e-05 2024-09-17T21:19:20.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:20.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249604+0000", 2024-09-17T21:19:20.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049634931 2024-09-17T21:19:20.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:20.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249738+0000", 2024-09-17T21:19:20.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013411599999999999 2024-09-17T21:19:20.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:20.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249917+0000", 2024-09-17T21:19:20.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179111 2024-09-17T21:19:20.989 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.989 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:20.989 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.989 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:560 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:20.989 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:20.989 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 54.706123061, 2024-09-17T21:19:20.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050173059999999998, 2024-09-17T21:19:20.989 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:20.989 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:20.989 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:20.990 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:20.990 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:20.990 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:20.990 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 560 2024-09-17T21:19:20.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.990 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:20.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:20.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:20.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:20.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:20.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:20.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199992+0000", 2024-09-17T21:19:20.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.517e-06 2024-09-17T21:19:20.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:20.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199998+0000", 2024-09-17T21:19:20.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1460000000000004e-06 2024-09-17T21:19:20.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:20.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:19:20.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7407000000000001e-05 2024-09-17T21:19:20.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:20.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200029+0000", 2024-09-17T21:19:20.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3499999999999999e-05 2024-09-17T21:19:20.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:20.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249968+0000", 2024-09-17T21:19:20.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049939799 2024-09-17T21:19:20.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:20.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250055+0000", 2024-09-17T21:19:20.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6731999999999996e-05 2024-09-17T21:19:20.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:20.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250164+0000", 2024-09-17T21:19:20.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108959 2024-09-17T21:19:20.993 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.993 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:20.993 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.994 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:19:20.994 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:20.994 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.096333352000002, 2024-09-17T21:19:20.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:19:20.994 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:20.994 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:20.994 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:20.994 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:20.994 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:19:20.994 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:19:20.994 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:19:20.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.994 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:20.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:20.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:20.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:20.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:20.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:20.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:20.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:19:20.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:19:20.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:20.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:19:20.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:19:20.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:20.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:19:20.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:19:20.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:20.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:19:20.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:19:20.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:20.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:19:20.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:19:20.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:20.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:19:20.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:19:20.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:20.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:19:20.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:19:20.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:20.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:19:20.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:19:20.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:20.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:19:20.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:19:20.998 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.998 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:20.999 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:20.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:20.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:20.999 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:19:20.999 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:20.999 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.541231566, 2024-09-17T21:19:20.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:19:20.999 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:20.999 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:20.999 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:20.999 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:20.999 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:19:20.999 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:21.000 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:19:21.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.000 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:21.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:21.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:21.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:21.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:21.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:21.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:19:21.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:19:21.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:21.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:19:21.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:19:21.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:21.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:19:21.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:19:21.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:21.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:19:21.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:19:21.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:21.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:19:21.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:19:21.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:21.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:19:21.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:19:21.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:21.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:19:21.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:19:21.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:21.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:19:21.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:19:21.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:21.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:19:21.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:19:21.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:21.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:19:21.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:19:21.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:21.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:19:21.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:19:21.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:21.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:19:21.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:19:21.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:21.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:19:21.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:19:21.005 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.005 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:21.005 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.005 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.005 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:21.005 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:21.007 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:19:21.007 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:21.004+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:21.107 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:21.103+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:21.174 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:21.175 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:21.175 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:19:21.175 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:21.189 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:19:21.208 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:21.204+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:21.309 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:19:21.310 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:21.306+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:21.353 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:21.353 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:21.353 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:21.353 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:21.353 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:21.366 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:19:21.413 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:21.410+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:21.512 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:21.508+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:21.527 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:19:21.527 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:19:21.527 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T21:19:21.527 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 0.756164 2024-09-17T21:19:21.527 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 0.7561636746818645 2024-09-17T21:19:21.535 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:21.535 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:21.536 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:21.536 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:21.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.536 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:12 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:19:21.536 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:19:21.536 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 68.201162574999998, 2024-09-17T21:19:21.536 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061340989999999996, 2024-09-17T21:19:21.536 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:21.536 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:21.536 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:21.536 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:21.536 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:19:21.536 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:21.536 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12 2024-09-17T21:19:21.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.537 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:21.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:21.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:19:21.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:21.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:19:21.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:21.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332336+0000", 2024-09-17T21:19:21.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5519999999999999e-06 2024-09-17T21:19:21.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:21.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332342+0000", 2024-09-17T21:19:21.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7180000000000001e-06 2024-09-17T21:19:21.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:21.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332352+0000", 2024-09-17T21:19:21.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:19:21.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:21.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332359+0000", 2024-09-17T21:19:21.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3479999999999998e-06 2024-09-17T21:19:21.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:21.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332413+0000", 2024-09-17T21:19:21.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4447999999999998e-05 2024-09-17T21:19:21.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-17T21:19:21.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332417+0000", 2024-09-17T21:19:21.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.895e-06 2024-09-17T21:19:21.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:21.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334755+0000", 2024-09-17T21:19:21.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023377189999999998 2024-09-17T21:19:21.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:21.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334903+0000", 2024-09-17T21:19:21.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014778999999999999 2024-09-17T21:19:21.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:19:21.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.335187+0000", 2024-09-17T21:19:21.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000284715 2024-09-17T21:19:21.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:21.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.337731+0000", 2024-09-17T21:19:21.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002543741 2024-09-17T21:19:21.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.541 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:21.541 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338382+0000", 2024-09-17T21:19:21.541 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00065029200000000001 2024-09-17T21:19:21.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.541 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:21.541 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338418+0000", 2024-09-17T21:19:21.541 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.697e-05 2024-09-17T21:19:21.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.541 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:21.541 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338469+0000", 2024-09-17T21:19:21.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0154999999999997e-05 2024-09-17T21:19:21.542 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.542 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:21.542 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.542 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:782 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'63, mlcod=28'60)", 2024-09-17T21:19:21.542 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:19:21.542 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.318732345000001, 2024-09-17T21:19:21.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006167283, 2024-09-17T21:19:21.542 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:21.542 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:21.542 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:21.542 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:21.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:21.543 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:19:21.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:21.543 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:19:21.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:21.543 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214767+0000", 2024-09-17T21:19:21.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:19:21.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.544 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:21.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214779+0000", 2024-09-17T21:19:21.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2627e-05 2024-09-17T21:19:21.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.544 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:21.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214805+0000", 2024-09-17T21:19:21.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5570000000000001e-05 2024-09-17T21:19:21.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.544 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:21.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214823+0000", 2024-09-17T21:19:21.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8202999999999998e-05 2024-09-17T21:19:21.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:21.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215170+0000", 2024-09-17T21:19:21.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034720399999999999 2024-09-17T21:19:21.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:21.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215235+0000", 2024-09-17T21:19:21.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4276000000000006e-05 2024-09-17T21:19:21.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:21.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220859+0000", 2024-09-17T21:19:21.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056240500000000002 2024-09-17T21:19:21.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:21.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220932+0000", 2024-09-17T21:19:21.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3503000000000004e-05 2024-09-17T21:19:21.546 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.546 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:21.546 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.546 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:19:21.546 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:21.547 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.316136817999997, 2024-09-17T21:19:21.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:19:21.547 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:21.547 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:21.547 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:21.547 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:21.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:21.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:21.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:21.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:21.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:21.548 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:19:21.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:19:21.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:21.548 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:19:21.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:19:21.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:21.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:19:21.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:19:21.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:21.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:19:21.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:19:21.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:21.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:19:21.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:19:21.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:21.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:19:21.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:19:21.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:21.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:19:21.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:19:21.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:21.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:19:21.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:19:21.550 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.550 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:21.550 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.551 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:790 3.5 e28/23 3:a14b1a37:::rbd_data.1152c6af092e.000000000000002f:head v 28'1, mlcod=0'0)", 2024-09-17T21:19:21.551 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:19:21.551 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.311098059999999, 2024-09-17T21:19:21.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059622240000000003, 2024-09-17T21:19:21.551 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:21.551 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:21.551 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:21.551 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:21.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:21.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:19:21.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:21.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:19:21.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:21.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222400+0000", 2024-09-17T21:19:21.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.02e-06 2024-09-17T21:19:21.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:21.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222411+0000", 2024-09-17T21:19:21.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.044e-05 2024-09-17T21:19:21.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:21.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222440+0000", 2024-09-17T21:19:21.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.932e-05 2024-09-17T21:19:21.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:21.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222458+0000", 2024-09-17T21:19:21.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8615999999999999e-05 2024-09-17T21:19:21.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:21.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222609+0000", 2024-09-17T21:19:21.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015102299999999999 2024-09-17T21:19:21.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:21.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222657+0000", 2024-09-17T21:19:21.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7206000000000002e-05 2024-09-17T21:19:21.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:21.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228278+0000", 2024-09-17T21:19:21.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056208100000000004 2024-09-17T21:19:21.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:21.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228361+0000", 2024-09-17T21:19:21.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3788999999999993e-05 2024-09-17T21:19:21.554 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.555 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:21.555 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.555 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:19:21.555 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:21.555 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 58.665836153999997, 2024-09-17T21:19:21.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:19:21.555 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:21.555 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:21.555 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:21.555 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:21.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:21.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:21.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:21.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:21.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:21.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:19:21.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:19:21.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:21.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:19:21.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:19:21.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:21.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:19:21.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:19:21.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:21.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:19:21.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:19:21.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:21.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:19:21.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:19:21.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:21.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:19:21.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:19:21.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:21.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:19:21.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:19:21.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:21.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:19:21.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:19:21.559 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.559 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:21.559 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.559 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:19:21.559 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:21.559 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 58.664571070999997, 2024-09-17T21:19:21.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:19:21.559 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:21.559 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:21.559 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:21.559 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:21.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.560 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:21.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:21.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.560 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:21.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:21.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.560 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:21.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:19:21.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:19:21.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:21.561 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:19:21.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:19:21.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:21.561 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:19:21.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:19:21.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:21.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:19:21.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:19:21.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:21.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:19:21.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:19:21.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:21.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:19:21.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:19:21.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:21.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:19:21.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:19:21.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:21.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:19:21.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:19:21.563 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.563 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:21.563 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.563 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:19:21.563 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:21.563 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 58.654791660999997, 2024-09-17T21:19:21.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:19:21.564 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:21.564 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:21.564 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:21.564 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:21.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:21.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:21.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:21.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:21.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.565 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:21.565 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:19:21.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:19:21.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.565 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:21.565 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:19:21.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:19:21.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.565 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:21.565 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:19:21.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:19:21.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:21.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:19:21.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:19:21.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:21.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:19:21.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:19:21.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:21.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:19:21.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:19:21.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.567 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:21.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:19:21.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:19:21.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.567 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:21.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:19:21.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:19:21.567 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.567 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:21.567 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.567 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:19:21.568 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:21.568 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 58.646069257999997, 2024-09-17T21:19:21.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:19:21.568 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:21.568 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:21.568 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:21.568 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:21.568 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.568 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:21.568 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:21.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.568 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.569 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:21.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:21.569 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.569 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.569 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:21.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:19:21.569 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:19:21.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.569 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.569 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:21.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:19:21.569 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:19:21.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.570 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:21.570 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:19:21.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:19:21.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.570 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:21.570 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:19:21.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:19:21.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.570 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:21.570 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:19:21.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:19:21.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.571 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.571 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:21.571 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:19:21.571 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:19:21.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.571 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.571 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:21.571 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:19:21.571 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:19:21.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.571 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.571 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:21.571 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:19:21.571 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:19:21.571 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.572 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:21.572 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.572 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:19:21.572 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:21.572 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 45.260978694000002, 2024-09-17T21:19:21.572 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:19:21.572 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:21.572 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:21.572 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:21.572 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:21.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.572 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:21.573 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:21.573 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.573 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.573 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:21.573 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:21.573 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.573 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.573 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:21.573 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:19:21.573 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:19:21.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.573 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.574 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:21.574 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:19:21.574 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:19:21.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.574 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:21.574 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:19:21.574 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:19:21.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.574 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:21.574 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:19:21.574 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:19:21.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.575 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:21.575 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:19:21.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:19:21.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.575 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.575 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:21.575 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:19:21.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:19:21.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.575 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.575 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:21.575 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:19:21.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:19:21.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.575 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.575 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:21.576 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:19:21.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:19:21.576 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.576 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:21.576 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.576 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.576 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:870 3.d e68/23 3:b061e05c:::rbd_data.115265216c1c.000000000000006b:head v 68'28, mlcod=68'25)", 2024-09-17T21:19:21.576 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:12.356263+0000", 2024-09-17T21:19:21.576 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.1772345000000008, 2024-09-17T21:19:21.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059492920000000001, 2024-09-17T21:19:21.576 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:21.576 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:21.576 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:21.577 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:21.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.577 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:21.577 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:12.356263+0000", 2024-09-17T21:19:21.577 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.577 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:21.577 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:12.356263+0000", 2024-09-17T21:19:21.577 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.577 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:21.577 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:12.356264+0000", 2024-09-17T21:19:21.578 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:19:21.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.578 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.578 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:21.578 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:12.356279+0000", 2024-09-17T21:19:21.578 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4790000000000001e-05 2024-09-17T21:19:21.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.578 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.578 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:21.578 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:12.356305+0000", 2024-09-17T21:19:21.578 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6041999999999999e-05 2024-09-17T21:19:21.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.578 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.578 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:21.579 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:12.356326+0000", 2024-09-17T21:19:21.579 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0772999999999999e-05 2024-09-17T21:19:21.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.579 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:21.579 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:12.356471+0000", 2024-09-17T21:19:21.579 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014509599999999999 2024-09-17T21:19:21.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.579 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:21.579 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:12.356551+0000", 2024-09-17T21:19:21.579 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0323999999999999e-05 2024-09-17T21:19:21.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.580 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:21.580 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:12.362103+0000", 2024-09-17T21:19:21.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0055514739999999998 2024-09-17T21:19:21.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.580 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.580 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:21.580 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:12.362212+0000", 2024-09-17T21:19:21.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000109193 2024-09-17T21:19:21.580 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.580 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:21.580 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.580 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.581 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1777 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'194, mlcod=69'189)", 2024-09-17T21:19:21.581 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.249616+0000", 2024-09-17T21:19:21.581 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.2838808200000003, 2024-09-17T21:19:21.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006283589, 2024-09-17T21:19:21.581 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:21.581 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:21.581 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:21.581 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:21.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.581 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:21.581 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.249616+0000", 2024-09-17T21:19:21.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.581 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:21.582 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.249616+0000", 2024-09-17T21:19:21.582 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.582 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.582 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:21.582 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.249618+0000", 2024-09-17T21:19:21.582 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2440000000000001e-06 2024-09-17T21:19:21.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.582 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.582 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:21.582 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.249626+0000", 2024-09-17T21:19:21.582 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.32e-06 2024-09-17T21:19:21.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.582 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.583 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:21.583 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.249645+0000", 2024-09-17T21:19:21.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9315000000000001e-05 2024-09-17T21:19:21.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.583 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:21.583 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.249656+0000", 2024-09-17T21:19:21.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543999999999999e-05 2024-09-17T21:19:21.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.583 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:21.583 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250345+0000", 2024-09-17T21:19:21.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00068912499999999996 2024-09-17T21:19:21.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.584 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:21.584 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250401+0000", 2024-09-17T21:19:21.584 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5985e-05 2024-09-17T21:19:21.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.584 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.584 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:21.584 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.255797+0000", 2024-09-17T21:19:21.584 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0053965790000000003 2024-09-17T21:19:21.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.584 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.584 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:21.584 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.255900+0000", 2024-09-17T21:19:21.584 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000102477 2024-09-17T21:19:21.584 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.584 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:21.585 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.585 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.585 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1780 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'195, mlcod=69'190)", 2024-09-17T21:19:21.585 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:19:21.585 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.2831230900000001, 2024-09-17T21:19:21.585 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075761769999999999, 2024-09-17T21:19:21.585 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:21.585 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:21.585 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:21.585 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:21.585 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.585 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:21.585 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:19:21.586 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.586 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.586 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:21.586 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:19:21.586 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.586 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.586 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:21.586 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250375+0000", 2024-09-17T21:19:21.586 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:19:21.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.586 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.586 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:21.586 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250383+0000", 2024-09-17T21:19:21.587 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7870000000000002e-06 2024-09-17T21:19:21.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.587 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:21.587 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250402+0000", 2024-09-17T21:19:21.587 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9346000000000001e-05 2024-09-17T21:19:21.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.587 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:21.587 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250413+0000", 2024-09-17T21:19:21.587 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.039e-05 2024-09-17T21:19:21.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.587 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:21.587 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251353+0000", 2024-09-17T21:19:21.587 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000940091 2024-09-17T21:19:21.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.588 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:21.588 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251407+0000", 2024-09-17T21:19:21.588 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4701999999999999e-05 2024-09-17T21:19:21.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.588 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:21.588 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257829+0000", 2024-09-17T21:19:21.588 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064212779999999999 2024-09-17T21:19:21.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.588 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:21.588 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257950+0000", 2024-09-17T21:19:21.588 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121623 2024-09-17T21:19:21.588 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.589 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:21.589 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.589 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.589 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1781 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'196, mlcod=69'190)", 2024-09-17T21:19:21.589 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:19:21.589 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.2820401389999994, 2024-09-17T21:19:21.589 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065632939999999999, 2024-09-17T21:19:21.589 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:21.589 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:21.589 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:21.589 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:21.589 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.589 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:21.590 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:19:21.590 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:21.590 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:19:21.590 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:21.590 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251459+0000", 2024-09-17T21:19:21.590 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.624e-06 2024-09-17T21:19:21.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:21.591 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251471+0000", 2024-09-17T21:19:21.591 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2636000000000001e-05 2024-09-17T21:19:21.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.591 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:21.591 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251498+0000", 2024-09-17T21:19:21.591 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:19:21.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.591 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:21.591 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251515+0000", 2024-09-17T21:19:21.591 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6939000000000002e-05 2024-09-17T21:19:21.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.591 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:21.591 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252938+0000", 2024-09-17T21:19:21.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001423082 2024-09-17T21:19:21.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.592 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:21.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252999+0000", 2024-09-17T21:19:21.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1138999999999999e-05 2024-09-17T21:19:21.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.592 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:21.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257955+0000", 2024-09-17T21:19:21.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049559640000000002 2024-09-17T21:19:21.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.592 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:21.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.258020+0000", 2024-09-17T21:19:21.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5504000000000002e-05 2024-09-17T21:19:21.593 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.593 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:21.593 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.593 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1788 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'198, mlcod=69'191)", 2024-09-17T21:19:21.593 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:19:21.593 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.28001465, 2024-09-17T21:19:21.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073690370000000002, 2024-09-17T21:19:21.593 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:21.593 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:21.593 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:21.593 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:21.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.594 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:21.594 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:19:21.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.594 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:21.594 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:19:21.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.594 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:21.594 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253484+0000", 2024-09-17T21:19:21.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2470000000000001e-06 2024-09-17T21:19:21.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.595 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:21.595 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253492+0000", 2024-09-17T21:19:21.595 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7159999999999992e-06 2024-09-17T21:19:21.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.595 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:21.595 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253509+0000", 2024-09-17T21:19:21.595 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.666e-05 2024-09-17T21:19:21.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.595 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:21.595 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253522+0000", 2024-09-17T21:19:21.595 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2643000000000001e-05 2024-09-17T21:19:21.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.596 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:21.596 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254853+0000", 2024-09-17T21:19:21.596 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331238 2024-09-17T21:19:21.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.596 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:21.596 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254885+0000", 2024-09-17T21:19:21.596 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1562e-05 2024-09-17T21:19:21.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.596 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:21.596 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260788+0000", 2024-09-17T21:19:21.596 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059030749999999998 2024-09-17T21:19:21.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:21.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260852+0000", 2024-09-17T21:19:21.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3895999999999999e-05 2024-09-17T21:19:21.597 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.597 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:21.597 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.597 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5746 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'477, mlcod=71'473)", 2024-09-17T21:19:21.597 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:19:21.597 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.274587092, 2024-09-17T21:19:21.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074182709999999997, 2024-09-17T21:19:21.597 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:21.597 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:21.597 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:21.598 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:21.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:21.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:19:21.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:21.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:19:21.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:21.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258912+0000", 2024-09-17T21:19:21.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:19:21.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:21.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258926+0000", 2024-09-17T21:19:21.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4249e-05 2024-09-17T21:19:21.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:21.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258946+0000", 2024-09-17T21:19:21.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9879999999999999e-05 2024-09-17T21:19:21.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:21.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258956+0000", 2024-09-17T21:19:21.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:19:21.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.600 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:21.600 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259380+0000", 2024-09-17T21:19:21.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042337599999999999 2024-09-17T21:19:21.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.600 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:21.600 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259412+0000", 2024-09-17T21:19:21.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:19:21.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.600 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:21.600 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266191+0000", 2024-09-17T21:19:21.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067793269999999999 2024-09-17T21:19:21.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:21.601 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266328+0000", 2024-09-17T21:19:21.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013690999999999999 2024-09-17T21:19:21.601 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.601 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:21.601 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.601 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5747 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'478, mlcod=71'473)", 2024-09-17T21:19:21.601 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:19:21.601 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.2738807430000003, 2024-09-17T21:19:21.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068779440000000004, 2024-09-17T21:19:21.601 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:21.601 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:21.601 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:21.601 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:21.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.602 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:21.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:19:21.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.602 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:21.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:19:21.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.602 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:21.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259618+0000", 2024-09-17T21:19:21.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6199999999999999e-06 2024-09-17T21:19:21.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:21.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259630+0000", 2024-09-17T21:19:21.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1884e-05 2024-09-17T21:19:21.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:21.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259657+0000", 2024-09-17T21:19:21.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7339e-05 2024-09-17T21:19:21.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:21.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259677+0000", 2024-09-17T21:19:21.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0156000000000001e-05 2024-09-17T21:19:21.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:21.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260521+0000", 2024-09-17T21:19:21.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084312899999999997 2024-09-17T21:19:21.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:21.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260585+0000", 2024-09-17T21:19:21.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4558000000000001e-05 2024-09-17T21:19:21.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:21.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266336+0000", 2024-09-17T21:19:21.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057505020000000002 2024-09-17T21:19:21.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.605 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:21.605 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266494+0000", 2024-09-17T21:19:21.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158756 2024-09-17T21:19:21.605 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.605 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:21.605 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.605 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5750 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'479, mlcod=71'473)", 2024-09-17T21:19:21.605 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:19:21.605 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.2731685500000003, 2024-09-17T21:19:21.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006302874, 2024-09-17T21:19:21.605 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:21.605 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:21.605 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:21.605 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:21.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:21.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:19:21.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:21.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:19:21.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:21.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260330+0000", 2024-09-17T21:19:21.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:19:21.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:21.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260343+0000", 2024-09-17T21:19:21.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2862e-05 2024-09-17T21:19:21.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:21.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260367+0000", 2024-09-17T21:19:21.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3723e-05 2024-09-17T21:19:21.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:21.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260386+0000", 2024-09-17T21:19:21.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8879999999999999e-05 2024-09-17T21:19:21.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:21.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261579+0000", 2024-09-17T21:19:21.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001193431 2024-09-17T21:19:21.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:21.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261608+0000", 2024-09-17T21:19:21.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8858999999999999e-05 2024-09-17T21:19:21.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:21.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266505+0000", 2024-09-17T21:19:21.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004896999 2024-09-17T21:19:21.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.609 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:21.609 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266631+0000", 2024-09-17T21:19:21.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012655300000000001 2024-09-17T21:19:21.609 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.609 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:21.609 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.609 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5751 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'480, mlcod=71'473)", 2024-09-17T21:19:21.609 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:19:21.609 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.2724613380000003, 2024-09-17T21:19:21.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075756299999999999, 2024-09-17T21:19:21.609 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:21.610 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:21.610 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:21.610 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:21.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:21.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:19:21.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:21.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:19:21.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:21.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261037+0000", 2024-09-17T21:19:21.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:19:21.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:21.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261049+0000", 2024-09-17T21:19:21.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1063999999999999e-05 2024-09-17T21:19:21.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:21.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261072+0000", 2024-09-17T21:19:21.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3102e-05 2024-09-17T21:19:21.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:21.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261133+0000", 2024-09-17T21:19:21.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1276000000000001e-05 2024-09-17T21:19:21.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:21.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:19:21.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015334319999999999 2024-09-17T21:19:21.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:21.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262702+0000", 2024-09-17T21:19:21.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5281999999999999e-05 2024-09-17T21:19:21.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:21.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268527+0000", 2024-09-17T21:19:21.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058250389999999997 2024-09-17T21:19:21.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:21.613 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268611+0000", 2024-09-17T21:19:21.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4724999999999999e-05 2024-09-17T21:19:21.613 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.613 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:21.613 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.613 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5756 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'481, mlcod=71'473)", 2024-09-17T21:19:21.613 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:19:21.613 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.2716004820000002, 2024-09-17T21:19:21.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006772386, 2024-09-17T21:19:21.613 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:21.614 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:21.614 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:21.614 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:21.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:21.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:19:21.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:21.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:19:21.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:21.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261899+0000", 2024-09-17T21:19:21.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.903e-06 2024-09-17T21:19:21.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:21.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261912+0000", 2024-09-17T21:19:21.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-05 2024-09-17T21:19:21.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:21.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261938+0000", 2024-09-17T21:19:21.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6098999999999999e-05 2024-09-17T21:19:21.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:21.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261955+0000", 2024-09-17T21:19:21.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.696e-05 2024-09-17T21:19:21.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:21.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263845+0000", 2024-09-17T21:19:21.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001890187 2024-09-17T21:19:21.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:21.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263907+0000", 2024-09-17T21:19:21.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1600999999999995e-05 2024-09-17T21:19:21.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:21.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268615+0000", 2024-09-17T21:19:21.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047086070000000001 2024-09-17T21:19:21.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:21.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268669+0000", 2024-09-17T21:19:21.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3588999999999998e-05 2024-09-17T21:19:21.617 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.617 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:21.617 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.617 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5757 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'482, mlcod=71'473)", 2024-09-17T21:19:21.617 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.262633+0000", 2024-09-17T21:19:21.618 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.2708645670000003, 2024-09-17T21:19:21.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061442279999999998, 2024-09-17T21:19:21.618 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:21.618 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:21.618 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:21.618 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:21.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:21.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262633+0000", 2024-09-17T21:19:21.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:21.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262633+0000", 2024-09-17T21:19:21.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:21.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:21.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262634+0000", 2024-09-17T21:19:21.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.587e-06 2024-09-17T21:19:21.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:21.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262645+0000", 2024-09-17T21:19:21.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0686000000000001e-05 2024-09-17T21:19:21.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:21.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:19:21.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0829999999999999e-05 2024-09-17T21:19:21.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:21.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262678+0000", 2024-09-17T21:19:21.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2583e-05 2024-09-17T21:19:21.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:21.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.265094+0000", 2024-09-17T21:19:21.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024153170000000002 2024-09-17T21:19:21.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:21.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.265167+0000", 2024-09-17T21:19:21.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3543999999999999e-05 2024-09-17T21:19:21.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:21.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268676+0000", 2024-09-17T21:19:21.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035090210000000002 2024-09-17T21:19:21.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:21.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:21.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:21.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268777+0000", 2024-09-17T21:19:21.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010066 2024-09-17T21:19:21.621 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.622 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:21.622 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.622 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:21.622 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:21.622 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:21.623 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:19:21.624 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:21.618+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:21.725 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:21.720+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:21.758 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:21.758 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:21.758 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:19:21.758 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:21.772 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:19:21.825 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:21.822+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:21.926 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:21.922+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:21.944 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:21.944 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:21.944 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:21.944 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:21.944 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:21.959 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:19:22.027 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:22.024+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:22.086 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:22.086 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:22.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:22.086 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:22.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.086 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:19:22.086 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:22.086 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 85.068434968999995, 2024-09-17T21:19:22.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:19:22.087 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.087 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.087 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:22.087 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.087 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:19:22.087 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:19:22.087 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:19:22.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.087 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:22.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:19:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:19:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:19:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:19:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:19:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:19:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:19:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:19:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:19:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:19:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:19:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:19:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:19:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:19:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:19:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:19:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:19:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:19:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:19:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:19:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:19:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:22.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:19:22.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:19:22.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:22.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:19:22.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:19:22.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:19:22.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:19:22.092 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.093 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.093 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.093 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:22.093 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:22.093 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.488270689000004, 2024-09-17T21:19:22.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:19:22.093 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.093 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.093 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.093 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.093 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:19:22.094 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:19:22.094 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:19:22.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.094 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:22.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:22.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:19:22.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:19:22.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:19:22.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:19:22.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:19:22.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:19:22.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:19:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:19:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:19:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:19:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:19:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:19:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:19:22.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:19:22.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:22.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:19:22.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:19:22.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:19:22.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:19:22.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:19:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:19:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:19:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:19:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.488173376999995, 2024-09-17T21:19:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:19:22.099 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.099 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.099 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.099 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.099 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:22.099 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:22.099 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:19:22.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.099 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:22.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:22.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:19:22.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:19:22.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:19:22.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:19:22.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:19:22.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:19:22.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:19:22.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:19:22.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:19:22.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:19:22.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:19:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:19:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:19:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:19:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:19:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:19:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:19:22.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:19:22.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:19:22.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:19:22.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:19:22.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:19:22.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:19:22.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:19:22.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:19:22.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:19:22.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:19:22.104 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.104 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.104 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.104 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:22.105 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:22.105 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.486086576000005, 2024-09-17T21:19:22.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:19:22.105 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.105 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.105 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:22.105 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.105 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:19:22.105 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:19:22.105 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:19:22.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.105 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:22.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:22.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:19:22.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:19:22.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:19:22.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:19:22.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:19:22.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:19:22.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:19:22.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:19:22.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:19:22.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:19:22.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:19:22.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:19:22.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:19:22.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:19:22.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:19:22.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:22.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:19:22.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:19:22.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:19:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:19:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:19:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:19:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:19:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:19:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:22.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:19:22.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:19:22.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:22.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:19:22.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:19:22.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:22.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:19:22.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:19:22.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:19:22.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:19:22.111 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.111 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.111 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.111 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:282 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.111 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:19:22.111 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.854381314999998, 2024-09-17T21:19:22.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023571801, 2024-09-17T21:19:22.111 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.111 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.111 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.111 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.112 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.112 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.112 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 282 2024-09-17T21:19:22.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.112 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:19:22.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227819+0000", 2024-09-17T21:19:22.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227820+0000", 2024-09-17T21:19:22.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.11e-06 2024-09-17T21:19:22.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227825+0000", 2024-09-17T21:19:22.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8099999999999997e-06 2024-09-17T21:19:22.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227848+0000", 2024-09-17T21:19:22.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2949999999999999e-05 2024-09-17T21:19:22.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227862+0000", 2024-09-17T21:19:22.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:19:22.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.247977+0000", 2024-09-17T21:19:22.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020114968 2024-09-17T21:19:22.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251141+0000", 2024-09-17T21:19:22.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031640689999999998 2024-09-17T21:19:22.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251391+0000", 2024-09-17T21:19:22.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025018499999999999 2024-09-17T21:19:22.115 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.115 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.115 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.115 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:283 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.115 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:19:22.115 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.854319476000001, 2024-09-17T21:19:22.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024022274999999999, 2024-09-17T21:19:22.116 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.116 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.116 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.116 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.116 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.116 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.116 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 283 2024-09-17T21:19:22.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.116 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:19:22.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227881+0000", 2024-09-17T21:19:22.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227882+0000", 2024-09-17T21:19:22.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-06 2024-09-17T21:19:22.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227887+0000", 2024-09-17T21:19:22.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1769999999999997e-06 2024-09-17T21:19:22.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227905+0000", 2024-09-17T21:19:22.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7235999999999999e-05 2024-09-17T21:19:22.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227918+0000", 2024-09-17T21:19:22.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3387000000000001e-05 2024-09-17T21:19:22.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251469+0000", 2024-09-17T21:19:22.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023550894999999999 2024-09-17T21:19:22.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251669+0000", 2024-09-17T21:19:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019992499999999999 2024-09-17T21:19:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251903+0000", 2024-09-17T21:19:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000234412 2024-09-17T21:19:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:284 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:19:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.854263922999998, 2024-09-17T21:19:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024450084, 2024-09-17T21:19:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 284 2024-09-17T21:19:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:19:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:19:22.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227937+0000", 2024-09-17T21:19:22.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6700000000000002e-07 2024-09-17T21:19:22.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227943+0000", 2024-09-17T21:19:22.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:19:22.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227961+0000", 2024-09-17T21:19:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.827e-05 2024-09-17T21:19:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227975+0000", 2024-09-17T21:19:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.394e-05 2024-09-17T21:19:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251977+0000", 2024-09-17T21:19:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024001340999999999 2024-09-17T21:19:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252169+0000", 2024-09-17T21:19:22.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019271199999999999 2024-09-17T21:19:22.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252387+0000", 2024-09-17T21:19:22.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021726100000000001 2024-09-17T21:19:22.123 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.123 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.123 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.123 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:285 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.124 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:19:22.124 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.854207633000001, 2024-09-17T21:19:22.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024884791, 2024-09-17T21:19:22.124 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.124 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.124 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.124 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.124 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.124 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.124 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 285 2024-09-17T21:19:22.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.124 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:19:22.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:19:22.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227994+0000", 2024-09-17T21:19:22.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1429999999999999e-06 2024-09-17T21:19:22.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227999+0000", 2024-09-17T21:19:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1669999999999998e-06 2024-09-17T21:19:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228015+0000", 2024-09-17T21:19:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6056e-05 2024-09-17T21:19:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228028+0000", 2024-09-17T21:19:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3196e-05 2024-09-17T21:19:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252458+0000", 2024-09-17T21:19:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024429751999999999 2024-09-17T21:19:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252650+0000", 2024-09-17T21:19:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019187800000000001 2024-09-17T21:19:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252878+0000", 2024-09-17T21:19:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022759899999999999 2024-09-17T21:19:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:286 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:19:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.854151874000003, 2024-09-17T21:19:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025299318000000001, 2024-09-17T21:19:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 286 2024-09-17T21:19:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:19:22.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:19:22.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228050+0000", 2024-09-17T21:19:22.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:19:22.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228055+0000", 2024-09-17T21:19:22.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4960000000000004e-06 2024-09-17T21:19:22.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228075+0000", 2024-09-17T21:19:22.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0109999999999999e-05 2024-09-17T21:19:22.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228089+0000", 2024-09-17T21:19:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4154e-05 2024-09-17T21:19:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252952+0000", 2024-09-17T21:19:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024862934 2024-09-17T21:19:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253141+0000", 2024-09-17T21:19:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000188782 2024-09-17T21:19:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253348+0000", 2024-09-17T21:19:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020684499999999999 2024-09-17T21:19:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:443 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:19:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.840564569000001, 2024-09-17T21:19:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023179569000000001, 2024-09-17T21:19:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 443 2024-09-17T21:19:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:19:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241636+0000", 2024-09-17T21:19:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241637+0000", 2024-09-17T21:19:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.46e-06 2024-09-17T21:19:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241643+0000", 2024-09-17T21:19:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1970000000000001e-06 2024-09-17T21:19:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241662+0000", 2024-09-17T21:19:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8712999999999999e-05 2024-09-17T21:19:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241676+0000", 2024-09-17T21:19:22.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.393e-05 2024-09-17T21:19:22.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264596+0000", 2024-09-17T21:19:22.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022919688000000001 2024-09-17T21:19:22.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264690+0000", 2024-09-17T21:19:22.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4145000000000001e-05 2024-09-17T21:19:22.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264815+0000", 2024-09-17T21:19:22.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012543600000000001 2024-09-17T21:19:22.136 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.136 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.136 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.136 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:444 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.136 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:19:22.136 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.840503519000002, 2024-09-17T21:19:22.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023260383999999999, 2024-09-17T21:19:22.136 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.136 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.136 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.137 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.137 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.137 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.137 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 444 2024-09-17T21:19:22.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.137 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:19:22.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241697+0000", 2024-09-17T21:19:22.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241698+0000", 2024-09-17T21:19:22.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2270000000000001e-06 2024-09-17T21:19:22.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241704+0000", 2024-09-17T21:19:22.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9800000000000003e-06 2024-09-17T21:19:22.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241725+0000", 2024-09-17T21:19:22.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0812999999999999e-05 2024-09-17T21:19:22.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241739+0000", 2024-09-17T21:19:22.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3773e-05 2024-09-17T21:19:22.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264850+0000", 2024-09-17T21:19:22.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023111326000000001 2024-09-17T21:19:22.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264897+0000", 2024-09-17T21:19:22.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6638999999999998e-05 2024-09-17T21:19:22.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264957+0000", 2024-09-17T21:19:22.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0625999999999999e-05 2024-09-17T21:19:22.140 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.140 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.140 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.141 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:445 3.13 3:c9079692:::rbd_data.11527824718.0000000000000000:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.141 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:19:22.141 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.840442850000002, 2024-09-17T21:19:22.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023315144999999999, 2024-09-17T21:19:22.141 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.141 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.141 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.141 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.141 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.141 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.141 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 445 2024-09-17T21:19:22.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.141 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:19:22.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241758+0000", 2024-09-17T21:19:22.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241759+0000", 2024-09-17T21:19:22.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-06 2024-09-17T21:19:22.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241764+0000", 2024-09-17T21:19:22.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5899999999999998e-06 2024-09-17T21:19:22.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241784+0000", 2024-09-17T21:19:22.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9204000000000001e-05 2024-09-17T21:19:22.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.241798+0000", 2024-09-17T21:19:22.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3953e-05 2024-09-17T21:19:22.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.264977+0000", 2024-09-17T21:19:22.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023179182 2024-09-17T21:19:22.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.265018+0000", 2024-09-17T21:19:22.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1171999999999998e-05 2024-09-17T21:19:22.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.265073+0000", 2024-09-17T21:19:22.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4741000000000003e-05 2024-09-17T21:19:22.144 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.145 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.145 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.145 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:679 3.13 3:cd0d74f9:::rbd_data.11527824718.000000000000000a:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.145 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:19:22.145 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.831521428999999, 2024-09-17T21:19:22.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023284978000000001, 2024-09-17T21:19:22.145 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.145 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.145 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.145 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.145 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.146 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.146 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 679 2024-09-17T21:19:22.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.146 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:19:22.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250679+0000", 2024-09-17T21:19:22.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250680+0000", 2024-09-17T21:19:22.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:19:22.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250685+0000", 2024-09-17T21:19:22.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1970000000000004e-06 2024-09-17T21:19:22.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250703+0000", 2024-09-17T21:19:22.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7396000000000001e-05 2024-09-17T21:19:22.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250716+0000", 2024-09-17T21:19:22.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.29e-05 2024-09-17T21:19:22.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273740+0000", 2024-09-17T21:19:22.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023024405000000001 2024-09-17T21:19:22.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273834+0000", 2024-09-17T21:19:22.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4124999999999998e-05 2024-09-17T21:19:22.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.273964+0000", 2024-09-17T21:19:22.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012963499999999999 2024-09-17T21:19:22.149 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.149 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.149 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.149 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:680 3.13 3:cd0d74f9:::rbd_data.11527824718.000000000000000a:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.149 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:19:22.149 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.831466349000003, 2024-09-17T21:19:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023687792999999999, 2024-09-17T21:19:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 680 2024-09-17T21:19:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:19:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250734+0000", 2024-09-17T21:19:22.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250735+0000", 2024-09-17T21:19:22.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2669999999999999e-06 2024-09-17T21:19:22.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250740+0000", 2024-09-17T21:19:22.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8060000000000002e-06 2024-09-17T21:19:22.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250757+0000", 2024-09-17T21:19:22.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7303e-05 2024-09-17T21:19:22.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250771+0000", 2024-09-17T21:19:22.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3876999999999999e-05 2024-09-17T21:19:22.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274022+0000", 2024-09-17T21:19:22.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023251135999999999 2024-09-17T21:19:22.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274211+0000", 2024-09-17T21:19:22.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018813599999999999 2024-09-17T21:19:22.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.274422+0000", 2024-09-17T21:19:22.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000211268 2024-09-17T21:19:22.153 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.153 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.153 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.154 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:22.154 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:22.154 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.802187795000002, 2024-09-17T21:19:22.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:19:22.154 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.154 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.154 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.154 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.154 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:22.154 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:22.154 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:19:22.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.154 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:22.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:22.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:19:22.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:19:22.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:19:22.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:19:22.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:19:22.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:19:22.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:19:22.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:19:22.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:19:22.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:19:22.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:22.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:19:22.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:19:22.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:19:22.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:19:22.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:22.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:19:22.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:19:22.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:19:22.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:19:22.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:19:22.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:19:22.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:19:22.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:19:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.793096083999998, 2024-09-17T21:19:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:19:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:19:22.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.160 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:22.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:22.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:19:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:19:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:19:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:19:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:19:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:19:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:19:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:19:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:19:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:19:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:19:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:19:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:19:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:19:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:19:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:19:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:19:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:19:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.792924032999998, 2024-09-17T21:19:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:19:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:19:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:19:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:19:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:19:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:19:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:19:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:19:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:19:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:19:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:19:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:19:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:19:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:19:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:19:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:19:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:19:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:19:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:19:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:19:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.617991285999999, 2024-09-17T21:19:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:19:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:19:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:19:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:19:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:19:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:19:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:19:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:19:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:19:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:19:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:19:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:19:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:19:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:19:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:19:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:19:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:19:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:19:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:19:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:19:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.617906728999998, 2024-09-17T21:19:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:19:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:19:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:19:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:19:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:19:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:19:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:19:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:19:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:19:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:19:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:19:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:19:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:19:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:19:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:19:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:19:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:19:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:19:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:19:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.617841091999999, 2024-09-17T21:19:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:19:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:19:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:19:22.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:19:22.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:19:22.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:19:22.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:19:22.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:19:22.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:19:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:19:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:19:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:19:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:19:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:19:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:19:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:19:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:19:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:19:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:19:22.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:19:22.183 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.183 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.183 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.183 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.183 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.183 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:22.184 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-17T21:19:22.185 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:19:22.185 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:22.180+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:22.285 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:22.282+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:22.366 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:22.366 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:22.366 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:19:22.366 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:22.382 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:19:22.431 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:22.428+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:22.486 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:22.484+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:22.504 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:22.504 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:22.504 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:22.504 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:22.504 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:22.515 INFO:teuthology.orchestra.run.smithi083.stderr:set osd.3 primary-affinity to 0.756164 (8495552) 2024-09-17T21:19:22.520 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:19:22.522 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-17T21:19:22.593 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:22.590+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:22.598 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:19:22.687 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:22.682+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:22.695 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:22.696 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:22.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:22.696 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:22.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.696 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:478 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 720896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.696 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:19:22.696 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.458552775999998, 2024-09-17T21:19:22.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016845300000000001, 2024-09-17T21:19:22.696 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.696 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.696 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.696 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.696 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.696 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.697 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 478 2024-09-17T21:19:22.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.697 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:19:22.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:19:22.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233785+0000", 2024-09-17T21:19:22.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.116e-06 2024-09-17T21:19:22.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233803+0000", 2024-09-17T21:19:22.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7932999999999999e-05 2024-09-17T21:19:22.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233822+0000", 2024-09-17T21:19:22.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9276999999999999e-05 2024-09-17T21:19:22.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233836+0000", 2024-09-17T21:19:22.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4053000000000001e-05 2024-09-17T21:19:22.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.248579+0000", 2024-09-17T21:19:22.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014743529 2024-09-17T21:19:22.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.248795+0000", 2024-09-17T21:19:22.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021535200000000001 2024-09-17T21:19:22.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250629+0000", 2024-09-17T21:19:22.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018340400000000001 2024-09-17T21:19:22.700 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.700 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.700 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.700 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:479 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 851968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.700 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:19:22.700 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.45846169, 2024-09-17T21:19:22.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017383900000000001, 2024-09-17T21:19:22.700 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.701 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.701 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.701 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.701 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.701 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.701 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 479 2024-09-17T21:19:22.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.701 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:19:22.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:19:22.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233876+0000", 2024-09-17T21:19:22.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2160000000000001e-06 2024-09-17T21:19:22.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233883+0000", 2024-09-17T21:19:22.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9199999999999998e-06 2024-09-17T21:19:22.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233900+0000", 2024-09-17T21:19:22.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7034e-05 2024-09-17T21:19:22.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233913+0000", 2024-09-17T21:19:22.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159000000000001e-05 2024-09-17T21:19:22.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250712+0000", 2024-09-17T21:19:22.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016798644000000001 2024-09-17T21:19:22.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250913+0000", 2024-09-17T21:19:22.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020127900000000001 2024-09-17T21:19:22.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251259+0000", 2024-09-17T21:19:22.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00034564799999999999 2024-09-17T21:19:22.704 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.704 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.704 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.704 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:480 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.704 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:19:22.705 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.458368845000003, 2024-09-17T21:19:22.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017902482000000001, 2024-09-17T21:19:22.705 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.705 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.705 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.705 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.705 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.705 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.705 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 480 2024-09-17T21:19:22.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.705 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:19:22.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:19:22.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233969+0000", 2024-09-17T21:19:22.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.454e-06 2024-09-17T21:19:22.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233981+0000", 2024-09-17T21:19:22.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1593000000000001e-05 2024-09-17T21:19:22.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233999+0000", 2024-09-17T21:19:22.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.806e-05 2024-09-17T21:19:22.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234012+0000", 2024-09-17T21:19:22.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:19:22.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251339+0000", 2024-09-17T21:19:22.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017326352 2024-09-17T21:19:22.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251534+0000", 2024-09-17T21:19:22.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000195842 2024-09-17T21:19:22.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251870+0000", 2024-09-17T21:19:22.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033551400000000003 2024-09-17T21:19:22.708 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.708 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.708 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.708 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:481 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1114112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.709 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:19:22.709 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.458300477999998, 2024-09-17T21:19:22.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018442621999999999, 2024-09-17T21:19:22.709 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.709 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.709 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.709 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.709 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.709 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.709 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 481 2024-09-17T21:19:22.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.709 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:19:22.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:19:22.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234037+0000", 2024-09-17T21:19:22.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:19:22.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234043+0000", 2024-09-17T21:19:22.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.49e-06 2024-09-17T21:19:22.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234061+0000", 2024-09-17T21:19:22.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8416000000000001e-05 2024-09-17T21:19:22.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234076+0000", 2024-09-17T21:19:22.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.456e-05 2024-09-17T21:19:22.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251948+0000", 2024-09-17T21:19:22.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017872774000000001 2024-09-17T21:19:22.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252144+0000", 2024-09-17T21:19:22.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019517300000000001 2024-09-17T21:19:22.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252478+0000", 2024-09-17T21:19:22.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033482899999999998 2024-09-17T21:19:22.712 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.712 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.713 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.713 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:482 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.713 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:19:22.713 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.458241925999999, 2024-09-17T21:19:22.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018988026000000002, 2024-09-17T21:19:22.713 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.713 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.713 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.713 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.713 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.713 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.713 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 482 2024-09-17T21:19:22.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.714 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:19:22.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:19:22.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234096+0000", 2024-09-17T21:19:22.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2139999999999999e-06 2024-09-17T21:19:22.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234101+0000", 2024-09-17T21:19:22.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7100000000000004e-06 2024-09-17T21:19:22.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234118+0000", 2024-09-17T21:19:22.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6486000000000001e-05 2024-09-17T21:19:22.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234131+0000", 2024-09-17T21:19:22.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2824e-05 2024-09-17T21:19:22.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252552+0000", 2024-09-17T21:19:22.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018421549999999998 2024-09-17T21:19:22.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252746+0000", 2024-09-17T21:19:22.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019424399999999999 2024-09-17T21:19:22.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253082+0000", 2024-09-17T21:19:22.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000335998 2024-09-17T21:19:22.717 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.717 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.717 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.717 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:483 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.717 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:19:22.717 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.458187305999999, 2024-09-17T21:19:22.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019462882000000001, 2024-09-17T21:19:22.718 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.718 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.718 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.718 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.718 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.718 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.718 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 483 2024-09-17T21:19:22.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.718 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:19:22.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:19:22.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234150+0000", 2024-09-17T21:19:22.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.127e-06 2024-09-17T21:19:22.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234155+0000", 2024-09-17T21:19:22.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:19:22.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234171+0000", 2024-09-17T21:19:22.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6160000000000001e-05 2024-09-17T21:19:22.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234185+0000", 2024-09-17T21:19:22.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3370000000000001e-05 2024-09-17T21:19:22.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253158+0000", 2024-09-17T21:19:22.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018972968999999999 2024-09-17T21:19:22.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253349+0000", 2024-09-17T21:19:22.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000190733 2024-09-17T21:19:22.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253612+0000", 2024-09-17T21:19:22.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026337000000000002 2024-09-17T21:19:22.721 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.721 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.721 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.722 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:484 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.722 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:19:22.722 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.458133279999998, 2024-09-17T21:19:22.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020029261999999999, 2024-09-17T21:19:22.722 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.722 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.722 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.722 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.722 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.722 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.722 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 484 2024-09-17T21:19:22.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.722 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:19:22.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:19:22.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234205+0000", 2024-09-17T21:19:22.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6470000000000001e-06 2024-09-17T21:19:22.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234210+0000", 2024-09-17T21:19:22.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:19:22.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234227+0000", 2024-09-17T21:19:22.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.73e-05 2024-09-17T21:19:22.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234241+0000", 2024-09-17T21:19:22.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3757000000000001e-05 2024-09-17T21:19:22.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253690+0000", 2024-09-17T21:19:22.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019449368000000002 2024-09-17T21:19:22.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253898+0000", 2024-09-17T21:19:22.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000207161 2024-09-17T21:19:22.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254232+0000", 2024-09-17T21:19:22.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033474299999999998 2024-09-17T21:19:22.726 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.726 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.726 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.726 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:485 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1638400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.726 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:19:22.726 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.458076052999999, 2024-09-17T21:19:22.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020553914, 2024-09-17T21:19:22.726 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.726 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.726 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.726 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.727 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.727 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.727 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 485 2024-09-17T21:19:22.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.727 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:19:22.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:19:22.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234261+0000", 2024-09-17T21:19:22.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.136e-06 2024-09-17T21:19:22.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234266+0000", 2024-09-17T21:19:22.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0669999999999999e-06 2024-09-17T21:19:22.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234284+0000", 2024-09-17T21:19:22.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7839999999999999e-05 2024-09-17T21:19:22.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234298+0000", 2024-09-17T21:19:22.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3852999999999999e-05 2024-09-17T21:19:22.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254311+0000", 2024-09-17T21:19:22.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020012973 2024-09-17T21:19:22.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254552+0000", 2024-09-17T21:19:22.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024116400000000001 2024-09-17T21:19:22.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254814+0000", 2024-09-17T21:19:22.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000261881 2024-09-17T21:19:22.730 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.730 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.730 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.731 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:486 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.731 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:19:22.731 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.458019663999998, 2024-09-17T21:19:22.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021010240999999999, 2024-09-17T21:19:22.731 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.731 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.731 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.731 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.731 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.731 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.731 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 486 2024-09-17T21:19:22.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.731 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:19:22.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:19:22.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234318+0000", 2024-09-17T21:19:22.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.37e-06 2024-09-17T21:19:22.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234324+0000", 2024-09-17T21:19:22.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:19:22.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234341+0000", 2024-09-17T21:19:22.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7127000000000001e-05 2024-09-17T21:19:22.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234354+0000", 2024-09-17T21:19:22.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3210000000000001e-05 2024-09-17T21:19:22.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254896+0000", 2024-09-17T21:19:22.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020541835000000001 2024-09-17T21:19:22.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255092+0000", 2024-09-17T21:19:22.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019634700000000001 2024-09-17T21:19:22.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255327+0000", 2024-09-17T21:19:22.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023475899999999999 2024-09-17T21:19:22.735 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.735 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.735 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.735 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:487 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.735 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:19:22.735 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.457963468000003, 2024-09-17T21:19:22.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021453911999999999, 2024-09-17T21:19:22.735 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.735 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.736 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.736 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.736 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.736 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.736 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 487 2024-09-17T21:19:22.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.736 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:19:22.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:19:22.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234374+0000", 2024-09-17T21:19:22.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3039999999999999e-06 2024-09-17T21:19:22.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234380+0000", 2024-09-17T21:19:22.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:19:22.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234397+0000", 2024-09-17T21:19:22.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7232999999999999e-05 2024-09-17T21:19:22.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234411+0000", 2024-09-17T21:19:22.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3784000000000001e-05 2024-09-17T21:19:22.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255406+0000", 2024-09-17T21:19:22.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020995204 2024-09-17T21:19:22.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255595+0000", 2024-09-17T21:19:22.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018903299999999999 2024-09-17T21:19:22.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255827+0000", 2024-09-17T21:19:22.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023190099999999999 2024-09-17T21:19:22.739 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.739 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.739 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:488 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:19:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.457907790999997, 2024-09-17T21:19:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026887966999999999, 2024-09-17T21:19:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 488 2024-09-17T21:19:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:19:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:19:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234430+0000", 2024-09-17T21:19:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1829999999999999e-06 2024-09-17T21:19:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234435+0000", 2024-09-17T21:19:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9869999999999999e-06 2024-09-17T21:19:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234451+0000", 2024-09-17T21:19:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6458999999999999e-05 2024-09-17T21:19:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234465+0000", 2024-09-17T21:19:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3730000000000001e-05 2024-09-17T21:19:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255908+0000", 2024-09-17T21:19:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021443413000000001 2024-09-17T21:19:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256098+0000", 2024-09-17T21:19:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018961 2024-09-17T21:19:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261317+0000", 2024-09-17T21:19:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052185850000000004 2024-09-17T21:19:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:489 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2162688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:19:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.457851951999999, 2024-09-17T21:19:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022632815000000001, 2024-09-17T21:19:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 489 2024-09-17T21:19:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:19:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:19:22.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234486+0000", 2024-09-17T21:19:22.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5099999999999999e-06 2024-09-17T21:19:22.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234491+0000", 2024-09-17T21:19:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.147e-06 2024-09-17T21:19:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234508+0000", 2024-09-17T21:19:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7405999999999999e-05 2024-09-17T21:19:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234522+0000", 2024-09-17T21:19:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3677e-05 2024-09-17T21:19:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256536+0000", 2024-09-17T21:19:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022013464999999999 2024-09-17T21:19:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256752+0000", 2024-09-17T21:19:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021637800000000001 2024-09-17T21:19:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257117+0000", 2024-09-17T21:19:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000365232 2024-09-17T21:19:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:490 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2293760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:19:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.457797161999999, 2024-09-17T21:19:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023146462999999999, 2024-09-17T21:19:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 490 2024-09-17T21:19:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:19:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:19:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234541+0000", 2024-09-17T21:19:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4869999999999999e-06 2024-09-17T21:19:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234559+0000", 2024-09-17T21:19:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8105999999999999e-05 2024-09-17T21:19:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234577+0000", 2024-09-17T21:19:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.836e-05 2024-09-17T21:19:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234592+0000", 2024-09-17T21:19:22.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4766000000000001e-05 2024-09-17T21:19:22.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257187+0000", 2024-09-17T21:19:22.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022594725 2024-09-17T21:19:22.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257374+0000", 2024-09-17T21:19:22.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018767299999999999 2024-09-17T21:19:22.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257686+0000", 2024-09-17T21:19:22.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000311346 2024-09-17T21:19:22.753 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.753 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.753 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.753 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:491 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.753 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:19:22.754 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.457724552999998, 2024-09-17T21:19:22.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023696872000000001, 2024-09-17T21:19:22.754 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.754 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.754 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.754 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.754 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.754 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.754 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 491 2024-09-17T21:19:22.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.754 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:19:22.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:19:22.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234613+0000", 2024-09-17T21:19:22.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-06 2024-09-17T21:19:22.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234618+0000", 2024-09-17T21:19:22.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3839999999999996e-06 2024-09-17T21:19:22.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234635+0000", 2024-09-17T21:19:22.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.685e-05 2024-09-17T21:19:22.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234648+0000", 2024-09-17T21:19:22.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.313e-05 2024-09-17T21:19:22.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257780+0000", 2024-09-17T21:19:22.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023131946 2024-09-17T21:19:22.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257985+0000", 2024-09-17T21:19:22.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020468400000000001 2024-09-17T21:19:22.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258309+0000", 2024-09-17T21:19:22.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032357499999999999 2024-09-17T21:19:22.758 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.758 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.758 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.758 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:492 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.758 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:19:22.758 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.457668966, 2024-09-17T21:19:22.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024258994999999998, 2024-09-17T21:19:22.758 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.758 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.759 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.759 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.759 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.759 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.759 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 492 2024-09-17T21:19:22.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.759 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:19:22.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:19:22.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234669+0000", 2024-09-17T21:19:22.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.313e-06 2024-09-17T21:19:22.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234674+0000", 2024-09-17T21:19:22.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1599999999999997e-06 2024-09-17T21:19:22.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234690+0000", 2024-09-17T21:19:22.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.666e-05 2024-09-17T21:19:22.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234704+0000", 2024-09-17T21:19:22.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3043e-05 2024-09-17T21:19:22.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258397+0000", 2024-09-17T21:19:22.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023693204999999998 2024-09-17T21:19:22.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258593+0000", 2024-09-17T21:19:22.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019578999999999999 2024-09-17T21:19:22.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258926+0000", 2024-09-17T21:19:22.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033382399999999998 2024-09-17T21:19:22.762 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.762 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.762 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.763 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:493 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.763 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:19:22.763 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.457612501, 2024-09-17T21:19:22.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024793461999999999, 2024-09-17T21:19:22.763 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.763 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.763 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.763 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.763 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.763 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.763 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 493 2024-09-17T21:19:22.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.763 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:19:22.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:19:22.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234725+0000", 2024-09-17T21:19:22.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.24e-06 2024-09-17T21:19:22.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234731+0000", 2024-09-17T21:19:22.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8939999999999998e-06 2024-09-17T21:19:22.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234749+0000", 2024-09-17T21:19:22.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7863000000000002e-05 2024-09-17T21:19:22.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234762+0000", 2024-09-17T21:19:22.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3322999999999999e-05 2024-09-17T21:19:22.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259022+0000", 2024-09-17T21:19:22.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024259691999999999 2024-09-17T21:19:22.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259259+0000", 2024-09-17T21:19:22.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023669199999999999 2024-09-17T21:19:22.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259517+0000", 2024-09-17T21:19:22.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025875799999999999 2024-09-17T21:19:22.767 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.767 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.767 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.767 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:494 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.767 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:19:22.767 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.457554801000001, 2024-09-17T21:19:22.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025251952000000001, 2024-09-17T21:19:22.767 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.768 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.768 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.768 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.768 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.768 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.768 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 494 2024-09-17T21:19:22.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.768 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:19:22.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:19:22.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234783+0000", 2024-09-17T21:19:22.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2440000000000001e-06 2024-09-17T21:19:22.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234789+0000", 2024-09-17T21:19:22.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:19:22.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234806+0000", 2024-09-17T21:19:22.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7450000000000001e-05 2024-09-17T21:19:22.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234819+0000", 2024-09-17T21:19:22.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3326000000000001e-05 2024-09-17T21:19:22.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259585+0000", 2024-09-17T21:19:22.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024765579999999999 2024-09-17T21:19:22.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259787+0000", 2024-09-17T21:19:22.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202444 2024-09-17T21:19:22.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260033+0000", 2024-09-17T21:19:22.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024611499999999998 2024-09-17T21:19:22.772 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.772 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.772 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.772 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:495 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.772 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:19:22.772 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.457499437999999, 2024-09-17T21:19:22.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025776342000000001, 2024-09-17T21:19:22.772 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.772 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.772 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.772 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.773 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.773 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.773 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 495 2024-09-17T21:19:22.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.773 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:19:22.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:19:22.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234838+0000", 2024-09-17T21:19:22.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2130000000000001e-06 2024-09-17T21:19:22.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234843+0000", 2024-09-17T21:19:22.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9899999999999997e-06 2024-09-17T21:19:22.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234860+0000", 2024-09-17T21:19:22.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6973000000000002e-05 2024-09-17T21:19:22.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234873+0000", 2024-09-17T21:19:22.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3179999999999999e-05 2024-09-17T21:19:22.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260135+0000", 2024-09-17T21:19:22.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025262118 2024-09-17T21:19:22.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260364+0000", 2024-09-17T21:19:22.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022830699999999999 2024-09-17T21:19:22.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260613+0000", 2024-09-17T21:19:22.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024956099999999998 2024-09-17T21:19:22.776 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.776 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.777 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.777 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:496 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.777 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:19:22.777 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.457446158000003, 2024-09-17T21:19:22.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026376773999999999, 2024-09-17T21:19:22.777 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.777 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.777 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:22.777 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.777 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.777 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.778 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 496 2024-09-17T21:19:22.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.778 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:19:22.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:19:22.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234891+0000", 2024-09-17T21:19:22.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1230000000000001e-06 2024-09-17T21:19:22.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234896+0000", 2024-09-17T21:19:22.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9599999999999999e-06 2024-09-17T21:19:22.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234913+0000", 2024-09-17T21:19:22.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6747000000000001e-05 2024-09-17T21:19:22.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234926+0000", 2024-09-17T21:19:22.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-05 2024-09-17T21:19:22.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260675+0000", 2024-09-17T21:19:22.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025748979000000002 2024-09-17T21:19:22.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260883+0000", 2024-09-17T21:19:22.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020848399999999999 2024-09-17T21:19:22.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261267+0000", 2024-09-17T21:19:22.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038345099999999998 2024-09-17T21:19:22.781 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.781 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.781 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.781 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:682 3.9 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2455552~12288,write 8126464~10240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:22.781 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:19:22.782 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.423509107000001, 2024-09-17T21:19:22.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018447261999999999, 2024-09-17T21:19:22.782 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:22.782 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:22.782 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:22.782 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:22.782 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:22.782 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:22.782 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 682 2024-09-17T21:19:22.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.782 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:22.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:22.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:19:22.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:22.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:19:22.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:22.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:22.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268830+0000", 2024-09-17T21:19:22.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6419999999999999e-06 2024-09-17T21:19:22.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:22.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271673+0000", 2024-09-17T21:19:22.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028432610000000001 2024-09-17T21:19:22.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:22.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271700+0000", 2024-09-17T21:19:22.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7200999999999999e-05 2024-09-17T21:19:22.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:22.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271715+0000", 2024-09-17T21:19:22.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5145e-05 2024-09-17T21:19:22.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:22.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271785+0000", 2024-09-17T21:19:22.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9519999999999998e-05 2024-09-17T21:19:22.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:22.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271918+0000", 2024-09-17T21:19:22.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013330899999999999 2024-09-17T21:19:22.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:22.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.272449+0000", 2024-09-17T21:19:22.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00053051200000000002 2024-09-17T21:19:22.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:22.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.278822+0000", 2024-09-17T21:19:22.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0063731619999999999 2024-09-17T21:19:22.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:22.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287167+0000", 2024-09-17T21:19:22.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083452149999999996 2024-09-17T21:19:22.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:22.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287213+0000", 2024-09-17T21:19:22.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5972e-05 2024-09-17T21:19:22.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:22.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:22.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:22.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287274+0000", 2024-09-17T21:19:22.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1322999999999995e-05 2024-09-17T21:19:22.787 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.787 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.787 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.787 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:22.787 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:22.787 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:22.788 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:22.789 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:22.789 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:22.790 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:22.790 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:19:22.791 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:22.791 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:22.791 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:22.786+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:22.795 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:22.795 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:22.815 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:22.815 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:22.832 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:22.833 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:22.897 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:22.894+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:22.914 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:22.914 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:22.928 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:19:22.990 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:22.988+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:23.056 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:23.056 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:23.071 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:19:23.118 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:23.116+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:23.192 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:23.188+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:23.241 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:23.241 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:23.256 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:19:23.301 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:23.298+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:23.363 INFO:teuthology.orchestra.run.smithi184.stdout:3058 write 0xa59c600 thru 0xa5a6985 (0xa386 bytes) 2024-09-17T21:19:23.392 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:23.392 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:23.393 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:23.390+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:23.395 INFO:teuthology.orchestra.run.smithi184.stdout:3060 write 0xa85d800 thru 0xa85e509 (0xd0a bytes) 2024-09-17T21:19:23.396 INFO:teuthology.orchestra.run.smithi184.stdout:3062 write 0x1c75400 thru 0x1c83413 (0xe014 bytes) 2024-09-17T21:19:23.407 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:19:23.450 INFO:teuthology.orchestra.run.smithi184.stdout:3063 write 0xe4f3600 thru 0xe4f6a7a (0x347b bytes) 2024-09-17T21:19:23.453 INFO:teuthology.orchestra.run.smithi184.stdout:3064 write 0xce1f600 thru 0xce2f296 (0xfc97 bytes) 2024-09-17T21:19:23.453 INFO:teuthology.orchestra.run.smithi184.stdout:3065 read 0x106be00 thru 0x10703ec (0x45ed bytes) 2024-09-17T21:19:23.461 INFO:teuthology.orchestra.run.smithi184.stdout:3070 trunc from 0xe4f6c00 to 0xa401200 2024-09-17T21:19:23.494 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:23.492+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:23.517 INFO:teuthology.orchestra.run.smithi184.stdout:3071 trunc from 0xa401200 to 0x3e15200 2024-09-17T21:19:23.569 INFO:teuthology.orchestra.run.smithi184.stdout:3072 write 0x528a000 thru 0x5291787 (0x7788 bytes) 2024-09-17T21:19:23.570 INFO:teuthology.orchestra.run.smithi184.stdout:3074 punch from 0x4045600 to 0x4048e00, (0x3800 bytes) 2024-09-17T21:19:23.586 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:23.586 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:23.595 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:23.590+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:23.602 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:19:23.607 INFO:teuthology.orchestra.run.smithi184.stdout:3079 write 0xd011400 thru 0xd01b33f (0x9f40 bytes) 2024-09-17T21:19:23.610 INFO:teuthology.orchestra.run.smithi184.stdout:3080 write 0xb304600 thru 0xb30ddc4 (0x97c5 bytes) 2024-09-17T21:19:23.623 INFO:teuthology.orchestra.run.smithi184.stdout:3081 trunc from 0xd01b400 to 0x7df3400 2024-09-17T21:19:23.679 INFO:teuthology.orchestra.run.smithi184.stdout:3082 trunc from 0x7df3400 to 0x24efc00 2024-09-17T21:19:23.697 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:23.692+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:23.713 INFO:teuthology.orchestra.run.smithi184.stdout:3083 punch from 0x7eca00 to 0x7f8e00, (0xc400 bytes) 2024-09-17T21:19:23.737 INFO:teuthology.orchestra.run.smithi184.stdout:3085 trunc from 0x24efc00 to 0xbdad000 2024-09-17T21:19:23.759 INFO:teuthology.orchestra.run.smithi184.stdout:3086 read 0x673a400 thru 0x6749f23 (0xfb24 bytes) 2024-09-17T21:19:23.760 INFO:teuthology.orchestra.run.smithi184.stdout:3088 write 0x968f600 thru 0x969edf9 (0xf7fa bytes) 2024-09-17T21:19:23.761 INFO:teuthology.orchestra.run.smithi184.stdout:3089 write 0x50ffc00 thru 0x510a665 (0xaa66 bytes) 2024-09-17T21:19:23.761 INFO:teuthology.orchestra.run.smithi184.stdout:3093 write 0xa1b4200 thru 0xa1c1e2f (0xdc30 bytes) 2024-09-17T21:19:23.761 INFO:teuthology.orchestra.run.smithi184.stdout:3094 punch from 0xb657200 to 0xb660e00, (0x9c00 bytes) 2024-09-17T21:19:23.762 INFO:teuthology.orchestra.run.smithi184.stdout:3095 write 0x1117c00 thru 0x111d568 (0x5969 bytes) 2024-09-17T21:19:23.776 INFO:teuthology.orchestra.run.smithi184.stdout:3096 trunc from 0xbdad000 to 0x3fb2600 2024-09-17T21:19:23.779 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:23.779 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:23.795 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:19:23.797 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:23.794+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:23.850 INFO:teuthology.orchestra.run.smithi184.stdout:3097 write 0x9ebaa00 thru 0x9ebcaef (0x20f0 bytes) 2024-09-17T21:19:23.892 INFO:teuthology.orchestra.run.smithi184.stdout:3098 write 0xddedc00 thru 0xddeedcc (0x11cd bytes) 2024-09-17T21:19:23.893 INFO:teuthology.orchestra.run.smithi184.stdout:3100 punch from 0x572a000 to 0x5733200, (0x9200 bytes) 2024-09-17T21:19:23.897 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:23.892+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:23.915 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:23.915 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:23.929 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:19:23.931 INFO:teuthology.orchestra.run.smithi184.stdout:3104 trunc from 0xddeee00 to 0x1021000 2024-09-17T21:19:23.998 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:23.994+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:24.015 INFO:teuthology.orchestra.run.smithi184.stdout:3105 trunc from 0x1021000 to 0xeac3600 2024-09-17T21:19:24.032 INFO:teuthology.orchestra.run.smithi184.stdout:3109 write 0x202000 thru 0x20d235 (0xb236 bytes) 2024-09-17T21:19:24.034 INFO:teuthology.orchestra.run.smithi184.stdout:3110 write 0xa529400 thru 0xa536b9d (0xd79e bytes) 2024-09-17T21:19:24.034 INFO:teuthology.orchestra.run.smithi184.stdout:3111 read 0xb28ba00 thru 0xb28d1bc (0x17bd bytes) 2024-09-17T21:19:24.034 INFO:teuthology.orchestra.run.smithi184.stdout:3112 write 0x2cdc400 thru 0x2ce89fd (0xc5fe bytes) 2024-09-17T21:19:24.034 INFO:teuthology.orchestra.run.smithi184.stdout:3113 read 0xb588000 thru 0xb589420 (0x1421 bytes) 2024-09-17T21:19:24.034 INFO:teuthology.orchestra.run.smithi184.stdout:3116 read 0xbd22200 thru 0xbd2e311 (0xc112 bytes) 2024-09-17T21:19:24.035 INFO:teuthology.orchestra.run.smithi184.stdout:3118 read 0xd18c600 thru 0xd194a35 (0x8436 bytes) 2024-09-17T21:19:24.035 INFO:teuthology.orchestra.run.smithi184.stdout:3120 write 0x6f6b200 thru 0x6f6e682 (0x3483 bytes) 2024-09-17T21:19:24.035 INFO:teuthology.orchestra.run.smithi184.stdout:3122 punch from 0x38a9a00 to 0x38ad000, (0x3600 bytes) 2024-09-17T21:19:24.036 INFO:teuthology.orchestra.run.smithi184.stdout:3123 write 0xca5f200 thru 0xca698ce (0xa6cf bytes) 2024-09-17T21:19:24.036 INFO:teuthology.orchestra.run.smithi184.stdout:3125 write 0x1390000 thru 0x139e19c (0xe19d bytes) 2024-09-17T21:19:24.057 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:24.057 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:24.059 INFO:teuthology.orchestra.run.smithi184.stdout:3126 trunc from 0xeac3600 to 0x2245a00 2024-09-17T21:19:24.073 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:19:24.101 INFO:teuthology.orchestra.run.smithi184.stdout:3134 punch from 0x4b3800 to 0x4bba00, (0x8200 bytes) 2024-09-17T21:19:24.105 INFO:teuthology.orchestra.run.smithi184.stdout:3135 punch from 0x1a1d800 to 0x1a1dc00, (0x400 bytes) 2024-09-17T21:19:24.117 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:24.114+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:24.122 INFO:teuthology.orchestra.run.smithi184.stdout:3137 trunc from 0x2245a00 to 0xaa57000 2024-09-17T21:19:24.141 INFO:teuthology.orchestra.run.smithi184.stdout:3139 punch from 0x38bd800 to 0x38c2c00, (0x5400 bytes) 2024-09-17T21:19:24.152 INFO:teuthology.orchestra.run.smithi184.stdout:3141 trunc from 0xaa57000 to 0x4f8f400 2024-09-17T21:19:24.199 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:24.196+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:24.204 INFO:teuthology.orchestra.run.smithi184.stdout:3142 write 0xeced800 thru 0xecfb079 (0xd87a bytes) 2024-09-17T21:19:24.238 INFO:teuthology.orchestra.run.smithi184.stdout:3143 write 0xee0b200 thru 0xee17e29 (0xcc2a bytes) 2024-09-17T21:19:24.244 INFO:teuthology.orchestra.run.smithi184.stdout:3147 write 0x694ae00 thru 0x6952d24 (0x7f25 bytes) 2024-09-17T21:19:24.244 INFO:teuthology.orchestra.run.smithi184.stdout:3149 punch from 0x6d41c00 to 0x6d42a00, (0xe00 bytes) 2024-09-17T21:19:24.245 INFO:teuthology.orchestra.run.smithi184.stdout:3152 write 0xda4a00 thru 0xdaf608 (0xac09 bytes) 2024-09-17T21:19:24.245 INFO:teuthology.orchestra.run.smithi184.stdout:3154 read 0xa666a00 thru 0xa66b7cc (0x4dcd bytes) 2024-09-17T21:19:24.245 INFO:teuthology.orchestra.run.smithi184.stdout:3156 write 0x88db400 thru 0x88e4a0e (0x960f bytes) 2024-09-17T21:19:24.245 INFO:teuthology.orchestra.run.smithi184.stdout:3157 write 0x2568c00 thru 0x25728af (0x9cb0 bytes) 2024-09-17T21:19:24.246 INFO:teuthology.orchestra.run.smithi184.stdout:3158 write 0x4bf3c00 thru 0x4bf826e (0x466f bytes) 2024-09-17T21:19:24.246 INFO:teuthology.orchestra.run.smithi184.stdout:3164 punch from 0x8827c00 to 0x8834600, (0xca00 bytes) 2024-09-17T21:19:24.246 INFO:teuthology.orchestra.run.smithi184.stdout:3165 read 0x5a5fe00 thru 0x5a618fe (0x1aff bytes) 2024-09-17T21:19:24.248 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:24.248 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:24.263 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:19:24.278 INFO:teuthology.orchestra.run.smithi184.stdout:3166 trunc from 0xee18000 to 0x492a00 2024-09-17T21:19:24.308 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:24.306+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:24.353 INFO:teuthology.orchestra.run.smithi184.stdout:3169 trunc from 0x492a00 to 0x86a00 2024-09-17T21:19:24.400 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:24.396+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:24.401 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:24.401 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:24.416 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:19:24.475 INFO:teuthology.orchestra.run.smithi184.stdout:3170 write 0xdb73200 thru 0xdb735e9 (0x3ea bytes) 2024-09-17T21:19:24.477 INFO:teuthology.orchestra.run.smithi184.stdout:3173 read 0x8a89000 thru 0x8a8aeec (0x1eed bytes) 2024-09-17T21:19:24.478 INFO:teuthology.orchestra.run.smithi184.stdout:3174 read 0x586e000 thru 0x58751b7 (0x71b8 bytes) 2024-09-17T21:19:24.501 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:24.496+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:24.503 INFO:teuthology.orchestra.run.smithi184.stdout:3175 trunc from 0xdb73600 to 0x3954800 2024-09-17T21:19:24.547 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:19:24.584 INFO:teuthology.orchestra.run.smithi184.stdout:3176 write 0xc431000 thru 0xc4318da (0x8db bytes) 2024-09-17T21:19:24.586 INFO:teuthology.orchestra.run.smithi184.stdout:3177 read 0x4196200 thru 0x419735c (0x115d bytes) 2024-09-17T21:19:24.586 INFO:teuthology.orchestra.run.smithi184.stdout:3180 write 0x6581200 thru 0x6583af6 (0x28f7 bytes) 2024-09-17T21:19:24.586 INFO:teuthology.orchestra.run.smithi184.stdout:3181 read 0x1da0000 thru 0x1da0821 (0x822 bytes) 2024-09-17T21:19:24.586 INFO:teuthology.orchestra.run.smithi184.stdout:3183 write 0xb007600 thru 0xb00c996 (0x5397 bytes) 2024-09-17T21:19:24.589 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:24.589 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:24.591 INFO:teuthology.orchestra.run.smithi184.stdout:3184 trunc from 0xc431a00 to 0x9026000 2024-09-17T21:19:24.601 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:24.598+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:24.604 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:19:24.632 INFO:teuthology.orchestra.run.smithi184.stdout:3186 read 0x309d600 thru 0x30a4d86 (0x7787 bytes) 2024-09-17T21:19:24.634 INFO:teuthology.orchestra.run.smithi184.stdout:3187 read 0x677c00 thru 0x678786 (0xb87 bytes) 2024-09-17T21:19:24.638 INFO:teuthology.orchestra.run.smithi184.stdout:3191 trunc from 0x9026000 to 0xb468e00 2024-09-17T21:19:24.665 INFO:teuthology.orchestra.run.smithi184.stdout:3194 trunc from 0xb468e00 to 0x914ce00 2024-09-17T21:19:24.685 INFO:teuthology.orchestra.run.smithi184.stdout:3196 punch from 0x7a38400 to 0x7a43600, (0xb200 bytes) 2024-09-17T21:19:24.685 INFO:teuthology.orchestra.run.smithi184.stdout:3197 read 0x5ef1a00 thru 0x5effa6c (0xe06d bytes) 2024-09-17T21:19:24.685 INFO:teuthology.orchestra.run.smithi184.stdout:3199 write 0x1fc3a00 thru 0x1fd0209 (0xc80a bytes) 2024-09-17T21:19:24.685 INFO:teuthology.orchestra.run.smithi184.stdout:3201 read 0x3082000 thru 0x308c5e1 (0xa5e2 bytes) 2024-09-17T21:19:24.691 INFO:teuthology.orchestra.run.smithi184.stdout:3203 trunc from 0x914ce00 to 0xba24e00 2024-09-17T21:19:24.702 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:24.698+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:24.713 INFO:teuthology.orchestra.run.smithi184.stdout:3205 read 0xbf3600 thru 0xbfe1b1 (0xabb2 bytes) 2024-09-17T21:19:24.714 INFO:teuthology.orchestra.run.smithi184.stdout:3206 clone 17 order 20 su 8192 sc 12 2024-09-17T21:19:24.773 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:24.773 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:24.787 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:19:24.828 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:24.826+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:24.850 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:19:24.903 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:24.898+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:24.917 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:24.917 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:24.930 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:19:24.991 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:24.992 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:25.004 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:24.998+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:25.047 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:25.048 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:25.062 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:19:25.106 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:25.104+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:25.179 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:25.179 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:25.193 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:19:25.206 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:25.202+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:25.214 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:25.214 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:25.234 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:25.234 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:25.255 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:25.256 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:25.282 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:25.282 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:25.306 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:25.304+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:25.308 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:25.308 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:25.356 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:25.356 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:25.356 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:19:25.356 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:25.370 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:19:25.408 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:25.402+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:25.508 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:25.506+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:25.541 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-17T21:19:25.544 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:25.544 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:25.544 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:25.544 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:25.544 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:25.560 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:19:25.612 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:25.610+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:25.669 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone16 from 0xba24e00 (overlap 0x86a00) to 0x1ae00 2024-09-17T21:19:25.697 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:478 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 720896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:19:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.457710841999997, 2024-09-17T21:19:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016845300000000001, 2024-09-17T21:19:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 478 2024-09-17T21:19:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:19:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233784+0000", 2024-09-17T21:19:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233785+0000", 2024-09-17T21:19:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.116e-06 2024-09-17T21:19:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233803+0000", 2024-09-17T21:19:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7932999999999999e-05 2024-09-17T21:19:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233822+0000", 2024-09-17T21:19:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9276999999999999e-05 2024-09-17T21:19:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233836+0000", 2024-09-17T21:19:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4053000000000001e-05 2024-09-17T21:19:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.248579+0000", 2024-09-17T21:19:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014743529 2024-09-17T21:19:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.248795+0000", 2024-09-17T21:19:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021535200000000001 2024-09-17T21:19:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250629+0000", 2024-09-17T21:19:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018340400000000001 2024-09-17T21:19:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:479 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 851968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:19:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.457619756, 2024-09-17T21:19:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017383900000000001, 2024-09-17T21:19:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 479 2024-09-17T21:19:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:19:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233875+0000", 2024-09-17T21:19:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233876+0000", 2024-09-17T21:19:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2160000000000001e-06 2024-09-17T21:19:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233883+0000", 2024-09-17T21:19:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9199999999999998e-06 2024-09-17T21:19:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233900+0000", 2024-09-17T21:19:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7034e-05 2024-09-17T21:19:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233913+0000", 2024-09-17T21:19:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159000000000001e-05 2024-09-17T21:19:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250712+0000", 2024-09-17T21:19:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016798644000000001 2024-09-17T21:19:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.250913+0000", 2024-09-17T21:19:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020127900000000001 2024-09-17T21:19:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251259+0000", 2024-09-17T21:19:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00034564799999999999 2024-09-17T21:19:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:480 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:19:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.457526911000002, 2024-09-17T21:19:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017902482000000001, 2024-09-17T21:19:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 480 2024-09-17T21:19:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:19:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:19:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233969+0000", 2024-09-17T21:19:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.454e-06 2024-09-17T21:19:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233981+0000", 2024-09-17T21:19:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1593000000000001e-05 2024-09-17T21:19:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233999+0000", 2024-09-17T21:19:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.806e-05 2024-09-17T21:19:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234012+0000", 2024-09-17T21:19:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:19:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251339+0000", 2024-09-17T21:19:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017326352 2024-09-17T21:19:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251534+0000", 2024-09-17T21:19:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000195842 2024-09-17T21:19:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251870+0000", 2024-09-17T21:19:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033551400000000003 2024-09-17T21:19:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:481 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1114112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:19:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.457458543999998, 2024-09-17T21:19:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018442621999999999, 2024-09-17T21:19:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 481 2024-09-17T21:19:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:19:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:19:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234037+0000", 2024-09-17T21:19:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:19:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234043+0000", 2024-09-17T21:19:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.49e-06 2024-09-17T21:19:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234061+0000", 2024-09-17T21:19:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8416000000000001e-05 2024-09-17T21:19:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234076+0000", 2024-09-17T21:19:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.456e-05 2024-09-17T21:19:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251948+0000", 2024-09-17T21:19:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017872774000000001 2024-09-17T21:19:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252144+0000", 2024-09-17T21:19:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019517300000000001 2024-09-17T21:19:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252478+0000", 2024-09-17T21:19:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033482899999999998 2024-09-17T21:19:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:482 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:19:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.457399991999999, 2024-09-17T21:19:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018988026000000002, 2024-09-17T21:19:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 482 2024-09-17T21:19:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:19:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:19:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234096+0000", 2024-09-17T21:19:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2139999999999999e-06 2024-09-17T21:19:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234101+0000", 2024-09-17T21:19:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7100000000000004e-06 2024-09-17T21:19:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234118+0000", 2024-09-17T21:19:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6486000000000001e-05 2024-09-17T21:19:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234131+0000", 2024-09-17T21:19:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2824e-05 2024-09-17T21:19:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252552+0000", 2024-09-17T21:19:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018421549999999998 2024-09-17T21:19:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252746+0000", 2024-09-17T21:19:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019424399999999999 2024-09-17T21:19:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253082+0000", 2024-09-17T21:19:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000335998 2024-09-17T21:19:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:483 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:19:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.457345371999999, 2024-09-17T21:19:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019462882000000001, 2024-09-17T21:19:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 483 2024-09-17T21:19:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:19:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:19:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234150+0000", 2024-09-17T21:19:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.127e-06 2024-09-17T21:19:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234155+0000", 2024-09-17T21:19:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:19:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234171+0000", 2024-09-17T21:19:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6160000000000001e-05 2024-09-17T21:19:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234185+0000", 2024-09-17T21:19:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3370000000000001e-05 2024-09-17T21:19:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253158+0000", 2024-09-17T21:19:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018972968999999999 2024-09-17T21:19:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253349+0000", 2024-09-17T21:19:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000190733 2024-09-17T21:19:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253612+0000", 2024-09-17T21:19:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026337000000000002 2024-09-17T21:19:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:484 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:19:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.457291345999998, 2024-09-17T21:19:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020029261999999999, 2024-09-17T21:19:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 484 2024-09-17T21:19:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:19:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:19:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234205+0000", 2024-09-17T21:19:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6470000000000001e-06 2024-09-17T21:19:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234210+0000", 2024-09-17T21:19:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:19:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234227+0000", 2024-09-17T21:19:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.73e-05 2024-09-17T21:19:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234241+0000", 2024-09-17T21:19:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3757000000000001e-05 2024-09-17T21:19:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253690+0000", 2024-09-17T21:19:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019449368000000002 2024-09-17T21:19:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253898+0000", 2024-09-17T21:19:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000207161 2024-09-17T21:19:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254232+0000", 2024-09-17T21:19:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033474299999999998 2024-09-17T21:19:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:485 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1638400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:19:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.457234118999999, 2024-09-17T21:19:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020553914, 2024-09-17T21:19:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 485 2024-09-17T21:19:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:19:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:19:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234261+0000", 2024-09-17T21:19:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.136e-06 2024-09-17T21:19:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234266+0000", 2024-09-17T21:19:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0669999999999999e-06 2024-09-17T21:19:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234284+0000", 2024-09-17T21:19:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7839999999999999e-05 2024-09-17T21:19:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234298+0000", 2024-09-17T21:19:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3852999999999999e-05 2024-09-17T21:19:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254311+0000", 2024-09-17T21:19:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020012973 2024-09-17T21:19:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254552+0000", 2024-09-17T21:19:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024116400000000001 2024-09-17T21:19:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254814+0000", 2024-09-17T21:19:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000261881 2024-09-17T21:19:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:486 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:19:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.457177729999998, 2024-09-17T21:19:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021010240999999999, 2024-09-17T21:19:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 486 2024-09-17T21:19:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:19:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:19:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234318+0000", 2024-09-17T21:19:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.37e-06 2024-09-17T21:19:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234324+0000", 2024-09-17T21:19:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:19:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234341+0000", 2024-09-17T21:19:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7127000000000001e-05 2024-09-17T21:19:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234354+0000", 2024-09-17T21:19:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3210000000000001e-05 2024-09-17T21:19:25.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:25.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254896+0000", 2024-09-17T21:19:25.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020541835000000001 2024-09-17T21:19:25.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:25.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255092+0000", 2024-09-17T21:19:25.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019634700000000001 2024-09-17T21:19:25.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:25.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255327+0000", 2024-09-17T21:19:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023475899999999999 2024-09-17T21:19:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:487 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:19:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.457121534000002, 2024-09-17T21:19:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021453911999999999, 2024-09-17T21:19:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 487 2024-09-17T21:19:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:19:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:19:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234374+0000", 2024-09-17T21:19:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3039999999999999e-06 2024-09-17T21:19:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234380+0000", 2024-09-17T21:19:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:19:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234397+0000", 2024-09-17T21:19:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7232999999999999e-05 2024-09-17T21:19:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234411+0000", 2024-09-17T21:19:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3784000000000001e-05 2024-09-17T21:19:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255406+0000", 2024-09-17T21:19:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020995204 2024-09-17T21:19:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255595+0000", 2024-09-17T21:19:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018903299999999999 2024-09-17T21:19:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255827+0000", 2024-09-17T21:19:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023190099999999999 2024-09-17T21:19:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:488 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:19:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.457065857000003, 2024-09-17T21:19:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026887966999999999, 2024-09-17T21:19:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 488 2024-09-17T21:19:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:19:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:19:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234430+0000", 2024-09-17T21:19:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1829999999999999e-06 2024-09-17T21:19:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234435+0000", 2024-09-17T21:19:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9869999999999999e-06 2024-09-17T21:19:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234451+0000", 2024-09-17T21:19:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6458999999999999e-05 2024-09-17T21:19:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234465+0000", 2024-09-17T21:19:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3730000000000001e-05 2024-09-17T21:19:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255908+0000", 2024-09-17T21:19:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021443413000000001 2024-09-17T21:19:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256098+0000", 2024-09-17T21:19:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018961 2024-09-17T21:19:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261317+0000", 2024-09-17T21:19:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052185850000000004 2024-09-17T21:19:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:489 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2162688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:19:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.457010017999998, 2024-09-17T21:19:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022632815000000001, 2024-09-17T21:19:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 489 2024-09-17T21:19:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:19:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:19:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234486+0000", 2024-09-17T21:19:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5099999999999999e-06 2024-09-17T21:19:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234491+0000", 2024-09-17T21:19:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.147e-06 2024-09-17T21:19:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234508+0000", 2024-09-17T21:19:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7405999999999999e-05 2024-09-17T21:19:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234522+0000", 2024-09-17T21:19:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3677e-05 2024-09-17T21:19:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256536+0000", 2024-09-17T21:19:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022013464999999999 2024-09-17T21:19:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256752+0000", 2024-09-17T21:19:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021637800000000001 2024-09-17T21:19:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257117+0000", 2024-09-17T21:19:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000365232 2024-09-17T21:19:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:490 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2293760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:19:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.456955227999998, 2024-09-17T21:19:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023146462999999999, 2024-09-17T21:19:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 490 2024-09-17T21:19:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:19:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:19:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234541+0000", 2024-09-17T21:19:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4869999999999999e-06 2024-09-17T21:19:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234559+0000", 2024-09-17T21:19:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8105999999999999e-05 2024-09-17T21:19:25.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:25.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234577+0000", 2024-09-17T21:19:25.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.836e-05 2024-09-17T21:19:25.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:25.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234592+0000", 2024-09-17T21:19:25.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4766000000000001e-05 2024-09-17T21:19:25.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257187+0000", 2024-09-17T21:19:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022594725 2024-09-17T21:19:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257374+0000", 2024-09-17T21:19:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018767299999999999 2024-09-17T21:19:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257686+0000", 2024-09-17T21:19:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000311346 2024-09-17T21:19:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:491 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:19:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.456882618999998, 2024-09-17T21:19:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023696872000000001, 2024-09-17T21:19:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 491 2024-09-17T21:19:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:19:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:19:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234613+0000", 2024-09-17T21:19:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-06 2024-09-17T21:19:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234618+0000", 2024-09-17T21:19:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3839999999999996e-06 2024-09-17T21:19:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234635+0000", 2024-09-17T21:19:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.685e-05 2024-09-17T21:19:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234648+0000", 2024-09-17T21:19:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.313e-05 2024-09-17T21:19:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257780+0000", 2024-09-17T21:19:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023131946 2024-09-17T21:19:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257985+0000", 2024-09-17T21:19:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020468400000000001 2024-09-17T21:19:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258309+0000", 2024-09-17T21:19:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032357499999999999 2024-09-17T21:19:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:492 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:19:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.456827032, 2024-09-17T21:19:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024258994999999998, 2024-09-17T21:19:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 492 2024-09-17T21:19:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:19:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:19:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234669+0000", 2024-09-17T21:19:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.313e-06 2024-09-17T21:19:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234674+0000", 2024-09-17T21:19:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1599999999999997e-06 2024-09-17T21:19:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234690+0000", 2024-09-17T21:19:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.666e-05 2024-09-17T21:19:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234704+0000", 2024-09-17T21:19:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3043e-05 2024-09-17T21:19:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258397+0000", 2024-09-17T21:19:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023693204999999998 2024-09-17T21:19:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258593+0000", 2024-09-17T21:19:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019578999999999999 2024-09-17T21:19:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258926+0000", 2024-09-17T21:19:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033382399999999998 2024-09-17T21:19:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:493 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:25.764 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:19:25.764 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.456770567, 2024-09-17T21:19:25.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024793461999999999, 2024-09-17T21:19:25.764 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:25.764 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:25.764 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:25.764 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:25.764 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:25.764 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 493 2024-09-17T21:19:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:19:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:19:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234725+0000", 2024-09-17T21:19:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.24e-06 2024-09-17T21:19:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234731+0000", 2024-09-17T21:19:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8939999999999998e-06 2024-09-17T21:19:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234749+0000", 2024-09-17T21:19:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7863000000000002e-05 2024-09-17T21:19:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234762+0000", 2024-09-17T21:19:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3322999999999999e-05 2024-09-17T21:19:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259022+0000", 2024-09-17T21:19:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024259691999999999 2024-09-17T21:19:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259259+0000", 2024-09-17T21:19:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023669199999999999 2024-09-17T21:19:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259517+0000", 2024-09-17T21:19:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025875799999999999 2024-09-17T21:19:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:494 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:19:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.456712867, 2024-09-17T21:19:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025251952000000001, 2024-09-17T21:19:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 494 2024-09-17T21:19:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:19:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:19:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234783+0000", 2024-09-17T21:19:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2440000000000001e-06 2024-09-17T21:19:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234789+0000", 2024-09-17T21:19:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:19:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234806+0000", 2024-09-17T21:19:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7450000000000001e-05 2024-09-17T21:19:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234819+0000", 2024-09-17T21:19:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3326000000000001e-05 2024-09-17T21:19:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259585+0000", 2024-09-17T21:19:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024765579999999999 2024-09-17T21:19:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259787+0000", 2024-09-17T21:19:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202444 2024-09-17T21:19:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260033+0000", 2024-09-17T21:19:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024611499999999998 2024-09-17T21:19:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:495 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:19:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.456657503999999, 2024-09-17T21:19:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025776342000000001, 2024-09-17T21:19:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 495 2024-09-17T21:19:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:19:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:19:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234838+0000", 2024-09-17T21:19:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2130000000000001e-06 2024-09-17T21:19:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234843+0000", 2024-09-17T21:19:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9899999999999997e-06 2024-09-17T21:19:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234860+0000", 2024-09-17T21:19:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6973000000000002e-05 2024-09-17T21:19:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234873+0000", 2024-09-17T21:19:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3179999999999999e-05 2024-09-17T21:19:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260135+0000", 2024-09-17T21:19:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025262118 2024-09-17T21:19:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260364+0000", 2024-09-17T21:19:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022830699999999999 2024-09-17T21:19:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260613+0000", 2024-09-17T21:19:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024956099999999998 2024-09-17T21:19:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:496 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:19:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.456604224000003, 2024-09-17T21:19:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026376773999999999, 2024-09-17T21:19:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 496 2024-09-17T21:19:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:19:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:19:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234891+0000", 2024-09-17T21:19:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1230000000000001e-06 2024-09-17T21:19:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234896+0000", 2024-09-17T21:19:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9599999999999999e-06 2024-09-17T21:19:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234913+0000", 2024-09-17T21:19:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6747000000000001e-05 2024-09-17T21:19:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234926+0000", 2024-09-17T21:19:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-05 2024-09-17T21:19:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260675+0000", 2024-09-17T21:19:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025748979000000002 2024-09-17T21:19:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260883+0000", 2024-09-17T21:19:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020848399999999999 2024-09-17T21:19:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261267+0000", 2024-09-17T21:19:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038345099999999998 2024-09-17T21:19:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:682 3.9 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2455552~12288,write 8126464~10240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:19:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.422667173000001, 2024-09-17T21:19:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018447261999999999, 2024-09-17T21:19:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 682 2024-09-17T21:19:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:19:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:19:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268830+0000", 2024-09-17T21:19:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6419999999999999e-06 2024-09-17T21:19:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271673+0000", 2024-09-17T21:19:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028432610000000001 2024-09-17T21:19:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271700+0000", 2024-09-17T21:19:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7200999999999999e-05 2024-09-17T21:19:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271715+0000", 2024-09-17T21:19:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5145e-05 2024-09-17T21:19:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271785+0000", 2024-09-17T21:19:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9519999999999998e-05 2024-09-17T21:19:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271918+0000", 2024-09-17T21:19:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013330899999999999 2024-09-17T21:19:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.272449+0000", 2024-09-17T21:19:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00053051200000000002 2024-09-17T21:19:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.278822+0000", 2024-09-17T21:19:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0063731619999999999 2024-09-17T21:19:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287167+0000", 2024-09-17T21:19:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083452149999999996 2024-09-17T21:19:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287213+0000", 2024-09-17T21:19:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5972e-05 2024-09-17T21:19:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:25.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287274+0000", 2024-09-17T21:19:25.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1322999999999995e-05 2024-09-17T21:19:25.787 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.787 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:25.787 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.787 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:25.787 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:25.787 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:25.788 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:19:25.789 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:25.786+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:25.889 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:25.884+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:25.918 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:25.918 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:25.918 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:19:25.918 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:25.933 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:19:26.018 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:26.016+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:26.091 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:26.086+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:26.097 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:26.097 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:26.097 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:26.097 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:26.097 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:26.111 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:19:26.163 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-17T21:19:26.198 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:26.196+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:26.283 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:26.283 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:26.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:26.283 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:19:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.675517334000006, 2024-09-17T21:19:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:19:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:19:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:19:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:19:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:19:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:19:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:19:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:19:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:19:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:19:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:19:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:19:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:19:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:19:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:19:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:19:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:19:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:19:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.665570697999996, 2024-09-17T21:19:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:19:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:19:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:19:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:19:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:19:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:19:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:19:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:19:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:19:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:19:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:19:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:19:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:19:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:19:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:19:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:19:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:19:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:19:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 63.422419857999998, 2024-09-17T21:19:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:19:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:19:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:19:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:19:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:19:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:19:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:19:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:19:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:19:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:19:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:19:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:19:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:19:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:19:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:19:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:19:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:19:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:19:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:19:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:19:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:19:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:19:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:19:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:19:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:19:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:19:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.299 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:19:26.299 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:26.299 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 63.412774372000001, 2024-09-17T21:19:26.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:19:26.299 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:26.299 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:26.299 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:26.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:26.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:19:26.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:19:26.299 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:19:26.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.299 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:26.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:26.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:26.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:26.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:26.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:26.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:19:26.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:19:26.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:26.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:19:26.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:19:26.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:26.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:19:26.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:19:26.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:19:26.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:19:26.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:26.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:19:26.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:19:26.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:26.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:19:26.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:19:26.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:26.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:19:26.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:19:26.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:26.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:19:26.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:19:26.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:26.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:19:26.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:19:26.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:26.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:19:26.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:19:26.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:26.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:19:26.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:19:26.304 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.304 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:26.304 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 50.043838338999997, 2024-09-17T21:19:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:19:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:19:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.305 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:26.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:19:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:19:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:19:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:19:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:19:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:19:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:19:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:19:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:19:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:19:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:19:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:19:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:19:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:19:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:565 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:19:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 50.040049373999999, 2024-09-17T21:19:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024193392000000001, 2024-09-17T21:19:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:26.309 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 565 2024-09-17T21:19:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:19:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:19:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239660+0000", 2024-09-17T21:19:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:19:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239666+0000", 2024-09-17T21:19:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8699999999999997e-06 2024-09-17T21:19:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:26.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239688+0000", 2024-09-17T21:19:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.213e-05 2024-09-17T21:19:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239703+0000", 2024-09-17T21:19:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4813e-05 2024-09-17T21:19:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263427+0000", 2024-09-17T21:19:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023724630999999999 2024-09-17T21:19:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263614+0000", 2024-09-17T21:19:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018664500000000001 2024-09-17T21:19:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263852+0000", 2024-09-17T21:19:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023793999999999999 2024-09-17T21:19:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:819 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5111808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.842266048999999, 2024-09-17T21:19:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024344811000000001, 2024-09-17T21:19:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 819 2024-09-17T21:19:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.314 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:26.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437443+0000", 2024-09-17T21:19:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1969999999999999e-06 2024-09-17T21:19:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437448+0000", 2024-09-17T21:19:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9490000000000004e-06 2024-09-17T21:19:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437467+0000", 2024-09-17T21:19:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8862999999999999e-05 2024-09-17T21:19:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437480+0000", 2024-09-17T21:19:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2833000000000001e-05 2024-09-17T21:19:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461552+0000", 2024-09-17T21:19:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024072695000000002 2024-09-17T21:19:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461639+0000", 2024-09-17T21:19:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6996000000000005e-05 2024-09-17T21:19:26.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461787+0000", 2024-09-17T21:19:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014727800000000001 2024-09-17T21:19:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.842153777, 2024-09-17T21:19:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:19:26.318 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:19:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:19:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:19:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:19:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:19:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:19:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:19:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:19:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:19:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:19:26.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:19:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:19:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:19:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:19:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:19:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:26.322 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:833 3.5 3:a772dfb1:::rbd_data.11523651b015.0000000000000001:d [sparse-read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.841719490999999, 2024-09-17T21:19:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024216024999999999, 2024-09-17T21:19:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 833 2024-09-17T21:19:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.323 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437990+0000", 2024-09-17T21:19:26.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2500000000000001e-06 2024-09-17T21:19:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437995+0000", 2024-09-17T21:19:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1240000000000004e-06 2024-09-17T21:19:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438012+0000", 2024-09-17T21:19:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7572e-05 2024-09-17T21:19:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438031+0000", 2024-09-17T21:19:26.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8612000000000001e-05 2024-09-17T21:19:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461830+0000", 2024-09-17T21:19:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023799135999999999 2024-09-17T21:19:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462005+0000", 2024-09-17T21:19:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000174747 2024-09-17T21:19:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:26.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:19:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019958399999999999 2024-09-17T21:19:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.841659199999999, 2024-09-17T21:19:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:19:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:26.327 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:19:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:19:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:19:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:19:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:19:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:19:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:19:26.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:19:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:19:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:19:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:19:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:19:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:19:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:19:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:19:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.841603818999999, 2024-09-17T21:19:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:19:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:26.331 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:19:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:19:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:19:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:19:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:19:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:19:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:19:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:19:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:19:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:19:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:19:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:19:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:19:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:19:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:19:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:26.335 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.841547541000001, 2024-09-17T21:19:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:19:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:19:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:26.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:26.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:26.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:19:26.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:19:26.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:26.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:19:26.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:19:26.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:19:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:19:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:19:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:19:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:19:26.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:19:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:19:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:19:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:19:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:19:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:26.342 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.840597901999999, 2024-09-17T21:19:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:19:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:19:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.343 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:19:26.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:19:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:19:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:19:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:19:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:19:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:19:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:19:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:19:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:19:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:19:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:19:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:19:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:19:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.840519281999999, 2024-09-17T21:19:26.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:19:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:19:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:26.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:19:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:19:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:19:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:19:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:19:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:19:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:19:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:19:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:19:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:19:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:19:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:19:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:19:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:19:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.351 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:847 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 4653056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439245+0000", 2024-09-17T21:19:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.840462844000001, 2024-09-17T21:19:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024172945000000001, 2024-09-17T21:19:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 847 2024-09-17T21:19:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439245+0000", 2024-09-17T21:19:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439245+0000", 2024-09-17T21:19:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439246+0000", 2024-09-17T21:19:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:19:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439252+0000", 2024-09-17T21:19:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3569999999999997e-06 2024-09-17T21:19:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439269+0000", 2024-09-17T21:19:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7316000000000002e-05 2024-09-17T21:19:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439282+0000", 2024-09-17T21:19:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3269e-05 2024-09-17T21:19:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463240+0000", 2024-09-17T21:19:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02395721 2024-09-17T21:19:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463306+0000", 2024-09-17T21:19:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6618000000000005e-05 2024-09-17T21:19:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463418+0000", 2024-09-17T21:19:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111672 2024-09-17T21:19:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.840406343000001, 2024-09-17T21:19:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:19:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:26.356 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:19:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:26.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:19:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:19:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:19:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:19:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:19:26.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:19:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:19:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:19:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:19:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:19:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:19:26.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:19:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:19:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:19:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:849 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5963776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.839812306999999, 2024-09-17T21:19:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024537514, 2024-09-17T21:19:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 849 2024-09-17T21:19:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439897+0000", 2024-09-17T21:19:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9099999999999999e-06 2024-09-17T21:19:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439905+0000", 2024-09-17T21:19:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0870000000000007e-06 2024-09-17T21:19:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439924+0000", 2024-09-17T21:19:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9042000000000002e-05 2024-09-17T21:19:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439941+0000", 2024-09-17T21:19:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.624e-05 2024-09-17T21:19:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464132+0000", 2024-09-17T21:19:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024191579000000001 2024-09-17T21:19:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464291+0000", 2024-09-17T21:19:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015904800000000001 2024-09-17T21:19:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464433+0000", 2024-09-17T21:19:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014160799999999999 2024-09-17T21:19:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:26.364 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.839745289, 2024-09-17T21:19:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:19:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:19:26.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:19:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:19:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:19:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:19:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:19:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:19:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:19:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:19:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:19:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:19:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:19:26.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:19:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:19:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:19:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:26.369 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.839680477999998, 2024-09-17T21:19:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:19:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:19:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:26.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:19:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:19:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:19:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:19:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:19:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:19:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:19:26.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:19:26.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:26.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:19:26.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:19:26.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:26.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:19:26.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:19:26.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:19:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:19:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.839619501000001, 2024-09-17T21:19:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:19:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:26.374 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:19:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:26.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:19:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:19:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:19:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:19:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:19:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:19:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:19:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:19:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:19:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:19:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:19:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:19:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:19:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:19:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:26.378 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:26.378 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:26.380 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:26.376+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:26.380 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:19:26.427 INFO:teuthology.orchestra.run.smithi184.stdout:3208 read 0xa49f000 thru 0xa4a766c (0x866d bytes) 2024-09-17T21:19:26.428 INFO:teuthology.orchestra.run.smithi184.stdout:3209 write 0x12f6400 thru 0x12f6631 (0x232 bytes) 2024-09-17T21:19:26.429 INFO:teuthology.orchestra.run.smithi184.stdout:3211 read 0x89df000 thru 0x89ecb29 (0xdb2a bytes) 2024-09-17T21:19:26.430 INFO:teuthology.orchestra.run.smithi184.stdout:3212 read 0xa5f9000 thru 0xa605cc1 (0xccc2 bytes) 2024-09-17T21:19:26.432 INFO:teuthology.orchestra.run.smithi184.stdout:3214 write 0x5f15600 thru 0x5f1b4de (0x5edf bytes) 2024-09-17T21:19:26.432 INFO:teuthology.orchestra.run.smithi184.stdout:3215 read 0x4d2f000 thru 0x4d3a4ae (0xb4af bytes) 2024-09-17T21:19:26.434 INFO:teuthology.orchestra.run.smithi184.stdout:3219 write 0x4afaa00 thru 0x4b053fd (0xa9fe bytes) 2024-09-17T21:19:26.435 INFO:teuthology.orchestra.run.smithi184.stdout:3220 punch from 0x5b43e00 to 0x5b50400, (0xc600 bytes) 2024-09-17T21:19:26.481 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:26.476+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:26.494 INFO:teuthology.orchestra.run.smithi184.stdout:3221 punch from 0x1b08600 to 0x1b13600, (0xb000 bytes) 2024-09-17T21:19:26.514 INFO:teuthology.orchestra.run.smithi184.stdout:3224 read 0x4754a00 thru 0x475adbe (0x63bf bytes) 2024-09-17T21:19:26.514 INFO:teuthology.orchestra.run.smithi184.stdout:3226 clone 18 order 19 su 262144 sc 2 2024-09-17T21:19:26.531 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:26.531 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:26.531 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:19:26.532 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:26.548 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:19:26.593 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:26.590+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:26.671 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:26.672 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:26.672 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:26.672 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:26.672 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:26.682 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:26.678+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:26.686 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:19:26.721 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone17 from 0xba24e00 (overlap 0xba24e00) to 0x129d400 2024-09-17T21:19:26.791 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:26.780+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:26.885 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:26.880+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:26.885 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:26.885 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:26.885 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:26.886 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:26.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.886 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:406 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 2359296~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:26.886 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:19:26.886 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.687957642000001, 2024-09-17T21:19:26.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042448018999999997, 2024-09-17T21:19:26.886 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:26.886 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:26.886 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:26.886 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:26.886 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:26.886 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:26.886 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 406 2024-09-17T21:19:26.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.887 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:26.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.887 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:26.887 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:19:26.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.887 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:26.887 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:19:26.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.887 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:26.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195056+0000", 2024-09-17T21:19:26.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2729999999999999e-06 2024-09-17T21:19:26.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:26.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195060+0000", 2024-09-17T21:19:26.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5399999999999997e-06 2024-09-17T21:19:26.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:26.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195076+0000", 2024-09-17T21:19:26.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6217000000000001e-05 2024-09-17T21:19:26.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195093+0000", 2024-09-17T21:19:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.683e-05 2024-09-17T21:19:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236217+0000", 2024-09-17T21:19:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041124102000000003 2024-09-17T21:19:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236339+0000", 2024-09-17T21:19:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121876 2024-09-17T21:19:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237502+0000", 2024-09-17T21:19:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011631809999999999 2024-09-17T21:19:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:416 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2228224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:19:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.687459615999998, 2024-09-17T21:19:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042361990000000002, 2024-09-17T21:19:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 416 2024-09-17T21:19:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:19:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:19:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195554+0000", 2024-09-17T21:19:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9300000000000002e-06 2024-09-17T21:19:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195563+0000", 2024-09-17T21:19:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2770000000000005e-06 2024-09-17T21:19:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195586+0000", 2024-09-17T21:19:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3889000000000001e-05 2024-09-17T21:19:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195603+0000", 2024-09-17T21:19:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6084000000000001e-05 2024-09-17T21:19:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237563+0000", 2024-09-17T21:19:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041960856999999997 2024-09-17T21:19:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237695+0000", 2024-09-17T21:19:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013144400000000001 2024-09-17T21:19:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237914+0000", 2024-09-17T21:19:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000219509 2024-09-17T21:19:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.895 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:439 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:26.895 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:19:26.895 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.686895714000002, 2024-09-17T21:19:26.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043053811999999997, 2024-09-17T21:19:26.895 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:26.895 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:26.895 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:26.895 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:26.895 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:26.895 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:26.895 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 439 2024-09-17T21:19:26.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.895 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:26.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:26.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:19:26.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:26.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:19:26.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:26.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196118+0000", 2024-09-17T21:19:26.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3969999999999999e-06 2024-09-17T21:19:26.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:26.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196157+0000", 2024-09-17T21:19:26.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9826999999999997e-05 2024-09-17T21:19:26.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:26.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196179+0000", 2024-09-17T21:19:26.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1756e-05 2024-09-17T21:19:26.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196194+0000", 2024-09-17T21:19:26.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4707e-05 2024-09-17T21:19:26.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:26.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237980+0000", 2024-09-17T21:19:26.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041786219999999999 2024-09-17T21:19:26.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:26.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.238065+0000", 2024-09-17T21:19:26.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4560000000000004e-05 2024-09-17T21:19:26.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:26.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239170+0000", 2024-09-17T21:19:26.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001105345 2024-09-17T21:19:26.899 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.899 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:26.899 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.899 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:440 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:26.899 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:19:26.899 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.686793211000001, 2024-09-17T21:19:26.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043890039999999998, 2024-09-17T21:19:26.900 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:26.900 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:26.900 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:26.900 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:26.900 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:26.900 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:26.900 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 440 2024-09-17T21:19:26.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.900 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:26.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:26.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:19:26.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:26.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:19:26.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:26.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196220+0000", 2024-09-17T21:19:26.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.254e-06 2024-09-17T21:19:26.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:26.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196225+0000", 2024-09-17T21:19:26.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7829999999999998e-06 2024-09-17T21:19:26.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:26.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196241+0000", 2024-09-17T21:19:26.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6322999999999999e-05 2024-09-17T21:19:26.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196258+0000", 2024-09-17T21:19:26.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7056000000000001e-05 2024-09-17T21:19:26.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:26.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239229+0000", 2024-09-17T21:19:26.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042971058999999999 2024-09-17T21:19:26.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:26.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239403+0000", 2024-09-17T21:19:26.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017399200000000001 2024-09-17T21:19:26.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:26.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240109+0000", 2024-09-17T21:19:26.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00070557300000000003 2024-09-17T21:19:26.903 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.903 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:26.903 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.904 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:450 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:26.904 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:19:26.904 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.686432259999997, 2024-09-17T21:19:26.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043994521000000002, 2024-09-17T21:19:26.904 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:26.904 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:26.904 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:26.904 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:26.904 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:26.904 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:26.904 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 450 2024-09-17T21:19:26.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.904 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:26.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:26.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:19:26.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:26.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:19:26.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:26.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196581+0000", 2024-09-17T21:19:26.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4500000000000001e-06 2024-09-17T21:19:26.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:26.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196588+0000", 2024-09-17T21:19:26.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7530000000000004e-06 2024-09-17T21:19:26.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:26.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196607+0000", 2024-09-17T21:19:26.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8987e-05 2024-09-17T21:19:26.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196620+0000", 2024-09-17T21:19:26.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3589e-05 2024-09-17T21:19:26.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:26.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240225+0000", 2024-09-17T21:19:26.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043604878999999999 2024-09-17T21:19:26.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:26.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240377+0000", 2024-09-17T21:19:26.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015146699999999999 2024-09-17T21:19:26.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:26.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240574+0000", 2024-09-17T21:19:26.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019739599999999999 2024-09-17T21:19:26.907 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.908 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:26.908 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.908 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:464 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:26.908 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:19:26.908 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.686181839, 2024-09-17T21:19:26.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044651692999999999, 2024-09-17T21:19:26.908 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:26.908 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:26.908 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:26.908 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:26.908 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:26.908 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:26.909 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 464 2024-09-17T21:19:26.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.909 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:26.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:26.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:19:26.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:26.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:19:26.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:26.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196838+0000", 2024-09-17T21:19:26.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4970000000000004e-06 2024-09-17T21:19:26.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:26.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196859+0000", 2024-09-17T21:19:26.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1166e-05 2024-09-17T21:19:26.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:26.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196879+0000", 2024-09-17T21:19:26.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.995e-05 2024-09-17T21:19:26.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196896+0000", 2024-09-17T21:19:26.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6929999999999999e-05 2024-09-17T21:19:26.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:26.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240617+0000", 2024-09-17T21:19:26.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043721784999999999 2024-09-17T21:19:26.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:26.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240702+0000", 2024-09-17T21:19:26.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4060999999999994e-05 2024-09-17T21:19:26.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:26.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241482+0000", 2024-09-17T21:19:26.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000780304 2024-09-17T21:19:26.912 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.912 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:26.912 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.912 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:475 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:26.912 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:19:26.912 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.685912707, 2024-09-17T21:19:26.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044661981000000003, 2024-09-17T21:19:26.912 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:26.912 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:26.912 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:26.912 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:26.913 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:26.913 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:26.913 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 475 2024-09-17T21:19:26.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.913 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:26.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:26.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:19:26.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:26.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:19:26.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:26.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197100+0000", 2024-09-17T21:19:26.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.206e-06 2024-09-17T21:19:26.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:26.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197105+0000", 2024-09-17T21:19:26.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.527e-06 2024-09-17T21:19:26.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:26.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197124+0000", 2024-09-17T21:19:26.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:19:26.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197136+0000", 2024-09-17T21:19:26.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2857000000000001e-05 2024-09-17T21:19:26.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:26.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241539+0000", 2024-09-17T21:19:26.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044402077999999998 2024-09-17T21:19:26.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:26.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241653+0000", 2024-09-17T21:19:26.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114931 2024-09-17T21:19:26.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:26.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241761+0000", 2024-09-17T21:19:26.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107753 2024-09-17T21:19:26.916 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.916 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:26.916 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.916 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:476 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:26.916 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:19:26.917 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.685858881999998, 2024-09-17T21:19:26.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045038976000000001, 2024-09-17T21:19:26.917 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:26.917 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:26.917 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:26.917 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:26.917 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:26.917 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:26.917 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 476 2024-09-17T21:19:26.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.917 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:26.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:26.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:19:26.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:26.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:19:26.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:26.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197154+0000", 2024-09-17T21:19:26.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.127e-06 2024-09-17T21:19:26.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:26.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197159+0000", 2024-09-17T21:19:26.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4569999999999998e-06 2024-09-17T21:19:26.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:26.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197174+0000", 2024-09-17T21:19:26.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5173e-05 2024-09-17T21:19:26.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197186+0000", 2024-09-17T21:19:26.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:19:26.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:26.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241863+0000", 2024-09-17T21:19:26.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044676826000000003 2024-09-17T21:19:26.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:26.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241998+0000", 2024-09-17T21:19:26.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013476100000000001 2024-09-17T21:19:26.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:26.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242192+0000", 2024-09-17T21:19:26.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019436900000000001 2024-09-17T21:19:26.920 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.920 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:26.920 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.921 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:494 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:26.921 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:26.921 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.685683699000002, 2024-09-17T21:19:26.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046244995999999997, 2024-09-17T21:19:26.921 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:26.921 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:26.921 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:26.921 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:26.921 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:26.921 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:26.921 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 494 2024-09-17T21:19:26.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.921 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:26.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:26.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:26.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:26.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:26.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:26.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197329+0000", 2024-09-17T21:19:26.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.173e-06 2024-09-17T21:19:26.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:26.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197334+0000", 2024-09-17T21:19:26.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0499999999999999e-06 2024-09-17T21:19:26.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:26.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197352+0000", 2024-09-17T21:19:26.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7133e-05 2024-09-17T21:19:26.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197367+0000", 2024-09-17T21:19:26.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5123000000000001e-05 2024-09-17T21:19:26.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:26.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:19:26.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044989829000000002 2024-09-17T21:19:26.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:26.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242526+0000", 2024-09-17T21:19:26.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016968199999999999 2024-09-17T21:19:26.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:26.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243573+0000", 2024-09-17T21:19:26.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010470060000000001 2024-09-17T21:19:26.924 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.924 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:26.925 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.925 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:502 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:26.925 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:19:26.925 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.683450039999997, 2024-09-17T21:19:26.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044346086999999999, 2024-09-17T21:19:26.925 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:26.925 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:26.925 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:26.925 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:26.925 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:26.925 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:26.925 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 502 2024-09-17T21:19:26.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.926 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:26.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:26.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:19:26.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:26.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:19:26.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:26.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199563+0000", 2024-09-17T21:19:26.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2929999999999999e-06 2024-09-17T21:19:26.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:26.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199569+0000", 2024-09-17T21:19:26.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5330000000000002e-06 2024-09-17T21:19:26.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:26.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199614+0000", 2024-09-17T21:19:26.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4793000000000003e-05 2024-09-17T21:19:26.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199628+0000", 2024-09-17T21:19:26.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4037e-05 2024-09-17T21:19:26.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:26.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243665+0000", 2024-09-17T21:19:26.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044037106999999999 2024-09-17T21:19:26.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:26.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243767+0000", 2024-09-17T21:19:26.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010279499999999999 2024-09-17T21:19:26.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:26.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243908+0000", 2024-09-17T21:19:26.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140529 2024-09-17T21:19:26.929 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.929 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:26.929 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.929 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:523 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:26.929 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:19:26.929 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.683003886999998, 2024-09-17T21:19:26.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044789645000000003, 2024-09-17T21:19:26.930 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:26.930 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:26.930 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:26.930 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:26.930 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:26.930 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:26.930 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 523 2024-09-17T21:19:26.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.930 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:26.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:26.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:19:26.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:26.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:19:26.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:26.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:19:26.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9e-06 2024-09-17T21:19:26.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:26.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200021+0000", 2024-09-17T21:19:26.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9499999999999998e-06 2024-09-17T21:19:26.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:26.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200038+0000", 2024-09-17T21:19:26.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.73e-05 2024-09-17T21:19:26.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200051+0000", 2024-09-17T21:19:26.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.322e-05 2024-09-17T21:19:26.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:26.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243987+0000", 2024-09-17T21:19:26.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043935386999999999 2024-09-17T21:19:26.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:26.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244136+0000", 2024-09-17T21:19:26.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014868300000000001 2024-09-17T21:19:26.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:26.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244798+0000", 2024-09-17T21:19:26.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00066220500000000002 2024-09-17T21:19:26.934 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.934 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:26.934 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.934 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:524 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:26.934 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:26.934 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.682937670999998, 2024-09-17T21:19:26.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045737414999999997, 2024-09-17T21:19:26.934 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:26.934 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:26.934 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:26.934 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:26.935 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:26.935 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:26.935 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 524 2024-09-17T21:19:26.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.935 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:26.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:26.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:26.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:26.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:26.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:26.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200076+0000", 2024-09-17T21:19:26.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:19:26.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:26.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200081+0000", 2024-09-17T21:19:26.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7030000000000003e-06 2024-09-17T21:19:26.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:26.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200099+0000", 2024-09-17T21:19:26.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7836999999999999e-05 2024-09-17T21:19:26.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200113+0000", 2024-09-17T21:19:26.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3570000000000001e-05 2024-09-17T21:19:26.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:26.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244866+0000", 2024-09-17T21:19:26.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044753573999999997 2024-09-17T21:19:26.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:26.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244972+0000", 2024-09-17T21:19:26.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106111 2024-09-17T21:19:26.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:26.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245812+0000", 2024-09-17T21:19:26.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083923999999999999 2024-09-17T21:19:26.938 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.938 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:26.938 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.939 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:525 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3670016~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:26.939 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:26.939 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.682786288999999, 2024-09-17T21:19:26.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046645697, 2024-09-17T21:19:26.939 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:26.939 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:26.939 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:26.939 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:26.939 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:26.939 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:26.939 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 525 2024-09-17T21:19:26.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.939 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:26.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:26.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:26.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:26.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:26.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:26.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:19:26.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:19:26.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:26.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200234+0000", 2024-09-17T21:19:26.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:19:26.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:26.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200254+0000", 2024-09-17T21:19:26.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9959999999999999e-05 2024-09-17T21:19:26.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200293+0000", 2024-09-17T21:19:26.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9755999999999997e-05 2024-09-17T21:19:26.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:26.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245871+0000", 2024-09-17T21:19:26.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045577871999999998 2024-09-17T21:19:26.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:26.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246030+0000", 2024-09-17T21:19:26.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158975 2024-09-17T21:19:26.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:26.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246871+0000", 2024-09-17T21:19:26.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084111699999999995 2024-09-17T21:19:26.943 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.943 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:26.943 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.943 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:26.943 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:26.943 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.682540613999997, 2024-09-17T21:19:26.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:19:26.943 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:26.943 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:26.943 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:26.943 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:26.944 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:26.944 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:26.944 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:19:26.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.944 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:26.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:26.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:26.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:26.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:26.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:26.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:19:26.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:19:26.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:26.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:19:26.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:19:26.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:26.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:19:26.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:19:26.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:19:26.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:19:26.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:26.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:19:26.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:19:26.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:26.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:19:26.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:19:26.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:26.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:19:26.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:19:26.947 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.947 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:26.947 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.947 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:26.947 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:26.948 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.682361931999999, 2024-09-17T21:19:26.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:19:26.948 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:26.948 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:26.948 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:26.948 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:26.948 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:26.948 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:26.948 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:19:26.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.948 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:26.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:26.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:26.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:26.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:26.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:26.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:19:26.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:19:26.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:26.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:19:26.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:19:26.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:26.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:19:26.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:19:26.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:19:26.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:19:26.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:26.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:19:26.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:19:26.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:26.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:19:26.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:19:26.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:26.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:19:26.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:19:26.953 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.954 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:26.954 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.954 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:26.954 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:26.954 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.682302659000001, 2024-09-17T21:19:26.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:19:26.954 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:26.954 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:26.954 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:26.954 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:26.954 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:26.955 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:26.955 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:19:26.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.955 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:26.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:26.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:26.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:26.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:26.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:26.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:19:26.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:19:26.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:26.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:19:26.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:19:26.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:26.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:19:26.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:19:26.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:19:26.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:19:26.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:26.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:19:26.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:19:26.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:26.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:19:26.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:19:26.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:26.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:19:26.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:19:26.958 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.958 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:26.958 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.959 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:26.959 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:26.959 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.682160463999999, 2024-09-17T21:19:26.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:19:26.959 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:26.959 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:26.959 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:26.959 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:26.959 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:26.959 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:26.959 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:19:26.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.960 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:26.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:26.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:26.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:26.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:26.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:26.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:19:26.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:19:26.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:26.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:19:26.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:19:26.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:26.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:19:26.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:19:26.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:19:26.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:19:26.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:26.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:19:26.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:19:26.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:26.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:19:26.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:19:26.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:26.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:19:26.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:19:26.963 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.963 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:26.963 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.963 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:26.963 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:26.963 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.682104541000001, 2024-09-17T21:19:26.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:19:26.964 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:26.964 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:26.964 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:26.964 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:26.964 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:26.964 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:26.964 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:19:26.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.964 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:26.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:26.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:26.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:26.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:26.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:26.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:19:26.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:19:26.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:26.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:19:26.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:19:26.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:26.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:19:26.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:19:26.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:19:26.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:19:26.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:26.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:19:26.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:19:26.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:26.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:19:26.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:19:26.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:26.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:19:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:19:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.682050924000002, 2024-09-17T21:19:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:19:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:19:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:19:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:19:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:19:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:19:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:19:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:19:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:19:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:19:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:19:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:19:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:19:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:19:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:19:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:19:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:19:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:19:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 10.372822682000001, 2024-09-17T21:19:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:19:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:19:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:19:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:19:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:19:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:19:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:19:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:19:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:19:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:19:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:19:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:19:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:19:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:19:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:19:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:19:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:19:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:19:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:19:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:19:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:19:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:19:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:19:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:26.978 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:26.979 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:19:26.986 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:26.980+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:27.087 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:27.082+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:27.108 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:27.108 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:27.108 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:19:27.108 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:27.122 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:19:27.187 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:27.184+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:27.289 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:27.284+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:27.298 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:27.298 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:27.298 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:27.298 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:27.298 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:27.314 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:19:27.330 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:19:27.389 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:27.384+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:27.436 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:27.436 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:27.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:27.436 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:27.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.436 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:491 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:27.437 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:19:27.437 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 61.234525857000001, 2024-09-17T21:19:27.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041839013000000001, 2024-09-17T21:19:27.437 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:27.437 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:27.437 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:27.437 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:27.437 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:27.437 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:27.437 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 491 2024-09-17T21:19:27.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.437 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:27.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:27.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:19:27.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:27.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:19:27.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:27.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198315+0000", 2024-09-17T21:19:27.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6599999999999996e-07 2024-09-17T21:19:27.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:27.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198317+0000", 2024-09-17T21:19:27.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6639999999999998e-06 2024-09-17T21:19:27.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:27.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198327+0000", 2024-09-17T21:19:27.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5629999999999998e-06 2024-09-17T21:19:27.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:27.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198337+0000", 2024-09-17T21:19:27.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0667e-05 2024-09-17T21:19:27.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:27.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239793+0000", 2024-09-17T21:19:27.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041455519000000003 2024-09-17T21:19:27.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:27.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239929+0000", 2024-09-17T21:19:27.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135716 2024-09-17T21:19:27.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:27.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240153+0000", 2024-09-17T21:19:27.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022421799999999999 2024-09-17T21:19:27.441 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.441 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:27.441 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.441 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:492 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:27.441 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:19:27.441 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 61.234487487000003, 2024-09-17T21:19:27.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042947165000000002, 2024-09-17T21:19:27.441 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:27.441 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:27.441 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:27.442 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:27.442 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:27.442 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:27.442 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 492 2024-09-17T21:19:27.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.442 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:27.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:27.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:19:27.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:27.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:19:27.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.443 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:27.443 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198354+0000", 2024-09-17T21:19:27.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:19:27.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.443 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:27.443 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198357+0000", 2024-09-17T21:19:27.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6969999999999999e-06 2024-09-17T21:19:27.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.443 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:27.443 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198367+0000", 2024-09-17T21:19:27.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8430000000000005e-06 2024-09-17T21:19:27.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:27.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198374+0000", 2024-09-17T21:19:27.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9600000000000003e-06 2024-09-17T21:19:27.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:27.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240226+0000", 2024-09-17T21:19:27.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041852157000000001 2024-09-17T21:19:27.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:27.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240409+0000", 2024-09-17T21:19:27.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018244800000000001 2024-09-17T21:19:27.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:27.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241299+0000", 2024-09-17T21:19:27.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000890554 2024-09-17T21:19:27.445 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.445 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:27.445 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.445 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:493 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 4063232~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:27.445 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:19:27.446 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 61.234450400999997, 2024-09-17T21:19:27.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043897209, 2024-09-17T21:19:27.446 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:27.446 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:27.446 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:27.446 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:27.446 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:27.446 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:27.446 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 493 2024-09-17T21:19:27.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.446 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:27.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:27.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:19:27.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:27.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:19:27.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:27.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198391+0000", 2024-09-17T21:19:27.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4699999999999999e-06 2024-09-17T21:19:27.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:27.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198396+0000", 2024-09-17T21:19:27.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:19:27.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:27.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198412+0000", 2024-09-17T21:19:27.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6507e-05 2024-09-17T21:19:27.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:27.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198425+0000", 2024-09-17T21:19:27.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.256e-05 2024-09-17T21:19:27.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:27.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241383+0000", 2024-09-17T21:19:27.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042958216 2024-09-17T21:19:27.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:27.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241525+0000", 2024-09-17T21:19:27.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014204500000000001 2024-09-17T21:19:27.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:27.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242287+0000", 2024-09-17T21:19:27.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076125800000000001 2024-09-17T21:19:27.449 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.449 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:27.449 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.450 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:496 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:27.450 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:19:27.450 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 61.234360002000003, 2024-09-17T21:19:27.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043732500000000001, 2024-09-17T21:19:27.450 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:27.450 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:27.450 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:27.450 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:27.450 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:27.450 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:27.450 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 496 2024-09-17T21:19:27.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.450 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:27.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:27.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:19:27.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:27.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:19:27.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:27.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198481+0000", 2024-09-17T21:19:27.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-17T21:19:27.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:27.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198488+0000", 2024-09-17T21:19:27.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8499999999999996e-06 2024-09-17T21:19:27.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:27.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198507+0000", 2024-09-17T21:19:27.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9276999999999999e-05 2024-09-17T21:19:27.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:27.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198521+0000", 2024-09-17T21:19:27.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3376e-05 2024-09-17T21:19:27.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:27.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241339+0000", 2024-09-17T21:19:27.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042818119000000002 2024-09-17T21:19:27.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:27.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241449+0000", 2024-09-17T21:19:27.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110209 2024-09-17T21:19:27.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:27.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242212+0000", 2024-09-17T21:19:27.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076329900000000005 2024-09-17T21:19:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:497 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:19:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 61.234279053000002, 2024-09-17T21:19:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044584278999999997, 2024-09-17T21:19:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:27.455 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:27.455 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 497 2024-09-17T21:19:27.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.455 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:27.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.455 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:27.455 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:19:27.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.455 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:27.455 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:19:27.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:27.456 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198562+0000", 2024-09-17T21:19:27.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-17T21:19:27.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:27.456 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198569+0000", 2024-09-17T21:19:27.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.494e-06 2024-09-17T21:19:27.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:27.456 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198588+0000", 2024-09-17T21:19:27.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8955999999999999e-05 2024-09-17T21:19:27.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:27.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198621+0000", 2024-09-17T21:19:27.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3837000000000001e-05 2024-09-17T21:19:27.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:27.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242258+0000", 2024-09-17T21:19:27.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043636900999999999 2024-09-17T21:19:27.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:27.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242343+0000", 2024-09-17T21:19:27.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5048000000000003e-05 2024-09-17T21:19:27.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:27.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243145+0000", 2024-09-17T21:19:27.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00080167300000000003 2024-09-17T21:19:27.458 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.458 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:27.458 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.458 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:503 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:27.458 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:19:27.458 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 61.234194529, 2024-09-17T21:19:27.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044220582000000001, 2024-09-17T21:19:27.458 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:27.458 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:27.459 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:27.459 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:27.459 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:27.459 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:27.459 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 503 2024-09-17T21:19:27.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.459 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:27.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:27.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:19:27.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:27.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:19:27.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.460 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:27.460 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198647+0000", 2024-09-17T21:19:27.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.483e-06 2024-09-17T21:19:27.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.460 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:27.460 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198653+0000", 2024-09-17T21:19:27.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4529999999999999e-06 2024-09-17T21:19:27.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.461 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:27.461 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198672+0000", 2024-09-17T21:19:27.461 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9193000000000001e-05 2024-09-17T21:19:27.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.461 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:27.461 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198685+0000", 2024-09-17T21:19:27.461 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.291e-05 2024-09-17T21:19:27.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.461 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:27.461 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:19:27.461 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043672141999999997 2024-09-17T21:19:27.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:27.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242716+0000", 2024-09-17T21:19:27.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00035901800000000001 2024-09-17T21:19:27.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:27.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242866+0000", 2024-09-17T21:19:27.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000149383 2024-09-17T21:19:27.462 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.462 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:27.462 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.463 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:504 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:27.463 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:19:27.463 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 61.234134736999998, 2024-09-17T21:19:27.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044611406999999999, 2024-09-17T21:19:27.463 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:27.463 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:27.463 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:27.463 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:27.463 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:27.463 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:27.463 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 504 2024-09-17T21:19:27.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.463 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:27.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:27.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:19:27.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:27.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:19:27.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:27.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198706+0000", 2024-09-17T21:19:27.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:19:27.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:27.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198712+0000", 2024-09-17T21:19:27.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5770000000000001e-06 2024-09-17T21:19:27.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:27.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198729+0000", 2024-09-17T21:19:27.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6673000000000001e-05 2024-09-17T21:19:27.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:27.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198742+0000", 2024-09-17T21:19:27.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3543000000000001e-05 2024-09-17T21:19:27.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.466 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:27.466 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242981+0000", 2024-09-17T21:19:27.466 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044238953999999997 2024-09-17T21:19:27.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.466 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:27.466 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243155+0000", 2024-09-17T21:19:27.466 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017401600000000001 2024-09-17T21:19:27.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.466 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:27.466 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243316+0000", 2024-09-17T21:19:27.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016137399999999999 2024-09-17T21:19:27.467 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.467 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:27.467 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.467 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:521 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:27.467 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:19:27.467 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 61.233587598, 2024-09-17T21:19:27.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045043685999999999, 2024-09-17T21:19:27.467 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:27.467 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:27.467 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:27.467 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:27.468 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:27.468 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:27.468 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 521 2024-09-17T21:19:27.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.468 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:27.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.468 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:27.468 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:19:27.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:27.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:19:27.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:27.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199253+0000", 2024-09-17T21:19:27.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1400000000000001e-06 2024-09-17T21:19:27.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:27.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199258+0000", 2024-09-17T21:19:27.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1800000000000004e-06 2024-09-17T21:19:27.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:27.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199276+0000", 2024-09-17T21:19:27.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7686999999999999e-05 2024-09-17T21:19:27.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:27.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199289+0000", 2024-09-17T21:19:27.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3263e-05 2024-09-17T21:19:27.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:27.471 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243397+0000", 2024-09-17T21:19:27.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044107636999999998 2024-09-17T21:19:27.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:27.471 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243527+0000", 2024-09-17T21:19:27.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013038399999999999 2024-09-17T21:19:27.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:27.471 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244296+0000", 2024-09-17T21:19:27.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076839499999999995 2024-09-17T21:19:27.472 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.472 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:27.472 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.472 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:522 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:27.472 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:19:27.472 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 61.233532295000003, 2024-09-17T21:19:27.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046049870999999999, 2024-09-17T21:19:27.472 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:27.472 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:27.472 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:27.472 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:27.472 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:27.473 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:27.473 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 522 2024-09-17T21:19:27.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.473 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:27.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.473 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:27.473 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:19:27.473 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.473 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:27.473 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:19:27.473 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:27.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199309+0000", 2024-09-17T21:19:27.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:19:27.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:27.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199315+0000", 2024-09-17T21:19:27.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2440000000000001e-06 2024-09-17T21:19:27.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:27.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199333+0000", 2024-09-17T21:19:27.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7886000000000001e-05 2024-09-17T21:19:27.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:27.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199379+0000", 2024-09-17T21:19:27.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6383e-05 2024-09-17T21:19:27.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:27.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244394+0000", 2024-09-17T21:19:27.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045014851000000002 2024-09-17T21:19:27.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:27.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244491+0000", 2024-09-17T21:19:27.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7245999999999997e-05 2024-09-17T21:19:27.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.476 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:27.476 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245357+0000", 2024-09-17T21:19:27.476 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00086626800000000001 2024-09-17T21:19:27.476 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.476 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:27.476 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.476 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:527 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:27.476 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:19:27.476 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 61.233435628999999, 2024-09-17T21:19:27.476 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045812522000000001, 2024-09-17T21:19:27.476 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:27.476 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:27.477 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:27.477 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:27.477 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:27.477 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:27.477 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 527 2024-09-17T21:19:27.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.477 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:27.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.477 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:27.477 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:19:27.477 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.477 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:27.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:19:27.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:27.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199406+0000", 2024-09-17T21:19:27.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.367e-06 2024-09-17T21:19:27.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:27.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199412+0000", 2024-09-17T21:19:27.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:19:27.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:27.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199429+0000", 2024-09-17T21:19:27.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7396000000000001e-05 2024-09-17T21:19:27.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:27.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199442+0000", 2024-09-17T21:19:27.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3327e-05 2024-09-17T21:19:27.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:27.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244361+0000", 2024-09-17T21:19:27.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044918197 2024-09-17T21:19:27.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:27.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244482+0000", 2024-09-17T21:19:27.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121945 2024-09-17T21:19:27.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:27.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245217+0000", 2024-09-17T21:19:27.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00073415999999999998 2024-09-17T21:19:27.480 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.480 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:27.480 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.480 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:534 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:27.480 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:19:27.481 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 61.233376923000002, 2024-09-17T21:19:27.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046292505999999997, 2024-09-17T21:19:27.481 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:27.481 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:27.481 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:27.481 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:27.481 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:27.481 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:27.481 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:19:27.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.481 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:27.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.481 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:27.482 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:19:27.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.482 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:27.482 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:19:27.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.482 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:27.482 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199464+0000", 2024-09-17T21:19:27.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:19:27.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.482 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:27.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199473+0000", 2024-09-17T21:19:27.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5629999999999992e-06 2024-09-17T21:19:27.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:27.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199492+0000", 2024-09-17T21:19:27.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9273999999999999e-05 2024-09-17T21:19:27.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:27.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199505+0000", 2024-09-17T21:19:27.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3443e-05 2024-09-17T21:19:27.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:27.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245357+0000", 2024-09-17T21:19:27.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045851714000000002 2024-09-17T21:19:27.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:27.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245550+0000", 2024-09-17T21:19:27.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019301600000000001 2024-09-17T21:19:27.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:27.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245755+0000", 2024-09-17T21:19:27.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205226 2024-09-17T21:19:27.484 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.485 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:27.485 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.485 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:535 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:27.485 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:19:27.485 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 61.233272366999998, 2024-09-17T21:19:27.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046637492000000003, 2024-09-17T21:19:27.485 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:27.485 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:27.485 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:27.485 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:27.485 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:27.486 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:27.486 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 535 2024-09-17T21:19:27.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.486 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:27.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:27.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:19:27.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:27.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:19:27.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:27.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199568+0000", 2024-09-17T21:19:27.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.037e-06 2024-09-17T21:19:27.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:27.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199572+0000", 2024-09-17T21:19:27.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5729999999999999e-06 2024-09-17T21:19:27.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:27.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199585+0000", 2024-09-17T21:19:27.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.279e-05 2024-09-17T21:19:27.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:27.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199592+0000", 2024-09-17T21:19:27.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.61e-06 2024-09-17T21:19:27.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:27.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245873+0000", 2024-09-17T21:19:27.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046280494999999998 2024-09-17T21:19:27.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:27.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246018+0000", 2024-09-17T21:19:27.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000144697 2024-09-17T21:19:27.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:27.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246205+0000", 2024-09-17T21:19:27.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018729 2024-09-17T21:19:27.489 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.489 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:27.489 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.489 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:543 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:27.489 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:27.490 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 61.233177120999997, 2024-09-17T21:19:27.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047564097999999999, 2024-09-17T21:19:27.490 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:27.490 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:27.490 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:27.490 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:27.490 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:27.490 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:27.490 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 543 2024-09-17T21:19:27.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.490 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:27.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:27.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:27.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:27.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:27.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:27.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:27.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9699999999999995e-07 2024-09-17T21:19:27.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:27.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199677+0000", 2024-09-17T21:19:27.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:19:27.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:27.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199690+0000", 2024-09-17T21:19:27.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2977e-05 2024-09-17T21:19:27.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:27.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199705+0000", 2024-09-17T21:19:27.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4656e-05 2024-09-17T21:19:27.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:27.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246307+0000", 2024-09-17T21:19:27.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046601780000000002 2024-09-17T21:19:27.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:27.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246475+0000", 2024-09-17T21:19:27.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016834200000000001 2024-09-17T21:19:27.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:27.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247227+0000", 2024-09-17T21:19:27.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00075173299999999998 2024-09-17T21:19:27.493 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.494 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:27.494 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.494 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:544 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:27.494 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:27.494 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 61.233118712, 2024-09-17T21:19:27.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049283559999999997, 2024-09-17T21:19:27.494 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:27.494 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:27.494 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:27.494 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:27.494 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:27.494 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:27.495 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 544 2024-09-17T21:19:27.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.495 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:27.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.495 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:27.495 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:27.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.495 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:27.495 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:27.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.495 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:27.496 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199722+0000", 2024-09-17T21:19:27.496 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.08e-06 2024-09-17T21:19:27.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.496 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:27.496 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199738+0000", 2024-09-17T21:19:27.496 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5943999999999998e-05 2024-09-17T21:19:27.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.496 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:27.496 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199748+0000", 2024-09-17T21:19:27.496 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0193e-05 2024-09-17T21:19:27.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.496 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:27.497 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199756+0000", 2024-09-17T21:19:27.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4200000000000001e-06 2024-09-17T21:19:27.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.497 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:27.497 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247311+0000", 2024-09-17T21:19:27.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047555300000000002 2024-09-17T21:19:27.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.497 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:27.497 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247571+0000", 2024-09-17T21:19:27.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025996999999999999 2024-09-17T21:19:27.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:27.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249005+0000", 2024-09-17T21:19:27.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001433653 2024-09-17T21:19:27.498 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.498 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:27.498 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.498 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:549 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:27.498 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:27.498 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 61.233073087999998, 2024-09-17T21:19:27.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048464528999999999, 2024-09-17T21:19:27.498 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:27.498 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:27.498 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:27.499 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:27.499 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:27.499 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:27.499 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 549 2024-09-17T21:19:27.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.499 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:27.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:27.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:27.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:27.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:27.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:27.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:27.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:19:27.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:27.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199770+0000", 2024-09-17T21:19:27.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9969999999999999e-06 2024-09-17T21:19:27.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:27.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199780+0000", 2024-09-17T21:19:27.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0003000000000001e-05 2024-09-17T21:19:27.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:27.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199793+0000", 2024-09-17T21:19:27.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2074e-05 2024-09-17T21:19:27.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:27.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247289+0000", 2024-09-17T21:19:27.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047496691000000001 2024-09-17T21:19:27.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:27.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247437+0000", 2024-09-17T21:19:27.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014807899999999999 2024-09-17T21:19:27.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:27.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248231+0000", 2024-09-17T21:19:27.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00079390899999999998 2024-09-17T21:19:27.502 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.502 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:27.502 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.502 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:558 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:27.503 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:27.503 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 61.233032575000003, 2024-09-17T21:19:27.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049709821000000001, 2024-09-17T21:19:27.503 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:27.503 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:27.503 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:27.503 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:27.503 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:27.503 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:27.503 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 558 2024-09-17T21:19:27.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.503 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:27.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:27.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:27.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:27.504 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:27.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:27.504 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199809+0000", 2024-09-17T21:19:27.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.333e-06 2024-09-17T21:19:27.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.505 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:27.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199814+0000", 2024-09-17T21:19:27.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0930000000000002e-06 2024-09-17T21:19:27.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.505 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:27.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199831+0000", 2024-09-17T21:19:27.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7580000000000001e-05 2024-09-17T21:19:27.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.505 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:27.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199860+0000", 2024-09-17T21:19:27.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8609999999999999e-05 2024-09-17T21:19:27.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:27.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249106+0000", 2024-09-17T21:19:27.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049246545000000003 2024-09-17T21:19:27.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:27.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249274+0000", 2024-09-17T21:19:27.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016806699999999999 2024-09-17T21:19:27.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:27.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249517+0000", 2024-09-17T21:19:27.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024259300000000001 2024-09-17T21:19:27.506 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.506 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:27.507 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.507 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:559 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:27.507 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:27.507 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 61.232913213000003, 2024-09-17T21:19:27.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049990694000000002, 2024-09-17T21:19:27.507 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:27.507 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:27.507 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:27.507 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:27.507 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:27.507 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:27.507 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 559 2024-09-17T21:19:27.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.508 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:27.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.508 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:27.508 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:27.508 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.508 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:27.508 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:27.508 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.508 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:27.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199928+0000", 2024-09-17T21:19:27.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:19:27.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:27.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199935+0000", 2024-09-17T21:19:27.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5429999999999999e-06 2024-09-17T21:19:27.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:27.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199954+0000", 2024-09-17T21:19:27.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.912e-05 2024-09-17T21:19:27.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:27.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199969+0000", 2024-09-17T21:19:27.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5305999999999999e-05 2024-09-17T21:19:27.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:27.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249604+0000", 2024-09-17T21:19:27.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049634931 2024-09-17T21:19:27.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:27.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249738+0000", 2024-09-17T21:19:27.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013411599999999999 2024-09-17T21:19:27.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:27.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249917+0000", 2024-09-17T21:19:27.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179111 2024-09-17T21:19:27.511 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.511 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:27.511 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.511 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:560 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:27.511 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:27.511 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 61.232848797000003, 2024-09-17T21:19:27.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050173059999999998, 2024-09-17T21:19:27.511 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:27.512 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:27.512 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:27.512 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:27.512 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:27.512 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:27.512 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 560 2024-09-17T21:19:27.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.512 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:27.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.512 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:27.512 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:27.512 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.513 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:27.513 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:27.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.513 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:27.513 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199992+0000", 2024-09-17T21:19:27.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.517e-06 2024-09-17T21:19:27.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.513 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:27.513 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199998+0000", 2024-09-17T21:19:27.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1460000000000004e-06 2024-09-17T21:19:27.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.514 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:27.514 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:19:27.514 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7407000000000001e-05 2024-09-17T21:19:27.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.514 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:27.514 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200029+0000", 2024-09-17T21:19:27.514 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3499999999999999e-05 2024-09-17T21:19:27.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.514 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:27.514 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249968+0000", 2024-09-17T21:19:27.514 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049939799 2024-09-17T21:19:27.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.514 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:27.515 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250055+0000", 2024-09-17T21:19:27.515 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6731999999999996e-05 2024-09-17T21:19:27.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.515 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:27.515 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250164+0000", 2024-09-17T21:19:27.515 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108959 2024-09-17T21:19:27.515 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.515 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:27.515 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.515 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:19:27.515 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:27.515 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 47.623059087999998, 2024-09-17T21:19:27.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:19:27.516 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:27.516 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:27.516 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:27.516 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:27.516 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:19:27.516 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:19:27.516 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:19:27.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.516 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:27.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.516 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:27.516 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:27.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:27.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:27.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:27.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:19:27.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:19:27.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:27.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:19:27.518 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:19:27.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.518 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:27.518 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:19:27.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:19:27.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.520 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:27.520 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:19:27.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:19:27.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:27.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:19:27.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:19:27.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:27.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:19:27.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:19:27.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:27.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:19:27.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:19:27.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.522 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.522 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:27.522 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:19:27.522 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:19:27.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.522 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.522 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:27.522 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:19:27.522 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:19:27.522 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.522 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:27.522 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.522 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.523 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:19:27.523 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:27.523 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 34.067957302000003, 2024-09-17T21:19:27.523 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:19:27.523 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:27.523 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:27.523 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:27.523 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:27.523 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:19:27.523 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:27.523 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:19:27.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.523 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:27.523 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:27.524 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:27.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:27.524 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:27.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:27.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:27.524 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:19:27.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:19:27.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.525 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:27.525 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:19:27.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:19:27.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.525 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.525 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:27.525 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:19:27.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:19:27.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.525 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.525 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:27.525 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:19:27.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:19:27.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.525 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:27.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:19:27.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:19:27.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:27.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:19:27.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:19:27.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:27.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:19:27.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:19:27.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:27.527 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:19:27.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:19:27.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:27.527 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:19:27.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:19:27.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:27.527 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:19:27.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:19:27.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:27.528 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:19:27.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:19:27.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.528 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:27.528 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:19:27.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:19:27.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:27.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:27.528 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:27.528 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:19:27.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:19:27.528 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.528 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:27.528 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.528 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:27.529 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:27.529 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:27.530 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-17T21:19:27.530 INFO:teuthology.orchestra.run.smithi184.stdout:3230 punch from 0x28f8200 to 0x2905400, (0xd200 bytes) 2024-09-17T21:19:27.531 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:19:27.531 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:27.531 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:27.531 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:27.532 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:27.532 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:27.526+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:27.536 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:27.536 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:27.544 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:19:27.544 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:19:27.544 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-17T21:19:27.544 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T21:19:27.544 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T21:19:27.545 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-17T21:19:27.559 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:27.559 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:27.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:19:27.580 INFO:teuthology.orchestra.run.smithi184.stdout:3232 write 0xdfe7400 thru 0xdfed0f2 (0x5cf3 bytes) 2024-09-17T21:19:27.584 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:27.584 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:27.584 INFO:teuthology.orchestra.run.smithi184.stdout:3234 read 0x89fa000 thru 0x89fd6fc (0x36fd bytes) 2024-09-17T21:19:27.609 INFO:teuthology.orchestra.run.smithi184.stdout:3236 write 0xe300a00 thru 0xe301815 (0xe16 bytes) 2024-09-17T21:19:27.611 INFO:teuthology.orchestra.run.smithi184.stdout:3237 write 0x570da00 thru 0x571d093 (0xf694 bytes) 2024-09-17T21:19:27.612 INFO:teuthology.orchestra.run.smithi184.stdout:3238 read 0xab01600 thru 0xab07ae8 (0x64e9 bytes) 2024-09-17T21:19:27.613 INFO:teuthology.orchestra.run.smithi184.stdout:3239 read 0xb05d200 thru 0xb06a87a (0xd67b bytes) 2024-09-17T21:19:27.613 INFO:teuthology.orchestra.run.smithi184.stdout:3245 punch from 0xbf6fe00 to 0xbf74400, (0x4600 bytes) 2024-09-17T21:19:27.615 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:27.616 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:27.616 INFO:teuthology.orchestra.run.smithi184.stdout:3246 trunc from 0xe301a00 to 0xdee9200 2024-09-17T21:19:27.630 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:27.626+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:27.656 INFO:teuthology.orchestra.run.smithi184.stdout:3247 read 0x8ae2e00 thru 0x8ae8b24 (0x5d25 bytes) 2024-09-17T21:19:27.659 INFO:teuthology.orchestra.run.smithi184.stdout:3248 punch from 0x9fc8c00 to 0x9fd2800, (0x9c00 bytes) 2024-09-17T21:19:27.659 INFO:teuthology.orchestra.run.smithi184.stdout:3250 read 0xb298c00 thru 0xb29b9a3 (0x2da4 bytes) 2024-09-17T21:19:27.660 INFO:teuthology.orchestra.run.smithi184.stdout:3251 punch from 0x573b800 to 0x573e400, (0x2c00 bytes) 2024-09-17T21:19:27.682 INFO:teuthology.orchestra.run.smithi184.stdout:3253 write 0xe186800 thru 0xe190510 (0x9d11 bytes) 2024-09-17T21:19:27.683 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:27.683 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:27.697 INFO:teuthology.orchestra.run.smithi184.stdout:3258 trunc from 0xe190600 to 0xb2a0200 2024-09-17T21:19:27.698 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:19:27.705 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:27.705 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T21:19:27.705 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:27.731 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:27.728+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:27.736 INFO:teuthology.orchestra.run.smithi184.stdout:3259 read 0x568e400 thru 0x5693a97 (0x5698 bytes) 2024-09-17T21:19:27.741 INFO:teuthology.orchestra.run.smithi184.stdout:3260 write 0xa599e00 thru 0xa5a7a51 (0xdc52 bytes) 2024-09-17T21:19:27.742 INFO:teuthology.orchestra.run.smithi184.stdout:3261 write 0x674a600 thru 0x6753d74 (0x9775 bytes) 2024-09-17T21:19:27.742 INFO:teuthology.orchestra.run.smithi184.stdout:3263 read 0x2e95a00 thru 0x2e9a900 (0x4f01 bytes) 2024-09-17T21:19:27.742 INFO:teuthology.orchestra.run.smithi184.stdout:3264 read 0xd1b800 thru 0xd222bd (0x6abe bytes) 2024-09-17T21:19:27.742 INFO:teuthology.orchestra.run.smithi184.stdout:3268 read 0x923d200 thru 0x9241e8d (0x4c8e bytes) 2024-09-17T21:19:27.825 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:27.825 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:27.832 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:27.830+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:27.839 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:19:27.933 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:27.928+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:27.988 INFO:teuthology.orchestra.run.smithi184.stdout:3272 write 0xd264400 thru 0xd26b95f (0x7560 bytes) 2024-09-17T21:19:28.019 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:28.019 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:28.032 INFO:teuthology.orchestra.run.smithi184.stdout:3273 trunc from 0xd26ba00 to 0x88f4e00 2024-09-17T21:19:28.034 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:19:28.035 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:28.030+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:28.065 INFO:teuthology.orchestra.run.smithi184.stdout:3274 write 0x499ae00 thru 0x49a5a1a (0xac1b bytes) 2024-09-17T21:19:28.067 INFO:teuthology.orchestra.run.smithi184.stdout:3277 read 0x2ac8200 thru 0x2ad34d5 (0xb2d6 bytes) 2024-09-17T21:19:28.068 INFO:teuthology.orchestra.run.smithi184.stdout:3280 trunc from 0x88f4e00 to 0x8d89800 2024-09-17T21:19:28.135 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:28.132+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:28.167 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:28.167 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:28.181 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:19:28.237 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:28.234+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:28.337 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:28.332+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:28.345 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:28.345 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:28.356 INFO:teuthology.orchestra.run.smithi184.stdout:3282 write 0x9ad8000 thru 0x9ad973e (0x173f bytes) 2024-09-17T21:19:28.356 INFO:teuthology.orchestra.run.smithi184.stdout:3284 punch from 0x1ff0e00 to 0x1ff8000, (0x7200 bytes) 2024-09-17T21:19:28.358 INFO:teuthology.orchestra.run.smithi184.stdout:3285 read 0x2d48a00 thru 0x2d4a12e (0x172f bytes) 2024-09-17T21:19:28.358 INFO:teuthology.orchestra.run.smithi184.stdout:3287 read 0x6abb600 thru 0x6ac08ac (0x52ad bytes) 2024-09-17T21:19:28.359 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:19:28.360 INFO:teuthology.orchestra.run.smithi184.stdout:3288 read 0x31f0200 thru 0x31f4a78 (0x4879 bytes) 2024-09-17T21:19:28.393 INFO:teuthology.orchestra.run.smithi184.stdout:3289 trunc from 0x9ad9800 to 0x66de800 2024-09-17T21:19:28.435 INFO:teuthology.orchestra.run.smithi184.stdout:3290 write 0x2af7200 thru 0x2b040c0 (0xcec1 bytes) 2024-09-17T21:19:28.437 INFO:teuthology.orchestra.run.smithi184.stdout:3292 punch from 0x4ec8800 to 0x4eccc00, (0x4400 bytes) 2024-09-17T21:19:28.446 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:28.444+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:28.470 INFO:teuthology.orchestra.run.smithi184.stdout:3295 write 0x73bf800 thru 0x73c72fa (0x7afb bytes) 2024-09-17T21:19:28.504 INFO:teuthology.orchestra.run.smithi184.stdout:3296 trunc from 0x73c7400 to 0x19f8000 2024-09-17T21:19:28.539 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:28.536+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:28.547 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:28.547 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:28.555 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-17T21:19:28.562 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:19:28.650 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:28.648+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:28.725 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:28.726 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:28.740 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:19:28.740 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:28.736+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:28.820 INFO:teuthology.orchestra.run.smithi184.stdout:3297 punch from 0x1723200 to 0x1730400, (0xd200 bytes) 2024-09-17T21:19:28.841 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:28.838+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:28.853 INFO:teuthology.orchestra.run.smithi184.stdout:3298 write 0x5d32c00 thru 0x5d41375 (0xe776 bytes) 2024-09-17T21:19:28.854 INFO:teuthology.orchestra.run.smithi184.stdout:3300 write 0x499d400 thru 0x49a7d96 (0xa997 bytes) 2024-09-17T21:19:28.854 INFO:teuthology.orchestra.run.smithi184.stdout:3303 punch from 0x13b9800 to 0x13be800, (0x5000 bytes) 2024-09-17T21:19:28.856 INFO:teuthology.orchestra.run.smithi184.stdout:3304 read 0x4e5b200 thru 0x4e680e4 (0xcee5 bytes) 2024-09-17T21:19:28.856 INFO:teuthology.orchestra.run.smithi184.stdout:3305 read 0x5b66000 thru 0x5b6b087 (0x5088 bytes) 2024-09-17T21:19:28.898 INFO:teuthology.orchestra.run.smithi184.stdout:3306 write 0xe869600 thru 0xe8781b1 (0xebb2 bytes) 2024-09-17T21:19:28.899 INFO:teuthology.orchestra.run.smithi184.stdout:3308 write 0x7513600 thru 0x75141cb (0xbcc bytes) 2024-09-17T21:19:28.899 INFO:teuthology.orchestra.run.smithi184.stdout:3312 punch from 0x9086400 to 0x9092e00, (0xca00 bytes) 2024-09-17T21:19:28.901 INFO:teuthology.orchestra.run.smithi184.stdout:3313 write 0x7815c00 thru 0x7819af3 (0x3ef4 bytes) 2024-09-17T21:19:28.901 INFO:teuthology.orchestra.run.smithi184.stdout:3316 punch from 0x781be00 to 0x7822200, (0x6400 bytes) 2024-09-17T21:19:28.901 INFO:teuthology.orchestra.run.smithi184.stdout:3318 punch from 0x44a5600 to 0x44b3a00, (0xe400 bytes) 2024-09-17T21:19:28.901 INFO:teuthology.orchestra.run.smithi184.stdout:3319 read 0x78ed400 thru 0x78fb9cd (0xe5ce bytes) 2024-09-17T21:19:28.902 INFO:teuthology.orchestra.run.smithi184.stdout:3321 write 0x54d0c00 thru 0x54d3812 (0x2c13 bytes) 2024-09-17T21:19:28.902 INFO:teuthology.orchestra.run.smithi184.stdout:3325 write 0x2c54c00 thru 0x2c5cafd (0x7efe bytes) 2024-09-17T21:19:28.902 INFO:teuthology.orchestra.run.smithi184.stdout:3326 punch from 0x9bd6400 to 0x9bd7200, (0xe00 bytes) 2024-09-17T21:19:28.903 INFO:teuthology.orchestra.run.smithi184.stdout:3327 write 0x96fe400 thru 0x970a230 (0xbe31 bytes) 2024-09-17T21:19:28.903 INFO:teuthology.orchestra.run.smithi184.stdout:3330 write 0x2113a00 thru 0x2113d00 (0x301 bytes) 2024-09-17T21:19:28.903 INFO:teuthology.orchestra.run.smithi184.stdout:3331 write 0x85df000 thru 0x85e46e0 (0x56e1 bytes) 2024-09-17T21:19:28.907 INFO:teuthology.orchestra.run.smithi184.stdout:3334 trunc from 0xe878200 to 0xc874e00 2024-09-17T21:19:28.917 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:28.917 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:28.937 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:19:28.982 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:28.980+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:28.985 INFO:teuthology.orchestra.run.smithi184.stdout:3335 write 0xe3d9c00 thru 0xe3dbe6c (0x226d bytes) 2024-09-17T21:19:28.986 INFO:teuthology.orchestra.run.smithi184.stdout:3336 write 0xa087200 thru 0xa090d23 (0x9b24 bytes) 2024-09-17T21:19:28.987 INFO:teuthology.orchestra.run.smithi184.stdout:3337 write 0x6bc5000 thru 0x6bc8718 (0x3719 bytes) 2024-09-17T21:19:28.998 INFO:teuthology.orchestra.run.smithi184.stdout:3338 trunc from 0xe3dc000 to 0x9f52a00 2024-09-17T21:19:29.042 INFO:teuthology.orchestra.run.smithi184.stdout:3340 write 0x3612400 thru 0x361e04f (0xbc50 bytes) 2024-09-17T21:19:29.042 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:29.038+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:29.044 INFO:teuthology.orchestra.run.smithi184.stdout:3342 read 0x7b05000 thru 0x7b0cef3 (0x7ef4 bytes) 2024-09-17T21:19:29.044 INFO:teuthology.orchestra.run.smithi184.stdout:3344 punch from 0x355a600 to 0x3565000, (0xaa00 bytes) 2024-09-17T21:19:29.045 INFO:teuthology.orchestra.run.smithi184.stdout:3345 punch from 0x4ed1600 to 0x4edb400, (0x9e00 bytes) 2024-09-17T21:19:29.045 INFO:teuthology.orchestra.run.smithi184.stdout:3346 write 0x6751000 thru 0x675fc75 (0xec76 bytes) 2024-09-17T21:19:29.045 INFO:teuthology.orchestra.run.smithi184.stdout:3347 punch from 0x9eea400 to 0x9eeee00, (0x4a00 bytes) 2024-09-17T21:19:29.046 INFO:teuthology.orchestra.run.smithi184.stdout:3352 read 0x6776400 thru 0x677aabb (0x46bc bytes) 2024-09-17T21:19:29.046 INFO:teuthology.orchestra.run.smithi184.stdout:3354 write 0x92bbc00 thru 0x92c3602 (0x7a03 bytes) 2024-09-17T21:19:29.052 INFO:teuthology.orchestra.run.smithi184.stdout:3355 trunc from 0x9f52a00 to 0xd85d000 2024-09-17T21:19:29.064 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:29.064 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:29.078 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:19:29.125 INFO:teuthology.orchestra.run.smithi184.stdout:3357 write 0xec94000 thru 0xec9dd22 (0x9d23 bytes) 2024-09-17T21:19:29.127 INFO:teuthology.orchestra.run.smithi184.stdout:3358 punch from 0x3d74800 to 0x3d7e800, (0xa000 bytes) 2024-09-17T21:19:29.128 INFO:teuthology.orchestra.run.smithi184.stdout:3360 read 0xe7d800 thru 0xe7f844 (0x2045 bytes) 2024-09-17T21:19:29.128 INFO:teuthology.orchestra.run.smithi184.stdout:3363 read 0xab71400 thru 0xab715bc (0x1bd bytes) 2024-09-17T21:19:29.129 INFO:teuthology.orchestra.run.smithi184.stdout:3365 punch from 0x257da00 to 0x2589400, (0xba00 bytes) 2024-09-17T21:19:29.129 INFO:teuthology.orchestra.run.smithi184.stdout:3367 read 0x279f600 thru 0x27ad66e (0xe06f bytes) 2024-09-17T21:19:29.129 INFO:teuthology.orchestra.run.smithi184.stdout:3369 read 0xc482200 thru 0xc48ac51 (0x8a52 bytes) 2024-09-17T21:19:29.142 INFO:teuthology.orchestra.run.smithi184.stdout:3370 trunc from 0xec9de00 to 0x87fc000 2024-09-17T21:19:29.143 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:29.138+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:29.203 INFO:teuthology.orchestra.run.smithi184.stdout:3374 trunc from 0x87fc000 to 0xcc9ee00 2024-09-17T21:19:29.225 INFO:teuthology.orchestra.run.smithi184.stdout:3375 write 0xa5a6a00 thru 0xa5b6245 (0xf846 bytes) 2024-09-17T21:19:29.226 INFO:teuthology.orchestra.run.smithi184.stdout:3376 punch from 0x761000 to 0x76a000, (0x9000 bytes) 2024-09-17T21:19:29.228 INFO:teuthology.orchestra.run.smithi184.stdout:3380 write 0x1507e00 thru 0x1507ed2 (0xd3 bytes) 2024-09-17T21:19:29.230 INFO:teuthology.orchestra.run.smithi184.stdout:3381 write 0x64b3c00 thru 0x64c2a84 (0xee85 bytes) 2024-09-17T21:19:29.230 INFO:teuthology.orchestra.run.smithi184.stdout:3382 punch from 0x8bea600 to 0x8bf9000, (0xea00 bytes) 2024-09-17T21:19:29.230 INFO:teuthology.orchestra.run.smithi184.stdout:3383 read 0xc6bd000 thru 0xc6c2dff (0x5e00 bytes) 2024-09-17T21:19:29.230 INFO:teuthology.orchestra.run.smithi184.stdout:3392 read 0x66b9600 thru 0x66c389b (0xa29c bytes) 2024-09-17T21:19:29.230 INFO:teuthology.orchestra.run.smithi184.stdout:3393 write 0x6842800 thru 0x68448c0 (0x20c1 bytes) 2024-09-17T21:19:29.230 INFO:teuthology.orchestra.run.smithi184.stdout:3395 read 0x1434e00 thru 0x143721e (0x241f bytes) 2024-09-17T21:19:29.235 INFO:teuthology.orchestra.run.smithi184.stdout:3397 trunc from 0xcc9ee00 to 0x9ad2600 2024-09-17T21:19:29.243 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:29.238+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:29.258 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:29.258 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:29.273 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:19:29.304 INFO:teuthology.orchestra.run.smithi184.stdout:3398 write 0xbaf7600 thru 0xbaf780a (0x20b bytes) 2024-09-17T21:19:29.306 INFO:teuthology.orchestra.run.smithi184.stdout:3399 read 0x5ca9a00 thru 0x5ca9b96 (0x197 bytes) 2024-09-17T21:19:29.306 INFO:teuthology.orchestra.run.smithi184.stdout:3400 write 0x8e47000 thru 0x8e52ce5 (0xbce6 bytes) 2024-09-17T21:19:29.308 INFO:teuthology.orchestra.run.smithi184.stdout:3401 trunc from 0xbaf7a00 to 0xc528600 2024-09-17T21:19:29.344 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:29.340+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:29.369 INFO:teuthology.orchestra.run.smithi184.stdout:3402 trunc from 0xc528600 to 0x26be000 2024-09-17T21:19:29.401 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:29.402 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:29.417 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:19:29.447 INFO:teuthology.orchestra.run.smithi184.stdout:3403 read 0xf1c800 thru 0xf23ea1 (0x76a2 bytes) 2024-09-17T21:19:29.458 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:29.456+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:29.492 INFO:teuthology.orchestra.run.smithi184.stdout:3407 write 0x7e5fe00 thru 0x7e6c4b7 (0xc6b8 bytes) 2024-09-17T21:19:29.493 INFO:teuthology.orchestra.run.smithi184.stdout:3409 read 0x1c31000 thru 0x1c3d2c9 (0xc2ca bytes) 2024-09-17T21:19:29.510 INFO:teuthology.orchestra.run.smithi184.stdout:3413 trunc from 0x7e6c600 to 0xe522400 2024-09-17T21:19:29.545 INFO:teuthology.orchestra.run.smithi184.stdout:3415 read 0x8faee00 thru 0x8fb4cc0 (0x5ec1 bytes) 2024-09-17T21:19:29.546 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:29.544+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:29.546 INFO:teuthology.orchestra.run.smithi184.stdout:3416 read 0x61fb600 thru 0x6206974 (0xb375 bytes) 2024-09-17T21:19:29.546 INFO:teuthology.orchestra.run.smithi184.stdout:3418 read 0x1dd200 thru 0x1ea8ea (0xd6eb bytes) 2024-09-17T21:19:29.578 INFO:teuthology.orchestra.run.smithi184.stdout:3421 trunc from 0xe522400 to 0x416c400 2024-09-17T21:19:29.592 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:29.592 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:29.608 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:19:29.622 INFO:teuthology.orchestra.run.smithi184.stdout:3425 punch from 0x28e4600 to 0x28f2a00, (0xe400 bytes) 2024-09-17T21:19:29.625 INFO:teuthology.orchestra.run.smithi184.stdout:3427 read 0x3dd4600 thru 0x3dddade (0x94df bytes) 2024-09-17T21:19:29.647 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:19:29.649 INFO:teuthology.orchestra.run.smithi184.stdout:3428 write 0x653d600 thru 0x6543dc8 (0x67c9 bytes) 2024-09-17T21:19:29.650 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:29.648+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:29.651 INFO:teuthology.orchestra.run.smithi184.stdout:3429 read 0x23ab800 thru 0x23b316c (0x796d bytes) 2024-09-17T21:19:29.651 INFO:teuthology.orchestra.run.smithi184.stdout:3430 read 0x5731600 thru 0x573d6df (0xc0e0 bytes) 2024-09-17T21:19:29.657 INFO:teuthology.orchestra.run.smithi184.stdout:3431 trunc from 0x6543e00 to 0x4008600 2024-09-17T21:19:29.714 INFO:teuthology.orchestra.run.smithi184.stdout:3432 write 0x98ff400 thru 0x99042b8 (0x4eb9 bytes) 2024-09-17T21:19:29.715 INFO:teuthology.orchestra.run.smithi184.stdout:3433 write 0x6b9a400 thru 0x6ba738a (0xcf8b bytes) 2024-09-17T21:19:29.715 INFO:teuthology.orchestra.run.smithi184.stdout:3435 read 0x4ee8600 thru 0x4eec7cf (0x41d0 bytes) 2024-09-17T21:19:29.716 INFO:teuthology.orchestra.run.smithi184.stdout:3439 read 0x27fa800 thru 0x2804eb6 (0xa6b7 bytes) 2024-09-17T21:19:29.716 INFO:teuthology.orchestra.run.smithi184.stdout:3441 punch from 0x8f5e400 to 0x8f60600, (0x2200 bytes) 2024-09-17T21:19:29.716 INFO:teuthology.orchestra.run.smithi184.stdout:3445 read 0x6285800 thru 0x6291241 (0xba42 bytes) 2024-09-17T21:19:29.716 INFO:teuthology.orchestra.run.smithi184.stdout:3447 punch from 0x7613800 to 0x7619c00, (0x6400 bytes) 2024-09-17T21:19:29.721 INFO:teuthology.orchestra.run.smithi184.stdout:3448 trunc from 0x9904400 to 0xc6a6a00 2024-09-17T21:19:29.747 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:29.742+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:29.751 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:29.752 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:29.754 INFO:teuthology.orchestra.run.smithi184.stdout:3449 write 0xf9d800 thru 0xf9fa02 (0x2203 bytes) 2024-09-17T21:19:29.756 INFO:teuthology.orchestra.run.smithi184.stdout:3450 read 0x5c25400 thru 0x5c2fcc2 (0xa8c3 bytes) 2024-09-17T21:19:29.756 INFO:teuthology.orchestra.run.smithi184.stdout:3453 read 0x35f9400 thru 0x360888a (0xf48b bytes) 2024-09-17T21:19:29.757 INFO:teuthology.orchestra.run.smithi184.stdout:3455 punch from 0xad18800 to 0xad27200, (0xea00 bytes) 2024-09-17T21:19:29.769 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:19:29.780 INFO:teuthology.orchestra.run.smithi184.stdout:3458 write 0xccdb000 thru 0xcce5c72 (0xac73 bytes) 2024-09-17T21:19:29.783 INFO:teuthology.orchestra.run.smithi184.stdout:3460 write 0xb108400 thru 0xb11505b (0xcc5c bytes) 2024-09-17T21:19:29.783 INFO:teuthology.orchestra.run.smithi184.stdout:3462 punch from 0x4169c00 to 0x4169e00, (0x200 bytes) 2024-09-17T21:19:29.784 INFO:teuthology.orchestra.run.smithi184.stdout:3463 punch from 0x451000 to 0x45c200, (0xb200 bytes) 2024-09-17T21:19:29.784 INFO:teuthology.orchestra.run.smithi184.stdout:3468 punch from 0x81a9000 to 0x81ac400, (0x3400 bytes) 2024-09-17T21:19:29.785 INFO:teuthology.orchestra.run.smithi184.stdout:3469 flatten 2024-09-17T21:19:29.851 INFO:teuthology.orchestra.run.smithi184.stdout:3473 read 0x1902a00 thru 0x1905998 (0x2f99 bytes) 2024-09-17T21:19:29.855 INFO:teuthology.orchestra.run.smithi184.stdout:3474 trunc from 0xcce5e00 to 0xe13d400 2024-09-17T21:19:29.856 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:29.854+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:29.869 INFO:teuthology.orchestra.run.smithi184.stdout:3475 read 0x46ac00 thru 0x4743d0 (0x97d1 bytes) 2024-09-17T21:19:29.871 INFO:teuthology.orchestra.run.smithi184.stdout:3480 write 0x51d7200 thru 0x51d79e1 (0x7e2 bytes) 2024-09-17T21:19:29.871 INFO:teuthology.orchestra.run.smithi184.stdout:3481 read 0x459cc00 thru 0x45a8db4 (0xc1b5 bytes) 2024-09-17T21:19:29.872 INFO:teuthology.orchestra.run.smithi184.stdout:3482 punch from 0x50cae00 to 0x50d7000, (0xc200 bytes) 2024-09-17T21:19:29.872 INFO:teuthology.orchestra.run.smithi184.stdout:3483 write 0x780b400 thru 0x7817bc6 (0xc7c7 bytes) 2024-09-17T21:19:29.895 INFO:teuthology.orchestra.run.smithi184.stdout:3484 trunc from 0xe13d400 to 0x20f7e00 2024-09-17T21:19:29.937 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:29.938 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:29.944 INFO:teuthology.orchestra.run.smithi184.stdout:3485 trunc from 0x20f7e00 to 0xc49600 2024-09-17T21:19:29.949 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:29.944+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:29.952 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:19:29.969 INFO:teuthology.orchestra.run.smithi184.stdout:3486 read 0x50fc00 thru 0x51e980 (0xed81 bytes) 2024-09-17T21:19:29.976 INFO:teuthology.orchestra.run.smithi184.stdout:3487 trunc from 0xc49600 to 0x220b200 2024-09-17T21:19:29.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T21:19:29.990+0000 7f7553fff640 0 --2- 172.21.15.83:0/422150656 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7554134db0 0x7f755411e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-17T21:19:30.016 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:30.017 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:30.020 INFO:teuthology.orchestra.run.smithi184.stdout:3488 write 0x6458a00 thru 0x645d930 (0x4f31 bytes) 2024-09-17T21:19:30.033 INFO:teuthology.orchestra.run.smithi184.stdout:3490 trunc from 0x645da00 to 0xc112000 2024-09-17T21:19:30.034 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:30.034 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:30.049 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:30.046+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:30.053 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:30.053 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:30.064 INFO:teuthology.orchestra.run.smithi184.stdout:3491 trunc from 0xc112000 to 0x6bcc000 2024-09-17T21:19:30.076 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:30.076 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:30.078 INFO:teuthology.orchestra.run.smithi184.stdout:3493 write 0x1f29c00 thru 0x1f38182 (0xe583 bytes) 2024-09-17T21:19:30.097 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:30.097 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:30.115 INFO:teuthology.orchestra.run.smithi184.stdout:3495 write 0xe311000 thru 0xe312cf1 (0x1cf2 bytes) 2024-09-17T21:19:30.117 INFO:teuthology.orchestra.run.smithi184.stdout:3501 write 0xbc9ee00 thru 0xbca08bf (0x1ac0 bytes) 2024-09-17T21:19:30.118 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:30.118 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:30.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:30.118 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:30.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.119 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:406 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 2359296~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.119 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:19:30.119 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.921353582999998, 2024-09-17T21:19:30.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042448018999999997, 2024-09-17T21:19:30.119 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.119 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.119 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.119 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.119 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.119 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.119 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 406 2024-09-17T21:19:30.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.119 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:19:30.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:19:30.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195056+0000", 2024-09-17T21:19:30.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2729999999999999e-06 2024-09-17T21:19:30.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195060+0000", 2024-09-17T21:19:30.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5399999999999997e-06 2024-09-17T21:19:30.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195076+0000", 2024-09-17T21:19:30.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6217000000000001e-05 2024-09-17T21:19:30.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195093+0000", 2024-09-17T21:19:30.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.683e-05 2024-09-17T21:19:30.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236217+0000", 2024-09-17T21:19:30.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041124102000000003 2024-09-17T21:19:30.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.123 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236339+0000", 2024-09-17T21:19:30.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121876 2024-09-17T21:19:30.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.123 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.123 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237502+0000", 2024-09-17T21:19:30.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011631809999999999 2024-09-17T21:19:30.123 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.123 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.123 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.123 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:416 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2228224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.124 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:19:30.124 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.920855557000003, 2024-09-17T21:19:30.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042361990000000002, 2024-09-17T21:19:30.124 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.124 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.124 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.124 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.124 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.124 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.124 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 416 2024-09-17T21:19:30.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.124 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:19:30.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:19:30.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195554+0000", 2024-09-17T21:19:30.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9300000000000002e-06 2024-09-17T21:19:30.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195563+0000", 2024-09-17T21:19:30.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2770000000000005e-06 2024-09-17T21:19:30.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195586+0000", 2024-09-17T21:19:30.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3889000000000001e-05 2024-09-17T21:19:30.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195603+0000", 2024-09-17T21:19:30.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6084000000000001e-05 2024-09-17T21:19:30.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237563+0000", 2024-09-17T21:19:30.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041960856999999997 2024-09-17T21:19:30.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237695+0000", 2024-09-17T21:19:30.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013144400000000001 2024-09-17T21:19:30.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237914+0000", 2024-09-17T21:19:30.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000219509 2024-09-17T21:19:30.128 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.128 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.128 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.128 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:439 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.128 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:19:30.128 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.920291655, 2024-09-17T21:19:30.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043053811999999997, 2024-09-17T21:19:30.128 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.129 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.129 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.129 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.129 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.129 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.129 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 439 2024-09-17T21:19:30.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.129 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.129 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.129 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:19:30.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:19:30.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196118+0000", 2024-09-17T21:19:30.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3969999999999999e-06 2024-09-17T21:19:30.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196157+0000", 2024-09-17T21:19:30.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9826999999999997e-05 2024-09-17T21:19:30.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196179+0000", 2024-09-17T21:19:30.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1756e-05 2024-09-17T21:19:30.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196194+0000", 2024-09-17T21:19:30.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4707e-05 2024-09-17T21:19:30.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237980+0000", 2024-09-17T21:19:30.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041786219999999999 2024-09-17T21:19:30.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.238065+0000", 2024-09-17T21:19:30.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4560000000000004e-05 2024-09-17T21:19:30.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239170+0000", 2024-09-17T21:19:30.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001105345 2024-09-17T21:19:30.132 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.132 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.132 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.133 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:440 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.133 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:19:30.133 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.920189151999999, 2024-09-17T21:19:30.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043890039999999998, 2024-09-17T21:19:30.133 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.133 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.133 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.133 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.133 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.133 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.133 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 440 2024-09-17T21:19:30.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.134 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:19:30.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:19:30.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196220+0000", 2024-09-17T21:19:30.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.254e-06 2024-09-17T21:19:30.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196225+0000", 2024-09-17T21:19:30.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7829999999999998e-06 2024-09-17T21:19:30.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196241+0000", 2024-09-17T21:19:30.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6322999999999999e-05 2024-09-17T21:19:30.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196258+0000", 2024-09-17T21:19:30.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7056000000000001e-05 2024-09-17T21:19:30.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239229+0000", 2024-09-17T21:19:30.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042971058999999999 2024-09-17T21:19:30.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.137 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239403+0000", 2024-09-17T21:19:30.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017399200000000001 2024-09-17T21:19:30.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.137 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.137 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240109+0000", 2024-09-17T21:19:30.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00070557300000000003 2024-09-17T21:19:30.137 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.137 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.137 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.137 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:450 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.137 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:19:30.138 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.919828201000001, 2024-09-17T21:19:30.138 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043994521000000002, 2024-09-17T21:19:30.138 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.138 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.138 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.138 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.138 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.138 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.138 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 450 2024-09-17T21:19:30.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.138 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.138 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.138 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:19:30.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:19:30.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196581+0000", 2024-09-17T21:19:30.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4500000000000001e-06 2024-09-17T21:19:30.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.140 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196588+0000", 2024-09-17T21:19:30.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7530000000000004e-06 2024-09-17T21:19:30.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.140 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196607+0000", 2024-09-17T21:19:30.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8987e-05 2024-09-17T21:19:30.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.140 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196620+0000", 2024-09-17T21:19:30.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3589e-05 2024-09-17T21:19:30.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240225+0000", 2024-09-17T21:19:30.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043604878999999999 2024-09-17T21:19:30.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240377+0000", 2024-09-17T21:19:30.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015146699999999999 2024-09-17T21:19:30.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240574+0000", 2024-09-17T21:19:30.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019739599999999999 2024-09-17T21:19:30.142 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.142 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.142 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.142 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:464 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.142 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:19:30.142 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.919577779999997, 2024-09-17T21:19:30.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044651692999999999, 2024-09-17T21:19:30.142 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.143 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.143 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.143 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.143 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.143 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.143 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 464 2024-09-17T21:19:30.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.143 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.143 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.143 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:19:30.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:19:30.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196838+0000", 2024-09-17T21:19:30.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4970000000000004e-06 2024-09-17T21:19:30.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196859+0000", 2024-09-17T21:19:30.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1166e-05 2024-09-17T21:19:30.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196879+0000", 2024-09-17T21:19:30.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.995e-05 2024-09-17T21:19:30.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196896+0000", 2024-09-17T21:19:30.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6929999999999999e-05 2024-09-17T21:19:30.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240617+0000", 2024-09-17T21:19:30.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043721784999999999 2024-09-17T21:19:30.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.146 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240702+0000", 2024-09-17T21:19:30.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4060999999999994e-05 2024-09-17T21:19:30.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.146 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241482+0000", 2024-09-17T21:19:30.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000780304 2024-09-17T21:19:30.146 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.146 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.147 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.147 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:475 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.147 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:19:30.147 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.919308647999998, 2024-09-17T21:19:30.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044661981000000003, 2024-09-17T21:19:30.147 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.147 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.147 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.147 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.147 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.147 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.147 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 475 2024-09-17T21:19:30.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.148 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:19:30.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:19:30.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197100+0000", 2024-09-17T21:19:30.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.206e-06 2024-09-17T21:19:30.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197105+0000", 2024-09-17T21:19:30.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.527e-06 2024-09-17T21:19:30.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197124+0000", 2024-09-17T21:19:30.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:19:30.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.150 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.150 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197136+0000", 2024-09-17T21:19:30.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2857000000000001e-05 2024-09-17T21:19:30.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.150 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.150 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241539+0000", 2024-09-17T21:19:30.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044402077999999998 2024-09-17T21:19:30.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.150 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.150 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241653+0000", 2024-09-17T21:19:30.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114931 2024-09-17T21:19:30.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.151 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.151 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241761+0000", 2024-09-17T21:19:30.151 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107753 2024-09-17T21:19:30.151 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.151 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.151 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.151 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:476 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:19:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.919254823000003, 2024-09-17T21:19:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045038976000000001, 2024-09-17T21:19:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 476 2024-09-17T21:19:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:19:30.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:19:30.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197154+0000", 2024-09-17T21:19:30.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.127e-06 2024-09-17T21:19:30.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197159+0000", 2024-09-17T21:19:30.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4569999999999998e-06 2024-09-17T21:19:30.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197174+0000", 2024-09-17T21:19:30.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5173e-05 2024-09-17T21:19:30.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.155 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197186+0000", 2024-09-17T21:19:30.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:19:30.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.155 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241863+0000", 2024-09-17T21:19:30.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044676826000000003 2024-09-17T21:19:30.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.155 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241998+0000", 2024-09-17T21:19:30.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013476100000000001 2024-09-17T21:19:30.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.156 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.156 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242192+0000", 2024-09-17T21:19:30.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019436900000000001 2024-09-17T21:19:30.156 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.156 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.156 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.156 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:494 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.156 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:30.156 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.91907964, 2024-09-17T21:19:30.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046244995999999997, 2024-09-17T21:19:30.156 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.157 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.157 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.157 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.157 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.157 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.157 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 494 2024-09-17T21:19:30.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.157 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.157 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.157 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.157 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:30.157 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.157 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:30.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197329+0000", 2024-09-17T21:19:30.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.173e-06 2024-09-17T21:19:30.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197334+0000", 2024-09-17T21:19:30.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0499999999999999e-06 2024-09-17T21:19:30.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197352+0000", 2024-09-17T21:19:30.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7133e-05 2024-09-17T21:19:30.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197367+0000", 2024-09-17T21:19:30.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5123000000000001e-05 2024-09-17T21:19:30.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:19:30.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044989829000000002 2024-09-17T21:19:30.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242526+0000", 2024-09-17T21:19:30.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016968199999999999 2024-09-17T21:19:30.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243573+0000", 2024-09-17T21:19:30.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010470060000000001 2024-09-17T21:19:30.160 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.160 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.160 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.161 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:502 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.161 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:19:30.161 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.916845981000002, 2024-09-17T21:19:30.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044346086999999999, 2024-09-17T21:19:30.161 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.161 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.161 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.161 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.161 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.161 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.161 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 502 2024-09-17T21:19:30.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.162 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:19:30.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:19:30.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.163 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199563+0000", 2024-09-17T21:19:30.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2929999999999999e-06 2024-09-17T21:19:30.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.163 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.163 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199569+0000", 2024-09-17T21:19:30.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5330000000000002e-06 2024-09-17T21:19:30.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.163 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.163 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199614+0000", 2024-09-17T21:19:30.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4793000000000003e-05 2024-09-17T21:19:30.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199628+0000", 2024-09-17T21:19:30.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4037e-05 2024-09-17T21:19:30.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243665+0000", 2024-09-17T21:19:30.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044037106999999999 2024-09-17T21:19:30.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243767+0000", 2024-09-17T21:19:30.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010279499999999999 2024-09-17T21:19:30.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243908+0000", 2024-09-17T21:19:30.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140529 2024-09-17T21:19:30.165 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.165 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.165 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.165 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:523 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.165 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:19:30.165 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.916399828000003, 2024-09-17T21:19:30.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044789645000000003, 2024-09-17T21:19:30.166 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.166 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.166 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.166 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.166 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.166 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.166 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 523 2024-09-17T21:19:30.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.166 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:19:30.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:19:30.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:19:30.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9e-06 2024-09-17T21:19:30.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200021+0000", 2024-09-17T21:19:30.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9499999999999998e-06 2024-09-17T21:19:30.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200038+0000", 2024-09-17T21:19:30.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.73e-05 2024-09-17T21:19:30.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200051+0000", 2024-09-17T21:19:30.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.322e-05 2024-09-17T21:19:30.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.169 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.169 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243987+0000", 2024-09-17T21:19:30.169 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043935386999999999 2024-09-17T21:19:30.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.169 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.169 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244136+0000", 2024-09-17T21:19:30.169 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014868300000000001 2024-09-17T21:19:30.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.169 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.169 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244798+0000", 2024-09-17T21:19:30.169 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00066220500000000002 2024-09-17T21:19:30.170 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.170 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.170 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.170 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:524 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.170 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:30.170 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.916333612000003, 2024-09-17T21:19:30.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045737414999999997, 2024-09-17T21:19:30.170 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.170 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.170 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.170 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.170 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.171 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.171 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 524 2024-09-17T21:19:30.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.171 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:30.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:30.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200076+0000", 2024-09-17T21:19:30.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:19:30.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200081+0000", 2024-09-17T21:19:30.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7030000000000003e-06 2024-09-17T21:19:30.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200099+0000", 2024-09-17T21:19:30.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7836999999999999e-05 2024-09-17T21:19:30.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200113+0000", 2024-09-17T21:19:30.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3570000000000001e-05 2024-09-17T21:19:30.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244866+0000", 2024-09-17T21:19:30.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044753573999999997 2024-09-17T21:19:30.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244972+0000", 2024-09-17T21:19:30.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106111 2024-09-17T21:19:30.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245812+0000", 2024-09-17T21:19:30.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083923999999999999 2024-09-17T21:19:30.174 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.174 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.174 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.174 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:525 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3670016~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.174 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:30.175 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.916182229999997, 2024-09-17T21:19:30.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046645697, 2024-09-17T21:19:30.175 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.175 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.175 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.175 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.175 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.175 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.175 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 525 2024-09-17T21:19:30.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.175 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:30.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.176 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.176 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:30.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.176 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.176 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:19:30.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:19:30.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.176 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.177 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200234+0000", 2024-09-17T21:19:30.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:19:30.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.177 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.177 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200254+0000", 2024-09-17T21:19:30.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9959999999999999e-05 2024-09-17T21:19:30.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.177 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200293+0000", 2024-09-17T21:19:30.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9755999999999997e-05 2024-09-17T21:19:30.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245871+0000", 2024-09-17T21:19:30.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045577871999999998 2024-09-17T21:19:30.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246030+0000", 2024-09-17T21:19:30.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158975 2024-09-17T21:19:30.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246871+0000", 2024-09-17T21:19:30.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084111699999999995 2024-09-17T21:19:30.179 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.179 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.179 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.179 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.179 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:30.179 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.915936555000002, 2024-09-17T21:19:30.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:19:30.179 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.179 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.180 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.180 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.180 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.180 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.180 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:19:30.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.180 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:30.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:30.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.182 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:19:30.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:19:30.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.182 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.182 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:19:30.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:19:30.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.182 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.182 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:19:30.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:19:30.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:19:30.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:19:30.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:19:30.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:19:30.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:19:30.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:19:30.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:19:30.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:19:30.184 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.184 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.184 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.184 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.184 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:30.185 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.915757872999997, 2024-09-17T21:19:30.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:19:30.185 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.185 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.185 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.185 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.185 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.185 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.185 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:19:30.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.185 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:19:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:19:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:19:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:19:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:19:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:19:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:19:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:19:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:19:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:19:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:19:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:19:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:19:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:19:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.915698599999999, 2024-09-17T21:19:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:19:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:19:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:19:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:19:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:19:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:19:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:19:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:19:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:19:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:19:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:19:30.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:19:30.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:19:30.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:19:30.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:19:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:19:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.915556404999997, 2024-09-17T21:19:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:19:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:19:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:19:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:19:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:19:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:19:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:19:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:19:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:19:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:19:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:19:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:19:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:19:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:19:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:19:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:19:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.915500481999999, 2024-09-17T21:19:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:19:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:19:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:19:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:19:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:19:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:19:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:19:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:19:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:19:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:19:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:19:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:19:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:19:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:19:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:19:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:19:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.915446865, 2024-09-17T21:19:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:19:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:19:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:19:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:19:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:19:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:19:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:19:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:19:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:19:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:19:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:19:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:19:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:19:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:19:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:19:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:19:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:19:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:19:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 13.606218623, 2024-09-17T21:19:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:19:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:19:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:19:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:19:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:19:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:19:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:19:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:19:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:19:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:19:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:19:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:19:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:19:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:19:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:19:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:19:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:19:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:19:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:19:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:19:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:19:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:19:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:19:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.213 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:30.217 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:30.217 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:30.218 INFO:teuthology.orchestra.run.smithi184.stdout:3503 trunc from 0xe312e00 to 0x5a2f400 2024-09-17T21:19:30.218 INFO:teuthology.orchestra.run.smithi184.stdout:3504 read 0x368a000 thru 0x368dc63 (0x3c64 bytes) 2024-09-17T21:19:30.218 INFO:teuthology.orchestra.run.smithi184.stdout:3505 write 0xab6ee00 thru 0xab77f2c (0x912d bytes) 2024-09-17T21:19:30.218 INFO:teuthology.orchestra.run.smithi184.stdout:3506 read 0x60600 thru 0x61597 (0xf98 bytes) 2024-09-17T21:19:30.218 INFO:teuthology.orchestra.run.smithi184.stdout:3507 read 0x8349200 thru 0x8354105 (0xaf06 bytes) 2024-09-17T21:19:30.218 INFO:teuthology.orchestra.run.smithi184.stdout:3508 trunc from 0xab78000 to 0xafe800 2024-09-17T21:19:30.219 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:19:30.220 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:30.214+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:30.262 INFO:teuthology.orchestra.run.smithi184.stdout:3513 write 0x4b72a00 thru 0x4b74b52 (0x2153 bytes) 2024-09-17T21:19:30.265 INFO:teuthology.orchestra.run.smithi184.stdout:3514 read 0x1dc4c00 thru 0x1dc78cb (0x2ccc bytes) 2024-09-17T21:19:30.265 INFO:teuthology.orchestra.run.smithi184.stdout:3517 read 0x336ea00 thru 0x337da2c (0xf02d bytes) 2024-09-17T21:19:30.266 INFO:teuthology.orchestra.run.smithi184.stdout:3520 trunc from 0x4b74c00 to 0x4766600 2024-09-17T21:19:30.316 INFO:teuthology.orchestra.run.smithi184.stdout:3522 write 0x8bed000 thru 0x8bf236c (0x536d bytes) 2024-09-17T21:19:30.318 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:30.314+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:30.318 INFO:teuthology.orchestra.run.smithi184.stdout:3524 read 0x4e6bc00 thru 0x4e78764 (0xcb65 bytes) 2024-09-17T21:19:30.319 INFO:teuthology.orchestra.run.smithi184.stdout:3525 read 0x26d2c00 thru 0x26df8d2 (0xccd3 bytes) 2024-09-17T21:19:30.338 INFO:teuthology.orchestra.run.smithi184.stdout:3526 trunc from 0x8bf2400 to 0xc07800 2024-09-17T21:19:30.380 INFO:teuthology.orchestra.run.smithi184.stdout:3527 punch from 0x3c6000 to 0x3c7e00, (0x1e00 bytes) 2024-09-17T21:19:30.397 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:30.397 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:30.397 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:19:30.397 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:30.410 INFO:teuthology.orchestra.run.smithi184.stdout:3529 write 0x142b000 thru 0x142c5f9 (0x15fa bytes) 2024-09-17T21:19:30.412 INFO:teuthology.orchestra.run.smithi184.stdout:3531 write 0xd33c00 thru 0xd38ab0 (0x4eb1 bytes) 2024-09-17T21:19:30.412 INFO:teuthology.orchestra.run.smithi184.stdout:3532 read 0x46ca00 thru 0x470b05 (0x4106 bytes) 2024-09-17T21:19:30.413 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:19:30.450 INFO:teuthology.orchestra.run.smithi184.stdout:3534 write 0x4eafa00 thru 0x4eb30bf (0x36c0 bytes) 2024-09-17T21:19:30.457 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:30.454+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:30.464 INFO:teuthology.orchestra.run.smithi184.stdout:3535 trunc from 0x4eb3200 to 0x9268600 2024-09-17T21:19:30.489 INFO:teuthology.orchestra.run.smithi184.stdout:3541 read 0x5929a00 thru 0x59370ad (0xd6ae bytes) 2024-09-17T21:19:30.500 INFO:teuthology.orchestra.run.smithi184.stdout:3542 trunc from 0x9268600 to 0xc235600 2024-09-17T21:19:30.521 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:30.518+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:30.522 INFO:teuthology.orchestra.run.smithi184.stdout:3543 clone 19 order 23 su 131072 sc 9 2024-09-17T21:19:30.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:19:30.585 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:30.585 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:30.585 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:30.585 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:30.586 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:30.601 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:19:30.648 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:30.646+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:30.726 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:30.722+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:30.767 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:30.767 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:30.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:491 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:19:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.566436612999993, 2024-09-17T21:19:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041839013000000001, 2024-09-17T21:19:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 491 2024-09-17T21:19:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:19:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:19:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198315+0000", 2024-09-17T21:19:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6599999999999996e-07 2024-09-17T21:19:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198317+0000", 2024-09-17T21:19:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6639999999999998e-06 2024-09-17T21:19:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198327+0000", 2024-09-17T21:19:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5629999999999998e-06 2024-09-17T21:19:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198337+0000", 2024-09-17T21:19:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0667e-05 2024-09-17T21:19:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239793+0000", 2024-09-17T21:19:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041455519000000003 2024-09-17T21:19:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239929+0000", 2024-09-17T21:19:30.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135716 2024-09-17T21:19:30.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240153+0000", 2024-09-17T21:19:30.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022421799999999999 2024-09-17T21:19:30.772 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.772 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:492 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:19:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.566398242999995, 2024-09-17T21:19:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042947165000000002, 2024-09-17T21:19:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 492 2024-09-17T21:19:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:19:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:19:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198354+0000", 2024-09-17T21:19:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:19:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198357+0000", 2024-09-17T21:19:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6969999999999999e-06 2024-09-17T21:19:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198367+0000", 2024-09-17T21:19:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8430000000000005e-06 2024-09-17T21:19:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198374+0000", 2024-09-17T21:19:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9600000000000003e-06 2024-09-17T21:19:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240226+0000", 2024-09-17T21:19:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041852157000000001 2024-09-17T21:19:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240409+0000", 2024-09-17T21:19:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018244800000000001 2024-09-17T21:19:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241299+0000", 2024-09-17T21:19:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000890554 2024-09-17T21:19:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:493 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 4063232~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:19:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.566361157000003, 2024-09-17T21:19:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043897209, 2024-09-17T21:19:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 493 2024-09-17T21:19:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:19:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:19:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198391+0000", 2024-09-17T21:19:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4699999999999999e-06 2024-09-17T21:19:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198396+0000", 2024-09-17T21:19:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:19:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198412+0000", 2024-09-17T21:19:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6507e-05 2024-09-17T21:19:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198425+0000", 2024-09-17T21:19:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.256e-05 2024-09-17T21:19:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241383+0000", 2024-09-17T21:19:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042958216 2024-09-17T21:19:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241525+0000", 2024-09-17T21:19:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014204500000000001 2024-09-17T21:19:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242287+0000", 2024-09-17T21:19:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076125800000000001 2024-09-17T21:19:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:496 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:19:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.566270758000002, 2024-09-17T21:19:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043732500000000001, 2024-09-17T21:19:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 496 2024-09-17T21:19:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:19:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:19:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198481+0000", 2024-09-17T21:19:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-17T21:19:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198488+0000", 2024-09-17T21:19:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8499999999999996e-06 2024-09-17T21:19:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198507+0000", 2024-09-17T21:19:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9276999999999999e-05 2024-09-17T21:19:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198521+0000", 2024-09-17T21:19:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3376e-05 2024-09-17T21:19:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241339+0000", 2024-09-17T21:19:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042818119000000002 2024-09-17T21:19:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241449+0000", 2024-09-17T21:19:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110209 2024-09-17T21:19:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242212+0000", 2024-09-17T21:19:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076329900000000005 2024-09-17T21:19:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:497 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:19:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.566189808999994, 2024-09-17T21:19:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044584278999999997, 2024-09-17T21:19:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 497 2024-09-17T21:19:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:19:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:19:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198562+0000", 2024-09-17T21:19:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-17T21:19:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198569+0000", 2024-09-17T21:19:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.494e-06 2024-09-17T21:19:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198588+0000", 2024-09-17T21:19:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8955999999999999e-05 2024-09-17T21:19:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198621+0000", 2024-09-17T21:19:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3837000000000001e-05 2024-09-17T21:19:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242258+0000", 2024-09-17T21:19:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043636900999999999 2024-09-17T21:19:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242343+0000", 2024-09-17T21:19:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5048000000000003e-05 2024-09-17T21:19:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243145+0000", 2024-09-17T21:19:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00080167300000000003 2024-09-17T21:19:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:503 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:19:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.566105285000006, 2024-09-17T21:19:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044220582000000001, 2024-09-17T21:19:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 503 2024-09-17T21:19:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:19:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:19:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198647+0000", 2024-09-17T21:19:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.483e-06 2024-09-17T21:19:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198653+0000", 2024-09-17T21:19:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4529999999999999e-06 2024-09-17T21:19:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198672+0000", 2024-09-17T21:19:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9193000000000001e-05 2024-09-17T21:19:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198685+0000", 2024-09-17T21:19:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.291e-05 2024-09-17T21:19:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:19:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043672141999999997 2024-09-17T21:19:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242716+0000", 2024-09-17T21:19:30.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00035901800000000001 2024-09-17T21:19:30.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242866+0000", 2024-09-17T21:19:30.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000149383 2024-09-17T21:19:30.797 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.797 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.797 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.797 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:504 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.797 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:19:30.798 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.566045493000004, 2024-09-17T21:19:30.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044611406999999999, 2024-09-17T21:19:30.798 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.798 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.798 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.798 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.798 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.798 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.798 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 504 2024-09-17T21:19:30.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.798 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:19:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:19:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198706+0000", 2024-09-17T21:19:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:19:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198712+0000", 2024-09-17T21:19:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5770000000000001e-06 2024-09-17T21:19:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198729+0000", 2024-09-17T21:19:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6673000000000001e-05 2024-09-17T21:19:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198742+0000", 2024-09-17T21:19:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3543000000000001e-05 2024-09-17T21:19:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242981+0000", 2024-09-17T21:19:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044238953999999997 2024-09-17T21:19:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243155+0000", 2024-09-17T21:19:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017401600000000001 2024-09-17T21:19:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243316+0000", 2024-09-17T21:19:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016137399999999999 2024-09-17T21:19:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:521 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:19:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.565498353999999, 2024-09-17T21:19:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045043685999999999, 2024-09-17T21:19:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 521 2024-09-17T21:19:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:19:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:19:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199253+0000", 2024-09-17T21:19:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1400000000000001e-06 2024-09-17T21:19:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199258+0000", 2024-09-17T21:19:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1800000000000004e-06 2024-09-17T21:19:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199276+0000", 2024-09-17T21:19:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7686999999999999e-05 2024-09-17T21:19:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199289+0000", 2024-09-17T21:19:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3263e-05 2024-09-17T21:19:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243397+0000", 2024-09-17T21:19:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044107636999999998 2024-09-17T21:19:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243527+0000", 2024-09-17T21:19:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013038399999999999 2024-09-17T21:19:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244296+0000", 2024-09-17T21:19:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076839499999999995 2024-09-17T21:19:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:522 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:19:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.565443051000003, 2024-09-17T21:19:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046049870999999999, 2024-09-17T21:19:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 522 2024-09-17T21:19:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:19:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:19:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199309+0000", 2024-09-17T21:19:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:19:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199315+0000", 2024-09-17T21:19:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2440000000000001e-06 2024-09-17T21:19:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199333+0000", 2024-09-17T21:19:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7886000000000001e-05 2024-09-17T21:19:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199379+0000", 2024-09-17T21:19:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6383e-05 2024-09-17T21:19:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244394+0000", 2024-09-17T21:19:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045014851000000002 2024-09-17T21:19:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244491+0000", 2024-09-17T21:19:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7245999999999997e-05 2024-09-17T21:19:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245357+0000", 2024-09-17T21:19:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00086626800000000001 2024-09-17T21:19:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:527 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:19:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.565346384999998, 2024-09-17T21:19:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045812522000000001, 2024-09-17T21:19:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 527 2024-09-17T21:19:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:19:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:19:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199406+0000", 2024-09-17T21:19:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.367e-06 2024-09-17T21:19:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199412+0000", 2024-09-17T21:19:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:19:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199429+0000", 2024-09-17T21:19:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7396000000000001e-05 2024-09-17T21:19:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199442+0000", 2024-09-17T21:19:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3327e-05 2024-09-17T21:19:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244361+0000", 2024-09-17T21:19:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044918197 2024-09-17T21:19:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244482+0000", 2024-09-17T21:19:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121945 2024-09-17T21:19:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245217+0000", 2024-09-17T21:19:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00073415999999999998 2024-09-17T21:19:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:534 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:19:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.565287678999994, 2024-09-17T21:19:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046292505999999997, 2024-09-17T21:19:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:19:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:19:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:19:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199464+0000", 2024-09-17T21:19:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:19:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199473+0000", 2024-09-17T21:19:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5629999999999992e-06 2024-09-17T21:19:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199492+0000", 2024-09-17T21:19:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9273999999999999e-05 2024-09-17T21:19:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199505+0000", 2024-09-17T21:19:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3443e-05 2024-09-17T21:19:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245357+0000", 2024-09-17T21:19:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045851714000000002 2024-09-17T21:19:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245550+0000", 2024-09-17T21:19:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019301600000000001 2024-09-17T21:19:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245755+0000", 2024-09-17T21:19:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205226 2024-09-17T21:19:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:535 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:19:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.565183122999997, 2024-09-17T21:19:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046637492000000003, 2024-09-17T21:19:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 535 2024-09-17T21:19:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:19:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:19:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199568+0000", 2024-09-17T21:19:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.037e-06 2024-09-17T21:19:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199572+0000", 2024-09-17T21:19:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5729999999999999e-06 2024-09-17T21:19:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199585+0000", 2024-09-17T21:19:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.279e-05 2024-09-17T21:19:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199592+0000", 2024-09-17T21:19:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.61e-06 2024-09-17T21:19:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245873+0000", 2024-09-17T21:19:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046280494999999998 2024-09-17T21:19:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.828 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.828 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246018+0000", 2024-09-17T21:19:30.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000144697 2024-09-17T21:19:30.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.828 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.828 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246205+0000", 2024-09-17T21:19:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018729 2024-09-17T21:19:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:543 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.565087876999996, 2024-09-17T21:19:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047564097999999999, 2024-09-17T21:19:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 543 2024-09-17T21:19:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9699999999999995e-07 2024-09-17T21:19:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199677+0000", 2024-09-17T21:19:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:19:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199690+0000", 2024-09-17T21:19:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2977e-05 2024-09-17T21:19:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199705+0000", 2024-09-17T21:19:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4656e-05 2024-09-17T21:19:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246307+0000", 2024-09-17T21:19:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046601780000000002 2024-09-17T21:19:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246475+0000", 2024-09-17T21:19:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016834200000000001 2024-09-17T21:19:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247227+0000", 2024-09-17T21:19:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00075173299999999998 2024-09-17T21:19:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:544 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.565029468000006, 2024-09-17T21:19:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049283559999999997, 2024-09-17T21:19:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 544 2024-09-17T21:19:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199722+0000", 2024-09-17T21:19:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.08e-06 2024-09-17T21:19:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199738+0000", 2024-09-17T21:19:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5943999999999998e-05 2024-09-17T21:19:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199748+0000", 2024-09-17T21:19:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0193e-05 2024-09-17T21:19:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199756+0000", 2024-09-17T21:19:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4200000000000001e-06 2024-09-17T21:19:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247311+0000", 2024-09-17T21:19:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047555300000000002 2024-09-17T21:19:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247571+0000", 2024-09-17T21:19:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025996999999999999 2024-09-17T21:19:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249005+0000", 2024-09-17T21:19:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001433653 2024-09-17T21:19:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:549 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.564983843999997, 2024-09-17T21:19:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048464528999999999, 2024-09-17T21:19:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 549 2024-09-17T21:19:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:19:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199770+0000", 2024-09-17T21:19:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9969999999999999e-06 2024-09-17T21:19:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.842 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.842 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199780+0000", 2024-09-17T21:19:30.842 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0003000000000001e-05 2024-09-17T21:19:30.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.842 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.842 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199793+0000", 2024-09-17T21:19:30.842 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2074e-05 2024-09-17T21:19:30.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.842 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247289+0000", 2024-09-17T21:19:30.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047496691000000001 2024-09-17T21:19:30.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.843 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247437+0000", 2024-09-17T21:19:30.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014807899999999999 2024-09-17T21:19:30.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.843 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248231+0000", 2024-09-17T21:19:30.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00079390899999999998 2024-09-17T21:19:30.843 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.843 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.844 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.844 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:558 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.844 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:30.844 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.564943330999995, 2024-09-17T21:19:30.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049709821000000001, 2024-09-17T21:19:30.844 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.844 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.844 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.844 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.844 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.844 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.845 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 558 2024-09-17T21:19:30.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.845 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:30.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:30.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199809+0000", 2024-09-17T21:19:30.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.333e-06 2024-09-17T21:19:30.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199814+0000", 2024-09-17T21:19:30.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0930000000000002e-06 2024-09-17T21:19:30.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.847 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199831+0000", 2024-09-17T21:19:30.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7580000000000001e-05 2024-09-17T21:19:30.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.847 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.847 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.847 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199860+0000", 2024-09-17T21:19:30.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8609999999999999e-05 2024-09-17T21:19:30.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.847 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.847 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.847 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249106+0000", 2024-09-17T21:19:30.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049246545000000003 2024-09-17T21:19:30.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.848 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.848 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.848 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249274+0000", 2024-09-17T21:19:30.848 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016806699999999999 2024-09-17T21:19:30.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.848 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.848 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.848 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249517+0000", 2024-09-17T21:19:30.848 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024259300000000001 2024-09-17T21:19:30.848 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.848 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.848 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.849 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:559 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.849 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:30.849 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.564823969000003, 2024-09-17T21:19:30.849 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049990694000000002, 2024-09-17T21:19:30.849 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.849 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.849 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.849 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.849 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.849 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.849 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 559 2024-09-17T21:19:30.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.849 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:30.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:30.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199928+0000", 2024-09-17T21:19:30.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:19:30.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.851 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199935+0000", 2024-09-17T21:19:30.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5429999999999999e-06 2024-09-17T21:19:30.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.851 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199954+0000", 2024-09-17T21:19:30.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.912e-05 2024-09-17T21:19:30.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.852 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.852 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199969+0000", 2024-09-17T21:19:30.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5305999999999999e-05 2024-09-17T21:19:30.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.852 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.852 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249604+0000", 2024-09-17T21:19:30.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049634931 2024-09-17T21:19:30.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.853 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.853 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.853 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249738+0000", 2024-09-17T21:19:30.853 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013411599999999999 2024-09-17T21:19:30.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.853 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.853 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.853 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249917+0000", 2024-09-17T21:19:30.853 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179111 2024-09-17T21:19:30.853 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.853 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.853 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.854 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.854 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:560 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:30.854 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:30.854 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.564759553000002, 2024-09-17T21:19:30.854 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050173059999999998, 2024-09-17T21:19:30.854 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.854 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.854 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.854 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.854 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:30.854 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:30.855 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 560 2024-09-17T21:19:30.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.855 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.855 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:30.855 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.855 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:30.855 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.856 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.856 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199992+0000", 2024-09-17T21:19:30.856 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.517e-06 2024-09-17T21:19:30.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.856 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.856 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199998+0000", 2024-09-17T21:19:30.856 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1460000000000004e-06 2024-09-17T21:19:30.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.856 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.857 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:19:30.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7407000000000001e-05 2024-09-17T21:19:30.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.857 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.857 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.857 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200029+0000", 2024-09-17T21:19:30.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3499999999999999e-05 2024-09-17T21:19:30.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.857 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.857 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.857 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249968+0000", 2024-09-17T21:19:30.858 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049939799 2024-09-17T21:19:30.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.858 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.858 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.858 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250055+0000", 2024-09-17T21:19:30.858 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6731999999999996e-05 2024-09-17T21:19:30.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.858 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.858 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.858 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250164+0000", 2024-09-17T21:19:30.858 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108959 2024-09-17T21:19:30.858 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.858 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.859 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.859 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.859 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:19:30.859 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:30.859 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.954969843999997, 2024-09-17T21:19:30.859 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:19:30.859 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.859 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.859 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:30.859 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.859 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:19:30.859 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:19:30.859 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:19:30.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.860 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:30.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:30.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:19:30.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:19:30.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:19:30.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:19:30.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:19:30.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:19:30.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.862 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.862 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:19:30.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:19:30.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.862 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.862 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:19:30.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:19:30.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.863 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:30.863 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:19:30.863 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:19:30.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.863 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.863 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:19:30.863 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:19:30.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:19:30.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:19:30.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:19:30.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:19:30.864 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.864 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.864 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.864 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:19:30.865 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:30.865 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.399868058000003, 2024-09-17T21:19:30.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:19:30.865 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:30.865 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:30.865 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:30.865 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:30.865 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:19:30.865 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:30.865 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:19:30.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.865 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:30.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.866 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:30.866 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:30.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.866 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:30.866 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:30.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:30.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.867 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:30.867 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:19:30.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:19:30.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.867 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:30.867 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:19:30.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:19:30.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:30.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:19:30.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:19:30.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:30.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:19:30.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:19:30.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:19:30.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:19:30.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:30.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:19:30.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:19:30.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:30.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:19:30.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:19:30.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:30.870 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:19:30.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:19:30.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:30.870 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:19:30.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:19:30.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:30.870 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:19:30.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:19:30.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.871 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:30.871 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:19:30.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:19:30.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.871 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:30.871 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:19:30.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:19:30.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:30.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:30.871 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:30.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:19:30.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:19:30.872 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.872 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.872 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.872 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:30.872 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:30.872 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:30.874 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:19:30.874 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:30.872+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:30.974 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:30.970+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:31.049 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:31.049 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:31.049 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:19:31.050 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:31.065 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:19:31.075 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:31.072+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:31.177 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:31.172+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:31.230 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:31.230 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:31.230 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:31.230 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:31.230 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:31.244 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:19:31.265 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-17T21:19:31.288 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:31.286+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:31.378 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:31.376+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:31.396 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:31.396 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:31.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:31.396 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:31.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.397 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:12 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T21:19:31.397 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:19:31.397 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 78.061196254999999, 2024-09-17T21:19:31.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061340989999999996, 2024-09-17T21:19:31.397 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:31.397 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:31.397 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:31.397 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:31.397 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:19:31.397 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:31.397 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12 2024-09-17T21:19:31.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.397 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:31.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:31.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:19:31.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:31.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332335+0000", 2024-09-17T21:19:31.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:31.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332336+0000", 2024-09-17T21:19:31.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5519999999999999e-06 2024-09-17T21:19:31.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:31.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332342+0000", 2024-09-17T21:19:31.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7180000000000001e-06 2024-09-17T21:19:31.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:31.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332352+0000", 2024-09-17T21:19:31.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:19:31.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332359+0000", 2024-09-17T21:19:31.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3479999999999998e-06 2024-09-17T21:19:31.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:31.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332413+0000", 2024-09-17T21:19:31.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4447999999999998e-05 2024-09-17T21:19:31.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-17T21:19:31.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.332417+0000", 2024-09-17T21:19:31.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.895e-06 2024-09-17T21:19:31.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:31.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334755+0000", 2024-09-17T21:19:31.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023377189999999998 2024-09-17T21:19:31.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:31.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.334903+0000", 2024-09-17T21:19:31.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014778999999999999 2024-09-17T21:19:31.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:19:31.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.335187+0000", 2024-09-17T21:19:31.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000284715 2024-09-17T21:19:31.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:31.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.337731+0000", 2024-09-17T21:19:31.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002543741 2024-09-17T21:19:31.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:31.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338382+0000", 2024-09-17T21:19:31.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00065029200000000001 2024-09-17T21:19:31.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:31.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338418+0000", 2024-09-17T21:19:31.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.697e-05 2024-09-17T21:19:31.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:31.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:13.338469+0000", 2024-09-17T21:19:31.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0154999999999997e-05 2024-09-17T21:19:31.403 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.403 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:31.403 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.404 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:782 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'63, mlcod=28'60)", 2024-09-17T21:19:31.404 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:19:31.404 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.178766025000002, 2024-09-17T21:19:31.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006167283, 2024-09-17T21:19:31.404 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:31.404 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:31.404 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:31.404 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:31.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:31.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:19:31.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:31.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214765+0000", 2024-09-17T21:19:31.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:31.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214767+0000", 2024-09-17T21:19:31.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:19:31.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:31.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214779+0000", 2024-09-17T21:19:31.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2627e-05 2024-09-17T21:19:31.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:31.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214805+0000", 2024-09-17T21:19:31.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5570000000000001e-05 2024-09-17T21:19:31.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.214823+0000", 2024-09-17T21:19:31.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8202999999999998e-05 2024-09-17T21:19:31.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:31.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215170+0000", 2024-09-17T21:19:31.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034720399999999999 2024-09-17T21:19:31.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:31.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.215235+0000", 2024-09-17T21:19:31.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4276000000000006e-05 2024-09-17T21:19:31.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:31.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220859+0000", 2024-09-17T21:19:31.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056240500000000002 2024-09-17T21:19:31.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:31.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220932+0000", 2024-09-17T21:19:31.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3503000000000004e-05 2024-09-17T21:19:31.408 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.408 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:31.408 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.408 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:19:31.408 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:31.408 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.176170498000005, 2024-09-17T21:19:31.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:19:31.408 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:31.408 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:31.408 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:31.409 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:31.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:31.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:31.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:31.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:31.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:31.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:19:31.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:19:31.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:31.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:19:31.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:19:31.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:31.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:19:31.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:19:31.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:19:31.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:19:31.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:31.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:19:31.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:19:31.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:31.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:19:31.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:19:31.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:31.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:19:31.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:19:31.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:31.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:19:31.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:19:31.412 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.412 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:31.413 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.413 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:790 3.5 e28/23 3:a14b1a37:::rbd_data.1152c6af092e.000000000000002f:head v 28'1, mlcod=0'0)", 2024-09-17T21:19:31.413 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:19:31.413 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.171131740000007, 2024-09-17T21:19:31.413 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059622240000000003, 2024-09-17T21:19:31.413 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:31.413 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:31.413 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:31.413 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:31.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.413 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:31.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:19:31.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:31.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222399+0000", 2024-09-17T21:19:31.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:31.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222400+0000", 2024-09-17T21:19:31.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.02e-06 2024-09-17T21:19:31.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:31.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222411+0000", 2024-09-17T21:19:31.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.044e-05 2024-09-17T21:19:31.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:31.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222440+0000", 2024-09-17T21:19:31.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.932e-05 2024-09-17T21:19:31.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222458+0000", 2024-09-17T21:19:31.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8615999999999999e-05 2024-09-17T21:19:31.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:31.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222609+0000", 2024-09-17T21:19:31.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015102299999999999 2024-09-17T21:19:31.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:31.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.222657+0000", 2024-09-17T21:19:31.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7206000000000002e-05 2024-09-17T21:19:31.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:31.417 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228278+0000", 2024-09-17T21:19:31.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056208100000000004 2024-09-17T21:19:31.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:31.417 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.228361+0000", 2024-09-17T21:19:31.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3788999999999993e-05 2024-09-17T21:19:31.417 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.417 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:31.417 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.417 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:19:31.418 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:31.418 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 68.525869834000005, 2024-09-17T21:19:31.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:19:31.418 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:31.418 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:31.418 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:31.418 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:31.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:31.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:31.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:31.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:31.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:31.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:19:31.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:19:31.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:31.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:19:31.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:19:31.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:31.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:19:31.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:19:31.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:19:31.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:19:31.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:31.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:19:31.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:19:31.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.421 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:31.421 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:19:31.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:19:31.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.421 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:31.421 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:19:31.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:19:31.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.421 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:31.421 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:19:31.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:19:31.422 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.422 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:31.422 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.422 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:19:31.422 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:31.422 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 68.524604750999998, 2024-09-17T21:19:31.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:19:31.422 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:31.422 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:31.422 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:31.422 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:31.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:31.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:31.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:31.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:31.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:31.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:19:31.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:19:31.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:31.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:19:31.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:19:31.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:31.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:19:31.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:19:31.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:19:31.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:19:31.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:31.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:19:31.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:19:31.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:31.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:19:31.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:19:31.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.426 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:31.426 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:19:31.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:19:31.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.426 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:31.426 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:19:31.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:19:31.426 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.426 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:31.426 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.426 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:19:31.427 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:31.427 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 68.514825341000005, 2024-09-17T21:19:31.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:19:31.427 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:31.427 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:31.427 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:31.427 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:31.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:31.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:31.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:31.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:31.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:31.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:19:31.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:19:31.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:31.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:19:31.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:19:31.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:31.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:19:31.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:19:31.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:19:31.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:19:31.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:31.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:19:31.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:19:31.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:31.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:19:31.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:19:31.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:31.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:19:31.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:19:31.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:31.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:19:31.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:19:31.431 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.431 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:31.431 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.431 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:19:31.431 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:31.431 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 68.506102937999998, 2024-09-17T21:19:31.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:19:31.431 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:31.431 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:31.431 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:31.431 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:31.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:31.432 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:31.432 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:31.432 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:31.432 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:31.432 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:19:31.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:19:31.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:31.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:19:31.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:19:31.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:31.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:19:31.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:19:31.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:19:31.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:19:31.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:31.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:19:31.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:19:31.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:31.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:19:31.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:19:31.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:31.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:19:31.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:19:31.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:31.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:19:31.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:19:31.435 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.435 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:31.435 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.435 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:19:31.436 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:31.436 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 55.121012374000003, 2024-09-17T21:19:31.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:19:31.436 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:31.436 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:31.436 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:31.436 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:31.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.436 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:31.436 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:31.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:31.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:31.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:31.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:19:31.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:19:31.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:31.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:19:31.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:19:31.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:31.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:19:31.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:19:31.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:19:31.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:19:31.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:31.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:19:31.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:19:31.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:31.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:19:31.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:19:31.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:31.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:19:31.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:19:31.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:31.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:19:31.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:19:31.440 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.440 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:31.440 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.440 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1777 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'194, mlcod=69'189)", 2024-09-17T21:19:31.440 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.249616+0000", 2024-09-17T21:19:31.440 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.143914500000001, 2024-09-17T21:19:31.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006283589, 2024-09-17T21:19:31.440 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:31.440 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:31.441 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:31.441 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:31.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.441 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:31.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.249616+0000", 2024-09-17T21:19:31.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.441 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:31.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.249616+0000", 2024-09-17T21:19:31.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:31.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.249618+0000", 2024-09-17T21:19:31.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2440000000000001e-06 2024-09-17T21:19:31.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:31.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.249626+0000", 2024-09-17T21:19:31.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.32e-06 2024-09-17T21:19:31.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:31.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.249645+0000", 2024-09-17T21:19:31.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9315000000000001e-05 2024-09-17T21:19:31.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.443 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.443 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.249656+0000", 2024-09-17T21:19:31.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543999999999999e-05 2024-09-17T21:19:31.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.443 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:31.443 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250345+0000", 2024-09-17T21:19:31.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00068912499999999996 2024-09-17T21:19:31.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.443 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:31.443 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250401+0000", 2024-09-17T21:19:31.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5985e-05 2024-09-17T21:19:31.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:31.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.255797+0000", 2024-09-17T21:19:31.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0053965790000000003 2024-09-17T21:19:31.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:31.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.255900+0000", 2024-09-17T21:19:31.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000102477 2024-09-17T21:19:31.444 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.444 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:31.444 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.445 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1780 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'195, mlcod=69'190)", 2024-09-17T21:19:31.445 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:19:31.445 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.143156770000001, 2024-09-17T21:19:31.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075761769999999999, 2024-09-17T21:19:31.445 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:31.445 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:31.445 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:31.445 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:31.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:31.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:19:31.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:31.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:19:31.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:31.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250375+0000", 2024-09-17T21:19:31.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:19:31.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:31.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250383+0000", 2024-09-17T21:19:31.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7870000000000002e-06 2024-09-17T21:19:31.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:31.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250402+0000", 2024-09-17T21:19:31.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9346000000000001e-05 2024-09-17T21:19:31.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250413+0000", 2024-09-17T21:19:31.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.039e-05 2024-09-17T21:19:31.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:31.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251353+0000", 2024-09-17T21:19:31.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000940091 2024-09-17T21:19:31.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:31.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251407+0000", 2024-09-17T21:19:31.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4701999999999999e-05 2024-09-17T21:19:31.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:31.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257829+0000", 2024-09-17T21:19:31.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064212779999999999 2024-09-17T21:19:31.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:31.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257950+0000", 2024-09-17T21:19:31.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121623 2024-09-17T21:19:31.449 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.449 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:31.449 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.449 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1781 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'196, mlcod=69'190)", 2024-09-17T21:19:31.449 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:19:31.449 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.142073819, 2024-09-17T21:19:31.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065632939999999999, 2024-09-17T21:19:31.450 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:31.450 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:31.450 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:31.450 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:31.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:31.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:19:31.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:31.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:19:31.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:31.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251459+0000", 2024-09-17T21:19:31.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.624e-06 2024-09-17T21:19:31.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:31.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251471+0000", 2024-09-17T21:19:31.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2636000000000001e-05 2024-09-17T21:19:31.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:31.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251498+0000", 2024-09-17T21:19:31.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:19:31.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251515+0000", 2024-09-17T21:19:31.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6939000000000002e-05 2024-09-17T21:19:31.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:31.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252938+0000", 2024-09-17T21:19:31.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001423082 2024-09-17T21:19:31.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:31.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252999+0000", 2024-09-17T21:19:31.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1138999999999999e-05 2024-09-17T21:19:31.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:31.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257955+0000", 2024-09-17T21:19:31.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049559640000000002 2024-09-17T21:19:31.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:31.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.258020+0000", 2024-09-17T21:19:31.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5504000000000002e-05 2024-09-17T21:19:31.454 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.454 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:31.454 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.454 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1788 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'198, mlcod=69'191)", 2024-09-17T21:19:31.454 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:19:31.454 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.140048329999999, 2024-09-17T21:19:31.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073690370000000002, 2024-09-17T21:19:31.454 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:31.454 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:31.454 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:31.454 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:31.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.455 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:31.455 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:19:31.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.455 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:31.455 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:19:31.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.455 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:31.455 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253484+0000", 2024-09-17T21:19:31.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2470000000000001e-06 2024-09-17T21:19:31.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:31.456 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253492+0000", 2024-09-17T21:19:31.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7159999999999992e-06 2024-09-17T21:19:31.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:31.456 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253509+0000", 2024-09-17T21:19:31.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.666e-05 2024-09-17T21:19:31.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.456 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253522+0000", 2024-09-17T21:19:31.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2643000000000001e-05 2024-09-17T21:19:31.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:31.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254853+0000", 2024-09-17T21:19:31.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331238 2024-09-17T21:19:31.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:31.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254885+0000", 2024-09-17T21:19:31.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1562e-05 2024-09-17T21:19:31.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:31.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260788+0000", 2024-09-17T21:19:31.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059030749999999998 2024-09-17T21:19:31.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:31.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260852+0000", 2024-09-17T21:19:31.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3895999999999999e-05 2024-09-17T21:19:31.458 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.458 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:31.458 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.458 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5746 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'477, mlcod=71'473)", 2024-09-17T21:19:31.458 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:19:31.458 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.134620772, 2024-09-17T21:19:31.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074182709999999997, 2024-09-17T21:19:31.459 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:31.459 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:31.459 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:31.459 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:31.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:31.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:19:31.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:31.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:19:31.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.460 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:31.460 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258912+0000", 2024-09-17T21:19:31.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:19:31.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.460 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:31.460 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258926+0000", 2024-09-17T21:19:31.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4249e-05 2024-09-17T21:19:31.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.460 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258946+0000", 2024-09-17T21:19:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9879999999999999e-05 2024-09-17T21:19:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258956+0000", 2024-09-17T21:19:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:19:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259380+0000", 2024-09-17T21:19:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042337599999999999 2024-09-17T21:19:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259412+0000", 2024-09-17T21:19:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:19:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266191+0000", 2024-09-17T21:19:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067793269999999999 2024-09-17T21:19:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266328+0000", 2024-09-17T21:19:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013690999999999999 2024-09-17T21:19:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5747 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'478, mlcod=71'473)", 2024-09-17T21:19:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:19:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.133914423, 2024-09-17T21:19:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068779440000000004, 2024-09-17T21:19:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:19:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:19:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259618+0000", 2024-09-17T21:19:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6199999999999999e-06 2024-09-17T21:19:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259630+0000", 2024-09-17T21:19:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1884e-05 2024-09-17T21:19:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259657+0000", 2024-09-17T21:19:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7339e-05 2024-09-17T21:19:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259677+0000", 2024-09-17T21:19:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0156000000000001e-05 2024-09-17T21:19:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260521+0000", 2024-09-17T21:19:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084312899999999997 2024-09-17T21:19:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260585+0000", 2024-09-17T21:19:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4558000000000001e-05 2024-09-17T21:19:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266336+0000", 2024-09-17T21:19:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057505020000000002 2024-09-17T21:19:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266494+0000", 2024-09-17T21:19:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158756 2024-09-17T21:19:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5750 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'479, mlcod=71'473)", 2024-09-17T21:19:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:19:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.13320223, 2024-09-17T21:19:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006302874, 2024-09-17T21:19:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:19:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:19:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260330+0000", 2024-09-17T21:19:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:19:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260343+0000", 2024-09-17T21:19:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2862e-05 2024-09-17T21:19:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:31.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260367+0000", 2024-09-17T21:19:31.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3723e-05 2024-09-17T21:19:31.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260386+0000", 2024-09-17T21:19:31.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8879999999999999e-05 2024-09-17T21:19:31.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:31.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261579+0000", 2024-09-17T21:19:31.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001193431 2024-09-17T21:19:31.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261608+0000", 2024-09-17T21:19:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8858999999999999e-05 2024-09-17T21:19:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266505+0000", 2024-09-17T21:19:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004896999 2024-09-17T21:19:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266631+0000", 2024-09-17T21:19:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012655300000000001 2024-09-17T21:19:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5751 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'480, mlcod=71'473)", 2024-09-17T21:19:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:19:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.132495018, 2024-09-17T21:19:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075756299999999999, 2024-09-17T21:19:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:19:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:19:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261037+0000", 2024-09-17T21:19:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:19:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261049+0000", 2024-09-17T21:19:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1063999999999999e-05 2024-09-17T21:19:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261072+0000", 2024-09-17T21:19:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3102e-05 2024-09-17T21:19:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261133+0000", 2024-09-17T21:19:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1276000000000001e-05 2024-09-17T21:19:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:19:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015334319999999999 2024-09-17T21:19:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262702+0000", 2024-09-17T21:19:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5281999999999999e-05 2024-09-17T21:19:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268527+0000", 2024-09-17T21:19:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058250389999999997 2024-09-17T21:19:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268611+0000", 2024-09-17T21:19:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4724999999999999e-05 2024-09-17T21:19:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5756 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'481, mlcod=71'473)", 2024-09-17T21:19:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:19:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.131634161999999, 2024-09-17T21:19:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006772386, 2024-09-17T21:19:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:19:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:19:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261899+0000", 2024-09-17T21:19:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.903e-06 2024-09-17T21:19:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261912+0000", 2024-09-17T21:19:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-05 2024-09-17T21:19:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261938+0000", 2024-09-17T21:19:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6098999999999999e-05 2024-09-17T21:19:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261955+0000", 2024-09-17T21:19:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.696e-05 2024-09-17T21:19:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263845+0000", 2024-09-17T21:19:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001890187 2024-09-17T21:19:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263907+0000", 2024-09-17T21:19:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1600999999999995e-05 2024-09-17T21:19:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268615+0000", 2024-09-17T21:19:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047086070000000001 2024-09-17T21:19:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268669+0000", 2024-09-17T21:19:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3588999999999998e-05 2024-09-17T21:19:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5757 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'482, mlcod=71'473)", 2024-09-17T21:19:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.262633+0000", 2024-09-17T21:19:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.130898246999999, 2024-09-17T21:19:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061442279999999998, 2024-09-17T21:19:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262633+0000", 2024-09-17T21:19:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262633+0000", 2024-09-17T21:19:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262634+0000", 2024-09-17T21:19:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.587e-06 2024-09-17T21:19:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262645+0000", 2024-09-17T21:19:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0686000000000001e-05 2024-09-17T21:19:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:19:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0829999999999999e-05 2024-09-17T21:19:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262678+0000", 2024-09-17T21:19:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2583e-05 2024-09-17T21:19:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.265094+0000", 2024-09-17T21:19:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024153170000000002 2024-09-17T21:19:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.265167+0000", 2024-09-17T21:19:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3543999999999999e-05 2024-09-17T21:19:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268676+0000", 2024-09-17T21:19:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035090210000000002 2024-09-17T21:19:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268777+0000", 2024-09-17T21:19:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010066 2024-09-17T21:19:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4978.0:598 3.9 e80/23 3:9701cc17:::rbd_data.1152724ff2a3.0000000000000019:head v 80'202, mlcod=80'198)", 2024-09-17T21:19:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:19:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.6300597789999998, 2024-09-17T21:19:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066473340000000004, 2024-09-17T21:19:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:19:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:19:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763472+0000", 2024-09-17T21:19:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6899999999999997e-07 2024-09-17T21:19:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763478+0000", 2024-09-17T21:19:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:19:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763489+0000", 2024-09-17T21:19:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0628999999999999e-05 2024-09-17T21:19:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763495+0000", 2024-09-17T21:19:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6649999999999997e-06 2024-09-17T21:19:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763907+0000", 2024-09-17T21:19:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041163899999999999 2024-09-17T21:19:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763976+0000", 2024-09-17T21:19:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8534999999999993e-05 2024-09-17T21:19:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.769900+0000", 2024-09-17T21:19:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059243270000000001 2024-09-17T21:19:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.770118+0000", 2024-09-17T21:19:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021842500000000001 2024-09-17T21:19:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:31.490 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:31.492 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:19:31.492 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:31.486+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:31.565 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-17T21:19:31.584 INFO:teuthology.orchestra.run.smithi184.stdout:3544 punch from 0xabba800 to 0xabc7200, (0xca00 bytes) 2024-09-17T21:19:31.592 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:31.590+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:31.613 INFO:teuthology.orchestra.run.smithi184.stdout:3545 write 0x8e71800 thru 0x8e799a1 (0x81a2 bytes) 2024-09-17T21:19:31.630 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:31.630 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:31.631 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:19:31.631 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:31.645 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:19:31.693 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:31.690+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:31.709 INFO:teuthology.orchestra.run.smithi184.stdout:3546 trunc from 0xc235600 to 0x220bc00 2024-09-17T21:19:31.776 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:31.777 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:31.777 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:31.777 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:31.777 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:31.779 INFO:teuthology.orchestra.run.smithi184.stdout:3547 read 0x19ede00 thru 0x19f6db9 (0x8fba bytes) 2024-09-17T21:19:31.791 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:19:31.793 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:31.790+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:31.794 INFO:teuthology.orchestra.run.smithi184.stdout:3548 trunc from 0x220bc00 to 0x5f78400 2024-09-17T21:19:31.813 INFO:teuthology.orchestra.run.smithi184.stdout:3549 write 0x2a48400 thru 0x2a4b43e (0x303f bytes) 2024-09-17T21:19:31.814 INFO:teuthology.orchestra.run.smithi184.stdout:3553 punch from 0x3fe6000 to 0x3ff4e00, (0xee00 bytes) 2024-09-17T21:19:31.815 INFO:teuthology.orchestra.run.smithi184.stdout:3554 punch from 0x49a2e00 to 0x49a8e00, (0x6000 bytes) 2024-09-17T21:19:31.815 INFO:teuthology.orchestra.run.smithi184.stdout:3555 write 0x56d7e00 thru 0x56dbd67 (0x3f68 bytes) 2024-09-17T21:19:31.861 INFO:teuthology.orchestra.run.smithi184.stdout:3556 write 0xe441400 thru 0xe44f2b3 (0xdeb4 bytes) 2024-09-17T21:19:31.863 INFO:teuthology.orchestra.run.smithi184.stdout:3557 punch from 0x9dc6e00 to 0x9dcb600, (0x4800 bytes) 2024-09-17T21:19:31.880 INFO:teuthology.orchestra.run.smithi184.stdout:3558 trunc from 0xe44f400 to 0x4746800 2024-09-17T21:19:31.894 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:31.890+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:31.915 INFO:teuthology.orchestra.run.smithi184.stdout:3559 punch from 0x463ac00 to 0x4641400, (0x6800 bytes) 2024-09-17T21:19:31.918 INFO:teuthology.orchestra.run.smithi184.stdout:3561 punch from 0xf8a400 to 0xf95600, (0xb200 bytes) 2024-09-17T21:19:31.918 INFO:teuthology.orchestra.run.smithi184.stdout:3566 write 0x45cb000 thru 0x45da899 (0xf89a bytes) 2024-09-17T21:19:31.921 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:31.921 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:31.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:31.922 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:31.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.922 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:19:31.922 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:31.922 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 94.904478284000007, 2024-09-17T21:19:31.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:19:31.922 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:31.922 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:31.922 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:31.922 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:31.923 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:19:31.923 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:19:31.923 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:19:31.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.923 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:31.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.923 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:31.923 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:31.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:31.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:31.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:31.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:19:31.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:19:31.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:31.925 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:19:31.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:19:31.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:31.925 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:19:31.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:19:31.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:19:31.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:19:31.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:19:31.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:19:31.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:31.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:19:31.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:19:31.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:19:31.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:19:31.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:31.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:19:31.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:19:31.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:19:31.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:19:31.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:19:31.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:31.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:19:31.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:19:31.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:31.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:19:31.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:19:31.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:31.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:19:31.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:19:31.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:31.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:19:31.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:19:31.930 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:31.930 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:31.930 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:31.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.930 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:31.930 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:31.930 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 75.324314004000001, 2024-09-17T21:19:31.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:19:31.930 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:31.930 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:31.930 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:31.931 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:31.931 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:19:31.931 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:19:31.931 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:19:31.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.931 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:31.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:31.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:31.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:31.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:31.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:31.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:19:31.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:19:31.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:31.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:19:31.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:19:31.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:31.933 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:19:31.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:19:31.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.933 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:19:31.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:19:31.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:19:31.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:19:31.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:31.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:19:31.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:19:31.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:19:31.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:19:31.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:31.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:19:31.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:19:31.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:19:31.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:19:31.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:31.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:19:31.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:19:31.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:31.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:19:31.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:19:31.936 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:31.936 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:31.937 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:31.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.937 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:31.937 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:31.937 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 75.324216691999993, 2024-09-17T21:19:31.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:19:31.937 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:31.937 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:31.937 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:31.937 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:31.937 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:31.938 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:31.938 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:19:31.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.938 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:31.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:31.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:31.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:31.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:31.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.939 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.939 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:31.939 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:19:31.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:19:31.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.939 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.939 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:31.939 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:19:31.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:19:31.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.940 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:31.940 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:19:31.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:19:31.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.940 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.940 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:19:31.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:19:31.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.941 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:19:31.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:19:31.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.941 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:31.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:19:31.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:19:31.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.941 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:19:31.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:19:31.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:31.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:19:31.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:19:31.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:19:31.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:19:31.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:19:31.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:19:31.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:19:31.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:19:31.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:19:31.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:31.944 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:19:31.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:19:31.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.944 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:31.944 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:19:31.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:19:31.944 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:31.944 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:31.944 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:31.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.945 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:31.945 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:31.945 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 75.322129891000003, 2024-09-17T21:19:31.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:19:31.945 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:31.945 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:31.945 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:31.945 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:31.945 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:19:31.945 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:19:31.945 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:19:31.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.946 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:31.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:31.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:31.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:31.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:31.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:31.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:19:31.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:19:31.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:31.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:19:31.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:19:31.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:31.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:19:31.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:19:31.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:19:31.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:19:31.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:19:31.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:19:31.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:19:31.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:19:31.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:19:31.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:19:31.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:19:31.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:31.950 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:19:31.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:19:31.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.950 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.950 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.950 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:19:31.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:19:31.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.950 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.950 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:31.950 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:19:31.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:19:31.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:31.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:19:31.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:19:31.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:31.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:19:31.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:19:31.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:31.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:19:31.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:19:31.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:31.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:19:31.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:19:31.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:31.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:19:31.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:19:31.953 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:31.953 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:31.953 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:31.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.953 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:284 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:31.953 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:19:31.953 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.690307238000003, 2024-09-17T21:19:31.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024450084, 2024-09-17T21:19:31.953 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:31.953 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:31.954 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:31.954 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:31.954 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:31.954 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:31.954 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 284 2024-09-17T21:19:31.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.954 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:31.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.954 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:31.954 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:19:31.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.955 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:31.955 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:19:31.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.955 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:31.955 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227937+0000", 2024-09-17T21:19:31.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6700000000000002e-07 2024-09-17T21:19:31.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.955 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:31.956 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227943+0000", 2024-09-17T21:19:31.956 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:19:31.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.956 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:31.956 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227961+0000", 2024-09-17T21:19:31.956 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.827e-05 2024-09-17T21:19:31.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.956 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.956 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227975+0000", 2024-09-17T21:19:31.956 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.394e-05 2024-09-17T21:19:31.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251977+0000", 2024-09-17T21:19:31.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024001340999999999 2024-09-17T21:19:31.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:31.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252169+0000", 2024-09-17T21:19:31.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019271199999999999 2024-09-17T21:19:31.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:31.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252387+0000", 2024-09-17T21:19:31.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021726100000000001 2024-09-17T21:19:31.958 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:31.958 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:31.958 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:31.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.958 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:285 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:31.958 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:19:31.958 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.690250947999999, 2024-09-17T21:19:31.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024884791, 2024-09-17T21:19:31.959 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:31.959 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:31.959 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:31.959 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:31.959 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:31.959 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:31.959 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 285 2024-09-17T21:19:31.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.959 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:31.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:31.959 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:19:31.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.960 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:31.960 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:19:31.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.960 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:31.960 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227994+0000", 2024-09-17T21:19:31.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1429999999999999e-06 2024-09-17T21:19:31.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:31.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227999+0000", 2024-09-17T21:19:31.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1669999999999998e-06 2024-09-17T21:19:31.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:31.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228015+0000", 2024-09-17T21:19:31.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6056e-05 2024-09-17T21:19:31.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.962 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228028+0000", 2024-09-17T21:19:31.962 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3196e-05 2024-09-17T21:19:31.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.962 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.962 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252458+0000", 2024-09-17T21:19:31.962 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024429751999999999 2024-09-17T21:19:31.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.962 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:31.962 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252650+0000", 2024-09-17T21:19:31.962 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019187800000000001 2024-09-17T21:19:31.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:31.963 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252878+0000", 2024-09-17T21:19:31.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022759899999999999 2024-09-17T21:19:31.963 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:31.963 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:31.963 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:31.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.963 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:286 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:31.963 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:19:31.964 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.690195189000001, 2024-09-17T21:19:31.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025299318000000001, 2024-09-17T21:19:31.964 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:31.964 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:31.964 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:31.964 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:31.964 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:31.964 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:31.964 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 286 2024-09-17T21:19:31.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.964 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:31.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.965 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:31.965 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:19:31.965 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.965 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:31.965 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:19:31.965 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.965 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:31.965 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228050+0000", 2024-09-17T21:19:31.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:19:31.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.966 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:31.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228055+0000", 2024-09-17T21:19:31.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4960000000000004e-06 2024-09-17T21:19:31.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.966 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:31.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228075+0000", 2024-09-17T21:19:31.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0109999999999999e-05 2024-09-17T21:19:31.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.967 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.967 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228089+0000", 2024-09-17T21:19:31.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4154e-05 2024-09-17T21:19:31.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.967 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.967 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252952+0000", 2024-09-17T21:19:31.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024862934 2024-09-17T21:19:31.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:31.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253141+0000", 2024-09-17T21:19:31.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000188782 2024-09-17T21:19:31.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:31.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253348+0000", 2024-09-17T21:19:31.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020684499999999999 2024-09-17T21:19:31.968 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:31.968 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:31.968 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:31.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.969 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:31.969 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:31.969 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 42.63823111, 2024-09-17T21:19:31.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:19:31.969 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:31.969 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:31.969 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:31.969 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:31.969 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:31.969 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:31.969 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:19:31.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.970 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:31.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.970 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:31.970 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:31.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.970 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:31.970 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:31.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:31.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:19:31.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:19:31.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:31.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:19:31.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:19:31.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:31.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:19:31.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:19:31.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:19:31.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:19:31.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:19:31.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:19:31.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:31.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:19:31.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:19:31.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:19:31.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:19:31.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:31.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:19:31.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:19:31.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:19:31.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:19:31.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:31.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:19:31.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:19:31.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.975 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:31.975 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:19:31.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:19:31.975 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:31.975 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:31.975 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:31.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.975 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:31.976 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:31.976 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 42.629139399000003, 2024-09-17T21:19:31.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:19:31.976 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:31.976 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:31.976 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:31.976 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:31.976 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:31.976 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:31.976 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:19:31.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.977 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:31.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:31.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:31.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:31.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:31.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:31.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:19:31.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:19:31.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:31.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:19:31.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:19:31.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:31.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:19:31.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:19:31.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:19:31.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:19:31.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:19:31.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:19:31.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:31.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:19:31.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:19:31.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:19:31.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:19:31.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:31.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:19:31.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:19:31.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:31.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:19:31.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:19:31.981 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:31.982 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:31.982 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:31.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.982 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:31.982 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:31.982 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 42.628967348000003, 2024-09-17T21:19:31.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:19:31.982 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:31.982 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:31.982 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:31.982 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:31.983 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:31.983 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:31.983 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:19:31.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.983 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:31.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:31.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:31.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:31.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:31.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:31.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:19:31.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:19:31.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:31.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:19:31.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:19:31.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:31.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:19:31.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:19:31.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:19:31.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:19:31.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.986 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:19:31.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:19:31.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:31.986 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:19:31.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:19:31.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:19:31.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:19:31.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:31.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:19:31.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:19:31.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:31.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:19:31.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:19:31.988 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:31.988 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:31.988 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:31.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.988 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:31.988 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:31.988 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 42.454034600999996, 2024-09-17T21:19:31.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:19:31.988 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:31.988 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:31.988 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:31.989 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:31.989 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:31.989 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:31.989 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:19:31.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.989 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:31.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:31.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:31.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:31.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:31.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:31.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:19:31.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:19:31.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:31.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:19:31.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:19:31.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:31.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:19:31.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:19:31.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:19:31.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:19:31.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.992 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.992 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:19:31.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:19:31.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.992 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:31.992 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:19:31.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:19:31.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.992 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:19:31.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:19:31.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:31.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:19:31.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:19:31.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:31.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:19:31.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:19:31.994 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:31.994 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:31.994 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:31.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.994 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:31.994 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:31.994 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 42.453950044000003, 2024-09-17T21:19:31.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:19:31.994 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:31.994 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:31.994 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:31.995 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:31.995 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:31.995 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:31.995 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:19:31.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.995 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:31.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:31.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:31.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:31.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:31.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:31.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:31.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:31.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:19:31.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:31.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:19:31.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:19:31.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:31.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:19:31.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:19:31.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:31.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:19:31.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:19:31.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:19:31.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:19:31.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:31.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:19:31.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:19:31.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:31.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:19:31.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:19:31.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:31.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:19:31.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:19:31.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:31.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:31.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:31.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:19:32.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:19:32.000 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:32.000 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:32.000 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:32.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.000 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:32.000 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:32.000 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 42.453884406999997, 2024-09-17T21:19:32.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:19:32.000 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:32.000 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:32.001 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:32.001 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:32.001 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:32.001 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:32.001 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:19:32.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.001 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:32.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:32.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:32.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:32.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:32.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:32.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:32.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:32.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:19:32.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:19:32.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:32.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:19:32.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:19:32.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:32.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:19:32.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:19:32.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:32.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:19:32.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:19:32.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:32.004 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:19:32.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:19:32.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:32.004 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:19:32.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:19:32.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:32.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:19:32.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:19:32.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:32.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:19:32.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:19:32.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:32.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:19:32.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:19:32.006 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:32.006 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:32.006 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:32.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.006 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:10979 3.a 3:53159310:::rbd_header.1152724ff2a3:head [watch reconnect cookie 139860846458096 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-17T21:19:32.006 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:26.748362+0000", 2024-09-17T21:19:32.006 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.1698813609999998, 2024-09-17T21:19:32.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025943728999999999, 2024-09-17T21:19:32.006 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:32.007 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:32.007 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:32.007 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:32.007 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:32.007 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:32.007 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10979 2024-09-17T21:19:32.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.007 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:32.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:32.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.748362+0000", 2024-09-17T21:19:32.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:32.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:32.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.748362+0000", 2024-09-17T21:19:32.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:32.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:32.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.748364+0000", 2024-09-17T21:19:32.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9039999999999999e-06 2024-09-17T21:19:32.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:32.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.748370+0000", 2024-09-17T21:19:32.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5890000000000002e-06 2024-09-17T21:19:32.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:32.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.748393+0000", 2024-09-17T21:19:32.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3462999999999999e-05 2024-09-17T21:19:32.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:32.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.748406+0000", 2024-09-17T21:19:32.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2317000000000001e-05 2024-09-17T21:19:32.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:32.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.767146+0000", 2024-09-17T21:19:32.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018740271999999999 2024-09-17T21:19:32.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:32.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.767277+0000", 2024-09-17T21:19:32.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013130300000000001 2024-09-17T21:19:32.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:32.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.774306+0000", 2024-09-17T21:19:32.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0070288809999999998 2024-09-17T21:19:32.011 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:32.011 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:32.011 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:32.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.011 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4978.0:564 3.a 3:53159310:::rbd_header.1152724ff2a3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-17T21:19:32.011 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:26.750343+0000", 2024-09-17T21:19:32.011 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.1679005580000004, 2024-09-17T21:19:32.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024755837999999999, 2024-09-17T21:19:32.012 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:32.012 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:32.012 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:32.012 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:32.012 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4978", 2024-09-17T21:19:32.012 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2995854560", 2024-09-17T21:19:32.012 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 564 2024-09-17T21:19:32.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.012 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:32.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:32.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750343+0000", 2024-09-17T21:19:32.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:32.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:32.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750343+0000", 2024-09-17T21:19:32.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:32.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:32.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750344+0000", 2024-09-17T21:19:32.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0669999999999999e-06 2024-09-17T21:19:32.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:32.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750348+0000", 2024-09-17T21:19:32.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9600000000000002e-06 2024-09-17T21:19:32.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:32.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750357+0000", 2024-09-17T21:19:32.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9560000000000003e-06 2024-09-17T21:19:32.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:32.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750380+0000", 2024-09-17T21:19:32.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2376999999999999e-05 2024-09-17T21:19:32.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:32.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.774378+0000", 2024-09-17T21:19:32.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023998609000000001 2024-09-17T21:19:32.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:32.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.774930+0000", 2024-09-17T21:19:32.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00055168100000000005 2024-09-17T21:19:32.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:32.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.775099+0000", 2024-09-17T21:19:32.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016918799999999999 2024-09-17T21:19:32.016 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:32.016 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:32.016 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:32.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.016 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4978.0:565 3.a 3:53159310:::rbd_header.1152724ff2a3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-17T21:19:32.017 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:26.750461+0000", 2024-09-17T21:19:32.017 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.16778241, 2024-09-17T21:19:32.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024983353999999999, 2024-09-17T21:19:32.017 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:32.017 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:32.017 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:32.017 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:32.017 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4978", 2024-09-17T21:19:32.017 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2995854560", 2024-09-17T21:19:32.017 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 565 2024-09-17T21:19:32.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.017 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:32.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:32.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750461+0000", 2024-09-17T21:19:32.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:32.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:32.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750461+0000", 2024-09-17T21:19:32.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:32.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:32.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750464+0000", 2024-09-17T21:19:32.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3130000000000001e-06 2024-09-17T21:19:32.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:32.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750472+0000", 2024-09-17T21:19:32.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2130000000000008e-06 2024-09-17T21:19:32.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:32.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750491+0000", 2024-09-17T21:19:32.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.927e-05 2024-09-17T21:19:32.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:32.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750505+0000", 2024-09-17T21:19:32.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3726999999999999e-05 2024-09-17T21:19:32.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:32.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.775162+0000", 2024-09-17T21:19:32.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024657070999999999 2024-09-17T21:19:32.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:32.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.775289+0000", 2024-09-17T21:19:32.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012754899999999999 2024-09-17T21:19:32.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:32.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.775445+0000", 2024-09-17T21:19:32.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015521099999999999 2024-09-17T21:19:32.021 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:32.021 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:32.021 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:32.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.022 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4978.0:567 3.a 3:53159310:::rbd_header.1152724ff2a3:head [notify cookie 140612330492592] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-17T21:19:32.022 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:26.750648+0000", 2024-09-17T21:19:32.022 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.1675959430000002, 2024-09-17T21:19:32.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026360126000000001, 2024-09-17T21:19:32.022 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:32.022 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:32.022 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:32.022 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:32.022 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4978", 2024-09-17T21:19:32.022 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2995854560", 2024-09-17T21:19:32.022 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 567 2024-09-17T21:19:32.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.023 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:32.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:32.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750648+0000", 2024-09-17T21:19:32.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:32.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:32.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750648+0000", 2024-09-17T21:19:32.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:32.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:32.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750650+0000", 2024-09-17T21:19:32.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8330000000000001e-06 2024-09-17T21:19:32.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:32.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750659+0000", 2024-09-17T21:19:32.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8160000000000005e-06 2024-09-17T21:19:32.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:32.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750695+0000", 2024-09-17T21:19:32.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5666e-05 2024-09-17T21:19:32.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:32.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750712+0000", 2024-09-17T21:19:32.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7003999999999998e-05 2024-09-17T21:19:32.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:32.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.775505+0000", 2024-09-17T21:19:32.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024792742999999999 2024-09-17T21:19:32.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:32.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.775660+0000", 2024-09-17T21:19:32.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015555500000000001 2024-09-17T21:19:32.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:32.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.777008+0000", 2024-09-17T21:19:32.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001347509 2024-09-17T21:19:32.026 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:32.026 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:32.026 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:32.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.027 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4978.0:568 3.a 3:53159310:::rbd_header.1152724ff2a3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-17T21:19:32.027 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:26.751223+0000", 2024-09-17T21:19:32.027 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.1670203079999997, 2024-09-17T21:19:32.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025333333999999999, 2024-09-17T21:19:32.027 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:32.027 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:32.027 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:32.027 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:32.027 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4978", 2024-09-17T21:19:32.027 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2995854560", 2024-09-17T21:19:32.028 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 568 2024-09-17T21:19:32.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.028 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:32.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:32.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.751223+0000", 2024-09-17T21:19:32.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:32.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:32.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.751223+0000", 2024-09-17T21:19:32.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:32.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:32.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.751226+0000", 2024-09-17T21:19:32.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4770000000000002e-06 2024-09-17T21:19:32.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:32.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.751238+0000", 2024-09-17T21:19:32.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1739999999999999e-05 2024-09-17T21:19:32.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:32.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.751277+0000", 2024-09-17T21:19:32.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9816e-05 2024-09-17T21:19:32.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:32.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.751295+0000", 2024-09-17T21:19:32.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7283e-05 2024-09-17T21:19:32.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:32.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.776198+0000", 2024-09-17T21:19:32.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024903444 2024-09-17T21:19:32.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:32.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.776387+0000", 2024-09-17T21:19:32.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000188522 2024-09-17T21:19:32.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:32.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.776557+0000", 2024-09-17T21:19:32.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017005199999999999 2024-09-17T21:19:32.031 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:32.032 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:32.032 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:32.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.032 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4978.0:575 3.a 3:53159310:::rbd_header.1152724ff2a3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-17T21:19:32.032 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:26.752087+0000", 2024-09-17T21:19:32.032 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.1661569969999999, 2024-09-17T21:19:32.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024864818, 2024-09-17T21:19:32.032 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:32.032 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:32.032 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:32.032 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:32.033 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4978", 2024-09-17T21:19:32.033 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2995854560", 2024-09-17T21:19:32.033 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 575 2024-09-17T21:19:32.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.033 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:32.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:32.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.752087+0000", 2024-09-17T21:19:32.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:32.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:32.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.752087+0000", 2024-09-17T21:19:32.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:32.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:32.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.752090+0000", 2024-09-17T21:19:32.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8569999999999999e-06 2024-09-17T21:19:32.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:32.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.752103+0000", 2024-09-17T21:19:32.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3752e-05 2024-09-17T21:19:32.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:32.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.752136+0000", 2024-09-17T21:19:32.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2403000000000001e-05 2024-09-17T21:19:32.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:32.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.752157+0000", 2024-09-17T21:19:32.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0973000000000001e-05 2024-09-17T21:19:32.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:32.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.776623+0000", 2024-09-17T21:19:32.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024466517 2024-09-17T21:19:32.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:32.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.776762+0000", 2024-09-17T21:19:32.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000139159 2024-09-17T21:19:32.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:32.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.776952+0000", 2024-09-17T21:19:32.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018915699999999999 2024-09-17T21:19:32.037 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:32.037 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:32.037 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:32.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.037 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4978.0:755 3.a 3:538046bd:::rbd_data.1152724ff2a3.000000000000006a:head [stat,delete] snapc 14=[14] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-17T21:19:32.037 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:26.798467+0000", 2024-09-17T21:19:32.037 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.1197769390000003, 2024-09-17T21:19:32.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024606805999999998, 2024-09-17T21:19:32.037 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:32.037 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:32.037 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:32.038 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:32.038 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4978", 2024-09-17T21:19:32.038 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2995854560", 2024-09-17T21:19:32.038 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 755 2024-09-17T21:19:32.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.038 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:32.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:32.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.798467+0000", 2024-09-17T21:19:32.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:32.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:32.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.798467+0000", 2024-09-17T21:19:32.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:32.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:32.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.798468+0000", 2024-09-17T21:19:32.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4200000000000004e-07 2024-09-17T21:19:32.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:32.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.798472+0000", 2024-09-17T21:19:32.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2320000000000003e-06 2024-09-17T21:19:32.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:32.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.798485+0000", 2024-09-17T21:19:32.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3106e-05 2024-09-17T21:19:32.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:32.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.798494+0000", 2024-09-17T21:19:32.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3840000000000003e-06 2024-09-17T21:19:32.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:32.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.817936+0000", 2024-09-17T21:19:32.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019441643000000002 2024-09-17T21:19:32.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:32.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.818103+0000", 2024-09-17T21:19:32.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016643800000000001 2024-09-17T21:19:32.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:32.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:32.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:32.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.823074+0000", 2024-09-17T21:19:32.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0049710609999999997 2024-09-17T21:19:32.042 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:32.042 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:32.042 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:32.042 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:32.042 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:32.042 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:32.044 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:19:32.045 INFO:teuthology.orchestra.run.smithi184.stdout:3567 punch from 0x3107200 to 0x310da00, (0x6800 bytes) 2024-09-17T21:19:32.045 INFO:teuthology.orchestra.run.smithi184.stdout:3568 read 0x13fa200 thru 0x1409dff (0xfc00 bytes) 2024-09-17T21:19:32.045 INFO:teuthology.orchestra.run.smithi184.stdout:3570 write 0x3d8fa00 thru 0x3d9ebc7 (0xf1c8 bytes) 2024-09-17T21:19:32.045 INFO:teuthology.orchestra.run.smithi184.stdout:3571 read 0x2a70200 thru 0x2a71f76 (0x1d77 bytes) 2024-09-17T21:19:32.045 INFO:teuthology.orchestra.run.smithi184.stdout:3572 trunc from 0x4746800 to 0x4ab2200 2024-09-17T21:19:32.045 INFO:teuthology.orchestra.run.smithi184.stdout:3573 write 0x2d6d800 thru 0x2d7d3f4 (0xfbf5 bytes) 2024-09-17T21:19:32.046 INFO:teuthology.orchestra.run.smithi184.stdout:3574 punch from 0x591400 to 0x59c200, (0xae00 bytes) 2024-09-17T21:19:32.046 INFO:teuthology.orchestra.run.smithi184.stdout:3576 read 0x170da00 thru 0x1718ad5 (0xb0d6 bytes) 2024-09-17T21:19:32.046 INFO:teuthology.orchestra.run.smithi184.stdout:3579 read 0x2ce1e00 thru 0x2cef533 (0xd734 bytes) 2024-09-17T21:19:32.046 INFO:teuthology.orchestra.run.smithi184.stdout:3580 clone 20 order 22 su 2097152 sc 9 2024-09-17T21:19:32.046 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:32.038+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:32.145 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:32.140+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:32.174 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:32.174 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:32.174 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:19:32.174 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:32.188 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:19:32.221 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:19:32.269 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:32.266+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:32.345 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:32.340+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:32.355 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:32.355 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:32.356 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:32.356 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:32.356 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:32.370 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:19:32.374 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:32.374 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:32.390 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:32.391 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:32.407 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:32.407 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:32.425 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:32.425 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:32.443 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:32.444 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:32.458 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:32.456+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:32.461 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:32.461 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:32.537 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:32.537 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:32.546 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:32.544+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:32.551 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:19:32.647 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:32.642+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:32.721 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:32.721 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:32.725 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:19:32.725 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:19:32.726 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T21:19:32.726 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 0.661009 2024-09-17T21:19:32.726 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 0.6610094144046175 2024-09-17T21:19:32.737 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:19:32.748 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone19 from 0x4ab2200 (overlap 0x220bc00) to 0x367800 2024-09-17T21:19:32.776 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:32.774+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:32.849 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:32.844+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:32.865 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:32.865 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:32.880 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:19:32.950 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:32.946+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:33.049 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:33.049 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:33.050 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:33.048+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:33.064 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:19:33.151 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:33.146+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:33.192 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:33.193 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:33.207 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:19:33.215 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-17T21:19:33.252 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:33.248+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:33.353 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:33.348+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:33.375 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:33.375 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:33.390 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:19:33.454 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:33.450+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:33.517 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:33.517 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:33.531 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:19:33.573 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:33.570+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:33.585 INFO:teuthology.orchestra.run.smithi083.stderr:set osd.3 primary-affinity to 0.661009 (8433192) 2024-09-17T21:19:33.592 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:19:33.660 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:33.658+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:33.701 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:33.701 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:33.716 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:19:33.756 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:33.752+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:33.856 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:33.852+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:33.872 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:33.872 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:33.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:19:33.957 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:33.952+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:34.014 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:34.014 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:34.028 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:19:34.072 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:34.070+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:34.150 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:34.150 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:34.158 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:34.156+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:34.164 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:19:34.259 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:34.254+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:34.328 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:34.328 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:34.342 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:19:34.385 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:34.382+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:34.461 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:34.458+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:34.479 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:19:34.502 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:34.502 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:34.517 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:19:34.562 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:34.560+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:34.601 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-17T21:19:34.631 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:34.631 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:34.648 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:34.649 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:34.654 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:34.654 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:34.662 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:34.660+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:34.666 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:34.666 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:34.669 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:19:34.686 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:34.686 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:34.705 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:34.705 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:34.725 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:34.784 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:34.782+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:34.844 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:34.844 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:34.844 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:34.844 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:34.844 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:34.858 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:19:34.864 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:34.858+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:34.965 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:34.960+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:34.985 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:34.985 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:34.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:34.986 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:34.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.986 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:19:34.986 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:34.986 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 97.968725268, 2024-09-17T21:19:34.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:19:34.986 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:34.986 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:34.986 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:34.986 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:34.986 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:19:34.986 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:19:34.986 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:19:34.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.987 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:34.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:34.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:34.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:34.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:34.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:34.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:34.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:34.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:19:34.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:19:34.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:34.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:19:34.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:19:34.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:34.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:19:34.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:19:34.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:34.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:19:34.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:19:34.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:34.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:19:34.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:19:34.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:34.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:19:34.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:19:34.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:34.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:19:34.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:19:34.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:34.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:19:34.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:19:34.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:19:34.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:19:34.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:19:34.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:34.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:19:34.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:19:34.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:34.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:19:34.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:19:34.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:34.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:19:34.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:19:34.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.992 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:34.992 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:19:34.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:19:34.992 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:34.992 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:34.992 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:34.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.992 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:34.992 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:34.992 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 78.388560987999995, 2024-09-17T21:19:34.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:19:34.993 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:34.993 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:34.993 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:34.993 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:34.993 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:19:34.993 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:19:34.993 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:19:34.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.993 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:34.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:34.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:34.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:34.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:34.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:34.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:34.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:34.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:19:34.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:19:34.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:34.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:19:34.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:19:34.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:34.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:19:34.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:19:34.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:34.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:19:34.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:19:34.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:34.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:19:34.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:19:34.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:34.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:19:34.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:19:34.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:34.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:19:34.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:19:34.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:34.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:19:34.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:19:34.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:34.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:19:34.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:19:34.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:34.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:19:34.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:19:34.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:34.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:19:34.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:19:34.998 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:34.998 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:34.998 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:34.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:34.998 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:34.998 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:34.998 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 78.388463676000001, 2024-09-17T21:19:34.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:19:34.999 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:34.999 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:34.999 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:34.999 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:34.999 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:34.999 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:34.999 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:19:34.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:34.999 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:34.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:35.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:35.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:19:35.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:19:35.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:19:35.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:19:35.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:19:35.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:19:35.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:19:35.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:19:35.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:19:35.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:19:35.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:35.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:19:35.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:19:35.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:19:35.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:19:35.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:35.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:19:35.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:19:35.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:19:35.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:19:35.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:19:35.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:19:35.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:19:35.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.004 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:19:35.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:19:35.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.004 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:19:35.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:19:35.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:19:35.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:19:35.005 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.005 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.005 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.005 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:35.005 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:35.005 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 78.386376874999996, 2024-09-17T21:19:35.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:19:35.005 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.005 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.005 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:35.006 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.006 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:19:35.006 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:19:35.006 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:19:35.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.006 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:35.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:35.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:19:35.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:19:35.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:19:35.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:19:35.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:19:35.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:19:35.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:19:35.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:19:35.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:19:35.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:19:35.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:19:35.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:19:35.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:19:35.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:19:35.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:19:35.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:35.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:19:35.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:19:35.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:19:35.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:19:35.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:19:35.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:19:35.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:35.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:19:35.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:19:35.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:35.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:19:35.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:19:35.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:35.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:19:35.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:19:35.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:35.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:19:35.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:19:35.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:19:35.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:19:35.012 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.012 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.012 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.012 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:284 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:35.012 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:19:35.013 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 58.754554222000003, 2024-09-17T21:19:35.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024450084, 2024-09-17T21:19:35.013 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.013 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.013 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.013 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.013 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:35.013 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:35.013 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 284 2024-09-17T21:19:35.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.013 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:19:35.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227936+0000", 2024-09-17T21:19:35.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227937+0000", 2024-09-17T21:19:35.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6700000000000002e-07 2024-09-17T21:19:35.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227943+0000", 2024-09-17T21:19:35.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:19:35.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227961+0000", 2024-09-17T21:19:35.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.827e-05 2024-09-17T21:19:35.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227975+0000", 2024-09-17T21:19:35.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.394e-05 2024-09-17T21:19:35.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251977+0000", 2024-09-17T21:19:35.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024001340999999999 2024-09-17T21:19:35.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252169+0000", 2024-09-17T21:19:35.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019271199999999999 2024-09-17T21:19:35.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252387+0000", 2024-09-17T21:19:35.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021726100000000001 2024-09-17T21:19:35.016 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.017 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.017 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.017 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:285 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:35.017 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:19:35.017 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 58.754497932, 2024-09-17T21:19:35.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024884791, 2024-09-17T21:19:35.017 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.017 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.017 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.017 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.017 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:35.017 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:35.018 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 285 2024-09-17T21:19:35.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.018 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:19:35.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227993+0000", 2024-09-17T21:19:35.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227994+0000", 2024-09-17T21:19:35.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1429999999999999e-06 2024-09-17T21:19:35.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.227999+0000", 2024-09-17T21:19:35.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1669999999999998e-06 2024-09-17T21:19:35.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228015+0000", 2024-09-17T21:19:35.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6056e-05 2024-09-17T21:19:35.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228028+0000", 2024-09-17T21:19:35.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3196e-05 2024-09-17T21:19:35.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252458+0000", 2024-09-17T21:19:35.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024429751999999999 2024-09-17T21:19:35.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252650+0000", 2024-09-17T21:19:35.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019187800000000001 2024-09-17T21:19:35.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252878+0000", 2024-09-17T21:19:35.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022759899999999999 2024-09-17T21:19:35.021 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.021 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.021 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.021 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:286 3.11 3:88d9a867:::rbd_data.11527824718.0000000000000004:9 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:35.021 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:19:35.021 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 58.754442173000001, 2024-09-17T21:19:35.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025299318000000001, 2024-09-17T21:19:35.021 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.022 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.022 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.022 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.022 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:35.022 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:35.022 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 286 2024-09-17T21:19:35.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.022 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:19:35.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228049+0000", 2024-09-17T21:19:35.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228050+0000", 2024-09-17T21:19:35.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:19:35.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228055+0000", 2024-09-17T21:19:35.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4960000000000004e-06 2024-09-17T21:19:35.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228075+0000", 2024-09-17T21:19:35.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0109999999999999e-05 2024-09-17T21:19:35.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.228089+0000", 2024-09-17T21:19:35.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4154e-05 2024-09-17T21:19:35.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252952+0000", 2024-09-17T21:19:35.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024862934 2024-09-17T21:19:35.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253141+0000", 2024-09-17T21:19:35.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000188782 2024-09-17T21:19:35.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253348+0000", 2024-09-17T21:19:35.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020684499999999999 2024-09-17T21:19:35.025 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.025 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.026 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.026 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:35.026 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:35.026 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.702478094, 2024-09-17T21:19:35.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:19:35.026 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.026 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.026 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.026 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.026 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:35.026 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:35.026 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:19:35.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.027 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:35.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:35.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:19:35.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:19:35.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:19:35.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:19:35.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:19:35.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:19:35.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:19:35.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:19:35.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:19:35.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:19:35.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:35.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:19:35.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:19:35.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:19:35.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:19:35.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:35.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:19:35.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:19:35.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:19:35.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:19:35.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:19:35.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:19:35.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:19:35.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:19:35.031 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.032 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.032 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.032 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:35.032 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:35.032 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.693386383000004, 2024-09-17T21:19:35.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:19:35.032 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.032 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.032 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.032 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.032 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:35.033 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:35.033 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:19:35.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.033 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:35.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:35.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:19:35.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:19:35.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:19:35.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:19:35.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:19:35.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:19:35.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:19:35.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:19:35.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:19:35.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:19:35.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:35.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:19:35.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:19:35.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:19:35.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:19:35.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:19:35.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:19:35.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:19:35.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:19:35.037 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.037 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.037 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.037 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:35.037 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:35.037 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.693214331999997, 2024-09-17T21:19:35.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:19:35.037 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.037 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.037 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.038 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.038 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:35.038 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:35.038 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:19:35.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.038 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:35.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:35.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:19:35.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:19:35.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:19:35.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:19:35.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:19:35.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:19:35.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:19:35.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:19:35.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:19:35.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:19:35.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:35.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:19:35.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:19:35.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:19:35.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:19:35.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:19:35.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:19:35.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:19:35.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:19:35.042 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.042 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.042 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.042 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:35.042 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:35.042 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.518281584999997, 2024-09-17T21:19:35.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:19:35.042 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.042 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.042 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.043 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.043 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:35.043 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:35.043 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:19:35.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.043 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:35.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:35.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:19:35.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:19:35.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:19:35.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:19:35.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:19:35.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:19:35.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:19:35.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:19:35.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:19:35.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:19:35.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:35.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:19:35.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:19:35.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:19:35.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:19:35.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:19:35.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:19:35.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:19:35.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:19:35.047 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.047 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.047 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.047 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:35.047 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:35.047 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.518197028000003, 2024-09-17T21:19:35.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:19:35.048 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.048 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.048 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.048 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.048 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:35.048 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:35.048 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:19:35.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.048 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:35.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:35.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:35.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:19:35.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:19:35.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:19:35.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:19:35.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:19:35.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:19:35.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:19:35.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:19:35.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:19:35.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:35.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:19:35.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:19:35.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:19:35.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:19:35.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:19:35.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:19:35.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:19:35.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:19:35.052 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.052 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.052 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.052 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:35.052 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:35.052 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.518131390999997, 2024-09-17T21:19:35.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:19:35.052 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.052 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.053 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.053 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.053 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:35.053 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:35.053 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:19:35.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.053 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:35.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:35.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:19:35.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:19:35.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:19:35.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:19:35.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:19:35.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:19:35.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:19:35.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:19:35.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:19:35.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:19:35.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:35.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:19:35.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:19:35.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:19:35.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:19:35.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:19:35.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:19:35.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:19:35.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:19:35.057 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.057 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.057 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.057 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:10979 3.a 3:53159310:::rbd_header.1152724ff2a3:head [watch reconnect cookie 139860846458096 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-17T21:19:35.057 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:26.748362+0000", 2024-09-17T21:19:35.057 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.2341283450000002, 2024-09-17T21:19:35.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025943728999999999, 2024-09-17T21:19:35.058 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.058 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.058 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.058 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.058 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:35.058 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:35.058 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10979 2024-09-17T21:19:35.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.058 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.748362+0000", 2024-09-17T21:19:35.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.748362+0000", 2024-09-17T21:19:35.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.748364+0000", 2024-09-17T21:19:35.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9039999999999999e-06 2024-09-17T21:19:35.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.748370+0000", 2024-09-17T21:19:35.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5890000000000002e-06 2024-09-17T21:19:35.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.748393+0000", 2024-09-17T21:19:35.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3462999999999999e-05 2024-09-17T21:19:35.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.748406+0000", 2024-09-17T21:19:35.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2317000000000001e-05 2024-09-17T21:19:35.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.767146+0000", 2024-09-17T21:19:35.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018740271999999999 2024-09-17T21:19:35.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.767277+0000", 2024-09-17T21:19:35.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013130300000000001 2024-09-17T21:19:35.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.774306+0000", 2024-09-17T21:19:35.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0070288809999999998 2024-09-17T21:19:35.061 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.061 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.061 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.062 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4978.0:564 3.a 3:53159310:::rbd_header.1152724ff2a3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-17T21:19:35.062 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:26.750343+0000", 2024-09-17T21:19:35.062 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.2321475419999999, 2024-09-17T21:19:35.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024755837999999999, 2024-09-17T21:19:35.062 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.062 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.062 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.062 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.062 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4978", 2024-09-17T21:19:35.062 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2995854560", 2024-09-17T21:19:35.062 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 564 2024-09-17T21:19:35.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.063 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750343+0000", 2024-09-17T21:19:35.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750343+0000", 2024-09-17T21:19:35.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750344+0000", 2024-09-17T21:19:35.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0669999999999999e-06 2024-09-17T21:19:35.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750348+0000", 2024-09-17T21:19:35.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9600000000000002e-06 2024-09-17T21:19:35.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750357+0000", 2024-09-17T21:19:35.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9560000000000003e-06 2024-09-17T21:19:35.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750380+0000", 2024-09-17T21:19:35.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2376999999999999e-05 2024-09-17T21:19:35.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.774378+0000", 2024-09-17T21:19:35.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023998609000000001 2024-09-17T21:19:35.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.774930+0000", 2024-09-17T21:19:35.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00055168100000000005 2024-09-17T21:19:35.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.775099+0000", 2024-09-17T21:19:35.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016918799999999999 2024-09-17T21:19:35.066 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.066 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.066 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.066 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4978.0:565 3.a 3:53159310:::rbd_header.1152724ff2a3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-17T21:19:35.066 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:26.750461+0000", 2024-09-17T21:19:35.066 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.2320293939999996, 2024-09-17T21:19:35.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024983353999999999, 2024-09-17T21:19:35.066 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.066 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.066 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.066 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.066 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4978", 2024-09-17T21:19:35.067 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2995854560", 2024-09-17T21:19:35.067 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 565 2024-09-17T21:19:35.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.067 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750461+0000", 2024-09-17T21:19:35.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750461+0000", 2024-09-17T21:19:35.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750464+0000", 2024-09-17T21:19:35.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3130000000000001e-06 2024-09-17T21:19:35.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750472+0000", 2024-09-17T21:19:35.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2130000000000008e-06 2024-09-17T21:19:35.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750491+0000", 2024-09-17T21:19:35.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.927e-05 2024-09-17T21:19:35.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750505+0000", 2024-09-17T21:19:35.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3726999999999999e-05 2024-09-17T21:19:35.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.775162+0000", 2024-09-17T21:19:35.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024657070999999999 2024-09-17T21:19:35.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.775289+0000", 2024-09-17T21:19:35.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012754899999999999 2024-09-17T21:19:35.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.775445+0000", 2024-09-17T21:19:35.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015521099999999999 2024-09-17T21:19:35.070 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.070 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.070 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.070 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4978.0:567 3.a 3:53159310:::rbd_header.1152724ff2a3:head [notify cookie 140612330492592] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-17T21:19:35.070 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:26.750648+0000", 2024-09-17T21:19:35.070 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.2318429270000006, 2024-09-17T21:19:35.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026360126000000001, 2024-09-17T21:19:35.070 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.070 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.071 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4978", 2024-09-17T21:19:35.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2995854560", 2024-09-17T21:19:35.071 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 567 2024-09-17T21:19:35.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.071 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750648+0000", 2024-09-17T21:19:35.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750648+0000", 2024-09-17T21:19:35.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750650+0000", 2024-09-17T21:19:35.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8330000000000001e-06 2024-09-17T21:19:35.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750659+0000", 2024-09-17T21:19:35.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8160000000000005e-06 2024-09-17T21:19:35.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750695+0000", 2024-09-17T21:19:35.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5666e-05 2024-09-17T21:19:35.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.750712+0000", 2024-09-17T21:19:35.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7003999999999998e-05 2024-09-17T21:19:35.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.775505+0000", 2024-09-17T21:19:35.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024792742999999999 2024-09-17T21:19:35.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.775660+0000", 2024-09-17T21:19:35.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015555500000000001 2024-09-17T21:19:35.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.777008+0000", 2024-09-17T21:19:35.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001347509 2024-09-17T21:19:35.074 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.074 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.074 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.074 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4978.0:568 3.a 3:53159310:::rbd_header.1152724ff2a3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-17T21:19:35.074 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:26.751223+0000", 2024-09-17T21:19:35.075 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.2312672920000001, 2024-09-17T21:19:35.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025333333999999999, 2024-09-17T21:19:35.075 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.075 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.075 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.075 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.075 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4978", 2024-09-17T21:19:35.075 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2995854560", 2024-09-17T21:19:35.075 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 568 2024-09-17T21:19:35.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.075 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.751223+0000", 2024-09-17T21:19:35.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.751223+0000", 2024-09-17T21:19:35.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.751226+0000", 2024-09-17T21:19:35.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4770000000000002e-06 2024-09-17T21:19:35.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.751238+0000", 2024-09-17T21:19:35.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1739999999999999e-05 2024-09-17T21:19:35.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.751277+0000", 2024-09-17T21:19:35.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9816e-05 2024-09-17T21:19:35.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.751295+0000", 2024-09-17T21:19:35.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7283e-05 2024-09-17T21:19:35.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.776198+0000", 2024-09-17T21:19:35.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024903444 2024-09-17T21:19:35.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.776387+0000", 2024-09-17T21:19:35.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000188522 2024-09-17T21:19:35.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.776557+0000", 2024-09-17T21:19:35.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017005199999999999 2024-09-17T21:19:35.078 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.079 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.079 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.079 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4978.0:575 3.a 3:53159310:::rbd_header.1152724ff2a3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-17T21:19:35.079 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:26.752087+0000", 2024-09-17T21:19:35.079 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.2304039810000003, 2024-09-17T21:19:35.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024864818, 2024-09-17T21:19:35.079 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.079 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.079 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.079 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.079 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4978", 2024-09-17T21:19:35.079 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2995854560", 2024-09-17T21:19:35.080 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 575 2024-09-17T21:19:35.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.080 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.752087+0000", 2024-09-17T21:19:35.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.752087+0000", 2024-09-17T21:19:35.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.752090+0000", 2024-09-17T21:19:35.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8569999999999999e-06 2024-09-17T21:19:35.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.752103+0000", 2024-09-17T21:19:35.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3752e-05 2024-09-17T21:19:35.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.752136+0000", 2024-09-17T21:19:35.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2403000000000001e-05 2024-09-17T21:19:35.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.752157+0000", 2024-09-17T21:19:35.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0973000000000001e-05 2024-09-17T21:19:35.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.776623+0000", 2024-09-17T21:19:35.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024466517 2024-09-17T21:19:35.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.776762+0000", 2024-09-17T21:19:35.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000139159 2024-09-17T21:19:35.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.776952+0000", 2024-09-17T21:19:35.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018915699999999999 2024-09-17T21:19:35.083 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.083 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.083 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.083 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4978.0:755 3.a 3:538046bd:::rbd_data.1152724ff2a3.000000000000006a:head [stat,delete] snapc 14=[14] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-17T21:19:35.083 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:26.798467+0000", 2024-09-17T21:19:35.083 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.1840239229999998, 2024-09-17T21:19:35.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024606805999999998, 2024-09-17T21:19:35.084 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.084 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.084 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.084 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.084 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4978", 2024-09-17T21:19:35.084 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2995854560", 2024-09-17T21:19:35.084 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 755 2024-09-17T21:19:35.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.084 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.798467+0000", 2024-09-17T21:19:35.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.798467+0000", 2024-09-17T21:19:35.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.798468+0000", 2024-09-17T21:19:35.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4200000000000004e-07 2024-09-17T21:19:35.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.798472+0000", 2024-09-17T21:19:35.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2320000000000003e-06 2024-09-17T21:19:35.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.798485+0000", 2024-09-17T21:19:35.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3106e-05 2024-09-17T21:19:35.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.798494+0000", 2024-09-17T21:19:35.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3840000000000003e-06 2024-09-17T21:19:35.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.817936+0000", 2024-09-17T21:19:35.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019441643000000002 2024-09-17T21:19:35.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.818103+0000", 2024-09-17T21:19:35.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016643800000000001 2024-09-17T21:19:35.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.823074+0000", 2024-09-17T21:19:35.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0049710609999999997 2024-09-17T21:19:35.087 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.087 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.087 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.087 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.088 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.088 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:35.089 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:19:35.089 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:35.084+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:35.190 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:35.188+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:35.262 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:35.262 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:35.262 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:19:35.262 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:35.276 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:19:35.291 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:35.286+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:35.392 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:35.388+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:35.439 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:35.439 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:35.440 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:35.440 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:35.440 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:35.453 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:19:35.492 INFO:teuthology.orchestra.run.smithi184.stdout:3581 read 0x3e75000 thru 0x3e81850 (0xc851 bytes) 2024-09-17T21:19:35.495 INFO:teuthology.orchestra.run.smithi184.stdout:3583 read 0x36cf400 thru 0x36d175e (0x235f bytes) 2024-09-17T21:19:35.498 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:35.496+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:35.513 INFO:teuthology.orchestra.run.smithi184.stdout:3588 trunc from 0x4ab2200 to 0x19e5800 2024-09-17T21:19:35.554 INFO:teuthology.orchestra.run.smithi184.stdout:3591 read 0x6e9000 thru 0x6eddc6 (0x4dc7 bytes) 2024-09-17T21:19:35.557 INFO:teuthology.orchestra.run.smithi184.stdout:3594 read 0x1772800 thru 0x17814ee (0xecef bytes) 2024-09-17T21:19:35.558 INFO:teuthology.orchestra.run.smithi184.stdout:3595 read 0x1935000 thru 0x193d3f1 (0x83f2 bytes) 2024-09-17T21:19:35.559 INFO:teuthology.orchestra.run.smithi184.stdout:3597 read 0x1590600 thru 0x15980c5 (0x7ac6 bytes) 2024-09-17T21:19:35.560 INFO:teuthology.orchestra.run.smithi184.stdout:3598 punch from 0x326600 to 0x328600, (0x2000 bytes) 2024-09-17T21:19:35.594 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:35.588+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:35.634 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:35.634 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:35.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:35.635 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:35.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.635 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:480 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:35.635 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:19:35.635 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.396927802, 2024-09-17T21:19:35.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017902482000000001, 2024-09-17T21:19:35.635 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.635 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.635 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.635 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.635 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:35.635 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:35.636 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 480 2024-09-17T21:19:35.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.636 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:19:35.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233967+0000", 2024-09-17T21:19:35.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233969+0000", 2024-09-17T21:19:35.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.454e-06 2024-09-17T21:19:35.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233981+0000", 2024-09-17T21:19:35.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1593000000000001e-05 2024-09-17T21:19:35.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.233999+0000", 2024-09-17T21:19:35.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.806e-05 2024-09-17T21:19:35.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234012+0000", 2024-09-17T21:19:35.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:19:35.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251339+0000", 2024-09-17T21:19:35.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017326352 2024-09-17T21:19:35.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251534+0000", 2024-09-17T21:19:35.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000195842 2024-09-17T21:19:35.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251870+0000", 2024-09-17T21:19:35.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033551400000000003 2024-09-17T21:19:35.639 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.639 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.640 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.640 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:481 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1114112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:35.640 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:19:35.640 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.396859435000003, 2024-09-17T21:19:35.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018442621999999999, 2024-09-17T21:19:35.640 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.640 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.640 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.640 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.640 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:35.640 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:35.641 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 481 2024-09-17T21:19:35.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.641 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:19:35.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234036+0000", 2024-09-17T21:19:35.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234037+0000", 2024-09-17T21:19:35.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:19:35.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234043+0000", 2024-09-17T21:19:35.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.49e-06 2024-09-17T21:19:35.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234061+0000", 2024-09-17T21:19:35.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8416000000000001e-05 2024-09-17T21:19:35.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234076+0000", 2024-09-17T21:19:35.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.456e-05 2024-09-17T21:19:35.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.251948+0000", 2024-09-17T21:19:35.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017872774000000001 2024-09-17T21:19:35.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252144+0000", 2024-09-17T21:19:35.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019517300000000001 2024-09-17T21:19:35.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252478+0000", 2024-09-17T21:19:35.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033482899999999998 2024-09-17T21:19:35.644 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.644 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.644 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.645 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:482 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:35.645 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:19:35.645 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.396800882999997, 2024-09-17T21:19:35.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018988026000000002, 2024-09-17T21:19:35.645 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.645 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.645 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.646 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.646 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:35.646 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:35.646 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 482 2024-09-17T21:19:35.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.646 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:19:35.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234094+0000", 2024-09-17T21:19:35.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234096+0000", 2024-09-17T21:19:35.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2139999999999999e-06 2024-09-17T21:19:35.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234101+0000", 2024-09-17T21:19:35.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7100000000000004e-06 2024-09-17T21:19:35.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234118+0000", 2024-09-17T21:19:35.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6486000000000001e-05 2024-09-17T21:19:35.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234131+0000", 2024-09-17T21:19:35.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2824e-05 2024-09-17T21:19:35.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252552+0000", 2024-09-17T21:19:35.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018421549999999998 2024-09-17T21:19:35.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.252746+0000", 2024-09-17T21:19:35.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019424399999999999 2024-09-17T21:19:35.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253082+0000", 2024-09-17T21:19:35.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000335998 2024-09-17T21:19:35.650 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.650 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.650 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.650 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:483 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:35.650 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:19:35.650 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.396746262999997, 2024-09-17T21:19:35.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019462882000000001, 2024-09-17T21:19:35.650 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.650 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.651 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.651 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.651 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:35.651 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:35.651 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 483 2024-09-17T21:19:35.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.651 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:19:35.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234149+0000", 2024-09-17T21:19:35.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234150+0000", 2024-09-17T21:19:35.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.127e-06 2024-09-17T21:19:35.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234155+0000", 2024-09-17T21:19:35.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:19:35.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234171+0000", 2024-09-17T21:19:35.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6160000000000001e-05 2024-09-17T21:19:35.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234185+0000", 2024-09-17T21:19:35.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3370000000000001e-05 2024-09-17T21:19:35.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253158+0000", 2024-09-17T21:19:35.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018972968999999999 2024-09-17T21:19:35.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253349+0000", 2024-09-17T21:19:35.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000190733 2024-09-17T21:19:35.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253612+0000", 2024-09-17T21:19:35.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026337000000000002 2024-09-17T21:19:35.654 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.655 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.655 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.655 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:484 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:35.655 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:19:35.655 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.396692237000003, 2024-09-17T21:19:35.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020029261999999999, 2024-09-17T21:19:35.655 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.655 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.655 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.655 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.655 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:35.655 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:35.655 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 484 2024-09-17T21:19:35.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.656 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:19:35.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234203+0000", 2024-09-17T21:19:35.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234205+0000", 2024-09-17T21:19:35.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6470000000000001e-06 2024-09-17T21:19:35.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234210+0000", 2024-09-17T21:19:35.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:19:35.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234227+0000", 2024-09-17T21:19:35.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.73e-05 2024-09-17T21:19:35.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234241+0000", 2024-09-17T21:19:35.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3757000000000001e-05 2024-09-17T21:19:35.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253690+0000", 2024-09-17T21:19:35.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019449368000000002 2024-09-17T21:19:35.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.253898+0000", 2024-09-17T21:19:35.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000207161 2024-09-17T21:19:35.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254232+0000", 2024-09-17T21:19:35.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033474299999999998 2024-09-17T21:19:35.659 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.659 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.659 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.659 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:485 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1638400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:35.659 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:19:35.659 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.396635009999997, 2024-09-17T21:19:35.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020553914, 2024-09-17T21:19:35.660 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.660 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.660 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:35.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:35.660 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 485 2024-09-17T21:19:35.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.660 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:19:35.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234260+0000", 2024-09-17T21:19:35.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234261+0000", 2024-09-17T21:19:35.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.136e-06 2024-09-17T21:19:35.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234266+0000", 2024-09-17T21:19:35.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0669999999999999e-06 2024-09-17T21:19:35.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234284+0000", 2024-09-17T21:19:35.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7839999999999999e-05 2024-09-17T21:19:35.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234298+0000", 2024-09-17T21:19:35.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3852999999999999e-05 2024-09-17T21:19:35.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254311+0000", 2024-09-17T21:19:35.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020012973 2024-09-17T21:19:35.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254552+0000", 2024-09-17T21:19:35.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024116400000000001 2024-09-17T21:19:35.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254814+0000", 2024-09-17T21:19:35.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000261881 2024-09-17T21:19:35.663 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.663 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.664 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.664 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:486 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:35.664 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:19:35.664 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.396578621000003, 2024-09-17T21:19:35.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021010240999999999, 2024-09-17T21:19:35.664 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.664 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.664 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.665 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.665 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:35.665 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:35.665 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 486 2024-09-17T21:19:35.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.665 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:19:35.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234317+0000", 2024-09-17T21:19:35.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234318+0000", 2024-09-17T21:19:35.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.37e-06 2024-09-17T21:19:35.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234324+0000", 2024-09-17T21:19:35.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:19:35.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234341+0000", 2024-09-17T21:19:35.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7127000000000001e-05 2024-09-17T21:19:35.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234354+0000", 2024-09-17T21:19:35.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3210000000000001e-05 2024-09-17T21:19:35.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.254896+0000", 2024-09-17T21:19:35.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020541835000000001 2024-09-17T21:19:35.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255092+0000", 2024-09-17T21:19:35.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019634700000000001 2024-09-17T21:19:35.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255327+0000", 2024-09-17T21:19:35.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023475899999999999 2024-09-17T21:19:35.668 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.668 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.669 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.669 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:487 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:35.669 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:19:35.669 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.396522425000001, 2024-09-17T21:19:35.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021453911999999999, 2024-09-17T21:19:35.669 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.669 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.669 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.669 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.669 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:35.669 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:35.669 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 487 2024-09-17T21:19:35.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.670 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:19:35.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234373+0000", 2024-09-17T21:19:35.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234374+0000", 2024-09-17T21:19:35.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3039999999999999e-06 2024-09-17T21:19:35.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234380+0000", 2024-09-17T21:19:35.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:19:35.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234397+0000", 2024-09-17T21:19:35.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7232999999999999e-05 2024-09-17T21:19:35.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234411+0000", 2024-09-17T21:19:35.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3784000000000001e-05 2024-09-17T21:19:35.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255406+0000", 2024-09-17T21:19:35.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020995204 2024-09-17T21:19:35.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255595+0000", 2024-09-17T21:19:35.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018903299999999999 2024-09-17T21:19:35.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255827+0000", 2024-09-17T21:19:35.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023190099999999999 2024-09-17T21:19:35.673 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.673 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.673 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.673 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:488 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:35.673 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:19:35.673 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.396466748000002, 2024-09-17T21:19:35.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026887966999999999, 2024-09-17T21:19:35.673 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.674 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.674 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.674 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.674 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:35.674 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:35.674 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 488 2024-09-17T21:19:35.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.674 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:19:35.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234429+0000", 2024-09-17T21:19:35.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234430+0000", 2024-09-17T21:19:35.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1829999999999999e-06 2024-09-17T21:19:35.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234435+0000", 2024-09-17T21:19:35.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9869999999999999e-06 2024-09-17T21:19:35.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234451+0000", 2024-09-17T21:19:35.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6458999999999999e-05 2024-09-17T21:19:35.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234465+0000", 2024-09-17T21:19:35.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3730000000000001e-05 2024-09-17T21:19:35.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.255908+0000", 2024-09-17T21:19:35.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021443413000000001 2024-09-17T21:19:35.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256098+0000", 2024-09-17T21:19:35.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018961 2024-09-17T21:19:35.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261317+0000", 2024-09-17T21:19:35.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052185850000000004 2024-09-17T21:19:35.678 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.678 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.678 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.678 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:489 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2162688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:35.678 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:19:35.678 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.396410908999997, 2024-09-17T21:19:35.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022632815000000001, 2024-09-17T21:19:35.678 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.678 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.678 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.678 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.678 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:35.679 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:35.679 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 489 2024-09-17T21:19:35.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.679 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:19:35.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234484+0000", 2024-09-17T21:19:35.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234486+0000", 2024-09-17T21:19:35.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5099999999999999e-06 2024-09-17T21:19:35.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234491+0000", 2024-09-17T21:19:35.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.147e-06 2024-09-17T21:19:35.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234508+0000", 2024-09-17T21:19:35.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7405999999999999e-05 2024-09-17T21:19:35.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234522+0000", 2024-09-17T21:19:35.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3677e-05 2024-09-17T21:19:35.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256536+0000", 2024-09-17T21:19:35.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022013464999999999 2024-09-17T21:19:35.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.256752+0000", 2024-09-17T21:19:35.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021637800000000001 2024-09-17T21:19:35.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257117+0000", 2024-09-17T21:19:35.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000365232 2024-09-17T21:19:35.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.682 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.683 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:490 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2293760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:35.683 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:19:35.683 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.396356119000004, 2024-09-17T21:19:35.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023146462999999999, 2024-09-17T21:19:35.683 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.683 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.683 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:35.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:35.683 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 490 2024-09-17T21:19:35.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.684 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:19:35.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234539+0000", 2024-09-17T21:19:35.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234541+0000", 2024-09-17T21:19:35.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4869999999999999e-06 2024-09-17T21:19:35.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234559+0000", 2024-09-17T21:19:35.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8105999999999999e-05 2024-09-17T21:19:35.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234577+0000", 2024-09-17T21:19:35.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.836e-05 2024-09-17T21:19:35.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234592+0000", 2024-09-17T21:19:35.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4766000000000001e-05 2024-09-17T21:19:35.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257187+0000", 2024-09-17T21:19:35.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022594725 2024-09-17T21:19:35.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257374+0000", 2024-09-17T21:19:35.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018767299999999999 2024-09-17T21:19:35.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257686+0000", 2024-09-17T21:19:35.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000311346 2024-09-17T21:19:35.687 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.687 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.687 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.687 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:491 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:35.687 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:19:35.688 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.396283510000004, 2024-09-17T21:19:35.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023696872000000001, 2024-09-17T21:19:35.688 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.688 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.688 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.688 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.688 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:35.688 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:35.688 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 491 2024-09-17T21:19:35.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.688 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:19:35.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234612+0000", 2024-09-17T21:19:35.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234613+0000", 2024-09-17T21:19:35.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-06 2024-09-17T21:19:35.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234618+0000", 2024-09-17T21:19:35.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3839999999999996e-06 2024-09-17T21:19:35.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234635+0000", 2024-09-17T21:19:35.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.685e-05 2024-09-17T21:19:35.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234648+0000", 2024-09-17T21:19:35.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.313e-05 2024-09-17T21:19:35.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257780+0000", 2024-09-17T21:19:35.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023131946 2024-09-17T21:19:35.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.257985+0000", 2024-09-17T21:19:35.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020468400000000001 2024-09-17T21:19:35.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258309+0000", 2024-09-17T21:19:35.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032357499999999999 2024-09-17T21:19:35.691 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.692 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.692 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.692 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:492 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:35.692 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:19:35.692 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.396227922999998, 2024-09-17T21:19:35.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024258994999999998, 2024-09-17T21:19:35.692 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.692 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.692 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:35.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:35.693 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 492 2024-09-17T21:19:35.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.693 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:19:35.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234667+0000", 2024-09-17T21:19:35.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234669+0000", 2024-09-17T21:19:35.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.313e-06 2024-09-17T21:19:35.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234674+0000", 2024-09-17T21:19:35.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1599999999999997e-06 2024-09-17T21:19:35.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234690+0000", 2024-09-17T21:19:35.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.666e-05 2024-09-17T21:19:35.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234704+0000", 2024-09-17T21:19:35.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3043e-05 2024-09-17T21:19:35.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258397+0000", 2024-09-17T21:19:35.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023693204999999998 2024-09-17T21:19:35.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258593+0000", 2024-09-17T21:19:35.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019578999999999999 2024-09-17T21:19:35.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.258926+0000", 2024-09-17T21:19:35.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033382399999999998 2024-09-17T21:19:35.696 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.696 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.696 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.696 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:493 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:35.697 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:19:35.697 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.396171457999998, 2024-09-17T21:19:35.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024793461999999999, 2024-09-17T21:19:35.697 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.697 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.697 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.697 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.697 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:35.697 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:35.697 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 493 2024-09-17T21:19:35.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.697 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:19:35.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234724+0000", 2024-09-17T21:19:35.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234725+0000", 2024-09-17T21:19:35.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.24e-06 2024-09-17T21:19:35.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234731+0000", 2024-09-17T21:19:35.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8939999999999998e-06 2024-09-17T21:19:35.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234749+0000", 2024-09-17T21:19:35.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7863000000000002e-05 2024-09-17T21:19:35.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234762+0000", 2024-09-17T21:19:35.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3322999999999999e-05 2024-09-17T21:19:35.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259022+0000", 2024-09-17T21:19:35.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024259691999999999 2024-09-17T21:19:35.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259259+0000", 2024-09-17T21:19:35.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023669199999999999 2024-09-17T21:19:35.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259517+0000", 2024-09-17T21:19:35.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025875799999999999 2024-09-17T21:19:35.701 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.701 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.701 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.701 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:494 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:35.701 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:19:35.701 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.396113757999998, 2024-09-17T21:19:35.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025251952000000001, 2024-09-17T21:19:35.701 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.701 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.702 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:35.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:35.702 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 494 2024-09-17T21:19:35.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.702 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:19:35.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234782+0000", 2024-09-17T21:19:35.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234783+0000", 2024-09-17T21:19:35.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2440000000000001e-06 2024-09-17T21:19:35.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234789+0000", 2024-09-17T21:19:35.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:19:35.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234806+0000", 2024-09-17T21:19:35.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7450000000000001e-05 2024-09-17T21:19:35.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234819+0000", 2024-09-17T21:19:35.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3326000000000001e-05 2024-09-17T21:19:35.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259585+0000", 2024-09-17T21:19:35.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024765579999999999 2024-09-17T21:19:35.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259787+0000", 2024-09-17T21:19:35.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202444 2024-09-17T21:19:35.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260033+0000", 2024-09-17T21:19:35.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024611499999999998 2024-09-17T21:19:35.705 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.705 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.705 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.706 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:495 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:35.706 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:19:35.706 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.396058394999997, 2024-09-17T21:19:35.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025776342000000001, 2024-09-17T21:19:35.706 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.706 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.706 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.706 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.706 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:35.706 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:35.706 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 495 2024-09-17T21:19:35.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.707 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:19:35.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234837+0000", 2024-09-17T21:19:35.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234838+0000", 2024-09-17T21:19:35.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2130000000000001e-06 2024-09-17T21:19:35.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234843+0000", 2024-09-17T21:19:35.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9899999999999997e-06 2024-09-17T21:19:35.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234860+0000", 2024-09-17T21:19:35.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6973000000000002e-05 2024-09-17T21:19:35.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234873+0000", 2024-09-17T21:19:35.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3179999999999999e-05 2024-09-17T21:19:35.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260135+0000", 2024-09-17T21:19:35.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025262118 2024-09-17T21:19:35.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260364+0000", 2024-09-17T21:19:35.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022830699999999999 2024-09-17T21:19:35.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260613+0000", 2024-09-17T21:19:35.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024956099999999998 2024-09-17T21:19:35.710 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.710 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.710 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.710 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:496 3.1a 3:5bb38ad9:::rbd_data.11527824718.0000000000000002:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:35.710 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:19:35.710 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.396005115000001, 2024-09-17T21:19:35.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026376773999999999, 2024-09-17T21:19:35.710 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.711 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.711 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.711 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.711 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:35.711 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:35.711 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 496 2024-09-17T21:19:35.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.711 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:19:35.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234890+0000", 2024-09-17T21:19:35.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234891+0000", 2024-09-17T21:19:35.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1230000000000001e-06 2024-09-17T21:19:35.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234896+0000", 2024-09-17T21:19:35.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9599999999999999e-06 2024-09-17T21:19:35.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234913+0000", 2024-09-17T21:19:35.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6747000000000001e-05 2024-09-17T21:19:35.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.234926+0000", 2024-09-17T21:19:35.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3030000000000001e-05 2024-09-17T21:19:35.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260675+0000", 2024-09-17T21:19:35.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025748979000000002 2024-09-17T21:19:35.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260883+0000", 2024-09-17T21:19:35.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020848399999999999 2024-09-17T21:19:35.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.261267+0000", 2024-09-17T21:19:35.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038345099999999998 2024-09-17T21:19:35.715 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.715 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.715 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.715 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:682 3.9 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2455552~12288,write 8126464~10240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:35.716 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:19:35.716 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.362068063999999, 2024-09-17T21:19:35.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018447261999999999, 2024-09-17T21:19:35.716 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.716 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.716 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:35.716 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.716 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:35.716 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:35.716 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 682 2024-09-17T21:19:35.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.716 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:19:35.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268827+0000", 2024-09-17T21:19:35.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.268830+0000", 2024-09-17T21:19:35.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6419999999999999e-06 2024-09-17T21:19:35.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271673+0000", 2024-09-17T21:19:35.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028432610000000001 2024-09-17T21:19:35.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271700+0000", 2024-09-17T21:19:35.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7200999999999999e-05 2024-09-17T21:19:35.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271715+0000", 2024-09-17T21:19:35.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5145e-05 2024-09-17T21:19:35.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271785+0000", 2024-09-17T21:19:35.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9519999999999998e-05 2024-09-17T21:19:35.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.271918+0000", 2024-09-17T21:19:35.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013330899999999999 2024-09-17T21:19:35.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:35.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.272449+0000", 2024-09-17T21:19:35.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00053051200000000002 2024-09-17T21:19:35.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:35.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.278822+0000", 2024-09-17T21:19:35.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0063731619999999999 2024-09-17T21:19:35.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:35.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287167+0000", 2024-09-17T21:19:35.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083452149999999996 2024-09-17T21:19:35.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:35.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287213+0000", 2024-09-17T21:19:35.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5972e-05 2024-09-17T21:19:35.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.287274+0000", 2024-09-17T21:19:35.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1322999999999995e-05 2024-09-17T21:19:35.721 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.721 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.721 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.722 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4978.0:619 3.1d 3:bf63d511:::rbd_data.1152724ff2a3.0000000000000041:head [stat,delete] snapc 14=[14] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-17T21:19:35.722 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:26.764551+0000", 2024-09-17T21:19:35.722 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.8663445920000008, 2024-09-17T21:19:35.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021971748999999999, 2024-09-17T21:19:35.722 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.722 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.722 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:35.722 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.722 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4978", 2024-09-17T21:19:35.722 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2995854560", 2024-09-17T21:19:35.722 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 619 2024-09-17T21:19:35.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.722 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.764551+0000", 2024-09-17T21:19:35.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.764551+0000", 2024-09-17T21:19:35.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.764552+0000", 2024-09-17T21:19:35.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.424e-06 2024-09-17T21:19:35.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.764560+0000", 2024-09-17T21:19:35.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7970000000000001e-06 2024-09-17T21:19:35.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.764585+0000", 2024-09-17T21:19:35.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4875000000000002e-05 2024-09-17T21:19:35.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.764600+0000", 2024-09-17T21:19:35.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4851999999999999e-05 2024-09-17T21:19:35.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.764712+0000", 2024-09-17T21:19:35.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112402 2024-09-17T21:19:35.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.765264+0000", 2024-09-17T21:19:35.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00055208500000000001 2024-09-17T21:19:35.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.786522+0000", 2024-09-17T21:19:35.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021258314 2024-09-17T21:19:35.726 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.726 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.726 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.726 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4978.0:642 3.1d 3:ba5eb57e:::rbd_data.1152724ff2a3.000000000000004a:head [stat,delete] snapc 14=[14] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-17T21:19:35.726 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:26.772024+0000", 2024-09-17T21:19:35.726 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.8588717060000004, 2024-09-17T21:19:35.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021513560000000001, 2024-09-17T21:19:35.726 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:35.726 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:35.726 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:35.727 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:35.727 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4978", 2024-09-17T21:19:35.727 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2995854560", 2024-09-17T21:19:35.727 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 642 2024-09-17T21:19:35.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.727 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:35.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:35.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.772024+0000", 2024-09-17T21:19:35.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:35.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.772024+0000", 2024-09-17T21:19:35.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:35.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:35.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.772025+0000", 2024-09-17T21:19:35.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5939999999999999e-06 2024-09-17T21:19:35.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:35.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.772033+0000", 2024-09-17T21:19:35.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5689999999999999e-06 2024-09-17T21:19:35.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:35.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.772054+0000", 2024-09-17T21:19:35.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1084999999999999e-05 2024-09-17T21:19:35.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:35.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.772097+0000", 2024-09-17T21:19:35.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3572999999999998e-05 2024-09-17T21:19:35.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:35.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.772232+0000", 2024-09-17T21:19:35.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013453799999999999 2024-09-17T21:19:35.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:35.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.772420+0000", 2024-09-17T21:19:35.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018777400000000001 2024-09-17T21:19:35.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:35.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.772894+0000", 2024-09-17T21:19:35.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000474556 2024-09-17T21:19:35.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:35.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.786644+0000", 2024-09-17T21:19:35.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013749924 2024-09-17T21:19:35.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:35.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.793382+0000", 2024-09-17T21:19:35.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067378940000000003 2024-09-17T21:19:35.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:35.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.793448+0000", 2024-09-17T21:19:35.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5524000000000005e-05 2024-09-17T21:19:35.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:35.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:35.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:35.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:26.793537+0000", 2024-09-17T21:19:35.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9529000000000003e-05 2024-09-17T21:19:35.732 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.732 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.732 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.732 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:35.732 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:35.732 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:35.734 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:19:35.735 INFO:teuthology.orchestra.run.smithi184.stdout:3599 write 0xe9c8800 thru 0xe9d264a (0x9e4b bytes) 2024-09-17T21:19:35.735 INFO:teuthology.orchestra.run.smithi184.stdout:3602 punch from 0x4ea7800 to 0x4eb7000, (0xf800 bytes) 2024-09-17T21:19:35.735 INFO:teuthology.orchestra.run.smithi184.stdout:3603 read 0xdc81000 thru 0xdc8bc67 (0xac68 bytes) 2024-09-17T21:19:35.735 INFO:teuthology.orchestra.run.smithi184.stdout:3605 punch from 0x3420600 to 0x3428200, (0x7c00 bytes) 2024-09-17T21:19:35.735 INFO:teuthology.orchestra.run.smithi184.stdout:3608 trunc from 0xe9d2800 to 0x34a5e00 2024-09-17T21:19:35.735 INFO:teuthology.orchestra.run.smithi184.stdout:3609 read 0x267c400 thru 0x268064c (0x424d bytes) 2024-09-17T21:19:35.735 INFO:teuthology.orchestra.run.smithi184.stdout:3610 read 0x193c200 thru 0x194a3c1 (0xe1c2 bytes) 2024-09-17T21:19:35.735 INFO:teuthology.orchestra.run.smithi184.stdout:3611 trunc from 0x34a5e00 to 0x5913e00 2024-09-17T21:19:35.736 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:35.730+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:35.768 INFO:teuthology.orchestra.run.smithi184.stdout:3612 write 0xdb1ca00 thru 0xdb2a2cb (0xd8cc bytes) 2024-09-17T21:19:35.771 INFO:teuthology.orchestra.run.smithi184.stdout:3613 read 0xd541600 thru 0xd54ce73 (0xb874 bytes) 2024-09-17T21:19:35.771 INFO:teuthology.orchestra.run.smithi184.stdout:3614 write 0x9a9b000 thru 0x9aa10bd (0x60be bytes) 2024-09-17T21:19:35.772 INFO:teuthology.orchestra.run.smithi184.stdout:3616 read 0xadfc200 thru 0xae0bdbb (0xfbbc bytes) 2024-09-17T21:19:35.772 INFO:teuthology.orchestra.run.smithi184.stdout:3618 punch from 0x88c2c00 to 0x88d1a00, (0xee00 bytes) 2024-09-17T21:19:35.773 INFO:teuthology.orchestra.run.smithi184.stdout:3622 read 0x14a4400 thru 0x14b3cf7 (0xf8f8 bytes) 2024-09-17T21:19:35.775 INFO:teuthology.orchestra.run.smithi184.stdout:3623 read 0x3289000 thru 0x328c859 (0x385a bytes) 2024-09-17T21:19:35.775 INFO:teuthology.orchestra.run.smithi184.stdout:3624 read 0xd547400 thru 0xd548f7c (0x1b7d bytes) 2024-09-17T21:19:35.791 INFO:teuthology.orchestra.run.smithi184.stdout:3625 trunc from 0xdb2a400 to 0x3bc7200 2024-09-17T21:19:35.815 INFO:teuthology.orchestra.run.smithi184.stdout:3628 punch from 0x9a0800 to 0x9ab400, (0xac00 bytes) 2024-09-17T21:19:35.819 INFO:teuthology.orchestra.run.smithi184.stdout:3629 trunc from 0x3bc7200 to 0x29e9c00 2024-09-17T21:19:35.834 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:35.828+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:35.845 INFO:teuthology.orchestra.run.smithi184.stdout:3631 punch from 0x1879e00 to 0x1887600, (0xd800 bytes) 2024-09-17T21:19:35.867 INFO:teuthology.orchestra.run.smithi184.stdout:3632 write 0x4b84c00 thru 0x4b8e103 (0x9504 bytes) 2024-09-17T21:19:35.868 INFO:teuthology.orchestra.run.smithi184.stdout:3633 punch from 0x3098e00 to 0x30a1400, (0x8600 bytes) 2024-09-17T21:19:35.869 INFO:teuthology.orchestra.run.smithi184.stdout:3636 read 0x2a3ec00 thru 0x2a42d8c (0x418d bytes) 2024-09-17T21:19:35.890 INFO:teuthology.orchestra.run.smithi184.stdout:3637 write 0x6a5d200 thru 0x6a65f06 (0x8d07 bytes) 2024-09-17T21:19:35.891 INFO:teuthology.orchestra.run.smithi184.stdout:3638 punch from 0x4f74400 to 0x4f7c800, (0x8400 bytes) 2024-09-17T21:19:35.892 INFO:teuthology.orchestra.run.smithi184.stdout:3639 read 0x6868a00 thru 0x68724f8 (0x9af9 bytes) 2024-09-17T21:19:35.892 INFO:teuthology.orchestra.run.smithi184.stdout:3640 read 0x4521200 thru 0x452438c (0x318d bytes) 2024-09-17T21:19:35.915 INFO:teuthology.orchestra.run.smithi184.stdout:3642 write 0xa190000 thru 0xa193656 (0x3657 bytes) 2024-09-17T21:19:35.916 INFO:teuthology.orchestra.run.smithi184.stdout:3645 read 0x546f800 thru 0x547ba1b (0xc21c bytes) 2024-09-17T21:19:35.924 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:35.924 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:35.924 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:19:35.924 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:35.934 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:35.932+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:35.938 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:19:35.945 INFO:teuthology.orchestra.run.smithi184.stdout:3648 trunc from 0xa193800 to 0x118f800 2024-09-17T21:19:36.002 INFO:teuthology.orchestra.run.smithi184.stdout:3653 write 0x3b95a00 thru 0x3b9bbcb (0x61cc bytes) 2024-09-17T21:19:36.004 INFO:teuthology.orchestra.run.smithi184.stdout:3654 read 0x383400 thru 0x389c53 (0x6854 bytes) 2024-09-17T21:19:36.005 INFO:teuthology.orchestra.run.smithi184.stdout:3658 read 0x1f3ae00 thru 0x1f3b42b (0x62c bytes) 2024-09-17T21:19:36.006 INFO:teuthology.orchestra.run.smithi184.stdout:3660 read 0x9cd200 thru 0x9db45d (0xe25e bytes) 2024-09-17T21:19:36.035 INFO:teuthology.orchestra.run.smithi184.stdout:3661 write 0x91cd600 thru 0x91d17c5 (0x41c6 bytes) 2024-09-17T21:19:36.035 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:36.030+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:36.066 INFO:teuthology.orchestra.run.smithi184.stdout:3666 write 0xe62ea00 thru 0xe63a788 (0xbd89 bytes) 2024-09-17T21:19:36.068 INFO:teuthology.orchestra.run.smithi184.stdout:3667 punch from 0x2f2bc00 to 0x2f3b400, (0xf800 bytes) 2024-09-17T21:19:36.069 INFO:teuthology.orchestra.run.smithi184.stdout:3668 read 0x3213800 thru 0x321ba7d (0x827e bytes) 2024-09-17T21:19:36.069 INFO:teuthology.orchestra.run.smithi184.stdout:3670 punch from 0xc8e5600 to 0xc8ef000, (0x9a00 bytes) 2024-09-17T21:19:36.092 INFO:teuthology.orchestra.run.smithi184.stdout:3671 trunc from 0xe63a800 to 0x195e200 2024-09-17T21:19:36.119 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:36.120 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:36.120 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:36.120 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:36.120 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:36.122 INFO:teuthology.orchestra.run.smithi184.stdout:3673 trunc from 0x195e200 to 0x309d600 2024-09-17T21:19:36.136 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:19:36.136 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:36.132+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:36.154 INFO:teuthology.orchestra.run.smithi184.stdout:3674 trunc from 0x309d600 to 0xdebe600 2024-09-17T21:19:36.165 INFO:teuthology.orchestra.run.smithi184.stdout:3675 punch from 0x924200 to 0x92bc00, (0x7a00 bytes) 2024-09-17T21:19:36.167 INFO:teuthology.orchestra.run.smithi184.stdout:3676 write 0x31a2a00 thru 0x31acb32 (0xa133 bytes) 2024-09-17T21:19:36.167 INFO:teuthology.orchestra.run.smithi184.stdout:3680 read 0x2452c00 thru 0x245616a (0x356b bytes) 2024-09-17T21:19:36.167 INFO:teuthology.orchestra.run.smithi184.stdout:3681 write 0xb2e8000 thru 0xb2e88be (0x8bf bytes) 2024-09-17T21:19:36.167 INFO:teuthology.orchestra.run.smithi184.stdout:3682 clone 21 order 18 su 65536 sc 10 2024-09-17T21:19:36.237 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:36.232+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:36.306 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:36.306 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:36.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:36.306 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:36.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.306 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:19:36.306 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:36.307 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.698157960000003, 2024-09-17T21:19:36.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:19:36.307 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:36.307 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:36.307 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:36.307 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:36.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:36.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:36.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:36.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:36.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:36.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:19:36.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:19:36.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:36.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:19:36.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:19:36.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:36.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:19:36.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:19:36.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:19:36.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:19:36.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:36.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:19:36.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:19:36.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:36.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:19:36.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:19:36.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:36.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:19:36.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:19:36.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:36.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:19:36.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:19:36.311 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.311 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:36.311 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.311 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:19:36.311 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:36.311 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.688211323999994, 2024-09-17T21:19:36.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:19:36.311 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:36.311 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:36.311 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:36.311 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:36.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:36.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:36.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:36.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:36.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:36.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:19:36.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:19:36.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:36.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:19:36.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:19:36.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:36.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:19:36.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:19:36.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:19:36.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:19:36.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:36.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:19:36.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:19:36.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:36.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:19:36.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:19:36.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:36.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:19:36.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:19:36.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:36.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:19:36.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:19:36.315 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.315 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:36.316 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.316 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:19:36.316 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:36.316 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 73.445060483999995, 2024-09-17T21:19:36.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:19:36.316 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:36.316 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:36.316 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:36.316 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:36.316 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:19:36.316 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:19:36.316 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:19:36.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.317 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:36.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:36.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:36.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:36.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:36.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:36.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:19:36.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:19:36.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:36.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:19:36.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:19:36.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:36.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:19:36.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:19:36.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:19:36.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:19:36.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:36.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:19:36.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:19:36.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:36.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:19:36.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:19:36.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:36.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:19:36.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:19:36.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:36.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:19:36.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:19:36.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:36.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:19:36.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:19:36.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:36.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:19:36.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:19:36.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:36.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:19:36.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:19:36.321 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.322 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:36.322 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.322 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:19:36.322 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:36.322 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 73.435414997999999, 2024-09-17T21:19:36.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:19:36.322 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:36.322 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:36.322 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:36.322 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:36.322 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:19:36.322 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:19:36.323 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:19:36.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.323 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:36.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:36.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:36.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:36.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:36.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:36.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:19:36.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:19:36.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:36.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:19:36.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:19:36.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:36.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:19:36.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:19:36.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:19:36.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:19:36.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:36.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:19:36.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:19:36.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:36.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:19:36.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:19:36.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:36.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:19:36.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:19:36.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:36.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:19:36.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:19:36.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:36.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:19:36.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:19:36.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:36.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:19:36.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:19:36.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:36.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:19:36.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:19:36.327 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.327 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:36.327 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.328 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:36.328 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:36.328 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 60.066478965000002, 2024-09-17T21:19:36.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:19:36.328 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:36.328 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:36.328 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:36.328 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:36.328 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:36.328 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:36.328 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:19:36.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.328 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:36.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:36.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:36.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:36.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:36.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:36.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:19:36.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:19:36.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:36.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:19:36.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:19:36.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:36.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:19:36.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:19:36.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:19:36.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:19:36.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:36.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:19:36.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:19:36.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:36.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:19:36.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:19:36.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:36.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:19:36.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:19:36.332 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.332 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:36.332 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.332 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:565 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:36.332 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:19:36.332 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 60.062690000000003, 2024-09-17T21:19:36.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024193392000000001, 2024-09-17T21:19:36.332 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:36.332 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:36.332 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:36.333 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:36.333 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:36.333 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:36.333 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 565 2024-09-17T21:19:36.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.333 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:36.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:36.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:19:36.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:36.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:19:36.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:36.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239660+0000", 2024-09-17T21:19:36.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:19:36.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:36.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239666+0000", 2024-09-17T21:19:36.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8699999999999997e-06 2024-09-17T21:19:36.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:36.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239688+0000", 2024-09-17T21:19:36.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.213e-05 2024-09-17T21:19:36.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239703+0000", 2024-09-17T21:19:36.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4813e-05 2024-09-17T21:19:36.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:36.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263427+0000", 2024-09-17T21:19:36.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023724630999999999 2024-09-17T21:19:36.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:36.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263614+0000", 2024-09-17T21:19:36.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018664500000000001 2024-09-17T21:19:36.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:36.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263852+0000", 2024-09-17T21:19:36.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023793999999999999 2024-09-17T21:19:36.339 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.339 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:36.339 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.339 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:819 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5111808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:36.339 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:36.339 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.864906675, 2024-09-17T21:19:36.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024344811000000001, 2024-09-17T21:19:36.340 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:36.340 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:36.340 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:36.340 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:36.340 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:36.340 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:36.340 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 819 2024-09-17T21:19:36.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.340 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:36.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:36.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:36.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:36.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:36.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:36.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437443+0000", 2024-09-17T21:19:36.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1969999999999999e-06 2024-09-17T21:19:36.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:36.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437448+0000", 2024-09-17T21:19:36.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9490000000000004e-06 2024-09-17T21:19:36.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:36.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437467+0000", 2024-09-17T21:19:36.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8862999999999999e-05 2024-09-17T21:19:36.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437480+0000", 2024-09-17T21:19:36.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2833000000000001e-05 2024-09-17T21:19:36.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:36.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461552+0000", 2024-09-17T21:19:36.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024072695000000002 2024-09-17T21:19:36.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:36.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461639+0000", 2024-09-17T21:19:36.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6996000000000005e-05 2024-09-17T21:19:36.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:36.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461787+0000", 2024-09-17T21:19:36.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014727800000000001 2024-09-17T21:19:36.343 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.344 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:36.344 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.344 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:36.344 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:36.344 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.864794402999998, 2024-09-17T21:19:36.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:19:36.344 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:36.344 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:36.344 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:36.344 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:36.344 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:36.344 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:36.345 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:19:36.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.345 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:36.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:36.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:36.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:36.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:36.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:36.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:19:36.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:19:36.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:36.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:19:36.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:19:36.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:36.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:19:36.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:19:36.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:19:36.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:19:36.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:36.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:19:36.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:19:36.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:36.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:19:36.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:19:36.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:36.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:19:36.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:19:36.348 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.348 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:36.348 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.348 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:833 3.5 3:a772dfb1:::rbd_data.11523651b015.0000000000000001:d [sparse-read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:36.348 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:36.348 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.864360116999997, 2024-09-17T21:19:36.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024216024999999999, 2024-09-17T21:19:36.348 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:36.348 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:36.348 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:36.349 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:36.349 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:36.349 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:36.349 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 833 2024-09-17T21:19:36.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.349 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:36.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:36.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:36.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:36.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:36.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:36.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437990+0000", 2024-09-17T21:19:36.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2500000000000001e-06 2024-09-17T21:19:36.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:36.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437995+0000", 2024-09-17T21:19:36.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1240000000000004e-06 2024-09-17T21:19:36.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:36.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438012+0000", 2024-09-17T21:19:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7572e-05 2024-09-17T21:19:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438031+0000", 2024-09-17T21:19:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8612000000000001e-05 2024-09-17T21:19:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461830+0000", 2024-09-17T21:19:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023799135999999999 2024-09-17T21:19:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:36.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462005+0000", 2024-09-17T21:19:36.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000174747 2024-09-17T21:19:36.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:36.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:19:36.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019958399999999999 2024-09-17T21:19:36.352 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.352 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:36.352 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.352 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:36.352 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:36.353 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.864299826, 2024-09-17T21:19:36.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:19:36.353 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:36.353 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:36.353 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:36.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:36.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:36.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:36.353 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:19:36.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.353 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:36.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:36.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:36.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:36.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:36.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:36.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:19:36.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:19:36.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:36.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:19:36.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:19:36.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:36.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:19:36.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:19:36.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:19:36.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:19:36.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:36.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:19:36.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:19:36.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:36.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:19:36.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:19:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:19:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:19:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.864244444999997, 2024-09-17T21:19:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:19:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:19:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:19:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:19:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:19:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:19:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:19:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:19:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:19:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:19:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:19:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:19:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:19:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:19:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:19:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:19:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.864188167000002, 2024-09-17T21:19:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:19:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:19:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:19:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:19:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:19:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:19:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:19:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:19:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:19:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:19:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:19:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:19:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:19:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:19:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:19:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:19:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.863238527999997, 2024-09-17T21:19:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:19:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:19:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:19:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:19:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:19:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:19:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:19:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:19:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:19:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:19:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:19:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:19:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:19:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:19:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:19:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:19:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.863159908, 2024-09-17T21:19:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:19:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:19:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:19:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:19:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:19:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:19:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:19:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:19:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:19:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:19:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:19:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:19:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:19:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:19:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:19:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:19:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:847 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 4653056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439245+0000", 2024-09-17T21:19:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.863103469999999, 2024-09-17T21:19:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024172945000000001, 2024-09-17T21:19:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 847 2024-09-17T21:19:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439245+0000", 2024-09-17T21:19:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439245+0000", 2024-09-17T21:19:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439246+0000", 2024-09-17T21:19:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:19:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439252+0000", 2024-09-17T21:19:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3569999999999997e-06 2024-09-17T21:19:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439269+0000", 2024-09-17T21:19:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7316000000000002e-05 2024-09-17T21:19:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439282+0000", 2024-09-17T21:19:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3269e-05 2024-09-17T21:19:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463240+0000", 2024-09-17T21:19:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02395721 2024-09-17T21:19:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463306+0000", 2024-09-17T21:19:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6618000000000005e-05 2024-09-17T21:19:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463418+0000", 2024-09-17T21:19:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111672 2024-09-17T21:19:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.863046969000003, 2024-09-17T21:19:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:19:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:19:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:19:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:19:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:19:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:19:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:19:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:19:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:19:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:19:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:19:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:19:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:19:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:19:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:19:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:19:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:849 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5963776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.862452933, 2024-09-17T21:19:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024537514, 2024-09-17T21:19:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 849 2024-09-17T21:19:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439897+0000", 2024-09-17T21:19:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9099999999999999e-06 2024-09-17T21:19:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439905+0000", 2024-09-17T21:19:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0870000000000007e-06 2024-09-17T21:19:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439924+0000", 2024-09-17T21:19:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9042000000000002e-05 2024-09-17T21:19:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439941+0000", 2024-09-17T21:19:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.624e-05 2024-09-17T21:19:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464132+0000", 2024-09-17T21:19:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024191579000000001 2024-09-17T21:19:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464291+0000", 2024-09-17T21:19:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015904800000000001 2024-09-17T21:19:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464433+0000", 2024-09-17T21:19:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014160799999999999 2024-09-17T21:19:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.862385914999997, 2024-09-17T21:19:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:19:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:19:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:19:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:19:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:19:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:19:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:19:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:19:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:19:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:19:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:19:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:19:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:19:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:19:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:19:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:19:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.862321104000003, 2024-09-17T21:19:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:19:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:19:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:19:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:19:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:19:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:19:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:19:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:19:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:19:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:19:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:19:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:19:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:19:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:19:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:19:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:19:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.862260126999999, 2024-09-17T21:19:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:19:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:19:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:19:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:19:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:19:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:19:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:19:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:19:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:19:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:19:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:19:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:19:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:19:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:19:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:19:36.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:19:36.401 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.401 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:36.401 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.401 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:36.401 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:36.401 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:36.402 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:19:36.403 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:36.398+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:36.504 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:36.500+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:36.533 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:36.534 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:36.534 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:19:36.534 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:36.548 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:19:36.604 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:36.598+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:36.625 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:19:36.670 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:36.670 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:36.670 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:36.670 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:36.670 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:36.685 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:19:36.728 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:36.726+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:36.744 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:19:36.792 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone20 from 0xdebe600 (overlap 0x118f800) to 0x0 2024-09-17T21:19:36.807 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:36.804+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:36.865 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:36.865 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:36.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:36.865 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:36.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.865 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:406 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 2359296~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:36.865 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:19:36.865 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.666469495000001, 2024-09-17T21:19:36.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042448018999999997, 2024-09-17T21:19:36.865 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:36.865 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:36.865 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:36.865 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:36.866 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:36.866 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:36.866 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 406 2024-09-17T21:19:36.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.866 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:36.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.866 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:36.866 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:19:36.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.866 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:36.866 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:19:36.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.867 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:36.867 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195056+0000", 2024-09-17T21:19:36.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2729999999999999e-06 2024-09-17T21:19:36.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.867 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:36.867 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195060+0000", 2024-09-17T21:19:36.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5399999999999997e-06 2024-09-17T21:19:36.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:36.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195076+0000", 2024-09-17T21:19:36.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6217000000000001e-05 2024-09-17T21:19:36.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195093+0000", 2024-09-17T21:19:36.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.683e-05 2024-09-17T21:19:36.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:36.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236217+0000", 2024-09-17T21:19:36.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041124102000000003 2024-09-17T21:19:36.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:36.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236339+0000", 2024-09-17T21:19:36.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121876 2024-09-17T21:19:36.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:36.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237502+0000", 2024-09-17T21:19:36.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011631809999999999 2024-09-17T21:19:36.869 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.869 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:36.869 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.870 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:416 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2228224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:36.870 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:19:36.870 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.665971468999999, 2024-09-17T21:19:36.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042361990000000002, 2024-09-17T21:19:36.870 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:36.870 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:36.870 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:36.870 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:36.870 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:36.870 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:36.870 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 416 2024-09-17T21:19:36.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.870 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:36.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.871 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:36.871 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:19:36.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.871 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:36.871 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:19:36.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.871 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:36.871 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195554+0000", 2024-09-17T21:19:36.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9300000000000002e-06 2024-09-17T21:19:36.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:36.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195563+0000", 2024-09-17T21:19:36.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2770000000000005e-06 2024-09-17T21:19:36.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:36.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195586+0000", 2024-09-17T21:19:36.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3889000000000001e-05 2024-09-17T21:19:36.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195603+0000", 2024-09-17T21:19:36.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6084000000000001e-05 2024-09-17T21:19:36.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.873 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:36.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237563+0000", 2024-09-17T21:19:36.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041960856999999997 2024-09-17T21:19:36.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.873 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:36.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237695+0000", 2024-09-17T21:19:36.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013144400000000001 2024-09-17T21:19:36.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.874 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:36.874 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237914+0000", 2024-09-17T21:19:36.874 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000219509 2024-09-17T21:19:36.874 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.874 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:36.874 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.874 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.874 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:439 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:36.874 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:19:36.874 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.665407567000003, 2024-09-17T21:19:36.874 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043053811999999997, 2024-09-17T21:19:36.875 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:36.875 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:36.875 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:36.875 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:36.875 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:36.875 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:36.875 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 439 2024-09-17T21:19:36.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.875 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:36.875 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.875 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:36.875 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:19:36.875 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.875 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.876 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:36.876 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:19:36.876 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.876 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:36.876 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196118+0000", 2024-09-17T21:19:36.876 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3969999999999999e-06 2024-09-17T21:19:36.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.876 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:36.876 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196157+0000", 2024-09-17T21:19:36.876 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9826999999999997e-05 2024-09-17T21:19:36.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:36.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196179+0000", 2024-09-17T21:19:36.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1756e-05 2024-09-17T21:19:36.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196194+0000", 2024-09-17T21:19:36.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4707e-05 2024-09-17T21:19:36.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:36.878 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237980+0000", 2024-09-17T21:19:36.878 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041786219999999999 2024-09-17T21:19:36.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.878 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:36.878 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.238065+0000", 2024-09-17T21:19:36.878 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4560000000000004e-05 2024-09-17T21:19:36.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.878 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:36.878 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239170+0000", 2024-09-17T21:19:36.878 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001105345 2024-09-17T21:19:36.878 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.878 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:36.879 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.879 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:440 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:36.879 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:19:36.879 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.665305063999995, 2024-09-17T21:19:36.879 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043890039999999998, 2024-09-17T21:19:36.879 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:36.879 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:36.879 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:36.879 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:36.879 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:36.879 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:36.879 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 440 2024-09-17T21:19:36.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.880 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:36.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.880 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:36.880 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:19:36.880 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.880 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:36.880 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:19:36.880 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:36.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196220+0000", 2024-09-17T21:19:36.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.254e-06 2024-09-17T21:19:36.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:36.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196225+0000", 2024-09-17T21:19:36.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7829999999999998e-06 2024-09-17T21:19:36.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:36.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196241+0000", 2024-09-17T21:19:36.882 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6322999999999999e-05 2024-09-17T21:19:36.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.882 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.882 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196258+0000", 2024-09-17T21:19:36.882 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7056000000000001e-05 2024-09-17T21:19:36.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.882 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:36.882 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239229+0000", 2024-09-17T21:19:36.882 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042971058999999999 2024-09-17T21:19:36.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.882 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:36.882 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239403+0000", 2024-09-17T21:19:36.883 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017399200000000001 2024-09-17T21:19:36.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.883 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:36.883 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240109+0000", 2024-09-17T21:19:36.883 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00070557300000000003 2024-09-17T21:19:36.883 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.883 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:36.883 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.883 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:450 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:36.883 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:19:36.883 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.664944113000004, 2024-09-17T21:19:36.883 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043994521000000002, 2024-09-17T21:19:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 450 2024-09-17T21:19:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:19:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.885 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:36.885 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:19:36.885 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.885 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:36.885 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196581+0000", 2024-09-17T21:19:36.885 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4500000000000001e-06 2024-09-17T21:19:36.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.885 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:36.885 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196588+0000", 2024-09-17T21:19:36.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7530000000000004e-06 2024-09-17T21:19:36.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.886 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:36.886 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196607+0000", 2024-09-17T21:19:36.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8987e-05 2024-09-17T21:19:36.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.886 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.886 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196620+0000", 2024-09-17T21:19:36.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3589e-05 2024-09-17T21:19:36.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.887 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:36.887 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240225+0000", 2024-09-17T21:19:36.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043604878999999999 2024-09-17T21:19:36.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.887 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:36.887 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240377+0000", 2024-09-17T21:19:36.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015146699999999999 2024-09-17T21:19:36.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.887 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:36.887 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240574+0000", 2024-09-17T21:19:36.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019739599999999999 2024-09-17T21:19:36.887 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.887 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:36.888 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.888 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:464 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:36.888 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:19:36.888 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.664693692, 2024-09-17T21:19:36.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044651692999999999, 2024-09-17T21:19:36.888 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:36.888 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:36.888 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:36.888 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:36.888 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:36.889 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:36.889 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 464 2024-09-17T21:19:36.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.889 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:36.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:36.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:19:36.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:36.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:19:36.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:36.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196838+0000", 2024-09-17T21:19:36.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4970000000000004e-06 2024-09-17T21:19:36.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:36.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196859+0000", 2024-09-17T21:19:36.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1166e-05 2024-09-17T21:19:36.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:36.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196879+0000", 2024-09-17T21:19:36.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.995e-05 2024-09-17T21:19:36.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196896+0000", 2024-09-17T21:19:36.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6929999999999999e-05 2024-09-17T21:19:36.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:36.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240617+0000", 2024-09-17T21:19:36.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043721784999999999 2024-09-17T21:19:36.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:36.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240702+0000", 2024-09-17T21:19:36.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4060999999999994e-05 2024-09-17T21:19:36.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:36.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241482+0000", 2024-09-17T21:19:36.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000780304 2024-09-17T21:19:36.892 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.892 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:36.892 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.893 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:475 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:36.893 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:19:36.893 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.66442456, 2024-09-17T21:19:36.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044661981000000003, 2024-09-17T21:19:36.893 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:36.893 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:36.893 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:36.893 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:36.893 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:36.893 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:36.893 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 475 2024-09-17T21:19:36.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.894 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:36.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:36.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:19:36.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:36.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:19:36.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:36.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197100+0000", 2024-09-17T21:19:36.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.206e-06 2024-09-17T21:19:36.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:36.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197105+0000", 2024-09-17T21:19:36.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.527e-06 2024-09-17T21:19:36.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:36.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197124+0000", 2024-09-17T21:19:36.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:19:36.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197136+0000", 2024-09-17T21:19:36.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2857000000000001e-05 2024-09-17T21:19:36.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:36.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241539+0000", 2024-09-17T21:19:36.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044402077999999998 2024-09-17T21:19:36.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:36.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241653+0000", 2024-09-17T21:19:36.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114931 2024-09-17T21:19:36.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:36.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241761+0000", 2024-09-17T21:19:36.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107753 2024-09-17T21:19:36.897 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.897 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:36.897 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.897 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:476 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:36.897 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:19:36.897 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.664370735000006, 2024-09-17T21:19:36.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045038976000000001, 2024-09-17T21:19:36.898 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:36.898 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:36.898 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:36.898 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:36.898 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:36.898 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:36.898 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 476 2024-09-17T21:19:36.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.898 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:36.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:36.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:19:36.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:36.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:19:36.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:36.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197154+0000", 2024-09-17T21:19:36.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.127e-06 2024-09-17T21:19:36.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:36.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197159+0000", 2024-09-17T21:19:36.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4569999999999998e-06 2024-09-17T21:19:36.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:36.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197174+0000", 2024-09-17T21:19:36.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5173e-05 2024-09-17T21:19:36.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197186+0000", 2024-09-17T21:19:36.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:19:36.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:36.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241863+0000", 2024-09-17T21:19:36.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044676826000000003 2024-09-17T21:19:36.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:36.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241998+0000", 2024-09-17T21:19:36.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013476100000000001 2024-09-17T21:19:36.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:36.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242192+0000", 2024-09-17T21:19:36.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019436900000000001 2024-09-17T21:19:36.902 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.902 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:36.902 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.902 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:494 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:36.902 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:36.902 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.664195551999995, 2024-09-17T21:19:36.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046244995999999997, 2024-09-17T21:19:36.902 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:36.902 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:36.902 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:36.902 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:36.903 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:36.903 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:36.903 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 494 2024-09-17T21:19:36.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.903 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:36.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:36.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:36.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:36.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:36.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:36.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197329+0000", 2024-09-17T21:19:36.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.173e-06 2024-09-17T21:19:36.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:36.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197334+0000", 2024-09-17T21:19:36.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0499999999999999e-06 2024-09-17T21:19:36.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:36.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197352+0000", 2024-09-17T21:19:36.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7133e-05 2024-09-17T21:19:36.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197367+0000", 2024-09-17T21:19:36.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5123000000000001e-05 2024-09-17T21:19:36.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:36.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:19:36.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044989829000000002 2024-09-17T21:19:36.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:36.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242526+0000", 2024-09-17T21:19:36.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016968199999999999 2024-09-17T21:19:36.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:36.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243573+0000", 2024-09-17T21:19:36.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010470060000000001 2024-09-17T21:19:36.906 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.906 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:36.906 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.907 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:502 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:36.907 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:19:36.907 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.661961892999997, 2024-09-17T21:19:36.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044346086999999999, 2024-09-17T21:19:36.907 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:36.907 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:36.907 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:36.907 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:36.907 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:36.907 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:36.907 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 502 2024-09-17T21:19:36.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.907 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:36.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:36.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:19:36.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:36.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:19:36.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:36.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199563+0000", 2024-09-17T21:19:36.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2929999999999999e-06 2024-09-17T21:19:36.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:36.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199569+0000", 2024-09-17T21:19:36.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5330000000000002e-06 2024-09-17T21:19:36.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:36.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199614+0000", 2024-09-17T21:19:36.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4793000000000003e-05 2024-09-17T21:19:36.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199628+0000", 2024-09-17T21:19:36.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4037e-05 2024-09-17T21:19:36.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:36.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243665+0000", 2024-09-17T21:19:36.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044037106999999999 2024-09-17T21:19:36.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:36.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243767+0000", 2024-09-17T21:19:36.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010279499999999999 2024-09-17T21:19:36.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:36.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243908+0000", 2024-09-17T21:19:36.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140529 2024-09-17T21:19:36.911 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.911 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:36.911 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.911 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:523 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:36.911 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:19:36.911 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.661515739999999, 2024-09-17T21:19:36.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044789645000000003, 2024-09-17T21:19:36.912 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:36.912 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:36.912 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:36.912 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:36.912 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:36.912 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:36.912 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 523 2024-09-17T21:19:36.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.912 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:36.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:36.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:19:36.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:36.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:19:36.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:36.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:19:36.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9e-06 2024-09-17T21:19:36.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:36.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200021+0000", 2024-09-17T21:19:36.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9499999999999998e-06 2024-09-17T21:19:36.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:36.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200038+0000", 2024-09-17T21:19:36.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.73e-05 2024-09-17T21:19:36.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200051+0000", 2024-09-17T21:19:36.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.322e-05 2024-09-17T21:19:36.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:36.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243987+0000", 2024-09-17T21:19:36.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043935386999999999 2024-09-17T21:19:36.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:36.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244136+0000", 2024-09-17T21:19:36.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014868300000000001 2024-09-17T21:19:36.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:36.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244798+0000", 2024-09-17T21:19:36.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00066220500000000002 2024-09-17T21:19:36.916 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.916 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:36.916 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.916 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:524 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:36.916 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:36.916 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.661449524000005, 2024-09-17T21:19:36.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045737414999999997, 2024-09-17T21:19:36.916 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:36.916 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:36.917 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:36.917 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:36.917 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:36.917 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:36.917 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 524 2024-09-17T21:19:36.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.917 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:36.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:36.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:36.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:36.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:36.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:36.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200076+0000", 2024-09-17T21:19:36.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:19:36.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:36.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200081+0000", 2024-09-17T21:19:36.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7030000000000003e-06 2024-09-17T21:19:36.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:36.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200099+0000", 2024-09-17T21:19:36.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7836999999999999e-05 2024-09-17T21:19:36.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200113+0000", 2024-09-17T21:19:36.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3570000000000001e-05 2024-09-17T21:19:36.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:36.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244866+0000", 2024-09-17T21:19:36.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044753573999999997 2024-09-17T21:19:36.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:36.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244972+0000", 2024-09-17T21:19:36.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106111 2024-09-17T21:19:36.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:36.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245812+0000", 2024-09-17T21:19:36.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083923999999999999 2024-09-17T21:19:36.920 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.920 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:36.920 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.921 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:525 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3670016~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:36.921 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:36.921 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.661298142000007, 2024-09-17T21:19:36.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046645697, 2024-09-17T21:19:36.921 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:36.921 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:36.921 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:36.921 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:36.921 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:36.921 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:36.921 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 525 2024-09-17T21:19:36.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.922 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:36.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:36.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:36.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:36.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:36.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:36.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:19:36.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:19:36.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:36.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200234+0000", 2024-09-17T21:19:36.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:19:36.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:36.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200254+0000", 2024-09-17T21:19:36.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9959999999999999e-05 2024-09-17T21:19:36.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200293+0000", 2024-09-17T21:19:36.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9755999999999997e-05 2024-09-17T21:19:36.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:36.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245871+0000", 2024-09-17T21:19:36.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045577871999999998 2024-09-17T21:19:36.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:36.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246030+0000", 2024-09-17T21:19:36.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158975 2024-09-17T21:19:36.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:36.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246871+0000", 2024-09-17T21:19:36.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084111699999999995 2024-09-17T21:19:36.925 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.925 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:36.925 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.926 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:36.926 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:36.926 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.661052467000005, 2024-09-17T21:19:36.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:19:36.926 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:36.926 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:36.926 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:36.926 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:36.926 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:36.926 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:36.926 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:19:36.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.926 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:36.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:36.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:36.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:36.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:36.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:36.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:19:36.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:19:36.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:36.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:19:36.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:19:36.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:36.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:19:36.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:19:36.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:19:36.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:19:36.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:36.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:19:36.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:19:36.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:36.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:19:36.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:19:36.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:36.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:19:36.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:19:36.930 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.930 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:36.930 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.930 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:36.930 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:36.930 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.660873785000007, 2024-09-17T21:19:36.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:19:36.931 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:36.931 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:36.931 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:36.931 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:36.931 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:36.931 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:36.931 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:19:36.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.931 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:36.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:36.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:36.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:36.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:36.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:36.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:19:36.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:19:36.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:36.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:19:36.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:19:36.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:36.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:19:36.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:19:36.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:19:36.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:19:36.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:36.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:19:36.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:19:36.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:36.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:19:36.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:19:36.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:36.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:19:36.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:19:36.935 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.935 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:36.935 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.935 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:36.935 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:36.935 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.660814512000002, 2024-09-17T21:19:36.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:19:36.935 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:36.935 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:36.936 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:36.936 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:36.936 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:36.936 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:36.936 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:19:36.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.936 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:36.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:36.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:36.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:36.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:36.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:36.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:19:36.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:19:36.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:36.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:19:36.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:19:36.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:36.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:19:36.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:19:36.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:19:36.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:19:36.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:36.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:19:36.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:19:36.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:36.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:19:36.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:19:36.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:36.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:19:36.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:19:36.939 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.940 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:36.940 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.940 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:36.940 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:36.940 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.660672317000007, 2024-09-17T21:19:36.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:19:36.940 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:36.940 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:36.940 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:36.940 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:36.940 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:36.941 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:36.941 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:19:36.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.941 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:36.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:36.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:36.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:36.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:36.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:36.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:19:36.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:19:36.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:36.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:19:36.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:19:36.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:36.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:19:36.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:19:36.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:19:36.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:19:36.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:36.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:19:36.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:19:36.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:36.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:19:36.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:19:36.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:36.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:19:36.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:19:36.945 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.945 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:36.945 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.948 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:36.948 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:36.948 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.660616394000002, 2024-09-17T21:19:36.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:19:36.949 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:36.949 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:36.949 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:36.949 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:36.949 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:36.949 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:36.949 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:19:36.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.949 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:36.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:36.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:36.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:36.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:36.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:36.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:19:36.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:19:36.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:36.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:19:36.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:19:36.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:36.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:19:36.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:19:36.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:19:36.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:19:36.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:36.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:19:36.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:19:36.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:36.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:19:36.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:19:36.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:36.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:19:36.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:19:36.953 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.953 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:36.953 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.953 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:36.954 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:36.954 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.660562776999996, 2024-09-17T21:19:36.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:19:36.954 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:36.954 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:36.954 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:36.954 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:36.954 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:36.954 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:36.954 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:19:36.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.954 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:36.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:36.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:36.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:36.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:36.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:36.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:19:36.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:19:36.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:36.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:19:36.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:19:36.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:36.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:19:36.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:19:36.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:19:36.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:19:36.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:36.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:19:36.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:19:36.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:36.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:19:36.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:19:36.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:36.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:19:36.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:19:36.958 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.958 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:36.958 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.959 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:19:36.959 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:19:36.959 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 20.351334534999999, 2024-09-17T21:19:36.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:19:36.959 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:36.959 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:36.959 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:19:36.959 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:36.959 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:19:36.959 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:19:36.959 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:19:36.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.960 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:36.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:36.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:19:36.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:36.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:19:36.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:36.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:36.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:19:36.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:19:36.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:36.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:19:36.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:19:36.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:36.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:19:36.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:19:36.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:36.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:19:36.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:19:36.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:36.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:19:36.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:19:36.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:36.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:19:36.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:19:36.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:36.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:19:36.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:19:36.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:36.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:36.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:36.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:19:36.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:19:36.964 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.964 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:36.964 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.964 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:36.964 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:36.964 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:36.966 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:19:36.966 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:36.967 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:36.967 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:36.967 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:36.968 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:36.968 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:36.968 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:36.968 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:36.969 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:36.962+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:36.986 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:37.008 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:37.008 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:37.068 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:37.066+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:37.106 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:37.106 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:37.121 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:19:37.208 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:37.206+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:37.269 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:37.264+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:37.292 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:37.292 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:37.307 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:19:37.327 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-17T21:19:37.370 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:37.364+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:37.430 INFO:teuthology.orchestra.run.smithi184.stdout:3683 write 0x72a6400 thru 0x72b537f (0xef80 bytes) 2024-09-17T21:19:37.431 INFO:teuthology.orchestra.run.smithi184.stdout:3686 write 0x150de00 thru 0x151a3c9 (0xc5ca bytes) 2024-09-17T21:19:37.432 INFO:teuthology.orchestra.run.smithi184.stdout:3687 read 0x5104600 thru 0x510d358 (0x8d59 bytes) 2024-09-17T21:19:37.433 INFO:teuthology.orchestra.run.smithi184.stdout:3688 write 0x88e8c00 thru 0x88ed088 (0x4489 bytes) 2024-09-17T21:19:37.434 INFO:teuthology.orchestra.run.smithi184.stdout:3690 read 0x4668c00 thru 0x4674497 (0xb898 bytes) 2024-09-17T21:19:37.435 INFO:teuthology.orchestra.run.smithi184.stdout:3691 read 0xd023800 thru 0xd032a2f (0xf230 bytes) 2024-09-17T21:19:37.436 INFO:teuthology.orchestra.run.smithi184.stdout:3694 write 0x5d98c00 thru 0x5da2253 (0x9654 bytes) 2024-09-17T21:19:37.438 INFO:teuthology.orchestra.run.smithi184.stdout:3696 read 0x5bb4e00 thru 0x5bc3bc1 (0xedc2 bytes) 2024-09-17T21:19:37.438 INFO:teuthology.orchestra.run.smithi184.stdout:3697 write 0xb434600 thru 0xb437341 (0x2d42 bytes) 2024-09-17T21:19:37.440 INFO:teuthology.orchestra.run.smithi184.stdout:3698 read 0x419de00 thru 0x41a6433 (0x8634 bytes) 2024-09-17T21:19:37.440 INFO:teuthology.orchestra.run.smithi184.stdout:3699 punch from 0xaed0200 to 0xaed9e00, (0x9c00 bytes) 2024-09-17T21:19:37.471 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:37.466+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:37.477 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:37.477 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:37.492 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:19:37.535 INFO:teuthology.orchestra.run.smithi184.stdout:3700 trunc from 0xdebe600 to 0x6021000 2024-09-17T21:19:37.572 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:37.568+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:37.618 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-17T21:19:37.641 INFO:teuthology.orchestra.run.smithi184.stdout:3701 write 0xb27ca00 thru 0xb27ce54 (0x455 bytes) 2024-09-17T21:19:37.642 INFO:teuthology.orchestra.run.smithi184.stdout:3702 write 0x25c7e00 thru 0x25ca67d (0x287e bytes) 2024-09-17T21:19:37.655 INFO:teuthology.orchestra.run.smithi184.stdout:3705 trunc from 0xb27d000 to 0x5d24c00 2024-09-17T21:19:37.669 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:37.669 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:37.673 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:37.668+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:37.686 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:19:37.728 INFO:teuthology.orchestra.run.smithi184.stdout:3707 read 0x3730000 thru 0x3738555 (0x8556 bytes) 2024-09-17T21:19:37.730 INFO:teuthology.orchestra.run.smithi184.stdout:3708 read 0x29cd600 thru 0x29da48a (0xce8b bytes) 2024-09-17T21:19:37.768 INFO:teuthology.orchestra.run.smithi184.stdout:3709 write 0xd950600 thru 0xd958165 (0x7b66 bytes) 2024-09-17T21:19:37.770 INFO:teuthology.orchestra.run.smithi184.stdout:3710 write 0x5edaa00 thru 0x5ee1393 (0x6994 bytes) 2024-09-17T21:19:37.771 INFO:teuthology.orchestra.run.smithi184.stdout:3711 read 0x24c800 thru 0x2547cb (0x7fcc bytes) 2024-09-17T21:19:37.773 INFO:teuthology.orchestra.run.smithi184.stdout:3713 read 0x299fc00 thru 0x29a39ee (0x3def bytes) 2024-09-17T21:19:37.773 INFO:teuthology.orchestra.run.smithi184.stdout:3714 write 0x50af400 thru 0x50b2284 (0x2e85 bytes) 2024-09-17T21:19:37.773 INFO:teuthology.orchestra.run.smithi184.stdout:3715 write 0x1f2c600 thru 0x1f3a506 (0xdf07 bytes) 2024-09-17T21:19:37.774 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:37.770+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:37.775 INFO:teuthology.orchestra.run.smithi184.stdout:3716 write 0x655b600 thru 0x656770c (0xc10d bytes) 2024-09-17T21:19:37.791 INFO:teuthology.orchestra.run.smithi184.stdout:3717 trunc from 0xd958200 to 0x6c19200 2024-09-17T21:19:37.858 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:37.858 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:37.873 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:19:37.874 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:37.872+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:37.884 INFO:teuthology.orchestra.run.smithi184.stdout:3719 trunc from 0x6c19200 to 0x36c0000 2024-09-17T21:19:37.937 INFO:teuthology.orchestra.run.smithi184.stdout:3720 read 0x30bd200 thru 0x30cb98f (0xe790 bytes) 2024-09-17T21:19:37.941 INFO:teuthology.orchestra.run.smithi184.stdout:3721 read 0x123ac00 thru 0x123c9bc (0x1dbd bytes) 2024-09-17T21:19:37.975 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:37.970+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:37.977 INFO:teuthology.orchestra.run.smithi184.stdout:3722 write 0x88b9c00 thru 0x88c0ac5 (0x6ec6 bytes) 2024-09-17T21:19:37.982 INFO:teuthology.orchestra.run.smithi184.stdout:3723 trunc from 0x88c0c00 to 0xa699600 2024-09-17T21:19:38.013 INFO:teuthology.orchestra.run.smithi184.stdout:3724 write 0x5e6bc00 thru 0x5e78a8e (0xce8f bytes) 2024-09-17T21:19:38.044 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:38.044 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:38.059 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:19:38.060 INFO:teuthology.orchestra.run.smithi184.stdout:3725 write 0xea0da00 thru 0xea12ea3 (0x54a4 bytes) 2024-09-17T21:19:38.064 INFO:teuthology.orchestra.run.smithi184.stdout:3727 write 0xa365c00 thru 0xa3759ee (0xfdef bytes) 2024-09-17T21:19:38.064 INFO:teuthology.orchestra.run.smithi184.stdout:3728 read 0x4cfc400 thru 0x4cffae4 (0x36e5 bytes) 2024-09-17T21:19:38.064 INFO:teuthology.orchestra.run.smithi184.stdout:3731 write 0x4934a00 thru 0x4935203 (0x804 bytes) 2024-09-17T21:19:38.096 INFO:teuthology.orchestra.run.smithi184.stdout:3733 trunc from 0xea13000 to 0xd7be00 2024-09-17T21:19:38.100 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:38.098+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:38.176 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:38.170+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:38.205 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:38.205 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:38.218 INFO:teuthology.orchestra.run.smithi184.stdout:3735 write 0xc71e200 thru 0xc71ee19 (0xc1a bytes) 2024-09-17T21:19:38.220 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:19:38.229 INFO:teuthology.orchestra.run.smithi184.stdout:3736 trunc from 0xc71f000 to 0x9885800 2024-09-17T21:19:38.271 INFO:teuthology.orchestra.run.smithi184.stdout:3738 trunc from 0x9885800 to 0x6059600 2024-09-17T21:19:38.277 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:38.274+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:38.293 INFO:teuthology.orchestra.run.smithi184.stdout:3742 write 0x2e4cc00 thru 0x2e58a8f (0xbe90 bytes) 2024-09-17T21:19:38.313 INFO:teuthology.orchestra.run.smithi184.stdout:3743 trunc from 0x6059600 to 0xcb84800 2024-09-17T21:19:38.344 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:38.345 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:38.346 INFO:teuthology.orchestra.run.smithi184.stdout:3751 write 0x5fdf200 thru 0x5fe69b9 (0x77ba bytes) 2024-09-17T21:19:38.359 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:19:38.377 INFO:teuthology.orchestra.run.smithi184.stdout:3752 trunc from 0xcb84800 to 0x16b2200 2024-09-17T21:19:38.410 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:38.408+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:38.422 INFO:teuthology.orchestra.run.smithi184.stdout:3753 read 0x2a7000 thru 0x2aec6d (0x7c6e bytes) 2024-09-17T21:19:38.425 INFO:teuthology.orchestra.run.smithi184.stdout:3754 read 0x4a3200 thru 0x4a323a (0x3b bytes) 2024-09-17T21:19:38.478 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:38.476+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:38.482 INFO:teuthology.orchestra.run.smithi184.stdout:3755 write 0xdf20200 thru 0xdf21ae8 (0x18e9 bytes) 2024-09-17T21:19:38.484 INFO:teuthology.orchestra.run.smithi184.stdout:3756 punch from 0x57ede00 to 0x57ee400, (0x600 bytes) 2024-09-17T21:19:38.485 INFO:teuthology.orchestra.run.smithi184.stdout:3757 punch from 0x49e1e00 to 0x49ea800, (0x8a00 bytes) 2024-09-17T21:19:38.485 INFO:teuthology.orchestra.run.smithi184.stdout:3760 read 0x1272000 thru 0x127d60d (0xb60e bytes) 2024-09-17T21:19:38.486 INFO:teuthology.orchestra.run.smithi184.stdout:3761 punch from 0xa19f600 to 0xa1a3600, (0x4000 bytes) 2024-09-17T21:19:38.486 INFO:teuthology.orchestra.run.smithi184.stdout:3762 write 0x2d09e00 thru 0x2d0bade (0x1cdf bytes) 2024-09-17T21:19:38.486 INFO:teuthology.orchestra.run.smithi184.stdout:3763 write 0x7f86600 thru 0x7f956e7 (0xf0e8 bytes) 2024-09-17T21:19:38.486 INFO:teuthology.orchestra.run.smithi184.stdout:3764 write 0xc676800 thru 0xc67f038 (0x8839 bytes) 2024-09-17T21:19:38.486 INFO:teuthology.orchestra.run.smithi184.stdout:3767 punch from 0x34fdc00 to 0x3501800, (0x3c00 bytes) 2024-09-17T21:19:38.487 INFO:teuthology.orchestra.run.smithi184.stdout:3768 write 0x66b2200 thru 0x66b89de (0x67df bytes) 2024-09-17T21:19:38.487 INFO:teuthology.orchestra.run.smithi184.stdout:3769 read 0x89fc200 thru 0x8a08b0e (0xc90f bytes) 2024-09-17T21:19:38.487 INFO:teuthology.orchestra.run.smithi184.stdout:3771 punch from 0x5da5200 to 0x5daf200, (0xa000 bytes) 2024-09-17T21:19:38.488 INFO:teuthology.orchestra.run.smithi184.stdout:3772 write 0x8869c00 thru 0x8873692 (0x9a93 bytes) 2024-09-17T21:19:38.488 INFO:teuthology.orchestra.run.smithi184.stdout:3773 clone 22 order 18 su 16384 sc 14 2024-09-17T21:19:38.501 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:38.501 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:38.521 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:19:38.610 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:38.608+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:38.613 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:19:38.613 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:19:38.613 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-17T21:19:38.613 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T21:19:38.613 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T21:19:38.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-17T21:19:38.680 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:38.676+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:38.694 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:38.694 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:38.709 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:19:38.753 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:38.753 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "" 2024-09-17T21:19:38.753 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:38.794 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:38.792+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:38.811 INFO:teuthology.orchestra.run.smithi184.stdout:leaving image image_client.0-clone21 intact 2024-09-17T21:19:38.875 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:38.875 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:38.881 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:38.876+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:38.890 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:19:38.982 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:38.980+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:39.026 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:19:39.056 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:39.056 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:39.071 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:19:39.083 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:39.078+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:39.183 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:39.184 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:39.178+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:39.200 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:39.217 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:39.218 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:39.236 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:39.241 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:39.241 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:39.255 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:19:39.258 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:39.259 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:39.278 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:39.278 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:39.298 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:39.296+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:39.379 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:39.379 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:39.379 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:39.380 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:39.380 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:39.386 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:39.384+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:39.394 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:19:39.407 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-17T21:19:39.487 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:39.486+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:39.524 INFO:teuthology.orchestra.run.smithi184.stdout:3775 trunc from 0xdf21c00 to 0x8eb1800 2024-09-17T21:19:39.576 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:39.576 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:39.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:39.576 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:39.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.576 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:19:39.576 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:39.577 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.968005067000007, 2024-09-17T21:19:39.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:19:39.577 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:39.577 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:39.577 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:39.577 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:39.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:39.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:39.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:39.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:39.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:39.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:19:39.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:19:39.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:39.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:19:39.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:19:39.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:39.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:19:39.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:19:39.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:39.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:19:39.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:19:39.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:39.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:19:39.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:19:39.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:39.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:19:39.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:19:39.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:39.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:19:39.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:19:39.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:39.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:19:39.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:19:39.580 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.581 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:39.581 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.581 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:19:39.581 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:39.581 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.958058430999998, 2024-09-17T21:19:39.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:19:39.581 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:39.581 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:39.581 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:39.581 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:39.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:39.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:39.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:39.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:39.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:39.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:19:39.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:19:39.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:39.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:19:39.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:19:39.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:39.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:19:39.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:19:39.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:39.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:19:39.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:19:39.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:39.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:19:39.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:19:39.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:39.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:19:39.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:19:39.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:39.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:19:39.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:19:39.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:39.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:19:39.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:19:39.585 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.585 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:39.585 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:19:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 76.714907590999999, 2024-09-17T21:19:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:19:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:19:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:19:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:19:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:19:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:19:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:39.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:19:39.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:19:39.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:39.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:19:39.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:19:39.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:39.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:19:39.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:19:39.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:39.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:19:39.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:19:39.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:39.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:19:39.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:19:39.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:39.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:19:39.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:19:39.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:39.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:19:39.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:19:39.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:39.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:19:39.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:19:39.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:39.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:19:39.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:19:39.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:39.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:19:39.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:19:39.591 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.591 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:39.591 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.591 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:19:39.592 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:39.592 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 76.705262105000003, 2024-09-17T21:19:39.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:19:39.592 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:39.592 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:39.592 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:39.592 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:39.592 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:19:39.592 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:19:39.592 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:19:39.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.592 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:39.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:39.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:39.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:39.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:39.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:39.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:19:39.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:19:39.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:39.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:19:39.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:19:39.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:39.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:19:39.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:19:39.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:39.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:19:39.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:19:39.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:39.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:19:39.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:19:39.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:39.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:19:39.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:19:39.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:39.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:19:39.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:19:39.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:39.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:19:39.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:19:39.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:39.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:19:39.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:19:39.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:39.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:19:39.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:19:39.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:39.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:19:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:19:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 63.336326071999999, 2024-09-17T21:19:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:19:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:19:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:19:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:19:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:19:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:19:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:19:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:19:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:19:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:19:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:19:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:19:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:19:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:19:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:19:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:19:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:565 3.1d 3:bbad910d:::rbd_data.11527824718.0000000000000005:9 [sparse-read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:19:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 63.332537107, 2024-09-17T21:19:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024193392000000001, 2024-09-17T21:19:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 565 2024-09-17T21:19:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:19:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239658+0000", 2024-09-17T21:19:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239660+0000", 2024-09-17T21:19:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:19:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239666+0000", 2024-09-17T21:19:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8699999999999997e-06 2024-09-17T21:19:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239688+0000", 2024-09-17T21:19:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.213e-05 2024-09-17T21:19:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:39.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.239703+0000", 2024-09-17T21:19:39.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4813e-05 2024-09-17T21:19:39.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:39.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263427+0000", 2024-09-17T21:19:39.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023724630999999999 2024-09-17T21:19:39.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:39.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263614+0000", 2024-09-17T21:19:39.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018664500000000001 2024-09-17T21:19:39.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.610 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:39.610 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.263852+0000", 2024-09-17T21:19:39.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023793999999999999 2024-09-17T21:19:39.610 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.610 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:39.610 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.610 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:819 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5111808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:39.610 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:39.610 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.134753781999997, 2024-09-17T21:19:39.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024344811000000001, 2024-09-17T21:19:39.610 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:39.611 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:39.611 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:39.611 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:39.611 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:39.611 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:39.611 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 819 2024-09-17T21:19:39.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.611 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:39.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:39.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:39.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:39.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:39.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:39.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437443+0000", 2024-09-17T21:19:39.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1969999999999999e-06 2024-09-17T21:19:39.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:39.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437448+0000", 2024-09-17T21:19:39.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9490000000000004e-06 2024-09-17T21:19:39.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:39.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437467+0000", 2024-09-17T21:19:39.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8862999999999999e-05 2024-09-17T21:19:39.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:39.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437480+0000", 2024-09-17T21:19:39.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2833000000000001e-05 2024-09-17T21:19:39.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:39.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461552+0000", 2024-09-17T21:19:39.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024072695000000002 2024-09-17T21:19:39.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.614 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:39.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461639+0000", 2024-09-17T21:19:39.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6996000000000005e-05 2024-09-17T21:19:39.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.614 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:39.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461787+0000", 2024-09-17T21:19:39.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014727800000000001 2024-09-17T21:19:39.614 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.614 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:39.614 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.615 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:39.615 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:39.615 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.134641510000002, 2024-09-17T21:19:39.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:19:39.615 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:39.615 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:39.615 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:39.615 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:39.615 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:39.615 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:39.615 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:19:39.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.616 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:39.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:39.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:39.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:39.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:39.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:39.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:19:39.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:19:39.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:39.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:19:39.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:19:39.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:39.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:19:39.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:19:39.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:833 3.5 3:a772dfb1:::rbd_data.11523651b015.0000000000000001:d [sparse-read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.134207224000001, 2024-09-17T21:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024216024999999999, 2024-09-17T21:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 833 2024-09-17T21:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437990+0000", 2024-09-17T21:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2500000000000001e-06 2024-09-17T21:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437995+0000", 2024-09-17T21:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1240000000000004e-06 2024-09-17T21:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438012+0000", 2024-09-17T21:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7572e-05 2024-09-17T21:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438031+0000", 2024-09-17T21:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8612000000000001e-05 2024-09-17T21:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461830+0000", 2024-09-17T21:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023799135999999999 2024-09-17T21:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462005+0000", 2024-09-17T21:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000174747 2024-09-17T21:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019958399999999999 2024-09-17T21:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.134146932999997, 2024-09-17T21:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:19:39.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:19:39.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:39.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:19:39.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:19:39.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.134091552000001, 2024-09-17T21:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:19:39.633 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.633 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:39.633 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.633 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:39.633 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:39.633 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.134035273999999, 2024-09-17T21:19:39.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:19:39.633 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:39.633 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:39.633 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:39.633 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.133085635, 2024-09-17T21:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.133007014999997, 2024-09-17T21:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:847 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 4653056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439245+0000", 2024-09-17T21:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.132950577000003, 2024-09-17T21:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024172945000000001, 2024-09-17T21:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 847 2024-09-17T21:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439245+0000", 2024-09-17T21:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439245+0000", 2024-09-17T21:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439246+0000", 2024-09-17T21:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439252+0000", 2024-09-17T21:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3569999999999997e-06 2024-09-17T21:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439269+0000", 2024-09-17T21:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7316000000000002e-05 2024-09-17T21:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439282+0000", 2024-09-17T21:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3269e-05 2024-09-17T21:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463240+0000", 2024-09-17T21:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02395721 2024-09-17T21:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463306+0000", 2024-09-17T21:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6618000000000005e-05 2024-09-17T21:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463418+0000", 2024-09-17T21:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111672 2024-09-17T21:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.132894075999999, 2024-09-17T21:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:849 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5963776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.132300039999997, 2024-09-17T21:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024537514, 2024-09-17T21:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 849 2024-09-17T21:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439897+0000", 2024-09-17T21:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9099999999999999e-06 2024-09-17T21:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439905+0000", 2024-09-17T21:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0870000000000007e-06 2024-09-17T21:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439924+0000", 2024-09-17T21:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9042000000000002e-05 2024-09-17T21:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439941+0000", 2024-09-17T21:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.624e-05 2024-09-17T21:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464132+0000", 2024-09-17T21:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024191579000000001 2024-09-17T21:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464291+0000", 2024-09-17T21:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015904800000000001 2024-09-17T21:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464433+0000", 2024-09-17T21:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014160799999999999 2024-09-17T21:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.132233022000001, 2024-09-17T21:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.132168211, 2024-09-17T21:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:19:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:19:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:19:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.132107234000003, 2024-09-17T21:19:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:19:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:19:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:19:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:19:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:19:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:19:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:19:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:19:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:19:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:19:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:19:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:39.674 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:39.674 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:19:39.676 INFO:teuthology.orchestra.run.smithi184.stdout:3778 read 0x4aaf200 thru 0x4ab4277 (0x5078 bytes) 2024-09-17T21:19:39.676 INFO:teuthology.orchestra.run.smithi184.stdout:3779 punch from 0x1b13e00 to 0x1b18000, (0x4200 bytes) 2024-09-17T21:19:39.676 INFO:teuthology.orchestra.run.smithi184.stdout:3780 punch from 0x2cde000 to 0x2ce3400, (0x5400 bytes) 2024-09-17T21:19:39.676 INFO:teuthology.orchestra.run.smithi184.stdout:3781 read 0x4a12400 thru 0x4a1870f (0x6310 bytes) 2024-09-17T21:19:39.676 INFO:teuthology.orchestra.run.smithi184.stdout:3783 read 0x56c1000 thru 0x56cfb50 (0xeb51 bytes) 2024-09-17T21:19:39.677 INFO:teuthology.orchestra.run.smithi184.stdout:3784 trunc from 0x8eb1800 to 0x25dae00 2024-09-17T21:19:39.677 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:39.672+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:39.677 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:19:39.730 INFO:teuthology.orchestra.run.smithi184.stdout:3785 trunc from 0x25dae00 to 0xdeb1800 2024-09-17T21:19:39.745 INFO:teuthology.orchestra.run.smithi184.stdout:3786 punch from 0x8b5be00 to 0x8b5f800, (0x3a00 bytes) 2024-09-17T21:19:39.746 INFO:teuthology.orchestra.run.smithi184.stdout:3787 write 0x5ea0000 thru 0x5ea5fc6 (0x5fc7 bytes) 2024-09-17T21:19:39.746 INFO:teuthology.orchestra.run.smithi184.stdout:3789 read 0x1bbf600 thru 0x1bcd500 (0xdf01 bytes) 2024-09-17T21:19:39.776 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:39.772+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:39.777 INFO:teuthology.orchestra.run.smithi184.stdout:3790 write 0xe751200 thru 0xe75b8be (0xa6bf bytes) 2024-09-17T21:19:39.778 INFO:teuthology.orchestra.run.smithi184.stdout:3791 read 0xce74200 thru 0xce748ed (0x6ee bytes) 2024-09-17T21:19:39.778 INFO:teuthology.orchestra.run.smithi184.stdout:3800 read 0x43ea400 thru 0x43eb24f (0xe50 bytes) 2024-09-17T21:19:39.778 INFO:teuthology.orchestra.run.smithi184.stdout:3805 read 0x4d5d200 thru 0x4d686bf (0xb4c0 bytes) 2024-09-17T21:19:39.778 INFO:teuthology.orchestra.run.smithi184.stdout:3806 read 0x8436600 thru 0x8440676 (0xa077 bytes) 2024-09-17T21:19:39.778 INFO:teuthology.orchestra.run.smithi184.stdout:3809 read 0x7180200 thru 0x718882e (0x862f bytes) 2024-09-17T21:19:39.778 INFO:teuthology.orchestra.run.smithi184.stdout:3812 write 0x5c8a00 thru 0x5d207e (0x967f bytes) 2024-09-17T21:19:39.781 INFO:teuthology.orchestra.run.smithi184.stdout:3813 punch from 0x74e3a00 to 0x74eec00, (0xb200 bytes) 2024-09-17T21:19:39.782 INFO:teuthology.orchestra.run.smithi184.stdout:3814 write 0x747d600 thru 0x7481991 (0x4392 bytes) 2024-09-17T21:19:39.782 INFO:teuthology.orchestra.run.smithi184.stdout:3815 punch from 0xa140800 to 0xa14a200, (0x9a00 bytes) 2024-09-17T21:19:39.783 INFO:teuthology.orchestra.run.smithi184.stdout:3817 punch from 0x7b5e00 to 0x7b9c00, (0x3e00 bytes) 2024-09-17T21:19:39.784 INFO:teuthology.orchestra.run.smithi184.stdout:3818 write 0x418a000 thru 0x418e848 (0x4849 bytes) 2024-09-17T21:19:39.784 INFO:teuthology.orchestra.run.smithi184.stdout:3820 punch from 0xaa9ac00 to 0xaaa7c00, (0xd000 bytes) 2024-09-17T21:19:39.784 INFO:teuthology.orchestra.run.smithi184.stdout:3821 read 0x49fea00 thru 0x4a0bb48 (0xd149 bytes) 2024-09-17T21:19:39.810 INFO:teuthology.orchestra.run.smithi184.stdout:3822 trunc from 0xe75ba00 to 0x204ba00 2024-09-17T21:19:39.869 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:39.869 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:39.869 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:19:39.869 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:39.876 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:39.874+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:39.892 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:19:39.907 INFO:teuthology.orchestra.run.smithi184.stdout:3823 trunc from 0x204ba00 to 0x63ed600 2024-09-17T21:19:39.920 INFO:teuthology.orchestra.run.smithi184.stdout:3825 read 0x62b4800 thru 0x62c2a9f (0xe2a0 bytes) 2024-09-17T21:19:39.922 INFO:teuthology.orchestra.run.smithi184.stdout:3826 trunc from 0x63ed600 to 0x5daec00 2024-09-17T21:19:39.946 INFO:teuthology.orchestra.run.smithi184.stdout:3830 trunc from 0x5daec00 to 0x99bf000 2024-09-17T21:19:39.980 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:39.978+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:39.985 INFO:teuthology.orchestra.run.smithi184.stdout:3832 write 0xb426600 thru 0xb4292a4 (0x2ca5 bytes) 2024-09-17T21:19:39.987 INFO:teuthology.orchestra.run.smithi184.stdout:3837 write 0x7eae400 thru 0x7eb7934 (0x9535 bytes) 2024-09-17T21:19:39.987 INFO:teuthology.orchestra.run.smithi184.stdout:3838 read 0x39d3400 thru 0x39dac5a (0x785b bytes) 2024-09-17T21:19:40.001 INFO:teuthology.orchestra.run.smithi184.stdout:3839 trunc from 0xb429400 to 0x60c0000 2024-09-17T21:19:40.048 INFO:teuthology.orchestra.run.smithi184.stdout:3843 read 0x5c20200 thru 0x5c205f3 (0x3f4 bytes) 2024-09-17T21:19:40.059 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:40.060 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:40.060 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:40.060 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:40.060 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:40.075 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:19:40.079 INFO:teuthology.orchestra.run.smithi184.stdout:3844 write 0xe61d600 thru 0xe628556 (0xaf57 bytes) 2024-09-17T21:19:40.081 INFO:teuthology.orchestra.run.smithi184.stdout:3845 write 0xb818800 thru 0xb81b8f2 (0x30f3 bytes) 2024-09-17T21:19:40.083 INFO:teuthology.orchestra.run.smithi184.stdout:3850 trunc from 0xe628600 to 0xd9b3400 2024-09-17T21:19:40.117 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:40.114+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:40.132 INFO:teuthology.orchestra.run.smithi184.stdout:3853 read 0x38e4000 thru 0x38f3d3d (0xfd3e bytes) 2024-09-17T21:19:40.132 INFO:teuthology.orchestra.run.smithi184.stdout:3854 read 0x4290400 thru 0x4297202 (0x6e03 bytes) 2024-09-17T21:19:40.132 INFO:teuthology.orchestra.run.smithi184.stdout:3855 read 0x402d600 thru 0x4039e4a (0xc84b bytes) 2024-09-17T21:19:40.132 INFO:teuthology.orchestra.run.smithi184.stdout:3856 punch from 0x209c00 to 0x216000, (0xc400 bytes) 2024-09-17T21:19:40.133 INFO:teuthology.orchestra.run.smithi184.stdout:3857 read 0x141ee00 thru 0x142c4d3 (0xd6d4 bytes) 2024-09-17T21:19:40.156 INFO:teuthology.orchestra.run.smithi184.stdout:3858 trunc from 0xd9b3400 to 0x523d000 2024-09-17T21:19:40.179 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:40.176+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:40.249 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:40.249 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:40.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:40.249 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:40.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.249 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:406 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 2359296~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.249 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:19:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.052122351999998, 2024-09-17T21:19:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042448018999999997, 2024-09-17T21:19:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 406 2024-09-17T21:19:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:19:40.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195054+0000", 2024-09-17T21:19:40.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195056+0000", 2024-09-17T21:19:40.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2729999999999999e-06 2024-09-17T21:19:40.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195060+0000", 2024-09-17T21:19:40.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5399999999999997e-06 2024-09-17T21:19:40.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195076+0000", 2024-09-17T21:19:40.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6217000000000001e-05 2024-09-17T21:19:40.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195093+0000", 2024-09-17T21:19:40.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.683e-05 2024-09-17T21:19:40.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236217+0000", 2024-09-17T21:19:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041124102000000003 2024-09-17T21:19:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.236339+0000", 2024-09-17T21:19:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121876 2024-09-17T21:19:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237502+0000", 2024-09-17T21:19:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011631809999999999 2024-09-17T21:19:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:416 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2228224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:19:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.051624325999995, 2024-09-17T21:19:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042361990000000002, 2024-09-17T21:19:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 416 2024-09-17T21:19:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:19:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195552+0000", 2024-09-17T21:19:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195554+0000", 2024-09-17T21:19:40.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9300000000000002e-06 2024-09-17T21:19:40.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195563+0000", 2024-09-17T21:19:40.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2770000000000005e-06 2024-09-17T21:19:40.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195586+0000", 2024-09-17T21:19:40.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3889000000000001e-05 2024-09-17T21:19:40.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.195603+0000", 2024-09-17T21:19:40.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6084000000000001e-05 2024-09-17T21:19:40.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237563+0000", 2024-09-17T21:19:40.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041960856999999997 2024-09-17T21:19:40.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237695+0000", 2024-09-17T21:19:40.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013144400000000001 2024-09-17T21:19:40.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237914+0000", 2024-09-17T21:19:40.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000219509 2024-09-17T21:19:40.258 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.258 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.258 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.259 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:439 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.259 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:19:40.259 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.051060423999999, 2024-09-17T21:19:40.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043053811999999997, 2024-09-17T21:19:40.259 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.259 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.259 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.259 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.259 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.259 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.259 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 439 2024-09-17T21:19:40.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.259 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:19:40.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196116+0000", 2024-09-17T21:19:40.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196118+0000", 2024-09-17T21:19:40.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3969999999999999e-06 2024-09-17T21:19:40.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196157+0000", 2024-09-17T21:19:40.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9826999999999997e-05 2024-09-17T21:19:40.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196179+0000", 2024-09-17T21:19:40.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1756e-05 2024-09-17T21:19:40.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196194+0000", 2024-09-17T21:19:40.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4707e-05 2024-09-17T21:19:40.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.237980+0000", 2024-09-17T21:19:40.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041786219999999999 2024-09-17T21:19:40.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.238065+0000", 2024-09-17T21:19:40.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4560000000000004e-05 2024-09-17T21:19:40.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239170+0000", 2024-09-17T21:19:40.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001105345 2024-09-17T21:19:40.263 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.263 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.263 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.263 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:440 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.263 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:19:40.263 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.050957921000006, 2024-09-17T21:19:40.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043890039999999998, 2024-09-17T21:19:40.263 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.263 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.263 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.264 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.264 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.264 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.264 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 440 2024-09-17T21:19:40.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.264 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:19:40.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196219+0000", 2024-09-17T21:19:40.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196220+0000", 2024-09-17T21:19:40.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.254e-06 2024-09-17T21:19:40.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196225+0000", 2024-09-17T21:19:40.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7829999999999998e-06 2024-09-17T21:19:40.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196241+0000", 2024-09-17T21:19:40.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6322999999999999e-05 2024-09-17T21:19:40.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196258+0000", 2024-09-17T21:19:40.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7056000000000001e-05 2024-09-17T21:19:40.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239229+0000", 2024-09-17T21:19:40.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042971058999999999 2024-09-17T21:19:40.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239403+0000", 2024-09-17T21:19:40.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017399200000000001 2024-09-17T21:19:40.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240109+0000", 2024-09-17T21:19:40.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00070557300000000003 2024-09-17T21:19:40.267 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.267 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.267 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.268 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:450 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.268 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:19:40.268 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.050596970000001, 2024-09-17T21:19:40.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043994521000000002, 2024-09-17T21:19:40.268 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.268 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.268 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.268 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.268 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.268 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.268 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 450 2024-09-17T21:19:40.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.269 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:19:40.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196580+0000", 2024-09-17T21:19:40.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196581+0000", 2024-09-17T21:19:40.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4500000000000001e-06 2024-09-17T21:19:40.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196588+0000", 2024-09-17T21:19:40.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7530000000000004e-06 2024-09-17T21:19:40.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196607+0000", 2024-09-17T21:19:40.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8987e-05 2024-09-17T21:19:40.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196620+0000", 2024-09-17T21:19:40.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3589e-05 2024-09-17T21:19:40.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240225+0000", 2024-09-17T21:19:40.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043604878999999999 2024-09-17T21:19:40.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240377+0000", 2024-09-17T21:19:40.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015146699999999999 2024-09-17T21:19:40.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240574+0000", 2024-09-17T21:19:40.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019739599999999999 2024-09-17T21:19:40.272 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.272 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.272 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.272 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:464 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.272 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:19:40.272 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.050346548999997, 2024-09-17T21:19:40.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044651692999999999, 2024-09-17T21:19:40.272 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.273 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.273 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.273 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.273 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.273 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.273 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 464 2024-09-17T21:19:40.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.273 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:19:40.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196830+0000", 2024-09-17T21:19:40.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196838+0000", 2024-09-17T21:19:40.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4970000000000004e-06 2024-09-17T21:19:40.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196859+0000", 2024-09-17T21:19:40.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1166e-05 2024-09-17T21:19:40.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196879+0000", 2024-09-17T21:19:40.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.995e-05 2024-09-17T21:19:40.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.196896+0000", 2024-09-17T21:19:40.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6929999999999999e-05 2024-09-17T21:19:40.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240617+0000", 2024-09-17T21:19:40.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043721784999999999 2024-09-17T21:19:40.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240702+0000", 2024-09-17T21:19:40.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4060999999999994e-05 2024-09-17T21:19:40.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241482+0000", 2024-09-17T21:19:40.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000780304 2024-09-17T21:19:40.276 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.276 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.276 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.277 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:475 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.277 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:19:40.277 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.050077416999997, 2024-09-17T21:19:40.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044661981000000003, 2024-09-17T21:19:40.277 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.277 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.277 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.277 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.277 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.277 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.277 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 475 2024-09-17T21:19:40.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.277 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:19:40.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197099+0000", 2024-09-17T21:19:40.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197100+0000", 2024-09-17T21:19:40.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.206e-06 2024-09-17T21:19:40.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197105+0000", 2024-09-17T21:19:40.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.527e-06 2024-09-17T21:19:40.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197124+0000", 2024-09-17T21:19:40.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:19:40.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197136+0000", 2024-09-17T21:19:40.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2857000000000001e-05 2024-09-17T21:19:40.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241539+0000", 2024-09-17T21:19:40.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044402077999999998 2024-09-17T21:19:40.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241653+0000", 2024-09-17T21:19:40.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114931 2024-09-17T21:19:40.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241761+0000", 2024-09-17T21:19:40.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107753 2024-09-17T21:19:40.281 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.281 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.281 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.281 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:476 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.281 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:19:40.281 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.050023592000002, 2024-09-17T21:19:40.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045038976000000001, 2024-09-17T21:19:40.281 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.281 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.282 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.282 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.282 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.282 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.282 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 476 2024-09-17T21:19:40.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.282 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:19:40.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197153+0000", 2024-09-17T21:19:40.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197154+0000", 2024-09-17T21:19:40.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.127e-06 2024-09-17T21:19:40.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197159+0000", 2024-09-17T21:19:40.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4569999999999998e-06 2024-09-17T21:19:40.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197174+0000", 2024-09-17T21:19:40.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5173e-05 2024-09-17T21:19:40.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197186+0000", 2024-09-17T21:19:40.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:19:40.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241863+0000", 2024-09-17T21:19:40.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044676826000000003 2024-09-17T21:19:40.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241998+0000", 2024-09-17T21:19:40.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013476100000000001 2024-09-17T21:19:40.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242192+0000", 2024-09-17T21:19:40.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019436900000000001 2024-09-17T21:19:40.285 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.285 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.285 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.286 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:494 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.286 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:40.286 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.049848409000006, 2024-09-17T21:19:40.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046244995999999997, 2024-09-17T21:19:40.286 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.286 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.286 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.286 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.286 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.286 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.286 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 494 2024-09-17T21:19:40.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.286 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:40.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:40.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197329+0000", 2024-09-17T21:19:40.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.173e-06 2024-09-17T21:19:40.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197334+0000", 2024-09-17T21:19:40.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0499999999999999e-06 2024-09-17T21:19:40.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197352+0000", 2024-09-17T21:19:40.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7133e-05 2024-09-17T21:19:40.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197367+0000", 2024-09-17T21:19:40.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5123000000000001e-05 2024-09-17T21:19:40.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:19:40.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044989829000000002 2024-09-17T21:19:40.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242526+0000", 2024-09-17T21:19:40.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016968199999999999 2024-09-17T21:19:40.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243573+0000", 2024-09-17T21:19:40.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010470060000000001 2024-09-17T21:19:40.290 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.290 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.290 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.290 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:502 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.290 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:19:40.290 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.047614749999994, 2024-09-17T21:19:40.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044346086999999999, 2024-09-17T21:19:40.290 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.290 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.290 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.291 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.291 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.291 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.291 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 502 2024-09-17T21:19:40.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.291 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:19:40.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199562+0000", 2024-09-17T21:19:40.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199563+0000", 2024-09-17T21:19:40.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2929999999999999e-06 2024-09-17T21:19:40.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199569+0000", 2024-09-17T21:19:40.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5330000000000002e-06 2024-09-17T21:19:40.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199614+0000", 2024-09-17T21:19:40.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4793000000000003e-05 2024-09-17T21:19:40.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199628+0000", 2024-09-17T21:19:40.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4037e-05 2024-09-17T21:19:40.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243665+0000", 2024-09-17T21:19:40.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044037106999999999 2024-09-17T21:19:40.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243767+0000", 2024-09-17T21:19:40.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010279499999999999 2024-09-17T21:19:40.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243908+0000", 2024-09-17T21:19:40.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140529 2024-09-17T21:19:40.294 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.294 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.294 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.295 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:523 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.295 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:19:40.295 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.047168596999995, 2024-09-17T21:19:40.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044789645000000003, 2024-09-17T21:19:40.295 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.295 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.295 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.295 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.295 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.295 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.295 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 523 2024-09-17T21:19:40.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.296 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:19:40.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200008+0000", 2024-09-17T21:19:40.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:19:40.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9e-06 2024-09-17T21:19:40.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200021+0000", 2024-09-17T21:19:40.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9499999999999998e-06 2024-09-17T21:19:40.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200038+0000", 2024-09-17T21:19:40.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.73e-05 2024-09-17T21:19:40.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200051+0000", 2024-09-17T21:19:40.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.322e-05 2024-09-17T21:19:40.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243987+0000", 2024-09-17T21:19:40.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043935386999999999 2024-09-17T21:19:40.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244136+0000", 2024-09-17T21:19:40.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014868300000000001 2024-09-17T21:19:40.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244798+0000", 2024-09-17T21:19:40.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00066220500000000002 2024-09-17T21:19:40.299 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.299 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.299 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.299 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:524 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.299 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:40.299 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.047102381000002, 2024-09-17T21:19:40.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045737414999999997, 2024-09-17T21:19:40.300 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.300 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.300 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.300 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.300 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.300 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.300 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 524 2024-09-17T21:19:40.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.300 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:40.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:40.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200076+0000", 2024-09-17T21:19:40.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:19:40.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200081+0000", 2024-09-17T21:19:40.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7030000000000003e-06 2024-09-17T21:19:40.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200099+0000", 2024-09-17T21:19:40.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7836999999999999e-05 2024-09-17T21:19:40.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200113+0000", 2024-09-17T21:19:40.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3570000000000001e-05 2024-09-17T21:19:40.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244866+0000", 2024-09-17T21:19:40.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044753573999999997 2024-09-17T21:19:40.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244972+0000", 2024-09-17T21:19:40.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106111 2024-09-17T21:19:40.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245812+0000", 2024-09-17T21:19:40.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083923999999999999 2024-09-17T21:19:40.303 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.303 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.304 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.304 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:525 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3670016~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.304 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:40.304 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.046950999000003, 2024-09-17T21:19:40.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046645697, 2024-09-17T21:19:40.304 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.304 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.304 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.304 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.304 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.304 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.304 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 525 2024-09-17T21:19:40.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.305 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:40.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:40.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:19:40.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:19:40.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200234+0000", 2024-09-17T21:19:40.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:19:40.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200254+0000", 2024-09-17T21:19:40.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9959999999999999e-05 2024-09-17T21:19:40.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200293+0000", 2024-09-17T21:19:40.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9755999999999997e-05 2024-09-17T21:19:40.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245871+0000", 2024-09-17T21:19:40.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045577871999999998 2024-09-17T21:19:40.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246030+0000", 2024-09-17T21:19:40.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158975 2024-09-17T21:19:40.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246871+0000", 2024-09-17T21:19:40.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084111699999999995 2024-09-17T21:19:40.308 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.308 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.308 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.308 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.308 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:40.308 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.046705324000001, 2024-09-17T21:19:40.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:19:40.309 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.309 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.309 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.309 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.309 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.309 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.309 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:19:40.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.309 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.309 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.309 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:40.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:40.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:19:40.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:19:40.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:19:40.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:19:40.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:19:40.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:19:40.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:19:40.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:19:40.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:19:40.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:19:40.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:19:40.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:19:40.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:19:40.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:19:40.312 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.313 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.313 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.313 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.313 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:40.313 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.046526642000003, 2024-09-17T21:19:40.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:19:40.313 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.313 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.314 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.314 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.314 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.314 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.314 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:19:40.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.314 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:40.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:40.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:19:40.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:19:40.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:19:40.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:19:40.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:19:40.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:19:40.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:19:40.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:19:40.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:19:40.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:19:40.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:19:40.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:19:40.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:19:40.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:19:40.317 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.317 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.317 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.318 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.318 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:40.318 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.046467368999998, 2024-09-17T21:19:40.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:19:40.318 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.318 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.318 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.318 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.318 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.318 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.318 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:19:40.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.319 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:40.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:40.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:19:40.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:19:40.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:19:40.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:19:40.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:19:40.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:19:40.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:19:40.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:19:40.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:19:40.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:19:40.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:19:40.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:19:40.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:19:40.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:19:40.322 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.322 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.322 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.322 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.322 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:40.322 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.046325174000003, 2024-09-17T21:19:40.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:19:40.323 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.323 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.323 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.323 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.323 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.323 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.323 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:19:40.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.323 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:40.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:40.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:19:40.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:19:40.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:19:40.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:19:40.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:19:40.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:19:40.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:19:40.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:19:40.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:19:40.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:19:40.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:19:40.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:19:40.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:19:40.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:19:40.326 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.327 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.327 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.327 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.327 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:40.327 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.046269250999998, 2024-09-17T21:19:40.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:19:40.327 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.327 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.327 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.327 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.327 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.327 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.328 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:19:40.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.328 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:40.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:40.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:19:40.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:19:40.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:19:40.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:19:40.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:19:40.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:19:40.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:19:40.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:19:40.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:19:40.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:19:40.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:19:40.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:19:40.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:19:40.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:19:40.331 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.331 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.331 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.332 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.332 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:40.332 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.046215634000006, 2024-09-17T21:19:40.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:19:40.332 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.332 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.332 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.332 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.332 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.332 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.332 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:19:40.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.332 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:40.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:40.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:19:40.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:19:40.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:19:40.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:19:40.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:19:40.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:19:40.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:19:40.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:19:40.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:19:40.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:19:40.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:19:40.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:19:40.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:19:40.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:19:40.336 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.336 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.336 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.336 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:19:40.336 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:19:40.336 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 23.736987392, 2024-09-17T21:19:40.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:19:40.336 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.337 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.337 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:19:40.337 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.337 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:19:40.337 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:19:40.337 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:19:40.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.337 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:19:40.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:19:40.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:19:40.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:19:40.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:19:40.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:19:40.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:19:40.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:19:40.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:19:40.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:19:40.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:19:40.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:19:40.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:40.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:19:40.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:19:40.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:19:40.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:19:40.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:19:40.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:19:40.341 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.341 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.341 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.341 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.341 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.341 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:40.343 INFO:teuthology.orchestra.run.smithi184.stdout:3860 write 0x9abac00 thru 0x9ac4b80 (0x9f81 bytes) 2024-09-17T21:19:40.343 INFO:teuthology.orchestra.run.smithi184.stdout:3861 punch from 0x2aaaa00 to 0x2aaec00, (0x4200 bytes) 2024-09-17T21:19:40.343 INFO:teuthology.orchestra.run.smithi184.stdout:3862 write 0xe8ba400 thru 0xe8c67a1 (0xc3a2 bytes) 2024-09-17T21:19:40.343 INFO:teuthology.orchestra.run.smithi184.stdout:3864 write 0x2d07e00 thru 0x2d1059b (0x879c bytes) 2024-09-17T21:19:40.343 INFO:teuthology.orchestra.run.smithi184.stdout:3865 trunc from 0xe8c6800 to 0x93f8a00 2024-09-17T21:19:40.343 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:19:40.344 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:40.340+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:40.370 INFO:teuthology.orchestra.run.smithi184.stdout:3868 punch from 0x79a9600 to 0x79b6200, (0xcc00 bytes) 2024-09-17T21:19:40.372 INFO:teuthology.orchestra.run.smithi184.stdout:3869 read 0x7e0d200 thru 0x7e18726 (0xb527 bytes) 2024-09-17T21:19:40.372 INFO:teuthology.orchestra.run.smithi184.stdout:3870 punch from 0x5469e00 to 0x5478a00, (0xec00 bytes) 2024-09-17T21:19:40.373 INFO:teuthology.orchestra.run.smithi184.stdout:3873 read 0xcb6400 thru 0xcb835e (0x1f5f bytes) 2024-09-17T21:19:40.374 INFO:teuthology.orchestra.run.smithi184.stdout:3874 write 0x334fa00 thru 0x3352118 (0x2719 bytes) 2024-09-17T21:19:40.374 INFO:teuthology.orchestra.run.smithi184.stdout:3877 punch from 0x1e39400 to 0x1e43200, (0x9e00 bytes) 2024-09-17T21:19:40.375 INFO:teuthology.orchestra.run.smithi184.stdout:3879 write 0x268f000 thru 0x2691aa1 (0x2aa2 bytes) 2024-09-17T21:19:40.375 INFO:teuthology.orchestra.run.smithi184.stdout:3880 write 0x31de400 thru 0x31e8527 (0xa128 bytes) 2024-09-17T21:19:40.443 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:40.440+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:40.444 INFO:teuthology.orchestra.run.smithi184.stdout:3887 write 0xec20a00 thru 0xec2f952 (0xef53 bytes) 2024-09-17T21:19:40.460 INFO:teuthology.orchestra.run.smithi184.stdout:3888 trunc from 0xec2fa00 to 0x9862e00 2024-09-17T21:19:40.486 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:40.486 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:40.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.487 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:13683 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-17T21:19:40.487 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.483483+0000", 2024-09-17T21:19:40.487 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0022729460000000001, 2024-09-17T21:19:40.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002315526, 2024-09-17T21:19:40.487 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.487 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.487 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.487 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.487 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:19:40.487 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:40.487 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 13683 2024-09-17T21:19:40.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.487 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.483483+0000", 2024-09-17T21:19:40.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.483483+0000", 2024-09-17T21:19:40.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.483484+0000", 2024-09-17T21:19:40.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7999999999999993e-07 2024-09-17T21:19:40.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.483490+0000", 2024-09-17T21:19:40.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4589999999999997e-06 2024-09-17T21:19:40.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.483501+0000", 2024-09-17T21:19:40.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1119e-05 2024-09-17T21:19:40.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.483525+0000", 2024-09-17T21:19:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4198000000000001e-05 2024-09-17T21:19:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.485436+0000", 2024-09-17T21:19:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019110119999999999 2024-09-17T21:19:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.485639+0000", 2024-09-17T21:19:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020231700000000001 2024-09-17T21:19:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:13684 3.4 3.4f4be604 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-17T21:19:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.483551+0000", 2024-09-17T21:19:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0022048860000000001, 2024-09-17T21:19:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023983899999999998, 2024-09-17T21:19:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:19:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:19:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 13684 2024-09-17T21:19:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.483551+0000", 2024-09-17T21:19:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.483551+0000", 2024-09-17T21:19:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.483552+0000", 2024-09-17T21:19:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7999999999999995e-07 2024-09-17T21:19:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.483732+0000", 2024-09-17T21:19:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179729 2024-09-17T21:19:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.483742+0000", 2024-09-17T21:19:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0237999999999999e-05 2024-09-17T21:19:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.483750+0000", 2024-09-17T21:19:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6539999999999999e-06 2024-09-17T21:19:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.485934+0000", 2024-09-17T21:19:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021847239999999999 2024-09-17T21:19:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:19:40.495 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 2 2024-09-17T21:19:40.495 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:40.502 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:19:40.503 INFO:teuthology.orchestra.run.smithi184.stdout:3892 read 0x1bcf800 thru 0x1bd3c83 (0x4484 bytes) 2024-09-17T21:19:40.505 INFO:teuthology.orchestra.run.smithi184.stdout:3893 read 0x4a90400 thru 0x4a92b5d (0x275e bytes) 2024-09-17T21:19:40.526 INFO:teuthology.orchestra.run.smithi184.stdout:3894 write 0xacaaa00 thru 0xacb32ff (0x8900 bytes) 2024-09-17T21:19:40.527 INFO:teuthology.orchestra.run.smithi184.stdout:3895 write 0x7a69e00 thru 0x7a6bb11 (0x1d12 bytes) 2024-09-17T21:19:40.527 INFO:teuthology.orchestra.run.smithi184.stdout:3898 punch from 0x9b90200 to 0x9b9c000, (0xbe00 bytes) 2024-09-17T21:19:40.543 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:40.540+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:40.569 INFO:teuthology.orchestra.run.smithi184.stdout:3900 write 0xc986200 thru 0xc98ab9b (0x499c bytes) 2024-09-17T21:19:40.571 INFO:teuthology.orchestra.run.smithi184.stdout:3902 punch from 0xcd7a00 to 0xcd8c00, (0x1200 bytes) 2024-09-17T21:19:40.572 INFO:teuthology.orchestra.run.smithi184.stdout:3904 punch from 0xa1a0e00 to 0xa1af400, (0xe600 bytes) 2024-09-17T21:19:40.572 INFO:teuthology.orchestra.run.smithi184.stdout:3905 read 0x27f4a00 thru 0x28026fd (0xdcfe bytes) 2024-09-17T21:19:40.573 INFO:teuthology.orchestra.run.smithi184.stdout:3907 read 0x5f1c000 thru 0x5f22b9c (0x6b9d bytes) 2024-09-17T21:19:40.590 INFO:teuthology.orchestra.run.smithi184.stdout:3908 trunc from 0xc98ac00 to 0x5df6c00 2024-09-17T21:19:40.627 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:40.627 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:40.627 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:40.627 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:40.628 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:40.628 INFO:teuthology.orchestra.run.smithi184.stdout:3910 clone 23 order 25 su 524288 sc 2 2024-09-17T21:19:40.642 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:19:40.644 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:40.640+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-17T21:19:40.744 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:40.742+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:40.766 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:40.766 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:40.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:40.766 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:40.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.767 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:491 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.767 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:19:40.767 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.566062135999999, 2024-09-17T21:19:40.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041839013000000001, 2024-09-17T21:19:40.767 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.767 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.767 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.767 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.767 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.767 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.767 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 491 2024-09-17T21:19:40.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.767 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:19:40.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198314+0000", 2024-09-17T21:19:40.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198315+0000", 2024-09-17T21:19:40.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6599999999999996e-07 2024-09-17T21:19:40.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198317+0000", 2024-09-17T21:19:40.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6639999999999998e-06 2024-09-17T21:19:40.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198327+0000", 2024-09-17T21:19:40.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5629999999999998e-06 2024-09-17T21:19:40.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198337+0000", 2024-09-17T21:19:40.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0667e-05 2024-09-17T21:19:40.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239793+0000", 2024-09-17T21:19:40.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041455519000000003 2024-09-17T21:19:40.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.239929+0000", 2024-09-17T21:19:40.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135716 2024-09-17T21:19:40.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240153+0000", 2024-09-17T21:19:40.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022421799999999999 2024-09-17T21:19:40.771 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.771 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.771 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.771 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:492 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.771 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:19:40.771 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.566023766000001, 2024-09-17T21:19:40.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042947165000000002, 2024-09-17T21:19:40.772 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.772 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.772 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.772 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.772 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.772 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.772 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 492 2024-09-17T21:19:40.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.772 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:19:40.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198352+0000", 2024-09-17T21:19:40.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198354+0000", 2024-09-17T21:19:40.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:19:40.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198357+0000", 2024-09-17T21:19:40.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6969999999999999e-06 2024-09-17T21:19:40.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198367+0000", 2024-09-17T21:19:40.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8430000000000005e-06 2024-09-17T21:19:40.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198374+0000", 2024-09-17T21:19:40.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9600000000000003e-06 2024-09-17T21:19:40.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240226+0000", 2024-09-17T21:19:40.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041852157000000001 2024-09-17T21:19:40.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.240409+0000", 2024-09-17T21:19:40.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018244800000000001 2024-09-17T21:19:40.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241299+0000", 2024-09-17T21:19:40.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000890554 2024-09-17T21:19:40.775 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.776 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.776 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.776 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:493 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 4063232~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.776 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:19:40.776 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.565986679999995, 2024-09-17T21:19:40.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043897209, 2024-09-17T21:19:40.776 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.776 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.776 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.776 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.776 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.776 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.777 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 493 2024-09-17T21:19:40.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.777 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:19:40.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198389+0000", 2024-09-17T21:19:40.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198391+0000", 2024-09-17T21:19:40.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4699999999999999e-06 2024-09-17T21:19:40.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198396+0000", 2024-09-17T21:19:40.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:19:40.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198412+0000", 2024-09-17T21:19:40.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6507e-05 2024-09-17T21:19:40.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198425+0000", 2024-09-17T21:19:40.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.256e-05 2024-09-17T21:19:40.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241383+0000", 2024-09-17T21:19:40.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042958216 2024-09-17T21:19:40.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241525+0000", 2024-09-17T21:19:40.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014204500000000001 2024-09-17T21:19:40.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242287+0000", 2024-09-17T21:19:40.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076125800000000001 2024-09-17T21:19:40.780 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.780 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.780 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.780 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:496 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.780 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:19:40.780 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.565896280999993, 2024-09-17T21:19:40.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043732500000000001, 2024-09-17T21:19:40.781 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.781 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.781 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.781 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.781 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.781 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.781 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 496 2024-09-17T21:19:40.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.781 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:19:40.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198480+0000", 2024-09-17T21:19:40.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198481+0000", 2024-09-17T21:19:40.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-17T21:19:40.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198488+0000", 2024-09-17T21:19:40.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8499999999999996e-06 2024-09-17T21:19:40.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198507+0000", 2024-09-17T21:19:40.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9276999999999999e-05 2024-09-17T21:19:40.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198521+0000", 2024-09-17T21:19:40.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3376e-05 2024-09-17T21:19:40.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241339+0000", 2024-09-17T21:19:40.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042818119000000002 2024-09-17T21:19:40.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.241449+0000", 2024-09-17T21:19:40.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110209 2024-09-17T21:19:40.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242212+0000", 2024-09-17T21:19:40.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076329900000000005 2024-09-17T21:19:40.784 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.784 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.785 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.785 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:497 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.785 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:19:40.785 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.565815332, 2024-09-17T21:19:40.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044584278999999997, 2024-09-17T21:19:40.785 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.785 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.785 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.785 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.785 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.785 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.785 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 497 2024-09-17T21:19:40.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.786 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:19:40.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198561+0000", 2024-09-17T21:19:40.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198562+0000", 2024-09-17T21:19:40.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-17T21:19:40.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198569+0000", 2024-09-17T21:19:40.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.494e-06 2024-09-17T21:19:40.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198588+0000", 2024-09-17T21:19:40.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8955999999999999e-05 2024-09-17T21:19:40.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198621+0000", 2024-09-17T21:19:40.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3837000000000001e-05 2024-09-17T21:19:40.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242258+0000", 2024-09-17T21:19:40.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043636900999999999 2024-09-17T21:19:40.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242343+0000", 2024-09-17T21:19:40.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5048000000000003e-05 2024-09-17T21:19:40.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243145+0000", 2024-09-17T21:19:40.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00080167300000000003 2024-09-17T21:19:40.789 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.789 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.789 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.789 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:503 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.789 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:19:40.789 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.565730807999998, 2024-09-17T21:19:40.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044220582000000001, 2024-09-17T21:19:40.789 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.789 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.790 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.790 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.790 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.790 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.790 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 503 2024-09-17T21:19:40.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.790 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:19:40.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198645+0000", 2024-09-17T21:19:40.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198647+0000", 2024-09-17T21:19:40.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.483e-06 2024-09-17T21:19:40.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198653+0000", 2024-09-17T21:19:40.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4529999999999999e-06 2024-09-17T21:19:40.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198672+0000", 2024-09-17T21:19:40.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9193000000000001e-05 2024-09-17T21:19:40.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198685+0000", 2024-09-17T21:19:40.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.291e-05 2024-09-17T21:19:40.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:19:40.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043672141999999997 2024-09-17T21:19:40.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242716+0000", 2024-09-17T21:19:40.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00035901800000000001 2024-09-17T21:19:40.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242866+0000", 2024-09-17T21:19:40.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000149383 2024-09-17T21:19:40.793 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.793 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.793 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.794 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:504 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.794 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:19:40.794 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.565671015999996, 2024-09-17T21:19:40.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044611406999999999, 2024-09-17T21:19:40.794 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.794 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.794 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.794 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.794 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.794 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.794 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 504 2024-09-17T21:19:40.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.794 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:19:40.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198705+0000", 2024-09-17T21:19:40.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198706+0000", 2024-09-17T21:19:40.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:19:40.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198712+0000", 2024-09-17T21:19:40.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5770000000000001e-06 2024-09-17T21:19:40.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198729+0000", 2024-09-17T21:19:40.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6673000000000001e-05 2024-09-17T21:19:40.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.198742+0000", 2024-09-17T21:19:40.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3543000000000001e-05 2024-09-17T21:19:40.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242981+0000", 2024-09-17T21:19:40.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044238953999999997 2024-09-17T21:19:40.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243155+0000", 2024-09-17T21:19:40.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017401600000000001 2024-09-17T21:19:40.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243316+0000", 2024-09-17T21:19:40.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016137399999999999 2024-09-17T21:19:40.798 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.798 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.798 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.798 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:521 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.798 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:19:40.798 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.565123877000005, 2024-09-17T21:19:40.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045043685999999999, 2024-09-17T21:19:40.798 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.798 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.798 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.798 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.799 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.799 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.799 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 521 2024-09-17T21:19:40.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.799 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:19:40.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199252+0000", 2024-09-17T21:19:40.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199253+0000", 2024-09-17T21:19:40.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1400000000000001e-06 2024-09-17T21:19:40.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199258+0000", 2024-09-17T21:19:40.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1800000000000004e-06 2024-09-17T21:19:40.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199276+0000", 2024-09-17T21:19:40.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7686999999999999e-05 2024-09-17T21:19:40.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199289+0000", 2024-09-17T21:19:40.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3263e-05 2024-09-17T21:19:40.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243397+0000", 2024-09-17T21:19:40.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044107636999999998 2024-09-17T21:19:40.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243527+0000", 2024-09-17T21:19:40.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013038399999999999 2024-09-17T21:19:40.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244296+0000", 2024-09-17T21:19:40.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076839499999999995 2024-09-17T21:19:40.805 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.805 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.805 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.805 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:522 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.805 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:19:40.806 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.565068573999994, 2024-09-17T21:19:40.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046049870999999999, 2024-09-17T21:19:40.806 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.806 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.806 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.806 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.806 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.806 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.806 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 522 2024-09-17T21:19:40.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.806 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:19:40.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199307+0000", 2024-09-17T21:19:40.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199309+0000", 2024-09-17T21:19:40.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:19:40.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199315+0000", 2024-09-17T21:19:40.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2440000000000001e-06 2024-09-17T21:19:40.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199333+0000", 2024-09-17T21:19:40.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7886000000000001e-05 2024-09-17T21:19:40.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199379+0000", 2024-09-17T21:19:40.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6383e-05 2024-09-17T21:19:40.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244394+0000", 2024-09-17T21:19:40.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045014851000000002 2024-09-17T21:19:40.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244491+0000", 2024-09-17T21:19:40.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7245999999999997e-05 2024-09-17T21:19:40.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245357+0000", 2024-09-17T21:19:40.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00086626800000000001 2024-09-17T21:19:40.810 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.810 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.810 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.810 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:527 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.810 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:19:40.810 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.564971908000004, 2024-09-17T21:19:40.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045812522000000001, 2024-09-17T21:19:40.810 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.810 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.810 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.810 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.810 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.811 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.811 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 527 2024-09-17T21:19:40.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.811 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:19:40.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199404+0000", 2024-09-17T21:19:40.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199406+0000", 2024-09-17T21:19:40.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.367e-06 2024-09-17T21:19:40.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199412+0000", 2024-09-17T21:19:40.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:19:40.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199429+0000", 2024-09-17T21:19:40.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7396000000000001e-05 2024-09-17T21:19:40.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199442+0000", 2024-09-17T21:19:40.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3327e-05 2024-09-17T21:19:40.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244361+0000", 2024-09-17T21:19:40.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044918197 2024-09-17T21:19:40.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244482+0000", 2024-09-17T21:19:40.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121945 2024-09-17T21:19:40.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245217+0000", 2024-09-17T21:19:40.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00073415999999999998 2024-09-17T21:19:40.814 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.814 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.814 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.815 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:534 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.815 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:19:40.815 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.564913202, 2024-09-17T21:19:40.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046292505999999997, 2024-09-17T21:19:40.815 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.815 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.815 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.815 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.815 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.815 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.815 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:19:40.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.815 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:19:40.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199463+0000", 2024-09-17T21:19:40.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199464+0000", 2024-09-17T21:19:40.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:19:40.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199473+0000", 2024-09-17T21:19:40.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5629999999999992e-06 2024-09-17T21:19:40.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199492+0000", 2024-09-17T21:19:40.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9273999999999999e-05 2024-09-17T21:19:40.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199505+0000", 2024-09-17T21:19:40.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3443e-05 2024-09-17T21:19:40.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245357+0000", 2024-09-17T21:19:40.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045851714000000002 2024-09-17T21:19:40.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245550+0000", 2024-09-17T21:19:40.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019301600000000001 2024-09-17T21:19:40.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245755+0000", 2024-09-17T21:19:40.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205226 2024-09-17T21:19:40.819 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.819 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.820 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.820 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:535 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.820 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:19:40.820 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.564808646000003, 2024-09-17T21:19:40.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046637492000000003, 2024-09-17T21:19:40.820 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.820 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.820 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.820 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.820 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.820 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.820 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 535 2024-09-17T21:19:40.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.821 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.821 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.821 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:19:40.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199567+0000", 2024-09-17T21:19:40.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199568+0000", 2024-09-17T21:19:40.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.037e-06 2024-09-17T21:19:40.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199572+0000", 2024-09-17T21:19:40.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5729999999999999e-06 2024-09-17T21:19:40.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199585+0000", 2024-09-17T21:19:40.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.279e-05 2024-09-17T21:19:40.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.826 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.826 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.826 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199592+0000", 2024-09-17T21:19:40.826 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.61e-06 2024-09-17T21:19:40.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.826 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.826 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.826 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245873+0000", 2024-09-17T21:19:40.826 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046280494999999998 2024-09-17T21:19:40.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.826 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.826 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.826 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246018+0000", 2024-09-17T21:19:40.826 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000144697 2024-09-17T21:19:40.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.827 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.827 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246205+0000", 2024-09-17T21:19:40.827 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018729 2024-09-17T21:19:40.827 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.827 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.827 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.827 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:543 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.827 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:40.827 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.564713400000002, 2024-09-17T21:19:40.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047564097999999999, 2024-09-17T21:19:40.828 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.828 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.828 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.828 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.828 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.828 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.828 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 543 2024-09-17T21:19:40.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.828 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.828 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.828 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:40.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:40.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:40.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9699999999999995e-07 2024-09-17T21:19:40.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199677+0000", 2024-09-17T21:19:40.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:19:40.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199690+0000", 2024-09-17T21:19:40.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2977e-05 2024-09-17T21:19:40.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199705+0000", 2024-09-17T21:19:40.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4656e-05 2024-09-17T21:19:40.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246307+0000", 2024-09-17T21:19:40.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046601780000000002 2024-09-17T21:19:40.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246475+0000", 2024-09-17T21:19:40.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016834200000000001 2024-09-17T21:19:40.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.832 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247227+0000", 2024-09-17T21:19:40.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00075173299999999998 2024-09-17T21:19:40.832 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.832 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.832 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.832 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:544 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.832 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:40.832 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.564654990999998, 2024-09-17T21:19:40.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049283559999999997, 2024-09-17T21:19:40.832 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.832 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.833 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.833 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.833 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.833 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.833 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 544 2024-09-17T21:19:40.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.833 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:40.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:40.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199722+0000", 2024-09-17T21:19:40.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.08e-06 2024-09-17T21:19:40.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199738+0000", 2024-09-17T21:19:40.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5943999999999998e-05 2024-09-17T21:19:40.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.835 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.835 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199748+0000", 2024-09-17T21:19:40.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0193e-05 2024-09-17T21:19:40.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.835 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.835 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199756+0000", 2024-09-17T21:19:40.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4200000000000001e-06 2024-09-17T21:19:40.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.835 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.835 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247311+0000", 2024-09-17T21:19:40.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047555300000000002 2024-09-17T21:19:40.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247571+0000", 2024-09-17T21:19:40.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025996999999999999 2024-09-17T21:19:40.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249005+0000", 2024-09-17T21:19:40.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001433653 2024-09-17T21:19:40.836 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.836 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.837 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.837 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:549 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.837 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:40.837 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.564609367000003, 2024-09-17T21:19:40.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048464528999999999, 2024-09-17T21:19:40.837 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.837 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.837 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.837 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.837 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.837 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.838 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 549 2024-09-17T21:19:40.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.838 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:40.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:40.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:40.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:19:40.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199770+0000", 2024-09-17T21:19:40.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9969999999999999e-06 2024-09-17T21:19:40.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199780+0000", 2024-09-17T21:19:40.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0003000000000001e-05 2024-09-17T21:19:40.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199793+0000", 2024-09-17T21:19:40.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2074e-05 2024-09-17T21:19:40.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247289+0000", 2024-09-17T21:19:40.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047496691000000001 2024-09-17T21:19:40.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247437+0000", 2024-09-17T21:19:40.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014807899999999999 2024-09-17T21:19:40.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.841 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.841 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248231+0000", 2024-09-17T21:19:40.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00079390899999999998 2024-09-17T21:19:40.841 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.841 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.841 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.841 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:558 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.841 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:40.842 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.564568854000001, 2024-09-17T21:19:40.842 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049709821000000001, 2024-09-17T21:19:40.842 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.842 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.842 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.842 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.842 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.842 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.842 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 558 2024-09-17T21:19:40.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.842 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.842 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:40.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.843 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:40.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.843 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199809+0000", 2024-09-17T21:19:40.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.333e-06 2024-09-17T21:19:40.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199814+0000", 2024-09-17T21:19:40.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0930000000000002e-06 2024-09-17T21:19:40.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199831+0000", 2024-09-17T21:19:40.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7580000000000001e-05 2024-09-17T21:19:40.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199860+0000", 2024-09-17T21:19:40.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8609999999999999e-05 2024-09-17T21:19:40.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249106+0000", 2024-09-17T21:19:40.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049246545000000003 2024-09-17T21:19:40.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249274+0000", 2024-09-17T21:19:40.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016806699999999999 2024-09-17T21:19:40.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249517+0000", 2024-09-17T21:19:40.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024259300000000001 2024-09-17T21:19:40.845 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.846 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.846 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.846 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:559 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.846 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:40.846 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.564449491999994, 2024-09-17T21:19:40.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049990694000000002, 2024-09-17T21:19:40.846 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.846 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.846 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.846 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.846 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.846 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.847 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 559 2024-09-17T21:19:40.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.847 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.847 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.847 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.847 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:40.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.847 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.847 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.847 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:40.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.848 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.848 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.848 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199928+0000", 2024-09-17T21:19:40.848 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:19:40.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.848 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.848 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.848 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199935+0000", 2024-09-17T21:19:40.848 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5429999999999999e-06 2024-09-17T21:19:40.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.848 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.848 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.848 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199954+0000", 2024-09-17T21:19:40.848 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.912e-05 2024-09-17T21:19:40.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.849 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.849 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199969+0000", 2024-09-17T21:19:40.849 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5305999999999999e-05 2024-09-17T21:19:40.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.849 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.849 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249604+0000", 2024-09-17T21:19:40.849 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049634931 2024-09-17T21:19:40.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.849 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249738+0000", 2024-09-17T21:19:40.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013411599999999999 2024-09-17T21:19:40.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249917+0000", 2024-09-17T21:19:40.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179111 2024-09-17T21:19:40.850 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.850 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.850 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.850 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:560 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:40.851 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:40.851 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 74.564385075999994, 2024-09-17T21:19:40.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050173059999999998, 2024-09-17T21:19:40.851 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.851 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.851 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.851 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.851 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:40.851 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:40.851 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 560 2024-09-17T21:19:40.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.851 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.852 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.852 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:40.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.852 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.852 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:40.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.852 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.852 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199992+0000", 2024-09-17T21:19:40.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.517e-06 2024-09-17T21:19:40.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.853 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.853 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.853 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199998+0000", 2024-09-17T21:19:40.853 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1460000000000004e-06 2024-09-17T21:19:40.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.853 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.853 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.853 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:19:40.853 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7407000000000001e-05 2024-09-17T21:19:40.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.854 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.854 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.854 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200029+0000", 2024-09-17T21:19:40.854 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3499999999999999e-05 2024-09-17T21:19:40.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.854 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.854 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.854 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249968+0000", 2024-09-17T21:19:40.854 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049939799 2024-09-17T21:19:40.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.854 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.854 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250055+0000", 2024-09-17T21:19:40.855 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6731999999999996e-05 2024-09-17T21:19:40.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.855 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250164+0000", 2024-09-17T21:19:40.855 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108959 2024-09-17T21:19:40.855 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.855 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.855 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.855 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:19:40.856 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:40.856 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.954595367000003, 2024-09-17T21:19:40.856 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:19:40.856 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.856 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.856 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:40.856 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.856 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:19:40.856 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:19:40.856 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:19:40.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.856 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.857 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.857 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:40.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.857 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.857 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.857 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:40.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.857 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.857 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.857 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:19:40.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:19:40.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.858 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.858 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.858 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:19:40.858 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:19:40.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.858 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.858 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.858 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:19:40.858 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:19:40.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.858 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.858 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.858 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:19:40.858 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:19:40.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.859 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.859 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.859 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:19:40.859 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:19:40.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.859 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.859 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:40.859 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:19:40.859 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:19:40.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.859 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.859 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:19:40.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:19:40.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:19:40.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:19:40.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:19:40.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:19:40.860 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.860 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.861 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.861 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:19:40.861 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:40.861 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 47.399493581000002, 2024-09-17T21:19:40.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:19:40.861 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:40.861 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:40.861 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:40.861 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:40.861 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:19:40.861 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:40.862 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:19:40.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.862 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:40.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.862 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:40.862 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:40.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.862 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:40.862 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:40.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:40.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.863 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:40.863 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:19:40.863 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:19:40.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.863 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:40.863 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:19:40.863 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:19:40.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.863 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:40.863 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:19:40.863 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:19:40.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:40.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:19:40.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:19:40.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:19:40.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:19:40.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:40.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:19:40.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:19:40.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:40.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:19:40.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:19:40.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:40.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:19:40.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:19:40.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.866 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:40.866 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:19:40.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:19:40.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.866 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:40.866 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:19:40.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:19:40.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.866 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:40.866 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:19:40.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:19:40.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.867 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:40.867 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:19:40.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:19:40.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:40.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:40.867 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:40.867 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:19:40.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:19:40.867 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.867 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.867 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.867 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:40.867 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:40.868 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:40.868 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone22 from 0x5df6c00 (overlap 0x204ba00) to 0x0 2024-09-17T21:19:40.869 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:19:40.873 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:40.866+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:40.970 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:40.966+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:41.056 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:41.056 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:41.057 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:19:41.057 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:41.071 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:41.066+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:41.075 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:19:41.172 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:41.168+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:41.235 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:41.235 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:41.235 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:41.235 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:41.235 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:41.250 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:19:41.296 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:19:41.297 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:41.294+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:41.374 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:41.368+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:41.375 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:41.375 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:41.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:41.375 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:41.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.375 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:19:41.375 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:41.375 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.154251187, 2024-09-17T21:19:41.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:19:41.375 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:41.376 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:41.376 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:41.376 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:41.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:41.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:41.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:41.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:41.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:41.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:19:41.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:19:41.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:41.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:19:41.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:19:41.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:41.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:19:41.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:19:41.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:41.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:19:41.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:19:41.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:41.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:19:41.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:19:41.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:41.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:19:41.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:19:41.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:41.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:19:41.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:19:41.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:41.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:19:41.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:19:41.380 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.380 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:41.380 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.380 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:19:41.380 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:41.380 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 78.503950523, 2024-09-17T21:19:41.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:19:41.380 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:41.380 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:41.381 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:41.381 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:41.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:41.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:41.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:41.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:41.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:41.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:19:41.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:19:41.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:41.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:19:41.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:19:41.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:41.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:19:41.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:19:41.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:41.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:19:41.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:19:41.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:41.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:19:41.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:19:41.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:41.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:19:41.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:19:41.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:41.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:19:41.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:19:41.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:41.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:19:41.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:19:41.384 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.384 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:41.385 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.385 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:19:41.385 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:41.385 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 78.502685439999993, 2024-09-17T21:19:41.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:19:41.385 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:41.385 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:41.385 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:41.385 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:41.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:41.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:41.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:41.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:41.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:41.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:19:41.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:19:41.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:41.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:19:41.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:19:41.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:41.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:19:41.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:19:41.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:41.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:19:41.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:19:41.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:41.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:19:41.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:19:41.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:41.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:19:41.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:19:41.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:41.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:19:41.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:19:41.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:41.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:19:41.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:19:41.390 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.390 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:41.390 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.390 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:19:41.390 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:41.390 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 78.49290603, 2024-09-17T21:19:41.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:19:41.390 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:41.390 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:41.391 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:41.391 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:41.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:41.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:41.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:41.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:41.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:41.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:19:41.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:19:41.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:41.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:19:41.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:19:41.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:41.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:19:41.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:19:41.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:41.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:19:41.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:19:41.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:41.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:19:41.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:19:41.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:41.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:19:41.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:19:41.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:41.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:19:41.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:19:41.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:41.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:19:41.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:19:41.394 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.394 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:41.395 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.395 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:19:41.395 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:41.395 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 78.484183626999993, 2024-09-17T21:19:41.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:19:41.395 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:41.395 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:41.395 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:41.395 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:41.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:41.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:41.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:41.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:41.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:41.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:19:41.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:19:41.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:41.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:19:41.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:19:41.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:41.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:19:41.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:19:41.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:41.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:19:41.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:19:41.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:41.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:19:41.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:19:41.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:41.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:19:41.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:19:41.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:41.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:19:41.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:19:41.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:41.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:19:41.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:19:41.399 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.399 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:41.399 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.399 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:19:41.400 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:41.400 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 65.099093062999998, 2024-09-17T21:19:41.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:19:41.400 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:41.400 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:41.400 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:41.400 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:41.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:41.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:41.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:41.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:41.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:41.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:19:41.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:19:41.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:41.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:19:41.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:19:41.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:41.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:19:41.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:19:41.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:41.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:19:41.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:19:41.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:41.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:19:41.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:19:41.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:41.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:19:41.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:19:41.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:41.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:19:41.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:19:41.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:41.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:19:41.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:19:41.404 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.404 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:41.404 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.405 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1780 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'195, mlcod=69'190)", 2024-09-17T21:19:41.405 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:19:41.405 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.121237459, 2024-09-17T21:19:41.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075761769999999999, 2024-09-17T21:19:41.405 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:41.405 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:41.405 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:41.405 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:41.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:41.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:19:41.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:41.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:19:41.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:41.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250375+0000", 2024-09-17T21:19:41.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:19:41.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:41.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250383+0000", 2024-09-17T21:19:41.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7870000000000002e-06 2024-09-17T21:19:41.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:41.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250402+0000", 2024-09-17T21:19:41.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9346000000000001e-05 2024-09-17T21:19:41.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:41.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250413+0000", 2024-09-17T21:19:41.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.039e-05 2024-09-17T21:19:41.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:41.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251353+0000", 2024-09-17T21:19:41.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000940091 2024-09-17T21:19:41.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:41.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251407+0000", 2024-09-17T21:19:41.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4701999999999999e-05 2024-09-17T21:19:41.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:41.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257829+0000", 2024-09-17T21:19:41.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064212779999999999 2024-09-17T21:19:41.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:41.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257950+0000", 2024-09-17T21:19:41.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121623 2024-09-17T21:19:41.409 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.409 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:41.409 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.409 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1781 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'196, mlcod=69'190)", 2024-09-17T21:19:41.409 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:19:41.410 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.120154507999999, 2024-09-17T21:19:41.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065632939999999999, 2024-09-17T21:19:41.410 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:41.410 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:41.410 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:41.410 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:41.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:41.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:19:41.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:41.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:19:41.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:41.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251459+0000", 2024-09-17T21:19:41.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.624e-06 2024-09-17T21:19:41.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:41.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251471+0000", 2024-09-17T21:19:41.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2636000000000001e-05 2024-09-17T21:19:41.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:41.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251498+0000", 2024-09-17T21:19:41.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:19:41.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:41.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251515+0000", 2024-09-17T21:19:41.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6939000000000002e-05 2024-09-17T21:19:41.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:41.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252938+0000", 2024-09-17T21:19:41.413 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001423082 2024-09-17T21:19:41.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.413 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:41.413 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252999+0000", 2024-09-17T21:19:41.413 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1138999999999999e-05 2024-09-17T21:19:41.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.413 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:41.413 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257955+0000", 2024-09-17T21:19:41.413 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049559640000000002 2024-09-17T21:19:41.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:41.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.258020+0000", 2024-09-17T21:19:41.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5504000000000002e-05 2024-09-17T21:19:41.414 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.414 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:41.414 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.414 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1788 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'198, mlcod=69'191)", 2024-09-17T21:19:41.414 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:19:41.414 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.118129019000001, 2024-09-17T21:19:41.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073690370000000002, 2024-09-17T21:19:41.415 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:41.415 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:41.415 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:41.415 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:41.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:41.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:19:41.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:41.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:19:41.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:41.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253484+0000", 2024-09-17T21:19:41.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2470000000000001e-06 2024-09-17T21:19:41.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:41.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253492+0000", 2024-09-17T21:19:41.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7159999999999992e-06 2024-09-17T21:19:41.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:41.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253509+0000", 2024-09-17T21:19:41.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.666e-05 2024-09-17T21:19:41.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:41.417 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253522+0000", 2024-09-17T21:19:41.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2643000000000001e-05 2024-09-17T21:19:41.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:41.417 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254853+0000", 2024-09-17T21:19:41.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331238 2024-09-17T21:19:41.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:41.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254885+0000", 2024-09-17T21:19:41.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1562e-05 2024-09-17T21:19:41.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:41.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260788+0000", 2024-09-17T21:19:41.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059030749999999998 2024-09-17T21:19:41.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:41.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260852+0000", 2024-09-17T21:19:41.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3895999999999999e-05 2024-09-17T21:19:41.419 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.419 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:41.419 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.419 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5746 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'477, mlcod=71'473)", 2024-09-17T21:19:41.419 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:19:41.419 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 25.112701461, 2024-09-17T21:19:41.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074182709999999997, 2024-09-17T21:19:41.419 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:41.419 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:41.419 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:41.420 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:41.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:41.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:19:41.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:41.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:19:41.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:41.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258912+0000", 2024-09-17T21:19:41.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:19:41.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.421 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:41.421 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258926+0000", 2024-09-17T21:19:41.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4249e-05 2024-09-17T21:19:41.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.421 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:41.421 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258946+0000", 2024-09-17T21:19:41.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9879999999999999e-05 2024-09-17T21:19:41.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:41.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258956+0000", 2024-09-17T21:19:41.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:19:41.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:41.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259380+0000", 2024-09-17T21:19:41.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042337599999999999 2024-09-17T21:19:41.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:41.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259412+0000", 2024-09-17T21:19:41.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:19:41.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:41.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266191+0000", 2024-09-17T21:19:41.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067793269999999999 2024-09-17T21:19:41.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:41.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266328+0000", 2024-09-17T21:19:41.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013690999999999999 2024-09-17T21:19:41.423 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.423 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:41.424 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.424 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5747 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'478, mlcod=71'473)", 2024-09-17T21:19:41.424 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:19:41.424 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 25.111995111999999, 2024-09-17T21:19:41.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068779440000000004, 2024-09-17T21:19:41.424 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:41.424 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:41.424 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:41.424 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:41.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:41.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:19:41.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:41.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:19:41.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:41.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259618+0000", 2024-09-17T21:19:41.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6199999999999999e-06 2024-09-17T21:19:41.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.426 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:41.426 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259630+0000", 2024-09-17T21:19:41.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1884e-05 2024-09-17T21:19:41.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.426 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:41.426 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259657+0000", 2024-09-17T21:19:41.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7339e-05 2024-09-17T21:19:41.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.426 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:41.426 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259677+0000", 2024-09-17T21:19:41.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0156000000000001e-05 2024-09-17T21:19:41.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:41.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260521+0000", 2024-09-17T21:19:41.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084312899999999997 2024-09-17T21:19:41.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:41.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260585+0000", 2024-09-17T21:19:41.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4558000000000001e-05 2024-09-17T21:19:41.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:41.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266336+0000", 2024-09-17T21:19:41.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057505020000000002 2024-09-17T21:19:41.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:41.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266494+0000", 2024-09-17T21:19:41.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158756 2024-09-17T21:19:41.428 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.428 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:41.428 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.428 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5750 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'479, mlcod=71'473)", 2024-09-17T21:19:41.428 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:19:41.428 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 25.111282919000001, 2024-09-17T21:19:41.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006302874, 2024-09-17T21:19:41.429 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:41.429 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:41.429 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:41.429 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:41.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:41.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:19:41.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:41.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:19:41.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:41.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260330+0000", 2024-09-17T21:19:41.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:19:41.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:41.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260343+0000", 2024-09-17T21:19:41.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2862e-05 2024-09-17T21:19:41.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.431 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:41.431 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260367+0000", 2024-09-17T21:19:41.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3723e-05 2024-09-17T21:19:41.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.431 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:41.431 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260386+0000", 2024-09-17T21:19:41.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8879999999999999e-05 2024-09-17T21:19:41.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.431 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:41.431 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261579+0000", 2024-09-17T21:19:41.432 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001193431 2024-09-17T21:19:41.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:41.432 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261608+0000", 2024-09-17T21:19:41.432 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8858999999999999e-05 2024-09-17T21:19:41.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:41.432 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266505+0000", 2024-09-17T21:19:41.432 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004896999 2024-09-17T21:19:41.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:41.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266631+0000", 2024-09-17T21:19:41.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012655300000000001 2024-09-17T21:19:41.433 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.433 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:41.433 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.434 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5751 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'480, mlcod=71'473)", 2024-09-17T21:19:41.434 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:19:41.434 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 25.110575706999999, 2024-09-17T21:19:41.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075756299999999999, 2024-09-17T21:19:41.434 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:41.434 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:41.434 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:41.434 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:41.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:41.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:19:41.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:41.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:19:41.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:41.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261037+0000", 2024-09-17T21:19:41.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:19:41.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:41.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261049+0000", 2024-09-17T21:19:41.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1063999999999999e-05 2024-09-17T21:19:41.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.436 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:41.436 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261072+0000", 2024-09-17T21:19:41.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3102e-05 2024-09-17T21:19:41.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.436 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:41.436 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261133+0000", 2024-09-17T21:19:41.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1276000000000001e-05 2024-09-17T21:19:41.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.436 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:41.436 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:19:41.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015334319999999999 2024-09-17T21:19:41.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:41.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262702+0000", 2024-09-17T21:19:41.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5281999999999999e-05 2024-09-17T21:19:41.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:41.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268527+0000", 2024-09-17T21:19:41.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058250389999999997 2024-09-17T21:19:41.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:41.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268611+0000", 2024-09-17T21:19:41.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4724999999999999e-05 2024-09-17T21:19:41.438 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.438 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:41.438 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.438 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5756 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'481, mlcod=71'473)", 2024-09-17T21:19:41.438 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:19:41.438 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 25.109714851, 2024-09-17T21:19:41.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006772386, 2024-09-17T21:19:41.438 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:41.438 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:41.439 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:41.439 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:41.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:41.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:19:41.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:41.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:19:41.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:41.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261899+0000", 2024-09-17T21:19:41.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.903e-06 2024-09-17T21:19:41.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:41.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261912+0000", 2024-09-17T21:19:41.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-05 2024-09-17T21:19:41.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:41.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261938+0000", 2024-09-17T21:19:41.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6098999999999999e-05 2024-09-17T21:19:41.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.441 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:41.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261955+0000", 2024-09-17T21:19:41.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.696e-05 2024-09-17T21:19:41.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.441 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:41.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263845+0000", 2024-09-17T21:19:41.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001890187 2024-09-17T21:19:41.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.441 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:41.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263907+0000", 2024-09-17T21:19:41.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1600999999999995e-05 2024-09-17T21:19:41.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:41.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268615+0000", 2024-09-17T21:19:41.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047086070000000001 2024-09-17T21:19:41.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:41.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268669+0000", 2024-09-17T21:19:41.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3588999999999998e-05 2024-09-17T21:19:41.442 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.442 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:41.443 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.443 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4978.0:598 3.9 e80/23 3:9701cc17:::rbd_data.1152724ff2a3.0000000000000019:head v 80'202, mlcod=80'198)", 2024-09-17T21:19:41.443 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:19:41.443 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.608140468, 2024-09-17T21:19:41.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066473340000000004, 2024-09-17T21:19:41.443 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:41.443 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:41.443 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:41.443 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:41.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.443 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:41.443 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:19:41.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:41.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:19:41.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:41.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763472+0000", 2024-09-17T21:19:41.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6899999999999997e-07 2024-09-17T21:19:41.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:41.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763478+0000", 2024-09-17T21:19:41.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:19:41.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:41.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763489+0000", 2024-09-17T21:19:41.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0628999999999999e-05 2024-09-17T21:19:41.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:41.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763495+0000", 2024-09-17T21:19:41.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6649999999999997e-06 2024-09-17T21:19:41.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:41.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763907+0000", 2024-09-17T21:19:41.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041163899999999999 2024-09-17T21:19:41.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:41.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763976+0000", 2024-09-17T21:19:41.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8534999999999993e-05 2024-09-17T21:19:41.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:41.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.769900+0000", 2024-09-17T21:19:41.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059243270000000001 2024-09-17T21:19:41.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:41.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.770118+0000", 2024-09-17T21:19:41.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021842500000000001 2024-09-17T21:19:41.447 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.447 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:41.447 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.448 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:468 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'94, mlcod=89'87)", 2024-09-17T21:19:41.448 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:19:41.448 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.685395169, 2024-09-17T21:19:41.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066528790000000004, 2024-09-17T21:19:41.448 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:41.448 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:41.448 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:41.448 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:41.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:41.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:19:41.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:41.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:19:41.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:41.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686218+0000", 2024-09-17T21:19:41.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3060000000000001e-06 2024-09-17T21:19:41.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:41.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686256+0000", 2024-09-17T21:19:41.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8333e-05 2024-09-17T21:19:41.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:41.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686272+0000", 2024-09-17T21:19:41.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:19:41.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:41.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686283+0000", 2024-09-17T21:19:41.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0903e-05 2024-09-17T21:19:41.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:41.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686682+0000", 2024-09-17T21:19:41.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039935099999999999 2024-09-17T21:19:41.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:41.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686785+0000", 2024-09-17T21:19:41.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010275199999999999 2024-09-17T21:19:41.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:41.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692816+0000", 2024-09-17T21:19:41.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060317579999999999 2024-09-17T21:19:41.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:41.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692869+0000", 2024-09-17T21:19:41.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2775999999999998e-05 2024-09-17T21:19:41.452 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.452 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:41.452 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.452 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:755 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'127, mlcod=89'122)", 2024-09-17T21:19:41.452 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:19:41.452 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.548939955, 2024-09-17T21:19:41.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084014080000000008, 2024-09-17T21:19:41.453 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:41.453 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:41.453 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:41.453 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:41.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:41.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:19:41.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:41.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:19:41.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.454 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:41.454 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:19:41.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:19:41.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.454 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:41.454 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822683+0000", 2024-09-17T21:19:41.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0674e-05 2024-09-17T21:19:41.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.454 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:41.454 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822698+0000", 2024-09-17T21:19:41.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5189e-05 2024-09-17T21:19:41.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.455 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:41.455 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822708+0000", 2024-09-17T21:19:41.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9119999999999993e-06 2024-09-17T21:19:41.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.455 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:41.455 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822748+0000", 2024-09-17T21:19:41.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9727000000000001e-05 2024-09-17T21:19:41.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.455 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:41.456 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822759+0000", 2024-09-17T21:19:41.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0865e-05 2024-09-17T21:19:41.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:41.456 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831034+0000", 2024-09-17T21:19:41.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082750710000000002 2024-09-17T21:19:41.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:41.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831073+0000", 2024-09-17T21:19:41.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9063999999999997e-05 2024-09-17T21:19:41.457 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.457 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:41.457 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.457 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:756 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'128, mlcod=89'124)", 2024-09-17T21:19:41.457 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:19:41.457 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.5470173229999999, 2024-09-17T21:19:41.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0076300270000000002, 2024-09-17T21:19:41.457 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:41.457 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:41.457 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:41.458 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:41.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:41.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:19:41.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:41.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:19:41.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:41.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824595+0000", 2024-09-17T21:19:41.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-17T21:19:41.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:41.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824600+0000", 2024-09-17T21:19:41.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:19:41.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:41.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824612+0000", 2024-09-17T21:19:41.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.181e-05 2024-09-17T21:19:41.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.460 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:41.460 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824619+0000", 2024-09-17T21:19:41.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:19:41.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.460 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:41.460 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824671+0000", 2024-09-17T21:19:41.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2262999999999998e-05 2024-09-17T21:19:41.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.460 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:41.461 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824695+0000", 2024-09-17T21:19:41.461 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3883000000000002e-05 2024-09-17T21:19:41.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.461 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:41.461 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832185+0000", 2024-09-17T21:19:41.461 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007490048 2024-09-17T21:19:41.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.461 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:41.461 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832224+0000", 2024-09-17T21:19:41.461 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8957e-05 2024-09-17T21:19:41.461 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.461 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:41.462 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.462 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2617 3.d e90/23 3:b0143718:::rbd_data.1152a8ae2019.0000000000000016:head v 90'249, mlcod=90'246)", 2024-09-17T21:19:41.462 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:19:41.462 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.52716632299999999, 2024-09-17T21:19:41.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063715559999999996, 2024-09-17T21:19:41.462 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:41.462 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:41.462 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:41.462 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:41.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:41.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:19:41.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:41.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:19:41.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:41.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844455+0000", 2024-09-17T21:19:41.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4259999999999992e-06 2024-09-17T21:19:41.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:41.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844470+0000", 2024-09-17T21:19:41.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5386000000000001e-05 2024-09-17T21:19:41.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:41.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844499+0000", 2024-09-17T21:19:41.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.883e-05 2024-09-17T21:19:41.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:41.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844518+0000", 2024-09-17T21:19:41.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8830000000000001e-05 2024-09-17T21:19:41.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:41.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844649+0000", 2024-09-17T21:19:41.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013143399999999999 2024-09-17T21:19:41.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:41.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844702+0000", 2024-09-17T21:19:41.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2876e-05 2024-09-17T21:19:41.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:41.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850696+0000", 2024-09-17T21:19:41.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059943729999999999 2024-09-17T21:19:41.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.466 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:41.466 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850817+0000", 2024-09-17T21:19:41.466 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120401 2024-09-17T21:19:41.466 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.466 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:41.466 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.466 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2635 3.d e90/23 3:b3d1e920:::rbd_data.1152a8ae2019.000000000000001f:head v 90'252, mlcod=90'246)", 2024-09-17T21:19:41.466 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:19:41.466 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.52445825300000004, 2024-09-17T21:19:41.466 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0077265190000000003, 2024-09-17T21:19:41.467 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:41.467 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:41.467 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:41.467 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:41.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.467 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:41.467 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:19:41.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.467 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:41.467 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:19:41.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:41.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.468 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:41.468 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847155+0000", 2024-09-17T21:19:41.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.544e-06 2024-09-17T21:19:41.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.468 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:41.468 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847165+0000", 2024-09-17T21:19:41.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.043e-05 2024-09-17T21:19:41.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:41.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847189+0000", 2024-09-17T21:19:41.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3659000000000002e-05 2024-09-17T21:19:41.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:41.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847210+0000", 2024-09-17T21:19:41.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1163e-05 2024-09-17T21:19:41.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:41.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848856+0000", 2024-09-17T21:19:41.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001645521 2024-09-17T21:19:41.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:41.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848905+0000", 2024-09-17T21:19:41.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9048999999999997e-05 2024-09-17T21:19:41.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:41.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854770+0000", 2024-09-17T21:19:41.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058652879999999998 2024-09-17T21:19:41.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:41.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:41.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:41.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854880+0000", 2024-09-17T21:19:41.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010986500000000001 2024-09-17T21:19:41.471 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.471 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:41.471 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.471 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:41.471 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:41.471 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:41.472 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:19:41.473 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:41.473 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:41.473 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:41.474 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:41.474 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:41.474 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:41.475 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:41.472+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:41.488 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:41.488 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:41.507 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:41.507 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:41.524 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:41.524 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:41.576 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:41.572+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:41.609 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:41.609 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:41.622 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:19:41.677 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:41.674+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:41.737 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-17T21:19:41.778 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:41.772+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:41.789 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:41.789 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:41.803 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:19:41.879 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:41.874+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:41.978 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:41.978 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:41.980 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:41.974+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:41.993 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:19:42.080 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:42.078+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:42.131 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:42.131 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:42.147 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:19:42.190 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:42.188+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:42.278 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:42.278 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:42.283 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:42.278+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:42.293 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:19:42.383 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:42.380+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:42.466 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:42.467 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:42.482 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:19:42.484 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:42.482+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:42.585 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:42.580+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:42.655 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:42.655 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:42.671 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:19:42.672 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:19:42.714 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:42.712+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:42.787 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:42.784+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:42.850 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:42.850 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:42.865 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:19:42.888 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:42.886+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:42.989 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:42.986+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:43.033 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:43.034 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:43.048 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:19:43.092 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:43.090+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:43.173 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:43.173 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:43.188 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:19:43.209 INFO:teuthology.orchestra.run.smithi184.stdout:3913 punch from 0x5783800 to 0x578f400, (0xbc00 bytes) 2024-09-17T21:19:43.233 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:43.230+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:43.271 INFO:teuthology.orchestra.run.smithi184.stdout:3914 trunc from 0x5df6c00 to 0xdda6a00 2024-09-17T21:19:43.291 INFO:teuthology.orchestra.run.smithi184.stdout:3915 write 0x4fe9c00 thru 0x4ff6342 (0xc743 bytes) 2024-09-17T21:19:43.292 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:43.288+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:43.294 INFO:teuthology.orchestra.run.smithi184.stdout:3916 write 0x9998e00 thru 0x999bba8 (0x2da9 bytes) 2024-09-17T21:19:43.302 INFO:teuthology.orchestra.run.smithi184.stdout:3917 trunc from 0xdda6a00 to 0x8c08a00 2024-09-17T21:19:43.330 INFO:teuthology.orchestra.run.smithi184.stdout:3919 clone 24 order 23 su 65536 sc 9 2024-09-17T21:19:43.363 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:43.363 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:43.377 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:19:43.393 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:43.388+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:43.495 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:43.488+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:43.497 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:43.497 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:43.511 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:19:43.542 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:19:43.595 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:43.592+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:43.635 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:43.635 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:43.650 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:19:43.671 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-17T21:19:43.696 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:43.690+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:43.713 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:43.713 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:43.732 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:43.732 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:43.751 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:43.769 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:19:43.769 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:19:43.769 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [1, 2, 3, 4, 5, 0] 2024-09-17T21:19:43.770 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-17T21:19:43.772 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:43.772 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:43.780 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:43.781 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:43.781 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:43.781 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:43.781 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:43.796 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:43.790+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:43.801 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:19:43.805 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:43.806 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:43.828 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:43.828 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:43.883 INFO:teuthology.orchestra.run.smithi184.stdout:leaving image image_client.0-clone23 intact 2024-09-17T21:19:43.897 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:43.892+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:43.924 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:43.924 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:43.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:43.924 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:43.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.924 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:543 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:43.924 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:43.924 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 77.722269889000003, 2024-09-17T21:19:43.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047564097999999999, 2024-09-17T21:19:43.925 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:43.925 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:43.925 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:43.925 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:43.925 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:43.925 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:43.925 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 543 2024-09-17T21:19:43.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.925 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:43.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:43.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:43.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:43.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:43.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:43.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:43.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9699999999999995e-07 2024-09-17T21:19:43.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:43.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199677+0000", 2024-09-17T21:19:43.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:19:43.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:43.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199690+0000", 2024-09-17T21:19:43.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2977e-05 2024-09-17T21:19:43.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:43.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199705+0000", 2024-09-17T21:19:43.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4656e-05 2024-09-17T21:19:43.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:43.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246307+0000", 2024-09-17T21:19:43.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046601780000000002 2024-09-17T21:19:43.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:43.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246475+0000", 2024-09-17T21:19:43.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016834200000000001 2024-09-17T21:19:43.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:43.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247227+0000", 2024-09-17T21:19:43.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00075173299999999998 2024-09-17T21:19:43.929 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.929 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:43.929 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.929 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:544 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:43.929 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:43.929 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 77.722211479999999, 2024-09-17T21:19:43.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049283559999999997, 2024-09-17T21:19:43.929 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:43.929 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:43.930 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:43.930 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:43.930 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:43.930 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:43.930 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 544 2024-09-17T21:19:43.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.930 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:43.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:43.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:43.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:43.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:43.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:43.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199722+0000", 2024-09-17T21:19:43.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.08e-06 2024-09-17T21:19:43.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:43.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199738+0000", 2024-09-17T21:19:43.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5943999999999998e-05 2024-09-17T21:19:43.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:43.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199748+0000", 2024-09-17T21:19:43.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0193e-05 2024-09-17T21:19:43.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:43.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199756+0000", 2024-09-17T21:19:43.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4200000000000001e-06 2024-09-17T21:19:43.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:43.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247311+0000", 2024-09-17T21:19:43.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047555300000000002 2024-09-17T21:19:43.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:43.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247571+0000", 2024-09-17T21:19:43.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025996999999999999 2024-09-17T21:19:43.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:43.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249005+0000", 2024-09-17T21:19:43.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001433653 2024-09-17T21:19:43.933 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.933 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:43.933 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.934 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:549 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:43.934 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:43.934 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 77.722165856000004, 2024-09-17T21:19:43.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048464528999999999, 2024-09-17T21:19:43.934 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:43.934 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:43.934 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:43.934 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:43.934 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:43.934 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:43.934 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 549 2024-09-17T21:19:43.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.934 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:43.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:43.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:43.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:43.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:43.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:43.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:43.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:19:43.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:43.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199770+0000", 2024-09-17T21:19:43.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9969999999999999e-06 2024-09-17T21:19:43.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:43.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199780+0000", 2024-09-17T21:19:43.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0003000000000001e-05 2024-09-17T21:19:43.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:43.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199793+0000", 2024-09-17T21:19:43.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2074e-05 2024-09-17T21:19:43.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:43.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247289+0000", 2024-09-17T21:19:43.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047496691000000001 2024-09-17T21:19:43.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:43.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247437+0000", 2024-09-17T21:19:43.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014807899999999999 2024-09-17T21:19:43.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:43.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248231+0000", 2024-09-17T21:19:43.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00079390899999999998 2024-09-17T21:19:43.938 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.938 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:43.938 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.938 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:558 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:43.938 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:43.938 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 77.722125343000002, 2024-09-17T21:19:43.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049709821000000001, 2024-09-17T21:19:43.938 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:43.938 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:43.938 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:43.939 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:43.939 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:43.939 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:43.939 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 558 2024-09-17T21:19:43.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.939 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:43.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:43.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:43.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:43.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:43.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:43.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199809+0000", 2024-09-17T21:19:43.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.333e-06 2024-09-17T21:19:43.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:43.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199814+0000", 2024-09-17T21:19:43.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0930000000000002e-06 2024-09-17T21:19:43.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:43.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199831+0000", 2024-09-17T21:19:43.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7580000000000001e-05 2024-09-17T21:19:43.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:43.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199860+0000", 2024-09-17T21:19:43.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8609999999999999e-05 2024-09-17T21:19:43.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:43.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249106+0000", 2024-09-17T21:19:43.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049246545000000003 2024-09-17T21:19:43.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:43.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249274+0000", 2024-09-17T21:19:43.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016806699999999999 2024-09-17T21:19:43.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:43.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249517+0000", 2024-09-17T21:19:43.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024259300000000001 2024-09-17T21:19:43.942 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.942 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:43.942 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.943 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:559 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:43.943 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:43.943 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 77.722005980999995, 2024-09-17T21:19:43.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049990694000000002, 2024-09-17T21:19:43.943 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:43.943 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:43.943 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:43.943 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:43.943 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:43.943 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:43.943 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 559 2024-09-17T21:19:43.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.944 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:43.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:43.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:43.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:43.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:43.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:43.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199928+0000", 2024-09-17T21:19:43.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:19:43.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:43.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199935+0000", 2024-09-17T21:19:43.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5429999999999999e-06 2024-09-17T21:19:43.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:43.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199954+0000", 2024-09-17T21:19:43.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.912e-05 2024-09-17T21:19:43.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:43.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199969+0000", 2024-09-17T21:19:43.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5305999999999999e-05 2024-09-17T21:19:43.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:43.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249604+0000", 2024-09-17T21:19:43.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049634931 2024-09-17T21:19:43.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:43.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249738+0000", 2024-09-17T21:19:43.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013411599999999999 2024-09-17T21:19:43.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:43.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249917+0000", 2024-09-17T21:19:43.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179111 2024-09-17T21:19:43.947 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.947 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:43.947 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.947 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:560 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:43.947 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:43.947 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 77.721941564999995, 2024-09-17T21:19:43.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050173059999999998, 2024-09-17T21:19:43.948 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:43.948 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:43.948 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:43.948 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:43.948 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:43.948 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:43.948 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 560 2024-09-17T21:19:43.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.948 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:43.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:43.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:43.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:43.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:43.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:43.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199992+0000", 2024-09-17T21:19:43.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.517e-06 2024-09-17T21:19:43.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:43.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199998+0000", 2024-09-17T21:19:43.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1460000000000004e-06 2024-09-17T21:19:43.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:43.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:19:43.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7407000000000001e-05 2024-09-17T21:19:43.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:43.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200029+0000", 2024-09-17T21:19:43.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3499999999999999e-05 2024-09-17T21:19:43.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:43.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249968+0000", 2024-09-17T21:19:43.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049939799 2024-09-17T21:19:43.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:43.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250055+0000", 2024-09-17T21:19:43.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6731999999999996e-05 2024-09-17T21:19:43.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:43.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250164+0000", 2024-09-17T21:19:43.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108959 2024-09-17T21:19:43.951 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.951 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:43.951 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.952 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:19:43.952 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:43.952 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.112151855999997, 2024-09-17T21:19:43.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:19:43.952 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:43.952 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:43.952 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:43.952 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:43.952 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:19:43.952 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:19:43.952 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:19:43.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.953 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:43.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:43.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:43.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:43.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:43.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:43.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:19:43.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:19:43.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:43.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:19:43.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:19:43.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:43.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:19:43.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:19:43.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:43.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:19:43.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:19:43.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:43.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:19:43.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:19:43.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:43.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:19:43.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:19:43.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:43.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:19:43.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:19:43.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:43.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:19:43.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:19:43.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:43.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:19:43.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:19:43.956 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.956 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:43.956 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.957 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:19:43.957 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:43.957 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.557050070000003, 2024-09-17T21:19:43.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:19:43.957 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:43.957 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:43.957 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:43.957 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:43.957 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:19:43.957 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:43.957 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:19:43.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.958 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:43.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:43.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:43.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:43.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:43.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:43.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:19:43.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:19:43.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:43.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:19:43.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:19:43.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:43.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:19:43.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:19:43.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:43.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:19:43.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:19:43.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:43.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:19:43.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:19:43.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:43.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:19:43.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:19:43.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:43.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:19:43.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:19:43.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:43.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:19:43.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:19:43.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:43.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:19:43.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:19:43.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:43.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:19:43.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:19:43.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:43.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:19:43.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:19:43.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:43.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:19:43.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:19:43.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:43.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:19:43.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:19:43.963 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.963 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:43.963 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.963 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3777 3.c 3:302136a0:::rbd_data.1152a8ae2019.000000000000004b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:43.963 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.902125+0000", 2024-09-17T21:19:43.963 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.019807444, 2024-09-17T21:19:43.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048610548000000003, 2024-09-17T21:19:43.963 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:43.963 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:43.963 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:43.963 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:43.964 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:43.964 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:43.964 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3777 2024-09-17T21:19:43.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.964 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:43.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:43.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902125+0000", 2024-09-17T21:19:43.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:43.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902125+0000", 2024-09-17T21:19:43.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:43.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902126+0000", 2024-09-17T21:19:43.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:19:43.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:43.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902138+0000", 2024-09-17T21:19:43.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1994000000000001e-05 2024-09-17T21:19:43.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:43.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902144+0000", 2024-09-17T21:19:43.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2820000000000002e-06 2024-09-17T21:19:43.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:43.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902149+0000", 2024-09-17T21:19:43.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.135e-06 2024-09-17T21:19:43.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:43.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.942910+0000", 2024-09-17T21:19:43.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.040761190000000003 2024-09-17T21:19:43.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:43.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.942986+0000", 2024-09-17T21:19:43.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6083999999999999e-05 2024-09-17T21:19:43.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:43.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.943319+0000", 2024-09-17T21:19:43.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00033264500000000001 2024-09-17T21:19:43.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:43.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944690+0000", 2024-09-17T21:19:43.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0013710549999999999 2024-09-17T21:19:43.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:43.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.950637+0000", 2024-09-17T21:19:43.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059473190000000004 2024-09-17T21:19:43.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:43.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.950696+0000", 2024-09-17T21:19:43.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8449999999999998e-05 2024-09-17T21:19:43.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:43.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.950736+0000", 2024-09-17T21:19:43.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9971000000000001e-05 2024-09-17T21:19:43.968 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.968 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:43.968 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.969 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3778 3.c 3:375c1b23:::rbd_data.1152a8ae2019.0000000000000052:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:43.969 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.902156+0000", 2024-09-17T21:19:43.969 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.0197760900000001, 2024-09-17T21:19:43.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049052074000000001, 2024-09-17T21:19:43.969 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:43.969 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:43.969 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:43.969 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:43.969 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:43.969 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:43.969 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3778 2024-09-17T21:19:43.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.969 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:43.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:43.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902156+0000", 2024-09-17T21:19:43.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:43.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902156+0000", 2024-09-17T21:19:43.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:43.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902157+0000", 2024-09-17T21:19:43.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5299999999999999e-07 2024-09-17T21:19:43.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:43.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902160+0000", 2024-09-17T21:19:43.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6359999999999998e-06 2024-09-17T21:19:43.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:43.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902165+0000", 2024-09-17T21:19:43.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9440000000000002e-06 2024-09-17T21:19:43.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:43.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902170+0000", 2024-09-17T21:19:43.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:19:43.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:43.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.943704+0000", 2024-09-17T21:19:43.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041533516999999999 2024-09-17T21:19:43.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:43.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.943760+0000", 2024-09-17T21:19:43.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6552000000000003e-05 2024-09-17T21:19:43.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:43.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.943879+0000", 2024-09-17T21:19:43.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118347 2024-09-17T21:19:43.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:43.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944733+0000", 2024-09-17T21:19:43.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00085437299999999996 2024-09-17T21:19:43.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:43.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951158+0000", 2024-09-17T21:19:43.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064243520000000004 2024-09-17T21:19:43.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:43.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951184+0000", 2024-09-17T21:19:43.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6913000000000001e-05 2024-09-17T21:19:43.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:43.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951209+0000", 2024-09-17T21:19:43.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4104000000000002e-05 2024-09-17T21:19:43.974 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.974 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:43.974 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.974 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3785 3.c 3:3414a45b:::rbd_data.1152a8ae2019.0000000000000074:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:43.974 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.902292+0000", 2024-09-17T21:19:43.974 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.0196405670000002, 2024-09-17T21:19:43.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049058004000000002, 2024-09-17T21:19:43.974 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:43.974 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:43.974 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:43.975 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:43.975 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:43.975 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:43.975 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3785 2024-09-17T21:19:43.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.975 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:43.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:43.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902292+0000", 2024-09-17T21:19:43.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:43.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902292+0000", 2024-09-17T21:19:43.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:43.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902292+0000", 2024-09-17T21:19:43.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:19:43.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:43.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902299+0000", 2024-09-17T21:19:43.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1210000000000001e-06 2024-09-17T21:19:43.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:43.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902305+0000", 2024-09-17T21:19:43.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6069999999999998e-06 2024-09-17T21:19:43.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:43.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902310+0000", 2024-09-17T21:19:43.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7509999999999999e-06 2024-09-17T21:19:43.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:43.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944093+0000", 2024-09-17T21:19:43.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041783582999999999 2024-09-17T21:19:43.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:43.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944159+0000", 2024-09-17T21:19:43.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5524999999999994e-05 2024-09-17T21:19:43.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:43.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944281+0000", 2024-09-17T21:19:43.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121573 2024-09-17T21:19:43.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:43.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.945107+0000", 2024-09-17T21:19:43.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00082616700000000005 2024-09-17T21:19:43.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:43.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951260+0000", 2024-09-17T21:19:43.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061530789999999997 2024-09-17T21:19:43.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:43.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951311+0000", 2024-09-17T21:19:43.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1678000000000002e-05 2024-09-17T21:19:43.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:43.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951350+0000", 2024-09-17T21:19:43.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8525000000000001e-05 2024-09-17T21:19:43.980 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.980 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:43.980 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.980 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3880 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:43.980 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.912848+0000", 2024-09-17T21:19:43.980 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.009084455, 2024-09-17T21:19:43.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046791626000000003, 2024-09-17T21:19:43.980 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:43.980 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:43.980 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:43.981 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:43.981 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:43.981 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:43.981 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3880 2024-09-17T21:19:43.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.981 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:43.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:43.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912848+0000", 2024-09-17T21:19:43.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:43.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912848+0000", 2024-09-17T21:19:43.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:43.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912849+0000", 2024-09-17T21:19:43.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2600000000000002e-07 2024-09-17T21:19:43.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:43.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912851+0000", 2024-09-17T21:19:43.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.475e-06 2024-09-17T21:19:43.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:43.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912857+0000", 2024-09-17T21:19:43.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1009999999999997e-06 2024-09-17T21:19:43.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:43.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912862+0000", 2024-09-17T21:19:43.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2639999999999999e-06 2024-09-17T21:19:43.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:43.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959555+0000", 2024-09-17T21:19:43.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046692589 2024-09-17T21:19:43.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:43.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959586+0000", 2024-09-17T21:19:43.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0978000000000001e-05 2024-09-17T21:19:43.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:43.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959640+0000", 2024-09-17T21:19:43.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3693000000000002e-05 2024-09-17T21:19:43.984 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.984 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:43.984 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.984 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3881 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:43.985 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.912869+0000", 2024-09-17T21:19:43.985 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.0090633320000002, 2024-09-17T21:19:43.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046894383999999997, 2024-09-17T21:19:43.985 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:43.985 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:43.985 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:43.985 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:43.985 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:43.985 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:43.985 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3881 2024-09-17T21:19:43.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.985 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:43.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:43.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912869+0000", 2024-09-17T21:19:43.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:43.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912869+0000", 2024-09-17T21:19:43.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:43.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912870+0000", 2024-09-17T21:19:43.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:19:43.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:43.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913014+0000", 2024-09-17T21:19:43.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000144473 2024-09-17T21:19:43.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:43.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913025+0000", 2024-09-17T21:19:43.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1341e-05 2024-09-17T21:19:43.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:43.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913033+0000", 2024-09-17T21:19:43.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.17e-06 2024-09-17T21:19:43.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:43.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959677+0000", 2024-09-17T21:19:43.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046644536 2024-09-17T21:19:43.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:43.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959709+0000", 2024-09-17T21:19:43.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2348000000000003e-05 2024-09-17T21:19:43.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:43.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959764+0000", 2024-09-17T21:19:43.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4120999999999999e-05 2024-09-17T21:19:43.988 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.989 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:43.989 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.989 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:14003 3.4 3.4f4be604 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:43.989 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.912905+0000", 2024-09-17T21:19:43.989 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.009027492, 2024-09-17T21:19:43.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046758252, 2024-09-17T21:19:43.989 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:43.989 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:43.989 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:19:43.989 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:43.989 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:19:43.989 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:43.990 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 14003 2024-09-17T21:19:43.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.990 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:43.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:43.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912905+0000", 2024-09-17T21:19:43.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:43.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912905+0000", 2024-09-17T21:19:43.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:43.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912906+0000", 2024-09-17T21:19:43.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.091e-06 2024-09-17T21:19:43.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:43.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912911+0000", 2024-09-17T21:19:43.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9409999999999998e-06 2024-09-17T21:19:43.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:43.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912920+0000", 2024-09-17T21:19:43.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3530000000000001e-06 2024-09-17T21:19:43.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:43.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912927+0000", 2024-09-17T21:19:43.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9979999999999997e-06 2024-09-17T21:19:43.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:43.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959659+0000", 2024-09-17T21:19:43.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046731111999999998 2024-09-17T21:19:43.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:43.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959663+0000", 2024-09-17T21:19:43.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7570000000000004e-06 2024-09-17T21:19:43.993 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.993 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:43.993 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.993 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3882 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:43.993 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913045+0000", 2024-09-17T21:19:43.993 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.008887976, 2024-09-17T21:19:43.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046829914, 2024-09-17T21:19:43.993 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:43.993 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:43.993 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:43.994 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:43.994 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:43.994 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:43.994 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3882 2024-09-17T21:19:43.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.994 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:43.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.994 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:43.994 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913045+0000", 2024-09-17T21:19:43.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.994 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:43.994 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913045+0000", 2024-09-17T21:19:43.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:43.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913045+0000", 2024-09-17T21:19:43.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1200000000000002e-07 2024-09-17T21:19:43.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:43.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913048+0000", 2024-09-17T21:19:43.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.179e-06 2024-09-17T21:19:43.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:43.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913055+0000", 2024-09-17T21:19:43.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0269999999999998e-06 2024-09-17T21:19:43.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:43.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913061+0000", 2024-09-17T21:19:43.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5709999999999997e-06 2024-09-17T21:19:43.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:43.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959782+0000", 2024-09-17T21:19:43.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046720708 2024-09-17T21:19:43.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:43.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959813+0000", 2024-09-17T21:19:43.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1358e-05 2024-09-17T21:19:43.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:43.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959874+0000", 2024-09-17T21:19:43.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1359000000000007e-05 2024-09-17T21:19:43.997 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.997 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:43.997 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:43.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.998 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3883 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:43.998 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913069+0000", 2024-09-17T21:19:43.998 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.008863608, 2024-09-17T21:19:43.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046905401999999999, 2024-09-17T21:19:43.998 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:43.998 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:43.998 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:43.998 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:43.998 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:43.998 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:43.998 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3883 2024-09-17T21:19:43.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.998 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:43.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:43.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913069+0000", 2024-09-17T21:19:43.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:43.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913069+0000", 2024-09-17T21:19:43.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:43.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:43.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913069+0000", 2024-09-17T21:19:43.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:19:43.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:43.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:43.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:43.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913072+0000", 2024-09-17T21:19:44.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6390000000000001e-06 2024-09-17T21:19:44.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913079+0000", 2024-09-17T21:19:44.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2640000000000002e-06 2024-09-17T21:19:44.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913085+0000", 2024-09-17T21:19:44.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4249999999999997e-06 2024-09-17T21:19:44.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959893+0000", 2024-09-17T21:19:44.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046808728000000001 2024-09-17T21:19:44.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959926+0000", 2024-09-17T21:19:44.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2437999999999997e-05 2024-09-17T21:19:44.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959974+0000", 2024-09-17T21:19:44.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8516000000000001e-05 2024-09-17T21:19:44.001 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.001 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.001 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.002 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3884 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:44.002 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913092+0000", 2024-09-17T21:19:44.002 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.0088409650000001, 2024-09-17T21:19:44.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047065517000000001, 2024-09-17T21:19:44.002 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.002 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.002 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:44.002 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:44.002 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:44.002 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:44.002 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3884 2024-09-17T21:19:44.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913092+0000", 2024-09-17T21:19:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913092+0000", 2024-09-17T21:19:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913670+0000", 2024-09-17T21:19:44.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00057830399999999997 2024-09-17T21:19:44.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913680+0000", 2024-09-17T21:19:44.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0426e-05 2024-09-17T21:19:44.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913691+0000", 2024-09-17T21:19:44.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0309000000000001e-05 2024-09-17T21:19:44.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913698+0000", 2024-09-17T21:19:44.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1760000000000004e-06 2024-09-17T21:19:44.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.960056+0000", 2024-09-17T21:19:44.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046358411000000002 2024-09-17T21:19:44.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.960089+0000", 2024-09-17T21:19:44.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2588e-05 2024-09-17T21:19:44.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.960157+0000", 2024-09-17T21:19:44.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8303e-05 2024-09-17T21:19:44.006 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.006 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.006 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.006 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:14004 3.4 3.4f4be604 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:44.006 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913136+0000", 2024-09-17T21:19:44.006 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.0087967290000002, 2024-09-17T21:19:44.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046862804000000001, 2024-09-17T21:19:44.006 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.006 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.006 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:19:44.006 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:44.007 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:19:44.007 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:44.007 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 14004 2024-09-17T21:19:44.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.007 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913136+0000", 2024-09-17T21:19:44.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913136+0000", 2024-09-17T21:19:44.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.008 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.008 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913447+0000", 2024-09-17T21:19:44.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031107200000000002 2024-09-17T21:19:44.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.008 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.008 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913460+0000", 2024-09-17T21:19:44.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2996999999999999e-05 2024-09-17T21:19:44.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.008 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913470+0000", 2024-09-17T21:19:44.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0037999999999999e-05 2024-09-17T21:19:44.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913477+0000", 2024-09-17T21:19:44.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:19:44.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959994+0000", 2024-09-17T21:19:44.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046517599999999999 2024-09-17T21:19:44.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959999+0000", 2024-09-17T21:19:44.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1400000000000002e-06 2024-09-17T21:19:44.010 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.010 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.010 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.010 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3885 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:44.010 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913710+0000", 2024-09-17T21:19:44.010 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.0082222820000002, 2024-09-17T21:19:44.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047449947999999999, 2024-09-17T21:19:44.010 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.011 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.011 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:44.011 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:44.011 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:44.011 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:44.011 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3885 2024-09-17T21:19:44.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.011 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.011 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.011 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913710+0000", 2024-09-17T21:19:44.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.011 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.012 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913710+0000", 2024-09-17T21:19:44.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.012 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.012 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.917352+0000", 2024-09-17T21:19:44.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0036412950000000001 2024-09-17T21:19:44.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.012 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.012 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.917357+0000", 2024-09-17T21:19:44.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4879999999999998e-06 2024-09-17T21:19:44.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.918061+0000", 2024-09-17T21:19:44.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00070362899999999999 2024-09-17T21:19:44.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.918069+0000", 2024-09-17T21:19:44.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4530000000000002e-06 2024-09-17T21:19:44.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.960956+0000", 2024-09-17T21:19:44.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042886951999999999 2024-09-17T21:19:44.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.961036+0000", 2024-09-17T21:19:44.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0010000000000001e-05 2024-09-17T21:19:44.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.961160+0000", 2024-09-17T21:19:44.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012412100000000001 2024-09-17T21:19:44.014 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.014 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.014 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.015 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:13984 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:44.015 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.921124+0000", 2024-09-17T21:19:44.015 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.0008083050000001, 2024-09-17T21:19:44.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047129407999999998, 2024-09-17T21:19:44.015 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.015 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.015 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:44.015 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:44.015 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:19:44.015 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:44.015 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 13984 2024-09-17T21:19:44.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.015 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.016 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.016 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.921124+0000", 2024-09-17T21:19:44.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.016 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.016 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.921124+0000", 2024-09-17T21:19:44.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.016 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.016 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.928309+0000", 2024-09-17T21:19:44.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0071847619999999999 2024-09-17T21:19:44.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.017 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.017 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.928316+0000", 2024-09-17T21:19:44.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2969999999999998e-06 2024-09-17T21:19:44.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.017 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.017 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.928330+0000", 2024-09-17T21:19:44.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3849000000000001e-05 2024-09-17T21:19:44.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.017 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.017 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.928338+0000", 2024-09-17T21:19:44.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.354e-06 2024-09-17T21:19:44.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.968167+0000", 2024-09-17T21:19:44.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.039828298999999998 2024-09-17T21:19:44.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.968201+0000", 2024-09-17T21:19:44.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3933000000000002e-05 2024-09-17T21:19:44.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.968254+0000", 2024-09-17T21:19:44.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2914000000000002e-05 2024-09-17T21:19:44.018 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.018 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.019 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.019 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.019 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.019 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:44.020 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:19:44.021 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:44.014+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:44.078 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-09-17T21:19:44.074+0000 7fe37768f640 -1 mon.a@0(leader).osd e91 definitely_dead 0 2024-09-17T21:19:44.120 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:44.118+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:44.189 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:44.189 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:44.189 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:19:44.189 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:44.204 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:19:44.221 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:44.216+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:44.322 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:44.318+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:44.366 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:44.366 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:44.366 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:44.366 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:44.366 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:44.379 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:19:44.423 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:44.420+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:44.523 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:44.520+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:44.545 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:44.545 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:19:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 86.325199635000004, 2024-09-17T21:19:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:19:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:19:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:19:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:19:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:19:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:19:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:19:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:19:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:19:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:19:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:19:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:19:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:19:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:19:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:19:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:19:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:19:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:19:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 81.674898971000005, 2024-09-17T21:19:44.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:19:44.551 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.551 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.551 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:44.552 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:44.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:44.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:19:44.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:19:44.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:19:44.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:19:44.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:19:44.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:19:44.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:19:44.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:19:44.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:19:44.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:19:44.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:19:44.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:19:44.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:44.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:19:44.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:19:44.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:19:44.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:19:44.555 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.555 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.555 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.556 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:19:44.556 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:44.556 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 81.673633887999998, 2024-09-17T21:19:44.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:19:44.556 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.556 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.556 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:44.556 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:44.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:44.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:19:44.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:19:44.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:19:44.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:19:44.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:19:44.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:19:44.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:19:44.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:19:44.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:19:44.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:19:44.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:19:44.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:19:44.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:44.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:19:44.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:19:44.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.560 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:19:44.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:19:44.560 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.560 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.560 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.560 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:19:44.560 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:44.560 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 81.663854478000005, 2024-09-17T21:19:44.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:19:44.560 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.560 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.561 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:44.561 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.561 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:44.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.561 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:44.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:19:44.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:19:44.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:19:44.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:19:44.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:19:44.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:19:44.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:19:44.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:19:44.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:19:44.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:19:44.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:19:44.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:19:44.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:44.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:19:44.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:19:44.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:19:44.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:19:44.565 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.565 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.565 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.565 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:19:44.565 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:44.565 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 81.655132074999997, 2024-09-17T21:19:44.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:19:44.565 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.565 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.565 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:44.565 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:44.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:44.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:19:44.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:19:44.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.567 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:19:44.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:19:44.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.567 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:19:44.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:19:44.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.567 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:19:44.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:19:44.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.568 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.568 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.568 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:19:44.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:19:44.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.568 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.568 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.568 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:19:44.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:19:44.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.568 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.568 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:44.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:19:44.569 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:19:44.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.569 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.569 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:19:44.569 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:19:44.569 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.569 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.569 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.569 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.569 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:19:44.569 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:44.570 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 68.270041511000002, 2024-09-17T21:19:44.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:19:44.570 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.570 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.570 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:44.570 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.570 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.570 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:44.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.570 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.571 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:44.571 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.571 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.571 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.571 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:19:44.571 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:19:44.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.571 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.571 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.571 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:19:44.572 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:19:44.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.572 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.572 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:19:44.572 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:19:44.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.572 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.572 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:19:44.572 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:19:44.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.573 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.573 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:19:44.573 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:19:44.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.573 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.573 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.573 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:19:44.573 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:19:44.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.573 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.573 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:44.573 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:19:44.573 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:19:44.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.574 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.574 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:19:44.574 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:19:44.574 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.574 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.574 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.574 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1780 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'195, mlcod=69'190)", 2024-09-17T21:19:44.574 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:19:44.574 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.292185907, 2024-09-17T21:19:44.574 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075761769999999999, 2024-09-17T21:19:44.575 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.575 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.575 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:44.575 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.575 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.575 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.575 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:19:44.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.575 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.575 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.575 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:19:44.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.576 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.576 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.576 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250375+0000", 2024-09-17T21:19:44.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:19:44.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.576 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.576 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.576 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250383+0000", 2024-09-17T21:19:44.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7870000000000002e-06 2024-09-17T21:19:44.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.576 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.576 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.576 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250402+0000", 2024-09-17T21:19:44.577 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9346000000000001e-05 2024-09-17T21:19:44.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.577 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.577 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250413+0000", 2024-09-17T21:19:44.577 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.039e-05 2024-09-17T21:19:44.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.577 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.577 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251353+0000", 2024-09-17T21:19:44.577 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000940091 2024-09-17T21:19:44.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.577 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.578 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251407+0000", 2024-09-17T21:19:44.578 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4701999999999999e-05 2024-09-17T21:19:44.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.578 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.578 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:44.578 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257829+0000", 2024-09-17T21:19:44.578 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064212779999999999 2024-09-17T21:19:44.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.578 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.578 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.578 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257950+0000", 2024-09-17T21:19:44.578 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121623 2024-09-17T21:19:44.578 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.578 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.579 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.579 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1781 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'196, mlcod=69'190)", 2024-09-17T21:19:44.579 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:19:44.579 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.291102956, 2024-09-17T21:19:44.579 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065632939999999999, 2024-09-17T21:19:44.579 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.579 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.579 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:44.579 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.579 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.579 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:19:44.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.580 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.580 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.580 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:19:44.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.580 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.580 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.580 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251459+0000", 2024-09-17T21:19:44.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.624e-06 2024-09-17T21:19:44.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.580 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.581 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.581 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251471+0000", 2024-09-17T21:19:44.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2636000000000001e-05 2024-09-17T21:19:44.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.581 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.581 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251498+0000", 2024-09-17T21:19:44.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:19:44.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.581 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.581 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251515+0000", 2024-09-17T21:19:44.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6939000000000002e-05 2024-09-17T21:19:44.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.582 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.582 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.582 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252938+0000", 2024-09-17T21:19:44.582 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001423082 2024-09-17T21:19:44.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.582 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.582 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.582 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252999+0000", 2024-09-17T21:19:44.582 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1138999999999999e-05 2024-09-17T21:19:44.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.582 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.582 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:44.582 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257955+0000", 2024-09-17T21:19:44.582 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049559640000000002 2024-09-17T21:19:44.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.583 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.583 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.258020+0000", 2024-09-17T21:19:44.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5504000000000002e-05 2024-09-17T21:19:44.583 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.583 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.583 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.583 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1788 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'198, mlcod=69'191)", 2024-09-17T21:19:44.583 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:19:44.583 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.289077466999998, 2024-09-17T21:19:44.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073690370000000002, 2024-09-17T21:19:44.584 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.584 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.584 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:44.584 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.584 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.584 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.584 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:19:44.584 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.584 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.584 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.584 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:19:44.584 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.585 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.585 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.585 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253484+0000", 2024-09-17T21:19:44.585 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2470000000000001e-06 2024-09-17T21:19:44.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.585 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.585 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.585 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253492+0000", 2024-09-17T21:19:44.585 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7159999999999992e-06 2024-09-17T21:19:44.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.585 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.585 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.585 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253509+0000", 2024-09-17T21:19:44.586 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.666e-05 2024-09-17T21:19:44.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.586 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.586 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.586 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253522+0000", 2024-09-17T21:19:44.586 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2643000000000001e-05 2024-09-17T21:19:44.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.586 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.586 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.586 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254853+0000", 2024-09-17T21:19:44.586 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331238 2024-09-17T21:19:44.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.587 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.587 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254885+0000", 2024-09-17T21:19:44.587 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1562e-05 2024-09-17T21:19:44.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.587 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:44.587 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260788+0000", 2024-09-17T21:19:44.587 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059030749999999998 2024-09-17T21:19:44.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.588 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.589 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260852+0000", 2024-09-17T21:19:44.589 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3895999999999999e-05 2024-09-17T21:19:44.589 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.589 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.589 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.589 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.589 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5746 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'477, mlcod=71'473)", 2024-09-17T21:19:44.589 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:19:44.589 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.283649909000001, 2024-09-17T21:19:44.589 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074182709999999997, 2024-09-17T21:19:44.589 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.589 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:19:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:19:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.591 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258912+0000", 2024-09-17T21:19:44.591 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:19:44.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.591 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.591 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258926+0000", 2024-09-17T21:19:44.591 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4249e-05 2024-09-17T21:19:44.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.591 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.591 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258946+0000", 2024-09-17T21:19:44.591 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9879999999999999e-05 2024-09-17T21:19:44.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.592 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258956+0000", 2024-09-17T21:19:44.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:19:44.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.592 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259380+0000", 2024-09-17T21:19:44.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042337599999999999 2024-09-17T21:19:44.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.592 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259412+0000", 2024-09-17T21:19:44.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:19:44.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.593 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:44.593 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266191+0000", 2024-09-17T21:19:44.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067793269999999999 2024-09-17T21:19:44.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.593 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.593 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266328+0000", 2024-09-17T21:19:44.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013690999999999999 2024-09-17T21:19:44.594 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.594 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.594 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.594 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5747 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'478, mlcod=71'473)", 2024-09-17T21:19:44.594 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:19:44.594 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.28294356, 2024-09-17T21:19:44.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068779440000000004, 2024-09-17T21:19:44.594 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.594 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.594 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:44.594 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.595 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.595 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:19:44.595 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.595 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.595 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:19:44.595 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.595 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.595 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259618+0000", 2024-09-17T21:19:44.595 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6199999999999999e-06 2024-09-17T21:19:44.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.596 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.596 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259630+0000", 2024-09-17T21:19:44.596 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1884e-05 2024-09-17T21:19:44.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.596 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.596 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259657+0000", 2024-09-17T21:19:44.596 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7339e-05 2024-09-17T21:19:44.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.596 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.596 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259677+0000", 2024-09-17T21:19:44.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0156000000000001e-05 2024-09-17T21:19:44.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260521+0000", 2024-09-17T21:19:44.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084312899999999997 2024-09-17T21:19:44.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260585+0000", 2024-09-17T21:19:44.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4558000000000001e-05 2024-09-17T21:19:44.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:44.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266336+0000", 2024-09-17T21:19:44.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057505020000000002 2024-09-17T21:19:44.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266494+0000", 2024-09-17T21:19:44.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158756 2024-09-17T21:19:44.599 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.599 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.599 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.599 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5750 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'479, mlcod=71'473)", 2024-09-17T21:19:44.599 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:19:44.599 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.282231367000001, 2024-09-17T21:19:44.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006302874, 2024-09-17T21:19:44.599 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.599 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.599 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:44.600 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.600 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.600 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:19:44.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.600 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.600 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:19:44.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.600 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.601 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260330+0000", 2024-09-17T21:19:44.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:19:44.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.601 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260343+0000", 2024-09-17T21:19:44.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2862e-05 2024-09-17T21:19:44.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.601 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260367+0000", 2024-09-17T21:19:44.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3723e-05 2024-09-17T21:19:44.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.602 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260386+0000", 2024-09-17T21:19:44.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8879999999999999e-05 2024-09-17T21:19:44.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.602 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261579+0000", 2024-09-17T21:19:44.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001193431 2024-09-17T21:19:44.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.602 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261608+0000", 2024-09-17T21:19:44.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8858999999999999e-05 2024-09-17T21:19:44.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:44.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266505+0000", 2024-09-17T21:19:44.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004896999 2024-09-17T21:19:44.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266631+0000", 2024-09-17T21:19:44.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012655300000000001 2024-09-17T21:19:44.603 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.603 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.603 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.604 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5751 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'480, mlcod=71'473)", 2024-09-17T21:19:44.604 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:19:44.604 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.281524155, 2024-09-17T21:19:44.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075756299999999999, 2024-09-17T21:19:44.604 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.604 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.604 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:44.604 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:19:44.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.605 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.605 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:19:44.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.605 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.605 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261037+0000", 2024-09-17T21:19:44.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:19:44.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.605 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261049+0000", 2024-09-17T21:19:44.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1063999999999999e-05 2024-09-17T21:19:44.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261072+0000", 2024-09-17T21:19:44.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3102e-05 2024-09-17T21:19:44.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261133+0000", 2024-09-17T21:19:44.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1276000000000001e-05 2024-09-17T21:19:44.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:19:44.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015334319999999999 2024-09-17T21:19:44.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262702+0000", 2024-09-17T21:19:44.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5281999999999999e-05 2024-09-17T21:19:44.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:44.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268527+0000", 2024-09-17T21:19:44.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058250389999999997 2024-09-17T21:19:44.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268611+0000", 2024-09-17T21:19:44.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4724999999999999e-05 2024-09-17T21:19:44.608 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.608 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.608 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.608 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5756 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'481, mlcod=71'473)", 2024-09-17T21:19:44.608 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:19:44.608 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.280663299, 2024-09-17T21:19:44.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006772386, 2024-09-17T21:19:44.609 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.609 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.609 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:44.609 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.609 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.609 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:19:44.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.609 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.609 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:19:44.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261899+0000", 2024-09-17T21:19:44.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.903e-06 2024-09-17T21:19:44.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261912+0000", 2024-09-17T21:19:44.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-05 2024-09-17T21:19:44.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261938+0000", 2024-09-17T21:19:44.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6098999999999999e-05 2024-09-17T21:19:44.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261955+0000", 2024-09-17T21:19:44.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.696e-05 2024-09-17T21:19:44.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263845+0000", 2024-09-17T21:19:44.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001890187 2024-09-17T21:19:44.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263907+0000", 2024-09-17T21:19:44.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1600999999999995e-05 2024-09-17T21:19:44.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:44.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268615+0000", 2024-09-17T21:19:44.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047086070000000001 2024-09-17T21:19:44.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268669+0000", 2024-09-17T21:19:44.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3588999999999998e-05 2024-09-17T21:19:44.612 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.613 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.613 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.613 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4978.0:598 3.9 e80/23 3:9701cc17:::rbd_data.1152724ff2a3.0000000000000019:head v 80'202, mlcod=80'198)", 2024-09-17T21:19:44.613 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:19:44.613 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 17.779088915999999, 2024-09-17T21:19:44.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066473340000000004, 2024-09-17T21:19:44.613 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.613 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.613 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:44.613 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:19:44.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:19:44.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763472+0000", 2024-09-17T21:19:44.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6899999999999997e-07 2024-09-17T21:19:44.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763478+0000", 2024-09-17T21:19:44.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:19:44.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763489+0000", 2024-09-17T21:19:44.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0628999999999999e-05 2024-09-17T21:19:44.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763495+0000", 2024-09-17T21:19:44.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6649999999999997e-06 2024-09-17T21:19:44.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763907+0000", 2024-09-17T21:19:44.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041163899999999999 2024-09-17T21:19:44.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763976+0000", 2024-09-17T21:19:44.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8534999999999993e-05 2024-09-17T21:19:44.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:44.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.769900+0000", 2024-09-17T21:19:44.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059243270000000001 2024-09-17T21:19:44.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.770118+0000", 2024-09-17T21:19:44.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021842500000000001 2024-09-17T21:19:44.617 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.617 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.617 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.617 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:468 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'94, mlcod=89'87)", 2024-09-17T21:19:44.617 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:19:44.618 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.8563436170000003, 2024-09-17T21:19:44.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066528790000000004, 2024-09-17T21:19:44.618 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.618 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.618 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:44.618 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:19:44.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:19:44.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686218+0000", 2024-09-17T21:19:44.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3060000000000001e-06 2024-09-17T21:19:44.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686256+0000", 2024-09-17T21:19:44.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8333e-05 2024-09-17T21:19:44.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686272+0000", 2024-09-17T21:19:44.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:19:44.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686283+0000", 2024-09-17T21:19:44.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0903e-05 2024-09-17T21:19:44.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686682+0000", 2024-09-17T21:19:44.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039935099999999999 2024-09-17T21:19:44.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686785+0000", 2024-09-17T21:19:44.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010275199999999999 2024-09-17T21:19:44.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:44.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692816+0000", 2024-09-17T21:19:44.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060317579999999999 2024-09-17T21:19:44.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692869+0000", 2024-09-17T21:19:44.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2775999999999998e-05 2024-09-17T21:19:44.622 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.622 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.622 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.622 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:755 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'127, mlcod=89'122)", 2024-09-17T21:19:44.622 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:19:44.622 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.7198884029999997, 2024-09-17T21:19:44.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084014080000000008, 2024-09-17T21:19:44.622 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.622 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.622 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:44.623 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:19:44.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:19:44.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:19:44.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:19:44.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822683+0000", 2024-09-17T21:19:44.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0674e-05 2024-09-17T21:19:44.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822698+0000", 2024-09-17T21:19:44.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5189e-05 2024-09-17T21:19:44.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822708+0000", 2024-09-17T21:19:44.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9119999999999993e-06 2024-09-17T21:19:44.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822748+0000", 2024-09-17T21:19:44.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9727000000000001e-05 2024-09-17T21:19:44.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822759+0000", 2024-09-17T21:19:44.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0865e-05 2024-09-17T21:19:44.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:44.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831034+0000", 2024-09-17T21:19:44.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082750710000000002 2024-09-17T21:19:44.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831073+0000", 2024-09-17T21:19:44.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9063999999999997e-05 2024-09-17T21:19:44.628 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.628 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.628 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.628 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:756 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'128, mlcod=89'124)", 2024-09-17T21:19:44.629 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:19:44.629 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.7179657710000003, 2024-09-17T21:19:44.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0076300270000000002, 2024-09-17T21:19:44.629 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.629 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.629 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:44.630 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:19:44.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:19:44.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824595+0000", 2024-09-17T21:19:44.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-17T21:19:44.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824600+0000", 2024-09-17T21:19:44.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:19:44.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824612+0000", 2024-09-17T21:19:44.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.181e-05 2024-09-17T21:19:44.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824619+0000", 2024-09-17T21:19:44.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:19:44.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824671+0000", 2024-09-17T21:19:44.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2262999999999998e-05 2024-09-17T21:19:44.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824695+0000", 2024-09-17T21:19:44.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3883000000000002e-05 2024-09-17T21:19:44.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:44.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832185+0000", 2024-09-17T21:19:44.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007490048 2024-09-17T21:19:44.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832224+0000", 2024-09-17T21:19:44.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8957e-05 2024-09-17T21:19:44.637 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.637 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.637 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.637 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2617 3.d e90/23 3:b0143718:::rbd_data.1152a8ae2019.0000000000000016:head v 90'249, mlcod=90'246)", 2024-09-17T21:19:44.637 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:19:44.637 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.6981147710000002, 2024-09-17T21:19:44.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063715559999999996, 2024-09-17T21:19:44.638 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.638 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.638 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:44.638 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:19:44.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:19:44.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844455+0000", 2024-09-17T21:19:44.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4259999999999992e-06 2024-09-17T21:19:44.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844470+0000", 2024-09-17T21:19:44.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5386000000000001e-05 2024-09-17T21:19:44.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844499+0000", 2024-09-17T21:19:44.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.883e-05 2024-09-17T21:19:44.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844518+0000", 2024-09-17T21:19:44.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8830000000000001e-05 2024-09-17T21:19:44.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844649+0000", 2024-09-17T21:19:44.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013143399999999999 2024-09-17T21:19:44.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844702+0000", 2024-09-17T21:19:44.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2876e-05 2024-09-17T21:19:44.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:44.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850696+0000", 2024-09-17T21:19:44.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059943729999999999 2024-09-17T21:19:44.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850817+0000", 2024-09-17T21:19:44.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120401 2024-09-17T21:19:44.644 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.644 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.645 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.645 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2635 3.d e90/23 3:b3d1e920:::rbd_data.1152a8ae2019.000000000000001f:head v 90'252, mlcod=90'246)", 2024-09-17T21:19:44.645 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:19:44.645 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.695406701, 2024-09-17T21:19:44.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0077265190000000003, 2024-09-17T21:19:44.645 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:44.645 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:44.645 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:44.645 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:44.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:44.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:19:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:19:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847155+0000", 2024-09-17T21:19:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.544e-06 2024-09-17T21:19:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847165+0000", 2024-09-17T21:19:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.043e-05 2024-09-17T21:19:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847189+0000", 2024-09-17T21:19:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3659000000000002e-05 2024-09-17T21:19:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847210+0000", 2024-09-17T21:19:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1163e-05 2024-09-17T21:19:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848856+0000", 2024-09-17T21:19:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001645521 2024-09-17T21:19:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848905+0000", 2024-09-17T21:19:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9048999999999997e-05 2024-09-17T21:19:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854770+0000", 2024-09-17T21:19:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058652879999999998 2024-09-17T21:19:44.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:44.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:44.649 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:44.649 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854880+0000", 2024-09-17T21:19:44.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010986500000000001 2024-09-17T21:19:44.649 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.649 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.649 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.649 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:44.649 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:44.649 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:44.650 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-17T21:19:44.651 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:19:44.651 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:44.646+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:44.671 INFO:teuthology.orchestra.run.smithi083.stderr:marked out osd.2. 2024-09-17T21:19:44.752 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:44.748+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:44.785 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:44.785 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:44.786 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:19:44.786 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:44.799 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:19:44.853 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:44.850+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:44.954 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:44.950+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:44.971 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:44.971 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:44.971 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:44.971 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:44.971 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:44.986 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:19:45.054 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:45.052+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:45.111 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:45.111 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:45.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:45.111 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:45.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.111 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:19:45.112 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:45.112 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 108.093219573, 2024-09-17T21:19:45.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:19:45.112 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.112 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.112 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:45.112 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.112 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:19:45.112 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:19:45.112 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:19:45.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.112 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:19:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:19:45.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:19:45.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:19:45.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:19:45.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:19:45.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:19:45.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:19:45.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:19:45.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:19:45.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:45.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:19:45.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:19:45.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:19:45.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:19:45.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:19:45.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:19:45.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:19:45.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:19:45.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:19:45.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:45.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:19:45.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:19:45.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:45.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:19:45.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:19:45.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:45.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:19:45.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:19:45.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:19:45.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:19:45.118 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.118 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.118 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.118 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:45.118 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:45.118 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 88.513055292999994, 2024-09-17T21:19:45.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:19:45.119 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.119 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.119 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.119 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.119 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:19:45.119 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:19:45.119 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:19:45.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.119 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:45.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:45.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:19:45.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:19:45.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:19:45.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:19:45.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:19:45.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:19:45.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:19:45.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:19:45.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:19:45.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:19:45.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:45.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:19:45.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:19:45.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:19:45.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:19:45.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:45.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:19:45.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:19:45.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:19:45.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:19:45.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:19:45.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:19:45.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:19:45.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:19:45.124 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.124 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.124 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.124 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:45.124 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:45.124 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 88.512957981, 2024-09-17T21:19:45.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:19:45.124 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.124 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.124 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.125 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.125 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:45.125 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:45.125 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:19:45.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.125 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:45.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:45.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:19:45.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:19:45.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:19:45.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:19:45.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:19:45.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:19:45.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:19:45.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:19:45.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:19:45.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:19:45.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:45.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:19:45.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:19:45.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:19:45.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:19:45.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:45.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:19:45.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:19:45.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:19:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:19:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:19:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:19:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:19:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:19:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:19:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:19:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:19:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:19:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:19:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 88.510871179999995, 2024-09-17T21:19:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:19:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:19:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:19:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:19:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:19:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:19:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:19:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:19:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:19:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:19:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:19:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:19:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:19:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:19:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:19:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:19:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:19:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:19:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:19:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:19:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:19:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:19:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:19:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:19:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:19:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:19:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:19:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:19:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:19:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:19:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:19:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:19:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:19:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:19:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:19:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.826972398999999, 2024-09-17T21:19:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:19:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:19:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:19:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:19:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:19:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:19:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:19:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:19:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:19:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:19:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:19:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:19:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:19:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:19:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:19:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:19:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:19:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:19:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:19:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:19:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:19:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:19:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:19:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:19:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.817880688000002, 2024-09-17T21:19:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:19:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:19:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:19:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:19:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:19:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:19:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:19:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:19:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:19:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:19:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:19:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:19:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:19:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:19:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:19:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:19:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:19:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:19:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:19:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:19:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.817708637000003, 2024-09-17T21:19:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:19:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:19:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:19:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:19:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:19:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:19:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:19:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:19:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:19:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:19:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:19:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:19:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:19:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:19:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:19:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:19:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:19:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:19:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:19:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:19:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.642775890000003, 2024-09-17T21:19:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:19:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:19:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:19:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:19:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:19:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:19:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:19:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:19:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:19:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:19:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:19:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:19:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:19:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:19:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:19:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:19:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:19:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:19:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:19:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:19:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.642691333000002, 2024-09-17T21:19:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:19:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:19:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:19:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:19:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:19:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:19:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:19:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:19:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:19:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:19:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:19:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:19:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:19:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:19:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:19:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:19:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:19:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:19:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:19:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.642625696000003, 2024-09-17T21:19:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:19:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:19:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:19:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:19:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:19:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:19:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:19:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:19:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:19:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:19:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:19:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:19:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:19:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:19:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:19:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:19:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:19:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:19:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:19:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:19:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:19:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.2690505989999998, 2024-09-17T21:19:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:19:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:19:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:19:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:19:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:19:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:19:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:19:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:19:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:19:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:19:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:19:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:19:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:19:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:19:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:19:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:19:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:19:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:19:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:19:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.2689310870000003, 2024-09-17T21:19:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:19:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:19:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:19:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:19:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:19:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:19:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:19:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:19:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:19:45.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:19:45.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:19:45.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:19:45.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:19:45.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:19:45.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:19:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:19:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:19:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:19:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2695 3.15 3:aedfd032:::rbd_data.1152a8ae2019.0000000000000041:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.843199+0000", 2024-09-17T21:19:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.2637857830000003, 2024-09-17T21:19:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079730569000000001, 2024-09-17T21:19:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2695 2024-09-17T21:19:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843199+0000", 2024-09-17T21:19:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843199+0000", 2024-09-17T21:19:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843201+0000", 2024-09-17T21:19:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0899999999999999e-06 2024-09-17T21:19:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843214+0000", 2024-09-17T21:19:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2233e-05 2024-09-17T21:19:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843245+0000", 2024-09-17T21:19:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1464000000000002e-05 2024-09-17T21:19:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843266+0000", 2024-09-17T21:19:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0879000000000001e-05 2024-09-17T21:19:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847920+0000", 2024-09-17T21:19:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.004654292 2024-09-17T21:19:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.848240+0000", 2024-09-17T21:19:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031994499999999997 2024-09-17T21:19:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922930+0000", 2024-09-17T21:19:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.074689666000000002 2024-09-17T21:19:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2732 3.15 3:ab9a4731:::rbd_data.1152a8ae2019.0000000000000058:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.846909+0000", 2024-09-17T21:19:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.2600757370000002, 2024-09-17T21:19:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076145476000000004, 2024-09-17T21:19:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2732 2024-09-17T21:19:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846909+0000", 2024-09-17T21:19:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846909+0000", 2024-09-17T21:19:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846911+0000", 2024-09-17T21:19:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.31e-06 2024-09-17T21:19:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846916+0000", 2024-09-17T21:19:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.023e-06 2024-09-17T21:19:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846931+0000", 2024-09-17T21:19:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5549999999999999e-05 2024-09-17T21:19:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846943+0000", 2024-09-17T21:19:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1783000000000001e-05 2024-09-17T21:19:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.850592+0000", 2024-09-17T21:19:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036486159999999999 2024-09-17T21:19:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.850870+0000", 2024-09-17T21:19:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027864200000000001 2024-09-17T21:19:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.923055+0000", 2024-09-17T21:19:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.072184551999999999 2024-09-17T21:19:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2737 3.15 3:a8c4e696:::rbd_data.1152a8ae2019.000000000000005b:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.847014+0000", 2024-09-17T21:19:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.2599706020000001, 2024-09-17T21:19:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076158271, 2024-09-17T21:19:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2737 2024-09-17T21:19:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847014+0000", 2024-09-17T21:19:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847014+0000", 2024-09-17T21:19:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847015+0000", 2024-09-17T21:19:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0699999999999999e-06 2024-09-17T21:19:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847021+0000", 2024-09-17T21:19:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.446e-06 2024-09-17T21:19:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847038+0000", 2024-09-17T21:19:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7257000000000001e-05 2024-09-17T21:19:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847051+0000", 2024-09-17T21:19:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2649999999999999e-05 2024-09-17T21:19:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.852127+0000", 2024-09-17T21:19:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050757420000000003 2024-09-17T21:19:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.852423+0000", 2024-09-17T21:19:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029608100000000001 2024-09-17T21:19:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.923173+0000", 2024-09-17T21:19:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.070750024999999994 2024-09-17T21:19:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3397 3.8 3:174f9d43:::rbd_data.1152a8ae2019.000000000000006e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.897797+0000", 2024-09-17T21:19:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.2091876570000002, 2024-09-17T21:19:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045478192000000001, 2024-09-17T21:19:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3397 2024-09-17T21:19:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897797+0000", 2024-09-17T21:19:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897797+0000", 2024-09-17T21:19:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897799+0000", 2024-09-17T21:19:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.238e-06 2024-09-17T21:19:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897806+0000", 2024-09-17T21:19:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5900000000000002e-06 2024-09-17T21:19:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897822+0000", 2024-09-17T21:19:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5659e-05 2024-09-17T21:19:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897832+0000", 2024-09-17T21:19:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0034000000000001e-05 2024-09-17T21:19:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.931249+0000", 2024-09-17T21:19:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033416700000000001 2024-09-17T21:19:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.931425+0000", 2024-09-17T21:19:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017629500000000001 2024-09-17T21:19:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943276+0000", 2024-09-17T21:19:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011850675999999999 2024-09-17T21:19:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3524 3.8 3:119ba024:::rbd_data.1152c6ce51b2.0000000000000005:17 [read 1261568~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.901789+0000", 2024-09-17T21:19:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.2051957299999998, 2024-09-17T21:19:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.042862082000000003, 2024-09-17T21:19:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3524 2024-09-17T21:19:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901789+0000", 2024-09-17T21:19:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901789+0000", 2024-09-17T21:19:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901790+0000", 2024-09-17T21:19:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0400000000000005e-07 2024-09-17T21:19:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901795+0000", 2024-09-17T21:19:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3839999999999999e-06 2024-09-17T21:19:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901807+0000", 2024-09-17T21:19:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2532e-05 2024-09-17T21:19:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901816+0000", 2024-09-17T21:19:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2220000000000008e-06 2024-09-17T21:19:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.944407+0000", 2024-09-17T21:19:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.042591088999999999 2024-09-17T21:19:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.944498+0000", 2024-09-17T21:19:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0883000000000004e-05 2024-09-17T21:19:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.944651+0000", 2024-09-17T21:19:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015306799999999999 2024-09-17T21:19:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3679 3.8 3:129b2dec:::rbd_data.1152a8ae2019.0000000000000012:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.908354+0000", 2024-09-17T21:19:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.1986313690000001, 2024-09-17T21:19:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046195880000000002, 2024-09-17T21:19:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3679 2024-09-17T21:19:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908354+0000", 2024-09-17T21:19:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908354+0000", 2024-09-17T21:19:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908355+0000", 2024-09-17T21:19:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9999999999999995e-07 2024-09-17T21:19:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908361+0000", 2024-09-17T21:19:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7399999999999998e-06 2024-09-17T21:19:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908375+0000", 2024-09-17T21:19:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.363e-05 2024-09-17T21:19:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908386+0000", 2024-09-17T21:19:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.06e-05 2024-09-17T21:19:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.945600+0000", 2024-09-17T21:19:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.037214199000000003 2024-09-17T21:19:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.945650+0000", 2024-09-17T21:19:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9737999999999997e-05 2024-09-17T21:19:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.945799+0000", 2024-09-17T21:19:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014921699999999999 2024-09-17T21:19:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954371+0000", 2024-09-17T21:19:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085719730000000001 2024-09-17T21:19:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954460+0000", 2024-09-17T21:19:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9596000000000001e-05 2024-09-17T21:19:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954511+0000", 2024-09-17T21:19:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0455999999999999e-05 2024-09-17T21:19:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954550+0000", 2024-09-17T21:19:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8730999999999999e-05 2024-09-17T21:19:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3703 3.13 3:cdb4d88a:::rbd_data.1152a8ae2019.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.908402+0000", 2024-09-17T21:19:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.1985833289999999, 2024-09-17T21:19:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046393446999999997, 2024-09-17T21:19:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3703 2024-09-17T21:19:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908402+0000", 2024-09-17T21:19:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908402+0000", 2024-09-17T21:19:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908403+0000", 2024-09-17T21:19:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0529999999999999e-06 2024-09-17T21:19:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908409+0000", 2024-09-17T21:19:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2219999999999999e-06 2024-09-17T21:19:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908422+0000", 2024-09-17T21:19:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3147e-05 2024-09-17T21:19:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908432+0000", 2024-09-17T21:19:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.039e-05 2024-09-17T21:19:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.942217+0000", 2024-09-17T21:19:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033784994999999998 2024-09-17T21:19:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.942296+0000", 2024-09-17T21:19:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8218000000000002e-05 2024-09-17T21:19:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943003+0000", 2024-09-17T21:19:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000707403 2024-09-17T21:19:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954635+0000", 2024-09-17T21:19:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011631786999999999 2024-09-17T21:19:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954706+0000", 2024-09-17T21:19:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1064999999999998e-05 2024-09-17T21:19:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954757+0000", 2024-09-17T21:19:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0760000000000002e-05 2024-09-17T21:19:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954795+0000", 2024-09-17T21:19:45.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8407e-05 2024-09-17T21:19:45.212 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.212 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.212 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.212 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3711 3.13 3:ca5a0e5e:::rbd_data.1152a8ae2019.0000000000000040:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.212 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.908514+0000", 2024-09-17T21:19:45.212 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.198471166, 2024-09-17T21:19:45.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046370871000000001, 2024-09-17T21:19:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3711 2024-09-17T21:19:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908514+0000", 2024-09-17T21:19:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908514+0000", 2024-09-17T21:19:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908515+0000", 2024-09-17T21:19:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0449999999999999e-06 2024-09-17T21:19:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908521+0000", 2024-09-17T21:19:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2179999999999996e-06 2024-09-17T21:19:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908535+0000", 2024-09-17T21:19:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4124999999999999e-05 2024-09-17T21:19:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908548+0000", 2024-09-17T21:19:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2680000000000001e-05 2024-09-17T21:19:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943530+0000", 2024-09-17T21:19:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034981655 2024-09-17T21:19:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943636+0000", 2024-09-17T21:19:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106086 2024-09-17T21:19:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943837+0000", 2024-09-17T21:19:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201826 2024-09-17T21:19:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954688+0000", 2024-09-17T21:19:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010850963 2024-09-17T21:19:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954805+0000", 2024-09-17T21:19:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000117025 2024-09-17T21:19:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954851+0000", 2024-09-17T21:19:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5065000000000003e-05 2024-09-17T21:19:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954885+0000", 2024-09-17T21:19:45.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4183000000000001e-05 2024-09-17T21:19:45.218 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.218 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.218 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.218 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.218 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.218 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:45.220 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:19:45.220 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:45.216+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:45.320 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:45.316+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:45.388 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:45.388 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:45.388 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:19:45.388 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:45.402 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:19:45.446 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:45.444+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:45.522 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:45.518+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:45.570 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:45.570 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:45.570 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:45.570 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:45.571 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:45.585 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:19:45.630 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:45.628+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:45.699 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:19:45.710 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:45.710 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:45.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:45.710 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:45.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.711 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3282 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 1572864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.711 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.882352+0000", 2024-09-17T21:19:45.711 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.824312624, 2024-09-17T21:19:45.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027540549000000001, 2024-09-17T21:19:45.711 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.711 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3282 2024-09-17T21:19:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882352+0000", 2024-09-17T21:19:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882352+0000", 2024-09-17T21:19:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882353+0000", 2024-09-17T21:19:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3e-07 2024-09-17T21:19:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882356+0000", 2024-09-17T21:19:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.163e-06 2024-09-17T21:19:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882365+0000", 2024-09-17T21:19:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9129999999999992e-06 2024-09-17T21:19:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882372+0000", 2024-09-17T21:19:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4070000000000004e-06 2024-09-17T21:19:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.909641+0000", 2024-09-17T21:19:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027268758000000001 2024-09-17T21:19:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.909753+0000", 2024-09-17T21:19:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112144 2024-09-17T21:19:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.909893+0000", 2024-09-17T21:19:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013943399999999999 2024-09-17T21:19:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3286 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 573440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.882381+0000", 2024-09-17T21:19:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8242833420000002, 2024-09-17T21:19:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027803568000000001, 2024-09-17T21:19:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3286 2024-09-17T21:19:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882381+0000", 2024-09-17T21:19:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882381+0000", 2024-09-17T21:19:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882382+0000", 2024-09-17T21:19:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6700000000000003e-07 2024-09-17T21:19:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882385+0000", 2024-09-17T21:19:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6759999999999999e-06 2024-09-17T21:19:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882393+0000", 2024-09-17T21:19:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3869999999999995e-06 2024-09-17T21:19:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882400+0000", 2024-09-17T21:19:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8630000000000002e-06 2024-09-17T21:19:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.909932+0000", 2024-09-17T21:19:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027532606000000001 2024-09-17T21:19:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.910041+0000", 2024-09-17T21:19:45.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000108047 2024-09-17T21:19:45.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.910185+0000", 2024-09-17T21:19:45.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144322 2024-09-17T21:19:45.720 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.720 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.720 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.720 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3287 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 114688~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.882411+0000", 2024-09-17T21:19:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8242539319999995, 2024-09-17T21:19:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028059356000000001, 2024-09-17T21:19:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3287 2024-09-17T21:19:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882411+0000", 2024-09-17T21:19:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882411+0000", 2024-09-17T21:19:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882411+0000", 2024-09-17T21:19:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7999999999999995e-07 2024-09-17T21:19:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882414+0000", 2024-09-17T21:19:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6699999999999998e-06 2024-09-17T21:19:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882423+0000", 2024-09-17T21:19:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5569999999999996e-06 2024-09-17T21:19:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882429+0000", 2024-09-17T21:19:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7859999999999999e-06 2024-09-17T21:19:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.910232+0000", 2024-09-17T21:19:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027802384999999999 2024-09-17T21:19:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.910347+0000", 2024-09-17T21:19:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000115118 2024-09-17T21:19:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.910470+0000", 2024-09-17T21:19:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012316000000000001 2024-09-17T21:19:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3288 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 802816~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.882439+0000", 2024-09-17T21:19:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8242259260000004, 2024-09-17T21:19:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028567248, 2024-09-17T21:19:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.727 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.727 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.727 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.727 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.727 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.727 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3288 2024-09-17T21:19:45.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.727 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882439+0000", 2024-09-17T21:19:45.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882439+0000", 2024-09-17T21:19:45.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882439+0000", 2024-09-17T21:19:45.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6700000000000003e-07 2024-09-17T21:19:45.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882442+0000", 2024-09-17T21:19:45.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7999999999999999e-06 2024-09-17T21:19:45.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882460+0000", 2024-09-17T21:19:45.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7363e-05 2024-09-17T21:19:45.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882474+0000", 2024-09-17T21:19:45.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4399999999999999e-05 2024-09-17T21:19:45.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.910686+0000", 2024-09-17T21:19:45.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028212002999999999 2024-09-17T21:19:45.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.910825+0000", 2024-09-17T21:19:45.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013920800000000001 2024-09-17T21:19:45.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.911006+0000", 2024-09-17T21:19:45.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018080700000000001 2024-09-17T21:19:45.731 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.731 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.731 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.732 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3290 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 344064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.732 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.882526+0000", 2024-09-17T21:19:45.732 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8241389110000004, 2024-09-17T21:19:45.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028845889999999999, 2024-09-17T21:19:45.732 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.732 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.732 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.733 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.733 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.733 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.733 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3290 2024-09-17T21:19:45.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.733 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882526+0000", 2024-09-17T21:19:45.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882526+0000", 2024-09-17T21:19:45.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882527+0000", 2024-09-17T21:19:45.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.294e-06 2024-09-17T21:19:45.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882532+0000", 2024-09-17T21:19:45.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4029999999999998e-06 2024-09-17T21:19:45.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882580+0000", 2024-09-17T21:19:45.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7342999999999997e-05 2024-09-17T21:19:45.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882599+0000", 2024-09-17T21:19:45.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9256e-05 2024-09-17T21:19:45.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.911060+0000", 2024-09-17T21:19:45.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028460587999999998 2024-09-17T21:19:45.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.911216+0000", 2024-09-17T21:19:45.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015654800000000001 2024-09-17T21:19:45.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.911372+0000", 2024-09-17T21:19:45.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000155458 2024-09-17T21:19:45.737 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.737 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.737 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.737 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3291 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 1032192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.737 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.882623+0000", 2024-09-17T21:19:45.737 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8240415150000002, 2024-09-17T21:19:45.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029044677000000001, 2024-09-17T21:19:45.737 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.737 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.737 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.737 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.738 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.738 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.738 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3291 2024-09-17T21:19:45.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.738 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882623+0000", 2024-09-17T21:19:45.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882623+0000", 2024-09-17T21:19:45.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882624+0000", 2024-09-17T21:19:45.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2270000000000001e-06 2024-09-17T21:19:45.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882630+0000", 2024-09-17T21:19:45.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6230000000000002e-06 2024-09-17T21:19:45.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882651+0000", 2024-09-17T21:19:45.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1449000000000001e-05 2024-09-17T21:19:45.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882665+0000", 2024-09-17T21:19:45.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3570000000000001e-05 2024-09-17T21:19:45.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.911417+0000", 2024-09-17T21:19:45.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028751826000000001 2024-09-17T21:19:45.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.911532+0000", 2024-09-17T21:19:45.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000114938 2024-09-17T21:19:45.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.911668+0000", 2024-09-17T21:19:45.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013604399999999999 2024-09-17T21:19:45.741 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.741 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.741 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.742 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3292 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.742 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.882683+0000", 2024-09-17T21:19:45.742 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8239814929999998, 2024-09-17T21:19:45.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029289778999999998, 2024-09-17T21:19:45.742 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.742 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.742 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.742 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.743 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.743 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.743 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3292 2024-09-17T21:19:45.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.743 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882683+0000", 2024-09-17T21:19:45.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882683+0000", 2024-09-17T21:19:45.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882684+0000", 2024-09-17T21:19:45.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.124e-06 2024-09-17T21:19:45.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882689+0000", 2024-09-17T21:19:45.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2460000000000003e-06 2024-09-17T21:19:45.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882706+0000", 2024-09-17T21:19:45.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.679e-05 2024-09-17T21:19:45.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882723+0000", 2024-09-17T21:19:45.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.624e-05 2024-09-17T21:19:45.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.911725+0000", 2024-09-17T21:19:45.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029002656000000002 2024-09-17T21:19:45.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.911837+0000", 2024-09-17T21:19:45.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011176200000000001 2024-09-17T21:19:45.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.911973+0000", 2024-09-17T21:19:45.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013596100000000001 2024-09-17T21:19:45.748 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.748 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.748 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.749 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3295 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 1261568~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.749 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.882742+0000", 2024-09-17T21:19:45.749 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8239230769999999, 2024-09-17T21:19:45.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029523122999999998, 2024-09-17T21:19:45.749 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.750 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.750 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.750 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.750 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.750 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.750 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3295 2024-09-17T21:19:45.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.751 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882742+0000", 2024-09-17T21:19:45.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882742+0000", 2024-09-17T21:19:45.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882743+0000", 2024-09-17T21:19:45.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.35e-06 2024-09-17T21:19:45.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882748+0000", 2024-09-17T21:19:45.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:19:45.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882768+0000", 2024-09-17T21:19:45.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.05e-05 2024-09-17T21:19:45.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882784+0000", 2024-09-17T21:19:45.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5398999999999999e-05 2024-09-17T21:19:45.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.912017+0000", 2024-09-17T21:19:45.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029233792000000001 2024-09-17T21:19:45.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.912135+0000", 2024-09-17T21:19:45.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000117378 2024-09-17T21:19:45.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.912265+0000", 2024-09-17T21:19:45.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012985399999999999 2024-09-17T21:19:45.758 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.758 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.758 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.759 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3297 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.759 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.882832+0000", 2024-09-17T21:19:45.759 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8238326210000002, 2024-09-17T21:19:45.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029711300999999999, 2024-09-17T21:19:45.759 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.760 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.760 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.760 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.760 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.760 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.760 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3297 2024-09-17T21:19:45.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.761 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882832+0000", 2024-09-17T21:19:45.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882832+0000", 2024-09-17T21:19:45.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882833+0000", 2024-09-17T21:19:45.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4959999999999999e-06 2024-09-17T21:19:45.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882839+0000", 2024-09-17T21:19:45.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7769999999999999e-06 2024-09-17T21:19:45.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882875+0000", 2024-09-17T21:19:45.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5265999999999997e-05 2024-09-17T21:19:45.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882893+0000", 2024-09-17T21:19:45.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8023e-05 2024-09-17T21:19:45.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.912309+0000", 2024-09-17T21:19:45.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029416787 2024-09-17T21:19:45.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.912421+0000", 2024-09-17T21:19:45.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011138599999999999 2024-09-17T21:19:45.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.912543+0000", 2024-09-17T21:19:45.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012256600000000001 2024-09-17T21:19:45.768 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.768 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.768 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.768 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3301 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.769 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.882919+0000", 2024-09-17T21:19:45.769 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8237453329999997, 2024-09-17T21:19:45.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.030138027000000001, 2024-09-17T21:19:45.769 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.769 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.769 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.770 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.770 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.770 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.770 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3301 2024-09-17T21:19:45.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.770 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882919+0000", 2024-09-17T21:19:45.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882919+0000", 2024-09-17T21:19:45.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882921+0000", 2024-09-17T21:19:45.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2929999999999999e-06 2024-09-17T21:19:45.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882926+0000", 2024-09-17T21:19:45.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7339999999999996e-06 2024-09-17T21:19:45.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882944+0000", 2024-09-17T21:19:45.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:19:45.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882973+0000", 2024-09-17T21:19:45.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8796e-05 2024-09-17T21:19:45.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.912633+0000", 2024-09-17T21:19:45.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029660338000000001 2024-09-17T21:19:45.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.912820+0000", 2024-09-17T21:19:45.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018654899999999999 2024-09-17T21:19:45.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.913057+0000", 2024-09-17T21:19:45.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023759399999999999 2024-09-17T21:19:45.777 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.777 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.777 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.778 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3461 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.778 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.889646+0000", 2024-09-17T21:19:45.778 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8170186279999996, 2024-09-17T21:19:45.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027582827000000001, 2024-09-17T21:19:45.779 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.779 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.779 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.779 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.779 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.779 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.779 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3461 2024-09-17T21:19:45.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.780 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.889646+0000", 2024-09-17T21:19:45.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.889646+0000", 2024-09-17T21:19:45.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.889647+0000", 2024-09-17T21:19:45.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4300000000000003e-07 2024-09-17T21:19:45.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.889649+0000", 2024-09-17T21:19:45.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3149999999999999e-06 2024-09-17T21:19:45.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.889656+0000", 2024-09-17T21:19:45.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8719999999999996e-06 2024-09-17T21:19:45.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.889661+0000", 2024-09-17T21:19:45.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6119999999999998e-06 2024-09-17T21:19:45.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.916968+0000", 2024-09-17T21:19:45.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027306674 2024-09-17T21:19:45.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.917120+0000", 2024-09-17T21:19:45.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015179000000000001 2024-09-17T21:19:45.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.917229+0000", 2024-09-17T21:19:45.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000109021 2024-09-17T21:19:45.787 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.787 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.787 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.788 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3465 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 131072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.788 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:19:45.788 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8161344960000001, 2024-09-17T21:19:45.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027910303000000001, 2024-09-17T21:19:45.789 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.789 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.789 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.789 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.789 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.789 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.789 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3465 2024-09-17T21:19:45.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.790 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:19:45.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:19:45.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890531+0000", 2024-09-17T21:19:45.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.08e-06 2024-09-17T21:19:45.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890551+0000", 2024-09-17T21:19:45.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0217e-05 2024-09-17T21:19:45.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890568+0000", 2024-09-17T21:19:45.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6575e-05 2024-09-17T21:19:45.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890580+0000", 2024-09-17T21:19:45.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2041999999999999e-05 2024-09-17T21:19:45.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.918065+0000", 2024-09-17T21:19:45.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027485328 2024-09-17T21:19:45.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.918214+0000", 2024-09-17T21:19:45.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148732 2024-09-17T21:19:45.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.918440+0000", 2024-09-17T21:19:45.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022632899999999999 2024-09-17T21:19:45.797 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.797 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.797 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.798 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3466 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.798 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.890599+0000", 2024-09-17T21:19:45.798 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8160659949999998, 2024-09-17T21:19:45.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028283418000000001, 2024-09-17T21:19:45.798 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.799 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.799 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.799 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.799 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.799 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.799 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3466 2024-09-17T21:19:45.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.800 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890599+0000", 2024-09-17T21:19:45.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890599+0000", 2024-09-17T21:19:45.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890600+0000", 2024-09-17T21:19:45.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.192e-06 2024-09-17T21:19:45.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890605+0000", 2024-09-17T21:19:45.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9579999999999998e-06 2024-09-17T21:19:45.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890620+0000", 2024-09-17T21:19:45.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759999999999999e-05 2024-09-17T21:19:45.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890631+0000", 2024-09-17T21:19:45.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1588e-05 2024-09-17T21:19:45.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.918499+0000", 2024-09-17T21:19:45.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027867507999999999 2024-09-17T21:19:45.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.918652+0000", 2024-09-17T21:19:45.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015290200000000001 2024-09-17T21:19:45.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.918882+0000", 2024-09-17T21:19:45.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023050999999999999 2024-09-17T21:19:45.807 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.807 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.807 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.808 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3468 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 360448~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.808 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.890869+0000", 2024-09-17T21:19:45.808 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8157955240000003, 2024-09-17T21:19:45.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02851074, 2024-09-17T21:19:45.808 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.809 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.809 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.809 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.809 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.809 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.809 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3468 2024-09-17T21:19:45.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.810 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890869+0000", 2024-09-17T21:19:45.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.811 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890869+0000", 2024-09-17T21:19:45.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890871+0000", 2024-09-17T21:19:45.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4929999999999999e-06 2024-09-17T21:19:45.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890877+0000", 2024-09-17T21:19:45.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9269999999999999e-06 2024-09-17T21:19:45.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890897+0000", 2024-09-17T21:19:45.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9323000000000001e-05 2024-09-17T21:19:45.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890910+0000", 2024-09-17T21:19:45.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.348e-05 2024-09-17T21:19:45.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.919052+0000", 2024-09-17T21:19:45.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028141301000000001 2024-09-17T21:19:45.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.919211+0000", 2024-09-17T21:19:45.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000159362 2024-09-17T21:19:45.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.919380+0000", 2024-09-17T21:19:45.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016885399999999999 2024-09-17T21:19:45.814 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.814 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.814 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.814 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3469 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.815 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.890932+0000", 2024-09-17T21:19:45.815 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8157327500000005, 2024-09-17T21:19:45.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028996609999999999, 2024-09-17T21:19:45.815 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.815 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.815 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.815 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.815 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.815 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.815 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3469 2024-09-17T21:19:45.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.815 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890932+0000", 2024-09-17T21:19:45.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890932+0000", 2024-09-17T21:19:45.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890933+0000", 2024-09-17T21:19:45.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.108e-06 2024-09-17T21:19:45.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890938+0000", 2024-09-17T21:19:45.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0549999999999999e-06 2024-09-17T21:19:45.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890960+0000", 2024-09-17T21:19:45.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1861999999999998e-05 2024-09-17T21:19:45.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890972+0000", 2024-09-17T21:19:45.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2068e-05 2024-09-17T21:19:45.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.919443+0000", 2024-09-17T21:19:45.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028471314000000001 2024-09-17T21:19:45.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.919602+0000", 2024-09-17T21:19:45.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015866200000000001 2024-09-17T21:19:45.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.919928+0000", 2024-09-17T21:19:45.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032654100000000002 2024-09-17T21:19:45.819 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.819 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.819 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.819 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3470 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 1736704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.819 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.890988+0000", 2024-09-17T21:19:45.819 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8156766879999999, 2024-09-17T21:19:45.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029367607, 2024-09-17T21:19:45.819 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.819 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.819 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.819 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.819 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.820 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.820 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3470 2024-09-17T21:19:45.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.820 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890988+0000", 2024-09-17T21:19:45.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890988+0000", 2024-09-17T21:19:45.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890989+0000", 2024-09-17T21:19:45.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0800000000000003e-07 2024-09-17T21:19:45.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890993+0000", 2024-09-17T21:19:45.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1200000000000004e-06 2024-09-17T21:19:45.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891007+0000", 2024-09-17T21:19:45.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3845e-05 2024-09-17T21:19:45.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891018+0000", 2024-09-17T21:19:45.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.084e-05 2024-09-17T21:19:45.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.920029+0000", 2024-09-17T21:19:45.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029010904000000001 2024-09-17T21:19:45.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.920182+0000", 2024-09-17T21:19:45.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000153704 2024-09-17T21:19:45.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.920356+0000", 2024-09-17T21:19:45.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017328600000000001 2024-09-17T21:19:45.823 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.823 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.823 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.823 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3473 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 1277952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.823 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.891033+0000", 2024-09-17T21:19:45.823 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.815631496, 2024-09-17T21:19:45.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029577203, 2024-09-17T21:19:45.824 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.824 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.824 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.824 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.824 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.824 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.824 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3473 2024-09-17T21:19:45.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.824 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891033+0000", 2024-09-17T21:19:45.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891033+0000", 2024-09-17T21:19:45.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891034+0000", 2024-09-17T21:19:45.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:19:45.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891038+0000", 2024-09-17T21:19:45.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3069999999999996e-06 2024-09-17T21:19:45.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891053+0000", 2024-09-17T21:19:45.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4208e-05 2024-09-17T21:19:45.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891064+0000", 2024-09-17T21:19:45.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1209999999999999e-05 2024-09-17T21:19:45.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.920434+0000", 2024-09-17T21:19:45.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029370250000000001 2024-09-17T21:19:45.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.920516+0000", 2024-09-17T21:19:45.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2455999999999998e-05 2024-09-17T21:19:45.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.920610+0000", 2024-09-17T21:19:45.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3834999999999999e-05 2024-09-17T21:19:45.827 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.827 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.828 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.828 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3475 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 1966080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.828 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.891083+0000", 2024-09-17T21:19:45.828 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8155816920000003, 2024-09-17T21:19:45.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029842481000000001, 2024-09-17T21:19:45.828 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.828 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.828 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.828 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.828 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.828 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.828 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3475 2024-09-17T21:19:45.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.829 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891083+0000", 2024-09-17T21:19:45.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891083+0000", 2024-09-17T21:19:45.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891084+0000", 2024-09-17T21:19:45.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2500000000000004e-07 2024-09-17T21:19:45.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891088+0000", 2024-09-17T21:19:45.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3950000000000003e-06 2024-09-17T21:19:45.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891102+0000", 2024-09-17T21:19:45.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4260000000000001e-05 2024-09-17T21:19:45.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891114+0000", 2024-09-17T21:19:45.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1127999999999999e-05 2024-09-17T21:19:45.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.920653+0000", 2024-09-17T21:19:45.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029539329 2024-09-17T21:19:45.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.920737+0000", 2024-09-17T21:19:45.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4512999999999995e-05 2024-09-17T21:19:45.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.920925+0000", 2024-09-17T21:19:45.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018793099999999999 2024-09-17T21:19:45.832 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.832 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.832 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.832 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3476 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 1507328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.832 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.891129+0000", 2024-09-17T21:19:45.832 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8155354570000002, 2024-09-17T21:19:45.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.030044952, 2024-09-17T21:19:45.832 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.832 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.833 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:45.833 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.833 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.833 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.833 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3476 2024-09-17T21:19:45.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.833 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.833 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.833 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891129+0000", 2024-09-17T21:19:45.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.833 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891129+0000", 2024-09-17T21:19:45.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891130+0000", 2024-09-17T21:19:45.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:19:45.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891134+0000", 2024-09-17T21:19:45.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1370000000000004e-06 2024-09-17T21:19:45.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891148+0000", 2024-09-17T21:19:45.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3740999999999999e-05 2024-09-17T21:19:45.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891159+0000", 2024-09-17T21:19:45.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0628999999999999e-05 2024-09-17T21:19:45.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.920965+0000", 2024-09-17T21:19:45.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029805940999999999 2024-09-17T21:19:45.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.921058+0000", 2024-09-17T21:19:45.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3642999999999998e-05 2024-09-17T21:19:45.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.921174+0000", 2024-09-17T21:19:45.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000115864 2024-09-17T21:19:45.836 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.836 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.836 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.837 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3709 3.1c 3:3f968e74:::rbd_data.1152a8ae2019.0000000000000048:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:45.837 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.897224+0000", 2024-09-17T21:19:45.837 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8094406330000004, 2024-09-17T21:19:45.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033869676000000001, 2024-09-17T21:19:45.837 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:45.837 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:45.837 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:45.837 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:45.837 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:45.837 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:45.837 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3709 2024-09-17T21:19:45.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.837 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:45.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:45.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897224+0000", 2024-09-17T21:19:45.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:45.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897224+0000", 2024-09-17T21:19:45.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:45.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:45.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897225+0000", 2024-09-17T21:19:45.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4079999999999999e-06 2024-09-17T21:19:45.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:45.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897233+0000", 2024-09-17T21:19:45.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9500000000000001e-06 2024-09-17T21:19:45.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:45.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897251+0000", 2024-09-17T21:19:45.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8108e-05 2024-09-17T21:19:45.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:45.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897265+0000", 2024-09-17T21:19:45.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3531e-05 2024-09-17T21:19:45.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:45.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922304+0000", 2024-09-17T21:19:45.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025038867999999999 2024-09-17T21:19:45.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:45.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922413+0000", 2024-09-17T21:19:45.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000109648 2024-09-17T21:19:45.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:45.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922738+0000", 2024-09-17T21:19:45.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032488200000000002 2024-09-17T21:19:45.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:45.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.928631+0000", 2024-09-17T21:19:45.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0058926430000000004 2024-09-17T21:19:45.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:45.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.930969+0000", 2024-09-17T21:19:45.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0023378320000000002 2024-09-17T21:19:45.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:45.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.931028+0000", 2024-09-17T21:19:45.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9262999999999998e-05 2024-09-17T21:19:45.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:45.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:45.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:45.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.931094+0000", 2024-09-17T21:19:45.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5543000000000006e-05 2024-09-17T21:19:45.842 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.842 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.842 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.842 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:45.842 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:45.842 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:45.844 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:19:45.844 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:45.840+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:45.847 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:19:45.945 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:45.942+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:45.984 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:45.984 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:46.000 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:46.000 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:46.016 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:46.016 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:46.018 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:46.018 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:46.018 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:19:46.018 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:46.032 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:19:46.037 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:46.038 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:46.046 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:46.042+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:46.056 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:46.056 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:46.073 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:46.074 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:46.146 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:46.142+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:46.155 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:46.155 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:46.169 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:19:46.248 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:46.246+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:46.331 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:46.331 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:46.332 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:19:46.376 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:46.374+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:46.449 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:46.444+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:46.462 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:46.462 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:46.478 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:19:46.550 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:46.547+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:46.645 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:46.645 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:46.651 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:46.646+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:46.659 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:19:46.680 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-17T21:19:46.734 INFO:teuthology.orchestra.run.smithi184.stdout:3921 write 0x6169400 thru 0x616965e (0x25f bytes) 2024-09-17T21:19:46.736 INFO:teuthology.orchestra.run.smithi184.stdout:3922 write 0x2bcf200 thru 0x2bd22b9 (0x30ba bytes) 2024-09-17T21:19:46.736 INFO:teuthology.orchestra.run.smithi184.stdout:3923 read 0x236600 thru 0x237656 (0x1057 bytes) 2024-09-17T21:19:46.737 INFO:teuthology.orchestra.run.smithi184.stdout:3924 read 0x636e800 thru 0x637ac26 (0xc427 bytes) 2024-09-17T21:19:46.739 INFO:teuthology.orchestra.run.smithi184.stdout:3927 write 0x6e06c00 thru 0x6e11911 (0xad12 bytes) 2024-09-17T21:19:46.740 INFO:teuthology.orchestra.run.smithi184.stdout:3928 read 0x5656200 thru 0x565965a (0x345b bytes) 2024-09-17T21:19:46.752 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:46.747+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:46.825 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:46.825 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:46.843 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:19:46.888 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:46.886+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:46.954 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:46.950+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:46.968 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:46.968 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:46.984 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:19:47.054 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:47.051+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:47.096 INFO:teuthology.orchestra.run.smithi184.stdout:3931 write 0xc30ca00 thru 0xc30cdab (0x3ac bytes) 2024-09-17T21:19:47.100 INFO:teuthology.orchestra.run.smithi184.stdout:3932 trunc from 0xc30ce00 to 0xab9ee00 2024-09-17T21:19:47.133 INFO:teuthology.orchestra.run.smithi184.stdout:3934 read 0x6251e00 thru 0x625660d (0x480e bytes) 2024-09-17T21:19:47.136 INFO:teuthology.orchestra.run.smithi184.stdout:3937 read 0xab70e00 thru 0xab801ae (0xf3af bytes) 2024-09-17T21:19:47.137 INFO:teuthology.orchestra.run.smithi184.stdout:3938 write 0x77c2a00 thru 0x77cac46 (0x8247 bytes) 2024-09-17T21:19:47.138 INFO:teuthology.orchestra.run.smithi184.stdout:3941 write 0x9d24c00 thru 0x9d269f9 (0x1dfa bytes) 2024-09-17T21:19:47.138 INFO:teuthology.orchestra.run.smithi184.stdout:3942 write 0x9b54a00 thru 0x9b572f8 (0x28f9 bytes) 2024-09-17T21:19:47.138 INFO:teuthology.orchestra.run.smithi184.stdout:3943 read 0x381000 thru 0x385102 (0x4103 bytes) 2024-09-17T21:19:47.140 INFO:teuthology.orchestra.run.smithi184.stdout:3944 write 0x7b68200 thru 0x7b77719 (0xf51a bytes) 2024-09-17T21:19:47.140 INFO:teuthology.orchestra.run.smithi184.stdout:3945 punch from 0x2265000 to 0x226b400, (0x6400 bytes) 2024-09-17T21:19:47.152 INFO:teuthology.orchestra.run.smithi184.stdout:3946 trunc from 0xab9ee00 to 0xa887600 2024-09-17T21:19:47.155 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:47.151+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:47.172 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:47.173 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:47.191 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:19:47.256 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:47.251+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:47.267 INFO:teuthology.orchestra.run.smithi184.stdout:3948 read 0x10e600 thru 0x112444 (0x3e45 bytes) 2024-09-17T21:19:47.269 INFO:teuthology.orchestra.run.smithi184.stdout:3953 write 0x61ab800 thru 0x61b8504 (0xcd05 bytes) 2024-09-17T21:19:47.270 INFO:teuthology.orchestra.run.smithi184.stdout:3954 read 0x2bf4c00 thru 0x2bf863c (0x3a3d bytes) 2024-09-17T21:19:47.271 INFO:teuthology.orchestra.run.smithi184.stdout:3957 read 0x7ae9600 thru 0x7af78e2 (0xe2e3 bytes) 2024-09-17T21:19:47.273 INFO:teuthology.orchestra.run.smithi184.stdout:3958 write 0x56b1a00 thru 0x56bfa11 (0xe012 bytes) 2024-09-17T21:19:47.274 INFO:teuthology.orchestra.run.smithi184.stdout:3960 punch from 0x3067800 to 0x3073400, (0xbc00 bytes) 2024-09-17T21:19:47.277 INFO:teuthology.orchestra.run.smithi184.stdout:3961 read 0x8294600 thru 0x8297807 (0x3208 bytes) 2024-09-17T21:19:47.289 INFO:teuthology.orchestra.run.smithi184.stdout:3964 read 0x13f5e00 thru 0x13fe5d5 (0x87d6 bytes) 2024-09-17T21:19:47.289 INFO:teuthology.orchestra.run.smithi184.stdout:3967 write 0x1f0be00 thru 0x1f0fe89 (0x408a bytes) 2024-09-17T21:19:47.292 INFO:teuthology.orchestra.run.smithi184.stdout:3970 read 0x6e9b200 thru 0x6ea5a47 (0xa848 bytes) 2024-09-17T21:19:47.319 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:47.319 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:47.333 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:19:47.381 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:47.378+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:47.448 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:47.448 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:47.459 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:47.455+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:47.462 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:19:47.560 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:47.554+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:47.633 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:47.633 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:47.648 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:19:47.693 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:47.690+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:47.762 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:47.758+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:47.812 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:47.812 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:47.827 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:19:47.875 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:47.871+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:47.959 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:47.959 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:47.964 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:47.958+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:47.974 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:19:48.065 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:48.059+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:48.091 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:19:48.141 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:48.141 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:48.155 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:19:48.166 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:48.162+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:48.246 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:48.246 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:48.263 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:48.263 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:48.266 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:48.263+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:48.280 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:48.280 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:48.297 INFO:teuthology.orchestra.run.smithi184.stdout:3971 write 0x857e00 thru 0x867bd4 (0xfdd5 bytes) 2024-09-17T21:19:48.303 INFO:teuthology.orchestra.run.smithi184.stdout:3977 punch from 0x8fa7800 to 0x8faa000, (0x2800 bytes) 2024-09-17T21:19:48.304 INFO:teuthology.orchestra.run.smithi184.stdout:3978 read 0x52cfc00 thru 0x52da426 (0xa827 bytes) 2024-09-17T21:19:48.305 INFO:teuthology.orchestra.run.smithi184.stdout:3980 write 0x92cf200 thru 0x92d0bd1 (0x19d2 bytes) 2024-09-17T21:19:48.305 INFO:teuthology.orchestra.run.smithi184.stdout:3981 read 0x8ce0e00 thru 0x8ceabdd (0x9dde bytes) 2024-09-17T21:19:48.306 INFO:teuthology.orchestra.run.smithi184.stdout:3983 write 0x3d10600 thru 0x3d1766f (0x7070 bytes) 2024-09-17T21:19:48.306 INFO:teuthology.orchestra.run.smithi184.stdout:3987 write 0x8b23200 thru 0x8b2518a (0x1f8b bytes) 2024-09-17T21:19:48.309 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:48.309 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:48.334 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:48.335 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:48.337 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:19:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.31927469599999, 2024-09-17T21:19:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:19:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:19:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:19:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:19:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:19:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:19:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:19:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:19:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:19:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:19:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:19:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:19:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:19:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:19:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:19:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:19:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:19:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:19:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:19:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:19:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:19:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:19:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:19:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:19:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:19:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:19:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:19:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:19:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:19:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:19:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:19:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 91.739110416000003, 2024-09-17T21:19:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:19:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:19:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:19:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:19:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:19:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:19:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:19:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:19:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:19:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:19:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:19:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:19:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:19:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:19:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:19:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:19:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:19:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:19:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:19:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:19:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:19:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:19:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:19:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:19:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:19:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:19:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 91.739013103999994, 2024-09-17T21:19:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:19:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:19:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:19:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:19:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:19:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:19:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:19:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:19:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:19:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:19:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:19:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:19:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:19:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:19:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:19:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:19:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:19:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:19:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:19:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:19:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:19:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:19:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:19:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:19:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:19:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:19:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:19:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:19:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:19:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 91.736926303000004, 2024-09-17T21:19:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:19:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:19:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:19:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:19:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:19:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:19:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:19:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:19:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:19:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:19:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:19:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:19:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:19:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:19:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:19:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:19:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:19:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:19:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:19:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:19:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:19:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:19:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:19:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:19:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:19:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:19:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:19:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:19:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:19:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:19:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:19:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:19:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:19:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:19:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:19:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.053027522000001, 2024-09-17T21:19:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:19:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:19:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:19:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:19:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:19:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:19:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:19:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:19:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:19:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:19:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:19:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:19:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:19:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:19:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:19:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:19:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:19:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:19:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:19:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:19:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:19:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:19:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:19:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:19:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.043935810999997, 2024-09-17T21:19:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:19:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:19:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:19:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:19:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:19:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:19:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:19:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:19:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:19:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:19:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:19:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:19:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:19:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:19:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:19:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:19:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:19:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:19:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:19:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:19:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.043763759999997, 2024-09-17T21:19:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:19:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:19:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:19:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:19:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:19:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:19:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:19:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:19:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:19:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:19:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:19:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:19:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:19:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:19:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:19:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:19:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:19:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:19:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:19:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:19:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 58.868831012999998, 2024-09-17T21:19:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:19:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:19:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:19:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:19:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:19:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:19:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:19:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:19:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:19:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:19:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:19:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:19:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:19:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:19:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:19:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:19:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:19:48.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:19:48.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:19:48.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:19:48.387 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.387 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 58.868746455999997, 2024-09-17T21:19:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:19:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:19:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:19:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:19:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:19:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:19:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:19:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:19:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:19:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:19:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:19:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:19:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:19:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:19:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:19:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:19:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:19:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:19:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:19:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 58.868680818999998, 2024-09-17T21:19:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:19:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:19:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:19:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:19:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:19:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:19:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:19:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:19:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:19:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:19:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:19:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:19:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:19:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:19:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:19:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:19:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:19:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:19:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:19:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:19:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:19:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.4951057219999999, 2024-09-17T21:19:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:19:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:19:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:19:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:19:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:19:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:19:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:19:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:19:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:19:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:19:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:19:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:19:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:19:48.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:19:48.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:19:48.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:19:48.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:19:48.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:19:48.402 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.403 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.403 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.403 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:48.403 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:19:48.403 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.4949862100000004, 2024-09-17T21:19:48.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:19:48.403 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.403 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.403 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:48.403 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.403 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:48.403 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:48.404 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:19:48.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.404 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.404 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.404 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:19:48.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.404 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.404 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:19:48.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:19:48.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:19:48.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:19:48.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:19:48.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:19:48.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:19:48.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:19:48.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:19:48.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:19:48.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:19:48.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:19:48.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:19:48.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:19:48.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:19:48.407 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.407 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.407 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.407 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2695 3.15 3:aedfd032:::rbd_data.1152a8ae2019.0000000000000041:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:48.407 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.843199+0000", 2024-09-17T21:19:48.407 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.4898409060000004, 2024-09-17T21:19:48.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079730569000000001, 2024-09-17T21:19:48.408 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.408 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.408 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:48.408 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.408 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:48.408 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:48.408 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2695 2024-09-17T21:19:48.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.408 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.408 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843199+0000", 2024-09-17T21:19:48.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843199+0000", 2024-09-17T21:19:48.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843201+0000", 2024-09-17T21:19:48.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0899999999999999e-06 2024-09-17T21:19:48.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843214+0000", 2024-09-17T21:19:48.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2233e-05 2024-09-17T21:19:48.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843245+0000", 2024-09-17T21:19:48.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1464000000000002e-05 2024-09-17T21:19:48.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843266+0000", 2024-09-17T21:19:48.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0879000000000001e-05 2024-09-17T21:19:48.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847920+0000", 2024-09-17T21:19:48.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.004654292 2024-09-17T21:19:48.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.848240+0000", 2024-09-17T21:19:48.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031994499999999997 2024-09-17T21:19:48.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922930+0000", 2024-09-17T21:19:48.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.074689666000000002 2024-09-17T21:19:48.411 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.411 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.412 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.412 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2732 3.15 3:ab9a4731:::rbd_data.1152a8ae2019.0000000000000058:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:48.412 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.846909+0000", 2024-09-17T21:19:48.412 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.4861308600000003, 2024-09-17T21:19:48.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076145476000000004, 2024-09-17T21:19:48.412 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.412 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.412 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:48.412 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.412 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:48.412 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:48.413 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2732 2024-09-17T21:19:48.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.413 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846909+0000", 2024-09-17T21:19:48.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846909+0000", 2024-09-17T21:19:48.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846911+0000", 2024-09-17T21:19:48.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.31e-06 2024-09-17T21:19:48.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846916+0000", 2024-09-17T21:19:48.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.023e-06 2024-09-17T21:19:48.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846931+0000", 2024-09-17T21:19:48.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5549999999999999e-05 2024-09-17T21:19:48.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846943+0000", 2024-09-17T21:19:48.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1783000000000001e-05 2024-09-17T21:19:48.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.850592+0000", 2024-09-17T21:19:48.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036486159999999999 2024-09-17T21:19:48.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.850870+0000", 2024-09-17T21:19:48.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027864200000000001 2024-09-17T21:19:48.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.923055+0000", 2024-09-17T21:19:48.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.072184551999999999 2024-09-17T21:19:48.416 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.416 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.416 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.416 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2737 3.15 3:a8c4e696:::rbd_data.1152a8ae2019.000000000000005b:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:48.417 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.847014+0000", 2024-09-17T21:19:48.417 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.4860257250000002, 2024-09-17T21:19:48.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076158271, 2024-09-17T21:19:48.417 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.417 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.417 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:48.417 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.417 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:48.417 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:48.417 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2737 2024-09-17T21:19:48.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.417 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847014+0000", 2024-09-17T21:19:48.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847014+0000", 2024-09-17T21:19:48.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847015+0000", 2024-09-17T21:19:48.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0699999999999999e-06 2024-09-17T21:19:48.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847021+0000", 2024-09-17T21:19:48.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.446e-06 2024-09-17T21:19:48.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847038+0000", 2024-09-17T21:19:48.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7257000000000001e-05 2024-09-17T21:19:48.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847051+0000", 2024-09-17T21:19:48.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2649999999999999e-05 2024-09-17T21:19:48.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.852127+0000", 2024-09-17T21:19:48.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050757420000000003 2024-09-17T21:19:48.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.852423+0000", 2024-09-17T21:19:48.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029608100000000001 2024-09-17T21:19:48.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.923173+0000", 2024-09-17T21:19:48.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.070750024999999994 2024-09-17T21:19:48.421 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.421 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.421 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.421 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3397 3.8 3:174f9d43:::rbd_data.1152a8ae2019.000000000000006e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:48.421 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.897797+0000", 2024-09-17T21:19:48.421 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.4352427800000003, 2024-09-17T21:19:48.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045478192000000001, 2024-09-17T21:19:48.421 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.421 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.421 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:48.421 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.422 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:48.422 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:48.422 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3397 2024-09-17T21:19:48.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.422 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.422 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897797+0000", 2024-09-17T21:19:48.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.422 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897797+0000", 2024-09-17T21:19:48.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897799+0000", 2024-09-17T21:19:48.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.238e-06 2024-09-17T21:19:48.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897806+0000", 2024-09-17T21:19:48.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5900000000000002e-06 2024-09-17T21:19:48.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897822+0000", 2024-09-17T21:19:48.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5659e-05 2024-09-17T21:19:48.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897832+0000", 2024-09-17T21:19:48.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0034000000000001e-05 2024-09-17T21:19:48.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.931249+0000", 2024-09-17T21:19:48.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033416700000000001 2024-09-17T21:19:48.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.931425+0000", 2024-09-17T21:19:48.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017629500000000001 2024-09-17T21:19:48.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943276+0000", 2024-09-17T21:19:48.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011850675999999999 2024-09-17T21:19:48.425 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.425 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.425 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.426 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3524 3.8 3:119ba024:::rbd_data.1152c6ce51b2.0000000000000005:17 [read 1261568~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:48.426 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.901789+0000", 2024-09-17T21:19:48.426 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.4312508529999999, 2024-09-17T21:19:48.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.042862082000000003, 2024-09-17T21:19:48.426 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.426 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.426 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:48.426 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.426 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:48.426 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:48.426 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3524 2024-09-17T21:19:48.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.426 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.426 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901789+0000", 2024-09-17T21:19:48.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901789+0000", 2024-09-17T21:19:48.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901790+0000", 2024-09-17T21:19:48.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0400000000000005e-07 2024-09-17T21:19:48.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901795+0000", 2024-09-17T21:19:48.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3839999999999999e-06 2024-09-17T21:19:48.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901807+0000", 2024-09-17T21:19:48.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2532e-05 2024-09-17T21:19:48.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901816+0000", 2024-09-17T21:19:48.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2220000000000008e-06 2024-09-17T21:19:48.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.944407+0000", 2024-09-17T21:19:48.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.042591088999999999 2024-09-17T21:19:48.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.944498+0000", 2024-09-17T21:19:48.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0883000000000004e-05 2024-09-17T21:19:48.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.944651+0000", 2024-09-17T21:19:48.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015306799999999999 2024-09-17T21:19:48.430 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.430 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.430 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.430 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3679 3.8 3:129b2dec:::rbd_data.1152a8ae2019.0000000000000012:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:48.430 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.908354+0000", 2024-09-17T21:19:48.430 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.4246864920000002, 2024-09-17T21:19:48.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046195880000000002, 2024-09-17T21:19:48.430 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.430 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.430 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:48.431 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.431 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:48.431 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:48.431 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3679 2024-09-17T21:19:48.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.431 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908354+0000", 2024-09-17T21:19:48.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908354+0000", 2024-09-17T21:19:48.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908355+0000", 2024-09-17T21:19:48.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9999999999999995e-07 2024-09-17T21:19:48.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908361+0000", 2024-09-17T21:19:48.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7399999999999998e-06 2024-09-17T21:19:48.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908375+0000", 2024-09-17T21:19:48.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.363e-05 2024-09-17T21:19:48.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908386+0000", 2024-09-17T21:19:48.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.06e-05 2024-09-17T21:19:48.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.945600+0000", 2024-09-17T21:19:48.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.037214199000000003 2024-09-17T21:19:48.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.945650+0000", 2024-09-17T21:19:48.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9737999999999997e-05 2024-09-17T21:19:48.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:48.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.945799+0000", 2024-09-17T21:19:48.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014921699999999999 2024-09-17T21:19:48.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:48.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954371+0000", 2024-09-17T21:19:48.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085719730000000001 2024-09-17T21:19:48.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:48.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954460+0000", 2024-09-17T21:19:48.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9596000000000001e-05 2024-09-17T21:19:48.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:48.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954511+0000", 2024-09-17T21:19:48.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0455999999999999e-05 2024-09-17T21:19:48.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954550+0000", 2024-09-17T21:19:48.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8730999999999999e-05 2024-09-17T21:19:48.436 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.436 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.436 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.436 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3703 3.13 3:cdb4d88a:::rbd_data.1152a8ae2019.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:48.436 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.908402+0000", 2024-09-17T21:19:48.436 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.4246384519999999, 2024-09-17T21:19:48.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046393446999999997, 2024-09-17T21:19:48.436 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.436 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.436 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:48.436 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.437 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:48.437 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:48.437 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3703 2024-09-17T21:19:48.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.437 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908402+0000", 2024-09-17T21:19:48.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908402+0000", 2024-09-17T21:19:48.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908403+0000", 2024-09-17T21:19:48.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0529999999999999e-06 2024-09-17T21:19:48.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908409+0000", 2024-09-17T21:19:48.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2219999999999999e-06 2024-09-17T21:19:48.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908422+0000", 2024-09-17T21:19:48.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3147e-05 2024-09-17T21:19:48.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908432+0000", 2024-09-17T21:19:48.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.039e-05 2024-09-17T21:19:48.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.942217+0000", 2024-09-17T21:19:48.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033784994999999998 2024-09-17T21:19:48.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.942296+0000", 2024-09-17T21:19:48.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8218000000000002e-05 2024-09-17T21:19:48.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:48.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943003+0000", 2024-09-17T21:19:48.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000707403 2024-09-17T21:19:48.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:48.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954635+0000", 2024-09-17T21:19:48.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011631786999999999 2024-09-17T21:19:48.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:48.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954706+0000", 2024-09-17T21:19:48.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1064999999999998e-05 2024-09-17T21:19:48.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:48.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954757+0000", 2024-09-17T21:19:48.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0760000000000002e-05 2024-09-17T21:19:48.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954795+0000", 2024-09-17T21:19:48.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8407e-05 2024-09-17T21:19:48.441 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.441 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.442 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.442 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3711 3.13 3:ca5a0e5e:::rbd_data.1152a8ae2019.0000000000000040:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:48.442 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.908514+0000", 2024-09-17T21:19:48.442 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.4245262890000001, 2024-09-17T21:19:48.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046370871000000001, 2024-09-17T21:19:48.442 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.442 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.442 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:48.442 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.442 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:48.442 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:48.442 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3711 2024-09-17T21:19:48.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.443 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908514+0000", 2024-09-17T21:19:48.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908514+0000", 2024-09-17T21:19:48.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908515+0000", 2024-09-17T21:19:48.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0449999999999999e-06 2024-09-17T21:19:48.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908521+0000", 2024-09-17T21:19:48.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2179999999999996e-06 2024-09-17T21:19:48.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908535+0000", 2024-09-17T21:19:48.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4124999999999999e-05 2024-09-17T21:19:48.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908548+0000", 2024-09-17T21:19:48.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2680000000000001e-05 2024-09-17T21:19:48.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943530+0000", 2024-09-17T21:19:48.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034981655 2024-09-17T21:19:48.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943636+0000", 2024-09-17T21:19:48.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106086 2024-09-17T21:19:48.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:48.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943837+0000", 2024-09-17T21:19:48.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201826 2024-09-17T21:19:48.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:48.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954688+0000", 2024-09-17T21:19:48.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010850963 2024-09-17T21:19:48.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:48.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954805+0000", 2024-09-17T21:19:48.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000117025 2024-09-17T21:19:48.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:48.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954851+0000", 2024-09-17T21:19:48.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5065000000000003e-05 2024-09-17T21:19:48.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954885+0000", 2024-09-17T21:19:48.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4183000000000001e-05 2024-09-17T21:19:48.447 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.447 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.447 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.447 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.447 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.448 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:48.448 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:48.450 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:19:48.450 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:48.443+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:48.450 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:48.549 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:48.543+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:48.617 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:48.617 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:48.617 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:19:48.617 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:48.631 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:19:48.675 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:48.671+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:48.713 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:19:48.751 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:48.747+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:48.799 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:48.799 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:48.799 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:48.799 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:48.799 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:48.813 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:19:48.851 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:48.846+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:48.953 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:48.950+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:48.973 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:48.973 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:48.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:48.974 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:48.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.974 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3286 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 573440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:48.974 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.882381+0000", 2024-09-17T21:19:48.974 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.0875867190000008, 2024-09-17T21:19:48.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027803568000000001, 2024-09-17T21:19:48.974 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.974 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.974 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:48.974 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.974 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:48.974 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:48.974 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3286 2024-09-17T21:19:48.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.975 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.975 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.975 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882381+0000", 2024-09-17T21:19:48.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.975 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.975 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882381+0000", 2024-09-17T21:19:48.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.975 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.976 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882382+0000", 2024-09-17T21:19:48.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6700000000000003e-07 2024-09-17T21:19:48.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.976 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882385+0000", 2024-09-17T21:19:48.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6759999999999999e-06 2024-09-17T21:19:48.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.976 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882393+0000", 2024-09-17T21:19:48.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3869999999999995e-06 2024-09-17T21:19:48.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882400+0000", 2024-09-17T21:19:48.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8630000000000002e-06 2024-09-17T21:19:48.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.909932+0000", 2024-09-17T21:19:48.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027532606000000001 2024-09-17T21:19:48.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.910041+0000", 2024-09-17T21:19:48.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000108047 2024-09-17T21:19:48.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.910185+0000", 2024-09-17T21:19:48.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144322 2024-09-17T21:19:48.978 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.978 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.978 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.978 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3287 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 114688~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:48.978 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.882411+0000", 2024-09-17T21:19:48.978 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.0875573089999993, 2024-09-17T21:19:48.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028059356000000001, 2024-09-17T21:19:48.978 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.979 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.979 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:48.979 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.979 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:48.979 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:48.979 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3287 2024-09-17T21:19:48.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.979 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882411+0000", 2024-09-17T21:19:48.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882411+0000", 2024-09-17T21:19:48.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882411+0000", 2024-09-17T21:19:48.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7999999999999995e-07 2024-09-17T21:19:48.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882414+0000", 2024-09-17T21:19:48.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6699999999999998e-06 2024-09-17T21:19:48.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882423+0000", 2024-09-17T21:19:48.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5569999999999996e-06 2024-09-17T21:19:48.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882429+0000", 2024-09-17T21:19:48.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7859999999999999e-06 2024-09-17T21:19:48.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.910232+0000", 2024-09-17T21:19:48.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027802384999999999 2024-09-17T21:19:48.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.910347+0000", 2024-09-17T21:19:48.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000115118 2024-09-17T21:19:48.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.910470+0000", 2024-09-17T21:19:48.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012316000000000001 2024-09-17T21:19:48.982 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.982 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.982 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.983 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3288 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 802816~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:48.983 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.882439+0000", 2024-09-17T21:19:48.983 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.0875293030000002, 2024-09-17T21:19:48.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028567248, 2024-09-17T21:19:48.983 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.983 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.983 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:48.983 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.983 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:48.983 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:48.983 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3288 2024-09-17T21:19:48.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.983 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882439+0000", 2024-09-17T21:19:48.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882439+0000", 2024-09-17T21:19:48.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882439+0000", 2024-09-17T21:19:48.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6700000000000003e-07 2024-09-17T21:19:48.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882442+0000", 2024-09-17T21:19:48.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7999999999999999e-06 2024-09-17T21:19:48.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882460+0000", 2024-09-17T21:19:48.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7363e-05 2024-09-17T21:19:48.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.986 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882474+0000", 2024-09-17T21:19:48.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4399999999999999e-05 2024-09-17T21:19:48.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.986 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.910686+0000", 2024-09-17T21:19:48.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028212002999999999 2024-09-17T21:19:48.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.986 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.910825+0000", 2024-09-17T21:19:48.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013920800000000001 2024-09-17T21:19:48.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.911006+0000", 2024-09-17T21:19:48.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018080700000000001 2024-09-17T21:19:48.987 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.987 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.987 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.987 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3290 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 344064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:48.987 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.882526+0000", 2024-09-17T21:19:48.987 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.0874422880000001, 2024-09-17T21:19:48.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028845889999999999, 2024-09-17T21:19:48.987 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.987 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.988 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:48.988 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.988 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:48.988 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:48.988 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3290 2024-09-17T21:19:48.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.988 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882526+0000", 2024-09-17T21:19:48.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882526+0000", 2024-09-17T21:19:48.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882527+0000", 2024-09-17T21:19:48.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.294e-06 2024-09-17T21:19:48.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882532+0000", 2024-09-17T21:19:48.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4029999999999998e-06 2024-09-17T21:19:48.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882580+0000", 2024-09-17T21:19:48.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7342999999999997e-05 2024-09-17T21:19:48.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882599+0000", 2024-09-17T21:19:48.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9256e-05 2024-09-17T21:19:48.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.911060+0000", 2024-09-17T21:19:48.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028460587999999998 2024-09-17T21:19:48.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.911216+0000", 2024-09-17T21:19:48.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015654800000000001 2024-09-17T21:19:48.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.911372+0000", 2024-09-17T21:19:48.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000155458 2024-09-17T21:19:48.991 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.991 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.991 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.992 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3291 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 1032192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:48.992 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.882623+0000", 2024-09-17T21:19:48.992 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.0873448920000008, 2024-09-17T21:19:48.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029044677000000001, 2024-09-17T21:19:48.992 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.992 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.992 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:48.992 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.992 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:48.992 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:48.992 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3291 2024-09-17T21:19:48.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.992 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882623+0000", 2024-09-17T21:19:48.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882623+0000", 2024-09-17T21:19:48.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882624+0000", 2024-09-17T21:19:48.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2270000000000001e-06 2024-09-17T21:19:48.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882630+0000", 2024-09-17T21:19:48.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6230000000000002e-06 2024-09-17T21:19:48.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882651+0000", 2024-09-17T21:19:48.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1449000000000001e-05 2024-09-17T21:19:48.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882665+0000", 2024-09-17T21:19:48.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3570000000000001e-05 2024-09-17T21:19:48.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.911417+0000", 2024-09-17T21:19:48.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028751826000000001 2024-09-17T21:19:48.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:48.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.911532+0000", 2024-09-17T21:19:48.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000114938 2024-09-17T21:19:48.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:48.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.911668+0000", 2024-09-17T21:19:48.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013604399999999999 2024-09-17T21:19:48.996 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.996 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:48.996 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:48.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.996 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3292 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:48.996 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.882683+0000", 2024-09-17T21:19:48.996 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.0872848699999995, 2024-09-17T21:19:48.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029289778999999998, 2024-09-17T21:19:48.996 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:48.996 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:48.996 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:48.996 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:48.997 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:48.997 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:48.997 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3292 2024-09-17T21:19:48.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.997 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:48.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:48.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882683+0000", 2024-09-17T21:19:48.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:48.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882683+0000", 2024-09-17T21:19:48.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:48.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:48.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882684+0000", 2024-09-17T21:19:48.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.124e-06 2024-09-17T21:19:48.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:48.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882689+0000", 2024-09-17T21:19:48.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2460000000000003e-06 2024-09-17T21:19:48.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:48.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882706+0000", 2024-09-17T21:19:48.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.679e-05 2024-09-17T21:19:48.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:48.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882723+0000", 2024-09-17T21:19:48.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.624e-05 2024-09-17T21:19:48.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:48.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.911725+0000", 2024-09-17T21:19:48.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029002656000000002 2024-09-17T21:19:48.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:48.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:48.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.911837+0000", 2024-09-17T21:19:49.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011176200000000001 2024-09-17T21:19:49.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.911973+0000", 2024-09-17T21:19:49.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013596100000000001 2024-09-17T21:19:49.000 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.000 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.000 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.000 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3295 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 1261568~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:49.000 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.882742+0000", 2024-09-17T21:19:49.001 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.0872264539999996, 2024-09-17T21:19:49.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029523122999999998, 2024-09-17T21:19:49.001 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.001 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.001 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.001 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.001 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:49.001 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:49.001 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3295 2024-09-17T21:19:49.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.001 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882742+0000", 2024-09-17T21:19:49.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882742+0000", 2024-09-17T21:19:49.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882743+0000", 2024-09-17T21:19:49.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.35e-06 2024-09-17T21:19:49.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882748+0000", 2024-09-17T21:19:49.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:19:49.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882768+0000", 2024-09-17T21:19:49.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.05e-05 2024-09-17T21:19:49.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882784+0000", 2024-09-17T21:19:49.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5398999999999999e-05 2024-09-17T21:19:49.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.004 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.912017+0000", 2024-09-17T21:19:49.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029233792000000001 2024-09-17T21:19:49.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.004 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.912135+0000", 2024-09-17T21:19:49.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000117378 2024-09-17T21:19:49.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.004 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.912265+0000", 2024-09-17T21:19:49.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012985399999999999 2024-09-17T21:19:49.004 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.005 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.005 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.005 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3297 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:49.005 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.882832+0000", 2024-09-17T21:19:49.005 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.0871359980000008, 2024-09-17T21:19:49.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029711300999999999, 2024-09-17T21:19:49.005 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.005 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.005 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.005 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.005 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:49.005 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:49.006 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3297 2024-09-17T21:19:49.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.006 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882832+0000", 2024-09-17T21:19:49.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882832+0000", 2024-09-17T21:19:49.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882833+0000", 2024-09-17T21:19:49.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4959999999999999e-06 2024-09-17T21:19:49.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882839+0000", 2024-09-17T21:19:49.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7769999999999999e-06 2024-09-17T21:19:49.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882875+0000", 2024-09-17T21:19:49.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5265999999999997e-05 2024-09-17T21:19:49.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882893+0000", 2024-09-17T21:19:49.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8023e-05 2024-09-17T21:19:49.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.912309+0000", 2024-09-17T21:19:49.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029416787 2024-09-17T21:19:49.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.912421+0000", 2024-09-17T21:19:49.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011138599999999999 2024-09-17T21:19:49.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.912543+0000", 2024-09-17T21:19:49.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012256600000000001 2024-09-17T21:19:49.009 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.009 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.009 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.009 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3301 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:49.009 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.882919+0000", 2024-09-17T21:19:49.009 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.0870487099999995, 2024-09-17T21:19:49.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.030138027000000001, 2024-09-17T21:19:49.010 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.010 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.010 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.010 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.010 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:49.010 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:49.010 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3301 2024-09-17T21:19:49.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.010 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882919+0000", 2024-09-17T21:19:49.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882919+0000", 2024-09-17T21:19:49.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882921+0000", 2024-09-17T21:19:49.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2929999999999999e-06 2024-09-17T21:19:49.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882926+0000", 2024-09-17T21:19:49.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7339999999999996e-06 2024-09-17T21:19:49.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882944+0000", 2024-09-17T21:19:49.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:19:49.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.882973+0000", 2024-09-17T21:19:49.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8796e-05 2024-09-17T21:19:49.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.912633+0000", 2024-09-17T21:19:49.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029660338000000001 2024-09-17T21:19:49.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.912820+0000", 2024-09-17T21:19:49.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018654899999999999 2024-09-17T21:19:49.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.913057+0000", 2024-09-17T21:19:49.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023759399999999999 2024-09-17T21:19:49.013 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.013 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.013 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.014 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3461 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:49.014 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.889646+0000", 2024-09-17T21:19:49.014 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.0803220049999993, 2024-09-17T21:19:49.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027582827000000001, 2024-09-17T21:19:49.014 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.014 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.014 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.014 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.014 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:49.014 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:49.014 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3461 2024-09-17T21:19:49.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.014 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.889646+0000", 2024-09-17T21:19:49.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.889646+0000", 2024-09-17T21:19:49.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.889647+0000", 2024-09-17T21:19:49.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4300000000000003e-07 2024-09-17T21:19:49.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.889649+0000", 2024-09-17T21:19:49.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3149999999999999e-06 2024-09-17T21:19:49.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.889656+0000", 2024-09-17T21:19:49.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8719999999999996e-06 2024-09-17T21:19:49.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.889661+0000", 2024-09-17T21:19:49.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6119999999999998e-06 2024-09-17T21:19:49.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.916968+0000", 2024-09-17T21:19:49.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027306674 2024-09-17T21:19:49.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.917120+0000", 2024-09-17T21:19:49.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015179000000000001 2024-09-17T21:19:49.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.917229+0000", 2024-09-17T21:19:49.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000109021 2024-09-17T21:19:49.018 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.018 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.018 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.018 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3465 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 131072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:49.018 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:19:49.018 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.0794378729999998, 2024-09-17T21:19:49.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027910303000000001, 2024-09-17T21:19:49.018 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.018 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.019 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.019 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.019 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:49.019 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:49.019 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3465 2024-09-17T21:19:49.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.019 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:19:49.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:19:49.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890531+0000", 2024-09-17T21:19:49.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.08e-06 2024-09-17T21:19:49.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890551+0000", 2024-09-17T21:19:49.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0217e-05 2024-09-17T21:19:49.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890568+0000", 2024-09-17T21:19:49.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6575e-05 2024-09-17T21:19:49.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890580+0000", 2024-09-17T21:19:49.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2041999999999999e-05 2024-09-17T21:19:49.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.918065+0000", 2024-09-17T21:19:49.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027485328 2024-09-17T21:19:49.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.918214+0000", 2024-09-17T21:19:49.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148732 2024-09-17T21:19:49.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.918440+0000", 2024-09-17T21:19:49.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022632899999999999 2024-09-17T21:19:49.022 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.022 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.022 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.023 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3466 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:49.023 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.890599+0000", 2024-09-17T21:19:49.023 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.0793693720000004, 2024-09-17T21:19:49.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028283418000000001, 2024-09-17T21:19:49.023 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.023 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.023 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.023 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.023 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:49.023 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:49.023 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3466 2024-09-17T21:19:49.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.023 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890599+0000", 2024-09-17T21:19:49.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890599+0000", 2024-09-17T21:19:49.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890600+0000", 2024-09-17T21:19:49.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.192e-06 2024-09-17T21:19:49.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890605+0000", 2024-09-17T21:19:49.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9579999999999998e-06 2024-09-17T21:19:49.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890620+0000", 2024-09-17T21:19:49.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759999999999999e-05 2024-09-17T21:19:49.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890631+0000", 2024-09-17T21:19:49.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1588e-05 2024-09-17T21:19:49.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.918499+0000", 2024-09-17T21:19:49.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027867507999999999 2024-09-17T21:19:49.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.918652+0000", 2024-09-17T21:19:49.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015290200000000001 2024-09-17T21:19:49.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.918882+0000", 2024-09-17T21:19:49.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023050999999999999 2024-09-17T21:19:49.027 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.027 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.027 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.027 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3468 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 360448~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:49.027 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.890869+0000", 2024-09-17T21:19:49.027 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.0790989010000001, 2024-09-17T21:19:49.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02851074, 2024-09-17T21:19:49.027 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.027 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.027 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.027 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.027 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:49.028 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:49.028 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3468 2024-09-17T21:19:49.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.028 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890869+0000", 2024-09-17T21:19:49.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890869+0000", 2024-09-17T21:19:49.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890871+0000", 2024-09-17T21:19:49.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4929999999999999e-06 2024-09-17T21:19:49.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890877+0000", 2024-09-17T21:19:49.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9269999999999999e-06 2024-09-17T21:19:49.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890897+0000", 2024-09-17T21:19:49.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9323000000000001e-05 2024-09-17T21:19:49.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890910+0000", 2024-09-17T21:19:49.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.348e-05 2024-09-17T21:19:49.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.919052+0000", 2024-09-17T21:19:49.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028141301000000001 2024-09-17T21:19:49.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.919211+0000", 2024-09-17T21:19:49.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000159362 2024-09-17T21:19:49.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.919380+0000", 2024-09-17T21:19:49.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016885399999999999 2024-09-17T21:19:49.031 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.031 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.031 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.031 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3469 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:49.031 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.890932+0000", 2024-09-17T21:19:49.032 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.0790361270000002, 2024-09-17T21:19:49.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028996609999999999, 2024-09-17T21:19:49.032 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.032 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.032 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.032 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.032 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:49.032 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:49.032 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3469 2024-09-17T21:19:49.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.032 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890932+0000", 2024-09-17T21:19:49.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890932+0000", 2024-09-17T21:19:49.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890933+0000", 2024-09-17T21:19:49.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.108e-06 2024-09-17T21:19:49.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890938+0000", 2024-09-17T21:19:49.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0549999999999999e-06 2024-09-17T21:19:49.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890960+0000", 2024-09-17T21:19:49.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1861999999999998e-05 2024-09-17T21:19:49.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890972+0000", 2024-09-17T21:19:49.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2068e-05 2024-09-17T21:19:49.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.919443+0000", 2024-09-17T21:19:49.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028471314000000001 2024-09-17T21:19:49.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.919602+0000", 2024-09-17T21:19:49.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015866200000000001 2024-09-17T21:19:49.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.919928+0000", 2024-09-17T21:19:49.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032654100000000002 2024-09-17T21:19:49.035 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.035 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.036 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.036 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3470 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 1736704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:49.036 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.890988+0000", 2024-09-17T21:19:49.036 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.0789800649999997, 2024-09-17T21:19:49.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029367607, 2024-09-17T21:19:49.036 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.036 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.036 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.036 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.036 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:49.036 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:49.036 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3470 2024-09-17T21:19:49.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.037 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890988+0000", 2024-09-17T21:19:49.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890988+0000", 2024-09-17T21:19:49.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890989+0000", 2024-09-17T21:19:49.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0800000000000003e-07 2024-09-17T21:19:49.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.890993+0000", 2024-09-17T21:19:49.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1200000000000004e-06 2024-09-17T21:19:49.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891007+0000", 2024-09-17T21:19:49.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3845e-05 2024-09-17T21:19:49.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891018+0000", 2024-09-17T21:19:49.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.084e-05 2024-09-17T21:19:49.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.920029+0000", 2024-09-17T21:19:49.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029010904000000001 2024-09-17T21:19:49.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.920182+0000", 2024-09-17T21:19:49.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000153704 2024-09-17T21:19:49.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.920356+0000", 2024-09-17T21:19:49.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017328600000000001 2024-09-17T21:19:49.040 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.040 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.040 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.040 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3473 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 1277952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:49.040 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.891033+0000", 2024-09-17T21:19:49.040 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.0789348729999997, 2024-09-17T21:19:49.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029577203, 2024-09-17T21:19:49.040 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.040 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.041 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.041 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.041 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:49.041 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:49.041 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3473 2024-09-17T21:19:49.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.041 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891033+0000", 2024-09-17T21:19:49.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891033+0000", 2024-09-17T21:19:49.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891034+0000", 2024-09-17T21:19:49.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:19:49.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891038+0000", 2024-09-17T21:19:49.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3069999999999996e-06 2024-09-17T21:19:49.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891053+0000", 2024-09-17T21:19:49.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4208e-05 2024-09-17T21:19:49.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891064+0000", 2024-09-17T21:19:49.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1209999999999999e-05 2024-09-17T21:19:49.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.920434+0000", 2024-09-17T21:19:49.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029370250000000001 2024-09-17T21:19:49.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.920516+0000", 2024-09-17T21:19:49.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2455999999999998e-05 2024-09-17T21:19:49.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.920610+0000", 2024-09-17T21:19:49.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3834999999999999e-05 2024-09-17T21:19:49.044 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.044 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.044 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.045 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3475 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 1966080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:49.045 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.891083+0000", 2024-09-17T21:19:49.045 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.078885069, 2024-09-17T21:19:49.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029842481000000001, 2024-09-17T21:19:49.045 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.045 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.045 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.045 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.045 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:49.045 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:49.045 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3475 2024-09-17T21:19:49.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.045 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891083+0000", 2024-09-17T21:19:49.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891083+0000", 2024-09-17T21:19:49.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891084+0000", 2024-09-17T21:19:49.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2500000000000004e-07 2024-09-17T21:19:49.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891088+0000", 2024-09-17T21:19:49.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3950000000000003e-06 2024-09-17T21:19:49.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891102+0000", 2024-09-17T21:19:49.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4260000000000001e-05 2024-09-17T21:19:49.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891114+0000", 2024-09-17T21:19:49.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1127999999999999e-05 2024-09-17T21:19:49.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.920653+0000", 2024-09-17T21:19:49.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029539329 2024-09-17T21:19:49.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.920737+0000", 2024-09-17T21:19:49.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4512999999999995e-05 2024-09-17T21:19:49.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.920925+0000", 2024-09-17T21:19:49.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018793099999999999 2024-09-17T21:19:49.049 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.049 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.049 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.049 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3476 3.1c 3:3ef14268:::rbd_data.1152c6ce51b2.0000000000000002:17 [read 1507328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:49.049 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.891129+0000", 2024-09-17T21:19:49.049 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.0788388340000008, 2024-09-17T21:19:49.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.030044952, 2024-09-17T21:19:49.049 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.049 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.049 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.049 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.050 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:49.050 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:49.050 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3476 2024-09-17T21:19:49.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.050 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891129+0000", 2024-09-17T21:19:49.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891129+0000", 2024-09-17T21:19:49.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891130+0000", 2024-09-17T21:19:49.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:19:49.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891134+0000", 2024-09-17T21:19:49.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1370000000000004e-06 2024-09-17T21:19:49.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891148+0000", 2024-09-17T21:19:49.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3740999999999999e-05 2024-09-17T21:19:49.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.891159+0000", 2024-09-17T21:19:49.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0628999999999999e-05 2024-09-17T21:19:49.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.920965+0000", 2024-09-17T21:19:49.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029805940999999999 2024-09-17T21:19:49.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.921058+0000", 2024-09-17T21:19:49.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3642999999999998e-05 2024-09-17T21:19:49.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.921174+0000", 2024-09-17T21:19:49.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000115864 2024-09-17T21:19:49.053 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.053 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.053 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.053 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3709 3.1c 3:3f968e74:::rbd_data.1152a8ae2019.0000000000000048:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:49.053 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.897224+0000", 2024-09-17T21:19:49.054 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.0727440099999992, 2024-09-17T21:19:49.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033869676000000001, 2024-09-17T21:19:49.054 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.054 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.054 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:49.054 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.054 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:49.054 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:49.054 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3709 2024-09-17T21:19:49.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.054 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897224+0000", 2024-09-17T21:19:49.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897224+0000", 2024-09-17T21:19:49.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897225+0000", 2024-09-17T21:19:49.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4079999999999999e-06 2024-09-17T21:19:49.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897233+0000", 2024-09-17T21:19:49.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9500000000000001e-06 2024-09-17T21:19:49.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897251+0000", 2024-09-17T21:19:49.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8108e-05 2024-09-17T21:19:49.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897265+0000", 2024-09-17T21:19:49.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3531e-05 2024-09-17T21:19:49.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922304+0000", 2024-09-17T21:19:49.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025038867999999999 2024-09-17T21:19:49.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922413+0000", 2024-09-17T21:19:49.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000109648 2024-09-17T21:19:49.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:49.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922738+0000", 2024-09-17T21:19:49.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032488200000000002 2024-09-17T21:19:49.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:49.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.928631+0000", 2024-09-17T21:19:49.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0058926430000000004 2024-09-17T21:19:49.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:49.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.930969+0000", 2024-09-17T21:19:49.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0023378320000000002 2024-09-17T21:19:49.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:49.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.931028+0000", 2024-09-17T21:19:49.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9262999999999998e-05 2024-09-17T21:19:49.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.931094+0000", 2024-09-17T21:19:49.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5543000000000006e-05 2024-09-17T21:19:49.059 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.059 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.059 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.059 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1613 3.1e e96/75 3:7d13ef1b:::rbd_data.115266d0b9f2.0000000000000000:head v 96'240, mlcod=91'238)", 2024-09-17T21:19:49.059 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:48.331412+0000", 2024-09-17T21:19:49.059 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.63855575099999995, 2024-09-17T21:19:49.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029103786999999999, 2024-09-17T21:19:49.060 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.060 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.060 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:49.060 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:48.331412+0000", 2024-09-17T21:19:49.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:48.331412+0000", 2024-09-17T21:19:49.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:48.331413+0000", 2024-09-17T21:19:49.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8699999999999995e-07 2024-09-17T21:19:49.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:48.342333+0000", 2024-09-17T21:19:49.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010920334 2024-09-17T21:19:49.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:48.342373+0000", 2024-09-17T21:19:49.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9960999999999999e-05 2024-09-17T21:19:49.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:48.342397+0000", 2024-09-17T21:19:49.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3683e-05 2024-09-17T21:19:49.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:48.342498+0000", 2024-09-17T21:19:49.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101135 2024-09-17T21:19:49.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:48.342530+0000", 2024-09-17T21:19:49.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1795999999999998e-05 2024-09-17T21:19:49.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:49.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:48.360418+0000", 2024-09-17T21:19:49.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017888134999999999 2024-09-17T21:19:49.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:48.360516+0000", 2024-09-17T21:19:49.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8256000000000003e-05 2024-09-17T21:19:49.063 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.063 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.063 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.064 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.064 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.064 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:49.065 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:19:49.065 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:49.059+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:49.165 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:49.159+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:49.233 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:49.233 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:49.233 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:19:49.233 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:49.247 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:19:49.266 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:49.262+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:49.289 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-17T21:19:49.307 INFO:teuthology.orchestra.run.smithi184.stdout:3989 punch from 0x61e2200 to 0x61f1000, (0xee00 bytes) 2024-09-17T21:19:49.321 INFO:teuthology.orchestra.run.smithi184.stdout:3990 punch from 0x9bd5a00 to 0x9bdbc00, (0x6200 bytes) 2024-09-17T21:19:49.321 INFO:teuthology.orchestra.run.smithi184.stdout:3991 read 0x7448200 thru 0x744a794 (0x2595 bytes) 2024-09-17T21:19:49.322 INFO:teuthology.orchestra.run.smithi184.stdout:3992 write 0x499e400 thru 0x49a9d98 (0xb999 bytes) 2024-09-17T21:19:49.323 INFO:teuthology.orchestra.run.smithi184.stdout:3993 read 0x1bf6800 thru 0x1bff5c2 (0x8dc3 bytes) 2024-09-17T21:19:49.328 INFO:teuthology.orchestra.run.smithi184.stdout:3995 write 0x10c6000 thru 0x10cbd69 (0x5d6a bytes) 2024-09-17T21:19:49.329 INFO:teuthology.orchestra.run.smithi184.stdout:3996 read 0x1010800 thru 0x10171be (0x69bf bytes) 2024-09-17T21:19:49.367 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:49.363+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:49.406 INFO:teuthology.orchestra.run.smithi184.stdout:3998 write 0xb611800 thru 0xb6181c3 (0x69c4 bytes) 2024-09-17T21:19:49.408 INFO:teuthology.orchestra.run.smithi184.stdout:3999 punch from 0x38dd800 to 0x38e9c00, (0xc400 bytes) 2024-09-17T21:19:49.409 INFO:teuthology.orchestra.run.smithi184.stdout:4001 read 0x4e22200 thru 0x4e26ce8 (0x4ae9 bytes) 2024-09-17T21:19:49.411 INFO:teuthology.orchestra.run.smithi184.stdout:4002 read 0x363da00 thru 0x36485ac (0xabad bytes) 2024-09-17T21:19:49.411 INFO:teuthology.orchestra.run.smithi184.stdout:4004 write 0x59a8600 thru 0x59b3571 (0xaf72 bytes) 2024-09-17T21:19:49.412 INFO:teuthology.orchestra.run.smithi184.stdout:4005 punch from 0xa966200 to 0xa968000, (0x1e00 bytes) 2024-09-17T21:19:49.413 INFO:teuthology.orchestra.run.smithi184.stdout:4006 read 0x669400 thru 0x678168 (0xed69 bytes) 2024-09-17T21:19:49.416 INFO:teuthology.orchestra.run.smithi184.stdout:4010 read 0xb024000 thru 0xb02f64f (0xb650 bytes) 2024-09-17T21:19:49.436 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:49.436 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:49.436 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:49.436 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:49.436 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:49.450 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:19:49.464 INFO:teuthology.orchestra.run.smithi184.stdout:4011 write 0xe0dbc00 thru 0xe0e4a50 (0x8e51 bytes) 2024-09-17T21:19:49.466 INFO:teuthology.orchestra.run.smithi184.stdout:4014 write 0x4b91e00 thru 0x4ba04cb (0xe6cc bytes) 2024-09-17T21:19:49.467 INFO:teuthology.orchestra.run.smithi184.stdout:4015 punch from 0xb363400 to 0xb368c00, (0x5800 bytes) 2024-09-17T21:19:49.468 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:49.462+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:49.469 INFO:teuthology.orchestra.run.smithi184.stdout:4016 read 0x67d0e00 thru 0x67d919d (0x839e bytes) 2024-09-17T21:19:49.470 INFO:teuthology.orchestra.run.smithi184.stdout:4017 write 0x46ad200 thru 0x46af49d (0x229e bytes) 2024-09-17T21:19:49.471 INFO:teuthology.orchestra.run.smithi184.stdout:4018 write 0x838200 thru 0x8411d1 (0x8fd2 bytes) 2024-09-17T21:19:49.472 INFO:teuthology.orchestra.run.smithi184.stdout:4020 write 0x1aa6600 thru 0x1aa7a89 (0x148a bytes) 2024-09-17T21:19:49.473 INFO:teuthology.orchestra.run.smithi184.stdout:4021 write 0xbcb5a00 thru 0xbcbbb15 (0x6116 bytes) 2024-09-17T21:19:49.473 INFO:teuthology.orchestra.run.smithi184.stdout:4022 write 0x150c600 thru 0x15139a9 (0x73aa bytes) 2024-09-17T21:19:49.474 INFO:teuthology.orchestra.run.smithi184.stdout:4025 punch from 0x3885400 to 0x3888c00, (0x3800 bytes) 2024-09-17T21:19:49.498 INFO:teuthology.orchestra.run.smithi184.stdout:4027 punch from 0xc16d200 to 0xc16dc00, (0xa00 bytes) 2024-09-17T21:19:49.499 INFO:teuthology.orchestra.run.smithi184.stdout:4028 clone 25 order 18 su 8192 sc 6 2024-09-17T21:19:49.569 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:49.563+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:49.633 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:49.633 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:49.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:49.634 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:49.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.634 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:19:49.634 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:49.634 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 88.026819345999996, 2024-09-17T21:19:49.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:19:49.634 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.634 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.634 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:49.634 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:49.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:49.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:19:49.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:19:49.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:19:49.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:19:49.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:19:49.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:19:49.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:19:49.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:19:49.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:19:49.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:19:49.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:19:49.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:19:49.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:49.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:19:49.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:19:49.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:19:49.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:19:49.638 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.638 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.638 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.639 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:19:49.639 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:49.639 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 88.016872710000001, 2024-09-17T21:19:49.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:19:49.639 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.639 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.639 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:49.639 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:49.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:49.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:19:49.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:19:49.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:19:49.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:19:49.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:19:49.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:19:49.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:19:49.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:19:49.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:19:49.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:19:49.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:19:49.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:19:49.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:49.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:19:49.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:19:49.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:19:49.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:19:49.643 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.643 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.643 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.643 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:19:49.643 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:49.643 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 86.773721870000003, 2024-09-17T21:19:49.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:19:49.643 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.643 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.643 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:49.643 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.644 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:19:49.644 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:19:49.644 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:19:49.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.644 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:49.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:49.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:19:49.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:19:49.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:19:49.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:19:49.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:19:49.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:19:49.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:19:49.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:19:49.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:19:49.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:19:49.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:19:49.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:19:49.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:49.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:19:49.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:19:49.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:49.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:19:49.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:19:49.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:49.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:19:49.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:19:49.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:49.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:19:49.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:19:49.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:19:49.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:19:49.649 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.649 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.649 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.649 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:19:49.649 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:49.649 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 86.764076384000006, 2024-09-17T21:19:49.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:19:49.650 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.650 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.650 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:49.650 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.650 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:19:49.650 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:19:49.650 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:19:49.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.650 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:49.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:49.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:19:49.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:19:49.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:19:49.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:19:49.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:19:49.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:19:49.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:19:49.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:19:49.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:19:49.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:19:49.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:19:49.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:19:49.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:49.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:19:49.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:19:49.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:49.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:19:49.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:19:49.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:49.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:19:49.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:19:49.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:49.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:19:49.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:19:49.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:19:49.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:19:49.655 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.655 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.655 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.655 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:49.655 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:49.656 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 73.395140350999995, 2024-09-17T21:19:49.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:19:49.656 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.656 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.656 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.656 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.656 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:49.656 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:49.656 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:19:49.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.656 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:49.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:49.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:19:49.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:19:49.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:19:49.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:19:49.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:19:49.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:19:49.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:19:49.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:19:49.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:19:49.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:19:49.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:19:49.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:19:49.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:19:49.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:19:49.659 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.660 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.660 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.660 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:819 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5111808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:49.660 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:49.660 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.193568061000001, 2024-09-17T21:19:49.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024344811000000001, 2024-09-17T21:19:49.660 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.660 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.660 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:49.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:49.661 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 819 2024-09-17T21:19:49.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.661 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:49.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:49.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437443+0000", 2024-09-17T21:19:49.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1969999999999999e-06 2024-09-17T21:19:49.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437448+0000", 2024-09-17T21:19:49.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9490000000000004e-06 2024-09-17T21:19:49.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437467+0000", 2024-09-17T21:19:49.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8862999999999999e-05 2024-09-17T21:19:49.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437480+0000", 2024-09-17T21:19:49.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2833000000000001e-05 2024-09-17T21:19:49.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461552+0000", 2024-09-17T21:19:49.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024072695000000002 2024-09-17T21:19:49.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461639+0000", 2024-09-17T21:19:49.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6996000000000005e-05 2024-09-17T21:19:49.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461787+0000", 2024-09-17T21:19:49.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014727800000000001 2024-09-17T21:19:49.664 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.664 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.664 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.664 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:49.664 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:49.665 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.193455788999998, 2024-09-17T21:19:49.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:19:49.665 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.665 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.665 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.665 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.665 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:49.665 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:49.665 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:19:49.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.665 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:49.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:49.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:19:49.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:19:49.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:19:49.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:19:49.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:19:49.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:19:49.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:19:49.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:19:49.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:19:49.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:19:49.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:19:49.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:19:49.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:19:49.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:19:49.669 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.669 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.669 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.669 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:833 3.5 3:a772dfb1:::rbd_data.11523651b015.0000000000000001:d [sparse-read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:49.669 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:49.669 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.193021502999997, 2024-09-17T21:19:49.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024216024999999999, 2024-09-17T21:19:49.669 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.669 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.669 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.669 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.669 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:49.670 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:49.670 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 833 2024-09-17T21:19:49.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.670 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:49.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:49.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437990+0000", 2024-09-17T21:19:49.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2500000000000001e-06 2024-09-17T21:19:49.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437995+0000", 2024-09-17T21:19:49.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1240000000000004e-06 2024-09-17T21:19:49.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438012+0000", 2024-09-17T21:19:49.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7572e-05 2024-09-17T21:19:49.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438031+0000", 2024-09-17T21:19:49.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8612000000000001e-05 2024-09-17T21:19:49.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461830+0000", 2024-09-17T21:19:49.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023799135999999999 2024-09-17T21:19:49.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462005+0000", 2024-09-17T21:19:49.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000174747 2024-09-17T21:19:49.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:19:49.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019958399999999999 2024-09-17T21:19:49.673 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.673 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.673 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.673 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:49.673 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:49.674 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.192961212, 2024-09-17T21:19:49.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:19:49.674 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.674 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.674 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.674 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.674 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:49.674 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:49.674 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:19:49.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.674 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:49.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:49.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:19:49.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:19:49.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:19:49.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:19:49.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:19:49.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:19:49.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:19:49.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:19:49.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:19:49.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:19:49.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:19:49.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:19:49.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:19:49.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:19:49.677 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.677 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.678 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.678 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:49.678 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:49.678 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.192905830999997, 2024-09-17T21:19:49.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:19:49.678 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.678 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.678 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.678 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.678 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:49.678 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:49.678 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:19:49.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.679 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:49.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:49.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:19:49.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:19:49.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:19:49.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:19:49.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:19:49.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:19:49.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:19:49.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:19:49.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:19:49.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:19:49.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:19:49.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:19:49.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:19:49.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:19:49.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.682 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.682 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:49.682 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:49.682 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.192849553000002, 2024-09-17T21:19:49.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:19:49.682 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.683 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.683 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:49.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:49.683 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:19:49.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.683 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:49.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:49.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:19:49.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:19:49.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:19:49.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:19:49.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:19:49.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:19:49.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:19:49.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:19:49.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:19:49.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:19:49.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:19:49.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:19:49.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:19:49.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:19:49.686 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.686 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.686 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.687 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:49.687 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:49.687 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.191899913999997, 2024-09-17T21:19:49.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:19:49.687 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.687 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.687 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.687 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.687 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:49.687 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:49.687 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:19:49.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.687 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:49.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:49.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:19:49.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:19:49.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:19:49.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:19:49.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:19:49.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:19:49.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:19:49.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:19:49.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:19:49.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:19:49.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:19:49.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:19:49.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:19:49.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:19:49.691 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.691 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.691 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.691 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:49.691 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:49.691 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.191821294, 2024-09-17T21:19:49.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:19:49.691 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.691 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.692 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:49.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:49.692 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:19:49.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.692 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:49.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:49.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:19:49.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:19:49.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:19:49.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:19:49.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:19:49.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:19:49.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:19:49.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:19:49.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:19:49.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:19:49.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:19:49.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:19:49.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:19:49.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:19:49.697 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.697 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.697 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.697 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:49.697 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:49.697 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.191708355000003, 2024-09-17T21:19:49.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:19:49.697 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.697 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.698 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.698 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.698 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:49.698 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:49.698 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:19:49.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.698 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:49.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:49.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:19:49.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:19:49.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:19:49.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:19:49.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:19:49.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:19:49.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:19:49.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:19:49.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:19:49.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:19:49.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:19:49.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:19:49.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:19:49.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:19:49.701 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.701 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.701 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.702 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:849 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5963776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:49.702 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:49.702 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.191114319, 2024-09-17T21:19:49.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024537514, 2024-09-17T21:19:49.702 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.702 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.702 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:49.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:49.702 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 849 2024-09-17T21:19:49.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.702 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:49.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:49.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439897+0000", 2024-09-17T21:19:49.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9099999999999999e-06 2024-09-17T21:19:49.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439905+0000", 2024-09-17T21:19:49.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0870000000000007e-06 2024-09-17T21:19:49.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439924+0000", 2024-09-17T21:19:49.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9042000000000002e-05 2024-09-17T21:19:49.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439941+0000", 2024-09-17T21:19:49.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.624e-05 2024-09-17T21:19:49.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464132+0000", 2024-09-17T21:19:49.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024191579000000001 2024-09-17T21:19:49.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464291+0000", 2024-09-17T21:19:49.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015904800000000001 2024-09-17T21:19:49.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464433+0000", 2024-09-17T21:19:49.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014160799999999999 2024-09-17T21:19:49.706 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.706 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.706 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.706 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:49.706 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:49.706 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.191047300999998, 2024-09-17T21:19:49.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:19:49.706 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.706 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.706 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.707 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.707 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:49.707 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:49.707 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:19:49.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.707 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:49.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:49.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:19:49.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:19:49.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:19:49.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:19:49.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:19:49.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:19:49.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:19:49.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:19:49.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:19:49.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:19:49.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:19:49.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:19:49.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:19:49.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:19:49.710 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.710 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.710 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.711 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:49.711 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:49.711 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.190982490000003, 2024-09-17T21:19:49.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:19:49.711 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.711 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.711 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.711 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.711 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:49.711 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:49.711 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:19:49.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.711 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:49.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:49.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:19:49.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:19:49.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:19:49.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:19:49.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:19:49.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:19:49.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:19:49.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:19:49.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:19:49.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:19:49.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:19:49.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:19:49.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:19:49.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:19:49.715 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.715 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.715 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.715 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:49.715 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:49.715 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.190921512999999, 2024-09-17T21:19:49.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:19:49.715 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.715 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.715 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:49.715 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:49.716 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:49.716 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:49.716 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:19:49.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.716 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:49.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:49.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:19:49.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:19:49.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:19:49.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:19:49.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:19:49.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:19:49.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:19:49.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:19:49.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:19:49.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:19:49.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:19:49.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:19:49.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:19:49.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:19:49.719 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.719 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.719 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.719 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:19:49.719 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:19:49.719 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.29335032799999999, 2024-09-17T21:19:49.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:19:49.720 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.720 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.720 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:49.720 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:19:49.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:19:49.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:19:49.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:19:49.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:19:49.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:19:49.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:19:49.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:19:49.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:19:49.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:19:49.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:19:49.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:19:49.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:19:49.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:19:49.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:49.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:19:49.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:19:49.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:19:49.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:19:49.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.724 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.724 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.724 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:19:49.724 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:19:49.724 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.084691933999999996, 2024-09-17T21:19:49.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:19:49.724 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:49.724 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:49.724 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:49.724 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:49.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:49.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:19:49.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:49.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:19:49.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:49.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:49.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:19:49.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:19:49.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:49.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:19:49.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:19:49.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:49.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:19:49.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:19:49.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:49.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:19:49.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:19:49.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:49.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:19:49.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:19:49.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:49.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:19:49.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:19:49.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:49.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:19:49.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:19:49.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:49.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:49.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:49.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:19:49.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:19:49.728 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.728 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.728 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.728 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:49.728 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:49.728 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:49.729 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 0] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:19:49.729 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:19:49.729 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-17T21:19:49.729 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T21:19:49.730 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:49.726+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:49.731 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:19:49.830 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:49.826+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:49.856 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:49.856 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:49.856 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:19:49.857 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:49.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:19:49.931 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:49.927+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:50.032 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:50.026+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:50.035 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:50.036 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:50.036 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:50.036 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:50.036 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:50.042 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:19:50.042 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:50.043 INFO:teuthology.orchestra.run.smithi083.stdout: "epoch": 97, 2024-09-17T21:19:50.043 INFO:teuthology.orchestra.run.smithi083.stdout: "fsid": "224a7a29-e112-472e-8a13-f67345d6d123", 2024-09-17T21:19:50.043 INFO:teuthology.orchestra.run.smithi083.stdout: "created": "2024-09-17T21:17:42.597312+0000", 2024-09-17T21:19:50.043 INFO:teuthology.orchestra.run.smithi083.stdout: "modified": "2024-09-17T21:19:49.285217+0000", 2024-09-17T21:19:50.043 INFO:teuthology.orchestra.run.smithi083.stdout: "last_up_change": "2024-09-17T21:17:53.976897+0000", 2024-09-17T21:19:50.043 INFO:teuthology.orchestra.run.smithi083.stdout: "last_in_change": "2024-09-17T21:19:44.668301+0000", 2024-09-17T21:19:50.043 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T21:19:50.043 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_num": 5806080, 2024-09-17T21:19:50.043 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_set": [ 2024-09-17T21:19:50.043 INFO:teuthology.orchestra.run.smithi083.stdout: "nodeep-scrub", 2024-09-17T21:19:50.043 INFO:teuthology.orchestra.run.smithi083.stdout: "noscrub", 2024-09-17T21:19:50.043 INFO:teuthology.orchestra.run.smithi083.stdout: "pglog_hardlimit", 2024-09-17T21:19:50.043 INFO:teuthology.orchestra.run.smithi083.stdout: "purged_snapdirs", 2024-09-17T21:19:50.043 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_deletes", 2024-09-17T21:19:50.044 INFO:teuthology.orchestra.run.smithi083.stdout: "sortbitwise" 2024-09-17T21:19:50.044 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:19:50.044 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_version": 5, 2024-09-17T21:19:50.044 INFO:teuthology.orchestra.run.smithi083.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T21:19:50.044 INFO:teuthology.orchestra.run.smithi083.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T21:19:50.044 INFO:teuthology.orchestra.run.smithi083.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T21:19:50.044 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_snapshot": "", 2024-09-17T21:19:50.044 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_max": 3, 2024-09-17T21:19:50.044 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd": 6, 2024-09-17T21:19:50.044 INFO:teuthology.orchestra.run.smithi083.stdout: "require_min_compat_client": "luminous", 2024-09-17T21:19:50.044 INFO:teuthology.orchestra.run.smithi083.stdout: "min_compat_client": "luminous", 2024-09-17T21:19:50.044 INFO:teuthology.orchestra.run.smithi083.stdout: "require_osd_release": "squid", 2024-09-17T21:19:50.044 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_crimson": false, 2024-09-17T21:19:50.045 INFO:teuthology.orchestra.run.smithi083.stdout: "pools": [ 2024-09-17T21:19:50.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.045 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 1, 2024-09-17T21:19:50.045 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": ".mgr", 2024-09-17T21:19:50.045 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-17T21:17:55.581350+0000", 2024-09-17T21:19:50.045 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-17T21:19:50.045 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-17T21:19:50.045 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-17T21:19:50.045 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 2, 2024-09-17T21:19:50.045 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 1, 2024-09-17T21:19:50.045 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-17T21:19:50.045 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:19:50.045 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:19:50.045 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:19:50.046 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:19:50.046 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-17T21:19:50.046 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-17T21:19:50.046 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:19:50.046 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 1, 2024-09-17T21:19:50.046 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 1, 2024-09-17T21:19:50.046 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 1, 2024-09-17T21:19:50.046 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 1, 2024-09-17T21:19:50.046 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 1, 2024-09-17T21:19:50.046 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-17T21:19:50.046 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-17T21:19:50.046 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-17T21:19:50.046 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-17T21:19:50.047 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-17T21:19:50.047 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-17T21:19:50.047 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-17T21:19:50.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.047 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "17", 2024-09-17T21:19:50.047 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-17T21:19:50.047 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:19:50.047 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:19:50.047 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-17T21:19:50.047 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:19:50.047 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-17T21:19:50.047 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-17T21:19:50.047 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-17T21:19:50.047 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-17T21:19:50.048 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-17T21:19:50.048 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-17T21:19:50.048 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-17T21:19:50.048 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-17T21:19:50.048 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-17T21:19:50.048 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-17T21:19:50.048 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-17T21:19:50.048 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-17T21:19:50.048 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-17T21:19:50.048 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:19:50.048 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:19:50.048 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:19:50.048 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-17T21:19:50.049 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-17T21:19:50.049 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-17T21:19:50.049 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-17T21:19:50.049 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-17T21:19:50.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.049 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-17T21:19:50.049 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-17T21:19:50.049 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-17T21:19:50.049 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:19:50.049 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:19:50.049 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:19:50.049 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:19:50.049 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-17T21:19:50.049 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-17T21:19:50.050 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-17T21:19:50.050 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-17T21:19:50.050 INFO:teuthology.orchestra.run.smithi083.stdout: "options": { 2024-09-17T21:19:50.050 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_max": 32, 2024-09-17T21:19:50.050 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_min": 1 2024-09-17T21:19:50.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.050 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-17T21:19:50.050 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": {} 2024-09-17T21:19:50.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.050 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-17T21:19:50.050 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-17T21:19:50.050 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 5, 2024-09-17T21:19:50.050 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 5, 2024-09-17T21:19:50.050 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 0.40000000596046448, 2024-09-17T21:19:50.051 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2, 2024-09-17T21:19:50.051 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2, 2024-09-17T21:19:50.051 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 0.65631616115570068, 2024-09-17T21:19:50.051 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 0.79000002145767212, 2024-09-17T21:19:50.051 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 0.6600000262260437 2024-09-17T21:19:50.051 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.051 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T21:19:50.051 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "rbd", 2024-09-17T21:19:50.051 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-17T21:17:59.603025+0000", 2024-09-17T21:19:50.051 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 8193, 2024-09-17T21:19:50.051 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:19:50.051 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-17T21:19:50.052 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 2, 2024-09-17T21:19:50.052 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 1, 2024-09-17T21:19:50.052 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-17T21:19:50.052 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:19:50.052 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:19:50.052 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:19:50.052 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:19:50.052 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-17T21:19:50.052 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-17T21:19:50.052 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:19:50.052 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 8, 2024-09-17T21:19:50.052 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 8, 2024-09-17T21:19:50.052 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 8, 2024-09-17T21:19:50.053 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 8, 2024-09-17T21:19:50.053 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 8, 2024-09-17T21:19:50.053 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-17T21:19:50.053 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-17T21:19:50.053 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-17T21:19:50.053 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-17T21:19:50.053 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-17T21:19:50.053 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-17T21:19:50.053 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-17T21:19:50.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.053 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "21", 2024-09-17T21:19:50.053 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-17T21:19:50.053 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:19:50.054 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:19:50.054 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-17T21:19:50.054 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:19:50.054 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 2, 2024-09-17T21:19:50.054 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 21, 2024-09-17T21:19:50.054 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-17T21:19:50.054 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-17T21:19:50.054 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-17T21:19:50.054 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-17T21:19:50.054 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-17T21:19:50.054 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-17T21:19:50.054 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-17T21:19:50.054 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-17T21:19:50.054 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-17T21:19:50.054 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-17T21:19:50.055 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-17T21:19:50.055 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:19:50.055 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:19:50.055 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:19:50.055 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-17T21:19:50.055 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-17T21:19:50.055 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-17T21:19:50.055 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-17T21:19:50.055 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-17T21:19:50.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.055 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-17T21:19:50.055 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-17T21:19:50.055 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-17T21:19:50.056 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:19:50.056 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:19:50.056 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:19:50.056 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:19:50.056 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-17T21:19:50.056 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-17T21:19:50.056 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-17T21:19:50.056 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-17T21:19:50.056 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-17T21:19:50.056 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-17T21:19:50.056 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd": {} 2024-09-17T21:19:50.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.056 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-17T21:19:50.056 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-17T21:19:50.057 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 3.130000114440918, 2024-09-17T21:19:50.057 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 3.130000114440918, 2024-09-17T21:19:50.057 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-17T21:19:50.057 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 3.130000114440918, 2024-09-17T21:19:50.057 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 3.130000114440918, 2024-09-17T21:19:50.057 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 0.65631616115570068, 2024-09-17T21:19:50.057 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 0.79000002145767212, 2024-09-17T21:19:50.057 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 0.6600000262260437 2024-09-17T21:19:50.057 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.057 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 3, 2024-09-17T21:19:50.057 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "pool_client.0", 2024-09-17T21:19:50.057 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-17T21:18:09.441137+0000", 2024-09-17T21:19:50.058 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 8193, 2024-09-17T21:19:50.058 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:19:50.058 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-17T21:19:50.058 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 2, 2024-09-17T21:19:50.058 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 1, 2024-09-17T21:19:50.058 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-17T21:19:50.058 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:19:50.058 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:19:50.058 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:19:50.058 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:19:50.058 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-17T21:19:50.058 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-17T21:19:50.058 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:19:50.058 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-17T21:19:50.059 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-17T21:19:50.059 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-17T21:19:50.059 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-17T21:19:50.059 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-17T21:19:50.059 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-17T21:19:50.059 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-17T21:19:50.059 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-17T21:19:50.059 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-17T21:19:50.059 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-17T21:19:50.059 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-17T21:19:50.059 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-17T21:19:50.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.059 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "91", 2024-09-17T21:19:50.059 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-17T21:19:50.060 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:19:50.060 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:19:50.060 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-17T21:19:50.060 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:19:50.060 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 26, 2024-09-17T21:19:50.060 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 91, 2024-09-17T21:19:50.060 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-17T21:19:50.060 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-17T21:19:50.060 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-17T21:19:50.060 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-17T21:19:50.060 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-17T21:19:50.060 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-17T21:19:50.060 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-17T21:19:50.061 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-17T21:19:50.061 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-17T21:19:50.061 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-17T21:19:50.061 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-17T21:19:50.061 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:19:50.061 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:19:50.061 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:19:50.061 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-17T21:19:50.061 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-17T21:19:50.061 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-17T21:19:50.061 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-17T21:19:50.061 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-17T21:19:50.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.061 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-17T21:19:50.062 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-17T21:19:50.062 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-17T21:19:50.062 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:19:50.062 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:19:50.062 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:19:50.062 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:19:50.062 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-17T21:19:50.062 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-17T21:19:50.062 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-17T21:19:50.062 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-17T21:19:50.062 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-17T21:19:50.062 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-17T21:19:50.062 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd": {} 2024-09-17T21:19:50.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.063 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-17T21:19:50.063 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-17T21:19:50.063 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.4099999666213989, 2024-09-17T21:19:50.063 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.4099999666213989, 2024-09-17T21:19:50.063 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-17T21:19:50.063 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.4099999666213989, 2024-09-17T21:19:50.063 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.4099999666213989, 2024-09-17T21:19:50.063 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 0.65631616115570068, 2024-09-17T21:19:50.063 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 0.79000002145767212, 2024-09-17T21:19:50.063 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 0.6600000262260437 2024-09-17T21:19:50.063 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.063 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.063 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:19:50.064 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-17T21:19:50.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.064 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-17T21:19:50.064 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "5bfd41c9-0cac-47c4-9a6e-8dce774c3314", 2024-09-17T21:19:50.064 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:19:50.064 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:19:50.064 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:19:50.064 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-17T21:19:50.064 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:19:50.064 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:19:50.064 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:19:50.064 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 95, 2024-09-17T21:19:50.064 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:19:50.064 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:19:50.065 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:19:50.065 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.065 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.065 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.065 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6816", 2024-09-17T21:19:50.065 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:19:50.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.065 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.065 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.065 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6817", 2024-09-17T21:19:50.065 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:19:50.065 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.065 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.066 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:19:50.066 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.066 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.066 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6818", 2024-09-17T21:19:50.066 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:19:50.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.066 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.066 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6819", 2024-09-17T21:19:50.066 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:19:50.066 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.066 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.067 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:19:50.067 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.067 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.067 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6822", 2024-09-17T21:19:50.067 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:19:50.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.067 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.067 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6823", 2024-09-17T21:19:50.067 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:19:50.067 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.067 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.068 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:19:50.068 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.068 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.068 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6820", 2024-09-17T21:19:50.068 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:19:50.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.068 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.068 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6821", 2024-09-17T21:19:50.068 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:19:50.068 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.068 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.069 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6817/1111036754", 2024-09-17T21:19:50.069 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6819/1111036754", 2024-09-17T21:19:50.069 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6823/1111036754", 2024-09-17T21:19:50.069 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6821/1111036754", 2024-09-17T21:19:50.069 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:19:50.069 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:19:50.069 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:19:50.069 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.069 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.069 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-17T21:19:50.070 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "78404e0d-d397-4806-8e90-5f32182643bb", 2024-09-17T21:19:50.070 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:19:50.070 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:19:50.070 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:19:50.070 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-17T21:19:50.070 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:19:50.070 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:19:50.070 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:19:50.070 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 95, 2024-09-17T21:19:50.070 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:19:50.070 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:19:50.070 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:19:50.070 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.071 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.071 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6808", 2024-09-17T21:19:50.071 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:19:50.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.071 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.071 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6809", 2024-09-17T21:19:50.071 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:19:50.071 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.071 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.071 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:19:50.071 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.072 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.072 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6810", 2024-09-17T21:19:50.072 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:19:50.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.072 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.072 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6811", 2024-09-17T21:19:50.072 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:19:50.072 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.072 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.072 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:19:50.072 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.073 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.073 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6814", 2024-09-17T21:19:50.073 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:19:50.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.073 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.073 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6815", 2024-09-17T21:19:50.073 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:19:50.073 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.073 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.073 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:19:50.073 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.074 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.074 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6812", 2024-09-17T21:19:50.074 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:19:50.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.074 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.074 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6813", 2024-09-17T21:19:50.074 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:19:50.074 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.074 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.074 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6809/1460363937", 2024-09-17T21:19:50.074 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6811/1460363937", 2024-09-17T21:19:50.075 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6815/1460363937", 2024-09-17T21:19:50.075 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6813/1460363937", 2024-09-17T21:19:50.075 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:19:50.075 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:19:50.075 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:19:50.075 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.075 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-17T21:19:50.075 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "286ed6e0-c121-4ebb-8cf6-37e506a397aa", 2024-09-17T21:19:50.075 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:19:50.075 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 0, 2024-09-17T21:19:50.075 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 0, 2024-09-17T21:19:50.075 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 0.2769012451171875, 2024-09-17T21:19:50.076 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:19:50.076 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:19:50.076 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 12, 2024-09-17T21:19:50.076 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 95, 2024-09-17T21:19:50.076 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:19:50.076 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:19:50.076 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:19:50.076 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.076 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.076 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.076 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6800", 2024-09-17T21:19:50.076 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:19:50.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.076 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.077 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.077 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6801", 2024-09-17T21:19:50.077 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:19:50.077 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.077 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.077 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:19:50.077 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.077 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.077 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6802", 2024-09-17T21:19:50.077 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:19:50.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.078 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.078 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6803", 2024-09-17T21:19:50.078 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:19:50.078 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.078 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.078 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:19:50.078 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.078 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.078 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6806", 2024-09-17T21:19:50.078 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:19:50.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.079 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.079 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6807", 2024-09-17T21:19:50.079 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:19:50.079 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.079 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.079 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:19:50.079 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.079 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.079 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6804", 2024-09-17T21:19:50.079 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:19:50.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.080 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.080 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6805", 2024-09-17T21:19:50.080 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:19:50.080 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.080 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.080 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6801/2837444034", 2024-09-17T21:19:50.080 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6803/2837444034", 2024-09-17T21:19:50.080 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6807/2837444034", 2024-09-17T21:19:50.080 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6805/2837444034", 2024-09-17T21:19:50.080 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:19:50.080 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:19:50.081 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:19:50.081 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.081 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-17T21:19:50.081 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "b0af73a1-4bca-4892-a550-2ced87f4c357", 2024-09-17T21:19:50.081 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:19:50.081 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:19:50.081 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:19:50.081 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 0.6609954833984375, 2024-09-17T21:19:50.081 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:19:50.081 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:19:50.081 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:19:50.081 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 96, 2024-09-17T21:19:50.082 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:19:50.082 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:19:50.082 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:19:50.082 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.082 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.082 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6808", 2024-09-17T21:19:50.082 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:19:50.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.082 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.082 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6809", 2024-09-17T21:19:50.082 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:19:50.082 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.083 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.083 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:19:50.083 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.083 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.083 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6810", 2024-09-17T21:19:50.083 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:19:50.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.083 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.083 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6811", 2024-09-17T21:19:50.083 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:19:50.083 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.084 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.084 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:19:50.084 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.084 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.084 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6814", 2024-09-17T21:19:50.084 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:19:50.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.084 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.084 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6815", 2024-09-17T21:19:50.084 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:19:50.084 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.085 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.085 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:19:50.085 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.085 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.085 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6812", 2024-09-17T21:19:50.085 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:19:50.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.085 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.085 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6813", 2024-09-17T21:19:50.085 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:19:50.086 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.086 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.086 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.97:6809/3637204865", 2024-09-17T21:19:50.086 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.97:6811/3637204865", 2024-09-17T21:19:50.086 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.97:6815/3637204865", 2024-09-17T21:19:50.086 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.97:6813/3637204865", 2024-09-17T21:19:50.086 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:19:50.086 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:19:50.086 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:19:50.086 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.086 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-17T21:19:50.087 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "2188361c-e1a1-45fa-bc01-da5a984ffd9f", 2024-09-17T21:19:50.087 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:19:50.087 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:19:50.087 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:19:50.087 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-17T21:19:50.087 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:19:50.087 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:19:50.087 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:19:50.087 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 96, 2024-09-17T21:19:50.087 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:19:50.087 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:19:50.087 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:19:50.087 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.088 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.088 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6800", 2024-09-17T21:19:50.088 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:19:50.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.088 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.088 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6801", 2024-09-17T21:19:50.088 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:19:50.088 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.088 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.088 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:19:50.088 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.089 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.089 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6802", 2024-09-17T21:19:50.089 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:19:50.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.089 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.089 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6803", 2024-09-17T21:19:50.089 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:19:50.089 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.089 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.089 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:19:50.089 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.090 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.090 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6806", 2024-09-17T21:19:50.090 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:19:50.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.090 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.090 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6807", 2024-09-17T21:19:50.090 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:19:50.090 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.090 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.090 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:19:50.090 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.091 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.091 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6804", 2024-09-17T21:19:50.091 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:19:50.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.091 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.091 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6805", 2024-09-17T21:19:50.091 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:19:50.091 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.091 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.091 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.97:6801/2391356911", 2024-09-17T21:19:50.091 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.97:6803/2391356911", 2024-09-17T21:19:50.092 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.97:6807/2391356911", 2024-09-17T21:19:50.092 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.97:6805/2391356911", 2024-09-17T21:19:50.092 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:19:50.092 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:19:50.092 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:19:50.092 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.092 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.092 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-17T21:19:50.092 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "7feccf4d-d8e4-4d06-ac83-bbc1bc623a5c", 2024-09-17T21:19:50.092 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:19:50.092 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:19:50.092 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:19:50.092 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 0, 2024-09-17T21:19:50.093 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:19:50.093 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:19:50.093 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:19:50.093 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 42, 2024-09-17T21:19:50.093 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:19:50.093 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:19:50.093 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:19:50.093 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.093 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.093 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6816", 2024-09-17T21:19:50.093 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:19:50.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.093 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.094 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6817", 2024-09-17T21:19:50.094 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:19:50.094 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.094 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.094 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:19:50.094 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.094 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.094 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6818", 2024-09-17T21:19:50.094 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:19:50.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.094 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.095 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6819", 2024-09-17T21:19:50.095 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:19:50.095 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.095 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.095 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:19:50.095 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.095 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.095 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6822", 2024-09-17T21:19:50.095 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:19:50.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.095 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.096 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6823", 2024-09-17T21:19:50.096 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:19:50.096 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.096 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.096 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:19:50.096 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:19:50.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.096 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:19:50.096 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6820", 2024-09-17T21:19:50.096 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:19:50.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.096 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:19:50.097 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6821", 2024-09-17T21:19:50.097 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:19:50.097 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.097 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.097 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.97:6817/2326316381", 2024-09-17T21:19:50.097 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.97:6819/2326316381", 2024-09-17T21:19:50.097 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.97:6823/2326316381", 2024-09-17T21:19:50.097 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.97:6821/2326316381", 2024-09-17T21:19:50.097 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:19:50.097 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:19:50.097 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:19:50.097 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.098 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.098 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:19:50.098 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_xinfo": [ 2024-09-17T21:19:50.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.098 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-17T21:19:50.098 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:19:50.098 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:19:50.098 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:19:50.098 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:19:50.098 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:19:50.098 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:51.138101+0000", 2024-09-17T21:19:50.098 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:19:50.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.098 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-17T21:19:50.099 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:19:50.099 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:19:50.099 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:19:50.099 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:19:50.099 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:19:50.099 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:50.581632+0000", 2024-09-17T21:19:50.099 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:19:50.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.099 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-17T21:19:50.099 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:19:50.099 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:19:50.099 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:19:50.099 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:19:50.100 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 65536, 2024-09-17T21:19:50.100 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:50.365193+0000", 2024-09-17T21:19:50.100 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:19:50.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.100 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-17T21:19:50.100 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:19:50.100 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:19:50.100 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:19:50.100 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:19:50.100 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:19:50.100 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:50.718974+0000", 2024-09-17T21:19:50.100 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:19:50.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.101 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-17T21:19:50.101 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:19:50.101 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:19:50.101 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:19:50.101 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:19:50.101 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:19:50.101 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:50.501878+0000", 2024-09-17T21:19:50.101 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:19:50.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.101 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-17T21:19:50.101 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:19:50.102 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:19:50.102 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:19:50.102 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:19:50.102 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:19:50.102 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:51.150747+0000", 2024-09-17T21:19:50.102 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:19:50.102 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.102 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:19:50.102 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap": [ 2024-09-17T21:19:50.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.102 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "3.8", 2024-09-17T21:19:50.102 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-17T21:19:50.102 INFO:teuthology.orchestra.run.smithi083.stdout: 2, 2024-09-17T21:19:50.102 INFO:teuthology.orchestra.run.smithi083.stdout: 3 2024-09-17T21:19:50.102 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.103 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "3.c", 2024-09-17T21:19:50.103 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-17T21:19:50.103 INFO:teuthology.orchestra.run.smithi083.stdout: 1, 2024-09-17T21:19:50.103 INFO:teuthology.orchestra.run.smithi083.stdout: 5 2024-09-17T21:19:50.103 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.103 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "3.1e", 2024-09-17T21:19:50.103 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-17T21:19:50.103 INFO:teuthology.orchestra.run.smithi083.stdout: 0, 2024-09-17T21:19:50.103 INFO:teuthology.orchestra.run.smithi083.stdout: 4 2024-09-17T21:19:50.103 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.104 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.104 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:19:50.104 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_items": [], 2024-09-17T21:19:50.104 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_primaries": [], 2024-09-17T21:19:50.104 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_temp": [], 2024-09-17T21:19:50.104 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_temp": [], 2024-09-17T21:19:50.104 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklist": {}, 2024-09-17T21:19:50.104 INFO:teuthology.orchestra.run.smithi083.stdout: "range_blocklist": {}, 2024-09-17T21:19:50.104 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profiles": { 2024-09-17T21:19:50.104 INFO:teuthology.orchestra.run.smithi083.stdout: "default": { 2024-09-17T21:19:50.104 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-failure-domain": "osd", 2024-09-17T21:19:50.104 INFO:teuthology.orchestra.run.smithi083.stdout: "k": "2", 2024-09-17T21:19:50.104 INFO:teuthology.orchestra.run.smithi083.stdout: "m": "1", 2024-09-17T21:19:50.104 INFO:teuthology.orchestra.run.smithi083.stdout: "plugin": "jerasure", 2024-09-17T21:19:50.105 INFO:teuthology.orchestra.run.smithi083.stdout: "technique": "reed_sol_van" 2024-09-17T21:19:50.105 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.105 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps_queue": [], 2024-09-17T21:19:50.105 INFO:teuthology.orchestra.run.smithi083.stdout: "new_removed_snaps": [], 2024-09-17T21:19:50.105 INFO:teuthology.orchestra.run.smithi083.stdout: "new_purged_snaps": [], 2024-09-17T21:19:50.105 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_node_flags": {}, 2024-09-17T21:19:50.105 INFO:teuthology.orchestra.run.smithi083.stdout: "device_class_flags": {}, 2024-09-17T21:19:50.105 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode": { 2024-09-17T21:19:50.105 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_enabled": false, 2024-09-17T21:19:50.105 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_bucket_count": 0, 2024-09-17T21:19:50.105 INFO:teuthology.orchestra.run.smithi083.stdout: "degraded_stretch_mode": 0, 2024-09-17T21:19:50.105 INFO:teuthology.orchestra.run.smithi083.stdout: "recovering_stretch_mode": 0, 2024-09-17T21:19:50.105 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_bucket": 0 2024-09-17T21:19:50.106 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.106 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:50.107 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:19:50.108 INFO:tasks.thrashosds.thrasher:j is {'epoch': 97, 'fsid': '224a7a29-e112-472e-8a13-f67345d6d123', 'created': '2024-09-17T21:17:42.597312+0000', 'modified': '2024-09-17T21:19:49.285217+0000', 'last_up_change': '2024-09-17T21:17:53.976897+0000', 'last_in_change': '2024-09-17T21:19:44.668301+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T21:17:55.581350+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5, 'score_stable': 5, 'optimal_score': 0.4000000059604645, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.6563161611557007, 'average_primary_affinity': 0.7900000214576721, 'average_primary_affinity_weighted': 0.6600000262260437}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T21:17:59.603025+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.130000114440918, 'score_stable': 3.130000114440918, 'optimal_score': 1, 'raw_score_acting': 3.130000114440918, 'raw_score_stable': 3.130000114440918, 'primary_affinity_weighted': 0.6563161611557007, 'average_primary_affinity': 0.7900000214576721, 'average_primary_affinity_weighted': 0.6600000262260437}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T21:18:09.441137+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 26, 'snap_epoch': 91, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.409999966621399, 'score_stable': 1.409999966621399, 'optimal_score': 1, 'raw_score_acting': 1.409999966621399, 'raw_score_stable': 1.409999966621399, 'primary_affinity_weighted': 0.6563161611557007, 'average_primary_affinity': 0.7900000214576721, 'average_primary_affinity_weighted': 0.6600000262260437}}], 'osds': [{'osd': 0, 'uuid': '5bfd41c9-0cac-47c4-9a6e-8dce774c3314', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6816', 'nonce': 1111036754}, {'type': 'v1', 'addr': '172.21.15.83:6817', 'nonce': 1111036754}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6818', 'nonce': 1111036754}, {'type': 'v1', 'addr': '172.21.15.83:6819', 'nonce': 1111036754}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6822', 'nonce': 1111036754}, {'type': 'v1', 'addr': '172.21.15.83:6823', 'nonce': 1111036754}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6820', 'nonce': 1111036754}, {'type': 'v1', 'addr': '172.21.15.83:6821', 'nonce': 1111036754}]}, 'public_addr': '172.21.15.83:6817/1111036754', 'cluster_addr': '172.21.15.83:6819/1111036754', 'heartbeat_back_addr': '172.21.15.83:6823/1111036754', 'heartbeat_front_addr': '172.21.15.83:6821/1111036754', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '78404e0d-d397-4806-8e90-5f32182643bb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6808', 'nonce': 1460363937}, {'type': 'v1', 'addr': '172.21.15.83:6809', 'nonce': 1460363937}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6810', 'nonce': 1460363937}, {'type': 'v1', 'addr': '172.21.15.83:6811', 'nonce': 1460363937}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6814', 'nonce': 1460363937}, {'type': 'v1', 'addr': '172.21.15.83:6815', 'nonce': 1460363937}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6812', 'nonce': 1460363937}, {'type': 'v1', 'addr': '172.21.15.83:6813', 'nonce': 1460363937}]}, 'public_addr': '172.21.15.83:6809/1460363937', 'cluster_addr': '172.21.15.83:6811/1460363937', 'heartbeat_back_addr': '172.21.15.83:6815/1460363937', 'heartbeat_front_addr': '172.21.15.83:6813/1460363937', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '286ed6e0-c121-4ebb-8cf6-37e506a397aa', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0.2769012451171875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6800', 'nonce': 2837444034}, {'type': 'v1', 'addr': '172.21.15.83:6801', 'nonce': 2837444034}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6802', 'nonce': 2837444034}, {'type': 'v1', 'addr': '172.21.15.83:6803', 'nonce': 2837444034}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6806', 'nonce': 2837444034}, {'type': 'v1', 'addr': '172.21.15.83:6807', 'nonce': 2837444034}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6804', 'nonce': 2837444034}, {'type': 'v1', 'addr': '172.21.15.83:6805', 'nonce': 2837444034}]}, 'public_addr': '172.21.15.83:6801/2837444034', 'cluster_addr': '172.21.15.83:6803/2837444034', 'heartbeat_back_addr': '172.21.15.83:6807/2837444034', 'heartbeat_front_addr': '172.21.15.83:6805/2837444034', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'b0af73a1-4bca-4892-a550-2ced87f4c357', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.6609954833984375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 96, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6808', 'nonce': 3637204865}, {'type': 'v1', 'addr': '172.21.15.97:6809', 'nonce': 3637204865}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6810', 'nonce': 3637204865}, {'type': 'v1', 'addr': '172.21.15.97:6811', 'nonce': 3637204865}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6814', 'nonce': 3637204865}, {'type': 'v1', 'addr': '172.21.15.97:6815', 'nonce': 3637204865}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6812', 'nonce': 3637204865}, {'type': 'v1', 'addr': '172.21.15.97:6813', 'nonce': 3637204865}]}, 'public_addr': '172.21.15.97:6809/3637204865', 'cluster_addr': '172.21.15.97:6811/3637204865', 'heartbeat_back_addr': '172.21.15.97:6815/3637204865', 'heartbeat_front_addr': '172.21.15.97:6813/3637204865', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2188361c-e1a1-45fa-bc01-da5a984ffd9f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 96, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6800', 'nonce': 2391356911}, {'type': 'v1', 'addr': '172.21.15.97:6801', 'nonce': 2391356911}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6802', 'nonce': 2391356911}, {'type': 'v1', 'addr': '172.21.15.97:6803', 'nonce': 2391356911}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6806', 'nonce': 2391356911}, {'type': 'v1', 'addr': '172.21.15.97:6807', 'nonce': 2391356911}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6804', 'nonce': 2391356911}, {'type': 'v1', 'addr': '172.21.15.97:6805', 'nonce': 2391356911}]}, 'public_addr': '172.21.15.97:6801/2391356911', 'cluster_addr': '172.21.15.97:6803/2391356911', 'heartbeat_back_addr': '172.21.15.97:6807/2391356911', 'heartbeat_front_addr': '172.21.15.97:6805/2391356911', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '7feccf4d-d8e4-4d06-ac83-bbc1bc623a5c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 42, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6816', 'nonce': 2326316381}, {'type': 'v1', 'addr': '172.21.15.97:6817', 'nonce': 2326316381}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6818', 'nonce': 2326316381}, {'type': 'v1', 'addr': '172.21.15.97:6819', 'nonce': 2326316381}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6822', 'nonce': 2326316381}, {'type': 'v1', 'addr': '172.21.15.97:6823', 'nonce': 2326316381}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6820', 'nonce': 2326316381}, {'type': 'v1', 'addr': '172.21.15.97:6821', 'nonce': 2326316381}]}, 'public_addr': '172.21.15.97:6817/2326316381', 'cluster_addr': '172.21.15.97:6819/2326316381', 'heartbeat_back_addr': '172.21.15.97:6823/2326316381', 'heartbeat_front_addr': '172.21.15.97:6821/2326316381', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:51.138101+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:50.581632+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-17T21:17:50.365193+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:50.718974+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:50.501878+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:51.150747+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.8', 'osds': [2, 3]}, {'pgid': '3.c', 'osds': [1, 5]}, {'pgid': '3.1e', 'osds': [0, 4]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T21:19:50.108 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:19:50.153 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:50.150+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:50.233 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:50.227+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:50.271 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:50.271 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:50.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:50.271 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:50.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.271 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:494 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:50.272 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:50.272 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 84.071765564000003, 2024-09-17T21:19:50.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046244995999999997, 2024-09-17T21:19:50.272 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:50.272 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:50.272 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:50.272 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:50.272 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:50.272 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:50.272 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 494 2024-09-17T21:19:50.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.272 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:50.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:50.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:50.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:50.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:50.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:50.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197329+0000", 2024-09-17T21:19:50.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.173e-06 2024-09-17T21:19:50.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:50.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197334+0000", 2024-09-17T21:19:50.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0499999999999999e-06 2024-09-17T21:19:50.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:50.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197352+0000", 2024-09-17T21:19:50.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7133e-05 2024-09-17T21:19:50.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:50.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197367+0000", 2024-09-17T21:19:50.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5123000000000001e-05 2024-09-17T21:19:50.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:50.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:19:50.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044989829000000002 2024-09-17T21:19:50.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:50.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242526+0000", 2024-09-17T21:19:50.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016968199999999999 2024-09-17T21:19:50.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:50.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243573+0000", 2024-09-17T21:19:50.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010470060000000001 2024-09-17T21:19:50.276 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.276 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.276 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.276 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:524 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:50.276 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:50.276 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 84.069019535999999, 2024-09-17T21:19:50.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045737414999999997, 2024-09-17T21:19:50.276 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:50.276 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:50.277 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:50.277 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:50.277 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:50.277 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:50.277 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 524 2024-09-17T21:19:50.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.277 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:50.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:50.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:50.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:50.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:50.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:50.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200076+0000", 2024-09-17T21:19:50.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:19:50.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:50.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200081+0000", 2024-09-17T21:19:50.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7030000000000003e-06 2024-09-17T21:19:50.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:50.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200099+0000", 2024-09-17T21:19:50.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7836999999999999e-05 2024-09-17T21:19:50.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:50.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200113+0000", 2024-09-17T21:19:50.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3570000000000001e-05 2024-09-17T21:19:50.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:50.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244866+0000", 2024-09-17T21:19:50.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044753573999999997 2024-09-17T21:19:50.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:50.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244972+0000", 2024-09-17T21:19:50.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106111 2024-09-17T21:19:50.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:50.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245812+0000", 2024-09-17T21:19:50.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083923999999999999 2024-09-17T21:19:50.280 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.280 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.280 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.280 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:525 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3670016~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:50.280 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:50.281 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 84.068868154, 2024-09-17T21:19:50.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046645697, 2024-09-17T21:19:50.281 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:50.281 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:50.281 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:50.281 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:50.281 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:50.281 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:50.281 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 525 2024-09-17T21:19:50.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.281 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:50.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:50.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:50.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:50.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:50.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:50.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:19:50.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:19:50.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:50.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200234+0000", 2024-09-17T21:19:50.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:19:50.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:50.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200254+0000", 2024-09-17T21:19:50.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9959999999999999e-05 2024-09-17T21:19:50.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:50.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200293+0000", 2024-09-17T21:19:50.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9755999999999997e-05 2024-09-17T21:19:50.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:50.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245871+0000", 2024-09-17T21:19:50.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045577871999999998 2024-09-17T21:19:50.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:50.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246030+0000", 2024-09-17T21:19:50.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158975 2024-09-17T21:19:50.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:50.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246871+0000", 2024-09-17T21:19:50.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084111699999999995 2024-09-17T21:19:50.284 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.284 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.285 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.285 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:50.285 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:50.285 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 84.068622478999998, 2024-09-17T21:19:50.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:19:50.285 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:50.285 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:50.285 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:50.285 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:50.285 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:50.285 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:50.285 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:19:50.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.286 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:50.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:50.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:50.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:50.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:50.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:50.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:19:50.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:19:50.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:50.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:19:50.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:19:50.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:50.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:19:50.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:19:50.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:50.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:19:50.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:19:50.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:50.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:19:50.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:19:50.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:50.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:19:50.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:19:50.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:50.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:19:50.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:19:50.289 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.289 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.289 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.289 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:50.289 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:50.289 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 84.068443797, 2024-09-17T21:19:50.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:19:50.289 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:50.290 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:50.290 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:50.290 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:50.290 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:50.290 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:50.290 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:19:50.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.290 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:50.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:50.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:50.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:50.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:50.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:50.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:19:50.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:19:50.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:50.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:19:50.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:19:50.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:50.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:19:50.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:19:50.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:50.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:19:50.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:19:50.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:50.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:19:50.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:19:50.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:50.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:19:50.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:19:50.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:50.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:19:50.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:19:50.293 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.293 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.293 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.294 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:50.294 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:50.294 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 84.068384523999995, 2024-09-17T21:19:50.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:19:50.294 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:50.294 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:50.294 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:50.294 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:50.294 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:50.294 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:50.294 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:19:50.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.294 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:50.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:50.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:50.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:50.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:50.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:50.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:19:50.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:19:50.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:50.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:19:50.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:19:50.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:50.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:19:50.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:19:50.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:50.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:19:50.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:19:50.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:50.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:19:50.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:19:50.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:50.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:19:50.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:19:50.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:50.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:19:50.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:19:50.297 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.298 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.298 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.298 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:50.298 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:50.298 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 84.068242329, 2024-09-17T21:19:50.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:19:50.298 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:50.298 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:50.298 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:50.298 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:50.298 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:50.298 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:50.299 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:19:50.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.299 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:50.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:50.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:50.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:50.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:50.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:50.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:19:50.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:19:50.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:50.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:19:50.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:19:50.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:50.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:19:50.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:19:50.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:50.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:19:50.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:19:50.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:50.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:19:50.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:19:50.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:50.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:19:50.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:19:50.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:50.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:19:50.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:19:50.302 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.302 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.302 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.302 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:50.302 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:50.302 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 84.068186405999995, 2024-09-17T21:19:50.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:19:50.302 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:50.303 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:50.303 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:50.303 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:50.303 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:50.303 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:50.303 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:19:50.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.303 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:50.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:50.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:50.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:50.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:50.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:50.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:19:50.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:19:50.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:50.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:19:50.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:19:50.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:50.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:19:50.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:19:50.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:50.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:19:50.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:19:50.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:50.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:19:50.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:19:50.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:50.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:19:50.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:19:50.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:50.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:19:50.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:19:50.306 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.306 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.306 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.307 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:50.307 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:50.307 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 84.068132789000003, 2024-09-17T21:19:50.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:19:50.307 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:50.307 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:50.307 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:50.307 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:50.307 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:50.307 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:50.307 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:19:50.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.308 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:50.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:50.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:50.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:50.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:50.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:50.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:19:50.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:19:50.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.309 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:50.309 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:19:50.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:19:50.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.309 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:50.309 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:19:50.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:19:50.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.309 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:50.309 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:19:50.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:19:50.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:50.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:19:50.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:19:50.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:50.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:19:50.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:19:50.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:50.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:19:50.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:19:50.311 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.311 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.311 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.311 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:19:50.311 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:19:50.311 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 33.758904547, 2024-09-17T21:19:50.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:19:50.311 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:50.311 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:50.311 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:19:50.311 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:50.312 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:19:50.312 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:19:50.312 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:19:50.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.312 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:50.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:50.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:19:50.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:50.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:19:50.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:50.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:19:50.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:19:50.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:50.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:19:50.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:19:50.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:50.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:19:50.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:19:50.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:50.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:19:50.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:19:50.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:50.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:19:50.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:19:50.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:50.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:19:50.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:19:50.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:50.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:19:50.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:19:50.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:50.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:19:50.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:19:50.315 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.316 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.316 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.316 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2597 3.1f 3:fb0e9615:::rbd_data.1152a8ae2019.000000000000000c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:50.316 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:19:50.316 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.4304681190000004, 2024-09-17T21:19:50.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097064994000000002, 2024-09-17T21:19:50.316 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:50.316 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:50.316 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:50.316 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:50.316 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:50.316 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:50.317 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2597 2024-09-17T21:19:50.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.317 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:50.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:50.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:19:50.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:50.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:19:50.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:50.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838628+0000", 2024-09-17T21:19:50.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.187e-06 2024-09-17T21:19:50.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:50.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838654+0000", 2024-09-17T21:19:50.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:19:50.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:50.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838696+0000", 2024-09-17T21:19:50.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1226000000000001e-05 2024-09-17T21:19:50.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:50.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838715+0000", 2024-09-17T21:19:50.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9646000000000001e-05 2024-09-17T21:19:50.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:50.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838937+0000", 2024-09-17T21:19:50.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022163 2024-09-17T21:19:50.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:50.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.839284+0000", 2024-09-17T21:19:50.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034753799999999999 2024-09-17T21:19:50.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:50.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935691+0000", 2024-09-17T21:19:50.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.096406360999999996 2024-09-17T21:19:50.320 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.320 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.320 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.320 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2641 3.12 3:488a729f:::rbd_data.1152a8ae2019.0000000000000022:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:50.320 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:19:50.320 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.4270113460000005, 2024-09-17T21:19:50.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098280953000000004, 2024-09-17T21:19:50.321 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:50.321 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:50.321 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:50.321 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:50.321 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:50.321 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:50.321 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2641 2024-09-17T21:19:50.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.321 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:50.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:50.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:19:50.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:50.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:19:50.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:50.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842083+0000", 2024-09-17T21:19:50.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5300000000000002e-07 2024-09-17T21:19:50.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:50.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842088+0000", 2024-09-17T21:19:50.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:19:50.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:50.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842105+0000", 2024-09-17T21:19:50.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6773e-05 2024-09-17T21:19:50.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:50.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842119+0000", 2024-09-17T21:19:50.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:19:50.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:50.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842320+0000", 2024-09-17T21:19:50.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020052700000000001 2024-09-17T21:19:50.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:50.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842845+0000", 2024-09-17T21:19:50.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052508600000000002 2024-09-17T21:19:50.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:50.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940363+0000", 2024-09-17T21:19:50.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097518795000000005 2024-09-17T21:19:50.324 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.324 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.324 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.325 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2674 3.12 3:4c87e3ac:::rbd_data.1152a8ae2019.0000000000000033:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:50.325 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:19:50.325 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.4266487090000002, 2024-09-17T21:19:50.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098025125000000005, 2024-09-17T21:19:50.325 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:50.325 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:50.325 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:50.325 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:50.325 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:50.325 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:50.325 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2674 2024-09-17T21:19:50.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.325 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:50.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:50.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:19:50.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:50.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:19:50.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:50.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842447+0000", 2024-09-17T21:19:50.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3999999999999999e-06 2024-09-17T21:19:50.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:50.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842453+0000", 2024-09-17T21:19:50.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:19:50.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:50.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842472+0000", 2024-09-17T21:19:50.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9006000000000001e-05 2024-09-17T21:19:50.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:50.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842487+0000", 2024-09-17T21:19:50.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4866e-05 2024-09-17T21:19:50.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:50.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843594+0000", 2024-09-17T21:19:50.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011073249999999999 2024-09-17T21:19:50.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:50.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843668+0000", 2024-09-17T21:19:50.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3188000000000005e-05 2024-09-17T21:19:50.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:19:50.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843893+0000", 2024-09-17T21:19:50.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022503399999999999 2024-09-17T21:19:50.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:50.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850439+0000", 2024-09-17T21:19:50.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065467880000000004 2024-09-17T21:19:50.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:50.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940409+0000", 2024-09-17T21:19:50.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089969511000000002 2024-09-17T21:19:50.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:50.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940444+0000", 2024-09-17T21:19:50.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5345999999999997e-05 2024-09-17T21:19:50.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:50.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940470+0000", 2024-09-17T21:19:50.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5964000000000001e-05 2024-09-17T21:19:50.330 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.330 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.331 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.331 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2707 3.1f 3:f805e65e:::rbd_data.1152a8ae2019.0000000000000049:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:50.331 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:19:50.331 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.4216020480000005, 2024-09-17T21:19:50.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088327300999999997, 2024-09-17T21:19:50.331 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:50.331 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:50.331 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:50.331 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:50.331 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:50.331 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:50.331 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2707 2024-09-17T21:19:50.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.332 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:50.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:50.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:19:50.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:50.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:19:50.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:50.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847494+0000", 2024-09-17T21:19:50.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:19:50.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:50.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847504+0000", 2024-09-17T21:19:50.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:19:50.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:50.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847526+0000", 2024-09-17T21:19:50.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1976000000000001e-05 2024-09-17T21:19:50.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:50.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847541+0000", 2024-09-17T21:19:50.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5277e-05 2024-09-17T21:19:50.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:50.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856142+0000", 2024-09-17T21:19:50.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086006710000000007 2024-09-17T21:19:50.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:50.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856435+0000", 2024-09-17T21:19:50.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029301499999999999 2024-09-17T21:19:50.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:50.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935819+0000", 2024-09-17T21:19:50.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079384448999999996 2024-09-17T21:19:50.335 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.335 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.335 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.335 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2730 3.12 3:48b3422a:::rbd_data.1152a8ae2019.0000000000000057:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:50.335 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:19:50.335 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.4124091490000001, 2024-09-17T21:19:50.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083879975999999995, 2024-09-17T21:19:50.335 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:50.336 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:50.336 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:50.336 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:50.336 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:50.336 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:50.336 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2730 2024-09-17T21:19:50.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.336 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:50.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:50.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:19:50.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:50.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:19:50.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:50.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856687+0000", 2024-09-17T21:19:50.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:19:50.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:50.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856697+0000", 2024-09-17T21:19:50.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0207000000000001e-05 2024-09-17T21:19:50.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:50.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856808+0000", 2024-09-17T21:19:50.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011071200000000001 2024-09-17T21:19:50.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:50.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856828+0000", 2024-09-17T21:19:50.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0219000000000001e-05 2024-09-17T21:19:50.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:50.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856907+0000", 2024-09-17T21:19:50.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9661999999999997e-05 2024-09-17T21:19:50.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:50.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.857019+0000", 2024-09-17T21:19:50.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011148499999999999 2024-09-17T21:19:50.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:50.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940565+0000", 2024-09-17T21:19:50.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083545791999999994 2024-09-17T21:19:50.339 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.339 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.339 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.340 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2743 3.12 3:4d036e62:::rbd_data.1152a8ae2019.000000000000005f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:50.340 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:19:50.340 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.4084636479999997, 2024-09-17T21:19:50.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080059962999999998, 2024-09-17T21:19:50.340 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:50.340 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:50.340 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:50.340 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:50.340 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:50.340 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:50.340 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2743 2024-09-17T21:19:50.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.341 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:50.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:50.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:19:50.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:50.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:19:50.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:50.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860631+0000", 2024-09-17T21:19:50.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:19:50.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:50.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860636+0000", 2024-09-17T21:19:50.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5279999999999997e-06 2024-09-17T21:19:50.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:50.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860649+0000", 2024-09-17T21:19:50.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-17T21:19:50.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:50.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860655+0000", 2024-09-17T21:19:50.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:19:50.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:50.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860731+0000", 2024-09-17T21:19:50.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5920000000000005e-05 2024-09-17T21:19:50.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:50.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.861070+0000", 2024-09-17T21:19:50.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338535 2024-09-17T21:19:50.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:50.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940690+0000", 2024-09-17T21:19:50.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079620503999999995 2024-09-17T21:19:50.344 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.344 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.344 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.344 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3378 3.12 3:4832a51b:::rbd_data.1152a8ae2019.000000000000006c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:50.344 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:19:50.344 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.3800997979999998, 2024-09-17T21:19:50.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051812516000000003, 2024-09-17T21:19:50.344 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:50.344 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:50.345 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:50.345 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:50.345 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:50.345 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:50.345 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3378 2024-09-17T21:19:50.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.345 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:50.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:50.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:19:50.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:50.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:19:50.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:50.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888995+0000", 2024-09-17T21:19:50.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:19:50.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:50.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889006+0000", 2024-09-17T21:19:50.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1012999999999999e-05 2024-09-17T21:19:50.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:50.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889032+0000", 2024-09-17T21:19:50.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6441e-05 2024-09-17T21:19:50.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:50.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889040+0000", 2024-09-17T21:19:50.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0639999999999994e-06 2024-09-17T21:19:50.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:50.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889278+0000", 2024-09-17T21:19:50.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023785899999999999 2024-09-17T21:19:50.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:50.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889408+0000", 2024-09-17T21:19:50.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129219 2024-09-17T21:19:50.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:19:50.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889723+0000", 2024-09-17T21:19:50.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000315264 2024-09-17T21:19:50.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:50.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.891972+0000", 2024-09-17T21:19:50.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002249633 2024-09-17T21:19:50.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:50.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940733+0000", 2024-09-17T21:19:50.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048760626000000001 2024-09-17T21:19:50.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:50.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940785+0000", 2024-09-17T21:19:50.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1906000000000001e-05 2024-09-17T21:19:50.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:50.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940807+0000", 2024-09-17T21:19:50.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1554999999999999e-05 2024-09-17T21:19:50.350 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.350 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.350 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.350 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3379 3.12 3:4e83fd71:::rbd_data.1152a8ae2019.000000000000006d:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:50.350 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:19:50.350 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.3800155909999994, 2024-09-17T21:19:50.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051804144000000003, 2024-09-17T21:19:50.350 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:50.350 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:50.350 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:50.351 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:50.351 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:50.351 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:50.351 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3379 2024-09-17T21:19:50.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.351 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:50.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:50.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:19:50.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:50.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:19:50.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:50.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889079+0000", 2024-09-17T21:19:50.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1800000000000004e-07 2024-09-17T21:19:50.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:50.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889083+0000", 2024-09-17T21:19:50.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:19:50.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:50.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889094+0000", 2024-09-17T21:19:50.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1562999999999999e-05 2024-09-17T21:19:50.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:50.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889102+0000", 2024-09-17T21:19:50.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.401e-06 2024-09-17T21:19:50.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:50.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:19:50.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014279259999999999 2024-09-17T21:19:50.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:50.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890609+0000", 2024-09-17T21:19:50.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9566000000000004e-05 2024-09-17T21:19:50.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:19:50.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890789+0000", 2024-09-17T21:19:50.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017949700000000001 2024-09-17T21:19:50.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:50.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.893929+0000", 2024-09-17T21:19:50.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003140538 2024-09-17T21:19:50.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:50.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940834+0000", 2024-09-17T21:19:50.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046904811999999997 2024-09-17T21:19:50.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:50.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940860+0000", 2024-09-17T21:19:50.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5341999999999999e-05 2024-09-17T21:19:50.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:50.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940882+0000", 2024-09-17T21:19:50.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2878999999999999e-05 2024-09-17T21:19:50.356 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.356 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.356 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.356 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:1613 3.1e 3:7d13ef1b:::rbd_data.115266d0b9f2.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 5666816~34816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-17T21:19:50.356 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:48.315514+0000", 2024-09-17T21:19:50.356 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.9535798120000001, 2024-09-17T21:19:50.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045611719000000002, 2024-09-17T21:19:50.356 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:50.356 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:50.357 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:50.357 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:50.357 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:19:50.357 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:19:50.357 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 1613 2024-09-17T21:19:50.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.357 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:50.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:50.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.315514+0000", 2024-09-17T21:19:50.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:50.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.315514+0000", 2024-09-17T21:19:50.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:50.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.315516+0000", 2024-09-17T21:19:50.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.497e-06 2024-09-17T21:19:50.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:50.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.330451+0000", 2024-09-17T21:19:50.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014935857 2024-09-17T21:19:50.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:50.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.330479+0000", 2024-09-17T21:19:50.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7151000000000001e-05 2024-09-17T21:19:50.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:50.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.330490+0000", 2024-09-17T21:19:50.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1250000000000001e-05 2024-09-17T21:19:50.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:50.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.330560+0000", 2024-09-17T21:19:50.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9901999999999996e-05 2024-09-17T21:19:50.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:50.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.330688+0000", 2024-09-17T21:19:50.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000128608 2024-09-17T21:19:50.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:19:50.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.331269+0000", 2024-09-17T21:19:50.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00058052799999999997 2024-09-17T21:19:50.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:50.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.350343+0000", 2024-09-17T21:19:50.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019074457 2024-09-17T21:19:50.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:50.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.361030+0000", 2024-09-17T21:19:50.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010686357 2024-09-17T21:19:50.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:50.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.361084+0000", 2024-09-17T21:19:50.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4097000000000001e-05 2024-09-17T21:19:50.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:50.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.361126+0000", 2024-09-17T21:19:50.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2014999999999997e-05 2024-09-17T21:19:50.361 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.362 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.362 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.362 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:2644 3.1e 3:79615585:::rbd_data.115266d0b9f2.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2475520~14848,write 3106304~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:19:50.362 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:19:50.362 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.75738718800000004, 2024-09-17T21:19:50.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050733484000000002, 2024-09-17T21:19:50.362 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:50.362 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:50.362 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:50.362 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:50.362 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:19:50.362 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:19:50.363 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2644 2024-09-17T21:19:50.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.363 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:50.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:50.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:19:50.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:50.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:19:50.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:50.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:50.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511708+0000", 2024-09-17T21:19:50.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-17T21:19:50.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:50.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522196+0000", 2024-09-17T21:19:50.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010487712999999999 2024-09-17T21:19:50.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:50.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522223+0000", 2024-09-17T21:19:50.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7064999999999999e-05 2024-09-17T21:19:50.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:50.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522236+0000", 2024-09-17T21:19:50.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2931e-05 2024-09-17T21:19:50.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:50.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522348+0000", 2024-09-17T21:19:50.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111906 2024-09-17T21:19:50.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:50.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522554+0000", 2024-09-17T21:19:50.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020632699999999999 2024-09-17T21:19:50.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:19:50.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.523185+0000", 2024-09-17T21:19:50.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063081400000000005 2024-09-17T21:19:50.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:50.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.547011+0000", 2024-09-17T21:19:50.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023825570000000001 2024-09-17T21:19:50.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:50.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562268+0000", 2024-09-17T21:19:50.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015257514 2024-09-17T21:19:50.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:50.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562367+0000", 2024-09-17T21:19:50.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8678000000000006e-05 2024-09-17T21:19:50.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:50.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:50.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:50.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562440+0000", 2024-09-17T21:19:50.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3337000000000006e-05 2024-09-17T21:19:50.367 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.367 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.367 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.367 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:50.368 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:50.368 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:50.369 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:19:50.370 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:50.363+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:50.451 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:19:50.466 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:19:50.467 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T21:19:50.470 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:50.463+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:50.486 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-17T21:19:50.294805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":921921123,"num_objects":963,"num_object_clones":195,"num_object_copies":1926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":963,"num_whiteouts":401,"num_read":118695,"num_read_kb":465188,"num_write":9683,"num_write_kb":212035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":918,"num_bytes_recovered":68527838,"num_keys_recovered":223,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8893,"ondisk_log_size":8893,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":397584,"kb_used_data":191192,"kb_used_omap":7,"kb_used_meta":198840,"kb_avail":471461616,"statfs":{"total":483183820800,"available":482776694784,"internally_reserved":0,"allocated":195780608,"data_stored":254833700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7978,"internal_metadata":203612374},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":97692676,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":2753,"num_read_kb":16660,"num_write":166,"num_write_kb":26773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":426,"num_bytes_recovered":21908224,"num_keys_recovered":80,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.613653"},"pg_stats":[{"pgid":"3.1f","version":"90'140","reported_seq":682,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.485920+0000","last_change":"2024-09-17T21:19:17.489680+0000","last_active":"2024-09-17T21:19:49.485920+0000","last_peered":"2024-09-17T21:19:49.485920+0000","last_clean":"2024-09-17T21:19:49.485920+0000","last_became_active":"2024-09-17T21:19:17.290274+0000","last_became_peered":"2024-09-17T21:19:17.290274+0000","last_unstale":"2024-09-17T21:19:49.485920+0000","last_undegraded":"2024-09-17T21:19:49.485920+0000","last_fullsized":"2024-09-17T21:19:49.485920+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:02:31.510422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088525599999999996,"stat_sum":{"num_bytes":6237411,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":11,"num_read":1298,"num_read_kb":9627,"num_write":148,"num_write_kb":1394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":568483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"97'241","reported_seq":2382,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.507824+0000","last_change":"2024-09-17T21:19:20.291890+0000","last_active":"2024-09-17T21:19:49.507824+0000","last_peered":"2024-09-17T21:19:49.507824+0000","last_clean":"2024-09-17T21:19:49.507824+0000","last_became_active":"2024-09-17T21:19:20.291104+0000","last_became_peered":"2024-09-17T21:19:20.291104+0000","last_unstale":"2024-09-17T21:19:49.507824+0000","last_undegraded":"2024-09-17T21:19:49.507824+0000","last_fullsized":"2024-09-17T21:19:49.507824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:08:17.742887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013256229999999999,"stat_sum":{"num_bytes":29310062,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":9,"num_read":2799,"num_read_kb":8481,"num_write":251,"num_write_kb":9099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":390798,"num_keys_recovered":27,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"97'323","reported_seq":5044,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.463844+0000","last_change":"2024-09-17T21:19:05.417322+0000","last_active":"2024-09-17T21:19:49.463844+0000","last_peered":"2024-09-17T21:19:49.463844+0000","last_clean":"2024-09-17T21:19:49.463844+0000","last_became_active":"2024-09-17T21:19:05.415967+0000","last_became_peered":"2024-09-17T21:19:05.415967+0000","last_unstale":"2024-09-17T21:19:49.463844+0000","last_undegraded":"2024-09-17T21:19:49.463844+0000","last_fullsized":"2024-09-17T21:19:49.463844+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:11:53.696730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018973079999999999,"stat_sum":{"num_bytes":31939689,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":14,"num_read":5343,"num_read_kb":15766,"num_write":366,"num_write_kb":8705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"91'123","reported_seq":1145,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.498049+0000","last_change":"2024-09-17T21:19:05.416924+0000","last_active":"2024-09-17T21:19:49.498049+0000","last_peered":"2024-09-17T21:19:49.498049+0000","last_clean":"2024-09-17T21:19:49.498049+0000","last_became_active":"2024-09-17T21:19:05.416249+0000","last_became_peered":"2024-09-17T21:19:05.416249+0000","last_unstale":"2024-09-17T21:19:49.498049+0000","last_undegraded":"2024-09-17T21:19:49.498049+0000","last_fullsized":"2024-09-17T21:19:49.498049+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:14:52.301787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034186630000000002,"stat_sum":{"num_bytes":6577168,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":906,"num_read_kb":8483,"num_write":116,"num_write_kb":2018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"97'378","reported_seq":4235,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.484941+0000","last_change":"2024-09-17T21:19:20.291675+0000","last_active":"2024-09-17T21:19:49.484941+0000","last_peered":"2024-09-17T21:19:49.484941+0000","last_clean":"2024-09-17T21:19:49.484941+0000","last_became_active":"2024-09-17T21:19:20.290844+0000","last_became_peered":"2024-09-17T21:19:20.290844+0000","last_unstale":"2024-09-17T21:19:49.484941+0000","last_undegraded":"2024-09-17T21:19:49.484941+0000","last_fullsized":"2024-09-17T21:19:49.484941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:52:21.185175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089504999999999999,"stat_sum":{"num_bytes":12681216,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":14,"num_read":6623,"num_read_kb":7636,"num_write":235,"num_write_kb":1682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":1443328,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"90'250","reported_seq":4932,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.482041+0000","last_change":"2024-09-17T21:18:13.314063+0000","last_active":"2024-09-17T21:19:49.482041+0000","last_peered":"2024-09-17T21:19:49.482041+0000","last_clean":"2024-09-17T21:19:49.482041+0000","last_became_active":"2024-09-17T21:18:11.301426+0000","last_became_peered":"2024-09-17T21:18:11.301426+0000","last_unstale":"2024-09-17T21:19:49.482041+0000","last_undegraded":"2024-09-17T21:19:49.482041+0000","last_fullsized":"2024-09-17T21:19:49.482041+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:36:57.997050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027366090000000001,"stat_sum":{"num_bytes":19332680,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":12,"num_read":6031,"num_read_kb":10890,"num_write":222,"num_write_kb":1830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"90'164","reported_seq":2477,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.491243+0000","last_change":"2024-09-17T21:18:13.314862+0000","last_active":"2024-09-17T21:19:49.491243+0000","last_peered":"2024-09-17T21:19:49.491243+0000","last_clean":"2024-09-17T21:19:49.491243+0000","last_became_active":"2024-09-17T21:18:11.300793+0000","last_became_peered":"2024-09-17T21:18:11.300793+0000","last_unstale":"2024-09-17T21:19:49.491243+0000","last_undegraded":"2024-09-17T21:19:49.491243+0000","last_fullsized":"2024-09-17T21:19:49.491243+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:00:17.519314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047673000000000002,"stat_sum":{"num_bytes":23871535,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":13,"num_read":2215,"num_read_kb":11197,"num_write":172,"num_write_kb":2905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"97'169","reported_seq":676,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.493256+0000","last_change":"2024-09-17T21:19:48.295801+0000","last_active":"2024-09-17T21:19:49.493256+0000","last_peered":"2024-09-17T21:19:49.493256+0000","last_clean":"2024-09-17T21:19:49.493256+0000","last_became_active":"2024-09-17T21:19:48.294966+0000","last_became_peered":"2024-09-17T21:19:48.294966+0000","last_unstale":"2024-09-17T21:19:49.493256+0000","last_undegraded":"2024-09-17T21:19:49.493256+0000","last_fullsized":"2024-09-17T21:19:49.493256+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:46:23.109713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14936387,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":17,"num_read":1033,"num_read_kb":4297,"num_write":192,"num_write_kb":1666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":1971619,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"97'176","reported_seq":339,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.489834+0000","last_change":"2024-09-17T21:19:17.563381+0000","last_active":"2024-09-17T21:19:49.489834+0000","last_peered":"2024-09-17T21:19:49.489834+0000","last_clean":"2024-09-17T21:19:49.489834+0000","last_became_active":"2024-09-17T21:19:17.291420+0000","last_became_peered":"2024-09-17T21:19:17.291420+0000","last_unstale":"2024-09-17T21:19:49.489834+0000","last_undegraded":"2024-09-17T21:19:49.489834+0000","last_fullsized":"2024-09-17T21:19:49.489834+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:05:14.977840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088962999999999998,"stat_sum":{"num_bytes":33351770,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":15,"num_read":1278,"num_read_kb":6176,"num_write":226,"num_write_kb":3801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":2246144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"90'427","reported_seq":1581,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.479578+0000","last_change":"2024-09-17T21:19:05.411424+0000","last_active":"2024-09-17T21:19:49.479578+0000","last_peered":"2024-09-17T21:19:49.479578+0000","last_clean":"2024-09-17T21:19:49.479578+0000","last_became_active":"2024-09-17T21:19:05.410746+0000","last_became_peered":"2024-09-17T21:19:05.410746+0000","last_unstale":"2024-09-17T21:19:49.479578+0000","last_undegraded":"2024-09-17T21:19:49.479578+0000","last_fullsized":"2024-09-17T21:19:49.479578+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:39:26.271869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001825658,"stat_sum":{"num_bytes":13797744,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":8,"num_read":2800,"num_read_kb":8057,"num_write":702,"num_write_kb":11584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"90'540","reported_seq":3794,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.292780+0000","last_change":"2024-09-17T21:19:20.288548+0000","last_active":"2024-09-17T21:19:49.292780+0000","last_peered":"2024-09-17T21:19:49.292780+0000","last_clean":"2024-09-17T21:19:49.292780+0000","last_became_active":"2024-09-17T21:19:20.287118+0000","last_became_peered":"2024-09-17T21:19:20.287118+0000","last_unstale":"2024-09-17T21:19:49.292780+0000","last_undegraded":"2024-09-17T21:19:49.292780+0000","last_fullsized":"2024-09-17T21:19:49.292780+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:24:35.882769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027112149999999999,"stat_sum":{"num_bytes":54361200,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":13,"num_read":4828,"num_read_kb":20619,"num_write":806,"num_write_kb":3688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":3174038,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"90'218","reported_seq":2891,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.498954+0000","last_change":"2024-09-17T21:18:13.307234+0000","last_active":"2024-09-17T21:19:49.498954+0000","last_peered":"2024-09-17T21:19:49.498954+0000","last_clean":"2024-09-17T21:19:49.498954+0000","last_became_active":"2024-09-17T21:18:11.299817+0000","last_became_peered":"2024-09-17T21:18:11.299817+0000","last_unstale":"2024-09-17T21:19:49.498954+0000","last_undegraded":"2024-09-17T21:19:49.498954+0000","last_fullsized":"2024-09-17T21:19:49.498954+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:11:19.875068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002281113,"stat_sum":{"num_bytes":40170775,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":16,"num_read":3510,"num_read_kb":12883,"num_write":172,"num_write_kb":12351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":172,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.292674+0000","last_change":"2024-09-17T21:18:03.059715+0000","last_active":"2024-09-17T21:19:49.292674+0000","last_peered":"2024-09-17T21:19:49.292674+0000","last_clean":"2024-09-17T21:19:49.292674+0000","last_became_active":"2024-09-17T21:18:01.213047+0000","last_became_peered":"2024-09-17T21:18:01.213047+0000","last_unstale":"2024-09-17T21:19:49.292674+0000","last_undegraded":"2024-09-17T21:19:49.292674+0000","last_fullsized":"2024-09-17T21:19:49.292674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:46:35.222373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"90'354","reported_seq":4095,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.503211+0000","last_change":"2024-09-17T21:19:20.290612+0000","last_active":"2024-09-17T21:19:49.503211+0000","last_peered":"2024-09-17T21:19:49.503211+0000","last_clean":"2024-09-17T21:19:49.503211+0000","last_became_active":"2024-09-17T21:19:20.289862+0000","last_became_peered":"2024-09-17T21:19:20.289862+0000","last_unstale":"2024-09-17T21:19:49.503211+0000","last_undegraded":"2024-09-17T21:19:49.503211+0000","last_fullsized":"2024-09-17T21:19:49.503211+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:11:18.433106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084327200000000005,"stat_sum":{"num_bytes":25392978,"num_objects":30,"num_object_clones":4,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":5715,"num_read_kb":28649,"num_write":368,"num_write_kb":13510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":17516408,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":172,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.314291+0000","last_change":"2024-09-17T21:18:03.073440+0000","last_active":"2024-09-17T21:19:49.314291+0000","last_peered":"2024-09-17T21:19:49.314291+0000","last_clean":"2024-09-17T21:19:49.314291+0000","last_became_active":"2024-09-17T21:18:01.048806+0000","last_became_peered":"2024-09-17T21:18:01.048806+0000","last_unstale":"2024-09-17T21:19:49.314291+0000","last_undegraded":"2024-09-17T21:19:49.314291+0000","last_fullsized":"2024-09-17T21:19:49.314291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:26.487163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094161400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"96'165","reported_seq":1772,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.496613+0000","last_change":"2024-09-17T21:19:48.301684+0000","last_active":"2024-09-17T21:19:49.496613+0000","last_peered":"2024-09-17T21:19:49.496613+0000","last_clean":"2024-09-17T21:19:49.496613+0000","last_became_active":"2024-09-17T21:19:48.301027+0000","last_became_peered":"2024-09-17T21:19:48.301027+0000","last_unstale":"2024-09-17T21:19:49.496613+0000","last_undegraded":"2024-09-17T21:19:49.496613+0000","last_fullsized":"2024-09-17T21:19:49.496613+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:51:22.426818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28021855,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":16,"num_read":1517,"num_read_kb":18207,"num_write":177,"num_write_kb":11777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":995423,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":184,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.292967+0000","last_change":"2024-09-17T21:19:17.294067+0000","last_active":"2024-09-17T21:19:49.292967+0000","last_peered":"2024-09-17T21:19:49.292967+0000","last_clean":"2024-09-17T21:19:49.292967+0000","last_became_active":"2024-09-17T21:19:17.293357+0000","last_became_peered":"2024-09-17T21:19:17.293357+0000","last_unstale":"2024-09-17T21:19:49.292967+0000","last_undegraded":"2024-09-17T21:19:49.292967+0000","last_fullsized":"2024-09-17T21:19:49.292967+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:13:10.070543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099219900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"94'209","reported_seq":4926,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.314588+0000","last_change":"2024-09-17T21:19:48.296853+0000","last_active":"2024-09-17T21:19:49.314588+0000","last_peered":"2024-09-17T21:19:49.314588+0000","last_clean":"2024-09-17T21:19:49.314588+0000","last_became_active":"2024-09-17T21:19:48.296024+0000","last_became_peered":"2024-09-17T21:19:48.296024+0000","last_unstale":"2024-09-17T21:19:49.314588+0000","last_undegraded":"2024-09-17T21:19:49.314588+0000","last_fullsized":"2024-09-17T21:19:49.314588+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:00:05.346630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078465700000000002,"stat_sum":{"num_bytes":29615652,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":15,"num_read":3396,"num_read_kb":14975,"num_write":158,"num_write_kb":6664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":5833764,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":184,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.312007+0000","last_change":"2024-09-17T21:19:17.294907+0000","last_active":"2024-09-17T21:19:49.312007+0000","last_peered":"2024-09-17T21:19:49.312007+0000","last_clean":"2024-09-17T21:19:49.312007+0000","last_became_active":"2024-09-17T21:19:17.293525+0000","last_became_peered":"2024-09-17T21:19:17.293525+0000","last_unstale":"2024-09-17T21:19:49.312007+0000","last_undegraded":"2024-09-17T21:19:49.312007+0000","last_fullsized":"2024-09-17T21:19:49.312007+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:32:46.627308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006222,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"90'672","reported_seq":10248,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.489413+0000","last_change":"2024-09-17T21:19:49.297844+0000","last_active":"2024-09-17T21:19:49.489413+0000","last_peered":"2024-09-17T21:19:49.489413+0000","last_clean":"2024-09-17T21:19:49.489413+0000","last_became_active":"2024-09-17T21:19:49.296523+0000","last_became_peered":"2024-09-17T21:19:49.296523+0000","last_unstale":"2024-09-17T21:19:49.489413+0000","last_undegraded":"2024-09-17T21:19:49.489413+0000","last_fullsized":"2024-09-17T21:19:49.489413+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:53:25.664030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036493720000000001,"stat_sum":{"num_bytes":60233759,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":14,"num_read":14589,"num_read_kb":25367,"num_write":411,"num_write_kb":16118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":5255199,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"94'141","reported_seq":1021,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.495190+0000","last_change":"2024-09-17T21:19:17.416920+0000","last_active":"2024-09-17T21:19:49.495190+0000","last_peered":"2024-09-17T21:19:49.495190+0000","last_clean":"2024-09-17T21:19:49.495190+0000","last_became_active":"2024-09-17T21:19:17.294385+0000","last_became_peered":"2024-09-17T21:19:17.294385+0000","last_unstale":"2024-09-17T21:19:49.495190+0000","last_undegraded":"2024-09-17T21:19:49.495190+0000","last_fullsized":"2024-09-17T21:19:49.495190+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:18:45.914186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084988700000000002,"stat_sum":{"num_bytes":9505792,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":12,"num_read":650,"num_read_kb":3757,"num_write":131,"num_write_kb":2398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":2000896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":94,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.313162+0000","last_change":"2024-09-17T21:19:05.415157+0000","last_active":"2024-09-17T21:19:49.313162+0000","last_peered":"2024-09-17T21:19:49.313162+0000","last_clean":"2024-09-17T21:19:49.313162+0000","last_became_active":"2024-09-17T21:19:05.414343+0000","last_became_peered":"2024-09-17T21:19:05.414343+0000","last_unstale":"2024-09-17T21:19:49.313162+0000","last_undegraded":"2024-09-17T21:19:49.313162+0000","last_fullsized":"2024-09-17T21:19:49.313162+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:27:06.755871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"97'93","reported_seq":1429,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.497721+0000","last_change":"2024-09-17T21:19:45.836556+0000","last_active":"2024-09-17T21:19:49.497721+0000","last_peered":"2024-09-17T21:19:49.497721+0000","last_clean":"2024-09-17T21:19:49.497721+0000","last_became_active":"2024-09-17T21:19:45.701151+0000","last_became_peered":"2024-09-17T21:19:45.701151+0000","last_unstale":"2024-09-17T21:19:49.497721+0000","last_undegraded":"2024-09-17T21:19:49.497721+0000","last_fullsized":"2024-09-17T21:19:49.497721+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:44:51.102256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049125000000000004,"stat_sum":{"num_bytes":29712416,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":9,"num_read":1088,"num_read_kb":9581,"num_write":90,"num_write_kb":3500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2501152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":156,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.312655+0000","last_change":"2024-09-17T21:19:45.702432+0000","last_active":"2024-09-17T21:19:49.312655+0000","last_peered":"2024-09-17T21:19:49.312655+0000","last_clean":"2024-09-17T21:19:49.312655+0000","last_became_active":"2024-09-17T21:19:45.701525+0000","last_became_peered":"2024-09-17T21:19:45.701525+0000","last_unstale":"2024-09-17T21:19:49.312655+0000","last_undegraded":"2024-09-17T21:19:49.312655+0000","last_fullsized":"2024-09-17T21:19:49.312655+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:39:24.504928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"90'653","reported_seq":1870,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.486502+0000","last_change":"2024-09-17T21:19:20.291992+0000","last_active":"2024-09-17T21:19:49.486502+0000","last_peered":"2024-09-17T21:19:49.486502+0000","last_clean":"2024-09-17T21:19:49.486502+0000","last_became_active":"2024-09-17T21:19:20.290804+0000","last_became_peered":"2024-09-17T21:19:20.290804+0000","last_unstale":"2024-09-17T21:19:49.486502+0000","last_undegraded":"2024-09-17T21:19:49.486502+0000","last_fullsized":"2024-09-17T21:19:49.486502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:18:36.769872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066398399999999995,"stat_sum":{"num_bytes":9214050,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":7,"num_read":3863,"num_read_kb":24077,"num_write":1056,"num_write_kb":6204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":8387726,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":172,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.292363+0000","last_change":"2024-09-17T21:18:03.059156+0000","last_active":"2024-09-17T21:19:49.292363+0000","last_peered":"2024-09-17T21:19:49.292363+0000","last_clean":"2024-09-17T21:19:49.292363+0000","last_became_active":"2024-09-17T21:18:01.049180+0000","last_became_peered":"2024-09-17T21:18:01.049180+0000","last_unstale":"2024-09-17T21:19:49.292363+0000","last_undegraded":"2024-09-17T21:19:49.292363+0000","last_fullsized":"2024-09-17T21:19:49.292363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:27.069794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077408,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"97'392","reported_seq":2544,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.504646+0000","last_change":"2024-09-17T21:18:13.315294+0000","last_active":"2024-09-17T21:19:49.504646+0000","last_peered":"2024-09-17T21:19:49.504646+0000","last_clean":"2024-09-17T21:19:49.504646+0000","last_became_active":"2024-09-17T21:18:11.294154+0000","last_became_peered":"2024-09-17T21:18:11.294154+0000","last_unstale":"2024-09-17T21:19:49.504646+0000","last_undegraded":"2024-09-17T21:19:49.504646+0000","last_fullsized":"2024-09-17T21:19:49.504646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:55:56.521608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049408099999999995,"stat_sum":{"num_bytes":48777593,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":14,"num_read":3903,"num_read_kb":22838,"num_write":506,"num_write_kb":9777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":23,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.313872+0000","last_change":"2024-09-17T21:19:45.706219+0000","last_active":"2024-09-17T21:19:49.313872+0000","last_peered":"2024-09-17T21:19:49.313872+0000","last_clean":"2024-09-17T21:19:49.313872+0000","last_became_active":"2024-09-17T21:19:45.698723+0000","last_became_peered":"2024-09-17T21:19:45.698723+0000","last_unstale":"2024-09-17T21:19:49.313872+0000","last_undegraded":"2024-09-17T21:19:49.313872+0000","last_fullsized":"2024-09-17T21:19:49.313872+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:16:21.602983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":287,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.292448+0000","last_change":"2024-09-17T21:19:17.427717+0000","last_active":"2024-09-17T21:19:49.292448+0000","last_peered":"2024-09-17T21:19:49.292448+0000","last_clean":"2024-09-17T21:19:49.292448+0000","last_became_active":"2024-09-17T21:19:17.297187+0000","last_became_peered":"2024-09-17T21:19:17.297187+0000","last_unstale":"2024-09-17T21:19:49.292448+0000","last_undegraded":"2024-09-17T21:19:49.292448+0000","last_fullsized":"2024-09-17T21:19:49.292448+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_clean_scrub_stamp":"2024-09-17T21:17:55.982339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:19:07.636190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'264","reported_seq":4986,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.509909+0000","last_change":"2024-09-17T21:19:48.300950+0000","last_active":"2024-09-17T21:19:49.509909+0000","last_peered":"2024-09-17T21:19:49.509909+0000","last_clean":"2024-09-17T21:19:49.509909+0000","last_became_active":"2024-09-17T21:19:48.298503+0000","last_became_peered":"2024-09-17T21:19:48.298503+0000","last_unstale":"2024-09-17T21:19:49.509909+0000","last_undegraded":"2024-09-17T21:19:49.509909+0000","last_fullsized":"2024-09-17T21:19:49.509909+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:43:29.626038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26432544,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":11,"num_read":5964,"num_read_kb":42085,"num_write":202,"num_write_kb":15914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":7335984,"num_keys_recovered":44,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'426","reported_seq":4379,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.488972+0000","last_change":"2024-09-17T21:19:48.302452+0000","last_active":"2024-09-17T21:19:49.488972+0000","last_peered":"2024-09-17T21:19:49.488972+0000","last_clean":"2024-09-17T21:19:49.488972+0000","last_became_active":"2024-09-17T21:19:48.300411+0000","last_became_peered":"2024-09-17T21:19:48.300411+0000","last_unstale":"2024-09-17T21:19:49.488972+0000","last_undegraded":"2024-09-17T21:19:49.488972+0000","last_fullsized":"2024-09-17T21:19:49.488972+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:00:02.549768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023749890000000001,"stat_sum":{"num_bytes":25120445,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":8,"num_read":5300,"num_read_kb":25933,"num_write":687,"num_write_kb":3800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":2266301,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"97'182","reported_seq":1467,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.486397+0000","last_change":"2024-09-17T21:19:34.613370+0000","last_active":"2024-09-17T21:19:49.486397+0000","last_peered":"2024-09-17T21:19:49.486397+0000","last_clean":"2024-09-17T21:19:49.486397+0000","last_became_active":"2024-09-17T21:19:34.611967+0000","last_became_peered":"2024-09-17T21:19:34.611967+0000","last_unstale":"2024-09-17T21:19:49.486397+0000","last_undegraded":"2024-09-17T21:19:49.486397+0000","last_fullsized":"2024-09-17T21:19:49.486397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:15:36.926436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021951829999999999,"stat_sum":{"num_bytes":44095586,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":16,"num_read":1148,"num_read_kb":6584,"num_write":156,"num_write_kb":1277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"97'218","reported_seq":2450,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.508647+0000","last_change":"2024-09-17T21:19:20.292113+0000","last_active":"2024-09-17T21:19:49.508647+0000","last_peered":"2024-09-17T21:19:49.508647+0000","last_clean":"2024-09-17T21:19:49.508647+0000","last_became_active":"2024-09-17T21:19:20.290717+0000","last_became_peered":"2024-09-17T21:19:20.290717+0000","last_unstale":"2024-09-17T21:19:49.508647+0000","last_undegraded":"2024-09-17T21:19:49.508647+0000","last_fullsized":"2024-09-17T21:19:49.508647+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:46:48.743549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048704800000000002,"stat_sum":{"num_bytes":40393376,"num_objects":29,"num_object_clones":4,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":10,"num_read":3008,"num_read_kb":13012,"num_write":189,"num_write_kb":12063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":1215535,"num_keys_recovered":36,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'162","reported_seq":316,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.482275+0000","last_change":"2024-09-17T21:18:54.392198+0000","last_active":"2024-09-17T21:19:49.482275+0000","last_peered":"2024-09-17T21:19:49.482275+0000","last_clean":"2024-09-17T21:19:49.482275+0000","last_became_active":"2024-09-17T21:18:54.343340+0000","last_became_peered":"2024-09-17T21:18:54.343340+0000","last_unstale":"2024-09-17T21:19:49.482275+0000","last_undegraded":"2024-09-17T21:19:49.482275+0000","last_fullsized":"2024-09-17T21:19:49.482275+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:57:24.876447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015319789999999999,"stat_sum":{"num_bytes":28475904,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":19,"num_read":2175,"num_read_kb":7251,"num_write":128,"num_write_kb":1081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":166400,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"97'271","reported_seq":684,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.485512+0000","last_change":"2024-09-17T21:19:49.303045+0000","last_active":"2024-09-17T21:19:49.485512+0000","last_peered":"2024-09-17T21:19:49.485512+0000","last_clean":"2024-09-17T21:19:49.485512+0000","last_became_active":"2024-09-17T21:19:49.298871+0000","last_became_peered":"2024-09-17T21:19:49.298871+0000","last_unstale":"2024-09-17T21:19:49.485512+0000","last_undegraded":"2024-09-17T21:19:49.485512+0000","last_fullsized":"2024-09-17T21:19:49.485512+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:12:55.990040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015238949999999999,"stat_sum":{"num_bytes":18204702,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":10,"num_read":1220,"num_read_kb":2792,"num_write":425,"num_write_kb":9544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":162846,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"97'317","reported_seq":6153,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.485787+0000","last_change":"2024-09-17T21:19:05.416474+0000","last_active":"2024-09-17T21:19:49.485787+0000","last_peered":"2024-09-17T21:19:49.485787+0000","last_clean":"2024-09-17T21:19:49.485787+0000","last_became_active":"2024-09-17T21:19:05.415977+0000","last_became_peered":"2024-09-17T21:19:05.415977+0000","last_unstale":"2024-09-17T21:19:49.485787+0000","last_undegraded":"2024-09-17T21:19:49.485787+0000","last_fullsized":"2024-09-17T21:19:49.485787+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:37:09.170992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003632209,"stat_sum":{"num_bytes":38429355,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":10,"num_read":6787,"num_read_kb":22180,"num_write":242,"num_write_kb":23905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"97'170","reported_seq":1749,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.508245+0000","last_change":"2024-09-17T21:19:05.410271+0000","last_active":"2024-09-17T21:19:49.508245+0000","last_peered":"2024-09-17T21:19:49.508245+0000","last_clean":"2024-09-17T21:19:49.508245+0000","last_became_active":"2024-09-17T21:19:05.409905+0000","last_became_peered":"2024-09-17T21:19:05.409905+0000","last_unstale":"2024-09-17T21:19:49.508245+0000","last_undegraded":"2024-09-17T21:19:49.508245+0000","last_fullsized":"2024-09-17T21:19:49.508245+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:58:18.528702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002454758,"stat_sum":{"num_bytes":25447531,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":15,"num_read":2096,"num_read_kb":4760,"num_write":160,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"97'145","reported_seq":441,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.467077+0000","last_change":"2024-09-17T21:19:17.606702+0000","last_active":"2024-09-17T21:19:49.467077+0000","last_peered":"2024-09-17T21:19:49.467077+0000","last_clean":"2024-09-17T21:19:49.467077+0000","last_became_active":"2024-09-17T21:19:17.296517+0000","last_became_peered":"2024-09-17T21:19:17.296517+0000","last_unstale":"2024-09-17T21:19:49.467077+0000","last_undegraded":"2024-09-17T21:19:49.467077+0000","last_fullsized":"2024-09-17T21:19:49.467077+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:27:30.015989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001519261,"stat_sum":{"num_bytes":9332826,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":1834,"num_read_kb":6391,"num_write":113,"num_write_kb":2167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":1208468,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"90'254","reported_seq":2742,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.479440+0000","last_change":"2024-09-17T21:18:13.305680+0000","last_active":"2024-09-17T21:19:49.479440+0000","last_peered":"2024-09-17T21:19:49.479440+0000","last_clean":"2024-09-17T21:19:49.479440+0000","last_became_active":"2024-09-17T21:18:11.299946+0000","last_became_peered":"2024-09-17T21:18:11.299946+0000","last_unstale":"2024-09-17T21:19:49.479440+0000","last_undegraded":"2024-09-17T21:19:49.479440+0000","last_fullsized":"2024-09-17T21:19:49.479440+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:46.539953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225884,"stat_sum":{"num_bytes":42040157,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":13,"num_read":2698,"num_read_kb":9618,"num_write":263,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"97'197","reported_seq":1967,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.498664+0000","last_change":"2024-09-17T21:19:17.399377+0000","last_active":"2024-09-17T21:19:49.498664+0000","last_peered":"2024-09-17T21:19:49.498664+0000","last_clean":"2024-09-17T21:19:49.498664+0000","last_became_active":"2024-09-17T21:19:17.290182+0000","last_became_peered":"2024-09-17T21:19:17.290182+0000","last_unstale":"2024-09-17T21:19:49.498664+0000","last_undegraded":"2024-09-17T21:19:49.498664+0000","last_fullsized":"2024-09-17T21:19:49.498664+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:08:43.183765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013045489999999999,"stat_sum":{"num_bytes":43837519,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":3776,"num_read_kb":11955,"num_write":175,"num_write_kb":3255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":2706590,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"90'384","reported_seq":8436,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T21:19:49.498316+0000","last_change":"2024-09-17T21:18:13.306520+0000","last_active":"2024-09-17T21:19:49.498316+0000","last_peered":"2024-09-17T21:19:49.498316+0000","last_clean":"2024-09-17T21:19:49.498316+0000","last_became_active":"2024-09-17T21:18:11.294953+0000","last_became_peered":"2024-09-17T21:18:11.294953+0000","last_unstale":"2024-09-17T21:19:49.498316+0000","last_undegraded":"2024-09-17T21:19:49.498316+0000","last_fullsized":"2024-09-17T21:19:49.498316+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:33.425191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019790290000000002,"stat_sum":{"num_bytes":52479059,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":18,"num_read":9238,"num_read_kb":41009,"num_write":335,"num_write_kb":2834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":921330736,"num_objects":959,"num_object_clones":195,"num_object_copies":1918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":959,"num_whiteouts":401,"num_read":118629,"num_read_kb":465133,"num_write":9580,"num_write_kb":210405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":913,"num_bytes_recovered":67347102,"num_keys_recovered":223,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210984960,"data_stored":278027421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8818,"ondisk_log_size":8818,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89412,"kb_used_data":45740,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94282428,"statfs":{"total":96636764160,"available":96545206272,"internally_reserved":0,"allocated":46837760,"data_stored":63652222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":44693956},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":102500,"kb_used_data":45824,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94269340,"statfs":{"total":96636764160,"available":96531804160,"internally_reserved":0,"allocated":46923776,"data_stored":53507981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":50395588},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56640,"kb_used_data":29612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94315200,"statfs":{"total":96636764160,"available":96578764800,"internally_reserved":0,"allocated":30322688,"data_stored":40166507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75644,"kb_used_data":36136,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94296196,"statfs":{"total":96636764160,"available":96559304704,"internally_reserved":0,"allocated":37003264,"data_stored":54620993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":40434116},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73388,"kb_used_data":33880,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94298452,"statfs":{"total":96636764160,"available":96561614848,"internally_reserved":0,"allocated":34693120,"data_stored":42885997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":40434116},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":51539607579,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":33034240,"data_stored":41893467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36012032,"data_stored":54289896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13430784,"data_stored":18128957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28819456,"data_stored":39310599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53899264,"data_stored":61138075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45789184,"data_stored":63266427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:19:50.488 INFO:tasks.thrashosds.thrasher:Setting 3.3 to [2, 4, 5, 1] 2024-09-17T21:19:50.488 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.3', '2', '4', '5', '1'] 2024-09-17T21:19:50.488 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.3 2 4 5 1 2024-09-17T21:19:50.491 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:50.492 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:50.492 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:19:50.492 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:50.509 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:19:50.532 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone24 from 0xe0e4c00 (overlap 0x8c08a00) to 0x4604400 2024-09-17T21:19:50.596 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:50.594+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:50.614 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:50.614 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:50.634 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:50.634 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:50.650 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:50.651 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:50.670 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:50.670 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:50.671 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:50.667+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:50.676 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:50.676 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:50.690 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:19:50.691 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:50.691 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:50.710 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:50.710 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:50.772 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:50.770+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:50.802 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:50.802 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:50.815 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:19:50.873 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:50.867+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:50.974 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:50.970+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:50.982 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:50.982 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:50.995 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:19:51.081 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:51.078+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:51.150 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:51.150 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:51.164 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:19:51.176 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:51.171+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:51.276 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:51.270+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:51.317 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:19:51.333 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:51.334 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:51.348 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:19:51.395 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:51.391+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:51.477 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:51.474+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:51.481 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:51.481 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:51.489 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-17T21:19:51.496 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:19:51.582 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:51.579+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:51.677 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:51.677 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:51.680 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:51.675+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:51.692 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:19:51.708 INFO:teuthology.orchestra.run.smithi083.stderr:set 3.3 pg_upmap_items mapping to [2->4,5->1] 2024-09-17T21:19:51.714 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-17T21:19:51.780 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:51.775+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:51.867 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:51.867 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:51.881 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:51.878+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:51.884 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:19:51.982 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:51.975+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:52.061 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:52.061 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:52.075 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:19:52.082 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:52.078+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:52.184 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:52.178+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:52.242 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:52.242 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:52.256 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:19:52.285 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:52.282+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:52.377 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:52.377 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:52.385 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:52.382+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:52.392 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:19:52.487 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:52.486+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:52.517 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:52.517 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:52.533 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:19:52.588 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:52.583+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:52.655 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:52.655 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:52.669 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:19:52.711 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:52.707+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:52.727 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:19:52.764 INFO:teuthology.orchestra.run.smithi184.stdout:4031 read 0x2353c00 thru 0x235a338 (0x6739 bytes) 2024-09-17T21:19:52.789 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:52.786+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:52.814 INFO:teuthology.orchestra.run.smithi184.stdout:4033 trunc from 0xe0e4c00 to 0x583f000 2024-09-17T21:19:52.839 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:52.839 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:52.857 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:19:52.868 INFO:teuthology.orchestra.run.smithi184.stdout:4035 punch from 0x438000 to 0x43dc00, (0x5c00 bytes) 2024-09-17T21:19:52.872 INFO:teuthology.orchestra.run.smithi184.stdout:4037 read 0x29fb400 thru 0x2a01f53 (0x6b54 bytes) 2024-09-17T21:19:52.883 INFO:teuthology.orchestra.run.smithi184.stdout:4038 clone 26 order 22 su 32768 sc 15 2024-09-17T21:19:52.890 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:52.883+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:52.897 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:52.898 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:52.915 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:52.915 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:52.933 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:52.933 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:52.953 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:52.953 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:52.973 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:52.973 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:52.979 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:52.979 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:52.979 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:19:52.979 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:52.991 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:52.987+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:52.993 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:19:52.994 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:52.994 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:53.093 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:53.087+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:53.108 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:53.108 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:53.108 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:53.108 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:53.108 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:53.122 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:19:53.193 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:53.191+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:53.246 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:53.246 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:53.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:53.246 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:53.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.246 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:494 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:53.246 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:53.246 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.046230163999994, 2024-09-17T21:19:53.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046244995999999997, 2024-09-17T21:19:53.246 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.247 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.247 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.247 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.247 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:53.247 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:53.247 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 494 2024-09-17T21:19:53.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.247 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:53.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:19:53.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197329+0000", 2024-09-17T21:19:53.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.173e-06 2024-09-17T21:19:53.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197334+0000", 2024-09-17T21:19:53.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0499999999999999e-06 2024-09-17T21:19:53.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197352+0000", 2024-09-17T21:19:53.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7133e-05 2024-09-17T21:19:53.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197367+0000", 2024-09-17T21:19:53.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5123000000000001e-05 2024-09-17T21:19:53.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:19:53.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044989829000000002 2024-09-17T21:19:53.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242526+0000", 2024-09-17T21:19:53.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016968199999999999 2024-09-17T21:19:53.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243573+0000", 2024-09-17T21:19:53.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010470060000000001 2024-09-17T21:19:53.251 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.251 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.251 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.251 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:524 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:53.251 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:53.251 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.043484136000004, 2024-09-17T21:19:53.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045737414999999997, 2024-09-17T21:19:53.252 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.252 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.252 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.252 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.252 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:53.252 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:53.252 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 524 2024-09-17T21:19:53.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.252 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:53.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:19:53.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200076+0000", 2024-09-17T21:19:53.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:19:53.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200081+0000", 2024-09-17T21:19:53.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7030000000000003e-06 2024-09-17T21:19:53.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200099+0000", 2024-09-17T21:19:53.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7836999999999999e-05 2024-09-17T21:19:53.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200113+0000", 2024-09-17T21:19:53.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3570000000000001e-05 2024-09-17T21:19:53.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244866+0000", 2024-09-17T21:19:53.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044753573999999997 2024-09-17T21:19:53.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244972+0000", 2024-09-17T21:19:53.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106111 2024-09-17T21:19:53.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245812+0000", 2024-09-17T21:19:53.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083923999999999999 2024-09-17T21:19:53.256 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.256 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.256 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.256 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:525 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3670016~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:53.256 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:53.256 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.043332754000005, 2024-09-17T21:19:53.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046645697, 2024-09-17T21:19:53.256 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.256 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.256 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.256 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.256 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:53.257 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:53.257 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 525 2024-09-17T21:19:53.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.257 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:53.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:19:53.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:19:53.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:19:53.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200234+0000", 2024-09-17T21:19:53.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:19:53.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200254+0000", 2024-09-17T21:19:53.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9959999999999999e-05 2024-09-17T21:19:53.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200293+0000", 2024-09-17T21:19:53.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9755999999999997e-05 2024-09-17T21:19:53.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245871+0000", 2024-09-17T21:19:53.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045577871999999998 2024-09-17T21:19:53.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246030+0000", 2024-09-17T21:19:53.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158975 2024-09-17T21:19:53.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246871+0000", 2024-09-17T21:19:53.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084111699999999995 2024-09-17T21:19:53.260 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.260 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.260 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.261 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:53.261 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:53.261 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.043087079000003, 2024-09-17T21:19:53.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:19:53.261 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.261 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.261 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.261 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.261 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:53.261 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:53.261 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:19:53.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.261 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:53.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:19:53.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:19:53.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:19:53.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:19:53.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:19:53.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:19:53.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:19:53.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:19:53.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:19:53.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:19:53.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:19:53.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:19:53.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:19:53.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:19:53.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:19:53.265 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.265 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.265 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.265 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:53.265 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:53.265 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.042908397000005, 2024-09-17T21:19:53.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:19:53.265 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.265 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.266 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.266 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.266 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:53.266 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:53.266 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:19:53.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.266 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:53.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:19:53.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:19:53.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:19:53.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:19:53.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:19:53.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:19:53.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:19:53.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:19:53.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:19:53.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:19:53.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:19:53.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:19:53.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:19:53.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:19:53.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:19:53.269 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.269 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.269 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.270 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:53.270 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:53.270 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.042849124, 2024-09-17T21:19:53.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:19:53.270 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.270 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.270 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.270 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.270 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:53.270 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:53.270 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:19:53.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.271 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:53.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:19:53.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:19:53.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:19:53.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:19:53.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:19:53.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:19:53.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:19:53.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:19:53.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:19:53.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:19:53.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:19:53.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:19:53.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:19:53.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:19:53.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:19:53.274 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.274 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.274 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.274 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:53.274 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:53.274 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.042706929000005, 2024-09-17T21:19:53.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:19:53.275 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.275 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.275 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.275 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.275 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:53.275 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:53.275 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:19:53.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.275 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:53.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:19:53.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:19:53.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:19:53.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:19:53.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:19:53.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:19:53.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:19:53.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:19:53.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:19:53.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:19:53.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:19:53.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:19:53.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:19:53.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:19:53.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:19:53.278 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.278 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.279 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.279 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:53.279 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:53.279 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.042651006, 2024-09-17T21:19:53.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:19:53.279 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.279 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.279 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.279 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.279 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:53.279 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:53.279 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:19:53.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.280 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:53.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:19:53.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:19:53.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:19:53.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:19:53.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:19:53.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:19:53.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:19:53.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:19:53.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:19:53.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:19:53.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:19:53.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:19:53.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:19:53.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:19:53.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:19:53.283 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.283 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.284 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.284 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:53.284 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:53.284 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.042597388999994, 2024-09-17T21:19:53.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:19:53.284 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.284 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.284 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.284 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.284 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:53.284 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:53.285 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:19:53.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.285 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:53.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:19:53.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:19:53.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:19:53.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:19:53.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:19:53.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:19:53.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:19:53.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:19:53.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:19:53.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:19:53.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:19:53.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:19:53.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:19:53.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:19:53.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:19:53.288 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.288 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.288 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.288 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:19:53.288 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:19:53.288 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 36.733369146999998, 2024-09-17T21:19:53.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:19:53.289 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.289 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.289 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:19:53.289 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.289 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:19:53.289 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:19:53.289 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:19:53.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.289 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:19:53.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:19:53.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:19:53.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:19:53.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:19:53.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:19:53.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:19:53.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:19:53.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:19:53.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:19:53.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:19:53.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:19:53.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:53.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:19:53.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:19:53.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:19:53.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:19:53.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:19:53.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:19:53.293 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.293 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.293 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.293 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2597 3.1f 3:fb0e9615:::rbd_data.1152a8ae2019.000000000000000c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:53.293 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:19:53.293 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.404932719, 2024-09-17T21:19:53.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097064994000000002, 2024-09-17T21:19:53.294 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.294 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.294 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.294 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.294 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:53.294 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:53.294 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2597 2024-09-17T21:19:53.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.294 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:19:53.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:19:53.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838628+0000", 2024-09-17T21:19:53.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.187e-06 2024-09-17T21:19:53.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838654+0000", 2024-09-17T21:19:53.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:19:53.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838696+0000", 2024-09-17T21:19:53.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1226000000000001e-05 2024-09-17T21:19:53.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838715+0000", 2024-09-17T21:19:53.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9646000000000001e-05 2024-09-17T21:19:53.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838937+0000", 2024-09-17T21:19:53.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022163 2024-09-17T21:19:53.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.839284+0000", 2024-09-17T21:19:53.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034753799999999999 2024-09-17T21:19:53.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935691+0000", 2024-09-17T21:19:53.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.096406360999999996 2024-09-17T21:19:53.298 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.298 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.298 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.298 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2641 3.12 3:488a729f:::rbd_data.1152a8ae2019.0000000000000022:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:53.298 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:19:53.298 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.401475946, 2024-09-17T21:19:53.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098280953000000004, 2024-09-17T21:19:53.298 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.298 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.298 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.298 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.298 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:53.299 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:53.299 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2641 2024-09-17T21:19:53.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.299 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:19:53.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:19:53.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842083+0000", 2024-09-17T21:19:53.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5300000000000002e-07 2024-09-17T21:19:53.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842088+0000", 2024-09-17T21:19:53.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:19:53.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842105+0000", 2024-09-17T21:19:53.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6773e-05 2024-09-17T21:19:53.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842119+0000", 2024-09-17T21:19:53.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:19:53.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842320+0000", 2024-09-17T21:19:53.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020052700000000001 2024-09-17T21:19:53.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842845+0000", 2024-09-17T21:19:53.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052508600000000002 2024-09-17T21:19:53.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940363+0000", 2024-09-17T21:19:53.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097518795000000005 2024-09-17T21:19:53.302 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.302 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.302 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.303 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2674 3.12 3:4c87e3ac:::rbd_data.1152a8ae2019.0000000000000033:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:53.303 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:19:53.303 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.401113308999999, 2024-09-17T21:19:53.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098025125000000005, 2024-09-17T21:19:53.303 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.303 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.303 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:53.303 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.303 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:53.303 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:53.303 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2674 2024-09-17T21:19:53.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.304 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:19:53.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:19:53.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842447+0000", 2024-09-17T21:19:53.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3999999999999999e-06 2024-09-17T21:19:53.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842453+0000", 2024-09-17T21:19:53.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:19:53.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842472+0000", 2024-09-17T21:19:53.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9006000000000001e-05 2024-09-17T21:19:53.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842487+0000", 2024-09-17T21:19:53.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4866e-05 2024-09-17T21:19:53.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843594+0000", 2024-09-17T21:19:53.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011073249999999999 2024-09-17T21:19:53.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843668+0000", 2024-09-17T21:19:53.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3188000000000005e-05 2024-09-17T21:19:53.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:19:53.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843893+0000", 2024-09-17T21:19:53.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022503399999999999 2024-09-17T21:19:53.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:53.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850439+0000", 2024-09-17T21:19:53.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065467880000000004 2024-09-17T21:19:53.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:53.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940409+0000", 2024-09-17T21:19:53.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089969511000000002 2024-09-17T21:19:53.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:53.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940444+0000", 2024-09-17T21:19:53.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5345999999999997e-05 2024-09-17T21:19:53.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940470+0000", 2024-09-17T21:19:53.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5964000000000001e-05 2024-09-17T21:19:53.308 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.308 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.308 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.309 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2707 3.1f 3:f805e65e:::rbd_data.1152a8ae2019.0000000000000049:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:53.309 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:19:53.309 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.396066648, 2024-09-17T21:19:53.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088327300999999997, 2024-09-17T21:19:53.309 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.309 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.309 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.309 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.309 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:53.309 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:53.309 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2707 2024-09-17T21:19:53.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.309 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:19:53.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:19:53.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847494+0000", 2024-09-17T21:19:53.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:19:53.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847504+0000", 2024-09-17T21:19:53.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:19:53.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847526+0000", 2024-09-17T21:19:53.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1976000000000001e-05 2024-09-17T21:19:53.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847541+0000", 2024-09-17T21:19:53.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5277e-05 2024-09-17T21:19:53.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856142+0000", 2024-09-17T21:19:53.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086006710000000007 2024-09-17T21:19:53.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856435+0000", 2024-09-17T21:19:53.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029301499999999999 2024-09-17T21:19:53.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935819+0000", 2024-09-17T21:19:53.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079384448999999996 2024-09-17T21:19:53.313 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.313 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.313 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.313 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2730 3.12 3:48b3422a:::rbd_data.1152a8ae2019.0000000000000057:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:53.313 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:19:53.313 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.386873748999999, 2024-09-17T21:19:53.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083879975999999995, 2024-09-17T21:19:53.313 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.313 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.314 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.314 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.314 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:53.314 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:53.314 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2730 2024-09-17T21:19:53.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.314 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:19:53.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:19:53.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856687+0000", 2024-09-17T21:19:53.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:19:53.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856697+0000", 2024-09-17T21:19:53.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0207000000000001e-05 2024-09-17T21:19:53.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856808+0000", 2024-09-17T21:19:53.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011071200000000001 2024-09-17T21:19:53.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856828+0000", 2024-09-17T21:19:53.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0219000000000001e-05 2024-09-17T21:19:53.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856907+0000", 2024-09-17T21:19:53.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9661999999999997e-05 2024-09-17T21:19:53.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.857019+0000", 2024-09-17T21:19:53.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011148499999999999 2024-09-17T21:19:53.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940565+0000", 2024-09-17T21:19:53.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083545791999999994 2024-09-17T21:19:53.317 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.317 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.317 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.317 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2743 3.12 3:4d036e62:::rbd_data.1152a8ae2019.000000000000005f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:53.317 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:19:53.318 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.382928248000001, 2024-09-17T21:19:53.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080059962999999998, 2024-09-17T21:19:53.318 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.318 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.318 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.318 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.318 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:53.318 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:53.318 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2743 2024-09-17T21:19:53.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.318 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:19:53.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:19:53.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860631+0000", 2024-09-17T21:19:53.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:19:53.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860636+0000", 2024-09-17T21:19:53.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5279999999999997e-06 2024-09-17T21:19:53.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860649+0000", 2024-09-17T21:19:53.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-17T21:19:53.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860655+0000", 2024-09-17T21:19:53.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:19:53.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860731+0000", 2024-09-17T21:19:53.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5920000000000005e-05 2024-09-17T21:19:53.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.861070+0000", 2024-09-17T21:19:53.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338535 2024-09-17T21:19:53.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940690+0000", 2024-09-17T21:19:53.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079620503999999995 2024-09-17T21:19:53.322 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.322 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.322 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.322 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3378 3.12 3:4832a51b:::rbd_data.1152a8ae2019.000000000000006c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:53.322 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:19:53.322 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.354564398000001, 2024-09-17T21:19:53.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051812516000000003, 2024-09-17T21:19:53.322 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.322 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.322 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:53.322 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.323 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:53.323 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:53.323 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3378 2024-09-17T21:19:53.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.323 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:19:53.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:19:53.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888995+0000", 2024-09-17T21:19:53.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:19:53.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889006+0000", 2024-09-17T21:19:53.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1012999999999999e-05 2024-09-17T21:19:53.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889032+0000", 2024-09-17T21:19:53.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6441e-05 2024-09-17T21:19:53.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889040+0000", 2024-09-17T21:19:53.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0639999999999994e-06 2024-09-17T21:19:53.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889278+0000", 2024-09-17T21:19:53.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023785899999999999 2024-09-17T21:19:53.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889408+0000", 2024-09-17T21:19:53.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129219 2024-09-17T21:19:53.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:19:53.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889723+0000", 2024-09-17T21:19:53.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000315264 2024-09-17T21:19:53.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:53.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.891972+0000", 2024-09-17T21:19:53.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002249633 2024-09-17T21:19:53.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:53.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940733+0000", 2024-09-17T21:19:53.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048760626000000001 2024-09-17T21:19:53.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:53.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940785+0000", 2024-09-17T21:19:53.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1906000000000001e-05 2024-09-17T21:19:53.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940807+0000", 2024-09-17T21:19:53.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1554999999999999e-05 2024-09-17T21:19:53.328 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.328 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.328 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.328 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3379 3.12 3:4e83fd71:::rbd_data.1152a8ae2019.000000000000006d:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:53.328 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:19:53.328 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.354480191, 2024-09-17T21:19:53.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051804144000000003, 2024-09-17T21:19:53.329 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.329 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.329 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:53.329 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.329 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:53.329 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:53.329 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3379 2024-09-17T21:19:53.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.329 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:19:53.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:19:53.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889079+0000", 2024-09-17T21:19:53.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1800000000000004e-07 2024-09-17T21:19:53.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889083+0000", 2024-09-17T21:19:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:19:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889094+0000", 2024-09-17T21:19:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1562999999999999e-05 2024-09-17T21:19:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889102+0000", 2024-09-17T21:19:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.401e-06 2024-09-17T21:19:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:19:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014279259999999999 2024-09-17T21:19:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890609+0000", 2024-09-17T21:19:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9566000000000004e-05 2024-09-17T21:19:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:19:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890789+0000", 2024-09-17T21:19:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017949700000000001 2024-09-17T21:19:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.893929+0000", 2024-09-17T21:19:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003140538 2024-09-17T21:19:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940834+0000", 2024-09-17T21:19:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046904811999999997 2024-09-17T21:19:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940860+0000", 2024-09-17T21:19:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5341999999999999e-05 2024-09-17T21:19:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940882+0000", 2024-09-17T21:19:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2878999999999999e-05 2024-09-17T21:19:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:1613 3.1e 3:7d13ef1b:::rbd_data.115266d0b9f2.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 5666816~34816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-17T21:19:53.335 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:48.315514+0000", 2024-09-17T21:19:53.335 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.9280444120000002, 2024-09-17T21:19:53.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045611719000000002, 2024-09-17T21:19:53.335 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.335 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.335 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:53.335 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.335 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:19:53.335 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:19:53.335 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 1613 2024-09-17T21:19:53.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.335 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.315514+0000", 2024-09-17T21:19:53.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.315514+0000", 2024-09-17T21:19:53.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.315516+0000", 2024-09-17T21:19:53.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.497e-06 2024-09-17T21:19:53.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.330451+0000", 2024-09-17T21:19:53.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014935857 2024-09-17T21:19:53.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.330479+0000", 2024-09-17T21:19:53.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7151000000000001e-05 2024-09-17T21:19:53.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.330490+0000", 2024-09-17T21:19:53.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1250000000000001e-05 2024-09-17T21:19:53.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.330560+0000", 2024-09-17T21:19:53.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9901999999999996e-05 2024-09-17T21:19:53.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.330688+0000", 2024-09-17T21:19:53.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000128608 2024-09-17T21:19:53.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:19:53.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.331269+0000", 2024-09-17T21:19:53.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00058052799999999997 2024-09-17T21:19:53.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:53.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.350343+0000", 2024-09-17T21:19:53.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019074457 2024-09-17T21:19:53.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:53.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.361030+0000", 2024-09-17T21:19:53.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010686357 2024-09-17T21:19:53.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:53.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.361084+0000", 2024-09-17T21:19:53.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4097000000000001e-05 2024-09-17T21:19:53.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.361126+0000", 2024-09-17T21:19:53.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2014999999999997e-05 2024-09-17T21:19:53.340 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.340 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.340 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.340 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:2644 3.1e 3:79615585:::rbd_data.115266d0b9f2.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2475520~14848,write 3106304~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:19:53.340 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:19:53.341 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.7318517880000002, 2024-09-17T21:19:53.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050733484000000002, 2024-09-17T21:19:53.341 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.341 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.341 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:53.341 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.341 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:19:53.341 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:19:53.341 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2644 2024-09-17T21:19:53.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.341 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:19:53.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:19:53.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511708+0000", 2024-09-17T21:19:53.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-17T21:19:53.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522196+0000", 2024-09-17T21:19:53.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010487712999999999 2024-09-17T21:19:53.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522223+0000", 2024-09-17T21:19:53.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7064999999999999e-05 2024-09-17T21:19:53.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522236+0000", 2024-09-17T21:19:53.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2931e-05 2024-09-17T21:19:53.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522348+0000", 2024-09-17T21:19:53.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111906 2024-09-17T21:19:53.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522554+0000", 2024-09-17T21:19:53.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020632699999999999 2024-09-17T21:19:53.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:19:53.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.523185+0000", 2024-09-17T21:19:53.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063081400000000005 2024-09-17T21:19:53.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:53.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.547011+0000", 2024-09-17T21:19:53.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023825570000000001 2024-09-17T21:19:53.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:53.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562268+0000", 2024-09-17T21:19:53.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015257514 2024-09-17T21:19:53.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:53.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562367+0000", 2024-09-17T21:19:53.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8678000000000006e-05 2024-09-17T21:19:53.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562440+0000", 2024-09-17T21:19:53.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3337000000000006e-05 2024-09-17T21:19:53.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.346 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.346 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.346 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:53.348 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:19:53.349 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:53.343+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:53.449 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:53.447+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:53.475 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:53.475 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:53.475 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:19:53.475 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:53.488 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:19:53.550 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:53.546+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:53.607 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:53.607 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:53.607 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:53.607 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:53.607 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:53.621 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:19:53.665 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:53.662+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:53.746 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:19:53.753 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:53.747+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:53.796 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:53.796 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:53.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:53.796 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:53.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.796 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:543 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:53.796 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:53.797 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.594011116999994, 2024-09-17T21:19:53.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047564097999999999, 2024-09-17T21:19:53.797 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.797 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.797 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.797 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.797 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:53.797 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:53.797 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 543 2024-09-17T21:19:53.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.797 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:53.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:53.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:19:53.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9699999999999995e-07 2024-09-17T21:19:53.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199677+0000", 2024-09-17T21:19:53.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:19:53.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199690+0000", 2024-09-17T21:19:53.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2977e-05 2024-09-17T21:19:53.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199705+0000", 2024-09-17T21:19:53.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4656e-05 2024-09-17T21:19:53.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246307+0000", 2024-09-17T21:19:53.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046601780000000002 2024-09-17T21:19:53.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246475+0000", 2024-09-17T21:19:53.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016834200000000001 2024-09-17T21:19:53.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247227+0000", 2024-09-17T21:19:53.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00075173299999999998 2024-09-17T21:19:53.801 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.801 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.801 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.801 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:544 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:53.801 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:53.801 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.593952708000003, 2024-09-17T21:19:53.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049283559999999997, 2024-09-17T21:19:53.801 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.801 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.801 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.801 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.801 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:53.802 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:53.802 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 544 2024-09-17T21:19:53.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.802 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:53.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:19:53.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199722+0000", 2024-09-17T21:19:53.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.08e-06 2024-09-17T21:19:53.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199738+0000", 2024-09-17T21:19:53.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5943999999999998e-05 2024-09-17T21:19:53.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199748+0000", 2024-09-17T21:19:53.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0193e-05 2024-09-17T21:19:53.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199756+0000", 2024-09-17T21:19:53.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4200000000000001e-06 2024-09-17T21:19:53.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247311+0000", 2024-09-17T21:19:53.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047555300000000002 2024-09-17T21:19:53.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247571+0000", 2024-09-17T21:19:53.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025996999999999999 2024-09-17T21:19:53.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249005+0000", 2024-09-17T21:19:53.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001433653 2024-09-17T21:19:53.805 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.805 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.805 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.805 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:549 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:53.805 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:53.805 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.593907083999994, 2024-09-17T21:19:53.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048464528999999999, 2024-09-17T21:19:53.806 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.806 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.806 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.806 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.806 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:53.806 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:53.806 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 549 2024-09-17T21:19:53.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.806 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:53.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:53.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:19:53.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:19:53.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199770+0000", 2024-09-17T21:19:53.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9969999999999999e-06 2024-09-17T21:19:53.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199780+0000", 2024-09-17T21:19:53.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0003000000000001e-05 2024-09-17T21:19:53.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199793+0000", 2024-09-17T21:19:53.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2074e-05 2024-09-17T21:19:53.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247289+0000", 2024-09-17T21:19:53.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047496691000000001 2024-09-17T21:19:53.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247437+0000", 2024-09-17T21:19:53.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014807899999999999 2024-09-17T21:19:53.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248231+0000", 2024-09-17T21:19:53.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00079390899999999998 2024-09-17T21:19:53.809 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.809 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.809 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.809 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:558 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:53.809 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:53.810 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.593866571000007, 2024-09-17T21:19:53.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049709821000000001, 2024-09-17T21:19:53.810 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.810 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.810 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.810 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.810 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:53.810 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:53.810 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 558 2024-09-17T21:19:53.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.810 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:53.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:19:53.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199809+0000", 2024-09-17T21:19:53.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.333e-06 2024-09-17T21:19:53.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199814+0000", 2024-09-17T21:19:53.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0930000000000002e-06 2024-09-17T21:19:53.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199831+0000", 2024-09-17T21:19:53.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7580000000000001e-05 2024-09-17T21:19:53.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199860+0000", 2024-09-17T21:19:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8609999999999999e-05 2024-09-17T21:19:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249106+0000", 2024-09-17T21:19:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049246545000000003 2024-09-17T21:19:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249274+0000", 2024-09-17T21:19:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016806699999999999 2024-09-17T21:19:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249517+0000", 2024-09-17T21:19:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024259300000000001 2024-09-17T21:19:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:559 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.593747209, 2024-09-17T21:19:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049990694000000002, 2024-09-17T21:19:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 559 2024-09-17T21:19:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:19:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199928+0000", 2024-09-17T21:19:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:19:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199935+0000", 2024-09-17T21:19:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5429999999999999e-06 2024-09-17T21:19:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199954+0000", 2024-09-17T21:19:53.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.912e-05 2024-09-17T21:19:53.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199969+0000", 2024-09-17T21:19:53.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5305999999999999e-05 2024-09-17T21:19:53.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249604+0000", 2024-09-17T21:19:53.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049634931 2024-09-17T21:19:53.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249738+0000", 2024-09-17T21:19:53.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013411599999999999 2024-09-17T21:19:53.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249917+0000", 2024-09-17T21:19:53.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179111 2024-09-17T21:19:53.818 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.818 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.818 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.818 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:560 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:19:53.818 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:53.819 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.593682792999999, 2024-09-17T21:19:53.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050173059999999998, 2024-09-17T21:19:53.819 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.819 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.819 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.819 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.819 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:19:53.819 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:19:53.819 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 560 2024-09-17T21:19:53.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.819 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:53.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.820 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.820 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:19:53.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.820 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.820 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199992+0000", 2024-09-17T21:19:53.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.517e-06 2024-09-17T21:19:53.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.820 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.821 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199998+0000", 2024-09-17T21:19:53.821 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1460000000000004e-06 2024-09-17T21:19:53.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.821 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.821 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.821 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:19:53.821 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7407000000000001e-05 2024-09-17T21:19:53.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.821 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.821 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.821 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200029+0000", 2024-09-17T21:19:53.821 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3499999999999999e-05 2024-09-17T21:19:53.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.821 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.822 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.822 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249968+0000", 2024-09-17T21:19:53.822 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049939799 2024-09-17T21:19:53.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.822 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.822 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250055+0000", 2024-09-17T21:19:53.822 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6731999999999996e-05 2024-09-17T21:19:53.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.822 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.822 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250164+0000", 2024-09-17T21:19:53.822 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108959 2024-09-17T21:19:53.823 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.823 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.823 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.823 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:19:53.823 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:53.823 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.983893084000002, 2024-09-17T21:19:53.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:19:53.823 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.823 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.823 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.823 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.823 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:19:53.824 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:19:53.824 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:19:53.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.824 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:53.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:19:53.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:19:53.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:19:53.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:19:53.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:19:53.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:19:53.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:19:53.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.826 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.826 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.826 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:19:53.826 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:19:53.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.826 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.826 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.826 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:19:53.826 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:19:53.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.826 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.826 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:53.826 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:19:53.827 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:19:53.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.827 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.827 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:19:53.827 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:19:53.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.827 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.827 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:19:53.827 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:19:53.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.827 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.828 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:19:53.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:19:53.828 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.828 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.828 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.828 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:19:53.828 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:53.828 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.428791298, 2024-09-17T21:19:53.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:19:53.828 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.828 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.829 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:53.829 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.829 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:19:53.829 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:53.829 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:19:53.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.829 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:53.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:19:53.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:19:53.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:19:53.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:19:53.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:19:53.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:19:53.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:19:53.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:19:53.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:19:53.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:19:53.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:19:53.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.832 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:19:53.832 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:19:53.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:19:53.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.832 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.832 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:19:53.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:19:53.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.832 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.832 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:19:53.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:19:53.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:53.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:19:53.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:19:53.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:53.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:19:53.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:19:53.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:53.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:19:53.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:19:53.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:53.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:19:53.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:19:53.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:19:53.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:19:53.834 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.835 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.835 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.835 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3777 3.c 3:302136a0:::rbd_data.1152a8ae2019.000000000000004b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:53.835 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.902125+0000", 2024-09-17T21:19:53.835 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.891548672000001, 2024-09-17T21:19:53.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048610548000000003, 2024-09-17T21:19:53.835 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.835 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.835 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:53.835 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.835 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:53.835 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:53.836 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3777 2024-09-17T21:19:53.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.836 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902125+0000", 2024-09-17T21:19:53.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902125+0000", 2024-09-17T21:19:53.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902126+0000", 2024-09-17T21:19:53.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:19:53.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902138+0000", 2024-09-17T21:19:53.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1994000000000001e-05 2024-09-17T21:19:53.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902144+0000", 2024-09-17T21:19:53.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2820000000000002e-06 2024-09-17T21:19:53.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902149+0000", 2024-09-17T21:19:53.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.135e-06 2024-09-17T21:19:53.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.942910+0000", 2024-09-17T21:19:53.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.040761190000000003 2024-09-17T21:19:53.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.942986+0000", 2024-09-17T21:19:53.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6083999999999999e-05 2024-09-17T21:19:53.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:53.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.943319+0000", 2024-09-17T21:19:53.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00033264500000000001 2024-09-17T21:19:53.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:53.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944690+0000", 2024-09-17T21:19:53.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0013710549999999999 2024-09-17T21:19:53.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:53.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.950637+0000", 2024-09-17T21:19:53.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059473190000000004 2024-09-17T21:19:53.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:53.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.950696+0000", 2024-09-17T21:19:53.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8449999999999998e-05 2024-09-17T21:19:53.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.950736+0000", 2024-09-17T21:19:53.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9971000000000001e-05 2024-09-17T21:19:53.840 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.841 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.841 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.841 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3778 3.c 3:375c1b23:::rbd_data.1152a8ae2019.0000000000000052:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:53.841 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.902156+0000", 2024-09-17T21:19:53.841 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.891517318, 2024-09-17T21:19:53.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049052074000000001, 2024-09-17T21:19:53.841 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.841 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.841 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:53.841 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.841 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:53.841 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:53.842 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3778 2024-09-17T21:19:53.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.842 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.842 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.842 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902156+0000", 2024-09-17T21:19:53.842 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.842 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.842 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902156+0000", 2024-09-17T21:19:53.842 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.843 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902157+0000", 2024-09-17T21:19:53.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5299999999999999e-07 2024-09-17T21:19:53.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.843 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902160+0000", 2024-09-17T21:19:53.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6359999999999998e-06 2024-09-17T21:19:53.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.843 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902165+0000", 2024-09-17T21:19:53.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9440000000000002e-06 2024-09-17T21:19:53.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902170+0000", 2024-09-17T21:19:53.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:19:53.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.943704+0000", 2024-09-17T21:19:53.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041533516999999999 2024-09-17T21:19:53.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.943760+0000", 2024-09-17T21:19:53.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6552000000000003e-05 2024-09-17T21:19:53.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:53.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.943879+0000", 2024-09-17T21:19:53.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118347 2024-09-17T21:19:53.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:53.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944733+0000", 2024-09-17T21:19:53.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00085437299999999996 2024-09-17T21:19:53.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:53.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951158+0000", 2024-09-17T21:19:53.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064243520000000004 2024-09-17T21:19:53.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:53.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951184+0000", 2024-09-17T21:19:53.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6913000000000001e-05 2024-09-17T21:19:53.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951209+0000", 2024-09-17T21:19:53.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4104000000000002e-05 2024-09-17T21:19:53.846 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.846 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.847 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.847 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.847 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3785 3.c 3:3414a45b:::rbd_data.1152a8ae2019.0000000000000074:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:53.847 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.902292+0000", 2024-09-17T21:19:53.847 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.891381795000001, 2024-09-17T21:19:53.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049058004000000002, 2024-09-17T21:19:53.847 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.847 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.847 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:53.847 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.847 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:53.847 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:53.847 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3785 2024-09-17T21:19:53.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.848 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.848 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.848 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.848 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902292+0000", 2024-09-17T21:19:53.848 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.848 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.848 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.848 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902292+0000", 2024-09-17T21:19:53.848 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.848 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.848 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.849 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902292+0000", 2024-09-17T21:19:53.849 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:19:53.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.849 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.849 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902299+0000", 2024-09-17T21:19:53.849 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1210000000000001e-06 2024-09-17T21:19:53.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.849 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.849 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902305+0000", 2024-09-17T21:19:53.849 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6069999999999998e-06 2024-09-17T21:19:53.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902310+0000", 2024-09-17T21:19:53.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7509999999999999e-06 2024-09-17T21:19:53.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944093+0000", 2024-09-17T21:19:53.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041783582999999999 2024-09-17T21:19:53.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944159+0000", 2024-09-17T21:19:53.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5524999999999994e-05 2024-09-17T21:19:53.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:53.851 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944281+0000", 2024-09-17T21:19:53.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121573 2024-09-17T21:19:53.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:19:53.851 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.945107+0000", 2024-09-17T21:19:53.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00082616700000000005 2024-09-17T21:19:53.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:53.851 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951260+0000", 2024-09-17T21:19:53.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061530789999999997 2024-09-17T21:19:53.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.852 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:53.852 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951311+0000", 2024-09-17T21:19:53.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1678000000000002e-05 2024-09-17T21:19:53.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.852 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.852 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951350+0000", 2024-09-17T21:19:53.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8525000000000001e-05 2024-09-17T21:19:53.852 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.852 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.852 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.853 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.853 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3880 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:53.853 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.912848+0000", 2024-09-17T21:19:53.853 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.880825682999999, 2024-09-17T21:19:53.853 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046791626000000003, 2024-09-17T21:19:53.853 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.853 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.853 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.853 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.853 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:53.853 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:53.853 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3880 2024-09-17T21:19:53.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.854 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.854 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.854 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.854 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912848+0000", 2024-09-17T21:19:53.854 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.854 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.854 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.854 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912848+0000", 2024-09-17T21:19:53.854 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.854 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.854 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912849+0000", 2024-09-17T21:19:53.855 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2600000000000002e-07 2024-09-17T21:19:53.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.855 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912851+0000", 2024-09-17T21:19:53.855 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.475e-06 2024-09-17T21:19:53.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.855 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912857+0000", 2024-09-17T21:19:53.855 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1009999999999997e-06 2024-09-17T21:19:53.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.856 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.856 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912862+0000", 2024-09-17T21:19:53.856 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2639999999999999e-06 2024-09-17T21:19:53.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.856 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.856 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959555+0000", 2024-09-17T21:19:53.856 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046692589 2024-09-17T21:19:53.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.856 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.856 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959586+0000", 2024-09-17T21:19:53.856 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0978000000000001e-05 2024-09-17T21:19:53.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.857 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.857 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.857 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959640+0000", 2024-09-17T21:19:53.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3693000000000002e-05 2024-09-17T21:19:53.857 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.857 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.857 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.857 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.857 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3881 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:53.857 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.912869+0000", 2024-09-17T21:19:53.857 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.88080456, 2024-09-17T21:19:53.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046894383999999997, 2024-09-17T21:19:53.857 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.858 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.858 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.858 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.858 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:53.858 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:53.858 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3881 2024-09-17T21:19:53.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.858 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.858 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.858 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.858 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912869+0000", 2024-09-17T21:19:53.858 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.858 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.858 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.859 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912869+0000", 2024-09-17T21:19:53.859 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.859 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.859 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.859 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912870+0000", 2024-09-17T21:19:53.859 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:19:53.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.859 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.859 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.859 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913014+0000", 2024-09-17T21:19:53.859 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000144473 2024-09-17T21:19:53.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.859 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913025+0000", 2024-09-17T21:19:53.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1341e-05 2024-09-17T21:19:53.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913033+0000", 2024-09-17T21:19:53.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.17e-06 2024-09-17T21:19:53.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959677+0000", 2024-09-17T21:19:53.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046644536 2024-09-17T21:19:53.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959709+0000", 2024-09-17T21:19:53.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2348000000000003e-05 2024-09-17T21:19:53.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959764+0000", 2024-09-17T21:19:53.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4120999999999999e-05 2024-09-17T21:19:53.861 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.861 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.861 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.862 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:14003 3.4 3.4f4be604 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:53.862 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.912905+0000", 2024-09-17T21:19:53.862 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.880768720000001, 2024-09-17T21:19:53.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046758252, 2024-09-17T21:19:53.862 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.862 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.862 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:19:53.862 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.862 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:19:53.862 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:53.862 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 14003 2024-09-17T21:19:53.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.862 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.863 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.863 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912905+0000", 2024-09-17T21:19:53.863 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.863 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.863 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912905+0000", 2024-09-17T21:19:53.863 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.863 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.863 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912906+0000", 2024-09-17T21:19:53.863 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.091e-06 2024-09-17T21:19:53.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912911+0000", 2024-09-17T21:19:53.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9409999999999998e-06 2024-09-17T21:19:53.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912920+0000", 2024-09-17T21:19:53.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3530000000000001e-06 2024-09-17T21:19:53.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912927+0000", 2024-09-17T21:19:53.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9979999999999997e-06 2024-09-17T21:19:53.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959659+0000", 2024-09-17T21:19:53.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046731111999999998 2024-09-17T21:19:53.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959663+0000", 2024-09-17T21:19:53.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7570000000000004e-06 2024-09-17T21:19:53.865 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.865 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.865 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.866 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3882 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:53.866 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913045+0000", 2024-09-17T21:19:53.866 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.880629204, 2024-09-17T21:19:53.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046829914, 2024-09-17T21:19:53.866 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.866 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.866 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.866 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.866 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:53.866 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:53.866 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3882 2024-09-17T21:19:53.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.867 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.867 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.867 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913045+0000", 2024-09-17T21:19:53.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.867 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.867 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913045+0000", 2024-09-17T21:19:53.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.867 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.867 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913045+0000", 2024-09-17T21:19:53.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1200000000000002e-07 2024-09-17T21:19:53.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913048+0000", 2024-09-17T21:19:53.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.179e-06 2024-09-17T21:19:53.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913055+0000", 2024-09-17T21:19:53.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0269999999999998e-06 2024-09-17T21:19:53.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913061+0000", 2024-09-17T21:19:53.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5709999999999997e-06 2024-09-17T21:19:53.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959782+0000", 2024-09-17T21:19:53.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046720708 2024-09-17T21:19:53.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959813+0000", 2024-09-17T21:19:53.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1358e-05 2024-09-17T21:19:53.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.870 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959874+0000", 2024-09-17T21:19:53.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1359000000000007e-05 2024-09-17T21:19:53.870 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.870 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.870 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.870 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3883 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:53.870 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913069+0000", 2024-09-17T21:19:53.870 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.880604836, 2024-09-17T21:19:53.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046905401999999999, 2024-09-17T21:19:53.870 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.871 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.871 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.871 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.871 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:53.871 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:53.871 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3883 2024-09-17T21:19:53.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.871 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.871 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.871 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913069+0000", 2024-09-17T21:19:53.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913069+0000", 2024-09-17T21:19:53.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913069+0000", 2024-09-17T21:19:53.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:19:53.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913072+0000", 2024-09-17T21:19:53.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6390000000000001e-06 2024-09-17T21:19:53.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.873 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913079+0000", 2024-09-17T21:19:53.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2640000000000002e-06 2024-09-17T21:19:53.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.873 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913085+0000", 2024-09-17T21:19:53.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4249999999999997e-06 2024-09-17T21:19:53.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.873 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959893+0000", 2024-09-17T21:19:53.874 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046808728000000001 2024-09-17T21:19:53.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.874 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.874 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.874 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959926+0000", 2024-09-17T21:19:53.874 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2437999999999997e-05 2024-09-17T21:19:53.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.874 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.874 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.874 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959974+0000", 2024-09-17T21:19:53.874 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8516000000000001e-05 2024-09-17T21:19:53.874 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.874 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.874 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.875 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.875 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3884 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:53.875 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913092+0000", 2024-09-17T21:19:53.875 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.880582193, 2024-09-17T21:19:53.875 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047065517000000001, 2024-09-17T21:19:53.875 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.875 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.875 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.875 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.875 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:53.875 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:53.875 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3884 2024-09-17T21:19:53.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.876 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.876 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.876 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913092+0000", 2024-09-17T21:19:53.876 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.876 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.876 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913092+0000", 2024-09-17T21:19:53.876 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.876 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.876 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913670+0000", 2024-09-17T21:19:53.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00057830399999999997 2024-09-17T21:19:53.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913680+0000", 2024-09-17T21:19:53.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0426e-05 2024-09-17T21:19:53.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913691+0000", 2024-09-17T21:19:53.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0309000000000001e-05 2024-09-17T21:19:53.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.878 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.878 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913698+0000", 2024-09-17T21:19:53.878 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1760000000000004e-06 2024-09-17T21:19:53.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.878 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.878 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.960056+0000", 2024-09-17T21:19:53.878 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046358411000000002 2024-09-17T21:19:53.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.878 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.878 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.960089+0000", 2024-09-17T21:19:53.878 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2588e-05 2024-09-17T21:19:53.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.879 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.879 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.960157+0000", 2024-09-17T21:19:53.879 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8303e-05 2024-09-17T21:19:53.879 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.879 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.879 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.879 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:14004 3.4 3.4f4be604 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:53.879 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913136+0000", 2024-09-17T21:19:53.879 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.880537957, 2024-09-17T21:19:53.879 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046862804000000001, 2024-09-17T21:19:53.880 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.880 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.880 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:19:53.880 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.880 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:19:53.880 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:53.880 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 14004 2024-09-17T21:19:53.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.880 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.880 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.880 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913136+0000", 2024-09-17T21:19:53.880 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913136+0000", 2024-09-17T21:19:53.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913447+0000", 2024-09-17T21:19:53.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031107200000000002 2024-09-17T21:19:53.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.882 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913460+0000", 2024-09-17T21:19:53.882 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2996999999999999e-05 2024-09-17T21:19:53.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.882 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.882 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913470+0000", 2024-09-17T21:19:53.882 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0037999999999999e-05 2024-09-17T21:19:53.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.882 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.883 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913477+0000", 2024-09-17T21:19:53.883 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:19:53.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.883 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.883 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959994+0000", 2024-09-17T21:19:53.883 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046517599999999999 2024-09-17T21:19:53.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.883 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.883 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959999+0000", 2024-09-17T21:19:53.883 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1400000000000002e-06 2024-09-17T21:19:53.883 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.883 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.883 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.884 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3885 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:53.884 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913710+0000", 2024-09-17T21:19:53.884 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.87996351, 2024-09-17T21:19:53.884 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047449947999999999, 2024-09-17T21:19:53.884 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.884 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.885 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.885 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.885 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:19:53.885 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:53.885 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3885 2024-09-17T21:19:53.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.885 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.885 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.885 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913710+0000", 2024-09-17T21:19:53.885 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.885 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.886 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913710+0000", 2024-09-17T21:19:53.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.886 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.886 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.917352+0000", 2024-09-17T21:19:53.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0036412950000000001 2024-09-17T21:19:53.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.886 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.886 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.917357+0000", 2024-09-17T21:19:53.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4879999999999998e-06 2024-09-17T21:19:53.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.887 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.887 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.918061+0000", 2024-09-17T21:19:53.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00070362899999999999 2024-09-17T21:19:53.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.887 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.887 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.918069+0000", 2024-09-17T21:19:53.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4530000000000002e-06 2024-09-17T21:19:53.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.887 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.887 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.960956+0000", 2024-09-17T21:19:53.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042886951999999999 2024-09-17T21:19:53.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.961036+0000", 2024-09-17T21:19:53.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0010000000000001e-05 2024-09-17T21:19:53.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.961160+0000", 2024-09-17T21:19:53.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012412100000000001 2024-09-17T21:19:53.888 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.888 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.888 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.889 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:13984 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:53.889 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.921124+0000", 2024-09-17T21:19:53.889 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.872549533000001, 2024-09-17T21:19:53.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047129407999999998, 2024-09-17T21:19:53.889 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:53.889 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:53.889 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:19:53.889 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:19:53.889 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:19:53.889 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:53.889 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 13984 2024-09-17T21:19:53.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.889 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:53.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:53.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.921124+0000", 2024-09-17T21:19:53.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:53.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.921124+0000", 2024-09-17T21:19:53.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:53.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:53.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.928309+0000", 2024-09-17T21:19:53.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0071847619999999999 2024-09-17T21:19:53.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.928316+0000", 2024-09-17T21:19:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2969999999999998e-06 2024-09-17T21:19:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.928330+0000", 2024-09-17T21:19:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3849000000000001e-05 2024-09-17T21:19:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.928338+0000", 2024-09-17T21:19:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.354e-06 2024-09-17T21:19:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.968167+0000", 2024-09-17T21:19:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.039828298999999998 2024-09-17T21:19:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.968201+0000", 2024-09-17T21:19:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3933000000000002e-05 2024-09-17T21:19:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.968254+0000", 2024-09-17T21:19:53.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2914000000000002e-05 2024-09-17T21:19:53.893 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.893 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.893 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.893 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:53.893 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:53.893 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:53.894 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:19:53.895 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:53.887+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:53.962 INFO:teuthology.orchestra.run.smithi184.stdout:flattening image image_client.0-clone25 2024-09-17T21:19:53.995 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:53.991+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:54.084 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:54.084 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:54.084 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:19:54.084 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:54.096 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:54.091+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:54.103 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:19:54.197 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:54.194+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:54.240 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:54.240 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:54.240 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:54.240 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:54.240 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:54.255 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:19:54.301 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:54.298+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:54.399 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:54.394+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:54.440 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:54.441 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:54.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:54.441 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:54.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.441 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:19:54.441 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:54.441 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 96.221381996999995, 2024-09-17T21:19:54.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:19:54.441 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:54.441 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:54.441 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:54.441 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:54.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:54.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:54.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:54.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:54.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:54.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:19:54.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:19:54.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.443 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:54.443 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:19:54.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:19:54.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.443 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:54.443 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:19:54.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:19:54.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.443 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:54.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:19:54.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:19:54.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:54.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:19:54.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:19:54.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:54.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:19:54.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:19:54.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:54.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:19:54.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:19:54.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:54.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:19:54.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:19:54.445 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.445 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:54.446 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.446 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:19:54.446 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:54.446 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.571081332999995, 2024-09-17T21:19:54.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:19:54.446 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:54.446 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:54.446 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:54.446 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:54.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:54.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:54.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:54.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:54.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:54.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:19:54.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:19:54.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:54.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:19:54.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:19:54.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:54.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:19:54.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:19:54.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:54.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:19:54.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:19:54.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:54.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:19:54.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:19:54.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:54.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:19:54.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:19:54.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:54.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:19:54.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:19:54.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:54.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:19:54.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:19:54.450 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.450 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:54.450 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.450 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:19:54.450 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:54.450 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.569816250000002, 2024-09-17T21:19:54.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:19:54.451 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:54.451 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:54.451 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:54.451 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:54.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:54.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:54.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:54.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:54.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:54.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:19:54.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:19:54.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:54.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:19:54.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:19:54.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:54.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:19:54.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:19:54.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:54.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:19:54.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:19:54.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:54.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:19:54.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:19:54.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.454 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:54.454 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:19:54.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:19:54.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.454 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:54.454 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:19:54.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:19:54.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.454 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:54.454 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:19:54.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:19:54.454 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.455 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:54.455 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.455 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:19:54.455 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:54.455 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.560036839999995, 2024-09-17T21:19:54.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:19:54.455 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:54.455 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:54.455 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:54.455 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:54.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:54.456 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:54.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:54.456 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:54.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:54.456 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:19:54.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:19:54.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:54.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:19:54.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:19:54.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:54.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:19:54.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:19:54.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:54.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:19:54.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:19:54.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:54.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:19:54.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:19:54.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:54.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:19:54.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:19:54.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:54.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:19:54.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:19:54.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:54.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:19:54.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:19:54.459 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.459 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:54.459 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.459 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:19:54.459 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:54.460 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.551314437000002, 2024-09-17T21:19:54.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:19:54.460 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:54.460 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:54.460 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:54.460 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:54.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.460 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:54.460 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:54.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.460 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:54.460 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:54.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.461 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:54.461 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:19:54.461 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:19:54.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.461 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:54.461 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:19:54.461 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:19:54.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.461 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:54.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:19:54.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:19:54.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:54.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:19:54.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:19:54.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:54.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:19:54.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:19:54.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:54.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:19:54.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:19:54.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:54.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:19:54.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:19:54.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:54.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:19:54.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:19:54.464 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.464 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:54.464 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.464 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:19:54.464 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:54.464 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 78.166223873000007, 2024-09-17T21:19:54.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:19:54.464 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:54.464 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:54.464 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:54.464 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:54.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:54.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:54.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:54.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:54.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:54.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:19:54.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:19:54.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.466 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:54.466 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:19:54.466 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:19:54.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.466 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:54.466 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:19:54.466 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:19:54.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.466 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:54.466 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:19:54.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:19:54.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.467 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:54.467 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:19:54.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:19:54.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.467 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:54.467 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:19:54.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:19:54.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.467 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:54.467 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:19:54.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:19:54.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.468 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:54.468 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:19:54.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:19:54.468 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.468 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:54.468 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.468 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1780 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'195, mlcod=69'190)", 2024-09-17T21:19:54.468 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:19:54.468 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.188368269000001, 2024-09-17T21:19:54.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075761769999999999, 2024-09-17T21:19:54.469 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:54.469 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:54.469 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:54.469 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:54.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:54.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:19:54.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:54.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:19:54.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:54.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250375+0000", 2024-09-17T21:19:54.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:19:54.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:54.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250383+0000", 2024-09-17T21:19:54.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7870000000000002e-06 2024-09-17T21:19:54.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:54.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250402+0000", 2024-09-17T21:19:54.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9346000000000001e-05 2024-09-17T21:19:54.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:54.471 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250413+0000", 2024-09-17T21:19:54.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.039e-05 2024-09-17T21:19:54.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:54.471 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251353+0000", 2024-09-17T21:19:54.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000940091 2024-09-17T21:19:54.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:54.472 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251407+0000", 2024-09-17T21:19:54.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4701999999999999e-05 2024-09-17T21:19:54.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.472 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:54.472 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257829+0000", 2024-09-17T21:19:54.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064212779999999999 2024-09-17T21:19:54.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.472 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:54.472 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257950+0000", 2024-09-17T21:19:54.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121623 2024-09-17T21:19:54.472 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.473 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:54.473 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.473 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1781 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'196, mlcod=69'190)", 2024-09-17T21:19:54.473 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:19:54.473 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.187285318000001, 2024-09-17T21:19:54.473 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065632939999999999, 2024-09-17T21:19:54.473 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:54.473 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:54.473 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:54.473 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:54.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:54.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:19:54.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:54.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:19:54.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:54.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251459+0000", 2024-09-17T21:19:54.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.624e-06 2024-09-17T21:19:54.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:54.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251471+0000", 2024-09-17T21:19:54.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2636000000000001e-05 2024-09-17T21:19:54.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:54.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251498+0000", 2024-09-17T21:19:54.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:19:54.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:54.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251515+0000", 2024-09-17T21:19:54.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6939000000000002e-05 2024-09-17T21:19:54.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.476 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:54.476 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252938+0000", 2024-09-17T21:19:54.476 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001423082 2024-09-17T21:19:54.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.476 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:54.476 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252999+0000", 2024-09-17T21:19:54.476 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1138999999999999e-05 2024-09-17T21:19:54.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.476 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:54.477 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257955+0000", 2024-09-17T21:19:54.477 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049559640000000002 2024-09-17T21:19:54.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.477 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:54.477 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.258020+0000", 2024-09-17T21:19:54.477 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5504000000000002e-05 2024-09-17T21:19:54.477 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.477 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:54.477 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.477 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1788 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'198, mlcod=69'191)", 2024-09-17T21:19:54.477 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:19:54.478 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.185259829000003, 2024-09-17T21:19:54.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073690370000000002, 2024-09-17T21:19:54.478 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:54.478 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:54.478 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:54.478 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:54.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:54.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:19:54.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:54.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:19:54.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:54.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253484+0000", 2024-09-17T21:19:54.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2470000000000001e-06 2024-09-17T21:19:54.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:54.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253492+0000", 2024-09-17T21:19:54.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7159999999999992e-06 2024-09-17T21:19:54.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:54.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253509+0000", 2024-09-17T21:19:54.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.666e-05 2024-09-17T21:19:54.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:54.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253522+0000", 2024-09-17T21:19:54.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2643000000000001e-05 2024-09-17T21:19:54.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:54.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254853+0000", 2024-09-17T21:19:54.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331238 2024-09-17T21:19:54.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.481 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:54.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254885+0000", 2024-09-17T21:19:54.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1562e-05 2024-09-17T21:19:54.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.481 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:54.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260788+0000", 2024-09-17T21:19:54.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059030749999999998 2024-09-17T21:19:54.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.481 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:54.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260852+0000", 2024-09-17T21:19:54.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3895999999999999e-05 2024-09-17T21:19:54.482 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.482 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:54.482 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.482 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5746 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'477, mlcod=71'473)", 2024-09-17T21:19:54.482 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:19:54.482 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 38.179832271000002, 2024-09-17T21:19:54.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074182709999999997, 2024-09-17T21:19:54.482 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:54.482 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:54.482 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:54.482 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:54.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:54.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:19:54.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:54.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:19:54.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:54.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258912+0000", 2024-09-17T21:19:54.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:19:54.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:54.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258926+0000", 2024-09-17T21:19:54.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4249e-05 2024-09-17T21:19:54.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:54.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258946+0000", 2024-09-17T21:19:54.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9879999999999999e-05 2024-09-17T21:19:54.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:54.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258956+0000", 2024-09-17T21:19:54.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:19:54.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:54.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259380+0000", 2024-09-17T21:19:54.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042337599999999999 2024-09-17T21:19:54.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:54.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259412+0000", 2024-09-17T21:19:54.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:19:54.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:54.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266191+0000", 2024-09-17T21:19:54.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067793269999999999 2024-09-17T21:19:54.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:54.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266328+0000", 2024-09-17T21:19:54.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013690999999999999 2024-09-17T21:19:54.487 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.487 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:54.487 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.487 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5747 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'478, mlcod=71'473)", 2024-09-17T21:19:54.488 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:19:54.488 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 38.179125921999997, 2024-09-17T21:19:54.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068779440000000004, 2024-09-17T21:19:54.488 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:54.488 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:54.488 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:54.488 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:54.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:54.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:19:54.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:54.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:19:54.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:54.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259618+0000", 2024-09-17T21:19:54.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6199999999999999e-06 2024-09-17T21:19:54.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:54.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259630+0000", 2024-09-17T21:19:54.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1884e-05 2024-09-17T21:19:54.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:54.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259657+0000", 2024-09-17T21:19:54.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7339e-05 2024-09-17T21:19:54.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:54.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259677+0000", 2024-09-17T21:19:54.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0156000000000001e-05 2024-09-17T21:19:54.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:54.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260521+0000", 2024-09-17T21:19:54.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084312899999999997 2024-09-17T21:19:54.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:54.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260585+0000", 2024-09-17T21:19:54.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4558000000000001e-05 2024-09-17T21:19:54.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:54.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266336+0000", 2024-09-17T21:19:54.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057505020000000002 2024-09-17T21:19:54.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:54.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266494+0000", 2024-09-17T21:19:54.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158756 2024-09-17T21:19:54.492 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.492 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:54.492 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.492 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5750 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'479, mlcod=71'473)", 2024-09-17T21:19:54.492 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:19:54.492 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 38.178413728999999, 2024-09-17T21:19:54.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006302874, 2024-09-17T21:19:54.492 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:54.492 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:54.492 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:54.492 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:54.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:54.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:19:54.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:54.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:19:54.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:54.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260330+0000", 2024-09-17T21:19:54.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:19:54.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:54.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260343+0000", 2024-09-17T21:19:54.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2862e-05 2024-09-17T21:19:54.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:54.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260367+0000", 2024-09-17T21:19:54.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3723e-05 2024-09-17T21:19:54.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:54.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260386+0000", 2024-09-17T21:19:54.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8879999999999999e-05 2024-09-17T21:19:54.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.495 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:54.495 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261579+0000", 2024-09-17T21:19:54.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001193431 2024-09-17T21:19:54.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.495 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:54.495 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261608+0000", 2024-09-17T21:19:54.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8858999999999999e-05 2024-09-17T21:19:54.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.495 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:54.495 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266505+0000", 2024-09-17T21:19:54.496 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004896999 2024-09-17T21:19:54.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.496 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:54.496 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266631+0000", 2024-09-17T21:19:54.496 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012655300000000001 2024-09-17T21:19:54.496 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.496 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:54.496 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.496 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5751 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'480, mlcod=71'473)", 2024-09-17T21:19:54.496 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:19:54.496 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 38.177706516999997, 2024-09-17T21:19:54.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075756299999999999, 2024-09-17T21:19:54.497 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:54.497 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:54.497 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:54.497 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:54.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.497 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:54.497 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:19:54.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.497 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:54.497 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:19:54.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:54.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261037+0000", 2024-09-17T21:19:54.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:19:54.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:54.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261049+0000", 2024-09-17T21:19:54.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1063999999999999e-05 2024-09-17T21:19:54.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:54.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261072+0000", 2024-09-17T21:19:54.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3102e-05 2024-09-17T21:19:54.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:54.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261133+0000", 2024-09-17T21:19:54.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1276000000000001e-05 2024-09-17T21:19:54.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:54.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:19:54.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015334319999999999 2024-09-17T21:19:54.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:54.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262702+0000", 2024-09-17T21:19:54.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5281999999999999e-05 2024-09-17T21:19:54.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:54.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268527+0000", 2024-09-17T21:19:54.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058250389999999997 2024-09-17T21:19:54.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:54.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268611+0000", 2024-09-17T21:19:54.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4724999999999999e-05 2024-09-17T21:19:54.501 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.501 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:54.501 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.501 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5756 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'481, mlcod=71'473)", 2024-09-17T21:19:54.501 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:19:54.501 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 38.176845661000002, 2024-09-17T21:19:54.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006772386, 2024-09-17T21:19:54.501 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:54.501 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:54.501 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:54.501 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:54.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:54.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:19:54.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:54.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:19:54.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:54.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261899+0000", 2024-09-17T21:19:54.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.903e-06 2024-09-17T21:19:54.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:54.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261912+0000", 2024-09-17T21:19:54.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-05 2024-09-17T21:19:54.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:54.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261938+0000", 2024-09-17T21:19:54.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6098999999999999e-05 2024-09-17T21:19:54.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:54.504 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261955+0000", 2024-09-17T21:19:54.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.696e-05 2024-09-17T21:19:54.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:54.504 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263845+0000", 2024-09-17T21:19:54.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001890187 2024-09-17T21:19:54.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:54.504 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263907+0000", 2024-09-17T21:19:54.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1600999999999995e-05 2024-09-17T21:19:54.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.505 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:54.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268615+0000", 2024-09-17T21:19:54.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047086070000000001 2024-09-17T21:19:54.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.505 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:54.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268669+0000", 2024-09-17T21:19:54.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3588999999999998e-05 2024-09-17T21:19:54.505 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.505 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:54.505 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.506 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4978.0:598 3.9 e80/23 3:9701cc17:::rbd_data.1152724ff2a3.0000000000000019:head v 80'202, mlcod=80'198)", 2024-09-17T21:19:54.506 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:19:54.506 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.675271278, 2024-09-17T21:19:54.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066473340000000004, 2024-09-17T21:19:54.506 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:54.506 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:54.506 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:54.506 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:54.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:54.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:19:54.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.507 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:54.507 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:19:54.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.507 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:54.507 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763472+0000", 2024-09-17T21:19:54.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6899999999999997e-07 2024-09-17T21:19:54.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.507 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:54.507 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763478+0000", 2024-09-17T21:19:54.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:19:54.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.508 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:54.508 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763489+0000", 2024-09-17T21:19:54.508 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0628999999999999e-05 2024-09-17T21:19:54.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.508 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:54.508 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763495+0000", 2024-09-17T21:19:54.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6649999999999997e-06 2024-09-17T21:19:54.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:54.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763907+0000", 2024-09-17T21:19:54.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041163899999999999 2024-09-17T21:19:54.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:54.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763976+0000", 2024-09-17T21:19:54.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8534999999999993e-05 2024-09-17T21:19:54.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:54.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.769900+0000", 2024-09-17T21:19:54.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059243270000000001 2024-09-17T21:19:54.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:54.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.770118+0000", 2024-09-17T21:19:54.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021842500000000001 2024-09-17T21:19:54.510 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.510 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:54.510 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.510 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:468 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'94, mlcod=89'87)", 2024-09-17T21:19:54.510 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:19:54.510 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.752525979, 2024-09-17T21:19:54.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066528790000000004, 2024-09-17T21:19:54.511 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:54.511 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:54.511 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:54.511 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:54.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:54.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:19:54.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:54.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:19:54.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.512 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:54.512 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686218+0000", 2024-09-17T21:19:54.512 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3060000000000001e-06 2024-09-17T21:19:54.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.512 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:54.512 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686256+0000", 2024-09-17T21:19:54.512 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8333e-05 2024-09-17T21:19:54.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.512 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:54.513 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686272+0000", 2024-09-17T21:19:54.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:19:54.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.513 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:54.513 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686283+0000", 2024-09-17T21:19:54.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0903e-05 2024-09-17T21:19:54.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.513 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:54.513 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686682+0000", 2024-09-17T21:19:54.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039935099999999999 2024-09-17T21:19:54.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.514 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:54.514 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686785+0000", 2024-09-17T21:19:54.514 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010275199999999999 2024-09-17T21:19:54.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.514 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:54.514 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692816+0000", 2024-09-17T21:19:54.514 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060317579999999999 2024-09-17T21:19:54.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.514 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:54.514 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692869+0000", 2024-09-17T21:19:54.514 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2775999999999998e-05 2024-09-17T21:19:54.515 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.515 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:54.515 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.515 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:755 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'127, mlcod=89'122)", 2024-09-17T21:19:54.515 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:19:54.515 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.616070765, 2024-09-17T21:19:54.515 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084014080000000008, 2024-09-17T21:19:54.515 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:54.515 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:54.515 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:54.515 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:54.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.515 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:54.516 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:19:54.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.516 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:54.516 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:19:54.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.516 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:54.516 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:19:54.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:19:54.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:54.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822683+0000", 2024-09-17T21:19:54.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0674e-05 2024-09-17T21:19:54.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:54.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822698+0000", 2024-09-17T21:19:54.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5189e-05 2024-09-17T21:19:54.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:54.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822708+0000", 2024-09-17T21:19:54.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9119999999999993e-06 2024-09-17T21:19:54.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.518 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:54.518 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822748+0000", 2024-09-17T21:19:54.518 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9727000000000001e-05 2024-09-17T21:19:54.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.518 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:54.518 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822759+0000", 2024-09-17T21:19:54.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0865e-05 2024-09-17T21:19:54.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.519 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:54.519 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831034+0000", 2024-09-17T21:19:54.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082750710000000002 2024-09-17T21:19:54.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.520 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:54.520 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831073+0000", 2024-09-17T21:19:54.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9063999999999997e-05 2024-09-17T21:19:54.520 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.520 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:54.520 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.520 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:756 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'128, mlcod=89'124)", 2024-09-17T21:19:54.520 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:19:54.520 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.614148133, 2024-09-17T21:19:54.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0076300270000000002, 2024-09-17T21:19:54.521 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:54.521 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:54.521 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:54.521 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:54.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:54.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:19:54.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:54.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:19:54.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.522 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.522 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:54.522 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824595+0000", 2024-09-17T21:19:54.522 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-17T21:19:54.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.523 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.523 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:54.524 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824600+0000", 2024-09-17T21:19:54.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:19:54.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:54.524 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824612+0000", 2024-09-17T21:19:54.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.181e-05 2024-09-17T21:19:54.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:54.524 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824619+0000", 2024-09-17T21:19:54.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:19:54.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.525 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.525 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:54.525 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824671+0000", 2024-09-17T21:19:54.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2262999999999998e-05 2024-09-17T21:19:54.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.525 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.525 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:54.525 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824695+0000", 2024-09-17T21:19:54.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3883000000000002e-05 2024-09-17T21:19:54.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.525 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.525 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:54.525 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832185+0000", 2024-09-17T21:19:54.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007490048 2024-09-17T21:19:54.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:54.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832224+0000", 2024-09-17T21:19:54.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8957e-05 2024-09-17T21:19:54.526 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.526 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:54.526 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.526 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2617 3.d e90/23 3:b0143718:::rbd_data.1152a8ae2019.0000000000000016:head v 90'249, mlcod=90'246)", 2024-09-17T21:19:54.526 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:19:54.527 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 13.594297133, 2024-09-17T21:19:54.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063715559999999996, 2024-09-17T21:19:54.527 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:54.527 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:54.527 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:54.527 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:54.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:54.527 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:19:54.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:54.527 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:19:54.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.528 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:54.528 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844455+0000", 2024-09-17T21:19:54.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4259999999999992e-06 2024-09-17T21:19:54.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.528 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:54.529 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844470+0000", 2024-09-17T21:19:54.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5386000000000001e-05 2024-09-17T21:19:54.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.529 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.529 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:54.529 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844499+0000", 2024-09-17T21:19:54.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.883e-05 2024-09-17T21:19:54.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.529 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.529 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:54.529 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844518+0000", 2024-09-17T21:19:54.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8830000000000001e-05 2024-09-17T21:19:54.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.530 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:54.530 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844649+0000", 2024-09-17T21:19:54.530 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013143399999999999 2024-09-17T21:19:54.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.530 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:54.530 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844702+0000", 2024-09-17T21:19:54.530 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2876e-05 2024-09-17T21:19:54.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.530 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:54.530 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850696+0000", 2024-09-17T21:19:54.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059943729999999999 2024-09-17T21:19:54.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:54.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850817+0000", 2024-09-17T21:19:54.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120401 2024-09-17T21:19:54.531 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.531 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:54.531 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.531 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2635 3.d e90/23 3:b3d1e920:::rbd_data.1152a8ae2019.000000000000001f:head v 90'252, mlcod=90'246)", 2024-09-17T21:19:54.531 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:19:54.531 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 13.591589063000001, 2024-09-17T21:19:54.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0077265190000000003, 2024-09-17T21:19:54.532 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:54.532 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:54.532 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:54.532 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:54.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:54.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:19:54.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:54.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:19:54.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:54.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:54.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847155+0000", 2024-09-17T21:19:54.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.544e-06 2024-09-17T21:19:54.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:54.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847165+0000", 2024-09-17T21:19:54.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.043e-05 2024-09-17T21:19:54.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:54.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847189+0000", 2024-09-17T21:19:54.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3659000000000002e-05 2024-09-17T21:19:54.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:54.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847210+0000", 2024-09-17T21:19:54.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1163e-05 2024-09-17T21:19:54.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:54.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848856+0000", 2024-09-17T21:19:54.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001645521 2024-09-17T21:19:54.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.535 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:54.535 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848905+0000", 2024-09-17T21:19:54.535 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9048999999999997e-05 2024-09-17T21:19:54.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.535 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:54.535 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854770+0000", 2024-09-17T21:19:54.535 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058652879999999998 2024-09-17T21:19:54.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:54.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:54.535 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:54.535 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854880+0000", 2024-09-17T21:19:54.535 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010986500000000001 2024-09-17T21:19:54.536 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.536 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:54.536 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.536 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:54.536 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:54.536 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:54.538 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:19:54.538 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:54.534+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:54.638 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:54.634+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:54.702 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:19:54.702 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:54.706 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-17T21:19:54.723 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:19:54.767 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:54.763+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:54.840 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:54.834+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:54.891 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:54.892 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:54.892 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:54.892 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:54.892 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:54.911 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:19:54.959 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:54.955+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:55.013 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:19:55.061 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:55.058+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:55.065 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:55.065 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:55.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:55.066 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:55.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.066 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:19:55.066 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:55.066 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 118.048251732, 2024-09-17T21:19:55.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:19:55.066 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:55.066 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:55.067 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:55.067 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:55.067 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:19:55.067 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:19:55.067 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:19:55.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.067 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:55.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:55.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:55.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:55.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:55.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:55.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:19:55.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:19:55.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:55.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:19:55.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:19:55.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:55.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:19:55.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:19:55.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:55.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:19:55.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:19:55.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:19:55.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:19:55.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:55.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:19:55.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:19:55.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:19:55.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:19:55.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:55.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:19:55.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:19:55.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:19:55.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:19:55.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:19:55.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:55.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:19:55.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:19:55.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:55.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:19:55.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:19:55.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:55.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:19:55.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:19:55.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:55.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:19:55.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:19:55.073 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.073 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:55.073 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.073 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:55.074 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:55.074 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 98.468087452000006, 2024-09-17T21:19:55.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:19:55.074 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:55.074 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:55.074 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:55.074 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:55.074 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:19:55.074 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:19:55.074 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:19:55.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.075 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:55.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:55.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:55.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:55.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:55.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:55.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:19:55.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:19:55.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:55.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:19:55.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:19:55.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:55.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:19:55.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:19:55.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:55.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:19:55.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:19:55.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:19:55.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:19:55.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:55.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:19:55.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:19:55.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:19:55.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:19:55.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:55.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:19:55.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:19:55.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:19:55.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:19:55.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:55.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:19:55.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:19:55.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:55.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:19:55.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:19:55.080 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.080 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:55.080 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.080 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:55.080 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:55.080 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 98.467990139999998, 2024-09-17T21:19:55.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:19:55.081 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:55.081 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:55.081 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:55.081 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:55.081 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:55.081 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:55.081 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:19:55.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.081 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:55.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:55.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:55.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:55.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:55.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:55.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:19:55.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:19:55.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:55.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:19:55.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:19:55.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:55.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:19:55.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:19:55.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:55.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:19:55.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:19:55.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:19:55.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:19:55.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:55.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:19:55.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:19:55.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:19:55.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:19:55.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:55.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:19:55.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:19:55.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:19:55.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:19:55.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:19:55.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:19:55.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:19:55.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:19:55.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:19:55.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:55.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:19:55.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:19:55.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:55.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:19:55.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:19:55.088 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.088 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:55.088 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.088 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:55.088 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:55.088 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 98.465903338999993, 2024-09-17T21:19:55.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:19:55.088 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:55.088 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:55.089 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:55.089 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:55.089 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:19:55.089 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:19:55.089 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:19:55.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.089 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:55.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:55.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:55.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:55.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:55.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:55.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:19:55.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:19:55.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:55.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:19:55.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:19:55.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:55.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:19:55.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:19:55.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:55.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:19:55.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:19:55.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:19:55.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:19:55.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:19:55.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:19:55.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:19:55.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:19:55.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:19:55.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:55.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:19:55.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:19:55.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:19:55.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:19:55.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:55.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:19:55.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:19:55.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:55.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:19:55.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:19:55.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:55.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:19:55.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:19:55.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:55.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:19:55.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:19:55.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:55.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:19:55.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:19:55.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:55.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:19:55.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:19:55.095 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.095 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:55.095 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.096 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:55.096 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:55.096 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.782004557999997, 2024-09-17T21:19:55.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:19:55.096 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:55.096 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:55.096 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:55.096 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:55.096 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:55.096 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:55.096 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:19:55.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.096 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:55.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:55.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:55.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:55.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:55.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:55.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:19:55.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:19:55.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:55.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:19:55.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:19:55.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:55.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:19:55.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:19:55.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:55.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:19:55.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:19:55.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:19:55.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:19:55.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:55.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:19:55.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:19:55.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:19:55.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:19:55.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:55.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:19:55.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:19:55.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:19:55.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:19:55.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:55.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:19:55.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:19:55.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:55.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:19:55.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:19:55.101 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.101 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:55.102 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.102 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:55.102 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:55.102 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.772912847000001, 2024-09-17T21:19:55.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:19:55.102 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:55.102 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:55.102 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:55.102 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:55.102 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:55.102 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:55.103 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:19:55.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.103 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:55.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:55.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:55.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:55.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:55.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:55.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:19:55.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:19:55.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:55.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:19:55.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:19:55.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:55.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:19:55.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:19:55.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:55.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:19:55.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:19:55.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:19:55.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:19:55.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:55.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:19:55.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:19:55.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:19:55.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:19:55.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:55.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:19:55.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:19:55.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:55.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:19:55.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:19:55.107 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.107 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:55.107 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.107 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:55.107 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:55.107 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.772740795999994, 2024-09-17T21:19:55.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:19:55.107 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:55.107 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:55.107 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:55.108 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:55.108 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:55.108 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:55.108 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:19:55.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.108 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:55.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:55.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:55.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:55.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:55.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:55.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:19:55.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:19:55.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:55.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:19:55.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:19:55.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:55.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:19:55.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:19:55.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:55.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:19:55.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:19:55.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:19:55.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:19:55.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:55.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:19:55.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:19:55.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:19:55.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:19:55.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:55.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:19:55.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:19:55.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:55.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:19:55.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:19:55.112 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.112 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:55.112 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.112 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:55.113 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:55.113 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.597808048999994, 2024-09-17T21:19:55.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:19:55.113 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:55.113 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:55.113 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:55.113 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:55.113 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:55.113 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:55.113 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:19:55.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.113 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:55.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:19:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:19:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:19:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:19:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:19:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:19:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:19:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:19:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:19:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:19:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:19:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:19:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:19:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:19:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:19:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:19:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:19:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:19:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.597723492, 2024-09-17T21:19:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:19:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:19:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:19:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:19:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:19:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:19:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:19:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:19:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:19:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:19:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:19:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:19:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:19:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:19:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:19:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:19:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:19:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:19:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:19:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.597657854999994, 2024-09-17T21:19:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:19:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:19:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:19:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:19:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:19:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:19:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:19:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:19:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:19:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:19:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:19:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:19:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:19:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:19:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:19:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:19:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:19:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:19:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:19:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:19:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:19:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.224082758, 2024-09-17T21:19:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:19:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:19:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:19:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:19:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:19:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:19:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:19:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:19:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:19:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:19:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:19:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:19:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:19:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:19:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:19:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:19:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:19:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:19:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.134 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:55.134 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:19:55.134 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.223963246, 2024-09-17T21:19:55.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:19:55.134 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:55.134 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:55.134 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:55.136 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:55.136 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:55.136 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:55.136 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:19:55.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.136 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:55.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:55.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:19:55.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:19:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:19:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:19:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:19:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:19:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:19:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:19:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:19:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:19:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:19:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:19:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:19:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:19:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:19:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:19:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2695 3.15 3:aedfd032:::rbd_data.1152a8ae2019.0000000000000041:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.843199+0000", 2024-09-17T21:19:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.218817941999999, 2024-09-17T21:19:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079730569000000001, 2024-09-17T21:19:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2695 2024-09-17T21:19:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843199+0000", 2024-09-17T21:19:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843199+0000", 2024-09-17T21:19:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843201+0000", 2024-09-17T21:19:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0899999999999999e-06 2024-09-17T21:19:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843214+0000", 2024-09-17T21:19:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2233e-05 2024-09-17T21:19:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843245+0000", 2024-09-17T21:19:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1464000000000002e-05 2024-09-17T21:19:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:55.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843266+0000", 2024-09-17T21:19:55.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0879000000000001e-05 2024-09-17T21:19:55.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847920+0000", 2024-09-17T21:19:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.004654292 2024-09-17T21:19:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.848240+0000", 2024-09-17T21:19:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031994499999999997 2024-09-17T21:19:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922930+0000", 2024-09-17T21:19:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.074689666000000002 2024-09-17T21:19:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2732 3.15 3:ab9a4731:::rbd_data.1152a8ae2019.0000000000000058:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.846909+0000", 2024-09-17T21:19:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.215107895999999, 2024-09-17T21:19:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076145476000000004, 2024-09-17T21:19:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2732 2024-09-17T21:19:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846909+0000", 2024-09-17T21:19:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846909+0000", 2024-09-17T21:19:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846911+0000", 2024-09-17T21:19:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.31e-06 2024-09-17T21:19:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846916+0000", 2024-09-17T21:19:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.023e-06 2024-09-17T21:19:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846931+0000", 2024-09-17T21:19:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5549999999999999e-05 2024-09-17T21:19:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846943+0000", 2024-09-17T21:19:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1783000000000001e-05 2024-09-17T21:19:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.850592+0000", 2024-09-17T21:19:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036486159999999999 2024-09-17T21:19:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.850870+0000", 2024-09-17T21:19:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027864200000000001 2024-09-17T21:19:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.923055+0000", 2024-09-17T21:19:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.072184551999999999 2024-09-17T21:19:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2737 3.15 3:a8c4e696:::rbd_data.1152a8ae2019.000000000000005b:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.847014+0000", 2024-09-17T21:19:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.215002760999999, 2024-09-17T21:19:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076158271, 2024-09-17T21:19:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2737 2024-09-17T21:19:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847014+0000", 2024-09-17T21:19:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847014+0000", 2024-09-17T21:19:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847015+0000", 2024-09-17T21:19:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0699999999999999e-06 2024-09-17T21:19:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847021+0000", 2024-09-17T21:19:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.446e-06 2024-09-17T21:19:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847038+0000", 2024-09-17T21:19:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7257000000000001e-05 2024-09-17T21:19:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847051+0000", 2024-09-17T21:19:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2649999999999999e-05 2024-09-17T21:19:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.852127+0000", 2024-09-17T21:19:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050757420000000003 2024-09-17T21:19:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.852423+0000", 2024-09-17T21:19:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029608100000000001 2024-09-17T21:19:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.923173+0000", 2024-09-17T21:19:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.070750024999999994 2024-09-17T21:19:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3397 3.8 3:174f9d43:::rbd_data.1152a8ae2019.000000000000006e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.897797+0000", 2024-09-17T21:19:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.164219815999999, 2024-09-17T21:19:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045478192000000001, 2024-09-17T21:19:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3397 2024-09-17T21:19:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897797+0000", 2024-09-17T21:19:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897797+0000", 2024-09-17T21:19:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897799+0000", 2024-09-17T21:19:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.238e-06 2024-09-17T21:19:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897806+0000", 2024-09-17T21:19:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5900000000000002e-06 2024-09-17T21:19:55.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:55.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897822+0000", 2024-09-17T21:19:55.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5659e-05 2024-09-17T21:19:55.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:55.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897832+0000", 2024-09-17T21:19:55.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0034000000000001e-05 2024-09-17T21:19:55.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.931249+0000", 2024-09-17T21:19:55.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033416700000000001 2024-09-17T21:19:55.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:55.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.931425+0000", 2024-09-17T21:19:55.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017629500000000001 2024-09-17T21:19:55.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:55.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943276+0000", 2024-09-17T21:19:55.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011850675999999999 2024-09-17T21:19:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3524 3.8 3:119ba024:::rbd_data.1152c6ce51b2.0000000000000005:17 [read 1261568~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.901789+0000", 2024-09-17T21:19:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.160227889, 2024-09-17T21:19:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.042862082000000003, 2024-09-17T21:19:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3524 2024-09-17T21:19:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901789+0000", 2024-09-17T21:19:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901789+0000", 2024-09-17T21:19:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901790+0000", 2024-09-17T21:19:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0400000000000005e-07 2024-09-17T21:19:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901795+0000", 2024-09-17T21:19:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3839999999999999e-06 2024-09-17T21:19:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901807+0000", 2024-09-17T21:19:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2532e-05 2024-09-17T21:19:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901816+0000", 2024-09-17T21:19:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2220000000000008e-06 2024-09-17T21:19:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.944407+0000", 2024-09-17T21:19:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.042591088999999999 2024-09-17T21:19:55.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:55.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.944498+0000", 2024-09-17T21:19:55.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0883000000000004e-05 2024-09-17T21:19:55.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:55.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.944651+0000", 2024-09-17T21:19:55.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015306799999999999 2024-09-17T21:19:55.163 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.163 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:55.163 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.164 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3679 3.8 3:129b2dec:::rbd_data.1152a8ae2019.0000000000000012:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:55.164 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.908354+0000", 2024-09-17T21:19:55.164 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.153663527999999, 2024-09-17T21:19:55.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046195880000000002, 2024-09-17T21:19:55.164 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:55.164 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:55.164 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:55.164 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:55.164 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:55.164 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:55.164 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3679 2024-09-17T21:19:55.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.165 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:55.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:55.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908354+0000", 2024-09-17T21:19:55.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:55.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908354+0000", 2024-09-17T21:19:55.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:55.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908355+0000", 2024-09-17T21:19:55.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9999999999999995e-07 2024-09-17T21:19:55.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:55.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908361+0000", 2024-09-17T21:19:55.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7399999999999998e-06 2024-09-17T21:19:55.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:55.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908375+0000", 2024-09-17T21:19:55.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.363e-05 2024-09-17T21:19:55.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:55.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908386+0000", 2024-09-17T21:19:55.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.06e-05 2024-09-17T21:19:55.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.945600+0000", 2024-09-17T21:19:55.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.037214199000000003 2024-09-17T21:19:55.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:55.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.945650+0000", 2024-09-17T21:19:55.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9737999999999997e-05 2024-09-17T21:19:55.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:55.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.945799+0000", 2024-09-17T21:19:55.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014921699999999999 2024-09-17T21:19:55.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:55.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954371+0000", 2024-09-17T21:19:55.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085719730000000001 2024-09-17T21:19:55.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:55.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954460+0000", 2024-09-17T21:19:55.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9596000000000001e-05 2024-09-17T21:19:55.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:55.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954511+0000", 2024-09-17T21:19:55.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0455999999999999e-05 2024-09-17T21:19:55.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:55.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954550+0000", 2024-09-17T21:19:55.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8730999999999999e-05 2024-09-17T21:19:55.169 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.169 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:55.169 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.170 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3703 3.13 3:cdb4d88a:::rbd_data.1152a8ae2019.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:55.170 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.908402+0000", 2024-09-17T21:19:55.170 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.153615488, 2024-09-17T21:19:55.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046393446999999997, 2024-09-17T21:19:55.170 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:55.170 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:55.170 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:55.170 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:55.170 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:55.170 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:55.170 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3703 2024-09-17T21:19:55.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.170 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:55.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:55.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908402+0000", 2024-09-17T21:19:55.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:55.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908402+0000", 2024-09-17T21:19:55.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:55.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908403+0000", 2024-09-17T21:19:55.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0529999999999999e-06 2024-09-17T21:19:55.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:55.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908409+0000", 2024-09-17T21:19:55.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2219999999999999e-06 2024-09-17T21:19:55.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:55.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908422+0000", 2024-09-17T21:19:55.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3147e-05 2024-09-17T21:19:55.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:55.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908432+0000", 2024-09-17T21:19:55.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.039e-05 2024-09-17T21:19:55.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.942217+0000", 2024-09-17T21:19:55.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033784994999999998 2024-09-17T21:19:55.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:55.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.942296+0000", 2024-09-17T21:19:55.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8218000000000002e-05 2024-09-17T21:19:55.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:55.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943003+0000", 2024-09-17T21:19:55.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000707403 2024-09-17T21:19:55.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:55.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954635+0000", 2024-09-17T21:19:55.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011631786999999999 2024-09-17T21:19:55.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:55.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954706+0000", 2024-09-17T21:19:55.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1064999999999998e-05 2024-09-17T21:19:55.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:55.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954757+0000", 2024-09-17T21:19:55.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0760000000000002e-05 2024-09-17T21:19:55.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:55.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954795+0000", 2024-09-17T21:19:55.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8407e-05 2024-09-17T21:19:55.175 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.175 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:55.175 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.175 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3711 3.13 3:ca5a0e5e:::rbd_data.1152a8ae2019.0000000000000040:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:55.176 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.908514+0000", 2024-09-17T21:19:55.176 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.153503325000001, 2024-09-17T21:19:55.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046370871000000001, 2024-09-17T21:19:55.176 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:55.176 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:55.176 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:55.176 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:55.176 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:55.176 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:55.176 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3711 2024-09-17T21:19:55.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.176 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:55.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:55.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908514+0000", 2024-09-17T21:19:55.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:55.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908514+0000", 2024-09-17T21:19:55.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:55.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:55.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908515+0000", 2024-09-17T21:19:55.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0449999999999999e-06 2024-09-17T21:19:55.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:55.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908521+0000", 2024-09-17T21:19:55.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2179999999999996e-06 2024-09-17T21:19:55.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:55.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908535+0000", 2024-09-17T21:19:55.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4124999999999999e-05 2024-09-17T21:19:55.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:55.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908548+0000", 2024-09-17T21:19:55.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2680000000000001e-05 2024-09-17T21:19:55.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:55.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943530+0000", 2024-09-17T21:19:55.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034981655 2024-09-17T21:19:55.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:55.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943636+0000", 2024-09-17T21:19:55.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106086 2024-09-17T21:19:55.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:55.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943837+0000", 2024-09-17T21:19:55.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201826 2024-09-17T21:19:55.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:55.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954688+0000", 2024-09-17T21:19:55.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010850963 2024-09-17T21:19:55.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:55.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954805+0000", 2024-09-17T21:19:55.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000117025 2024-09-17T21:19:55.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:55.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954851+0000", 2024-09-17T21:19:55.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5065000000000003e-05 2024-09-17T21:19:55.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:55.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:55.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:55.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954885+0000", 2024-09-17T21:19:55.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4183000000000001e-05 2024-09-17T21:19:55.181 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.181 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:55.181 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.181 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:55.181 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:55.182 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:55.183 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:55.183 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:55.184 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:55.184 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:55.184 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:19:55.185 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:55.178+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:55.197 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:55.197 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:55.215 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:55.215 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:55.234 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:55.234 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:55.252 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:55.253 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:55.315 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:55.315 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:55.317 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:55.314+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:55.329 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:19:55.385 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:55.382+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:55.453 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:55.453 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:55.467 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:19:55.485 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:55.482+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:55.582 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:55.583 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:55.586 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:55.582+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:55.596 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:19:55.609 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-17T21:19:55.687 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:55.682+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:55.726 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:55.726 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:55.742 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:19:55.788 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:55.786+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:55.879 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:55.879 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:55.889 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:55.886+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:55.896 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:19:55.991 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:55.987+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:56.032 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:56.032 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:56.049 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:19:56.096 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:56.094+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:56.190 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:56.190 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:56.192 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:56.186+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:56.205 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:19:56.293 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:56.290+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:56.377 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:56.377 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:56.391 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:19:56.433 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:56.430+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:56.446 INFO:teuthology.orchestra.run.smithi184.stdout:4040 trunc from 0x583f000 to 0x6a60000 2024-09-17T21:19:56.472 INFO:teuthology.orchestra.run.smithi184.stdout:4041 read 0x908e00 thru 0x90c3f3 (0x35f4 bytes) 2024-09-17T21:19:56.474 INFO:teuthology.orchestra.run.smithi184.stdout:4042 write 0x6cf400 thru 0x6d4b1e (0x571f bytes) 2024-09-17T21:19:56.474 INFO:teuthology.orchestra.run.smithi184.stdout:4043 read 0x39af600 thru 0x39b3ab2 (0x44b3 bytes) 2024-09-17T21:19:56.474 INFO:teuthology.orchestra.run.smithi184.stdout:4044 write 0x4e8cc00 thru 0x4e9bf3f (0xf340 bytes) 2024-09-17T21:19:56.474 INFO:teuthology.orchestra.run.smithi184.stdout:4045 read 0x6389a00 thru 0x638d713 (0x3d14 bytes) 2024-09-17T21:19:56.474 INFO:teuthology.orchestra.run.smithi184.stdout:4047 punch from 0x15e800 to 0x165e00, (0x7600 bytes) 2024-09-17T21:19:56.490 INFO:teuthology.orchestra.run.smithi184.stdout:4049 read 0x9b1200 thru 0x9b192a (0x72b bytes) 2024-09-17T21:19:56.490 INFO:teuthology.orchestra.run.smithi184.stdout:4050 write 0x1951200 thru 0x1956617 (0x5418 bytes) 2024-09-17T21:19:56.490 INFO:teuthology.orchestra.run.smithi184.stdout:4055 write 0x843600 thru 0x848b09 (0x550a bytes) 2024-09-17T21:19:56.495 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:56.490+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:56.509 INFO:teuthology.orchestra.run.smithi184.stdout:4056 trunc from 0x6a60000 to 0x6bcd400 2024-09-17T21:19:56.572 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:56.572 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:56.588 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:19:56.595 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:56.591+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:56.667 INFO:teuthology.orchestra.run.smithi184.stdout:4057 write 0xace8600 thru 0xace8db7 (0x7b8 bytes) 2024-09-17T21:19:56.696 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:56.689+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:56.700 INFO:teuthology.orchestra.run.smithi184.stdout:4058 trunc from 0xace8e00 to 0x452b600 2024-09-17T21:19:56.731 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:19:56.744 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 0] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:19:56.744 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:19:56.744 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-17T21:19:56.744 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-17T21:19:56.761 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:56.761 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:56.781 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:19:56.825 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:56.821+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:56.879 INFO:teuthology.orchestra.run.smithi184.stdout:4059 trunc from 0x452b600 to 0xd730200 2024-09-17T21:19:56.896 INFO:teuthology.orchestra.run.smithi184.stdout:4060 read 0x3ec7000 thru 0x3eca853 (0x3854 bytes) 2024-09-17T21:19:56.897 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:56.893+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:56.899 INFO:teuthology.orchestra.run.smithi184.stdout:4061 punch from 0x7c86800 to 0x7c92e00, (0xc600 bytes) 2024-09-17T21:19:56.899 INFO:teuthology.orchestra.run.smithi184.stdout:4062 punch from 0x7cb0c00 to 0x7cb0e00, (0x200 bytes) 2024-09-17T21:19:56.899 INFO:teuthology.orchestra.run.smithi184.stdout:4065 read 0x3914600 thru 0x39178a1 (0x32a2 bytes) 2024-09-17T21:19:56.900 INFO:teuthology.orchestra.run.smithi184.stdout:4066 read 0x199c000 thru 0x199ef1a (0x2f1b bytes) 2024-09-17T21:19:56.901 INFO:teuthology.orchestra.run.smithi184.stdout:4067 read 0xd1bee00 thru 0xd1c49f7 (0x5bf8 bytes) 2024-09-17T21:19:56.901 INFO:teuthology.orchestra.run.smithi184.stdout:4068 punch from 0x2a57e00 to 0x2a5ea00, (0x6c00 bytes) 2024-09-17T21:19:56.910 INFO:teuthology.orchestra.run.smithi184.stdout:4069 punch from 0xcaee200 to 0xcafd400, (0xf200 bytes) 2024-09-17T21:19:56.910 INFO:teuthology.orchestra.run.smithi184.stdout:4070 write 0x5a51400 thru 0x5a5f449 (0xe04a bytes) 2024-09-17T21:19:56.934 INFO:teuthology.orchestra.run.smithi184.stdout:4072 trunc from 0xd730200 to 0x5151200 2024-09-17T21:19:56.955 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-17T21:19:56.964 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:56.965 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:56.975 INFO:teuthology.orchestra.run.smithi184.stdout:4074 read 0x3d48600 thru 0x3d53907 (0xb308 bytes) 2024-09-17T21:19:56.984 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:19:56.994 INFO:teuthology.orchestra.run.smithi184.stdout:4075 trunc from 0x5151200 to 0xb887000 2024-09-17T21:19:56.998 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:56.991+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:57.023 INFO:teuthology.orchestra.run.smithi184.stdout:4077 write 0xc83ce00 thru 0xc84ab2a (0xdd2b bytes) 2024-09-17T21:19:57.024 INFO:teuthology.orchestra.run.smithi184.stdout:4078 read 0x5654000 thru 0x5659263 (0x5264 bytes) 2024-09-17T21:19:57.025 INFO:teuthology.orchestra.run.smithi184.stdout:4079 write 0xc5fc000 thru 0xc5fee83 (0x2e84 bytes) 2024-09-17T21:19:57.025 INFO:teuthology.orchestra.run.smithi184.stdout:4080 punch from 0x4219400 to 0x4224000, (0xac00 bytes) 2024-09-17T21:19:57.026 INFO:teuthology.orchestra.run.smithi184.stdout:4084 write 0x7e77a00 thru 0x7e7b0aa (0x36ab bytes) 2024-09-17T21:19:57.027 INFO:teuthology.orchestra.run.smithi184.stdout:4087 read 0x2994800 thru 0x299e095 (0x9896 bytes) 2024-09-17T21:19:57.050 INFO:teuthology.orchestra.run.smithi184.stdout:4092 trunc from 0xc84ac00 to 0x2f01e00 2024-09-17T21:19:57.100 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:57.095+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:57.162 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-17T21:19:57.170 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:57.170 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:57.180 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-17T21:19:57.190 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:19:57.205 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:57.201+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:57.227 INFO:teuthology.orchestra.run.smithi184.stdout:4093 write 0xdc9d800 thru 0xdca0537 (0x2d38 bytes) 2024-09-17T21:19:57.229 INFO:teuthology.orchestra.run.smithi184.stdout:4094 read 0x307bc00 thru 0x30840b4 (0x84b5 bytes) 2024-09-17T21:19:57.230 INFO:teuthology.orchestra.run.smithi184.stdout:4095 punch from 0x9cf000 to 0x9d1e00, (0x2e00 bytes) 2024-09-17T21:19:57.231 INFO:teuthology.orchestra.run.smithi184.stdout:4098 read 0x94ff600 thru 0x95032c5 (0x3cc6 bytes) 2024-09-17T21:19:57.231 INFO:teuthology.orchestra.run.smithi184.stdout:4101 write 0x6324600 thru 0x632b16b (0x6b6c bytes) 2024-09-17T21:19:57.231 INFO:teuthology.orchestra.run.smithi184.stdout:4102 read 0x89bba00 thru 0x89c63da (0xa9db bytes) 2024-09-17T21:19:57.242 INFO:teuthology.orchestra.run.smithi184.stdout:4103 trunc from 0xdca0600 to 0x7344e00 2024-09-17T21:19:57.269 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:19:57.278 INFO:teuthology.orchestra.run.smithi184.stdout:4105 read 0x5a9fc00 thru 0x5aa69e6 (0x6de7 bytes) 2024-09-17T21:19:57.280 INFO:teuthology.orchestra.run.smithi184.stdout:4106 read 0x11b2c00 thru 0x11bc63f (0x9a40 bytes) 2024-09-17T21:19:57.281 INFO:teuthology.orchestra.run.smithi184.stdout:4107 clone 27 order 20 su 131072 sc 3 2024-09-17T21:19:57.300 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:57.295+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:57.335 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-17T21:19:57.373 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:57.373 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:57.389 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:19:57.430 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:57.431 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:57.431 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:57.425+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:57.449 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:57.449 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:57.466 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:57.467 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:57.488 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:57.488 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:57.502 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:57.497+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:57.509 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:57.509 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:57.530 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:57.530 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:57.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-17T21:19:57.548 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:57.548 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:57.548 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:19:57.548 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:19:57.548 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:57.563 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:19:57.603 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:57.599+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:57.703 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:57.699+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:57.721 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:57.721 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:19:57.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:19:57.721 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:19:57.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.722 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:19:57.722 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:57.722 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 99.502316707999995, 2024-09-17T21:19:57.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:19:57.722 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:57.722 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:57.722 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:57.722 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:57.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:57.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:57.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:57.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:19:57.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:57.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:19:57.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:19:57.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:57.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:19:57.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:19:57.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:57.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:19:57.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:19:57.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:57.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:19:57.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:19:57.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:57.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:19:57.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:19:57.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:57.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:19:57.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:19:57.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:57.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:19:57.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:19:57.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:57.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:19:57.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:19:57.726 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.726 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:57.726 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.726 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:19:57.727 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:57.727 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 94.852016043999996, 2024-09-17T21:19:57.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:19:57.727 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:57.727 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:57.727 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:57.727 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:57.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:57.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:57.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:57.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:19:57.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:57.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:19:57.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:19:57.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:57.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:19:57.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:19:57.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:57.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:19:57.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:19:57.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:57.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:19:57.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:19:57.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:57.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:19:57.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:19:57.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:57.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:19:57.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:19:57.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:57.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:19:57.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:19:57.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:57.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:19:57.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:19:57.731 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.731 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:57.731 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.731 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:19:57.731 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:57.731 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 94.850750961000003, 2024-09-17T21:19:57.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:19:57.732 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:57.732 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:57.732 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:57.732 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:57.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:57.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:57.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:57.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:19:57.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:57.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:19:57.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:19:57.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:57.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:19:57.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:19:57.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:57.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:19:57.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:19:57.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:57.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:19:57.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:19:57.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:57.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:19:57.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:19:57.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:57.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:19:57.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:19:57.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:57.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:19:57.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:19:57.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:57.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:19:57.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:19:57.736 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.736 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:57.736 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.736 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:19:57.736 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:57.736 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 94.840971550999996, 2024-09-17T21:19:57.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:19:57.736 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:57.736 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:57.736 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:57.736 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:57.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:57.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:57.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:57.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:19:57.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:57.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:19:57.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:19:57.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:57.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:19:57.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:19:57.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:57.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:19:57.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:19:57.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:57.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:19:57.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:19:57.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:57.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:19:57.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:19:57.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:57.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:19:57.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:19:57.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:57.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:19:57.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:19:57.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:57.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:19:57.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:19:57.740 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.740 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:57.740 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.740 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:19:57.740 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:57.741 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 94.832249148000002, 2024-09-17T21:19:57.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:19:57.741 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:57.741 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:57.741 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:57.741 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:57.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:57.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:57.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:57.742 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:19:57.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:57.742 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:19:57.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:19:57.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:57.742 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:19:57.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:19:57.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:19:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:19:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:19:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:19:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:19:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:19:57.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:57.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:19:57.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:19:57.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:57.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:19:57.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:19:57.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:57.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:19:57.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:19:57.745 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.745 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:57.745 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.745 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:19:57.745 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:57.745 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 81.447158583999993, 2024-09-17T21:19:57.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:19:57.745 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:57.745 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:57.745 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:57.746 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:57.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:57.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:57.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:57.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:19:57.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:57.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:19:57.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:19:57.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.747 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:57.747 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:19:57.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:19:57.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.747 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:57.747 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:19:57.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:19:57.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:57.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:19:57.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:19:57.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:57.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:19:57.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:19:57.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:57.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:19:57.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:19:57.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:57.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:19:57.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:19:57.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:57.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:19:57.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:19:57.749 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.750 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:57.750 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.750 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1780 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'195, mlcod=69'190)", 2024-09-17T21:19:57.750 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:19:57.750 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 44.469302980000002, 2024-09-17T21:19:57.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075761769999999999, 2024-09-17T21:19:57.750 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:57.750 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:57.750 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:57.750 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:57.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:57.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:19:57.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:57.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:19:57.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:57.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250375+0000", 2024-09-17T21:19:57.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:19:57.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.752 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:57.752 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250383+0000", 2024-09-17T21:19:57.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7870000000000002e-06 2024-09-17T21:19:57.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.752 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:57.752 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250402+0000", 2024-09-17T21:19:57.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9346000000000001e-05 2024-09-17T21:19:57.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.752 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:57.752 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250413+0000", 2024-09-17T21:19:57.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.039e-05 2024-09-17T21:19:57.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:57.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251353+0000", 2024-09-17T21:19:57.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000940091 2024-09-17T21:19:57.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:57.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251407+0000", 2024-09-17T21:19:57.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4701999999999999e-05 2024-09-17T21:19:57.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:57.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257829+0000", 2024-09-17T21:19:57.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064212779999999999 2024-09-17T21:19:57.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:57.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257950+0000", 2024-09-17T21:19:57.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121623 2024-09-17T21:19:57.754 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.754 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:57.754 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.754 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1781 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'196, mlcod=69'190)", 2024-09-17T21:19:57.754 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:19:57.754 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 44.468220029000001, 2024-09-17T21:19:57.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065632939999999999, 2024-09-17T21:19:57.755 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:57.755 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:57.755 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:57.755 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:57.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:57.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:19:57.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:57.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:19:57.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:57.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251459+0000", 2024-09-17T21:19:57.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.624e-06 2024-09-17T21:19:57.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:57.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251471+0000", 2024-09-17T21:19:57.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2636000000000001e-05 2024-09-17T21:19:57.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:57.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251498+0000", 2024-09-17T21:19:57.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:19:57.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:57.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251515+0000", 2024-09-17T21:19:57.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6939000000000002e-05 2024-09-17T21:19:57.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:57.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252938+0000", 2024-09-17T21:19:57.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001423082 2024-09-17T21:19:57.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:57.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252999+0000", 2024-09-17T21:19:57.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1138999999999999e-05 2024-09-17T21:19:57.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:57.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257955+0000", 2024-09-17T21:19:57.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049559640000000002 2024-09-17T21:19:57.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:57.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.258020+0000", 2024-09-17T21:19:57.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5504000000000002e-05 2024-09-17T21:19:57.760 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.760 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:57.760 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.761 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1788 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'198, mlcod=69'191)", 2024-09-17T21:19:57.761 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:19:57.761 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 44.466194539999996, 2024-09-17T21:19:57.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073690370000000002, 2024-09-17T21:19:57.761 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:57.761 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:57.761 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:57.761 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:57.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:57.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:19:57.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:57.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:19:57.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:57.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253484+0000", 2024-09-17T21:19:57.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2470000000000001e-06 2024-09-17T21:19:57.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:57.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253492+0000", 2024-09-17T21:19:57.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7159999999999992e-06 2024-09-17T21:19:57.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:57.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253509+0000", 2024-09-17T21:19:57.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.666e-05 2024-09-17T21:19:57.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:57.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253522+0000", 2024-09-17T21:19:57.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2643000000000001e-05 2024-09-17T21:19:57.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:57.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254853+0000", 2024-09-17T21:19:57.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331238 2024-09-17T21:19:57.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:57.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254885+0000", 2024-09-17T21:19:57.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1562e-05 2024-09-17T21:19:57.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:57.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260788+0000", 2024-09-17T21:19:57.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059030749999999998 2024-09-17T21:19:57.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:57.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260852+0000", 2024-09-17T21:19:57.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3895999999999999e-05 2024-09-17T21:19:57.765 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.765 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:57.765 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.765 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5746 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'477, mlcod=71'473)", 2024-09-17T21:19:57.765 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:19:57.765 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.460766982000003, 2024-09-17T21:19:57.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074182709999999997, 2024-09-17T21:19:57.765 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:57.765 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:57.766 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:57.766 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:57.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:57.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:19:57.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:57.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:19:57.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:57.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258912+0000", 2024-09-17T21:19:57.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:19:57.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:57.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258926+0000", 2024-09-17T21:19:57.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4249e-05 2024-09-17T21:19:57.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:57.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258946+0000", 2024-09-17T21:19:57.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9879999999999999e-05 2024-09-17T21:19:57.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:57.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258956+0000", 2024-09-17T21:19:57.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:19:57.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:57.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259380+0000", 2024-09-17T21:19:57.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042337599999999999 2024-09-17T21:19:57.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:57.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259412+0000", 2024-09-17T21:19:57.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:19:57.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266191+0000", 2024-09-17T21:19:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067793269999999999 2024-09-17T21:19:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266328+0000", 2024-09-17T21:19:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013690999999999999 2024-09-17T21:19:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5747 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'478, mlcod=71'473)", 2024-09-17T21:19:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:19:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.460060632999998, 2024-09-17T21:19:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068779440000000004, 2024-09-17T21:19:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:19:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:19:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259618+0000", 2024-09-17T21:19:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6199999999999999e-06 2024-09-17T21:19:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259630+0000", 2024-09-17T21:19:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1884e-05 2024-09-17T21:19:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259657+0000", 2024-09-17T21:19:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7339e-05 2024-09-17T21:19:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259677+0000", 2024-09-17T21:19:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0156000000000001e-05 2024-09-17T21:19:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260521+0000", 2024-09-17T21:19:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084312899999999997 2024-09-17T21:19:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260585+0000", 2024-09-17T21:19:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4558000000000001e-05 2024-09-17T21:19:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266336+0000", 2024-09-17T21:19:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057505020000000002 2024-09-17T21:19:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266494+0000", 2024-09-17T21:19:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158756 2024-09-17T21:19:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5750 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'479, mlcod=71'473)", 2024-09-17T21:19:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:19:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.459348439999999, 2024-09-17T21:19:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006302874, 2024-09-17T21:19:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:19:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:19:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260330+0000", 2024-09-17T21:19:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:19:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260343+0000", 2024-09-17T21:19:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2862e-05 2024-09-17T21:19:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260367+0000", 2024-09-17T21:19:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3723e-05 2024-09-17T21:19:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260386+0000", 2024-09-17T21:19:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8879999999999999e-05 2024-09-17T21:19:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261579+0000", 2024-09-17T21:19:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001193431 2024-09-17T21:19:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261608+0000", 2024-09-17T21:19:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8858999999999999e-05 2024-09-17T21:19:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266505+0000", 2024-09-17T21:19:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004896999 2024-09-17T21:19:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266631+0000", 2024-09-17T21:19:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012655300000000001 2024-09-17T21:19:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5751 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'480, mlcod=71'473)", 2024-09-17T21:19:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:19:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.458641227999998, 2024-09-17T21:19:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075756299999999999, 2024-09-17T21:19:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:19:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:19:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261037+0000", 2024-09-17T21:19:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:19:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261049+0000", 2024-09-17T21:19:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1063999999999999e-05 2024-09-17T21:19:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261072+0000", 2024-09-17T21:19:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3102e-05 2024-09-17T21:19:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261133+0000", 2024-09-17T21:19:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1276000000000001e-05 2024-09-17T21:19:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:19:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015334319999999999 2024-09-17T21:19:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262702+0000", 2024-09-17T21:19:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5281999999999999e-05 2024-09-17T21:19:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268527+0000", 2024-09-17T21:19:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058250389999999997 2024-09-17T21:19:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268611+0000", 2024-09-17T21:19:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4724999999999999e-05 2024-09-17T21:19:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5756 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'481, mlcod=71'473)", 2024-09-17T21:19:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:19:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.457780372000002, 2024-09-17T21:19:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006772386, 2024-09-17T21:19:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:19:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:19:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261899+0000", 2024-09-17T21:19:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.903e-06 2024-09-17T21:19:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261912+0000", 2024-09-17T21:19:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-05 2024-09-17T21:19:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261938+0000", 2024-09-17T21:19:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6098999999999999e-05 2024-09-17T21:19:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261955+0000", 2024-09-17T21:19:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.696e-05 2024-09-17T21:19:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263845+0000", 2024-09-17T21:19:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001890187 2024-09-17T21:19:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263907+0000", 2024-09-17T21:19:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1600999999999995e-05 2024-09-17T21:19:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268615+0000", 2024-09-17T21:19:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047086070000000001 2024-09-17T21:19:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268669+0000", 2024-09-17T21:19:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3588999999999998e-05 2024-09-17T21:19:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4978.0:598 3.9 e80/23 3:9701cc17:::rbd_data.1152724ff2a3.0000000000000019:head v 80'202, mlcod=80'198)", 2024-09-17T21:19:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:19:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 30.956205989000001, 2024-09-17T21:19:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066473340000000004, 2024-09-17T21:19:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:19:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:19:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763472+0000", 2024-09-17T21:19:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6899999999999997e-07 2024-09-17T21:19:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763478+0000", 2024-09-17T21:19:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:19:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763489+0000", 2024-09-17T21:19:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0628999999999999e-05 2024-09-17T21:19:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763495+0000", 2024-09-17T21:19:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6649999999999997e-06 2024-09-17T21:19:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763907+0000", 2024-09-17T21:19:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041163899999999999 2024-09-17T21:19:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763976+0000", 2024-09-17T21:19:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8534999999999993e-05 2024-09-17T21:19:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.769900+0000", 2024-09-17T21:19:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059243270000000001 2024-09-17T21:19:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.770118+0000", 2024-09-17T21:19:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021842500000000001 2024-09-17T21:19:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:468 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'94, mlcod=89'87)", 2024-09-17T21:19:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:19:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.033460689999998, 2024-09-17T21:19:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066528790000000004, 2024-09-17T21:19:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:19:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:19:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686218+0000", 2024-09-17T21:19:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3060000000000001e-06 2024-09-17T21:19:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686256+0000", 2024-09-17T21:19:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8333e-05 2024-09-17T21:19:57.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:57.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686272+0000", 2024-09-17T21:19:57.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:19:57.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:57.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686283+0000", 2024-09-17T21:19:57.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0903e-05 2024-09-17T21:19:57.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:57.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686682+0000", 2024-09-17T21:19:57.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039935099999999999 2024-09-17T21:19:57.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:57.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686785+0000", 2024-09-17T21:19:57.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010275199999999999 2024-09-17T21:19:57.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:57.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692816+0000", 2024-09-17T21:19:57.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060317579999999999 2024-09-17T21:19:57.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692869+0000", 2024-09-17T21:19:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2775999999999998e-05 2024-09-17T21:19:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:755 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'127, mlcod=89'122)", 2024-09-17T21:19:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:19:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 17.897005476, 2024-09-17T21:19:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084014080000000008, 2024-09-17T21:19:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:19:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:19:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:19:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:19:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822683+0000", 2024-09-17T21:19:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0674e-05 2024-09-17T21:19:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822698+0000", 2024-09-17T21:19:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5189e-05 2024-09-17T21:19:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822708+0000", 2024-09-17T21:19:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9119999999999993e-06 2024-09-17T21:19:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822748+0000", 2024-09-17T21:19:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9727000000000001e-05 2024-09-17T21:19:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822759+0000", 2024-09-17T21:19:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0865e-05 2024-09-17T21:19:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831034+0000", 2024-09-17T21:19:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082750710000000002 2024-09-17T21:19:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831073+0000", 2024-09-17T21:19:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9063999999999997e-05 2024-09-17T21:19:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:756 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'128, mlcod=89'124)", 2024-09-17T21:19:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:19:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 17.895082844000001, 2024-09-17T21:19:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0076300270000000002, 2024-09-17T21:19:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:19:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:19:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824595+0000", 2024-09-17T21:19:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-17T21:19:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824600+0000", 2024-09-17T21:19:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:19:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824612+0000", 2024-09-17T21:19:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.181e-05 2024-09-17T21:19:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824619+0000", 2024-09-17T21:19:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:19:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824671+0000", 2024-09-17T21:19:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2262999999999998e-05 2024-09-17T21:19:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824695+0000", 2024-09-17T21:19:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3883000000000002e-05 2024-09-17T21:19:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832185+0000", 2024-09-17T21:19:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007490048 2024-09-17T21:19:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832224+0000", 2024-09-17T21:19:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8957e-05 2024-09-17T21:19:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2617 3.d e90/23 3:b0143718:::rbd_data.1152a8ae2019.0000000000000016:head v 90'249, mlcod=90'246)", 2024-09-17T21:19:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:19:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 16.875231843999998, 2024-09-17T21:19:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063715559999999996, 2024-09-17T21:19:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:19:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:19:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844455+0000", 2024-09-17T21:19:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4259999999999992e-06 2024-09-17T21:19:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844470+0000", 2024-09-17T21:19:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5386000000000001e-05 2024-09-17T21:19:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844499+0000", 2024-09-17T21:19:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.883e-05 2024-09-17T21:19:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844518+0000", 2024-09-17T21:19:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8830000000000001e-05 2024-09-17T21:19:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844649+0000", 2024-09-17T21:19:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013143399999999999 2024-09-17T21:19:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844702+0000", 2024-09-17T21:19:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2876e-05 2024-09-17T21:19:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850696+0000", 2024-09-17T21:19:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059943729999999999 2024-09-17T21:19:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850817+0000", 2024-09-17T21:19:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120401 2024-09-17T21:19:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2635 3.d e90/23 3:b3d1e920:::rbd_data.1152a8ae2019.000000000000001f:head v 90'252, mlcod=90'246)", 2024-09-17T21:19:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:19:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 16.872523774000001, 2024-09-17T21:19:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0077265190000000003, 2024-09-17T21:19:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:19:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:19:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:19:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:19:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:19:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:19:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:19:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:19:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:19:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847155+0000", 2024-09-17T21:19:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.544e-06 2024-09-17T21:19:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:19:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847165+0000", 2024-09-17T21:19:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.043e-05 2024-09-17T21:19:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:19:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847189+0000", 2024-09-17T21:19:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3659000000000002e-05 2024-09-17T21:19:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:19:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847210+0000", 2024-09-17T21:19:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1163e-05 2024-09-17T21:19:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:19:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848856+0000", 2024-09-17T21:19:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001645521 2024-09-17T21:19:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:19:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848905+0000", 2024-09-17T21:19:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9048999999999997e-05 2024-09-17T21:19:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:19:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854770+0000", 2024-09-17T21:19:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058652879999999998 2024-09-17T21:19:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:19:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:19:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:19:57.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854880+0000", 2024-09-17T21:19:57.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010986500000000001 2024-09-17T21:19:57.815 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.815 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:57.815 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.815 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:19:57.815 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:19:57.815 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:57.817 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:19:57.817 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:57.813+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:57.918 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:57.913+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:57.948 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:57.948 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:57.948 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:19:57.948 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:57.962 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:19:58.019 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:58.015+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:58.083 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:58.083 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:58.084 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:58.084 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:58.084 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:58.097 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:19:58.139 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:58.135+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:58.221 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:58.215+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:58.265 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:58.265 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:58.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:58.265 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:58.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.265 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:19:58.265 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:58.265 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 121.24831780700001, 2024-09-17T21:19:58.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:19:58.266 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.266 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.266 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:58.266 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.266 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:19:58.266 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:19:58.266 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:19:58.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.266 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:58.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:19:58.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:19:58.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:19:58.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:19:58.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:19:58.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:19:58.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:19:58.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:19:58.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:19:58.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:19:58.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:19:58.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:58.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:19:58.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:19:58.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:19:58.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:19:58.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:19:58.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:19:58.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:19:58.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:19:58.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:19:58.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:58.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:19:58.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:19:58.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:58.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:19:58.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:19:58.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:58.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:19:58.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:19:58.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:19:58.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:19:58.272 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.272 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.272 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.272 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:58.272 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:58.272 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.668153527, 2024-09-17T21:19:58.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:19:58.272 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.272 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.272 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.272 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.273 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:19:58.273 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:19:58.273 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:19:58.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.273 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:58.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:19:58.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:19:58.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:19:58.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:19:58.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:19:58.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:19:58.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:19:58.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:19:58.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:19:58.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:19:58.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:19:58.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:58.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:19:58.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:19:58.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:19:58.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:19:58.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:58.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:19:58.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:19:58.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:19:58.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:19:58.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:19:58.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:19:58.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:19:58.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:19:58.277 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.277 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.278 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.278 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:58.278 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:58.278 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.66805621500001, 2024-09-17T21:19:58.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:19:58.278 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.278 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.278 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:58.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:58.278 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:19:58.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.279 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:58.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:19:58.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:19:58.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:19:58.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:19:58.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:19:58.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:19:58.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:19:58.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:19:58.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:19:58.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:19:58.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:19:58.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:58.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:19:58.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:19:58.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:19:58.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:19:58.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:58.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:19:58.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:19:58.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:19:58.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:19:58.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:19:58.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:19:58.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:19:58.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:19:58.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:19:58.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:19:58.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:19:58.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:19:58.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:19:58.284 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.284 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.284 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.284 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:19:58.284 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:58.284 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.665969414, 2024-09-17T21:19:58.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:19:58.285 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.285 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.285 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:58.285 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.285 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:19:58.285 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:19:58.285 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:19:58.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.285 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:58.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:19:58.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:19:58.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:19:58.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:19:58.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:19:58.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:19:58.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:19:58.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:19:58.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:19:58.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:19:58.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:19:58.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:19:58.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:19:58.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:19:58.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:19:58.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:19:58.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:58.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:19:58.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:19:58.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:19:58.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:19:58.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:19:58.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:19:58.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:58.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:19:58.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:19:58.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:58.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:19:58.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:19:58.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:58.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:19:58.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:19:58.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:58.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:19:58.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:19:58.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:19:58.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:19:58.291 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.291 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.291 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.292 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:58.292 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:58.292 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.982070633000006, 2024-09-17T21:19:58.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:19:58.292 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.292 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.292 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.292 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.292 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:58.292 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:58.292 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:19:58.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.292 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:58.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:19:58.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:19:58.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:19:58.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:19:58.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:19:58.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:19:58.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:19:58.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:19:58.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:19:58.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:19:58.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:19:58.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:19:58.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:19:58.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:19:58.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:19:58.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:19:58.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:58.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:19:58.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:19:58.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:19:58.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:19:58.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:19:58.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:19:58.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:19:58.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:19:58.297 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.297 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.297 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.297 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:58.298 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:58.298 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.972978921999996, 2024-09-17T21:19:58.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:19:58.298 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.298 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.298 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.298 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.298 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:58.298 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:58.298 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:19:58.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.298 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:58.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:19:58.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:19:58.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:19:58.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:19:58.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:19:58.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:19:58.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:19:58.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:19:58.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:19:58.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:19:58.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:19:58.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:58.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:19:58.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:19:58.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:19:58.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:19:58.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:19:58.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:19:58.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:19:58.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:19:58.302 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.302 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.302 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.303 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:58.303 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:58.303 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.972806871000003, 2024-09-17T21:19:58.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:19:58.303 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.303 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.303 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.303 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.303 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:19:58.303 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:19:58.303 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:19:58.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.303 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:58.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:19:58.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:19:58.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:19:58.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:19:58.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:19:58.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:19:58.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:19:58.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:19:58.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:19:58.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:19:58.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:19:58.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:58.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:19:58.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:19:58.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:19:58.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:19:58.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:19:58.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:19:58.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:19:58.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:19:58.307 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.307 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.307 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.308 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:58.308 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:58.308 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.797874124000003, 2024-09-17T21:19:58.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:19:58.308 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.308 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.308 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.308 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.308 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:58.308 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:58.308 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:19:58.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.308 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:58.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:19:58.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:19:58.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:19:58.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:19:58.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:19:58.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:19:58.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:19:58.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:19:58.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:19:58.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:19:58.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:19:58.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:58.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:19:58.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:19:58.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:19:58.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:19:58.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:19:58.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:19:58.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:19:58.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:19:58.312 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.312 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.312 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.313 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:58.313 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:58.313 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.797789566999995, 2024-09-17T21:19:58.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:19:58.313 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.313 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.313 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.313 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.313 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:58.313 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:58.313 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:19:58.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.314 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:58.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:58.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:19:58.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:19:58.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:19:58.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:19:58.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:19:58.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:19:58.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:19:58.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:19:58.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:19:58.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:19:58.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:58.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:19:58.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:19:58.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:19:58.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:19:58.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:19:58.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:19:58.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:19:58.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:19:58.317 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.317 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.317 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.318 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:19:58.318 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:58.318 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.797723930000004, 2024-09-17T21:19:58.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:19:58.318 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.318 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.318 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.318 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.318 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:19:58.318 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:19:58.318 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:19:58.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.318 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:58.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:19:58.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:19:58.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:19:58.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:19:58.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:19:58.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:19:58.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:19:58.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:19:58.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:19:58.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:19:58.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:19:58.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:19:58.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:19:58.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:19:58.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:19:58.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:19:58.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:19:58.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:19:58.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:19:58.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:19:58.322 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.322 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.322 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.323 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:58.323 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:19:58.323 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.424148833, 2024-09-17T21:19:58.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:19:58.323 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.323 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.323 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.323 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.323 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:58.323 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:58.323 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:19:58.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.323 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:19:58.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:19:58.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:19:58.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:19:58.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:19:58.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:19:58.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:19:58.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:19:58.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:19:58.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:19:58.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:19:58.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:19:58.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:19:58.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:19:58.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:19:58.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:19:58.327 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.327 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.327 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.327 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:58.327 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:19:58.327 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.424029320999999, 2024-09-17T21:19:58.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:19:58.327 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.327 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.327 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.327 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.328 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:58.328 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:58.328 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:19:58.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.328 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:19:58.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:19:58.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:19:58.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:19:58.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:19:58.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:19:58.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:19:58.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:19:58.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:19:58.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:19:58.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:19:58.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:19:58.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:19:58.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:19:58.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:19:58.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:19:58.331 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.331 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.331 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.332 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2695 3.15 3:aedfd032:::rbd_data.1152a8ae2019.0000000000000041:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:58.332 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.843199+0000", 2024-09-17T21:19:58.332 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.418884017, 2024-09-17T21:19:58.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079730569000000001, 2024-09-17T21:19:58.332 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.332 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.332 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:58.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:58.332 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2695 2024-09-17T21:19:58.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.332 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843199+0000", 2024-09-17T21:19:58.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843199+0000", 2024-09-17T21:19:58.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843201+0000", 2024-09-17T21:19:58.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0899999999999999e-06 2024-09-17T21:19:58.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843214+0000", 2024-09-17T21:19:58.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2233e-05 2024-09-17T21:19:58.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843245+0000", 2024-09-17T21:19:58.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1464000000000002e-05 2024-09-17T21:19:58.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843266+0000", 2024-09-17T21:19:58.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0879000000000001e-05 2024-09-17T21:19:58.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847920+0000", 2024-09-17T21:19:58.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.004654292 2024-09-17T21:19:58.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.848240+0000", 2024-09-17T21:19:58.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031994499999999997 2024-09-17T21:19:58.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922930+0000", 2024-09-17T21:19:58.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.074689666000000002 2024-09-17T21:19:58.336 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.336 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.336 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.336 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2732 3.15 3:ab9a4731:::rbd_data.1152a8ae2019.0000000000000058:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:58.336 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.846909+0000", 2024-09-17T21:19:58.336 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.415173971000002, 2024-09-17T21:19:58.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076145476000000004, 2024-09-17T21:19:58.336 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.336 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.336 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.337 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.337 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:58.337 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:58.337 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2732 2024-09-17T21:19:58.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.337 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846909+0000", 2024-09-17T21:19:58.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846909+0000", 2024-09-17T21:19:58.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846911+0000", 2024-09-17T21:19:58.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.31e-06 2024-09-17T21:19:58.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846916+0000", 2024-09-17T21:19:58.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.023e-06 2024-09-17T21:19:58.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846931+0000", 2024-09-17T21:19:58.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5549999999999999e-05 2024-09-17T21:19:58.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846943+0000", 2024-09-17T21:19:58.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1783000000000001e-05 2024-09-17T21:19:58.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.850592+0000", 2024-09-17T21:19:58.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036486159999999999 2024-09-17T21:19:58.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.850870+0000", 2024-09-17T21:19:58.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027864200000000001 2024-09-17T21:19:58.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.923055+0000", 2024-09-17T21:19:58.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.072184551999999999 2024-09-17T21:19:58.340 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.340 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.340 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.340 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2737 3.15 3:a8c4e696:::rbd_data.1152a8ae2019.000000000000005b:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:58.340 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.847014+0000", 2024-09-17T21:19:58.340 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.415068836, 2024-09-17T21:19:58.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076158271, 2024-09-17T21:19:58.341 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.341 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.341 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.341 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.341 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:58.341 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:58.341 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2737 2024-09-17T21:19:58.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.341 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847014+0000", 2024-09-17T21:19:58.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847014+0000", 2024-09-17T21:19:58.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847015+0000", 2024-09-17T21:19:58.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0699999999999999e-06 2024-09-17T21:19:58.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847021+0000", 2024-09-17T21:19:58.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.446e-06 2024-09-17T21:19:58.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847038+0000", 2024-09-17T21:19:58.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7257000000000001e-05 2024-09-17T21:19:58.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847051+0000", 2024-09-17T21:19:58.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2649999999999999e-05 2024-09-17T21:19:58.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.852127+0000", 2024-09-17T21:19:58.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050757420000000003 2024-09-17T21:19:58.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.852423+0000", 2024-09-17T21:19:58.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029608100000000001 2024-09-17T21:19:58.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.923173+0000", 2024-09-17T21:19:58.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.070750024999999994 2024-09-17T21:19:58.344 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.344 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.344 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.345 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3397 3.8 3:174f9d43:::rbd_data.1152a8ae2019.000000000000006e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:58.345 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.897797+0000", 2024-09-17T21:19:58.345 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.364285891000002, 2024-09-17T21:19:58.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045478192000000001, 2024-09-17T21:19:58.345 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.345 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.345 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.345 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.345 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:58.345 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:58.345 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3397 2024-09-17T21:19:58.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.345 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897797+0000", 2024-09-17T21:19:58.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897797+0000", 2024-09-17T21:19:58.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897799+0000", 2024-09-17T21:19:58.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.238e-06 2024-09-17T21:19:58.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897806+0000", 2024-09-17T21:19:58.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5900000000000002e-06 2024-09-17T21:19:58.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897822+0000", 2024-09-17T21:19:58.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5659e-05 2024-09-17T21:19:58.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897832+0000", 2024-09-17T21:19:58.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0034000000000001e-05 2024-09-17T21:19:58.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.931249+0000", 2024-09-17T21:19:58.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033416700000000001 2024-09-17T21:19:58.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.931425+0000", 2024-09-17T21:19:58.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017629500000000001 2024-09-17T21:19:58.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943276+0000", 2024-09-17T21:19:58.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011850675999999999 2024-09-17T21:19:58.349 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.349 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.349 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.349 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3524 3.8 3:119ba024:::rbd_data.1152c6ce51b2.0000000000000005:17 [read 1261568~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:58.349 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.901789+0000", 2024-09-17T21:19:58.349 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.360293964, 2024-09-17T21:19:58.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.042862082000000003, 2024-09-17T21:19:58.349 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.349 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.350 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.350 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.350 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:58.350 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:58.350 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3524 2024-09-17T21:19:58.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.350 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901789+0000", 2024-09-17T21:19:58.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901789+0000", 2024-09-17T21:19:58.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901790+0000", 2024-09-17T21:19:58.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0400000000000005e-07 2024-09-17T21:19:58.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901795+0000", 2024-09-17T21:19:58.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3839999999999999e-06 2024-09-17T21:19:58.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901807+0000", 2024-09-17T21:19:58.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2532e-05 2024-09-17T21:19:58.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901816+0000", 2024-09-17T21:19:58.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2220000000000008e-06 2024-09-17T21:19:58.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.944407+0000", 2024-09-17T21:19:58.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.042591088999999999 2024-09-17T21:19:58.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.944498+0000", 2024-09-17T21:19:58.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0883000000000004e-05 2024-09-17T21:19:58.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.944651+0000", 2024-09-17T21:19:58.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015306799999999999 2024-09-17T21:19:58.353 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.353 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.353 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.353 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3679 3.8 3:129b2dec:::rbd_data.1152a8ae2019.0000000000000012:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:58.353 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.908354+0000", 2024-09-17T21:19:58.354 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.353729603000001, 2024-09-17T21:19:58.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046195880000000002, 2024-09-17T21:19:58.354 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.354 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.354 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:58.354 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.354 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:58.354 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:58.354 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3679 2024-09-17T21:19:58.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.354 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908354+0000", 2024-09-17T21:19:58.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908354+0000", 2024-09-17T21:19:58.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908355+0000", 2024-09-17T21:19:58.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9999999999999995e-07 2024-09-17T21:19:58.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908361+0000", 2024-09-17T21:19:58.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7399999999999998e-06 2024-09-17T21:19:58.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908375+0000", 2024-09-17T21:19:58.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.363e-05 2024-09-17T21:19:58.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908386+0000", 2024-09-17T21:19:58.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.06e-05 2024-09-17T21:19:58.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.945600+0000", 2024-09-17T21:19:58.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.037214199000000003 2024-09-17T21:19:58.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.945650+0000", 2024-09-17T21:19:58.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9737999999999997e-05 2024-09-17T21:19:58.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:58.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.945799+0000", 2024-09-17T21:19:58.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014921699999999999 2024-09-17T21:19:58.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:58.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954371+0000", 2024-09-17T21:19:58.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085719730000000001 2024-09-17T21:19:58.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:58.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954460+0000", 2024-09-17T21:19:58.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9596000000000001e-05 2024-09-17T21:19:58.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:58.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954511+0000", 2024-09-17T21:19:58.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0455999999999999e-05 2024-09-17T21:19:58.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954550+0000", 2024-09-17T21:19:58.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8730999999999999e-05 2024-09-17T21:19:58.359 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.359 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.359 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.359 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3703 3.13 3:cdb4d88a:::rbd_data.1152a8ae2019.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:58.359 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.908402+0000", 2024-09-17T21:19:58.359 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.353681562999999, 2024-09-17T21:19:58.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046393446999999997, 2024-09-17T21:19:58.359 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.359 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.359 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:58.359 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.360 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:58.360 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:58.360 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3703 2024-09-17T21:19:58.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.360 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908402+0000", 2024-09-17T21:19:58.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908402+0000", 2024-09-17T21:19:58.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908403+0000", 2024-09-17T21:19:58.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0529999999999999e-06 2024-09-17T21:19:58.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908409+0000", 2024-09-17T21:19:58.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2219999999999999e-06 2024-09-17T21:19:58.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908422+0000", 2024-09-17T21:19:58.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3147e-05 2024-09-17T21:19:58.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908432+0000", 2024-09-17T21:19:58.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.039e-05 2024-09-17T21:19:58.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.942217+0000", 2024-09-17T21:19:58.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033784994999999998 2024-09-17T21:19:58.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.942296+0000", 2024-09-17T21:19:58.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8218000000000002e-05 2024-09-17T21:19:58.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:58.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943003+0000", 2024-09-17T21:19:58.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000707403 2024-09-17T21:19:58.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:58.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954635+0000", 2024-09-17T21:19:58.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011631786999999999 2024-09-17T21:19:58.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:58.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954706+0000", 2024-09-17T21:19:58.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1064999999999998e-05 2024-09-17T21:19:58.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:58.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954757+0000", 2024-09-17T21:19:58.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0760000000000002e-05 2024-09-17T21:19:58.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954795+0000", 2024-09-17T21:19:58.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8407e-05 2024-09-17T21:19:58.365 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.365 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.365 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.365 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3711 3.13 3:ca5a0e5e:::rbd_data.1152a8ae2019.0000000000000040:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:19:58.365 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.908514+0000", 2024-09-17T21:19:58.365 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.353569400000001, 2024-09-17T21:19:58.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046370871000000001, 2024-09-17T21:19:58.366 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.366 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.366 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:58.366 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.366 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:19:58.366 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:19:58.366 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3711 2024-09-17T21:19:58.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.366 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908514+0000", 2024-09-17T21:19:58.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908514+0000", 2024-09-17T21:19:58.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908515+0000", 2024-09-17T21:19:58.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0449999999999999e-06 2024-09-17T21:19:58.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908521+0000", 2024-09-17T21:19:58.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2179999999999996e-06 2024-09-17T21:19:58.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908535+0000", 2024-09-17T21:19:58.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4124999999999999e-05 2024-09-17T21:19:58.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908548+0000", 2024-09-17T21:19:58.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2680000000000001e-05 2024-09-17T21:19:58.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943530+0000", 2024-09-17T21:19:58.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034981655 2024-09-17T21:19:58.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943636+0000", 2024-09-17T21:19:58.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106086 2024-09-17T21:19:58.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:19:58.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943837+0000", 2024-09-17T21:19:58.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201826 2024-09-17T21:19:58.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:58.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954688+0000", 2024-09-17T21:19:58.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010850963 2024-09-17T21:19:58.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:58.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954805+0000", 2024-09-17T21:19:58.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000117025 2024-09-17T21:19:58.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:58.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954851+0000", 2024-09-17T21:19:58.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5065000000000003e-05 2024-09-17T21:19:58.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954885+0000", 2024-09-17T21:19:58.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4183000000000001e-05 2024-09-17T21:19:58.371 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.371 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.371 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.371 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.371 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.371 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:58.372 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone26 from 0x7344e00 (overlap 0x2f01e00) to 0x0 2024-09-17T21:19:58.373 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:19:58.373 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:58.365+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:58.473 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:58.469+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:58.551 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:58.551 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:58.551 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:19:58.551 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:58.565 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:19:58.574 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:58.569+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:58.675 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:58.671+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:58.690 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:58.690 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:58.690 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:58.690 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:58.690 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:58.705 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:19:58.775 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:58.769+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:58.873 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-17T21:19:58.876 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:19:58.871+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:19:58.880 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:58.880 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:58.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:58.880 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:58.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.880 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5541.0:2643 3.1a 3:5ab2b679:::rbd_data.115266d0b9f2.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 917504~4608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:19:58.880 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.510099+0000", 2024-09-17T21:19:58.880 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.3655856249999996, 2024-09-17T21:19:58.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.053163717999999999, 2024-09-17T21:19:58.880 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.880 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.881 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:58.881 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.881 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5541", 2024-09-17T21:19:58.881 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:19:58.881 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2643 2024-09-17T21:19:58.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.881 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.510099+0000", 2024-09-17T21:19:58.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.510099+0000", 2024-09-17T21:19:58.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.510100+0000", 2024-09-17T21:19:58.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.668e-06 2024-09-17T21:19:58.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527298+0000", 2024-09-17T21:19:58.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017198108 2024-09-17T21:19:58.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527333+0000", 2024-09-17T21:19:58.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4304999999999998e-05 2024-09-17T21:19:58.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527414+0000", 2024-09-17T21:19:58.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1009999999999999e-05 2024-09-17T21:19:58.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527529+0000", 2024-09-17T21:19:58.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000115122 2024-09-17T21:19:58.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527653+0000", 2024-09-17T21:19:58.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012405900000000001 2024-09-17T21:19:58.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:19:58.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527950+0000", 2024-09-17T21:19:58.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029670599999999999 2024-09-17T21:19:58.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:58.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.553334+0000", 2024-09-17T21:19:58.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025384750000000001 2024-09-17T21:19:58.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:58.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.563128+0000", 2024-09-17T21:19:58.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0097937179999999999 2024-09-17T21:19:58.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:58.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.563201+0000", 2024-09-17T21:19:58.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3128999999999997e-05 2024-09-17T21:19:58.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.886 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.563262+0000", 2024-09-17T21:19:58.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1142999999999994e-05 2024-09-17T21:19:58.886 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.886 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.886 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.886 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5541.0:3004 3.a 3:524d69f6:::rbd_data.115266d0b9f2.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T21:19:58.886 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:50.606531+0000", 2024-09-17T21:19:58.886 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.2691536729999999, 2024-09-17T21:19:58.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049461896999999998, 2024-09-17T21:19:58.886 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.886 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.887 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:58.887 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.887 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5541", 2024-09-17T21:19:58.887 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:19:58.887 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3004 2024-09-17T21:19:58.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.887 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.606531+0000", 2024-09-17T21:19:58.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.606531+0000", 2024-09-17T21:19:58.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.888 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.606533+0000", 2024-09-17T21:19:58.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.03e-06 2024-09-17T21:19:58.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.888 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617600+0000", 2024-09-17T21:19:58.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011067169999999999 2024-09-17T21:19:58.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617620+0000", 2024-09-17T21:19:58.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0627000000000001e-05 2024-09-17T21:19:58.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617631+0000", 2024-09-17T21:19:58.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0991e-05 2024-09-17T21:19:58.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617713+0000", 2024-09-17T21:19:58.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1241000000000004e-05 2024-09-17T21:19:58.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617903+0000", 2024-09-17T21:19:58.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019016599999999999 2024-09-17T21:19:58.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 3", 2024-09-17T21:19:58.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.618842+0000", 2024-09-17T21:19:58.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00093906399999999998 2024-09-17T21:19:58.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:58.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.654887+0000", 2024-09-17T21:19:58.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.036045176999999998 2024-09-17T21:19:58.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:58.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.655762+0000", 2024-09-17T21:19:58.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00087534500000000003 2024-09-17T21:19:58.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:58.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.655900+0000", 2024-09-17T21:19:58.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013781700000000001 2024-09-17T21:19:58.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.655992+0000", 2024-09-17T21:19:58.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2268999999999994e-05 2024-09-17T21:19:58.891 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.892 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.892 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.892 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1489 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:19:58.892 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.014521+0000", 2024-09-17T21:19:58.892 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8611629189999999, 2024-09-17T21:19:58.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047072140999999998, 2024-09-17T21:19:58.892 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.892 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.892 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.892 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.892 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:19:58.892 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:19:58.893 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1489 2024-09-17T21:19:58.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.893 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.893 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014521+0000", 2024-09-17T21:19:58.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.893 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014521+0000", 2024-09-17T21:19:58.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014522+0000", 2024-09-17T21:19:58.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8199999999999999e-07 2024-09-17T21:19:58.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014525+0000", 2024-09-17T21:19:58.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2799999999999999e-06 2024-09-17T21:19:58.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014536+0000", 2024-09-17T21:19:58.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0463e-05 2024-09-17T21:19:58.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014544+0000", 2024-09-17T21:19:58.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:19:58.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.061185+0000", 2024-09-17T21:19:58.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046641021999999997 2024-09-17T21:19:58.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.061263+0000", 2024-09-17T21:19:58.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7973999999999996e-05 2024-09-17T21:19:58.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.061593+0000", 2024-09-17T21:19:58.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033020199999999998 2024-09-17T21:19:58.896 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.896 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.896 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.896 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1521 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 638976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:19:58.896 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.018585+0000", 2024-09-17T21:19:58.897 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8570992410000002, 2024-09-17T21:19:58.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046751124999999998, 2024-09-17T21:19:58.897 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.897 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.897 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.897 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.897 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:19:58.897 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:19:58.897 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1521 2024-09-17T21:19:58.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.897 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.897 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.897 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018585+0000", 2024-09-17T21:19:58.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018585+0000", 2024-09-17T21:19:58.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018586+0000", 2024-09-17T21:19:58.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7999999999999993e-07 2024-09-17T21:19:58.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018590+0000", 2024-09-17T21:19:58.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2719999999999999e-06 2024-09-17T21:19:58.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018605+0000", 2024-09-17T21:19:58.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4637000000000001e-05 2024-09-17T21:19:58.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018615+0000", 2024-09-17T21:19:58.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.048e-05 2024-09-17T21:19:58.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065019+0000", 2024-09-17T21:19:58.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046403563000000002 2024-09-17T21:19:58.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065073+0000", 2024-09-17T21:19:58.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4521000000000002e-05 2024-09-17T21:19:58.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065336+0000", 2024-09-17T21:19:58.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026267199999999998 2024-09-17T21:19:58.900 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.900 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.900 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.901 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1522 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:19:58.901 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.018637+0000", 2024-09-17T21:19:58.901 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8570473930000002, 2024-09-17T21:19:58.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047019501999999998, 2024-09-17T21:19:58.901 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.901 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.901 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.901 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.901 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:19:58.901 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:19:58.901 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1522 2024-09-17T21:19:58.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.902 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.902 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018637+0000", 2024-09-17T21:19:58.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.902 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018637+0000", 2024-09-17T21:19:58.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018638+0000", 2024-09-17T21:19:58.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3570000000000001e-06 2024-09-17T21:19:58.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018643+0000", 2024-09-17T21:19:58.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7400000000000004e-06 2024-09-17T21:19:58.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018657+0000", 2024-09-17T21:19:58.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3617e-05 2024-09-17T21:19:58.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018667+0000", 2024-09-17T21:19:58.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0392e-05 2024-09-17T21:19:58.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065370+0000", 2024-09-17T21:19:58.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046703100999999997 2024-09-17T21:19:58.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065427+0000", 2024-09-17T21:19:58.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6920999999999999e-05 2024-09-17T21:19:58.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065656+0000", 2024-09-17T21:19:58.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022937399999999999 2024-09-17T21:19:58.905 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.905 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.905 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.905 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1523 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 704512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:19:58.905 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.018682+0000", 2024-09-17T21:19:58.905 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8570019950000001, 2024-09-17T21:19:58.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047547589000000001, 2024-09-17T21:19:58.905 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.905 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.906 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.906 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.906 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:19:58.906 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:19:58.906 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1523 2024-09-17T21:19:58.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.906 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018682+0000", 2024-09-17T21:19:58.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018682+0000", 2024-09-17T21:19:58.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018683+0000", 2024-09-17T21:19:58.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1400000000000001e-06 2024-09-17T21:19:58.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018688+0000", 2024-09-17T21:19:58.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4900000000000002e-06 2024-09-17T21:19:58.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018703+0000", 2024-09-17T21:19:58.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4837e-05 2024-09-17T21:19:58.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018714+0000", 2024-09-17T21:19:58.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0882e-05 2024-09-17T21:19:58.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065769+0000", 2024-09-17T21:19:58.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047055579 2024-09-17T21:19:58.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065910+0000", 2024-09-17T21:19:58.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014135100000000001 2024-09-17T21:19:58.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.909 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.909 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.066230+0000", 2024-09-17T21:19:58.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031931 2024-09-17T21:19:58.909 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.909 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.909 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.909 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1552 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:19:58.909 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022280+0000", 2024-09-17T21:19:58.909 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8534044549999997, 2024-09-17T21:19:58.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047789951999999997, 2024-09-17T21:19:58.909 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.910 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.910 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.910 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.910 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:19:58.910 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:19:58.910 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1552 2024-09-17T21:19:58.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.910 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.910 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.910 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022280+0000", 2024-09-17T21:19:58.910 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.911 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022280+0000", 2024-09-17T21:19:58.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.911 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022280+0000", 2024-09-17T21:19:58.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:19:58.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.911 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022283+0000", 2024-09-17T21:19:58.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4080000000000001e-06 2024-09-17T21:19:58.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022289+0000", 2024-09-17T21:19:58.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.477e-06 2024-09-17T21:19:58.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022295+0000", 2024-09-17T21:19:58.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5439999999999998e-06 2024-09-17T21:19:58.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.069004+0000", 2024-09-17T21:19:58.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046709222000000002 2024-09-17T21:19:58.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.913 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.069152+0000", 2024-09-17T21:19:58.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014832300000000001 2024-09-17T21:19:58.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.913 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.070070+0000", 2024-09-17T21:19:58.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00091743300000000005 2024-09-17T21:19:58.913 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.913 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.913 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.914 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.914 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1553 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:19:58.914 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022302+0000", 2024-09-17T21:19:58.914 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8533820680000002, 2024-09-17T21:19:58.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048707517999999998, 2024-09-17T21:19:58.914 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.914 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.914 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.914 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.914 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:19:58.914 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:19:58.914 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1553 2024-09-17T21:19:58.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.915 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.915 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.915 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022302+0000", 2024-09-17T21:19:58.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.915 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.915 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022302+0000", 2024-09-17T21:19:58.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.915 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.915 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022303+0000", 2024-09-17T21:19:58.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2300000000000001e-07 2024-09-17T21:19:58.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022305+0000", 2024-09-17T21:19:58.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4700000000000001e-06 2024-09-17T21:19:58.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022312+0000", 2024-09-17T21:19:58.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.494e-06 2024-09-17T21:19:58.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022320+0000", 2024-09-17T21:19:58.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1969999999999996e-06 2024-09-17T21:19:58.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.917 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.070138+0000", 2024-09-17T21:19:58.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047818685 2024-09-17T21:19:58.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.917 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.070323+0000", 2024-09-17T21:19:58.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000184186 2024-09-17T21:19:58.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.917 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071010+0000", 2024-09-17T21:19:58.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00068696299999999996 2024-09-17T21:19:58.918 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.918 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.918 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.918 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1554 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:19:58.918 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022328+0000", 2024-09-17T21:19:58.918 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8533562259999998, 2024-09-17T21:19:58.918 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049212936999999998, 2024-09-17T21:19:58.918 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.918 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.918 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.918 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.919 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:19:58.919 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:19:58.919 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1554 2024-09-17T21:19:58.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.919 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.919 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022328+0000", 2024-09-17T21:19:58.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.919 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022328+0000", 2024-09-17T21:19:58.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022328+0000", 2024-09-17T21:19:58.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3000000000000001e-07 2024-09-17T21:19:58.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022331+0000", 2024-09-17T21:19:58.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3539999999999998e-06 2024-09-17T21:19:58.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022338+0000", 2024-09-17T21:19:58.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8800000000000002e-06 2024-09-17T21:19:58.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022343+0000", 2024-09-17T21:19:58.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6149999999999996e-06 2024-09-17T21:19:58.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071051+0000", 2024-09-17T21:19:58.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048707179000000003 2024-09-17T21:19:58.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071171+0000", 2024-09-17T21:19:58.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000120036 2024-09-17T21:19:58.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071541+0000", 2024-09-17T21:19:58.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037034299999999997 2024-09-17T21:19:58.922 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.922 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.922 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.922 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1555 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:19:58.922 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022352+0000", 2024-09-17T21:19:58.923 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8533320599999996, 2024-09-17T21:19:58.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049642617, 2024-09-17T21:19:58.923 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.923 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.923 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.923 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.923 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:19:58.923 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:19:58.923 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1555 2024-09-17T21:19:58.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.923 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.923 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.923 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022352+0000", 2024-09-17T21:19:58.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022352+0000", 2024-09-17T21:19:58.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022353+0000", 2024-09-17T21:19:58.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7000000000000005e-07 2024-09-17T21:19:58.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022355+0000", 2024-09-17T21:19:58.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4569999999999999e-06 2024-09-17T21:19:58.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.925 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022364+0000", 2024-09-17T21:19:58.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2550000000000003e-06 2024-09-17T21:19:58.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.925 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022374+0000", 2024-09-17T21:19:58.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2820000000000002e-06 2024-09-17T21:19:58.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071626+0000", 2024-09-17T21:19:58.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049251892999999998 2024-09-17T21:19:58.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071703+0000", 2024-09-17T21:19:58.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7273000000000004e-05 2024-09-17T21:19:58.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071995+0000", 2024-09-17T21:19:58.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029188700000000001 2024-09-17T21:19:58.926 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.927 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.927 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.927 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1556 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 663552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:19:58.927 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022388+0000", 2024-09-17T21:19:58.927 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8532965160000003, 2024-09-17T21:19:58.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050001666, 2024-09-17T21:19:58.927 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.927 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.927 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.927 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.927 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:19:58.928 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:19:58.928 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1556 2024-09-17T21:19:58.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.928 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022388+0000", 2024-09-17T21:19:58.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022388+0000", 2024-09-17T21:19:58.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022389+0000", 2024-09-17T21:19:58.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.282e-06 2024-09-17T21:19:58.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022394+0000", 2024-09-17T21:19:58.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6779999999999999e-06 2024-09-17T21:19:58.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022407+0000", 2024-09-17T21:19:58.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3309000000000001e-05 2024-09-17T21:19:58.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.930 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.930 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022417+0000", 2024-09-17T21:19:58.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0152e-05 2024-09-17T21:19:58.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.930 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.930 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072033+0000", 2024-09-17T21:19:58.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049616014999999999 2024-09-17T21:19:58.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.930 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072137+0000", 2024-09-17T21:19:58.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104214 2024-09-17T21:19:58.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072389+0000", 2024-09-17T21:19:58.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025201599999999998 2024-09-17T21:19:58.931 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.931 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.931 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.931 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1557 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:19:58.931 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022434+0000", 2024-09-17T21:19:58.932 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8532504630000002, 2024-09-17T21:19:58.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050377024999999999, 2024-09-17T21:19:58.932 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.932 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.932 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.932 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.932 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:19:58.932 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:19:58.932 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1557 2024-09-17T21:19:58.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.932 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.933 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022434+0000", 2024-09-17T21:19:58.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.933 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022434+0000", 2024-09-17T21:19:58.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.933 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022435+0000", 2024-09-17T21:19:58.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9700000000000005e-07 2024-09-17T21:19:58.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022439+0000", 2024-09-17T21:19:58.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1899999999999997e-06 2024-09-17T21:19:58.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022452+0000", 2024-09-17T21:19:58.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2865e-05 2024-09-17T21:19:58.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022462+0000", 2024-09-17T21:19:58.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.91e-06 2024-09-17T21:19:58.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072452+0000", 2024-09-17T21:19:58.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049990290999999999 2024-09-17T21:19:58.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072541+0000", 2024-09-17T21:19:58.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9297999999999998e-05 2024-09-17T21:19:58.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072811+0000", 2024-09-17T21:19:58.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026957400000000002 2024-09-17T21:19:58.936 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.936 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.936 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.936 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1584 3.e 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:1b [read 753664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:19:58.936 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.026146+0000", 2024-09-17T21:19:58.936 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8495382720000002, 2024-09-17T21:19:58.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049235534999999997, 2024-09-17T21:19:58.936 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.936 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.937 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.937 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.937 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:19:58.937 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:19:58.937 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1584 2024-09-17T21:19:58.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.937 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.937 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.937 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026146+0000", 2024-09-17T21:19:58.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026146+0000", 2024-09-17T21:19:58.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026147+0000", 2024-09-17T21:19:58.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0249999999999999e-06 2024-09-17T21:19:58.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026151+0000", 2024-09-17T21:19:58.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:19:58.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.939 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.939 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.939 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026169+0000", 2024-09-17T21:19:58.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7294000000000001e-05 2024-09-17T21:19:58.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.939 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.939 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.939 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026179+0000", 2024-09-17T21:19:58.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0084999999999999e-05 2024-09-17T21:19:58.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.939 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.939 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.939 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072874+0000", 2024-09-17T21:19:58.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046695497000000002 2024-09-17T21:19:58.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.940 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.940 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072976+0000", 2024-09-17T21:19:58.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101358 2024-09-17T21:19:58.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.940 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.940 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.075381+0000", 2024-09-17T21:19:58.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002405726 2024-09-17T21:19:58.940 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.940 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.940 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.941 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1585 3.e 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:1b [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:19:58.941 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.026195+0000", 2024-09-17T21:19:58.941 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8494891439999996, 2024-09-17T21:19:58.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051427650999999998, 2024-09-17T21:19:58.941 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.941 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.941 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.941 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.941 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:19:58.941 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:19:58.941 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1585 2024-09-17T21:19:58.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.941 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026195+0000", 2024-09-17T21:19:58.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026195+0000", 2024-09-17T21:19:58.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026196+0000", 2024-09-17T21:19:58.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7300000000000004e-07 2024-09-17T21:19:58.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026200+0000", 2024-09-17T21:19:58.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.425e-06 2024-09-17T21:19:58.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026214+0000", 2024-09-17T21:19:58.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3108999999999999e-05 2024-09-17T21:19:58.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026224+0000", 2024-09-17T21:19:58.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.042e-05 2024-09-17T21:19:58.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.944 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.944 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.075431+0000", 2024-09-17T21:19:58.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049206543999999998 2024-09-17T21:19:58.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.944 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.944 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.075489+0000", 2024-09-17T21:19:58.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8053000000000001e-05 2024-09-17T21:19:58.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.945 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.945 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.077623+0000", 2024-09-17T21:19:58.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002134127 2024-09-17T21:19:58.945 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.945 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.945 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.945 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.945 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1586 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:19:58.945 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.026238+0000", 2024-09-17T21:19:58.945 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8494463860000003, 2024-09-17T21:19:58.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051776152999999998, 2024-09-17T21:19:58.945 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.945 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.945 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.946 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.946 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:19:58.946 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:19:58.946 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1586 2024-09-17T21:19:58.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.946 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026238+0000", 2024-09-17T21:19:58.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026238+0000", 2024-09-17T21:19:58.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026239+0000", 2024-09-17T21:19:58.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0380000000000001e-06 2024-09-17T21:19:58.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026243+0000", 2024-09-17T21:19:58.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3499999999999999e-06 2024-09-17T21:19:58.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026259+0000", 2024-09-17T21:19:58.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5928999999999999e-05 2024-09-17T21:19:58.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026270+0000", 2024-09-17T21:19:58.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0702e-05 2024-09-17T21:19:58.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.077682+0000", 2024-09-17T21:19:58.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051411865000000001 2024-09-17T21:19:58.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.077751+0000", 2024-09-17T21:19:58.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9606999999999999e-05 2024-09-17T21:19:58.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078014+0000", 2024-09-17T21:19:58.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026266199999999999 2024-09-17T21:19:58.949 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.949 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.949 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.949 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1587 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 802816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:19:58.950 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.026286+0000", 2024-09-17T21:19:58.950 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8493986070000004, 2024-09-17T21:19:58.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.052384495000000003, 2024-09-17T21:19:58.950 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.950 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.950 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.950 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.950 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:19:58.950 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:19:58.950 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1587 2024-09-17T21:19:58.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.950 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.950 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.950 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026286+0000", 2024-09-17T21:19:58.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026286+0000", 2024-09-17T21:19:58.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026287+0000", 2024-09-17T21:19:58.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0049999999999999e-06 2024-09-17T21:19:58.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026291+0000", 2024-09-17T21:19:58.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3479999999999997e-06 2024-09-17T21:19:58.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026304+0000", 2024-09-17T21:19:58.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3288999999999999e-05 2024-09-17T21:19:58.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026314+0000", 2024-09-17T21:19:58.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9920000000000002e-06 2024-09-17T21:19:58.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.953 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.953 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078052+0000", 2024-09-17T21:19:58.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051737299 2024-09-17T21:19:58.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.953 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.953 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078118+0000", 2024-09-17T21:19:58.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6180999999999995e-05 2024-09-17T21:19:58.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.953 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.953 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078670+0000", 2024-09-17T21:19:58.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00055238100000000001 2024-09-17T21:19:58.954 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.954 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.954 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.954 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1618 3.e 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:1b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:19:58.954 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.031597+0000", 2024-09-17T21:19:58.954 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8440871659999996, 2024-09-17T21:19:58.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047545707999999999, 2024-09-17T21:19:58.954 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.954 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.954 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.954 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.955 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:19:58.955 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:19:58.955 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1618 2024-09-17T21:19:58.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.955 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.955 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.955 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031597+0000", 2024-09-17T21:19:58.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.955 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.955 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031597+0000", 2024-09-17T21:19:58.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.956 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.956 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031598+0000", 2024-09-17T21:19:58.956 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:19:58.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.956 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.956 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031602+0000", 2024-09-17T21:19:58.956 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:19:58.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.956 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.956 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031630+0000", 2024-09-17T21:19:58.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8058000000000001e-05 2024-09-17T21:19:58.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031643+0000", 2024-09-17T21:19:58.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2628e-05 2024-09-17T21:19:58.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078728+0000", 2024-09-17T21:19:58.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047085044 2024-09-17T21:19:58.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078830+0000", 2024-09-17T21:19:58.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102493 2024-09-17T21:19:58.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079143+0000", 2024-09-17T21:19:58.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031263599999999998 2024-09-17T21:19:58.958 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.958 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.958 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.958 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1619 3.e 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:1b [read 827392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:19:58.959 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.031661+0000", 2024-09-17T21:19:58.959 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8440235989999998, 2024-09-17T21:19:58.959 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047908566999999999, 2024-09-17T21:19:58.959 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.959 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.959 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.959 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.959 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:19:58.959 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:19:58.959 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1619 2024-09-17T21:19:58.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.959 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.960 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031661+0000", 2024-09-17T21:19:58.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.960 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.960 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031661+0000", 2024-09-17T21:19:58.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.960 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.960 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031662+0000", 2024-09-17T21:19:58.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.252e-06 2024-09-17T21:19:58.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031666+0000", 2024-09-17T21:19:58.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6299999999999997e-06 2024-09-17T21:19:58.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031683+0000", 2024-09-17T21:19:58.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6035000000000001e-05 2024-09-17T21:19:58.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031693+0000", 2024-09-17T21:19:58.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0492000000000001e-05 2024-09-17T21:19:58.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.962 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.962 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079186+0000", 2024-09-17T21:19:58.962 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047493441999999997 2024-09-17T21:19:58.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.962 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.962 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079315+0000", 2024-09-17T21:19:58.962 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012860000000000001 2024-09-17T21:19:58.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.962 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.962 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079569+0000", 2024-09-17T21:19:58.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025411599999999998 2024-09-17T21:19:58.963 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.963 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.963 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.963 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1620 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:19:58.963 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.031706+0000", 2024-09-17T21:19:58.963 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8439781750000002, 2024-09-17T21:19:58.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048561143000000001, 2024-09-17T21:19:58.963 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.963 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.963 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.963 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.963 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:19:58.964 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:19:58.964 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1620 2024-09-17T21:19:58.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.964 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031706+0000", 2024-09-17T21:19:58.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031706+0000", 2024-09-17T21:19:58.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.965 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.965 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031707+0000", 2024-09-17T21:19:58.965 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5799999999999998e-07 2024-09-17T21:19:58.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.965 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.965 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031711+0000", 2024-09-17T21:19:58.965 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1930000000000003e-06 2024-09-17T21:19:58.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.965 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.965 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031724+0000", 2024-09-17T21:19:58.965 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.24e-05 2024-09-17T21:19:58.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.966 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031733+0000", 2024-09-17T21:19:58.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6239999999999997e-06 2024-09-17T21:19:58.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.966 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079613+0000", 2024-09-17T21:19:58.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047880082999999997 2024-09-17T21:19:58.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.966 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079719+0000", 2024-09-17T21:19:58.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010559499999999999 2024-09-17T21:19:58.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.967 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.967 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.080267+0000", 2024-09-17T21:19:58.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00054829 2024-09-17T21:19:58.967 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.967 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.967 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.967 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1621 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:19:58.967 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.031746+0000", 2024-09-17T21:19:58.967 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8439377009999998, 2024-09-17T21:19:58.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048925189000000001, 2024-09-17T21:19:58.967 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:58.968 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:58.968 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:58.968 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:58.968 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:19:58.968 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:19:58.968 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1621 2024-09-17T21:19:58.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.968 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:58.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:58.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031746+0000", 2024-09-17T21:19:58.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:58.969 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031746+0000", 2024-09-17T21:19:58.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:58.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.969 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:58.969 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031747+0000", 2024-09-17T21:19:58.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3900000000000003e-07 2024-09-17T21:19:58.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.969 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:58.969 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031752+0000", 2024-09-17T21:19:58.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.172e-06 2024-09-17T21:19:58.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.969 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:58.970 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031767+0000", 2024-09-17T21:19:58.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5517000000000001e-05 2024-09-17T21:19:58.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.970 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:58.970 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031777+0000", 2024-09-17T21:19:58.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8579999999999995e-06 2024-09-17T21:19:58.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.970 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:58.970 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.080318+0000", 2024-09-17T21:19:58.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048541009000000003 2024-09-17T21:19:58.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:58.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.080415+0000", 2024-09-17T21:19:58.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7034999999999995e-05 2024-09-17T21:19:58.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:58.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:58.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:58.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.080672+0000", 2024-09-17T21:19:58.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025665900000000001 2024-09-17T21:19:58.971 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.971 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.971 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.971 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:58.971 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:58.971 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:58.972 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:19:58.977 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:58.973+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:59.078 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:59.073+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:59.166 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:19:59.166 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:59.179 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:19:59.173+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:19:59.184 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:19:59.206 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:19:59.280 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:19:59.275+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:19:59.318 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:59.318 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:19:59.318 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:19:59.318 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:19:59.319 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:59.335 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:19:59.380 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:59.373+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:59.481 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:19:59.477+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:19:59.509 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:19:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:19:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:19:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:19:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:19:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 97.901473030999995, 2024-09-17T21:19:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:19:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:19:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:19:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:19:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:19:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:19:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:19:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:19:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:19:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:19:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:19:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:19:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:19:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:19:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:19:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:19:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:19:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:19:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:19:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 97.891526395, 2024-09-17T21:19:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:19:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:19:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:19:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:19:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:19:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:19:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:19:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:19:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:19:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:19:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:19:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:19:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:19:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:19:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:19:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:19:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:19:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:19:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:19:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 96.648375555000001, 2024-09-17T21:19:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:19:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:19:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:19:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:19:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:19:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:19:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:19:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:19:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:19:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:19:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:19:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:19:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:19:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:19:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:19:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:19:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:19:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:19:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:19:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:19:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:19:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:19:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:19:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:19:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:19:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:19:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:19:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:19:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 96.638730069000005, 2024-09-17T21:19:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:19:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:19:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:19:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:19:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:19:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:19:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:19:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:19:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:19:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:19:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:19:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:19:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:19:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:19:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:19:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:19:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:19:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:19:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:19:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:19:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:19:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:19:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:19:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:19:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:19:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:19:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:19:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:19:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:19:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:19:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:19:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.269794035999993, 2024-09-17T21:19:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:19:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:19:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:19:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:19:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:19:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:19:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:19:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:19:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:19:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:19:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:19:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:19:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:19:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:19:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:19:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:19:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:19:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:19:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:19:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:819 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5111808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.068221746000006, 2024-09-17T21:19:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024344811000000001, 2024-09-17T21:19:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 819 2024-09-17T21:19:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:19:59.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:59.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437443+0000", 2024-09-17T21:19:59.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1969999999999999e-06 2024-09-17T21:19:59.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:59.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437448+0000", 2024-09-17T21:19:59.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9490000000000004e-06 2024-09-17T21:19:59.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:59.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437467+0000", 2024-09-17T21:19:59.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8862999999999999e-05 2024-09-17T21:19:59.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:59.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437480+0000", 2024-09-17T21:19:59.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2833000000000001e-05 2024-09-17T21:19:59.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:59.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461552+0000", 2024-09-17T21:19:59.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024072695000000002 2024-09-17T21:19:59.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:59.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461639+0000", 2024-09-17T21:19:59.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6996000000000005e-05 2024-09-17T21:19:59.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:59.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461787+0000", 2024-09-17T21:19:59.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014727800000000001 2024-09-17T21:19:59.540 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.540 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:59.540 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.541 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:59.541 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:59.541 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.068109473999996, 2024-09-17T21:19:59.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:19:59.541 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:59.541 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:59.541 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:59.541 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:59.541 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:59.541 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:59.541 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:19:59.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.541 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:59.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:59.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:59.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:59.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:19:59.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:59.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:19:59.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:19:59.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:59.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:19:59.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:19:59.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:59.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:19:59.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:19:59.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:59.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:19:59.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:19:59.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:59.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:19:59.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:19:59.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:59.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:19:59.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:19:59.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:59.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:19:59.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:19:59.545 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.545 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:59.545 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.545 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:833 3.5 3:a772dfb1:::rbd_data.11523651b015.0000000000000001:d [sparse-read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:59.545 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:59.545 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.067675187999996, 2024-09-17T21:19:59.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024216024999999999, 2024-09-17T21:19:59.545 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:59.545 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:59.545 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:59.546 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:59.546 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:59.546 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:59.546 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 833 2024-09-17T21:19:59.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.546 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:59.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:59.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:59.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:59.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:19:59.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:59.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437990+0000", 2024-09-17T21:19:59.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2500000000000001e-06 2024-09-17T21:19:59.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:59.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437995+0000", 2024-09-17T21:19:59.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1240000000000004e-06 2024-09-17T21:19:59.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:59.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438012+0000", 2024-09-17T21:19:59.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7572e-05 2024-09-17T21:19:59.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:59.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438031+0000", 2024-09-17T21:19:59.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8612000000000001e-05 2024-09-17T21:19:59.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:59.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461830+0000", 2024-09-17T21:19:59.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023799135999999999 2024-09-17T21:19:59.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:59.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462005+0000", 2024-09-17T21:19:59.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000174747 2024-09-17T21:19:59.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:59.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:19:59.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019958399999999999 2024-09-17T21:19:59.549 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.549 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:59.549 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.550 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:59.550 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:59.550 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.067614896999999, 2024-09-17T21:19:59.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:19:59.550 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:59.550 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:59.550 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:59.550 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:59.550 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:59.550 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:59.550 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:19:59.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.551 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:59.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:59.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:59.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:59.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:19:59.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:59.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:19:59.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:19:59.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:59.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:19:59.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:19:59.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:59.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:19:59.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:19:59.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:59.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:19:59.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:19:59.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.553 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:59.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:19:59.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:19:59.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.553 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:59.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:19:59.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:19:59.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:59.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:19:59.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:19:59.554 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.554 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:59.554 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.554 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:59.554 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:59.554 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.067559516000003, 2024-09-17T21:19:59.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:19:59.555 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:59.555 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:59.555 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:59.555 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:59.555 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:59.555 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:59.555 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:19:59.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.555 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:59.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:59.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:59.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:59.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:19:59.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:59.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:19:59.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:19:59.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:59.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:19:59.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:19:59.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.557 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:59.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:19:59.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:19:59.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.557 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:59.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:19:59.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:19:59.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.557 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:59.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:19:59.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:19:59.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:59.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:19:59.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:19:59.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:59.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:19:59.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:19:59.558 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.559 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:59.559 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.559 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:59.559 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:59.559 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.067503238, 2024-09-17T21:19:59.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:19:59.559 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:59.559 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:59.559 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:59.559 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:59.560 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:59.560 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:59.560 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:19:59.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.560 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:59.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:59.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:59.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:59.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:19:59.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:59.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:19:59.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:19:59.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:59.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:19:59.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:19:59.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.562 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:59.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:19:59.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:19:59.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.562 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:59.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:19:59.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:19:59.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.562 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:59.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:19:59.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:19:59.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:59.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:19:59.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:19:59.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:59.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:19:59.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:19:59.563 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.563 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:59.563 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.564 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:59.564 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:59.564 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.066553599000002, 2024-09-17T21:19:59.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:19:59.564 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:59.564 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:59.564 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:59.564 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:59.564 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:59.564 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:59.564 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:19:59.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.565 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:59.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:59.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:59.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:59.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:19:59.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:59.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:19:59.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:19:59.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:59.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:19:59.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:19:59.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:59.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:19:59.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:19:59.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:59.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:19:59.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:19:59.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:59.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:19:59.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:19:59.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:59.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:19:59.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:19:59.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:59.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:19:59.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:19:59.568 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.568 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:59.568 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.568 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:59.568 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:59.569 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.066474979000006, 2024-09-17T21:19:59.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:19:59.569 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:59.569 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:59.569 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:59.569 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:59.569 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:59.569 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:59.569 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:19:59.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.569 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:59.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:59.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:59.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:59.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:19:59.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:59.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:19:59.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:19:59.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:59.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:19:59.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:19:59.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:59.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:19:59.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:19:59.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:59.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:19:59.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:19:59.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:59.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:19:59.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:19:59.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:59.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:19:59.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:19:59.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:59.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:19:59.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:19:59.573 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.573 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:59.573 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.573 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:59.573 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:59.573 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.066362040000001, 2024-09-17T21:19:59.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:19:59.573 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:59.573 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:59.573 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:59.574 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:59.574 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:59.574 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:59.574 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:19:59.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.574 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:59.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:59.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:59.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:59.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:19:59.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:59.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:19:59.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:19:59.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:59.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:19:59.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:19:59.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:59.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:19:59.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:19:59.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:59.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:19:59.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:19:59.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:59.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:19:59.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:19:59.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:59.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:19:59.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:19:59.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:59.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:19:59.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:19:59.577 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.577 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:59.577 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.578 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:849 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5963776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:59.578 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:59.578 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.065768004000006, 2024-09-17T21:19:59.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024537514, 2024-09-17T21:19:59.578 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:59.578 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:59.578 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:59.578 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:59.578 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:59.578 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:59.578 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 849 2024-09-17T21:19:59.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.578 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:59.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:59.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:59.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:59.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:19:59.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:59.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439897+0000", 2024-09-17T21:19:59.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9099999999999999e-06 2024-09-17T21:19:59.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:59.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439905+0000", 2024-09-17T21:19:59.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0870000000000007e-06 2024-09-17T21:19:59.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:59.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439924+0000", 2024-09-17T21:19:59.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9042000000000002e-05 2024-09-17T21:19:59.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:59.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439941+0000", 2024-09-17T21:19:59.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.624e-05 2024-09-17T21:19:59.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:59.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464132+0000", 2024-09-17T21:19:59.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024191579000000001 2024-09-17T21:19:59.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:59.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464291+0000", 2024-09-17T21:19:59.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015904800000000001 2024-09-17T21:19:59.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:59.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464433+0000", 2024-09-17T21:19:59.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014160799999999999 2024-09-17T21:19:59.582 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.582 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:59.582 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.582 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:59.582 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:59.582 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.065700985999996, 2024-09-17T21:19:59.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:19:59.582 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:59.582 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:59.583 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:59.583 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:59.583 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:59.583 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:59.583 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:19:59.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.583 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:59.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:59.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:59.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:59.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:19:59.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:59.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:19:59.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:19:59.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:59.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:19:59.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:19:59.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:59.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:19:59.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:19:59.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:59.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:19:59.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:19:59.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:59.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:19:59.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:19:59.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.586 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:59.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:19:59.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:19:59.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.586 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:59.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:19:59.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:19:59.586 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.587 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:59.587 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.587 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:59.587 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:59.587 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.065636174999995, 2024-09-17T21:19:59.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:19:59.587 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:59.587 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:59.588 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:59.588 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:59.588 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:59.588 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:59.588 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:19:59.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.588 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:59.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:59.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:59.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:59.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:19:59.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:59.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:19:59.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:19:59.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:59.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:19:59.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:19:59.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:59.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:19:59.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:19:59.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:59.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:19:59.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:19:59.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:59.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:19:59.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:19:59.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:59.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:19:59.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:19:59.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:59.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:19:59.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:19:59.591 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.591 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:59.591 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.592 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:19:59.592 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:59.592 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.065575198000005, 2024-09-17T21:19:59.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:19:59.592 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:59.592 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:59.592 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:19:59.592 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:19:59.592 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:19:59.592 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:19:59.592 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:19:59.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.592 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:59.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:59.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:59.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:59.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:19:59.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:59.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:19:59.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:19:59.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:59.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:19:59.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:19:59.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:59.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:19:59.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:19:59.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:59.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:19:59.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:19:59.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:59.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:19:59.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:19:59.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:59.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:19:59.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:19:59.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:59.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:19:59.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:19:59.596 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.596 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:59.596 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.596 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:19:59.597 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:19:59.597 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.168004013000001, 2024-09-17T21:19:59.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:19:59.597 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:59.597 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:59.597 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:59.597 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:59.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:59.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:19:59.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:59.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:19:59.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:59.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:19:59.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:19:59.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:59.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:19:59.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:19:59.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:59.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:19:59.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:19:59.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:59.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:19:59.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:19:59.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:59.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:19:59.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:19:59.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:59.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:19:59.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:19:59.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:59.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:19:59.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:19:59.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:59.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:19:59.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:19:59.601 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.601 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:59.601 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.601 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:19:59.602 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:19:59.602 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.9593456190000005, 2024-09-17T21:19:59.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:19:59.602 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:19:59.602 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:19:59.602 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:19:59.602 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:19:59.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:19:59.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:19:59.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:19:59.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:19:59.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:19:59.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:19:59.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:19:59.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:19:59.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:19:59.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:19:59.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:19:59.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:19:59.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:19:59.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:19:59.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:19:59.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:19:59.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:19:59.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:19:59.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:19:59.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:19:59.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:19:59.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:19:59.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:19:59.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:19:59.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:19:59.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:19:59.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:19:59.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:19:59.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:19:59.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:19:59.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:19:59.606 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.606 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:59.606 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.606 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:19:59.606 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:19:59.606 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:19:59.606 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:19:59.608 INFO:teuthology.orchestra.run.smithi184.stdout:4109 trunc from 0x7344e00 to 0x4c09200 2024-09-17T21:19:59.609 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:59.603+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:59.609 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:19:59.639 INFO:teuthology.orchestra.run.smithi184.stdout:4113 read 0x237fa00 thru 0x238ae78 (0xb479 bytes) 2024-09-17T21:19:59.642 INFO:teuthology.orchestra.run.smithi184.stdout:4115 read 0x46f5e00 thru 0x4701eda (0xc0db bytes) 2024-09-17T21:19:59.643 INFO:teuthology.orchestra.run.smithi184.stdout:4117 read 0x4b09200 thru 0x4b0f3cb (0x61cc bytes) 2024-09-17T21:19:59.708 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:59.701+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:59.722 INFO:teuthology.orchestra.run.smithi184.stdout:4118 write 0xe6eee00 thru 0xe6f7231 (0x8432 bytes) 2024-09-17T21:19:59.724 INFO:teuthology.orchestra.run.smithi184.stdout:4122 write 0x9b4f400 thru 0x9b5890e (0x950f bytes) 2024-09-17T21:19:59.724 INFO:teuthology.orchestra.run.smithi184.stdout:4123 punch from 0x9f46200 to 0x9f51c00, (0xba00 bytes) 2024-09-17T21:19:59.725 INFO:teuthology.orchestra.run.smithi184.stdout:4124 read 0xc1fae00 thru 0xc205ff8 (0xb1f9 bytes) 2024-09-17T21:19:59.739 INFO:teuthology.orchestra.run.smithi184.stdout:4125 trunc from 0xe6f7400 to 0x633fe00 2024-09-17T21:19:59.739 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:19:59.740 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:19:59.740 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:19:59.740 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:19:59.757 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:19:59.760 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:19:59.760 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:19:59.777 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:19:59.778 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:19:59.789 INFO:teuthology.orchestra.run.smithi184.stdout:4129 trunc from 0x633fe00 to 0x2140400 2024-09-17T21:19:59.797 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:19:59.797 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:19:59.809 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:19:59.803+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:19:59.820 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:19:59.820 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:19:59.831 INFO:teuthology.orchestra.run.smithi184.stdout:4130 read 0xd8ec00 thru 0xd9c74b (0xdb4c bytes) 2024-09-17T21:19:59.841 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:19:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:19:59.841 INFO:teuthology.orchestra.run.smithi184.stdout:4131 punch from 0x926600 to 0x934200, (0xdc00 bytes) 2024-09-17T21:19:59.842 INFO:teuthology.orchestra.run.smithi184.stdout:4133 read 0x8ca00 thru 0x9c374 (0xf975 bytes) 2024-09-17T21:19:59.844 INFO:teuthology.orchestra.run.smithi184.stdout:4136 read 0xeb3400 thru 0xebff37 (0xcb38 bytes) 2024-09-17T21:19:59.864 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:19:59.864 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:19:59.873 INFO:teuthology.orchestra.run.smithi184.stdout:4137 write 0x4ab0e00 thru 0x4abbaab (0xacac bytes) 2024-09-17T21:19:59.874 INFO:teuthology.orchestra.run.smithi184.stdout:4143 write 0x378ba00 thru 0x3799a25 (0xe026 bytes) 2024-09-17T21:19:59.874 INFO:teuthology.orchestra.run.smithi184.stdout:4144 read 0x3fdc400 thru 0x3fdc9fd (0x5fe bytes) 2024-09-17T21:19:59.874 INFO:teuthology.orchestra.run.smithi184.stdout:4146 read 0x46f8e00 thru 0x4703ce4 (0xaee5 bytes) 2024-09-17T21:19:59.875 INFO:teuthology.orchestra.run.smithi184.stdout:4147 read 0x3b8d200 thru 0x3b8fecc (0x2ccd bytes) 2024-09-17T21:19:59.875 INFO:teuthology.orchestra.run.smithi184.stdout:4150 read 0x37df400 thru 0x37ece4e (0xda4f bytes) 2024-09-17T21:19:59.875 INFO:teuthology.orchestra.run.smithi184.stdout:4152 read 0xdfa000 thru 0xe073f7 (0xd3f8 bytes) 2024-09-17T21:19:59.876 INFO:teuthology.orchestra.run.smithi184.stdout:4153 write 0x4841c00 thru 0x48483c3 (0x67c4 bytes) 2024-09-17T21:19:59.877 INFO:teuthology.orchestra.run.smithi184.stdout:4154 read 0xe5c200 thru 0xe64a61 (0x8862 bytes) 2024-09-17T21:19:59.878 INFO:teuthology.orchestra.run.smithi184.stdout:4155 trunc from 0x4abbc00 to 0x43bfe00 2024-09-17T21:19:59.884 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:19:59.884 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:19:59.900 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:19:59.910 INFO:teuthology.orchestra.run.smithi184.stdout:4156 read 0x50ec00 thru 0x515736 (0x6b37 bytes) 2024-09-17T21:19:59.914 INFO:teuthology.orchestra.run.smithi184.stdout:4159 read 0x17c4c00 thru 0x17c67e7 (0x1be8 bytes) 2024-09-17T21:19:59.915 INFO:teuthology.orchestra.run.smithi184.stdout:4163 punch from 0x1ca6200 to 0x1caec00, (0x8a00 bytes) 2024-09-17T21:19:59.915 INFO:teuthology.orchestra.run.smithi184.stdout:4165 punch from 0x3611200 to 0x361ba00, (0xa800 bytes) 2024-09-17T21:19:59.916 INFO:teuthology.orchestra.run.smithi184.stdout:4167 read 0x1979200 thru 0x1979464 (0x265 bytes) 2024-09-17T21:19:59.940 INFO:teuthology.orchestra.run.smithi184.stdout:4168 write 0x6304000 thru 0x630fe32 (0xbe33 bytes) 2024-09-17T21:19:59.942 INFO:teuthology.orchestra.run.smithi184.stdout:4169 write 0x3a97800 thru 0x3aa5862 (0xe063 bytes) 2024-09-17T21:19:59.943 INFO:teuthology.orchestra.run.smithi184.stdout:4170 punch from 0x199ec00 to 0x19a1a00, (0x2e00 bytes) 2024-09-17T21:19:59.945 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:19:59.941+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:19:59.982 INFO:teuthology.orchestra.run.smithi184.stdout:4171 write 0xee5b400 thru 0xee5c160 (0xd61 bytes) 2024-09-17T21:19:59.983 INFO:teuthology.orchestra.run.smithi184.stdout:4172 write 0xeaa7200 thru 0xeaa8f02 (0x1d03 bytes) 2024-09-17T21:19:59.983 INFO:teuthology.orchestra.run.smithi184.stdout:4173 read 0x9bf5200 thru 0x9bfd975 (0x8776 bytes) 2024-09-17T21:20:00.004 INFO:teuthology.orchestra.run.smithi184.stdout:4174 trunc from 0xee5c200 to 0x3275a00 2024-09-17T21:20:00.010 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:00.003+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:00.040 INFO:teuthology.orchestra.run.smithi184.stdout:4176 write 0xb1c400 thru 0xb2701d (0xac1e bytes) 2024-09-17T21:20:00.042 INFO:teuthology.orchestra.run.smithi184.stdout:4177 trunc from 0x3275a00 to 0x358ee00 2024-09-17T21:20:00.072 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:00.073 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:00.087 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:20:00.099 INFO:teuthology.orchestra.run.smithi184.stdout:4180 write 0x905e400 thru 0x9064fe4 (0x6be5 bytes) 2024-09-17T21:20:00.101 INFO:teuthology.orchestra.run.smithi184.stdout:4184 read 0x72ac00 thru 0x73a36d (0xf76e bytes) 2024-09-17T21:20:00.106 INFO:teuthology.orchestra.run.smithi184.stdout:4185 write 0x1aeb400 thru 0x1af90f0 (0xdcf1 bytes) 2024-09-17T21:20:00.108 INFO:teuthology.orchestra.run.smithi184.stdout:4187 write 0xd2f400 thru 0xd38796 (0x9397 bytes) 2024-09-17T21:20:00.109 INFO:teuthology.orchestra.run.smithi184.stdout:4188 read 0x914a00 thru 0x920b4f (0xc150 bytes) 2024-09-17T21:20:00.110 INFO:teuthology.orchestra.run.smithi184.stdout:4189 punch from 0x5f55800 to 0x5f64400, (0xec00 bytes) 2024-09-17T21:20:00.111 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:00.107+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:00.112 INFO:teuthology.orchestra.run.smithi184.stdout:4190 punch from 0x40d200 to 0x411800, (0x4600 bytes) 2024-09-17T21:20:00.112 INFO:teuthology.orchestra.run.smithi184.stdout:4192 read 0xffec00 thru 0x1006b42 (0x7f43 bytes) 2024-09-17T21:20:00.115 INFO:teuthology.orchestra.run.smithi184.stdout:4193 read 0x6a58e00 thru 0x6a648f9 (0xbafa bytes) 2024-09-17T21:20:00.116 INFO:teuthology.orchestra.run.smithi184.stdout:4197 write 0x6f2ac00 thru 0x6f2fc28 (0x5029 bytes) 2024-09-17T21:20:00.116 INFO:teuthology.orchestra.run.smithi184.stdout:4198 punch from 0x615fa00 to 0x6164e00, (0x5400 bytes) 2024-09-17T21:20:00.116 INFO:teuthology.orchestra.run.smithi184.stdout:4199 read 0x5db0400 thru 0x5dbb279 (0xae7a bytes) 2024-09-17T21:20:00.117 INFO:teuthology.orchestra.run.smithi184.stdout:4203 write 0x5b8d200 thru 0x5b8de3b (0xc3c bytes) 2024-09-17T21:20:00.151 INFO:teuthology.orchestra.run.smithi184.stdout:4206 write 0x98d6e00 thru 0x98dfba5 (0x8da6 bytes) 2024-09-17T21:20:00.153 INFO:teuthology.orchestra.run.smithi184.stdout:4207 write 0x3fb8e00 thru 0x3fc4a46 (0xbc47 bytes) 2024-09-17T21:20:00.193 INFO:teuthology.orchestra.run.smithi184.stdout:4209 write 0xcd60000 thru 0xcd6e371 (0xe372 bytes) 2024-09-17T21:20:00.195 INFO:teuthology.orchestra.run.smithi184.stdout:4210 write 0x3d60200 thru 0x3d65f25 (0x5d26 bytes) 2024-09-17T21:20:00.195 INFO:teuthology.orchestra.run.smithi184.stdout:4211 read 0x4eda600 thru 0x4ee8498 (0xde99 bytes) 2024-09-17T21:20:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-17T21:20:00.209 INFO:teuthology.orchestra.run.smithi184.stdout:4213 trunc from 0xcd6e400 to 0x5f7a600 2024-09-17T21:20:00.212 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:00.207+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:00.217 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:00.217 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:00.236 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:20:00.278 INFO:teuthology.orchestra.run.smithi184.stdout:4214 write 0x8024600 thru 0x802f415 (0xae16 bytes) 2024-09-17T21:20:00.279 INFO:teuthology.orchestra.run.smithi184.stdout:4215 punch from 0x2ca2400 to 0x2ca8400, (0x6000 bytes) 2024-09-17T21:20:00.304 INFO:teuthology.orchestra.run.smithi184.stdout:4216 write 0xbcc4800 thru 0xbccae56 (0x6657 bytes) 2024-09-17T21:20:00.312 INFO:teuthology.orchestra.run.smithi184.stdout:4218 trunc from 0xbccb000 to 0x85d7400 2024-09-17T21:20:00.320 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:00.317+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:00.341 INFO:teuthology.orchestra.run.smithi184.stdout:4219 punch from 0x8474600 to 0x847a400, (0x5e00 bytes) 2024-09-17T21:20:00.368 INFO:teuthology.orchestra.run.smithi184.stdout:4220 write 0xbd33000 thru 0xbd3e043 (0xb044 bytes) 2024-09-17T21:20:00.396 INFO:teuthology.orchestra.run.smithi184.stdout:4221 write 0xd274800 thru 0xd282256 (0xda57 bytes) 2024-09-17T21:20:00.397 INFO:teuthology.orchestra.run.smithi184.stdout:4223 punch from 0xa3c3a00 to 0xa3cca00, (0x9000 bytes) 2024-09-17T21:20:00.398 INFO:teuthology.orchestra.run.smithi184.stdout:4228 write 0x6f5b000 thru 0x6f5f5ba (0x45bb bytes) 2024-09-17T21:20:00.399 INFO:teuthology.orchestra.run.smithi184.stdout:4230 read 0x79dce00 thru 0x79e9920 (0xcb21 bytes) 2024-09-17T21:20:00.399 INFO:teuthology.orchestra.run.smithi184.stdout:4234 write 0xa608400 thru 0xa6160d0 (0xdcd1 bytes) 2024-09-17T21:20:00.399 INFO:teuthology.orchestra.run.smithi184.stdout:4235 write 0xcc2ce00 thru 0xcc39c31 (0xce32 bytes) 2024-09-17T21:20:00.399 INFO:teuthology.orchestra.run.smithi184.stdout:4237 read 0x46d8c00 thru 0x46e5c2d (0xd02e bytes) 2024-09-17T21:20:00.400 INFO:teuthology.orchestra.run.smithi184.stdout:4238 read 0x7cff600 thru 0x7d0286b (0x326c bytes) 2024-09-17T21:20:00.400 INFO:teuthology.orchestra.run.smithi184.stdout:4241 punch from 0x6763e00 to 0x6769000, (0x5200 bytes) 2024-09-17T21:20:00.400 INFO:teuthology.orchestra.run.smithi184.stdout:4242 punch from 0xb175600 to 0xb184a00, (0xf400 bytes) 2024-09-17T21:20:00.401 INFO:teuthology.orchestra.run.smithi184.stdout:4244 read 0xb1ad200 thru 0xb1b1ced (0x4aee bytes) 2024-09-17T21:20:00.402 INFO:teuthology.orchestra.run.smithi184.stdout:4246 read 0x514e800 thru 0x5158ef7 (0xa6f8 bytes) 2024-09-17T21:20:00.402 INFO:teuthology.orchestra.run.smithi184.stdout:4248 write 0xa475c00 thru 0xa47fa42 (0x9e43 bytes) 2024-09-17T21:20:00.402 INFO:teuthology.orchestra.run.smithi184.stdout:4249 write 0x86a3400 thru 0x86af215 (0xbe16 bytes) 2024-09-17T21:20:00.405 INFO:teuthology.orchestra.run.smithi184.stdout:4250 read 0x5a41400 thru 0x5a495f7 (0x81f8 bytes) 2024-09-17T21:20:00.406 INFO:teuthology.orchestra.run.smithi184.stdout:4252 write 0x8f96200 thru 0x8f9c98e (0x678f bytes) 2024-09-17T21:20:00.406 INFO:teuthology.orchestra.run.smithi184.stdout:4254 write 0x78a6c00 thru 0x78ab9ba (0x4dbb bytes) 2024-09-17T21:20:00.408 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:00.408 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:00.413 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:00.407+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:00.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:20:00.442 INFO:teuthology.orchestra.run.smithi184.stdout:4261 write 0xe808e00 thru 0xe80fca7 (0x6ea8 bytes) 2024-09-17T21:20:00.444 INFO:teuthology.orchestra.run.smithi184.stdout:4263 read 0x2470800 thru 0x2475a7e (0x527f bytes) 2024-09-17T21:20:00.444 INFO:teuthology.orchestra.run.smithi184.stdout:4265 trunc from 0xe80fe00 to 0xe757c00 2024-09-17T21:20:00.471 INFO:teuthology.orchestra.run.smithi184.stdout:4267 write 0xb4a200 thru 0xb541ff (0xa000 bytes) 2024-09-17T21:20:00.485 INFO:teuthology.orchestra.run.smithi184.stdout:4268 trunc from 0xe757c00 to 0x99d8600 2024-09-17T21:20:00.513 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:00.509+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:00.528 INFO:teuthology.orchestra.run.smithi184.stdout:4270 write 0x19cfe00 thru 0x19da601 (0xa802 bytes) 2024-09-17T21:20:00.528 INFO:teuthology.orchestra.run.smithi184.stdout:4271 read 0x9683a00 thru 0x9685d5a (0x235b bytes) 2024-09-17T21:20:00.537 INFO:teuthology.orchestra.run.smithi184.stdout:4272 trunc from 0x99d8600 to 0x5aa5a00 2024-09-17T21:20:00.590 INFO:teuthology.orchestra.run.smithi184.stdout:4275 trunc from 0x5aa5a00 to 0xbbdbe00 2024-09-17T21:20:00.600 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:00.601 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:00.614 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:00.607+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:00.616 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:20:00.634 INFO:teuthology.orchestra.run.smithi184.stdout:4276 write 0xd414e00 thru 0xd424931 (0xfb32 bytes) 2024-09-17T21:20:00.666 INFO:teuthology.orchestra.run.smithi184.stdout:4277 write 0xdce9a00 thru 0xdcf0127 (0x6728 bytes) 2024-09-17T21:20:00.668 INFO:teuthology.orchestra.run.smithi184.stdout:4278 write 0x6804e00 thru 0x68081fe (0x33ff bytes) 2024-09-17T21:20:00.668 INFO:teuthology.orchestra.run.smithi184.stdout:4279 read 0x1b57800 thru 0x1b5a81a (0x301b bytes) 2024-09-17T21:20:00.673 INFO:teuthology.orchestra.run.smithi184.stdout:4281 write 0x45b5600 thru 0x45b7cb1 (0x26b2 bytes) 2024-09-17T21:20:00.673 INFO:teuthology.orchestra.run.smithi184.stdout:4283 read 0xb93c000 thru 0xb93e217 (0x2218 bytes) 2024-09-17T21:20:00.673 INFO:teuthology.orchestra.run.smithi184.stdout:4285 read 0xb523600 thru 0xb52f4ed (0xbeee bytes) 2024-09-17T21:20:00.673 INFO:teuthology.orchestra.run.smithi184.stdout:truncating to largest ever: 0xee4a200 2024-09-17T21:20:00.676 INFO:teuthology.orchestra.run.smithi184.stdout:4287 trunc from 0xdcf0200 to 0xee4a200 2024-09-17T21:20:00.704 INFO:teuthology.orchestra.run.smithi184.stdout:4289 write 0x3689600 thru 0x3696c74 (0xd675 bytes) 2024-09-17T21:20:00.705 INFO:teuthology.orchestra.run.smithi184.stdout:4291 read 0x22b9400 thru 0x22bad38 (0x1939 bytes) 2024-09-17T21:20:00.706 INFO:teuthology.orchestra.run.smithi184.stdout:4293 read 0x5410400 thru 0x541a977 (0xa578 bytes) 2024-09-17T21:20:00.707 INFO:teuthology.orchestra.run.smithi184.stdout:4294 write 0x762a200 thru 0x76389e3 (0xe7e4 bytes) 2024-09-17T21:20:00.707 INFO:teuthology.orchestra.run.smithi184.stdout:4295 punch from 0xb308e00 to 0xb312600, (0x9800 bytes) 2024-09-17T21:20:00.709 INFO:teuthology.orchestra.run.smithi184.stdout:4296 write 0x7798200 thru 0x77a3cd7 (0xbad8 bytes) 2024-09-17T21:20:00.709 INFO:teuthology.orchestra.run.smithi184.stdout:4297 read 0x2321200 thru 0x232ca9c (0xb89d bytes) 2024-09-17T21:20:00.710 INFO:teuthology.orchestra.run.smithi184.stdout:4299 punch from 0x9f80c00 to 0x9f8d800, (0xcc00 bytes) 2024-09-17T21:20:00.711 INFO:teuthology.orchestra.run.smithi184.stdout:4300 write 0x5183800 thru 0x518c3eb (0x8bec bytes) 2024-09-17T21:20:00.711 INFO:teuthology.orchestra.run.smithi184.stdout:4302 write 0x875ee00 thru 0x8768145 (0x9346 bytes) 2024-09-17T21:20:00.714 INFO:teuthology.orchestra.run.smithi184.stdout:4305 trunc from 0xee4a200 to 0xd8e7600 2024-09-17T21:20:00.714 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:00.711+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:00.755 INFO:teuthology.orchestra.run.smithi184.stdout:4306 read 0x7011200 thru 0x701d649 (0xc44a bytes) 2024-09-17T21:20:00.758 INFO:teuthology.orchestra.run.smithi184.stdout:4308 trunc from 0xd8e7600 to 0xe90a000 2024-09-17T21:20:00.785 INFO:teuthology.orchestra.run.smithi184.stdout:4309 write 0x984b600 thru 0x985054a (0x4f4b bytes) 2024-09-17T21:20:00.787 INFO:teuthology.orchestra.run.smithi184.stdout:4310 write 0x4808800 thru 0x480ca22 (0x4223 bytes) 2024-09-17T21:20:00.788 INFO:teuthology.orchestra.run.smithi184.stdout:4311 write 0x454d200 thru 0x4551b24 (0x4925 bytes) 2024-09-17T21:20:00.788 INFO:teuthology.orchestra.run.smithi184.stdout:4313 read 0x8eb2000 thru 0x8ebd477 (0xb478 bytes) 2024-09-17T21:20:00.789 INFO:teuthology.orchestra.run.smithi184.stdout:4314 write 0xabdd800 thru 0xabe2aa2 (0x52a3 bytes) 2024-09-17T21:20:00.789 INFO:teuthology.orchestra.run.smithi184.stdout:4315 write 0x4f0b800 thru 0x4f0fc31 (0x4432 bytes) 2024-09-17T21:20:00.790 INFO:teuthology.orchestra.run.smithi184.stdout:4316 write 0xc639000 thru 0xc64191a (0x891b bytes) 2024-09-17T21:20:00.790 INFO:teuthology.orchestra.run.smithi184.stdout:4317 read 0x7254600 thru 0x7254ed4 (0x8d5 bytes) 2024-09-17T21:20:00.790 INFO:teuthology.orchestra.run.smithi184.stdout:4319 write 0x394aa00 thru 0x3959343 (0xe944 bytes) 2024-09-17T21:20:00.791 INFO:teuthology.orchestra.run.smithi184.stdout:4320 write 0x42ff200 thru 0x42ff740 (0x541 bytes) 2024-09-17T21:20:00.791 INFO:teuthology.orchestra.run.smithi184.stdout:4321 write 0x808d400 thru 0x8095276 (0x7e77 bytes) 2024-09-17T21:20:00.791 INFO:teuthology.orchestra.run.smithi184.stdout:4322 write 0x57bee00 thru 0x57cc3e2 (0xd5e3 bytes) 2024-09-17T21:20:00.791 INFO:teuthology.orchestra.run.smithi184.stdout:4325 punch from 0x34ec00 to 0x35aa00, (0xbe00 bytes) 2024-09-17T21:20:00.795 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:00.796 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:00.805 INFO:teuthology.orchestra.run.smithi184.stdout:4326 trunc from 0xe90a000 to 0x6f8c200 2024-09-17T21:20:00.815 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:20:00.816 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:00.811+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:00.863 INFO:teuthology.orchestra.run.smithi184.stdout:4327 read 0x4a14e00 thru 0x4a205e0 (0xb7e1 bytes) 2024-09-17T21:20:00.864 INFO:teuthology.orchestra.run.smithi184.stdout:4328 read 0x40b1400 thru 0x40be5ef (0xd1f0 bytes) 2024-09-17T21:20:00.865 INFO:teuthology.orchestra.run.smithi184.stdout:4329 read 0x2d48200 thru 0x2d4a4f8 (0x22f9 bytes) 2024-09-17T21:20:00.865 INFO:teuthology.orchestra.run.smithi184.stdout:4330 read 0x629e800 thru 0x629fbd3 (0x13d4 bytes) 2024-09-17T21:20:00.865 INFO:teuthology.orchestra.run.smithi184.stdout:4332 read 0x5071400 thru 0x507b704 (0xa305 bytes) 2024-09-17T21:20:00.873 INFO:teuthology.orchestra.run.smithi184.stdout:4336 trunc from 0x6f8c200 to 0x31af600 2024-09-17T21:20:00.916 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:00.911+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:00.919 INFO:teuthology.orchestra.run.smithi184.stdout:4341 trunc from 0x31af600 to 0x4026c00 2024-09-17T21:20:00.943 INFO:teuthology.orchestra.run.smithi184.stdout:4348 write 0x1d4c200 thru 0x1d4da98 (0x1899 bytes) 2024-09-17T21:20:00.950 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:00.950 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:00.965 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:20:00.969 INFO:teuthology.orchestra.run.smithi184.stdout:4349 trunc from 0x4026c00 to 0xc7b5a00 2024-09-17T21:20:00.997 INFO:teuthology.orchestra.run.smithi184.stdout:4352 read 0x9254c00 thru 0x9264475 (0xf876 bytes) 2024-09-17T21:20:00.999 INFO:teuthology.orchestra.run.smithi184.stdout:4353 punch from 0x7c67600 to 0x7c6c600, (0x5000 bytes) 2024-09-17T21:20:01.000 INFO:teuthology.orchestra.run.smithi184.stdout:4354 read 0x5629600 thru 0x562f193 (0x5b94 bytes) 2024-09-17T21:20:01.009 INFO:teuthology.orchestra.run.smithi184.stdout:4361 trunc from 0xc7b5a00 to 0x95f1600 2024-09-17T21:20:01.016 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:01.011+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:01.069 INFO:teuthology.orchestra.run.smithi184.stdout:4363 trunc from 0x95f1600 to 0x77ee00 2024-09-17T21:20:01.098 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:01.098 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:01.115 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:20:01.117 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:01.111+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:01.123 INFO:teuthology.orchestra.run.smithi184.stdout:4367 punch from 0x27b000 to 0x28a200, (0xf200 bytes) 2024-09-17T21:20:01.127 INFO:teuthology.orchestra.run.smithi184.stdout:4368 read 0x23e00 thru 0x2f00b (0xb20c bytes) 2024-09-17T21:20:01.133 INFO:teuthology.orchestra.run.smithi184.stdout:4369 punch from 0x544800 to 0x54bc00, (0x7400 bytes) 2024-09-17T21:20:01.160 INFO:teuthology.orchestra.run.smithi184.stdout:4370 write 0x24ffe00 thru 0x2507802 (0x7a03 bytes) 2024-09-17T21:20:01.215 INFO:teuthology.orchestra.run.smithi184.stdout:4372 write 0xbdb3600 thru 0xbdbe57e (0xaf7f bytes) 2024-09-17T21:20:01.218 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:01.213+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:01.222 INFO:teuthology.orchestra.run.smithi184.stdout:4373 trunc from 0xbdbe600 to 0xda64800 2024-09-17T21:20:01.250 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:01.250 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:01.251 INFO:teuthology.orchestra.run.smithi184.stdout:4374 write 0xbb1d800 thru 0xbb26a9e (0x929f bytes) 2024-09-17T21:20:01.252 INFO:teuthology.orchestra.run.smithi184.stdout:4375 write 0xc036400 thru 0xc039980 (0x3581 bytes) 2024-09-17T21:20:01.253 INFO:teuthology.orchestra.run.smithi184.stdout:4377 read 0x5304200 thru 0x5304f81 (0xd82 bytes) 2024-09-17T21:20:01.253 INFO:teuthology.orchestra.run.smithi184.stdout:4380 write 0x336c200 thru 0x336f349 (0x314a bytes) 2024-09-17T21:20:01.253 INFO:teuthology.orchestra.run.smithi184.stdout:4381 punch from 0x885c400 to 0x886b600, (0xf200 bytes) 2024-09-17T21:20:01.254 INFO:teuthology.orchestra.run.smithi184.stdout:4383 read 0xd88fa00 thru 0xd89de9b (0xe49c bytes) 2024-09-17T21:20:01.254 INFO:teuthology.orchestra.run.smithi184.stdout:4385 read 0x5d77c00 thru 0x5d8690f (0xed10 bytes) 2024-09-17T21:20:01.254 INFO:teuthology.orchestra.run.smithi184.stdout:4390 write 0x73e5000 thru 0x73f1a2d (0xca2e bytes) 2024-09-17T21:20:01.255 INFO:teuthology.orchestra.run.smithi184.stdout:4392 write 0x6c1bc00 thru 0x6c284f1 (0xc8f2 bytes) 2024-09-17T21:20:01.264 INFO:teuthology.orchestra.run.smithi184.stdout:4394 trunc from 0xda64800 to 0x8f0c600 2024-09-17T21:20:01.267 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:20:01.306 INFO:teuthology.orchestra.run.smithi184.stdout:4396 write 0xd33800 thru 0xd3793e (0x413f bytes) 2024-09-17T21:20:01.308 INFO:teuthology.orchestra.run.smithi184.stdout:4399 read 0x4fd2400 thru 0x4fd310d (0xd0e bytes) 2024-09-17T21:20:01.308 INFO:teuthology.orchestra.run.smithi184.stdout:4402 write 0x49f7800 thru 0x4a027c1 (0xafc2 bytes) 2024-09-17T21:20:01.308 INFO:teuthology.orchestra.run.smithi184.stdout:4404 read 0x38b3c00 thru 0x38b4501 (0x902 bytes) 2024-09-17T21:20:01.309 INFO:teuthology.orchestra.run.smithi184.stdout:4405 read 0x89e3800 thru 0x89e8cc4 (0x54c5 bytes) 2024-09-17T21:20:01.315 INFO:teuthology.orchestra.run.smithi184.stdout:4406 trunc from 0x8f0c600 to 0x6ff4400 2024-09-17T21:20:01.318 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:01.311+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:01.363 INFO:teuthology.orchestra.run.smithi184.stdout:4408 read 0x1413800 thru 0x14175fa (0x3dfb bytes) 2024-09-17T21:20:01.365 INFO:teuthology.orchestra.run.smithi184.stdout:4413 punch from 0x6302c00 to 0x6308c00, (0x6000 bytes) 2024-09-17T21:20:01.365 INFO:teuthology.orchestra.run.smithi184.stdout:4414 write 0xc2aa00 thru 0xc3329e (0x889f bytes) 2024-09-17T21:20:01.380 INFO:teuthology.orchestra.run.smithi184.stdout:4416 trunc from 0x6ff4400 to 0xc7fba00 2024-09-17T21:20:01.401 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:01.401 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:01.414 INFO:teuthology.orchestra.run.smithi184.stdout:4417 trunc from 0xc7fba00 to 0xc425400 2024-09-17T21:20:01.416 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:20:01.439 INFO:teuthology.orchestra.run.smithi184.stdout:4420 read 0xb45a800 thru 0xb461df3 (0x75f4 bytes) 2024-09-17T21:20:01.439 INFO:teuthology.orchestra.run.smithi184.stdout:4421 punch from 0x8910200 to 0x8919e00, (0x9c00 bytes) 2024-09-17T21:20:01.445 INFO:teuthology.orchestra.run.smithi184.stdout:4422 trunc from 0xc425400 to 0xaa3fe00 2024-09-17T21:20:01.459 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:01.455+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:01.471 INFO:teuthology.orchestra.run.smithi184.stdout:4423 read 0x2a06400 thru 0x2a0c5d0 (0x61d1 bytes) 2024-09-17T21:20:01.498 INFO:teuthology.orchestra.run.smithi184.stdout:4426 write 0xb88a800 thru 0xb88c5eb (0x1dec bytes) 2024-09-17T21:20:01.499 INFO:teuthology.orchestra.run.smithi184.stdout:4427 punch from 0xa104a00 to 0xa106800, (0x1e00 bytes) 2024-09-17T21:20:01.499 INFO:teuthology.orchestra.run.smithi184.stdout:4428 read 0x6092400 thru 0x60a123e (0xee3f bytes) 2024-09-17T21:20:01.500 INFO:teuthology.orchestra.run.smithi184.stdout:4429 read 0x11d7c00 thru 0x11da238 (0x2639 bytes) 2024-09-17T21:20:01.501 INFO:teuthology.orchestra.run.smithi184.stdout:4430 read 0xa24d800 thru 0xa259cff (0xc500 bytes) 2024-09-17T21:20:01.501 INFO:teuthology.orchestra.run.smithi184.stdout:4432 read 0x24c4800 thru 0x24c9aba (0x52bb bytes) 2024-09-17T21:20:01.501 INFO:teuthology.orchestra.run.smithi184.stdout:4433 punch from 0xd5e600 to 0xd66c00, (0x8600 bytes) 2024-09-17T21:20:01.520 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:01.513+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:01.532 INFO:teuthology.orchestra.run.smithi184.stdout:4434 write 0xd46c600 thru 0xd4729a1 (0x63a2 bytes) 2024-09-17T21:20:01.535 INFO:teuthology.orchestra.run.smithi184.stdout:4435 write 0x9758800 thru 0x9767f92 (0xf793 bytes) 2024-09-17T21:20:01.568 INFO:teuthology.orchestra.run.smithi184.stdout:4436 write 0xe9c2a00 thru 0xe9cab55 (0x8156 bytes) 2024-09-17T21:20:01.571 INFO:teuthology.orchestra.run.smithi184.stdout:4437 write 0xc1b2400 thru 0xc1b9439 (0x703a bytes) 2024-09-17T21:20:01.571 INFO:teuthology.orchestra.run.smithi184.stdout:4438 write 0x4e5ca00 thru 0x4e69d0a (0xd30b bytes) 2024-09-17T21:20:01.571 INFO:teuthology.orchestra.run.smithi184.stdout:4440 write 0x4b39e00 thru 0x4b498b6 (0xfab7 bytes) 2024-09-17T21:20:01.571 INFO:teuthology.orchestra.run.smithi184.stdout:4443 write 0xcd50000 thru 0xcd5c9b4 (0xc9b5 bytes) 2024-09-17T21:20:01.574 INFO:teuthology.orchestra.run.smithi184.stdout:4444 trunc from 0xe9cac00 to 0xd49b800 2024-09-17T21:20:01.590 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:01.591 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:01.606 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:20:01.669 INFO:teuthology.orchestra.run.smithi184.stdout:4445 trunc from 0xd49b800 to 0x6d44200 2024-09-17T21:20:01.671 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:01.665+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:01.685 INFO:teuthology.orchestra.run.smithi184.stdout:4446 write 0x3646e00 thru 0x3652e37 (0xc038 bytes) 2024-09-17T21:20:01.686 INFO:teuthology.orchestra.run.smithi184.stdout:4447 read 0x5273e00 thru 0x527c8c0 (0x8ac1 bytes) 2024-09-17T21:20:01.686 INFO:teuthology.orchestra.run.smithi184.stdout:4448 write 0x2731200 thru 0x273cf7e (0xbd7f bytes) 2024-09-17T21:20:01.687 INFO:teuthology.orchestra.run.smithi184.stdout:4451 read 0x618a400 thru 0x61962d9 (0xbeda bytes) 2024-09-17T21:20:01.687 INFO:teuthology.orchestra.run.smithi184.stdout:4452 read 0x1fc6a00 thru 0x1fd5d2b (0xf32c bytes) 2024-09-17T21:20:01.687 INFO:teuthology.orchestra.run.smithi184.stdout:4454 read 0x2943400 thru 0x294d9d8 (0xa5d9 bytes) 2024-09-17T21:20:01.688 INFO:teuthology.orchestra.run.smithi184.stdout:4455 punch from 0x3531e00 to 0x3540a00, (0xec00 bytes) 2024-09-17T21:20:01.688 INFO:teuthology.orchestra.run.smithi184.stdout:4458 write 0x5094000 thru 0x509d057 (0x9058 bytes) 2024-09-17T21:20:01.689 INFO:teuthology.orchestra.run.smithi184.stdout:4460 write 0xadf800 thru 0xae9c9e (0xa49f bytes) 2024-09-17T21:20:01.730 INFO:teuthology.orchestra.run.smithi184.stdout:4467 write 0xd1a6c00 thru 0xd1b0e98 (0xa299 bytes) 2024-09-17T21:20:01.731 INFO:teuthology.orchestra.run.smithi184.stdout:4468 read 0x942c600 thru 0x9439554 (0xcf55 bytes) 2024-09-17T21:20:01.731 INFO:teuthology.orchestra.run.smithi184.stdout:4470 clone 28 order 21 su 1048576 sc 7 2024-09-17T21:20:01.771 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:01.765+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:01.800 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:01.800 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:01.814 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:20:01.871 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:01.865+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:01.882 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:20:01.936 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:01.936 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:01.950 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:20:01.972 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:01.965+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:02.035 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:02.036 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:02.053 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:02.053 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:02.070 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:02.070 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:02.072 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:02.065+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:02.072 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:02.073 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:02.087 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:20:02.088 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:02.089 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:02.107 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:02.108 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:02.124 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:02.124 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:02.185 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:02.181+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:02.208 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:02.209 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:02.209 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:20:02.209 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:02.224 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:20:02.230 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:20:02.277 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:02.273+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:02.351 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:02.351 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:02.351 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:02.351 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:02.351 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:02.366 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:20:02.379 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone27 from 0xd1b1000 (overlap 0x77ee00) to 0x3bf600 2024-09-17T21:20:02.411 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:02.407+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:02.475 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:02.469+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:02.500 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:02.500 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:02.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:02.500 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:02.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.500 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:20:02.500 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 100.892145771, 2024-09-17T21:20:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:20:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:20:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:20:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:20:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:20:02.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:20:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:20:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:20:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:20:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:20:02.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:20:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:20:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:20:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:20:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:20:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:02.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:20:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:20:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:20:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 100.88219913499999, 2024-09-17T21:20:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:20:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:20:02.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:20:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:20:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:20:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:20:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:20:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:20:02.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:20:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:20:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:20:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:20:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:20:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:02.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:20:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:20:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:20:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:20:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:20:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:02.509 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 99.639048294999995, 2024-09-17T21:20:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:20:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:20:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:20:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:20:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:02.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:20:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:20:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:02.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:20:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:20:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:20:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:20:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:20:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:20:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:20:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:20:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:20:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:20:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:20:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:20:02.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:20:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:20:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:20:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:20:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:20:02.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:20:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:20:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:20:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:20:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 99.629402808999998, 2024-09-17T21:20:02.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:20:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:20:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:20:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:20:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:02.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:02.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:02.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:02.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:20:02.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:20:02.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:02.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:20:02.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:20:02.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:02.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:20:02.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:20:02.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:02.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:20:02.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:20:02.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:02.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:20:02.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:20:02.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:02.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:20:02.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:20:02.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:02.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:20:02.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:20:02.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:02.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:20:02.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:20:02.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:02.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:20:02.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:20:02.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:02.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:20:02.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:20:02.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:02.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:20:02.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:20:02.521 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.521 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:02.521 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.521 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:20:02.521 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:02.521 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 86.260466776000001, 2024-09-17T21:20:02.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:20:02.521 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:02.521 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:02.522 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:02.522 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:02.522 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:20:02.522 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:20:02.522 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:20:02.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.522 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:02.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:02.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:02.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:02.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:02.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:02.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:20:02.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:20:02.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:02.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:20:02.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:20:02.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:02.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:20:02.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:20:02.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:02.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:20:02.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:20:02.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:02.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:20:02.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:20:02.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:02.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:20:02.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:20:02.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:02.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:20:02.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:20:02.525 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.525 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:02.525 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.526 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:819 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5111808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:02.526 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:02.526 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.058894486, 2024-09-17T21:20:02.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024344811000000001, 2024-09-17T21:20:02.526 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:02.526 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:02.526 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:02.526 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:02.526 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:02.526 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:02.526 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 819 2024-09-17T21:20:02.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.526 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:02.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:02.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:02.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:02.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:02.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:02.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437443+0000", 2024-09-17T21:20:02.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1969999999999999e-06 2024-09-17T21:20:02.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:02.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437448+0000", 2024-09-17T21:20:02.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9490000000000004e-06 2024-09-17T21:20:02.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:02.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437467+0000", 2024-09-17T21:20:02.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8862999999999999e-05 2024-09-17T21:20:02.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:02.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437480+0000", 2024-09-17T21:20:02.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2833000000000001e-05 2024-09-17T21:20:02.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:02.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461552+0000", 2024-09-17T21:20:02.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024072695000000002 2024-09-17T21:20:02.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:02.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461639+0000", 2024-09-17T21:20:02.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6996000000000005e-05 2024-09-17T21:20:02.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:02.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461787+0000", 2024-09-17T21:20:02.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014727800000000001 2024-09-17T21:20:02.530 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.530 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:02.530 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.530 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:02.530 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:02.530 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.058782214000004, 2024-09-17T21:20:02.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:20:02.530 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:02.530 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:02.530 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:02.531 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:02.531 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:02.531 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:02.531 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:20:02.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.531 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:02.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:02.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:02.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:02.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:02.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:02.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:20:02.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:20:02.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:02.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:20:02.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:20:02.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:02.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:20:02.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:20:02.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:02.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:20:02.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:20:02.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:02.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:20:02.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:20:02.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:02.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:20:02.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:20:02.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:02.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:20:02.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:20:02.535 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.535 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:02.535 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.535 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:833 3.5 3:a772dfb1:::rbd_data.11523651b015.0000000000000001:d [sparse-read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:02.535 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:02.535 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.058347928000003, 2024-09-17T21:20:02.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024216024999999999, 2024-09-17T21:20:02.535 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:02.535 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:02.535 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:02.535 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:02.535 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:02.536 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:02.536 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 833 2024-09-17T21:20:02.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.536 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:02.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:02.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:02.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:02.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:02.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:02.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437990+0000", 2024-09-17T21:20:02.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2500000000000001e-06 2024-09-17T21:20:02.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:02.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437995+0000", 2024-09-17T21:20:02.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1240000000000004e-06 2024-09-17T21:20:02.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:02.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438012+0000", 2024-09-17T21:20:02.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7572e-05 2024-09-17T21:20:02.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:02.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438031+0000", 2024-09-17T21:20:02.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8612000000000001e-05 2024-09-17T21:20:02.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:02.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461830+0000", 2024-09-17T21:20:02.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023799135999999999 2024-09-17T21:20:02.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:02.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462005+0000", 2024-09-17T21:20:02.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000174747 2024-09-17T21:20:02.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:02.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:20:02.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019958399999999999 2024-09-17T21:20:02.539 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.539 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:02.539 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.539 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:02.539 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:02.539 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.058287637000006, 2024-09-17T21:20:02.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:20:02.540 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:02.540 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:02.540 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:02.540 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:02.540 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:02.540 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:02.540 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:20:02.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.540 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:02.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:02.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:02.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:02.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:02.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:02.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:20:02.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:20:02.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:02.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:20:02.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:20:02.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:02.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:20:02.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:20:02.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:02.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:20:02.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:20:02.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:02.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:20:02.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:20:02.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:02.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:20:02.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:20:02.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:02.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:20:02.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:20:02.544 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.544 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:02.544 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.544 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:02.544 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:02.544 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.058232255999997, 2024-09-17T21:20:02.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:20:02.544 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:02.544 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:02.545 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:02.545 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:02.545 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:02.545 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:02.545 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:20:02.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.545 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:02.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:02.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:02.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:02.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:02.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:02.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:20:02.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:20:02.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:02.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:20:02.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:20:02.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:02.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:20:02.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:20:02.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:02.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:20:02.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:20:02.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:02.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:20:02.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:20:02.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:02.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:20:02.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:20:02.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:02.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:20:02.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:20:02.548 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.548 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:02.548 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.548 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:02.548 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:02.549 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.058175977999994, 2024-09-17T21:20:02.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:20:02.549 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:02.549 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:02.549 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:02.549 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:02.549 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:02.549 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:02.549 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:20:02.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.549 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:02.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:02.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:02.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:02.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:02.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:02.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:20:02.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:20:02.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:02.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:20:02.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:20:02.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:02.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:20:02.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:20:02.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:02.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:20:02.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:20:02.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:02.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:20:02.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:20:02.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:02.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:20:02.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:20:02.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:02.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:20:02.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:20:02.553 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.553 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:02.553 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.553 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:02.553 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:02.553 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.057226338999996, 2024-09-17T21:20:02.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:20:02.553 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:02.553 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:02.553 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:02.553 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:02.554 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:02.554 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:02.554 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:20:02.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.554 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:02.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:02.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:02.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:02.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:02.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:02.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:20:02.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:20:02.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:02.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:20:02.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:20:02.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:02.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:20:02.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:20:02.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:02.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:20:02.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:20:02.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:02.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:20:02.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:20:02.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.557 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:02.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:20:02.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:20:02.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.557 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:02.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:20:02.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:20:02.557 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.557 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:02.558 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.558 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:02.558 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:02.558 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.057147719, 2024-09-17T21:20:02.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:20:02.558 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:02.558 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:02.558 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:02.558 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:02.558 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:02.558 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:02.559 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:20:02.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.559 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:02.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:02.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:02.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:02.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:02.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:02.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:20:02.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:20:02.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:02.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:20:02.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:20:02.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:02.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:20:02.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:20:02.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:02.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:20:02.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:20:02.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:02.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:20:02.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:20:02.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:02.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:20:02.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:20:02.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.562 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:02.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:20:02.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:20:02.562 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.562 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:02.562 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.562 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:02.562 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:02.563 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.057034779999995, 2024-09-17T21:20:02.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:20:02.563 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:02.563 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:02.563 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:02.563 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:02.563 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:02.563 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:02.563 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:20:02.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.563 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:02.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:02.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:02.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:02.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:02.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:02.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:20:02.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:20:02.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:02.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:20:02.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:20:02.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:02.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:20:02.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:20:02.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:02.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:20:02.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:20:02.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:02.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:20:02.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:20:02.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:02.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:20:02.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:20:02.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:02.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:20:02.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:20:02.567 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.567 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:02.567 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.567 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:849 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5963776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:02.567 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:02.567 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.056440744, 2024-09-17T21:20:02.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024537514, 2024-09-17T21:20:02.567 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:02.567 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:02.567 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:02.568 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:02.568 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:02.568 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:02.568 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 849 2024-09-17T21:20:02.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.568 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:02.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:02.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:02.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:02.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:02.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:02.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439897+0000", 2024-09-17T21:20:02.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9099999999999999e-06 2024-09-17T21:20:02.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:02.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439905+0000", 2024-09-17T21:20:02.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0870000000000007e-06 2024-09-17T21:20:02.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:02.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439924+0000", 2024-09-17T21:20:02.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9042000000000002e-05 2024-09-17T21:20:02.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:02.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439941+0000", 2024-09-17T21:20:02.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.624e-05 2024-09-17T21:20:02.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:02.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464132+0000", 2024-09-17T21:20:02.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024191579000000001 2024-09-17T21:20:02.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:02.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464291+0000", 2024-09-17T21:20:02.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015904800000000001 2024-09-17T21:20:02.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:02.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464433+0000", 2024-09-17T21:20:02.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014160799999999999 2024-09-17T21:20:02.571 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.571 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:02.571 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.572 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:02.572 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:02.572 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.056373726000004, 2024-09-17T21:20:02.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:20:02.572 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:02.572 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:02.572 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:02.572 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:02.572 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:02.572 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:02.572 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:20:02.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.572 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:02.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:02.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:02.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:02.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:02.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:02.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:20:02.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:20:02.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:02.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:20:02.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:20:02.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:02.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:20:02.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:20:02.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:02.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:20:02.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:20:02.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:02.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:20:02.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:20:02.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:02.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:20:02.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:20:02.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:02.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:20:02.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:20:02.576 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.576 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:02.576 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.576 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:02.576 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:02.576 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.056308915000002, 2024-09-17T21:20:02.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:20:02.576 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:02.577 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:02.577 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:02.577 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:02.577 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:02.577 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:02.577 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:20:02.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.577 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:02.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:02.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:02.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:02.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:02.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:02.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:20:02.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:20:02.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:02.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:20:02.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:20:02.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:02.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:20:02.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:20:02.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:02.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:20:02.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:20:02.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:02.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:20:02.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:20:02.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:02.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:20:02.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:20:02.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:02.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:20:02.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:20:02.580 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.580 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:02.580 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.581 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:02.581 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:02.581 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.056247937999998, 2024-09-17T21:20:02.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:20:02.581 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:02.581 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:02.581 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:02.581 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:02.581 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:02.581 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:02.581 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:20:02.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.582 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:02.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:02.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:02.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:02.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:02.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:02.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:20:02.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:20:02.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:02.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:20:02.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:20:02.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:02.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:20:02.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:20:02.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:02.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:20:02.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:20:02.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:02.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:20:02.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:20:02.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:02.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:20:02.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:20:02.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:02.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:20:02.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:20:02.585 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.585 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:02.585 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.585 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:20:02.585 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:02.586 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.158676753, 2024-09-17T21:20:02.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:20:02.586 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:02.586 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:02.586 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:02.586 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:02.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.586 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:02.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:02.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.586 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:02.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:02.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:02.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:20:02.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:20:02.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:02.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:20:02.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:20:02.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:02.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:20:02.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:20:02.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:02.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:20:02.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:20:02.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:02.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:20:02.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:20:02.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:02.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:20:02.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:20:02.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:02.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:20:02.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:20:02.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:02.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:20:02.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:20:02.590 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.590 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:02.590 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.590 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:20:02.590 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:02.590 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.950018359, 2024-09-17T21:20:02.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:20:02.590 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:02.590 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:02.590 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:02.591 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:02.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:02.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:02.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:02.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:02.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:02.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:02.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:20:02.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:20:02.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:02.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:20:02.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:20:02.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:02.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:20:02.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:20:02.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:02.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:20:02.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:20:02.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:02.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:20:02.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:20:02.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:02.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:20:02.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:20:02.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:02.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:20:02.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:20:02.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:02.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:02.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:02.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:20:02.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:20:02.594 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.594 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:02.594 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.594 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:02.595 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:02.595 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:02.596 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:20:02.597 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:02.589+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:02.691 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 0] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:20:02.691 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:20:02.691 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-17T21:20:02.691 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T21:20:02.691 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T21:20:02.691 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-17T21:20:02.696 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:02.693+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:02.717 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:02.718 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:02.718 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:02.718 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:02.731 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:20:02.817 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:02.813+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:02.824 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:02.824 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T21:20:02.824 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:02.895 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-17T21:20:02.898 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:02.891+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:02.900 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:02.900 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:02.900 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:02.900 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:02.900 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:02.914 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:20:02.959 INFO:teuthology.orchestra.run.smithi184.stdout:4471 read 0x6d78000 thru 0x6d7870b (0x70c bytes) 2024-09-17T21:20:02.960 INFO:teuthology.orchestra.run.smithi184.stdout:4473 punch from 0x66d0400 to 0x66d6800, (0x6400 bytes) 2024-09-17T21:20:02.977 INFO:teuthology.orchestra.run.smithi184.stdout:4474 read 0x4c96400 thru 0x4c9a1d6 (0x3dd7 bytes) 2024-09-17T21:20:03.001 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:02.997+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:03.066 INFO:teuthology.orchestra.run.smithi184.stdout:4476 trunc from 0xd1b1000 to 0x1eb3a00 2024-09-17T21:20:03.082 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:03.082 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:03.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:03.082 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:03.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.083 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:494 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:03.083 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:20:03.083 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 96.881203114000002, 2024-09-17T21:20:03.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046244995999999997, 2024-09-17T21:20:03.083 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.083 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.083 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.083 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.083 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:03.083 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:03.083 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 494 2024-09-17T21:20:03.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.083 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:20:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:20:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197329+0000", 2024-09-17T21:20:03.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.173e-06 2024-09-17T21:20:03.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197334+0000", 2024-09-17T21:20:03.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0499999999999999e-06 2024-09-17T21:20:03.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197352+0000", 2024-09-17T21:20:03.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7133e-05 2024-09-17T21:20:03.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197367+0000", 2024-09-17T21:20:03.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5123000000000001e-05 2024-09-17T21:20:03.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:20:03.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044989829000000002 2024-09-17T21:20:03.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242526+0000", 2024-09-17T21:20:03.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016968199999999999 2024-09-17T21:20:03.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.087 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.087 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243573+0000", 2024-09-17T21:20:03.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010470060000000001 2024-09-17T21:20:03.087 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.087 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.087 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.087 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:524 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:03.087 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:20:03.087 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 96.878457085999997, 2024-09-17T21:20:03.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045737414999999997, 2024-09-17T21:20:03.087 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.087 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.088 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.088 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.088 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:03.088 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:03.088 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 524 2024-09-17T21:20:03.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.088 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.088 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.088 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:20:03.088 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:20:03.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200076+0000", 2024-09-17T21:20:03.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:20:03.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200081+0000", 2024-09-17T21:20:03.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7030000000000003e-06 2024-09-17T21:20:03.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200099+0000", 2024-09-17T21:20:03.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7836999999999999e-05 2024-09-17T21:20:03.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200113+0000", 2024-09-17T21:20:03.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3570000000000001e-05 2024-09-17T21:20:03.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244866+0000", 2024-09-17T21:20:03.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044753573999999997 2024-09-17T21:20:03.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.091 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.091 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244972+0000", 2024-09-17T21:20:03.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106111 2024-09-17T21:20:03.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.091 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.091 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245812+0000", 2024-09-17T21:20:03.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083923999999999999 2024-09-17T21:20:03.091 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.091 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.091 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.092 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.092 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:525 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3670016~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:03.092 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:03.092 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 96.878305703999999, 2024-09-17T21:20:03.092 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046645697, 2024-09-17T21:20:03.092 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.092 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.092 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.092 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.092 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:03.092 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:03.092 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 525 2024-09-17T21:20:03.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.093 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.093 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.093 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:03.093 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.093 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.093 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:03.093 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.093 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:20:03.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:20:03.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200234+0000", 2024-09-17T21:20:03.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:20:03.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200254+0000", 2024-09-17T21:20:03.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9959999999999999e-05 2024-09-17T21:20:03.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200293+0000", 2024-09-17T21:20:03.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9755999999999997e-05 2024-09-17T21:20:03.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245871+0000", 2024-09-17T21:20:03.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045577871999999998 2024-09-17T21:20:03.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246030+0000", 2024-09-17T21:20:03.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158975 2024-09-17T21:20:03.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.096 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.096 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246871+0000", 2024-09-17T21:20:03.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084111699999999995 2024-09-17T21:20:03.096 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.096 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.096 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.096 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:03.096 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:03.096 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 96.878060028999997, 2024-09-17T21:20:03.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:20:03.097 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.097 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.097 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.097 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.097 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:03.097 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:03.097 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:20:03.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.097 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.097 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:03.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:03.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:20:03.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:20:03.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:20:03.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:20:03.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:20:03.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:20:03.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:20:03.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:20:03.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:20:03.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:20:03.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:20:03.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:20:03.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:20:03.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:20:03.100 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.100 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.100 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.101 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:03.101 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:03.101 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 96.877881346999999, 2024-09-17T21:20:03.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:20:03.101 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.101 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.101 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.101 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.101 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:03.101 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:03.102 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:20:03.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.102 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.102 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.102 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:03.102 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.102 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.102 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:03.102 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:20:03.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:20:03.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:20:03.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:20:03.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:20:03.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:20:03.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:20:03.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:20:03.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:20:03.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:20:03.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:20:03.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:20:03.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:20:03.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:20:03.105 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.105 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.105 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.105 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:03.105 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:03.106 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 96.877822073999994, 2024-09-17T21:20:03.106 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:20:03.106 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.106 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.106 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.106 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.106 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:03.106 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:03.106 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:20:03.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.106 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.106 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.107 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:03.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.107 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:03.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.107 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:20:03.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:20:03.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:20:03.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:20:03.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:20:03.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:20:03.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:20:03.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:20:03.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:20:03.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:20:03.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:20:03.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:20:03.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:20:03.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:20:03.110 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.110 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.110 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.110 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:03.110 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:03.110 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 96.877679878999999, 2024-09-17T21:20:03.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:20:03.110 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.110 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.110 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.110 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.111 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:03.111 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:03.111 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:20:03.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.111 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:03.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:03.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:20:03.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:20:03.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:20:03.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:20:03.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:20:03.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:20:03.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:20:03.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:20:03.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:20:03.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:20:03.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:20:03.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:20:03.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:20:03.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:20:03.114 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.114 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.114 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.114 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:03.114 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:03.115 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 96.877623955999994, 2024-09-17T21:20:03.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:20:03.115 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.115 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.115 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.115 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.115 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:03.115 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:03.115 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:20:03.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.115 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:03.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:03.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:20:03.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:20:03.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:20:03.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:20:03.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:20:03.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:20:03.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:20:03.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:20:03.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:20:03.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:20:03.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:20:03.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:20:03.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:20:03.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:20:03.119 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.119 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.119 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.119 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:03.119 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:03.119 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 96.877570339000002, 2024-09-17T21:20:03.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:20:03.119 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.119 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.119 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.119 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.120 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:03.120 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:03.120 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:20:03.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.120 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:03.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:03.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:20:03.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:20:03.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:20:03.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:20:03.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:20:03.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:20:03.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:20:03.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:20:03.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:20:03.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:20:03.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.123 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.123 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:20:03.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:20:03.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.123 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.123 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:20:03.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:20:03.123 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.123 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.123 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.124 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:20:03.124 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:03.124 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 46.568342096999999, 2024-09-17T21:20:03.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:20:03.124 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.124 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.124 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:20:03.124 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.124 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:20:03.124 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:20:03.124 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:20:03.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.124 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:03.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:03.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:20:03.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:20:03.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:20:03.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:20:03.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:20:03.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:20:03.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:20:03.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:20:03.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:20:03.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:20:03.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:03.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:20:03.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:20:03.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:20:03.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:20:03.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:20:03.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:20:03.128 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.128 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.128 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.128 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2597 3.1f 3:fb0e9615:::rbd_data.1152a8ae2019.000000000000000c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:03.129 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:03.129 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.239905668999999, 2024-09-17T21:20:03.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097064994000000002, 2024-09-17T21:20:03.129 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.129 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.129 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.129 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.129 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:03.129 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:03.129 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2597 2024-09-17T21:20:03.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.129 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:03.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:03.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838628+0000", 2024-09-17T21:20:03.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.187e-06 2024-09-17T21:20:03.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838654+0000", 2024-09-17T21:20:03.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:20:03.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838696+0000", 2024-09-17T21:20:03.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1226000000000001e-05 2024-09-17T21:20:03.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838715+0000", 2024-09-17T21:20:03.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9646000000000001e-05 2024-09-17T21:20:03.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838937+0000", 2024-09-17T21:20:03.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022163 2024-09-17T21:20:03.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.839284+0000", 2024-09-17T21:20:03.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034753799999999999 2024-09-17T21:20:03.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935691+0000", 2024-09-17T21:20:03.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.096406360999999996 2024-09-17T21:20:03.133 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.133 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.133 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.133 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2641 3.12 3:488a729f:::rbd_data.1152a8ae2019.0000000000000022:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:03.133 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:03.133 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.236448895999999, 2024-09-17T21:20:03.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098280953000000004, 2024-09-17T21:20:03.133 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.133 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.133 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.133 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.134 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:03.134 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:03.134 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2641 2024-09-17T21:20:03.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.134 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:03.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:03.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842083+0000", 2024-09-17T21:20:03.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5300000000000002e-07 2024-09-17T21:20:03.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842088+0000", 2024-09-17T21:20:03.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:20:03.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842105+0000", 2024-09-17T21:20:03.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6773e-05 2024-09-17T21:20:03.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842119+0000", 2024-09-17T21:20:03.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:20:03.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842320+0000", 2024-09-17T21:20:03.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020052700000000001 2024-09-17T21:20:03.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842845+0000", 2024-09-17T21:20:03.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052508600000000002 2024-09-17T21:20:03.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.137 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.137 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940363+0000", 2024-09-17T21:20:03.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097518795000000005 2024-09-17T21:20:03.137 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.137 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.137 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.137 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2674 3.12 3:4c87e3ac:::rbd_data.1152a8ae2019.0000000000000033:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:03.137 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:03.138 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.236086259, 2024-09-17T21:20:03.138 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098025125000000005, 2024-09-17T21:20:03.138 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.138 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.138 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:03.138 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.138 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:03.138 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:03.138 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2674 2024-09-17T21:20:03.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.138 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.138 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.138 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.138 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:03.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:03.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842447+0000", 2024-09-17T21:20:03.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3999999999999999e-06 2024-09-17T21:20:03.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842453+0000", 2024-09-17T21:20:03.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:20:03.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.140 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842472+0000", 2024-09-17T21:20:03.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9006000000000001e-05 2024-09-17T21:20:03.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.140 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842487+0000", 2024-09-17T21:20:03.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4866e-05 2024-09-17T21:20:03.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843594+0000", 2024-09-17T21:20:03.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011073249999999999 2024-09-17T21:20:03.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843668+0000", 2024-09-17T21:20:03.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3188000000000005e-05 2024-09-17T21:20:03.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:03.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843893+0000", 2024-09-17T21:20:03.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022503399999999999 2024-09-17T21:20:03.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:03.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850439+0000", 2024-09-17T21:20:03.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065467880000000004 2024-09-17T21:20:03.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:03.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940409+0000", 2024-09-17T21:20:03.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089969511000000002 2024-09-17T21:20:03.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:03.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940444+0000", 2024-09-17T21:20:03.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5345999999999997e-05 2024-09-17T21:20:03.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.143 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.143 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940470+0000", 2024-09-17T21:20:03.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5964000000000001e-05 2024-09-17T21:20:03.143 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.143 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.143 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.143 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2707 3.1f 3:f805e65e:::rbd_data.1152a8ae2019.0000000000000049:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:03.143 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:03.143 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.231039597999999, 2024-09-17T21:20:03.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088327300999999997, 2024-09-17T21:20:03.144 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.144 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.144 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.144 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.144 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:03.144 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:03.144 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2707 2024-09-17T21:20:03.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.144 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:03.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:03.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847494+0000", 2024-09-17T21:20:03.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:20:03.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.146 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847504+0000", 2024-09-17T21:20:03.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:20:03.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.146 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847526+0000", 2024-09-17T21:20:03.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1976000000000001e-05 2024-09-17T21:20:03.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.146 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847541+0000", 2024-09-17T21:20:03.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5277e-05 2024-09-17T21:20:03.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.147 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856142+0000", 2024-09-17T21:20:03.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086006710000000007 2024-09-17T21:20:03.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.147 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856435+0000", 2024-09-17T21:20:03.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029301499999999999 2024-09-17T21:20:03.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.147 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935819+0000", 2024-09-17T21:20:03.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079384448999999996 2024-09-17T21:20:03.148 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.148 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.148 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.148 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2730 3.12 3:48b3422a:::rbd_data.1152a8ae2019.0000000000000057:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:03.148 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:03.148 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.221846699, 2024-09-17T21:20:03.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083879975999999995, 2024-09-17T21:20:03.148 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.148 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.148 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.148 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.149 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:03.149 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:03.149 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2730 2024-09-17T21:20:03.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.149 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:03.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:03.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.150 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.150 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856687+0000", 2024-09-17T21:20:03.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:20:03.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.150 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.150 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856697+0000", 2024-09-17T21:20:03.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0207000000000001e-05 2024-09-17T21:20:03.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.150 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.150 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856808+0000", 2024-09-17T21:20:03.151 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011071200000000001 2024-09-17T21:20:03.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.151 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.151 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856828+0000", 2024-09-17T21:20:03.151 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0219000000000001e-05 2024-09-17T21:20:03.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.151 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.151 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856907+0000", 2024-09-17T21:20:03.151 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9661999999999997e-05 2024-09-17T21:20:03.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.151 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.857019+0000", 2024-09-17T21:20:03.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011148499999999999 2024-09-17T21:20:03.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940565+0000", 2024-09-17T21:20:03.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083545791999999994 2024-09-17T21:20:03.152 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.152 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.152 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.152 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2743 3.12 3:4d036e62:::rbd_data.1152a8ae2019.000000000000005f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:03.152 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:03.153 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.217901198, 2024-09-17T21:20:03.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080059962999999998, 2024-09-17T21:20:03.153 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.153 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.153 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.153 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.153 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:03.153 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:03.153 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2743 2024-09-17T21:20:03.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.153 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:03.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:03.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860631+0000", 2024-09-17T21:20:03.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:20:03.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860636+0000", 2024-09-17T21:20:03.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5279999999999997e-06 2024-09-17T21:20:03.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.155 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860649+0000", 2024-09-17T21:20:03.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-17T21:20:03.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.155 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860655+0000", 2024-09-17T21:20:03.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:20:03.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.156 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.156 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860731+0000", 2024-09-17T21:20:03.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5920000000000005e-05 2024-09-17T21:20:03.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.156 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.156 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.861070+0000", 2024-09-17T21:20:03.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338535 2024-09-17T21:20:03.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.156 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.156 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940690+0000", 2024-09-17T21:20:03.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079620503999999995 2024-09-17T21:20:03.157 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.157 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.157 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.157 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.157 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3378 3.12 3:4832a51b:::rbd_data.1152a8ae2019.000000000000006c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:03.157 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:03.157 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.189537348000002, 2024-09-17T21:20:03.157 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051812516000000003, 2024-09-17T21:20:03.157 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.157 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.157 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:03.157 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.157 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:03.158 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:03.158 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3378 2024-09-17T21:20:03.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.158 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:03.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:03.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888995+0000", 2024-09-17T21:20:03.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:20:03.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889006+0000", 2024-09-17T21:20:03.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1012999999999999e-05 2024-09-17T21:20:03.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889032+0000", 2024-09-17T21:20:03.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6441e-05 2024-09-17T21:20:03.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889040+0000", 2024-09-17T21:20:03.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0639999999999994e-06 2024-09-17T21:20:03.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889278+0000", 2024-09-17T21:20:03.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023785899999999999 2024-09-17T21:20:03.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889408+0000", 2024-09-17T21:20:03.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129219 2024-09-17T21:20:03.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.161 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:03.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889723+0000", 2024-09-17T21:20:03.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000315264 2024-09-17T21:20:03.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.161 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:03.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.891972+0000", 2024-09-17T21:20:03.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002249633 2024-09-17T21:20:03.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:03.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940733+0000", 2024-09-17T21:20:03.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048760626000000001 2024-09-17T21:20:03.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:03.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940785+0000", 2024-09-17T21:20:03.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1906000000000001e-05 2024-09-17T21:20:03.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940807+0000", 2024-09-17T21:20:03.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1554999999999999e-05 2024-09-17T21:20:03.162 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.163 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.163 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.163 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3379 3.12 3:4e83fd71:::rbd_data.1152a8ae2019.000000000000006d:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:03.163 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:03.163 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.189453141000001, 2024-09-17T21:20:03.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051804144000000003, 2024-09-17T21:20:03.163 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.163 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.163 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:03.163 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.163 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:03.163 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:03.164 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3379 2024-09-17T21:20:03.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.164 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:03.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:03.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889079+0000", 2024-09-17T21:20:03.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1800000000000004e-07 2024-09-17T21:20:03.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889083+0000", 2024-09-17T21:20:03.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:20:03.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889094+0000", 2024-09-17T21:20:03.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1562999999999999e-05 2024-09-17T21:20:03.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889102+0000", 2024-09-17T21:20:03.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.401e-06 2024-09-17T21:20:03.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:20:03.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014279259999999999 2024-09-17T21:20:03.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890609+0000", 2024-09-17T21:20:03.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9566000000000004e-05 2024-09-17T21:20:03.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:03.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890789+0000", 2024-09-17T21:20:03.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017949700000000001 2024-09-17T21:20:03.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:03.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.893929+0000", 2024-09-17T21:20:03.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003140538 2024-09-17T21:20:03.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:03.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940834+0000", 2024-09-17T21:20:03.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046904811999999997 2024-09-17T21:20:03.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:03.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940860+0000", 2024-09-17T21:20:03.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5341999999999999e-05 2024-09-17T21:20:03.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940882+0000", 2024-09-17T21:20:03.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2878999999999999e-05 2024-09-17T21:20:03.168 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.169 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.169 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.169 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:1613 3.1e 3:7d13ef1b:::rbd_data.115266d0b9f2.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 5666816~34816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-17T21:20:03.169 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:48.315514+0000", 2024-09-17T21:20:03.169 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.763017361999999, 2024-09-17T21:20:03.169 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045611719000000002, 2024-09-17T21:20:03.169 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.169 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.169 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:03.169 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.169 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:20:03.169 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:20:03.170 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 1613 2024-09-17T21:20:03.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.170 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.170 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.170 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.315514+0000", 2024-09-17T21:20:03.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.170 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.170 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.315514+0000", 2024-09-17T21:20:03.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.315516+0000", 2024-09-17T21:20:03.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.497e-06 2024-09-17T21:20:03.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.330451+0000", 2024-09-17T21:20:03.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014935857 2024-09-17T21:20:03.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.330479+0000", 2024-09-17T21:20:03.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7151000000000001e-05 2024-09-17T21:20:03.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.330490+0000", 2024-09-17T21:20:03.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1250000000000001e-05 2024-09-17T21:20:03.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.330560+0000", 2024-09-17T21:20:03.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9901999999999996e-05 2024-09-17T21:20:03.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.330688+0000", 2024-09-17T21:20:03.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000128608 2024-09-17T21:20:03.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:20:03.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.331269+0000", 2024-09-17T21:20:03.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00058052799999999997 2024-09-17T21:20:03.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:03.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.350343+0000", 2024-09-17T21:20:03.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019074457 2024-09-17T21:20:03.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:03.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.361030+0000", 2024-09-17T21:20:03.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010686357 2024-09-17T21:20:03.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:03.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.361084+0000", 2024-09-17T21:20:03.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4097000000000001e-05 2024-09-17T21:20:03.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.361126+0000", 2024-09-17T21:20:03.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2014999999999997e-05 2024-09-17T21:20:03.174 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.175 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.175 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.175 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:2644 3.1e 3:79615585:::rbd_data.115266d0b9f2.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2475520~14848,write 3106304~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:20:03.175 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:03.175 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 13.566824737999999, 2024-09-17T21:20:03.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050733484000000002, 2024-09-17T21:20:03.175 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.175 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.175 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:03.175 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.175 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:20:03.175 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:20:03.176 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2644 2024-09-17T21:20:03.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.176 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.176 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.176 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:03.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.176 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.176 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:03.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.177 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.177 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511708+0000", 2024-09-17T21:20:03.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-17T21:20:03.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.177 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.177 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522196+0000", 2024-09-17T21:20:03.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010487712999999999 2024-09-17T21:20:03.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.177 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.177 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522223+0000", 2024-09-17T21:20:03.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7064999999999999e-05 2024-09-17T21:20:03.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522236+0000", 2024-09-17T21:20:03.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2931e-05 2024-09-17T21:20:03.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522348+0000", 2024-09-17T21:20:03.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111906 2024-09-17T21:20:03.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522554+0000", 2024-09-17T21:20:03.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020632699999999999 2024-09-17T21:20:03.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:20:03.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.523185+0000", 2024-09-17T21:20:03.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063081400000000005 2024-09-17T21:20:03.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:03.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.547011+0000", 2024-09-17T21:20:03.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023825570000000001 2024-09-17T21:20:03.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:03.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562268+0000", 2024-09-17T21:20:03.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015257514 2024-09-17T21:20:03.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:03.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562367+0000", 2024-09-17T21:20:03.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8678000000000006e-05 2024-09-17T21:20:03.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562440+0000", 2024-09-17T21:20:03.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3337000000000006e-05 2024-09-17T21:20:03.180 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.180 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.181 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.181 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.181 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.181 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:03.182 INFO:teuthology.orchestra.run.smithi184.stdout:4477 write 0xb022400 thru 0xb02e400 (0xc001 bytes) 2024-09-17T21:20:03.182 INFO:teuthology.orchestra.run.smithi184.stdout:4479 read 0x3876200 thru 0x38859a6 (0xf7a7 bytes) 2024-09-17T21:20:03.182 INFO:teuthology.orchestra.run.smithi184.stdout:4480 trunc from 0xb02e600 to 0x5c13a00 2024-09-17T21:20:03.183 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:20:03.183 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:03.175+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:03.227 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-17T21:20:03.229 INFO:teuthology.orchestra.run.smithi184.stdout:4482 trunc from 0x5c13a00 to 0xce3d400 2024-09-17T21:20:03.257 INFO:teuthology.orchestra.run.smithi184.stdout:4484 write 0x5d42a00 thru 0x5d43b4d (0x114e bytes) 2024-09-17T21:20:03.258 INFO:teuthology.orchestra.run.smithi184.stdout:4486 read 0xd88a00 thru 0xd8ea42 (0x6043 bytes) 2024-09-17T21:20:03.259 INFO:teuthology.orchestra.run.smithi184.stdout:4487 read 0x1dd5a00 thru 0x1de1b7a (0xc17b bytes) 2024-09-17T21:20:03.260 INFO:teuthology.orchestra.run.smithi184.stdout:4488 write 0xc84e800 thru 0xc85a625 (0xbe26 bytes) 2024-09-17T21:20:03.261 INFO:teuthology.orchestra.run.smithi184.stdout:4489 write 0x8c33a00 thru 0x8c42a2b (0xf02c bytes) 2024-09-17T21:20:03.261 INFO:teuthology.orchestra.run.smithi184.stdout:4490 write 0x62ce800 thru 0x62daf9a (0xc79b bytes) 2024-09-17T21:20:03.261 INFO:teuthology.orchestra.run.smithi184.stdout:4491 write 0x9ea600 thru 0x9f36a3 (0x90a4 bytes) 2024-09-17T21:20:03.262 INFO:teuthology.orchestra.run.smithi184.stdout:4492 write 0x1f75a00 thru 0x1f7d340 (0x7941 bytes) 2024-09-17T21:20:03.283 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:03.279+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:03.305 INFO:teuthology.orchestra.run.smithi184.stdout:4493 write 0xdacb800 thru 0xdad937e (0xdb7f bytes) 2024-09-17T21:20:03.309 INFO:teuthology.orchestra.run.smithi184.stdout:4495 read 0x1601000 thru 0x160bf2e (0xaf2f bytes) 2024-09-17T21:20:03.309 INFO:teuthology.orchestra.run.smithi184.stdout:4500 write 0x5eb2e00 thru 0x5ebe1ca (0xb3cb bytes) 2024-09-17T21:20:03.309 INFO:teuthology.orchestra.run.smithi184.stdout:4508 write 0xa7fe200 thru 0xa809649 (0xb44a bytes) 2024-09-17T21:20:03.310 INFO:teuthology.orchestra.run.smithi184.stdout:4509 punch from 0x2faf000 to 0x2fb1600, (0x2600 bytes) 2024-09-17T21:20:03.317 INFO:teuthology.orchestra.run.smithi184.stdout:4510 trunc from 0xdad9400 to 0xb5ab800 2024-09-17T21:20:03.354 INFO:teuthology.orchestra.run.smithi184.stdout:4511 read 0x2347200 thru 0x234dc24 (0x6a25 bytes) 2024-09-17T21:20:03.356 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:03.356 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:03.356 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:03.356 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:03.357 INFO:teuthology.orchestra.run.smithi184.stdout:4513 read 0xb276e00 thru 0xb27a13d (0x333e bytes) 2024-09-17T21:20:03.357 INFO:teuthology.orchestra.run.smithi184.stdout:4515 read 0x66f6a00 thru 0x67033a6 (0xc9a7 bytes) 2024-09-17T21:20:03.357 INFO:teuthology.orchestra.run.smithi184.stdout:4516 read 0x8c400 thru 0x99141 (0xcd42 bytes) 2024-09-17T21:20:03.358 INFO:teuthology.orchestra.run.smithi184.stdout:4517 write 0x9e03000 thru 0x9e0d9b3 (0xa9b4 bytes) 2024-09-17T21:20:03.359 INFO:teuthology.orchestra.run.smithi184.stdout:4518 write 0x7e22c00 thru 0x7e26727 (0x3b28 bytes) 2024-09-17T21:20:03.359 INFO:teuthology.orchestra.run.smithi184.stdout:4520 write 0xa2ff400 thru 0xa30222b (0x2e2c bytes) 2024-09-17T21:20:03.359 INFO:teuthology.orchestra.run.smithi184.stdout:4522 punch from 0x8927000 to 0x8929c00, (0x2c00 bytes) 2024-09-17T21:20:03.360 INFO:teuthology.orchestra.run.smithi184.stdout:4524 write 0x45f9600 thru 0x46028cc (0x92cd bytes) 2024-09-17T21:20:03.361 INFO:teuthology.orchestra.run.smithi184.stdout:4525 write 0x3567c00 thru 0x35757ba (0xdbbb bytes) 2024-09-17T21:20:03.361 INFO:teuthology.orchestra.run.smithi184.stdout:4526 read 0x6d41800 thru 0x6d4d35b (0xbb5c bytes) 2024-09-17T21:20:03.370 INFO:teuthology.orchestra.run.smithi184.stdout:4527 trunc from 0xb5ab800 to 0xe4f1c00 2024-09-17T21:20:03.373 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:20:03.403 INFO:teuthology.orchestra.run.smithi184.stdout:4528 write 0x2df9a00 thru 0x2dff375 (0x5976 bytes) 2024-09-17T21:20:03.407 INFO:teuthology.orchestra.run.smithi184.stdout:4529 flatten 2024-09-17T21:20:03.416 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:03.413+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:03.458 INFO:teuthology.orchestra.run.smithi184.stdout:4530 punch from 0x65d7800 to 0x65df600, (0x7e00 bytes) 2024-09-17T21:20:03.459 INFO:teuthology.orchestra.run.smithi184.stdout:4531 write 0x7c4ec00 thru 0x7c4fa96 (0xe97 bytes) 2024-09-17T21:20:03.460 INFO:teuthology.orchestra.run.smithi184.stdout:4532 read 0x8626600 thru 0x8627c35 (0x1636 bytes) 2024-09-17T21:20:03.460 INFO:teuthology.orchestra.run.smithi184.stdout:4534 write 0x3b46800 thru 0x3b4a8c9 (0x40ca bytes) 2024-09-17T21:20:03.460 INFO:teuthology.orchestra.run.smithi184.stdout:4535 write 0x86e1200 thru 0x86e6229 (0x502a bytes) 2024-09-17T21:20:03.460 INFO:teuthology.orchestra.run.smithi184.stdout:4536 read 0x747d800 thru 0x748b748 (0xdf49 bytes) 2024-09-17T21:20:03.460 INFO:teuthology.orchestra.run.smithi184.stdout:4537 read 0x8330200 thru 0x8339e64 (0x9c65 bytes) 2024-09-17T21:20:03.460 INFO:teuthology.orchestra.run.smithi184.stdout:4538 read 0x40b3e00 thru 0x40bc28e (0x848f bytes) 2024-09-17T21:20:03.460 INFO:teuthology.orchestra.run.smithi184.stdout:4540 clone 29 order 22 su 16384 sc 14 2024-09-17T21:20:03.485 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:03.481+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:03.498 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:03.498 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:03.498 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:03.498 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:03.498 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:03.513 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:20:03.586 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:03.579+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:03.685 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:543 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:20:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.481332952000002, 2024-09-17T21:20:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047564097999999999, 2024-09-17T21:20:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 543 2024-09-17T21:20:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:20:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:20:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:20:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9699999999999995e-07 2024-09-17T21:20:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199677+0000", 2024-09-17T21:20:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:20:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199690+0000", 2024-09-17T21:20:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2977e-05 2024-09-17T21:20:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199705+0000", 2024-09-17T21:20:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4656e-05 2024-09-17T21:20:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246307+0000", 2024-09-17T21:20:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046601780000000002 2024-09-17T21:20:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246475+0000", 2024-09-17T21:20:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016834200000000001 2024-09-17T21:20:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247227+0000", 2024-09-17T21:20:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00075173299999999998 2024-09-17T21:20:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:544 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:20:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.481274542999998, 2024-09-17T21:20:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049283559999999997, 2024-09-17T21:20:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 544 2024-09-17T21:20:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:20:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:20:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199722+0000", 2024-09-17T21:20:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.08e-06 2024-09-17T21:20:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199738+0000", 2024-09-17T21:20:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5943999999999998e-05 2024-09-17T21:20:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199748+0000", 2024-09-17T21:20:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0193e-05 2024-09-17T21:20:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199756+0000", 2024-09-17T21:20:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4200000000000001e-06 2024-09-17T21:20:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247311+0000", 2024-09-17T21:20:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047555300000000002 2024-09-17T21:20:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247571+0000", 2024-09-17T21:20:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025996999999999999 2024-09-17T21:20:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249005+0000", 2024-09-17T21:20:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001433653 2024-09-17T21:20:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:549 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:20:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.481228919000003, 2024-09-17T21:20:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048464528999999999, 2024-09-17T21:20:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 549 2024-09-17T21:20:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:20:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:20:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:20:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:20:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199770+0000", 2024-09-17T21:20:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9969999999999999e-06 2024-09-17T21:20:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199780+0000", 2024-09-17T21:20:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0003000000000001e-05 2024-09-17T21:20:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199793+0000", 2024-09-17T21:20:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2074e-05 2024-09-17T21:20:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247289+0000", 2024-09-17T21:20:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047496691000000001 2024-09-17T21:20:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247437+0000", 2024-09-17T21:20:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014807899999999999 2024-09-17T21:20:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248231+0000", 2024-09-17T21:20:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00079390899999999998 2024-09-17T21:20:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:558 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:20:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.481188406000001, 2024-09-17T21:20:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049709821000000001, 2024-09-17T21:20:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 558 2024-09-17T21:20:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:20:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:20:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199809+0000", 2024-09-17T21:20:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.333e-06 2024-09-17T21:20:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199814+0000", 2024-09-17T21:20:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0930000000000002e-06 2024-09-17T21:20:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199831+0000", 2024-09-17T21:20:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7580000000000001e-05 2024-09-17T21:20:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199860+0000", 2024-09-17T21:20:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8609999999999999e-05 2024-09-17T21:20:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249106+0000", 2024-09-17T21:20:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049246545000000003 2024-09-17T21:20:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249274+0000", 2024-09-17T21:20:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016806699999999999 2024-09-17T21:20:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249517+0000", 2024-09-17T21:20:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024259300000000001 2024-09-17T21:20:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:559 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:20:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.481069043999995, 2024-09-17T21:20:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049990694000000002, 2024-09-17T21:20:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 559 2024-09-17T21:20:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:20:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:20:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199928+0000", 2024-09-17T21:20:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:20:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199935+0000", 2024-09-17T21:20:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5429999999999999e-06 2024-09-17T21:20:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199954+0000", 2024-09-17T21:20:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.912e-05 2024-09-17T21:20:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199969+0000", 2024-09-17T21:20:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5305999999999999e-05 2024-09-17T21:20:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249604+0000", 2024-09-17T21:20:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049634931 2024-09-17T21:20:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249738+0000", 2024-09-17T21:20:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013411599999999999 2024-09-17T21:20:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249917+0000", 2024-09-17T21:20:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179111 2024-09-17T21:20:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:560 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:20:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.481004627999994, 2024-09-17T21:20:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050173059999999998, 2024-09-17T21:20:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 560 2024-09-17T21:20:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:20:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:20:03.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199992+0000", 2024-09-17T21:20:03.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.517e-06 2024-09-17T21:20:03.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199998+0000", 2024-09-17T21:20:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1460000000000004e-06 2024-09-17T21:20:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:20:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7407000000000001e-05 2024-09-17T21:20:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200029+0000", 2024-09-17T21:20:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3499999999999999e-05 2024-09-17T21:20:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249968+0000", 2024-09-17T21:20:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049939799 2024-09-17T21:20:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250055+0000", 2024-09-17T21:20:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6731999999999996e-05 2024-09-17T21:20:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250164+0000", 2024-09-17T21:20:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108959 2024-09-17T21:20:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:20:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.871214918999996, 2024-09-17T21:20:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:20:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:20:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:20:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:20:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:20:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:20:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:20:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:20:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:20:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:20:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:20:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:20:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:20:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:20:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:20:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:20:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:20:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:20:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:20:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:20:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:20:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:20:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:20:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.316113133000002, 2024-09-17T21:20:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:20:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:20:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:20:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:20:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:20:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:20:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:20:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:20:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:20:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:20:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:20:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:20:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:20:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:20:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:20:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:20:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:20:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:20:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:20:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:20:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:20:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:20:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:20:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:20:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:20:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:20:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:20:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:20:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:20:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3777 3.c 3:302136a0:::rbd_data.1152a8ae2019.000000000000004b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.902125+0000", 2024-09-17T21:20:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.778870507000001, 2024-09-17T21:20:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048610548000000003, 2024-09-17T21:20:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3777 2024-09-17T21:20:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902125+0000", 2024-09-17T21:20:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902125+0000", 2024-09-17T21:20:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902126+0000", 2024-09-17T21:20:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:20:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902138+0000", 2024-09-17T21:20:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1994000000000001e-05 2024-09-17T21:20:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902144+0000", 2024-09-17T21:20:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2820000000000002e-06 2024-09-17T21:20:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902149+0000", 2024-09-17T21:20:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.135e-06 2024-09-17T21:20:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.942910+0000", 2024-09-17T21:20:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.040761190000000003 2024-09-17T21:20:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.942986+0000", 2024-09-17T21:20:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6083999999999999e-05 2024-09-17T21:20:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.943319+0000", 2024-09-17T21:20:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00033264500000000001 2024-09-17T21:20:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944690+0000", 2024-09-17T21:20:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0013710549999999999 2024-09-17T21:20:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.950637+0000", 2024-09-17T21:20:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059473190000000004 2024-09-17T21:20:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.950696+0000", 2024-09-17T21:20:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8449999999999998e-05 2024-09-17T21:20:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.950736+0000", 2024-09-17T21:20:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9971000000000001e-05 2024-09-17T21:20:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3778 3.c 3:375c1b23:::rbd_data.1152a8ae2019.0000000000000052:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.902156+0000", 2024-09-17T21:20:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.778839153, 2024-09-17T21:20:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049052074000000001, 2024-09-17T21:20:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3778 2024-09-17T21:20:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902156+0000", 2024-09-17T21:20:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902156+0000", 2024-09-17T21:20:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902157+0000", 2024-09-17T21:20:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5299999999999999e-07 2024-09-17T21:20:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902160+0000", 2024-09-17T21:20:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6359999999999998e-06 2024-09-17T21:20:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902165+0000", 2024-09-17T21:20:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9440000000000002e-06 2024-09-17T21:20:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902170+0000", 2024-09-17T21:20:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:20:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.943704+0000", 2024-09-17T21:20:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041533516999999999 2024-09-17T21:20:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.943760+0000", 2024-09-17T21:20:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6552000000000003e-05 2024-09-17T21:20:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.943879+0000", 2024-09-17T21:20:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118347 2024-09-17T21:20:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944733+0000", 2024-09-17T21:20:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00085437299999999996 2024-09-17T21:20:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951158+0000", 2024-09-17T21:20:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064243520000000004 2024-09-17T21:20:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951184+0000", 2024-09-17T21:20:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6913000000000001e-05 2024-09-17T21:20:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951209+0000", 2024-09-17T21:20:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4104000000000002e-05 2024-09-17T21:20:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3785 3.c 3:3414a45b:::rbd_data.1152a8ae2019.0000000000000074:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.902292+0000", 2024-09-17T21:20:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.778703629999999, 2024-09-17T21:20:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049058004000000002, 2024-09-17T21:20:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3785 2024-09-17T21:20:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902292+0000", 2024-09-17T21:20:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902292+0000", 2024-09-17T21:20:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902292+0000", 2024-09-17T21:20:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:20:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902299+0000", 2024-09-17T21:20:03.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1210000000000001e-06 2024-09-17T21:20:03.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.742 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902305+0000", 2024-09-17T21:20:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6069999999999998e-06 2024-09-17T21:20:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902310+0000", 2024-09-17T21:20:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7509999999999999e-06 2024-09-17T21:20:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944093+0000", 2024-09-17T21:20:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041783582999999999 2024-09-17T21:20:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944159+0000", 2024-09-17T21:20:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5524999999999994e-05 2024-09-17T21:20:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944281+0000", 2024-09-17T21:20:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121573 2024-09-17T21:20:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.945107+0000", 2024-09-17T21:20:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00082616700000000005 2024-09-17T21:20:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951260+0000", 2024-09-17T21:20:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061530789999999997 2024-09-17T21:20:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951311+0000", 2024-09-17T21:20:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1678000000000002e-05 2024-09-17T21:20:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951350+0000", 2024-09-17T21:20:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8525000000000001e-05 2024-09-17T21:20:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3880 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.912848+0000", 2024-09-17T21:20:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.768147517999999, 2024-09-17T21:20:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046791626000000003, 2024-09-17T21:20:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3880 2024-09-17T21:20:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912848+0000", 2024-09-17T21:20:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912848+0000", 2024-09-17T21:20:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912849+0000", 2024-09-17T21:20:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2600000000000002e-07 2024-09-17T21:20:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912851+0000", 2024-09-17T21:20:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.475e-06 2024-09-17T21:20:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912857+0000", 2024-09-17T21:20:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1009999999999997e-06 2024-09-17T21:20:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912862+0000", 2024-09-17T21:20:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2639999999999999e-06 2024-09-17T21:20:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959555+0000", 2024-09-17T21:20:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046692589 2024-09-17T21:20:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959586+0000", 2024-09-17T21:20:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0978000000000001e-05 2024-09-17T21:20:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959640+0000", 2024-09-17T21:20:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3693000000000002e-05 2024-09-17T21:20:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3881 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.912869+0000", 2024-09-17T21:20:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.768126394999999, 2024-09-17T21:20:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046894383999999997, 2024-09-17T21:20:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3881 2024-09-17T21:20:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912869+0000", 2024-09-17T21:20:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912869+0000", 2024-09-17T21:20:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912870+0000", 2024-09-17T21:20:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:20:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913014+0000", 2024-09-17T21:20:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000144473 2024-09-17T21:20:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913025+0000", 2024-09-17T21:20:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1341e-05 2024-09-17T21:20:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913033+0000", 2024-09-17T21:20:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.17e-06 2024-09-17T21:20:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959677+0000", 2024-09-17T21:20:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046644536 2024-09-17T21:20:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959709+0000", 2024-09-17T21:20:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2348000000000003e-05 2024-09-17T21:20:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959764+0000", 2024-09-17T21:20:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4120999999999999e-05 2024-09-17T21:20:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:14003 3.4 3.4f4be604 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.912905+0000", 2024-09-17T21:20:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.768090555000001, 2024-09-17T21:20:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046758252, 2024-09-17T21:20:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:20:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:20:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 14003 2024-09-17T21:20:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912905+0000", 2024-09-17T21:20:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912905+0000", 2024-09-17T21:20:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912906+0000", 2024-09-17T21:20:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.091e-06 2024-09-17T21:20:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912911+0000", 2024-09-17T21:20:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9409999999999998e-06 2024-09-17T21:20:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912920+0000", 2024-09-17T21:20:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3530000000000001e-06 2024-09-17T21:20:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912927+0000", 2024-09-17T21:20:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9979999999999997e-06 2024-09-17T21:20:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959659+0000", 2024-09-17T21:20:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046731111999999998 2024-09-17T21:20:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959663+0000", 2024-09-17T21:20:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7570000000000004e-06 2024-09-17T21:20:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3882 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913045+0000", 2024-09-17T21:20:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.767951039, 2024-09-17T21:20:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046829914, 2024-09-17T21:20:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3882 2024-09-17T21:20:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913045+0000", 2024-09-17T21:20:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913045+0000", 2024-09-17T21:20:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913045+0000", 2024-09-17T21:20:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1200000000000002e-07 2024-09-17T21:20:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913048+0000", 2024-09-17T21:20:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.179e-06 2024-09-17T21:20:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913055+0000", 2024-09-17T21:20:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0269999999999998e-06 2024-09-17T21:20:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913061+0000", 2024-09-17T21:20:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5709999999999997e-06 2024-09-17T21:20:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959782+0000", 2024-09-17T21:20:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046720708 2024-09-17T21:20:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959813+0000", 2024-09-17T21:20:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1358e-05 2024-09-17T21:20:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959874+0000", 2024-09-17T21:20:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1359000000000007e-05 2024-09-17T21:20:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3883 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913069+0000", 2024-09-17T21:20:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.767926671000001, 2024-09-17T21:20:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046905401999999999, 2024-09-17T21:20:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3883 2024-09-17T21:20:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913069+0000", 2024-09-17T21:20:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913069+0000", 2024-09-17T21:20:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913069+0000", 2024-09-17T21:20:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:20:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913072+0000", 2024-09-17T21:20:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6390000000000001e-06 2024-09-17T21:20:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913079+0000", 2024-09-17T21:20:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2640000000000002e-06 2024-09-17T21:20:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913085+0000", 2024-09-17T21:20:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4249999999999997e-06 2024-09-17T21:20:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959893+0000", 2024-09-17T21:20:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046808728000000001 2024-09-17T21:20:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959926+0000", 2024-09-17T21:20:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2437999999999997e-05 2024-09-17T21:20:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959974+0000", 2024-09-17T21:20:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8516000000000001e-05 2024-09-17T21:20:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3884 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913092+0000", 2024-09-17T21:20:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.767904028, 2024-09-17T21:20:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047065517000000001, 2024-09-17T21:20:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3884 2024-09-17T21:20:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913092+0000", 2024-09-17T21:20:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913092+0000", 2024-09-17T21:20:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913670+0000", 2024-09-17T21:20:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00057830399999999997 2024-09-17T21:20:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913680+0000", 2024-09-17T21:20:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0426e-05 2024-09-17T21:20:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913691+0000", 2024-09-17T21:20:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0309000000000001e-05 2024-09-17T21:20:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913698+0000", 2024-09-17T21:20:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1760000000000004e-06 2024-09-17T21:20:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.960056+0000", 2024-09-17T21:20:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046358411000000002 2024-09-17T21:20:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.960089+0000", 2024-09-17T21:20:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2588e-05 2024-09-17T21:20:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.960157+0000", 2024-09-17T21:20:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8303e-05 2024-09-17T21:20:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:14004 3.4 3.4f4be604 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913136+0000", 2024-09-17T21:20:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.767859791999999, 2024-09-17T21:20:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046862804000000001, 2024-09-17T21:20:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:20:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:20:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 14004 2024-09-17T21:20:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913136+0000", 2024-09-17T21:20:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913136+0000", 2024-09-17T21:20:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913447+0000", 2024-09-17T21:20:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031107200000000002 2024-09-17T21:20:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913460+0000", 2024-09-17T21:20:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2996999999999999e-05 2024-09-17T21:20:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913470+0000", 2024-09-17T21:20:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0037999999999999e-05 2024-09-17T21:20:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913477+0000", 2024-09-17T21:20:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:20:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959994+0000", 2024-09-17T21:20:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046517599999999999 2024-09-17T21:20:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959999+0000", 2024-09-17T21:20:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1400000000000002e-06 2024-09-17T21:20:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3885 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913710+0000", 2024-09-17T21:20:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.767285345000001, 2024-09-17T21:20:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047449947999999999, 2024-09-17T21:20:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3885 2024-09-17T21:20:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913710+0000", 2024-09-17T21:20:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913710+0000", 2024-09-17T21:20:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.917352+0000", 2024-09-17T21:20:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0036412950000000001 2024-09-17T21:20:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.917357+0000", 2024-09-17T21:20:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4879999999999998e-06 2024-09-17T21:20:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.918061+0000", 2024-09-17T21:20:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00070362899999999999 2024-09-17T21:20:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.918069+0000", 2024-09-17T21:20:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4530000000000002e-06 2024-09-17T21:20:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.960956+0000", 2024-09-17T21:20:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042886951999999999 2024-09-17T21:20:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.961036+0000", 2024-09-17T21:20:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0010000000000001e-05 2024-09-17T21:20:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.961160+0000", 2024-09-17T21:20:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012412100000000001 2024-09-17T21:20:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:13984 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.921124+0000", 2024-09-17T21:20:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 22.759871367999999, 2024-09-17T21:20:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047129407999999998, 2024-09-17T21:20:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:20:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 13984 2024-09-17T21:20:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.921124+0000", 2024-09-17T21:20:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.921124+0000", 2024-09-17T21:20:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.928309+0000", 2024-09-17T21:20:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0071847619999999999 2024-09-17T21:20:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.928316+0000", 2024-09-17T21:20:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2969999999999998e-06 2024-09-17T21:20:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.928330+0000", 2024-09-17T21:20:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3849000000000001e-05 2024-09-17T21:20:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.928338+0000", 2024-09-17T21:20:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.354e-06 2024-09-17T21:20:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.968167+0000", 2024-09-17T21:20:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.039828298999999998 2024-09-17T21:20:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.968201+0000", 2024-09-17T21:20:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3933000000000002e-05 2024-09-17T21:20:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.968254+0000", 2024-09-17T21:20:03.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2914000000000002e-05 2024-09-17T21:20:03.786 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.786 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.786 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.786 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:03.786 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:03.786 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:03.788 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:20:03.788 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:03.785+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:03.888 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:03.881+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:03.955 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:03.955 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:03.955 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:03.955 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:03.968 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:20:03.989 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:03.983+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:04.089 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:04.085+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:04.134 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:04.135 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:04.141 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:20:04.147 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:20:04.190 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:04.183+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:04.291 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:04.287+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:04.311 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:04.311 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:04.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:04.311 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:20:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 106.091785852, 2024-09-17T21:20:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:20:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:20:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:20:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:20:04.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:20:04.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:04.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:20:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:20:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:20:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:20:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:20:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:20:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:20:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:20:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:20:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:20:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:20:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:20:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:20:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 101.441485188, 2024-09-17T21:20:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:20:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:20:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:20:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:20:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:20:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:20:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:20:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:20:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:20:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:20:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:20:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:20:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:20:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:20:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:20:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:20:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:20:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:20:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 101.44022010499999, 2024-09-17T21:20:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:20:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:20:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:20:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:20:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:20:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:20:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:20:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:20:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:20:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:20:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:20:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:20:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:20:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:20:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:20:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:20:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:20:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:20:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 101.430440695, 2024-09-17T21:20:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:20:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:20:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:20:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:20:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:20:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:20:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:20:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:20:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:20:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:20:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:20:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:20:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:20:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:20:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:20:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:20:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:20:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:20:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 101.42171829199999, 2024-09-17T21:20:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:20:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:20:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:20:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:20:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:20:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:20:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:20:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:20:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:20:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:20:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:20:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:20:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:20:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:20:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:20:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:20:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:20:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:20:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 88.036627727999999, 2024-09-17T21:20:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:20:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:20:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:20:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:20:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:20:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:20:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:20:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:20:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:20:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:20:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:20:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:20:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:20:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:20:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:20:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:20:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:20:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1780 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'195, mlcod=69'190)", 2024-09-17T21:20:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.058772124000001, 2024-09-17T21:20:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075761769999999999, 2024-09-17T21:20:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250375+0000", 2024-09-17T21:20:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:20:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250383+0000", 2024-09-17T21:20:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7870000000000002e-06 2024-09-17T21:20:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250402+0000", 2024-09-17T21:20:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9346000000000001e-05 2024-09-17T21:20:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250413+0000", 2024-09-17T21:20:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.039e-05 2024-09-17T21:20:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251353+0000", 2024-09-17T21:20:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000940091 2024-09-17T21:20:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251407+0000", 2024-09-17T21:20:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4701999999999999e-05 2024-09-17T21:20:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257829+0000", 2024-09-17T21:20:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064212779999999999 2024-09-17T21:20:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257950+0000", 2024-09-17T21:20:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121623 2024-09-17T21:20:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:04.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.346 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1781 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'196, mlcod=69'190)", 2024-09-17T21:20:04.346 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:04.346 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.057689173, 2024-09-17T21:20:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065632939999999999, 2024-09-17T21:20:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251459+0000", 2024-09-17T21:20:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.624e-06 2024-09-17T21:20:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251471+0000", 2024-09-17T21:20:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2636000000000001e-05 2024-09-17T21:20:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251498+0000", 2024-09-17T21:20:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:20:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251515+0000", 2024-09-17T21:20:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6939000000000002e-05 2024-09-17T21:20:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252938+0000", 2024-09-17T21:20:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001423082 2024-09-17T21:20:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252999+0000", 2024-09-17T21:20:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1138999999999999e-05 2024-09-17T21:20:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257955+0000", 2024-09-17T21:20:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049559640000000002 2024-09-17T21:20:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.258020+0000", 2024-09-17T21:20:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5504000000000002e-05 2024-09-17T21:20:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1788 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'198, mlcod=69'191)", 2024-09-17T21:20:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.055663684000002, 2024-09-17T21:20:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073690370000000002, 2024-09-17T21:20:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253484+0000", 2024-09-17T21:20:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2470000000000001e-06 2024-09-17T21:20:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253492+0000", 2024-09-17T21:20:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7159999999999992e-06 2024-09-17T21:20:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253509+0000", 2024-09-17T21:20:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.666e-05 2024-09-17T21:20:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253522+0000", 2024-09-17T21:20:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2643000000000001e-05 2024-09-17T21:20:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254853+0000", 2024-09-17T21:20:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331238 2024-09-17T21:20:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254885+0000", 2024-09-17T21:20:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1562e-05 2024-09-17T21:20:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260788+0000", 2024-09-17T21:20:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059030749999999998 2024-09-17T21:20:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260852+0000", 2024-09-17T21:20:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3895999999999999e-05 2024-09-17T21:20:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5746 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'477, mlcod=71'473)", 2024-09-17T21:20:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 48.050236126000001, 2024-09-17T21:20:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074182709999999997, 2024-09-17T21:20:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258912+0000", 2024-09-17T21:20:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:20:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258926+0000", 2024-09-17T21:20:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4249e-05 2024-09-17T21:20:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258946+0000", 2024-09-17T21:20:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9879999999999999e-05 2024-09-17T21:20:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258956+0000", 2024-09-17T21:20:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:20:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259380+0000", 2024-09-17T21:20:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042337599999999999 2024-09-17T21:20:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259412+0000", 2024-09-17T21:20:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:20:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266191+0000", 2024-09-17T21:20:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067793269999999999 2024-09-17T21:20:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266328+0000", 2024-09-17T21:20:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013690999999999999 2024-09-17T21:20:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5747 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'478, mlcod=71'473)", 2024-09-17T21:20:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 48.049529776999996, 2024-09-17T21:20:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068779440000000004, 2024-09-17T21:20:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259618+0000", 2024-09-17T21:20:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6199999999999999e-06 2024-09-17T21:20:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259630+0000", 2024-09-17T21:20:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1884e-05 2024-09-17T21:20:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259657+0000", 2024-09-17T21:20:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7339e-05 2024-09-17T21:20:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259677+0000", 2024-09-17T21:20:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0156000000000001e-05 2024-09-17T21:20:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260521+0000", 2024-09-17T21:20:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084312899999999997 2024-09-17T21:20:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260585+0000", 2024-09-17T21:20:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4558000000000001e-05 2024-09-17T21:20:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266336+0000", 2024-09-17T21:20:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057505020000000002 2024-09-17T21:20:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266494+0000", 2024-09-17T21:20:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158756 2024-09-17T21:20:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5750 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'479, mlcod=71'473)", 2024-09-17T21:20:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 48.048817583999998, 2024-09-17T21:20:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006302874, 2024-09-17T21:20:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260330+0000", 2024-09-17T21:20:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:20:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260343+0000", 2024-09-17T21:20:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2862e-05 2024-09-17T21:20:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260367+0000", 2024-09-17T21:20:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3723e-05 2024-09-17T21:20:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260386+0000", 2024-09-17T21:20:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8879999999999999e-05 2024-09-17T21:20:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261579+0000", 2024-09-17T21:20:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001193431 2024-09-17T21:20:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261608+0000", 2024-09-17T21:20:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8858999999999999e-05 2024-09-17T21:20:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266505+0000", 2024-09-17T21:20:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004896999 2024-09-17T21:20:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266631+0000", 2024-09-17T21:20:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012655300000000001 2024-09-17T21:20:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5751 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'480, mlcod=71'473)", 2024-09-17T21:20:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 48.048110371999996, 2024-09-17T21:20:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075756299999999999, 2024-09-17T21:20:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261037+0000", 2024-09-17T21:20:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:20:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261049+0000", 2024-09-17T21:20:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1063999999999999e-05 2024-09-17T21:20:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261072+0000", 2024-09-17T21:20:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3102e-05 2024-09-17T21:20:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261133+0000", 2024-09-17T21:20:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1276000000000001e-05 2024-09-17T21:20:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:20:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015334319999999999 2024-09-17T21:20:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262702+0000", 2024-09-17T21:20:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5281999999999999e-05 2024-09-17T21:20:04.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:04.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268527+0000", 2024-09-17T21:20:04.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058250389999999997 2024-09-17T21:20:04.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268611+0000", 2024-09-17T21:20:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4724999999999999e-05 2024-09-17T21:20:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5756 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'481, mlcod=71'473)", 2024-09-17T21:20:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 48.047249516000001, 2024-09-17T21:20:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006772386, 2024-09-17T21:20:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261899+0000", 2024-09-17T21:20:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.903e-06 2024-09-17T21:20:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261912+0000", 2024-09-17T21:20:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-05 2024-09-17T21:20:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261938+0000", 2024-09-17T21:20:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6098999999999999e-05 2024-09-17T21:20:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261955+0000", 2024-09-17T21:20:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.696e-05 2024-09-17T21:20:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263845+0000", 2024-09-17T21:20:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001890187 2024-09-17T21:20:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263907+0000", 2024-09-17T21:20:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1600999999999995e-05 2024-09-17T21:20:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268615+0000", 2024-09-17T21:20:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047086070000000001 2024-09-17T21:20:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268669+0000", 2024-09-17T21:20:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3588999999999998e-05 2024-09-17T21:20:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4978.0:598 3.9 e80/23 3:9701cc17:::rbd_data.1152724ff2a3.0000000000000019:head v 80'202, mlcod=80'198)", 2024-09-17T21:20:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.545675133000003, 2024-09-17T21:20:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066473340000000004, 2024-09-17T21:20:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763472+0000", 2024-09-17T21:20:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6899999999999997e-07 2024-09-17T21:20:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763478+0000", 2024-09-17T21:20:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:20:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763489+0000", 2024-09-17T21:20:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0628999999999999e-05 2024-09-17T21:20:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763495+0000", 2024-09-17T21:20:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6649999999999997e-06 2024-09-17T21:20:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763907+0000", 2024-09-17T21:20:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041163899999999999 2024-09-17T21:20:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763976+0000", 2024-09-17T21:20:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8534999999999993e-05 2024-09-17T21:20:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.769900+0000", 2024-09-17T21:20:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059243270000000001 2024-09-17T21:20:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.770118+0000", 2024-09-17T21:20:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021842500000000001 2024-09-17T21:20:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:468 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'94, mlcod=89'87)", 2024-09-17T21:20:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 24.622929834000001, 2024-09-17T21:20:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066528790000000004, 2024-09-17T21:20:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686218+0000", 2024-09-17T21:20:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3060000000000001e-06 2024-09-17T21:20:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686256+0000", 2024-09-17T21:20:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8333e-05 2024-09-17T21:20:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686272+0000", 2024-09-17T21:20:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:20:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686283+0000", 2024-09-17T21:20:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0903e-05 2024-09-17T21:20:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686682+0000", 2024-09-17T21:20:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039935099999999999 2024-09-17T21:20:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686785+0000", 2024-09-17T21:20:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010275199999999999 2024-09-17T21:20:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692816+0000", 2024-09-17T21:20:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060317579999999999 2024-09-17T21:20:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692869+0000", 2024-09-17T21:20:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2775999999999998e-05 2024-09-17T21:20:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:755 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'127, mlcod=89'122)", 2024-09-17T21:20:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 24.486474619999999, 2024-09-17T21:20:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084014080000000008, 2024-09-17T21:20:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:20:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822683+0000", 2024-09-17T21:20:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0674e-05 2024-09-17T21:20:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822698+0000", 2024-09-17T21:20:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5189e-05 2024-09-17T21:20:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822708+0000", 2024-09-17T21:20:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9119999999999993e-06 2024-09-17T21:20:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822748+0000", 2024-09-17T21:20:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9727000000000001e-05 2024-09-17T21:20:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822759+0000", 2024-09-17T21:20:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0865e-05 2024-09-17T21:20:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831034+0000", 2024-09-17T21:20:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082750710000000002 2024-09-17T21:20:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831073+0000", 2024-09-17T21:20:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9063999999999997e-05 2024-09-17T21:20:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:756 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'128, mlcod=89'124)", 2024-09-17T21:20:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 24.484551988, 2024-09-17T21:20:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0076300270000000002, 2024-09-17T21:20:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824595+0000", 2024-09-17T21:20:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-17T21:20:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824600+0000", 2024-09-17T21:20:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:20:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824612+0000", 2024-09-17T21:20:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.181e-05 2024-09-17T21:20:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824619+0000", 2024-09-17T21:20:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:20:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824671+0000", 2024-09-17T21:20:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2262999999999998e-05 2024-09-17T21:20:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824695+0000", 2024-09-17T21:20:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3883000000000002e-05 2024-09-17T21:20:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832185+0000", 2024-09-17T21:20:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007490048 2024-09-17T21:20:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832224+0000", 2024-09-17T21:20:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8957e-05 2024-09-17T21:20:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2617 3.d e90/23 3:b0143718:::rbd_data.1152a8ae2019.0000000000000016:head v 90'249, mlcod=90'246)", 2024-09-17T21:20:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 23.464700988000001, 2024-09-17T21:20:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063715559999999996, 2024-09-17T21:20:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844455+0000", 2024-09-17T21:20:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4259999999999992e-06 2024-09-17T21:20:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844470+0000", 2024-09-17T21:20:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5386000000000001e-05 2024-09-17T21:20:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844499+0000", 2024-09-17T21:20:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.883e-05 2024-09-17T21:20:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844518+0000", 2024-09-17T21:20:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8830000000000001e-05 2024-09-17T21:20:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844649+0000", 2024-09-17T21:20:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013143399999999999 2024-09-17T21:20:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844702+0000", 2024-09-17T21:20:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2876e-05 2024-09-17T21:20:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850696+0000", 2024-09-17T21:20:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059943729999999999 2024-09-17T21:20:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850817+0000", 2024-09-17T21:20:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120401 2024-09-17T21:20:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2635 3.d e90/23 3:b3d1e920:::rbd_data.1152a8ae2019.000000000000001f:head v 90'252, mlcod=90'246)", 2024-09-17T21:20:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 23.461992918, 2024-09-17T21:20:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0077265190000000003, 2024-09-17T21:20:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847155+0000", 2024-09-17T21:20:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.544e-06 2024-09-17T21:20:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847165+0000", 2024-09-17T21:20:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.043e-05 2024-09-17T21:20:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847189+0000", 2024-09-17T21:20:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3659000000000002e-05 2024-09-17T21:20:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847210+0000", 2024-09-17T21:20:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1163e-05 2024-09-17T21:20:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848856+0000", 2024-09-17T21:20:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001645521 2024-09-17T21:20:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848905+0000", 2024-09-17T21:20:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9048999999999997e-05 2024-09-17T21:20:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854770+0000", 2024-09-17T21:20:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058652879999999998 2024-09-17T21:20:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854880+0000", 2024-09-17T21:20:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010986500000000001 2024-09-17T21:20:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:04.406 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:04.408 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:20:04.408 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:04.401+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:04.493 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:04.493 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:04.508 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:04.501+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:04.521 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:04.537 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:04.537 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:04.538 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:20:04.538 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:04.548 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:04.549 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:04.552 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:20:04.573 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:04.573 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:04.597 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:04.598 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:04.609 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:04.605+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:04.620 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:04.674 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:04.674 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:04.688 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:20:04.710 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:04.705+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:04.811 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:04.807+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:04.813 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:04.813 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:04.829 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:20:04.912 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:04.905+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:05.000 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:05.000 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:05.004 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-17T21:20:05.013 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:05.009+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:05.014 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:20:05.115 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:05.109+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:05.139 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:05.139 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:05.153 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:20:05.186 INFO:teuthology.orchestra.run.smithi184.stdout:4541 trunc from 0xe4f1c00 to 0x5cf7c00 2024-09-17T21:20:05.217 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:05.209+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:05.252 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:20:05.294 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:05.294 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:05.300 INFO:teuthology.orchestra.run.smithi184.stdout:4546 punch from 0x3f0b800 to 0x3f15600, (0x9e00 bytes) 2024-09-17T21:20:05.302 INFO:teuthology.orchestra.run.smithi184.stdout:4547 read 0x3733600 thru 0x3742e76 (0xf877 bytes) 2024-09-17T21:20:05.309 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:20:05.335 INFO:teuthology.orchestra.run.smithi184.stdout:4550 write 0xde77200 thru 0xde7eb81 (0x7982 bytes) 2024-09-17T21:20:05.355 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:05.351+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:05.400 INFO:teuthology.orchestra.run.smithi184.stdout:4551 trunc from 0xde7ec00 to 0x225cc00 2024-09-17T21:20:05.417 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:05.411+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:05.446 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:05.446 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:05.466 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:20:05.518 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:05.513+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:05.531 INFO:teuthology.orchestra.run.smithi184.stdout:4552 read 0x1e90c00 thru 0x1e93646 (0x2a47 bytes) 2024-09-17T21:20:05.534 INFO:teuthology.orchestra.run.smithi184.stdout:4554 read 0x992000 thru 0x996a72 (0x4a73 bytes) 2024-09-17T21:20:05.535 INFO:teuthology.orchestra.run.smithi184.stdout:4556 write 0x1e16800 thru 0x1e24a93 (0xe294 bytes) 2024-09-17T21:20:05.541 INFO:teuthology.orchestra.run.smithi184.stdout:4558 trunc from 0x225cc00 to 0x3b81200 2024-09-17T21:20:05.560 INFO:teuthology.orchestra.run.smithi184.stdout:4560 read 0x26bbe00 thru 0x26bd95f (0x1b60 bytes) 2024-09-17T21:20:05.560 INFO:teuthology.orchestra.run.smithi184.stdout:4561 write 0x11d3000 thru 0x11db28a (0x828b bytes) 2024-09-17T21:20:05.561 INFO:teuthology.orchestra.run.smithi184.stdout:4562 write 0x1fb3c00 thru 0x1fbe6b7 (0xaab8 bytes) 2024-09-17T21:20:05.561 INFO:teuthology.orchestra.run.smithi184.stdout:4564 read 0x31ea200 thru 0x31f70cd (0xcece bytes) 2024-09-17T21:20:05.562 INFO:teuthology.orchestra.run.smithi184.stdout:4566 read 0x194c000 thru 0x194cc07 (0xc08 bytes) 2024-09-17T21:20:05.573 INFO:teuthology.orchestra.run.smithi184.stdout:4568 trunc from 0x3b81200 to 0x7a03600 2024-09-17T21:20:05.596 INFO:teuthology.orchestra.run.smithi184.stdout:4574 read 0x4e48a00 thru 0x4e5738d (0xe98e bytes) 2024-09-17T21:20:05.596 INFO:teuthology.orchestra.run.smithi184.stdout:4575 read 0x61ec800 thru 0x61ee73d (0x1f3e bytes) 2024-09-17T21:20:05.597 INFO:teuthology.orchestra.run.smithi184.stdout:4577 write 0x466f400 thru 0x467acc7 (0xb8c8 bytes) 2024-09-17T21:20:05.617 INFO:teuthology.orchestra.run.smithi184.stdout:4582 trunc from 0x7a03600 to 0xf4ee00 2024-09-17T21:20:05.619 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:05.613+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:05.635 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:05.635 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:05.652 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:20:05.708 INFO:teuthology.orchestra.run.smithi184.stdout:4584 write 0xdcc6e00 thru 0xdcc6e57 (0x58 bytes) 2024-09-17T21:20:05.710 INFO:teuthology.orchestra.run.smithi184.stdout:4585 write 0x73f3200 thru 0x73ff831 (0xc632 bytes) 2024-09-17T21:20:05.710 INFO:teuthology.orchestra.run.smithi184.stdout:4587 punch from 0x6d88000 to 0x6d96200, (0xe200 bytes) 2024-09-17T21:20:05.712 INFO:teuthology.orchestra.run.smithi184.stdout:4588 write 0x9924a00 thru 0x992db60 (0x9161 bytes) 2024-09-17T21:20:05.712 INFO:teuthology.orchestra.run.smithi184.stdout:4589 read 0x170b000 thru 0x170f3d1 (0x43d2 bytes) 2024-09-17T21:20:05.713 INFO:teuthology.orchestra.run.smithi184.stdout:4590 read 0x9aa1800 thru 0x9aa61e2 (0x49e3 bytes) 2024-09-17T21:20:05.713 INFO:teuthology.orchestra.run.smithi184.stdout:4591 read 0x2a64000 thru 0x2a659c6 (0x19c7 bytes) 2024-09-17T21:20:05.720 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:05.715+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:05.736 INFO:teuthology.orchestra.run.smithi184.stdout:4594 trunc from 0xdcc7000 to 0x1bcd600 2024-09-17T21:20:05.787 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:05.788 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:05.790 INFO:teuthology.orchestra.run.smithi184.stdout:4601 trunc from 0x1bcd600 to 0x312ac00 2024-09-17T21:20:05.803 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:20:05.813 INFO:teuthology.orchestra.run.smithi184.stdout:4602 read 0x7ed400 thru 0x7f9fbc (0xcbbd bytes) 2024-09-17T21:20:05.817 INFO:teuthology.orchestra.run.smithi184.stdout:4603 read 0xacfc00 thru 0xad293d (0x2d3e bytes) 2024-09-17T21:20:05.819 INFO:teuthology.orchestra.run.smithi184.stdout:4606 punch from 0x591400 to 0x595a00, (0x4600 bytes) 2024-09-17T21:20:05.819 INFO:teuthology.orchestra.run.smithi184.stdout:4608 punch from 0x2564200 to 0x2571600, (0xd400 bytes) 2024-09-17T21:20:05.846 INFO:teuthology.orchestra.run.smithi184.stdout:4610 trunc from 0x312ac00 to 0xd160400 2024-09-17T21:20:05.847 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:05.843+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:05.866 INFO:teuthology.orchestra.run.smithi184.stdout:4611 punch from 0x61fd200 to 0x6203e00, (0x6c00 bytes) 2024-09-17T21:20:05.868 INFO:teuthology.orchestra.run.smithi184.stdout:4612 read 0xb465c00 thru 0xb46caa7 (0x6ea8 bytes) 2024-09-17T21:20:05.868 INFO:teuthology.orchestra.run.smithi184.stdout:4613 write 0x7c18800 thru 0x7c24cf8 (0xc4f9 bytes) 2024-09-17T21:20:05.868 INFO:teuthology.orchestra.run.smithi184.stdout:4614 punch from 0x6f59800 to 0x6f67a00, (0xe200 bytes) 2024-09-17T21:20:05.869 INFO:teuthology.orchestra.run.smithi184.stdout:4616 read 0xc427e00 thru 0xc436f14 (0xf115 bytes) 2024-09-17T21:20:05.870 INFO:teuthology.orchestra.run.smithi184.stdout:4617 write 0xd097200 thru 0xd097aec (0x8ed bytes) 2024-09-17T21:20:05.878 INFO:teuthology.orchestra.run.smithi184.stdout:4619 trunc from 0xd160400 to 0x8738000 2024-09-17T21:20:05.921 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:05.915+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:05.947 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:05.948 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:05.950 INFO:teuthology.orchestra.run.smithi184.stdout:4621 trunc from 0x8738000 to 0x328d800 2024-09-17T21:20:05.962 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:20:05.992 INFO:teuthology.orchestra.run.smithi184.stdout:4623 write 0x1a99600 thru 0x1aa6f61 (0xd962 bytes) 2024-09-17T21:20:06.022 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:06.015+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:06.053 INFO:teuthology.orchestra.run.smithi184.stdout:4625 write 0xd116e00 thru 0xd120a3b (0x9c3c bytes) 2024-09-17T21:20:06.055 INFO:teuthology.orchestra.run.smithi184.stdout:4626 read 0x6ea9e00 thru 0x6eb536f (0xb570 bytes) 2024-09-17T21:20:06.055 INFO:teuthology.orchestra.run.smithi184.stdout:4627 write 0xc88f200 thru 0xc88fb26 (0x927 bytes) 2024-09-17T21:20:06.055 INFO:teuthology.orchestra.run.smithi184.stdout:4628 punch from 0x378b400 to 0x3790400, (0x5000 bytes) 2024-09-17T21:20:06.056 INFO:teuthology.orchestra.run.smithi184.stdout:4629 clone 30 order 20 su 65536 sc 10 2024-09-17T21:20:06.090 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:06.090 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:06.105 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:20:06.122 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:06.119+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:06.223 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:06.217+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:06.224 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:06.224 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:06.238 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:20:06.247 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-17T21:20:06.324 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:06.319+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:06.372 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:06.372 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:06.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:20:06.401 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone29 from 0xd120c00 (overlap 0xf4ee00) to 0xab7200 2024-09-17T21:20:06.433 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:06.429+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:06.516 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:06.516 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:06.526 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:06.519+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:06.531 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:20:06.628 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:06.623+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:06.643 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:20:06.702 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:06.702 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:06.718 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:20:06.765 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:06.761+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:06.801 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:06.801 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:06.819 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:06.819 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:06.828 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:06.821+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:06.850 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:06.850 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:06.858 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:06.858 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:06.858 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:06.859 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:06.871 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:06.871 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:06.873 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:20:06.892 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:06.892 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:06.913 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:06.913 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:06.922 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-17T21:20:06.929 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:06.925+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:07.007 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:07.008 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:07.008 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:07.008 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:07.008 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:07.021 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:20:07.069 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:07.065+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:07.130 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:07.125+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:07.149 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:20:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 108.928452478, 2024-09-17T21:20:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:20:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:20:07.151 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:20:07.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:07.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:20:07.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:20:07.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:07.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:20:07.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:20:07.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:20:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:20:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:20:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:20:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:20:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:20:07.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:20:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:20:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:20:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:20:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:20:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 104.278151814, 2024-09-17T21:20:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:20:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:20:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:20:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:20:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:20:07.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:20:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:20:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:20:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:20:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:20:07.157 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:20:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:20:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:20:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:20:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:20:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:07.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:20:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:20:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:20:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 104.276886731, 2024-09-17T21:20:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:20:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:20:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:20:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:20:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:20:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:20:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:20:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.161 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:20:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:20:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:20:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:20:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:20:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:20:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:20:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:20:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:20:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:20:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:20:07.163 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 104.267107321, 2024-09-17T21:20:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:20:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:20:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:20:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:20:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:20:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:20:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:20:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:20:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:20:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:20:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:20:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:20:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:20:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:20:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:20:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:20:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:20:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.167 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:20:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 104.258384918, 2024-09-17T21:20:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:20:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:07.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:20:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:20:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:07.169 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:20:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:20:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:20:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:20:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:20:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:20:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.170 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:20:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:20:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:20:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:20:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:20:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:20:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:20:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:20:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:20:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 90.873294353999995, 2024-09-17T21:20:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:20:07.172 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:20:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:20:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:20:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:20:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:20:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:20:07.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:20:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:20:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:20:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:20:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:20:07.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:20:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:20:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:20:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:20:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:20:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1780 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'195, mlcod=69'190)", 2024-09-17T21:20:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 53.895438749999997, 2024-09-17T21:20:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075761769999999999, 2024-09-17T21:20:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250375+0000", 2024-09-17T21:20:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:20:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250383+0000", 2024-09-17T21:20:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7870000000000002e-06 2024-09-17T21:20:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250402+0000", 2024-09-17T21:20:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9346000000000001e-05 2024-09-17T21:20:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250413+0000", 2024-09-17T21:20:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.039e-05 2024-09-17T21:20:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251353+0000", 2024-09-17T21:20:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000940091 2024-09-17T21:20:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251407+0000", 2024-09-17T21:20:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4701999999999999e-05 2024-09-17T21:20:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257829+0000", 2024-09-17T21:20:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064212779999999999 2024-09-17T21:20:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257950+0000", 2024-09-17T21:20:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121623 2024-09-17T21:20:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1781 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'196, mlcod=69'190)", 2024-09-17T21:20:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 53.894355799000003, 2024-09-17T21:20:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065632939999999999, 2024-09-17T21:20:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.181 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:07.182 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251459+0000", 2024-09-17T21:20:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.624e-06 2024-09-17T21:20:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251471+0000", 2024-09-17T21:20:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2636000000000001e-05 2024-09-17T21:20:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251498+0000", 2024-09-17T21:20:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:20:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251515+0000", 2024-09-17T21:20:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6939000000000002e-05 2024-09-17T21:20:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252938+0000", 2024-09-17T21:20:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001423082 2024-09-17T21:20:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252999+0000", 2024-09-17T21:20:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1138999999999999e-05 2024-09-17T21:20:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257955+0000", 2024-09-17T21:20:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049559640000000002 2024-09-17T21:20:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.258020+0000", 2024-09-17T21:20:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5504000000000002e-05 2024-09-17T21:20:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1788 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'198, mlcod=69'191)", 2024-09-17T21:20:07.185 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 53.892330309999998, 2024-09-17T21:20:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073690370000000002, 2024-09-17T21:20:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253484+0000", 2024-09-17T21:20:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2470000000000001e-06 2024-09-17T21:20:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253492+0000", 2024-09-17T21:20:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7159999999999992e-06 2024-09-17T21:20:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253509+0000", 2024-09-17T21:20:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.666e-05 2024-09-17T21:20:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253522+0000", 2024-09-17T21:20:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2643000000000001e-05 2024-09-17T21:20:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254853+0000", 2024-09-17T21:20:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331238 2024-09-17T21:20:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254885+0000", 2024-09-17T21:20:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1562e-05 2024-09-17T21:20:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260788+0000", 2024-09-17T21:20:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059030749999999998 2024-09-17T21:20:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260852+0000", 2024-09-17T21:20:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3895999999999999e-05 2024-09-17T21:20:07.189 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5746 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'477, mlcod=71'473)", 2024-09-17T21:20:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.886902751999997, 2024-09-17T21:20:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074182709999999997, 2024-09-17T21:20:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258912+0000", 2024-09-17T21:20:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:20:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258926+0000", 2024-09-17T21:20:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4249e-05 2024-09-17T21:20:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258946+0000", 2024-09-17T21:20:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9879999999999999e-05 2024-09-17T21:20:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258956+0000", 2024-09-17T21:20:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:20:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259380+0000", 2024-09-17T21:20:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042337599999999999 2024-09-17T21:20:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259412+0000", 2024-09-17T21:20:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:20:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266191+0000", 2024-09-17T21:20:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067793269999999999 2024-09-17T21:20:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266328+0000", 2024-09-17T21:20:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013690999999999999 2024-09-17T21:20:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5747 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'478, mlcod=71'473)", 2024-09-17T21:20:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.886196403, 2024-09-17T21:20:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068779440000000004, 2024-09-17T21:20:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259618+0000", 2024-09-17T21:20:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6199999999999999e-06 2024-09-17T21:20:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259630+0000", 2024-09-17T21:20:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1884e-05 2024-09-17T21:20:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259657+0000", 2024-09-17T21:20:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7339e-05 2024-09-17T21:20:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259677+0000", 2024-09-17T21:20:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0156000000000001e-05 2024-09-17T21:20:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260521+0000", 2024-09-17T21:20:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084312899999999997 2024-09-17T21:20:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260585+0000", 2024-09-17T21:20:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4558000000000001e-05 2024-09-17T21:20:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266336+0000", 2024-09-17T21:20:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057505020000000002 2024-09-17T21:20:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266494+0000", 2024-09-17T21:20:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158756 2024-09-17T21:20:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5750 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'479, mlcod=71'473)", 2024-09-17T21:20:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.885484210000001, 2024-09-17T21:20:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006302874, 2024-09-17T21:20:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260330+0000", 2024-09-17T21:20:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:20:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260343+0000", 2024-09-17T21:20:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2862e-05 2024-09-17T21:20:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260367+0000", 2024-09-17T21:20:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3723e-05 2024-09-17T21:20:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260386+0000", 2024-09-17T21:20:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8879999999999999e-05 2024-09-17T21:20:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261579+0000", 2024-09-17T21:20:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001193431 2024-09-17T21:20:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261608+0000", 2024-09-17T21:20:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8858999999999999e-05 2024-09-17T21:20:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266505+0000", 2024-09-17T21:20:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004896999 2024-09-17T21:20:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266631+0000", 2024-09-17T21:20:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012655300000000001 2024-09-17T21:20:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5751 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'480, mlcod=71'473)", 2024-09-17T21:20:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.884776998, 2024-09-17T21:20:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075756299999999999, 2024-09-17T21:20:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261037+0000", 2024-09-17T21:20:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:20:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261049+0000", 2024-09-17T21:20:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1063999999999999e-05 2024-09-17T21:20:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261072+0000", 2024-09-17T21:20:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3102e-05 2024-09-17T21:20:07.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261133+0000", 2024-09-17T21:20:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1276000000000001e-05 2024-09-17T21:20:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:20:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015334319999999999 2024-09-17T21:20:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:07.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262702+0000", 2024-09-17T21:20:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5281999999999999e-05 2024-09-17T21:20:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268527+0000", 2024-09-17T21:20:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058250389999999997 2024-09-17T21:20:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268611+0000", 2024-09-17T21:20:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4724999999999999e-05 2024-09-17T21:20:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.208 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5756 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'481, mlcod=71'473)", 2024-09-17T21:20:07.208 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:07.208 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.883916141999997, 2024-09-17T21:20:07.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006772386, 2024-09-17T21:20:07.208 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:07.208 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:07.208 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.208 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:07.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:07.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:07.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:07.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:07.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:07.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261899+0000", 2024-09-17T21:20:07.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.903e-06 2024-09-17T21:20:07.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:07.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261912+0000", 2024-09-17T21:20:07.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-05 2024-09-17T21:20:07.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:07.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261938+0000", 2024-09-17T21:20:07.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6098999999999999e-05 2024-09-17T21:20:07.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261955+0000", 2024-09-17T21:20:07.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.696e-05 2024-09-17T21:20:07.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:07.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263845+0000", 2024-09-17T21:20:07.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001890187 2024-09-17T21:20:07.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:07.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263907+0000", 2024-09-17T21:20:07.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1600999999999995e-05 2024-09-17T21:20:07.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:07.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268615+0000", 2024-09-17T21:20:07.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047086070000000001 2024-09-17T21:20:07.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:07.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268669+0000", 2024-09-17T21:20:07.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3588999999999998e-05 2024-09-17T21:20:07.212 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.212 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:07.212 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.212 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4978.0:598 3.9 e80/23 3:9701cc17:::rbd_data.1152724ff2a3.0000000000000019:head v 80'202, mlcod=80'198)", 2024-09-17T21:20:07.212 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:07.212 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.382341758999999, 2024-09-17T21:20:07.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066473340000000004, 2024-09-17T21:20:07.212 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:07.212 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:07.213 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.213 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:07.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:07.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:07.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:07.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:07.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:07.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763472+0000", 2024-09-17T21:20:07.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6899999999999997e-07 2024-09-17T21:20:07.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:07.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763478+0000", 2024-09-17T21:20:07.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:20:07.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:07.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763489+0000", 2024-09-17T21:20:07.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0628999999999999e-05 2024-09-17T21:20:07.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763495+0000", 2024-09-17T21:20:07.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6649999999999997e-06 2024-09-17T21:20:07.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:07.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763907+0000", 2024-09-17T21:20:07.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041163899999999999 2024-09-17T21:20:07.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:07.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763976+0000", 2024-09-17T21:20:07.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8534999999999993e-05 2024-09-17T21:20:07.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:07.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.769900+0000", 2024-09-17T21:20:07.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059243270000000001 2024-09-17T21:20:07.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:07.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.770118+0000", 2024-09-17T21:20:07.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021842500000000001 2024-09-17T21:20:07.216 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.216 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:07.216 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.217 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:468 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'94, mlcod=89'87)", 2024-09-17T21:20:07.217 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:07.217 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.45959646, 2024-09-17T21:20:07.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066528790000000004, 2024-09-17T21:20:07.217 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:07.217 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:07.217 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.217 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:07.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:07.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:07.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:07.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:07.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:07.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686218+0000", 2024-09-17T21:20:07.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3060000000000001e-06 2024-09-17T21:20:07.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:07.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686256+0000", 2024-09-17T21:20:07.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8333e-05 2024-09-17T21:20:07.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:07.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686272+0000", 2024-09-17T21:20:07.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:20:07.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686283+0000", 2024-09-17T21:20:07.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0903e-05 2024-09-17T21:20:07.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:07.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686682+0000", 2024-09-17T21:20:07.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039935099999999999 2024-09-17T21:20:07.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:07.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686785+0000", 2024-09-17T21:20:07.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010275199999999999 2024-09-17T21:20:07.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:07.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692816+0000", 2024-09-17T21:20:07.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060317579999999999 2024-09-17T21:20:07.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:07.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692869+0000", 2024-09-17T21:20:07.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2775999999999998e-05 2024-09-17T21:20:07.221 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.221 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:07.221 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.221 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:755 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'127, mlcod=89'122)", 2024-09-17T21:20:07.221 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:07.221 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.323141245999999, 2024-09-17T21:20:07.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084014080000000008, 2024-09-17T21:20:07.221 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:07.221 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:07.222 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.222 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:07.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:07.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:07.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:07.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:07.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:07.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:07.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:20:07.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:07.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822683+0000", 2024-09-17T21:20:07.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0674e-05 2024-09-17T21:20:07.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:07.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822698+0000", 2024-09-17T21:20:07.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5189e-05 2024-09-17T21:20:07.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822708+0000", 2024-09-17T21:20:07.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9119999999999993e-06 2024-09-17T21:20:07.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:07.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822748+0000", 2024-09-17T21:20:07.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9727000000000001e-05 2024-09-17T21:20:07.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:07.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822759+0000", 2024-09-17T21:20:07.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0865e-05 2024-09-17T21:20:07.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:07.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831034+0000", 2024-09-17T21:20:07.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082750710000000002 2024-09-17T21:20:07.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:07.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831073+0000", 2024-09-17T21:20:07.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9063999999999997e-05 2024-09-17T21:20:07.225 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.225 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:07.225 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.226 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:756 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'128, mlcod=89'124)", 2024-09-17T21:20:07.226 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:07.226 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.321218613999999, 2024-09-17T21:20:07.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0076300270000000002, 2024-09-17T21:20:07.226 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:07.226 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:07.226 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.226 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:07.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:07.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:07.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:07.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:07.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:07.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824595+0000", 2024-09-17T21:20:07.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-17T21:20:07.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:07.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824600+0000", 2024-09-17T21:20:07.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:20:07.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:07.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824612+0000", 2024-09-17T21:20:07.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.181e-05 2024-09-17T21:20:07.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824619+0000", 2024-09-17T21:20:07.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:20:07.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:07.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824671+0000", 2024-09-17T21:20:07.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2262999999999998e-05 2024-09-17T21:20:07.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:07.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824695+0000", 2024-09-17T21:20:07.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3883000000000002e-05 2024-09-17T21:20:07.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:07.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832185+0000", 2024-09-17T21:20:07.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007490048 2024-09-17T21:20:07.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:07.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832224+0000", 2024-09-17T21:20:07.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8957e-05 2024-09-17T21:20:07.230 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.230 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:07.230 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.230 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2617 3.d e90/23 3:b0143718:::rbd_data.1152a8ae2019.0000000000000016:head v 90'249, mlcod=90'246)", 2024-09-17T21:20:07.230 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:07.230 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 26.301367614, 2024-09-17T21:20:07.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063715559999999996, 2024-09-17T21:20:07.230 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:07.230 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:07.230 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.231 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:07.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:07.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:07.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:07.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:07.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:07.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844455+0000", 2024-09-17T21:20:07.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4259999999999992e-06 2024-09-17T21:20:07.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:07.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844470+0000", 2024-09-17T21:20:07.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5386000000000001e-05 2024-09-17T21:20:07.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:07.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844499+0000", 2024-09-17T21:20:07.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.883e-05 2024-09-17T21:20:07.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844518+0000", 2024-09-17T21:20:07.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8830000000000001e-05 2024-09-17T21:20:07.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:07.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844649+0000", 2024-09-17T21:20:07.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013143399999999999 2024-09-17T21:20:07.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:07.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844702+0000", 2024-09-17T21:20:07.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2876e-05 2024-09-17T21:20:07.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:07.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850696+0000", 2024-09-17T21:20:07.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059943729999999999 2024-09-17T21:20:07.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:07.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850817+0000", 2024-09-17T21:20:07.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120401 2024-09-17T21:20:07.234 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.234 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:07.234 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.234 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2635 3.d e90/23 3:b3d1e920:::rbd_data.1152a8ae2019.000000000000001f:head v 90'252, mlcod=90'246)", 2024-09-17T21:20:07.235 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:07.235 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 26.298659543999999, 2024-09-17T21:20:07.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0077265190000000003, 2024-09-17T21:20:07.235 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:07.235 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:07.235 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.235 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:07.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:07.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:07.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:07.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:07.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:07.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:07.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847155+0000", 2024-09-17T21:20:07.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.544e-06 2024-09-17T21:20:07.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:07.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847165+0000", 2024-09-17T21:20:07.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.043e-05 2024-09-17T21:20:07.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:07.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847189+0000", 2024-09-17T21:20:07.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3659000000000002e-05 2024-09-17T21:20:07.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847210+0000", 2024-09-17T21:20:07.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1163e-05 2024-09-17T21:20:07.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:07.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848856+0000", 2024-09-17T21:20:07.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001645521 2024-09-17T21:20:07.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:07.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848905+0000", 2024-09-17T21:20:07.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9048999999999997e-05 2024-09-17T21:20:07.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:07.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854770+0000", 2024-09-17T21:20:07.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058652879999999998 2024-09-17T21:20:07.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:07.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:07.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:07.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854880+0000", 2024-09-17T21:20:07.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010986500000000001 2024-09-17T21:20:07.238 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.239 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:07.239 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.239 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:07.239 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:07.239 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:07.240 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:20:07.241 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:07.235+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:07.341 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:07.337+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:07.377 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:07.377 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:07.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.377 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5405.0:755 3.5 3:a0698a22:::rbd_data.11527d2b15f6.0000000000000029:head [sparse-read 528384~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-17T21:20:07.377 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:07.375158+0000", 2024-09-17T21:20:07.377 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.00040277100000000001, 2024-09-17T21:20:07.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00042614299999999998, 2024-09-17T21:20:07.377 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:07.377 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:07.377 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "reached pg", 2024-09-17T21:20:07.378 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:07.378 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5405", 2024-09-17T21:20:07.378 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3178358100", 2024-09-17T21:20:07.378 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 755 2024-09-17T21:20:07.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.378 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:07.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:07.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:07.375158+0000", 2024-09-17T21:20:07.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:07.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:07.375158+0000", 2024-09-17T21:20:07.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:07.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:07.375159+0000", 2024-09-17T21:20:07.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.204e-06 2024-09-17T21:20:07.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:07.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:07.375165+0000", 2024-09-17T21:20:07.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1360000000000002e-06 2024-09-17T21:20:07.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:07.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:07.375187+0000", 2024-09-17T21:20:07.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2025999999999999e-05 2024-09-17T21:20:07.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:07.375202+0000", 2024-09-17T21:20:07.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4443e-05 2024-09-17T21:20:07.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:07.375494+0000", 2024-09-17T21:20:07.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029245200000000001 2024-09-17T21:20:07.381 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.381 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:07.382 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.382 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.382 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-09-17T21:20:07.382 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 1 2024-09-17T21:20:07.382 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:07.391 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:20:07.442 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:07.435+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:07.515 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:07.515 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:07.516 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:07.516 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:07.516 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:07.529 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:20:07.543 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:07.537+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:07.644 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:07.637+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:07.708 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:20:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:20:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 130.690311788, 2024-09-17T21:20:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:20:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:20:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:20:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:20:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:20:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:20:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:20:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:20:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:20:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:20:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:20:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:20:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:20:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:20:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:20:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:20:07.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:07.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:20:07.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:20:07.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:20:07.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:20:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:20:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:20:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:20:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:20:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:20:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:20:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:20:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:20:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:20:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:20:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:20:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:20:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:20:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:20:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:20:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.110147508, 2024-09-17T21:20:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:20:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:20:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:20:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:20:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:20:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:20:07.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:07.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:20:07.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:20:07.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:20:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:20:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:20:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:20:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:20:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:20:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:20:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:20:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:20:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:20:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:20:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:20:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:20:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:20:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:20:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:20:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:20:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:20:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:20:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:20:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:20:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:20:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:20:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.110050196, 2024-09-17T21:20:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:20:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:20:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:20:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:20:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:20:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:20:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:20:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:20:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:20:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:20:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:20:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:20:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:20:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:20:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:20:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:20:07.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:20:07.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:20:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:20:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:20:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:20:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:20:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:20:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:20:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:20:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:20:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:20:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:20:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:20:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:20:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:20:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:20:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:20:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:20:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.107963395, 2024-09-17T21:20:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:20:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:20:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:20:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:20:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:20:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:20:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:20:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:20:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:20:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:20:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:20:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:20:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:20:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:20:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:20:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:20:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:20:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:20:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:20:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:20:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:20:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:20:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:20:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:20:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:20:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:20:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:20:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:20:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:20:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:20:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:20:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:20:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:20:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:20:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:20:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:20:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:20:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:20:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 78.424064614000002, 2024-09-17T21:20:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:20:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:20:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:20:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:20:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:20:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:20:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:20:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:20:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:20:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:20:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:20:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:20:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:20:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:20:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:20:07.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:20:07.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:20:07.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:20:07.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:20:07.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:20:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:20:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:20:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:20:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:20:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:20:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:20:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:20:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:20:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:20:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:20:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 78.414972903000006, 2024-09-17T21:20:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:20:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:20:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:20:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:20:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:20:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:20:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:20:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:20:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:20:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:20:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:20:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:20:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:20:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:20:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:20:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:20:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:20:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:20:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:20:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:20:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:20:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:20:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:20:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:20:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:20:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 78.414800851999999, 2024-09-17T21:20:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:20:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:20:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:20:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:20:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:20:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:20:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:20:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:20:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:20:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:20:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:20:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:20:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:20:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:20:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:20:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:20:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:20:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:20:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:20:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:20:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:20:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:20:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:20:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:20:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:20:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 78.239868104999999, 2024-09-17T21:20:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:20:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:20:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:20:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:20:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:20:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:20:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:20:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:20:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:20:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:20:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:20:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:20:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:20:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:20:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:20:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:20:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:20:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:20:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:20:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:20:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:20:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:20:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 78.239783548000005, 2024-09-17T21:20:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:20:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:20:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:20:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:20:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:20:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:20:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:20:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:20:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:20:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:20:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:20:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:20:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:20:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:20:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:20:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:20:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:20:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:20:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:20:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:20:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 78.239717911, 2024-09-17T21:20:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:20:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:20:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:20:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:20:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:20:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:20:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:20:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:20:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:20:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:20:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:20:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:20:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:20:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:20:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:20:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:20:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:20:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:20:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:20:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:20:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:20:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:20:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:20:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.866142814, 2024-09-17T21:20:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:20:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:20:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:20:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:20:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:20:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:20:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:20:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:20:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:20:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:20:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:20:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:20:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:20:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:20:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:20:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:20:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:20:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:20:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:20:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.866023301999999, 2024-09-17T21:20:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:20:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:20:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:20:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:20:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:07.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:20:07.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:20:07.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:07.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:20:07.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:20:07.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:07.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:20:07.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:20:07.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:20:07.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:20:07.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:20:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:20:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:20:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:20:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:20:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:20:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2695 3.15 3:aedfd032:::rbd_data.1152a8ae2019.0000000000000041:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.843199+0000", 2024-09-17T21:20:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.860877997999999, 2024-09-17T21:20:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079730569000000001, 2024-09-17T21:20:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2695 2024-09-17T21:20:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843199+0000", 2024-09-17T21:20:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843199+0000", 2024-09-17T21:20:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843201+0000", 2024-09-17T21:20:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0899999999999999e-06 2024-09-17T21:20:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843214+0000", 2024-09-17T21:20:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2233e-05 2024-09-17T21:20:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843245+0000", 2024-09-17T21:20:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1464000000000002e-05 2024-09-17T21:20:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843266+0000", 2024-09-17T21:20:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0879000000000001e-05 2024-09-17T21:20:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847920+0000", 2024-09-17T21:20:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.004654292 2024-09-17T21:20:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.848240+0000", 2024-09-17T21:20:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031994499999999997 2024-09-17T21:20:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922930+0000", 2024-09-17T21:20:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.074689666000000002 2024-09-17T21:20:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2732 3.15 3:ab9a4731:::rbd_data.1152a8ae2019.0000000000000058:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.846909+0000", 2024-09-17T21:20:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.857167952000001, 2024-09-17T21:20:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076145476000000004, 2024-09-17T21:20:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2732 2024-09-17T21:20:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846909+0000", 2024-09-17T21:20:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846909+0000", 2024-09-17T21:20:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846911+0000", 2024-09-17T21:20:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.31e-06 2024-09-17T21:20:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846916+0000", 2024-09-17T21:20:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.023e-06 2024-09-17T21:20:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846931+0000", 2024-09-17T21:20:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5549999999999999e-05 2024-09-17T21:20:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846943+0000", 2024-09-17T21:20:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1783000000000001e-05 2024-09-17T21:20:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.850592+0000", 2024-09-17T21:20:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036486159999999999 2024-09-17T21:20:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.850870+0000", 2024-09-17T21:20:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027864200000000001 2024-09-17T21:20:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.923055+0000", 2024-09-17T21:20:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.072184551999999999 2024-09-17T21:20:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2737 3.15 3:a8c4e696:::rbd_data.1152a8ae2019.000000000000005b:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.847014+0000", 2024-09-17T21:20:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.857062816999999, 2024-09-17T21:20:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076158271, 2024-09-17T21:20:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2737 2024-09-17T21:20:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847014+0000", 2024-09-17T21:20:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847014+0000", 2024-09-17T21:20:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847015+0000", 2024-09-17T21:20:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0699999999999999e-06 2024-09-17T21:20:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847021+0000", 2024-09-17T21:20:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.446e-06 2024-09-17T21:20:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847038+0000", 2024-09-17T21:20:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7257000000000001e-05 2024-09-17T21:20:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847051+0000", 2024-09-17T21:20:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2649999999999999e-05 2024-09-17T21:20:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.852127+0000", 2024-09-17T21:20:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050757420000000003 2024-09-17T21:20:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.852423+0000", 2024-09-17T21:20:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029608100000000001 2024-09-17T21:20:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.923173+0000", 2024-09-17T21:20:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.070750024999999994 2024-09-17T21:20:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3397 3.8 3:174f9d43:::rbd_data.1152a8ae2019.000000000000006e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.897797+0000", 2024-09-17T21:20:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.806279872000001, 2024-09-17T21:20:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045478192000000001, 2024-09-17T21:20:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3397 2024-09-17T21:20:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897797+0000", 2024-09-17T21:20:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897797+0000", 2024-09-17T21:20:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897799+0000", 2024-09-17T21:20:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.238e-06 2024-09-17T21:20:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897806+0000", 2024-09-17T21:20:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5900000000000002e-06 2024-09-17T21:20:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897822+0000", 2024-09-17T21:20:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5659e-05 2024-09-17T21:20:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897832+0000", 2024-09-17T21:20:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0034000000000001e-05 2024-09-17T21:20:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.931249+0000", 2024-09-17T21:20:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033416700000000001 2024-09-17T21:20:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.931425+0000", 2024-09-17T21:20:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017629500000000001 2024-09-17T21:20:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943276+0000", 2024-09-17T21:20:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011850675999999999 2024-09-17T21:20:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3524 3.8 3:119ba024:::rbd_data.1152c6ce51b2.0000000000000005:17 [read 1261568~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.901789+0000", 2024-09-17T21:20:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.802287945, 2024-09-17T21:20:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.042862082000000003, 2024-09-17T21:20:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3524 2024-09-17T21:20:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901789+0000", 2024-09-17T21:20:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901789+0000", 2024-09-17T21:20:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901790+0000", 2024-09-17T21:20:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0400000000000005e-07 2024-09-17T21:20:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901795+0000", 2024-09-17T21:20:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3839999999999999e-06 2024-09-17T21:20:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901807+0000", 2024-09-17T21:20:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2532e-05 2024-09-17T21:20:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901816+0000", 2024-09-17T21:20:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2220000000000008e-06 2024-09-17T21:20:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.944407+0000", 2024-09-17T21:20:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.042591088999999999 2024-09-17T21:20:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.944498+0000", 2024-09-17T21:20:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0883000000000004e-05 2024-09-17T21:20:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.944651+0000", 2024-09-17T21:20:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015306799999999999 2024-09-17T21:20:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3679 3.8 3:129b2dec:::rbd_data.1152a8ae2019.0000000000000012:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.908354+0000", 2024-09-17T21:20:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.795723584000001, 2024-09-17T21:20:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046195880000000002, 2024-09-17T21:20:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3679 2024-09-17T21:20:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908354+0000", 2024-09-17T21:20:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908354+0000", 2024-09-17T21:20:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908355+0000", 2024-09-17T21:20:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9999999999999995e-07 2024-09-17T21:20:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908361+0000", 2024-09-17T21:20:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7399999999999998e-06 2024-09-17T21:20:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908375+0000", 2024-09-17T21:20:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.363e-05 2024-09-17T21:20:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908386+0000", 2024-09-17T21:20:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.06e-05 2024-09-17T21:20:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.945600+0000", 2024-09-17T21:20:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.037214199000000003 2024-09-17T21:20:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.945650+0000", 2024-09-17T21:20:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9737999999999997e-05 2024-09-17T21:20:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.945799+0000", 2024-09-17T21:20:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014921699999999999 2024-09-17T21:20:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954371+0000", 2024-09-17T21:20:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085719730000000001 2024-09-17T21:20:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954460+0000", 2024-09-17T21:20:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9596000000000001e-05 2024-09-17T21:20:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954511+0000", 2024-09-17T21:20:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0455999999999999e-05 2024-09-17T21:20:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954550+0000", 2024-09-17T21:20:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8730999999999999e-05 2024-09-17T21:20:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.808 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3703 3.13 3:cdb4d88a:::rbd_data.1152a8ae2019.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:07.808 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.908402+0000", 2024-09-17T21:20:07.808 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.795675544000002, 2024-09-17T21:20:07.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046393446999999997, 2024-09-17T21:20:07.808 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:07.808 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:07.808 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.808 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3703 2024-09-17T21:20:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908402+0000", 2024-09-17T21:20:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908402+0000", 2024-09-17T21:20:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908403+0000", 2024-09-17T21:20:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0529999999999999e-06 2024-09-17T21:20:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908409+0000", 2024-09-17T21:20:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2219999999999999e-06 2024-09-17T21:20:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908422+0000", 2024-09-17T21:20:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3147e-05 2024-09-17T21:20:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908432+0000", 2024-09-17T21:20:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.039e-05 2024-09-17T21:20:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.942217+0000", 2024-09-17T21:20:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033784994999999998 2024-09-17T21:20:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.942296+0000", 2024-09-17T21:20:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8218000000000002e-05 2024-09-17T21:20:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943003+0000", 2024-09-17T21:20:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000707403 2024-09-17T21:20:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954635+0000", 2024-09-17T21:20:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011631786999999999 2024-09-17T21:20:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954706+0000", 2024-09-17T21:20:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1064999999999998e-05 2024-09-17T21:20:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954757+0000", 2024-09-17T21:20:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0760000000000002e-05 2024-09-17T21:20:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954795+0000", 2024-09-17T21:20:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8407e-05 2024-09-17T21:20:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3711 3.13 3:ca5a0e5e:::rbd_data.1152a8ae2019.0000000000000040:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.908514+0000", 2024-09-17T21:20:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.795563381000001, 2024-09-17T21:20:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046370871000000001, 2024-09-17T21:20:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3711 2024-09-17T21:20:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908514+0000", 2024-09-17T21:20:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908514+0000", 2024-09-17T21:20:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908515+0000", 2024-09-17T21:20:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0449999999999999e-06 2024-09-17T21:20:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908521+0000", 2024-09-17T21:20:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2179999999999996e-06 2024-09-17T21:20:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908535+0000", 2024-09-17T21:20:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4124999999999999e-05 2024-09-17T21:20:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908548+0000", 2024-09-17T21:20:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2680000000000001e-05 2024-09-17T21:20:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943530+0000", 2024-09-17T21:20:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034981655 2024-09-17T21:20:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943636+0000", 2024-09-17T21:20:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106086 2024-09-17T21:20:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943837+0000", 2024-09-17T21:20:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201826 2024-09-17T21:20:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954688+0000", 2024-09-17T21:20:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010850963 2024-09-17T21:20:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954805+0000", 2024-09-17T21:20:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000117025 2024-09-17T21:20:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954851+0000", 2024-09-17T21:20:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5065000000000003e-05 2024-09-17T21:20:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954885+0000", 2024-09-17T21:20:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4183000000000001e-05 2024-09-17T21:20:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.820 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:07.820 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:07.820 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:07.821 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:20:07.822 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:07.815+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:07.839 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 0] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:20:07.839 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:20:07.839 INFO:tasks.thrashosds.thrasher:Removing osd 5, in_osds are: [1, 3, 4, 5, 0] 2024-09-17T21:20:07.839 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 5 2024-09-17T21:20:07.922 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:07.917+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:07.995 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:07.995 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:07.995 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:20:07.995 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:08.010 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:20:08.023 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:08.017+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:08.124 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:08.117+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:08.129 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:08.129 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:08.129 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:08.130 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:08.130 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:08.145 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:20:08.180 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-09-17T21:20:08.173+0000 7fe37768f640 -1 mon.a@0(leader).osd e108 definitely_dead 0 2024-09-17T21:20:08.224 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:08.219+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:marked out osd.5. 2024-09-17T21:20:08.273 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:20:08.274 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:08.274 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:08.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:08.275 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:08.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.275 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5541.0:2643 3.1a 3:5ab2b679:::rbd_data.115266d0b9f2.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 917504~4608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:20:08.275 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.510099+0000", 2024-09-17T21:20:08.275 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.761391092, 2024-09-17T21:20:08.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.053163717999999999, 2024-09-17T21:20:08.275 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.275 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.275 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:08.275 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.275 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5541", 2024-09-17T21:20:08.275 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:20:08.275 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2643 2024-09-17T21:20:08.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.275 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.510099+0000", 2024-09-17T21:20:08.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.510099+0000", 2024-09-17T21:20:08.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.510100+0000", 2024-09-17T21:20:08.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.668e-06 2024-09-17T21:20:08.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527298+0000", 2024-09-17T21:20:08.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017198108 2024-09-17T21:20:08.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527333+0000", 2024-09-17T21:20:08.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4304999999999998e-05 2024-09-17T21:20:08.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527414+0000", 2024-09-17T21:20:08.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1009999999999999e-05 2024-09-17T21:20:08.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527529+0000", 2024-09-17T21:20:08.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000115122 2024-09-17T21:20:08.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527653+0000", 2024-09-17T21:20:08.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012405900000000001 2024-09-17T21:20:08.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:20:08.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527950+0000", 2024-09-17T21:20:08.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029670599999999999 2024-09-17T21:20:08.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:08.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.553334+0000", 2024-09-17T21:20:08.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025384750000000001 2024-09-17T21:20:08.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:08.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.563128+0000", 2024-09-17T21:20:08.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0097937179999999999 2024-09-17T21:20:08.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:08.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.563201+0000", 2024-09-17T21:20:08.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3128999999999997e-05 2024-09-17T21:20:08.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.563262+0000", 2024-09-17T21:20:08.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1142999999999994e-05 2024-09-17T21:20:08.280 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.281 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.281 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.281 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5541.0:3004 3.a 3:524d69f6:::rbd_data.115266d0b9f2.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T21:20:08.281 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:50.606531+0000", 2024-09-17T21:20:08.281 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.664959140000001, 2024-09-17T21:20:08.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049461896999999998, 2024-09-17T21:20:08.281 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.281 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.281 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:08.281 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.281 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5541", 2024-09-17T21:20:08.281 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:20:08.281 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3004 2024-09-17T21:20:08.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.282 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.606531+0000", 2024-09-17T21:20:08.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.606531+0000", 2024-09-17T21:20:08.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.606533+0000", 2024-09-17T21:20:08.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.03e-06 2024-09-17T21:20:08.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617600+0000", 2024-09-17T21:20:08.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011067169999999999 2024-09-17T21:20:08.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617620+0000", 2024-09-17T21:20:08.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0627000000000001e-05 2024-09-17T21:20:08.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617631+0000", 2024-09-17T21:20:08.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0991e-05 2024-09-17T21:20:08.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617713+0000", 2024-09-17T21:20:08.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1241000000000004e-05 2024-09-17T21:20:08.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617903+0000", 2024-09-17T21:20:08.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019016599999999999 2024-09-17T21:20:08.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:08.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.618842+0000", 2024-09-17T21:20:08.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00093906399999999998 2024-09-17T21:20:08.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:08.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.654887+0000", 2024-09-17T21:20:08.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.036045176999999998 2024-09-17T21:20:08.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:08.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.655762+0000", 2024-09-17T21:20:08.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00087534500000000003 2024-09-17T21:20:08.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:08.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.655900+0000", 2024-09-17T21:20:08.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013781700000000001 2024-09-17T21:20:08.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.655992+0000", 2024-09-17T21:20:08.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2268999999999994e-05 2024-09-17T21:20:08.286 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.286 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.286 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.286 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1489 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:08.286 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.014521+0000", 2024-09-17T21:20:08.286 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.256968386, 2024-09-17T21:20:08.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047072140999999998, 2024-09-17T21:20:08.286 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.287 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.287 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.287 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.287 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:08.287 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:08.287 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1489 2024-09-17T21:20:08.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.287 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014521+0000", 2024-09-17T21:20:08.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014521+0000", 2024-09-17T21:20:08.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014522+0000", 2024-09-17T21:20:08.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8199999999999999e-07 2024-09-17T21:20:08.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014525+0000", 2024-09-17T21:20:08.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2799999999999999e-06 2024-09-17T21:20:08.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014536+0000", 2024-09-17T21:20:08.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0463e-05 2024-09-17T21:20:08.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014544+0000", 2024-09-17T21:20:08.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:20:08.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.061185+0000", 2024-09-17T21:20:08.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046641021999999997 2024-09-17T21:20:08.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.061263+0000", 2024-09-17T21:20:08.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7973999999999996e-05 2024-09-17T21:20:08.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.061593+0000", 2024-09-17T21:20:08.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033020199999999998 2024-09-17T21:20:08.290 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.290 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.290 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.291 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1521 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 638976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:08.291 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.018585+0000", 2024-09-17T21:20:08.291 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.252904707999999, 2024-09-17T21:20:08.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046751124999999998, 2024-09-17T21:20:08.291 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.291 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.291 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.291 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.291 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:08.291 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:08.291 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1521 2024-09-17T21:20:08.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.291 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018585+0000", 2024-09-17T21:20:08.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018585+0000", 2024-09-17T21:20:08.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018586+0000", 2024-09-17T21:20:08.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7999999999999993e-07 2024-09-17T21:20:08.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018590+0000", 2024-09-17T21:20:08.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2719999999999999e-06 2024-09-17T21:20:08.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018605+0000", 2024-09-17T21:20:08.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4637000000000001e-05 2024-09-17T21:20:08.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018615+0000", 2024-09-17T21:20:08.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.048e-05 2024-09-17T21:20:08.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065019+0000", 2024-09-17T21:20:08.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046403563000000002 2024-09-17T21:20:08.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065073+0000", 2024-09-17T21:20:08.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4521000000000002e-05 2024-09-17T21:20:08.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065336+0000", 2024-09-17T21:20:08.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026267199999999998 2024-09-17T21:20:08.294 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.294 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.295 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.295 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1522 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:08.295 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.018637+0000", 2024-09-17T21:20:08.295 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.252852860000001, 2024-09-17T21:20:08.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047019501999999998, 2024-09-17T21:20:08.295 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.295 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.295 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.295 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.295 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:08.295 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:08.295 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1522 2024-09-17T21:20:08.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.296 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018637+0000", 2024-09-17T21:20:08.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018637+0000", 2024-09-17T21:20:08.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018638+0000", 2024-09-17T21:20:08.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3570000000000001e-06 2024-09-17T21:20:08.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018643+0000", 2024-09-17T21:20:08.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7400000000000004e-06 2024-09-17T21:20:08.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018657+0000", 2024-09-17T21:20:08.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3617e-05 2024-09-17T21:20:08.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018667+0000", 2024-09-17T21:20:08.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0392e-05 2024-09-17T21:20:08.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065370+0000", 2024-09-17T21:20:08.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046703100999999997 2024-09-17T21:20:08.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065427+0000", 2024-09-17T21:20:08.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6920999999999999e-05 2024-09-17T21:20:08.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065656+0000", 2024-09-17T21:20:08.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022937399999999999 2024-09-17T21:20:08.299 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.299 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.299 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.299 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1523 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 704512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:08.299 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.018682+0000", 2024-09-17T21:20:08.299 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.252807462, 2024-09-17T21:20:08.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047547589000000001, 2024-09-17T21:20:08.299 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.299 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.299 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:08.300 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:08.300 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1523 2024-09-17T21:20:08.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.300 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018682+0000", 2024-09-17T21:20:08.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018682+0000", 2024-09-17T21:20:08.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018683+0000", 2024-09-17T21:20:08.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1400000000000001e-06 2024-09-17T21:20:08.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018688+0000", 2024-09-17T21:20:08.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4900000000000002e-06 2024-09-17T21:20:08.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018703+0000", 2024-09-17T21:20:08.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4837e-05 2024-09-17T21:20:08.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018714+0000", 2024-09-17T21:20:08.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0882e-05 2024-09-17T21:20:08.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065769+0000", 2024-09-17T21:20:08.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047055579 2024-09-17T21:20:08.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065910+0000", 2024-09-17T21:20:08.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014135100000000001 2024-09-17T21:20:08.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.066230+0000", 2024-09-17T21:20:08.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031931 2024-09-17T21:20:08.303 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.303 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.303 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.303 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1552 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:08.303 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022280+0000", 2024-09-17T21:20:08.304 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.249209922, 2024-09-17T21:20:08.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047789951999999997, 2024-09-17T21:20:08.304 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.304 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.304 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.304 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.304 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:08.304 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:08.304 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1552 2024-09-17T21:20:08.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.304 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022280+0000", 2024-09-17T21:20:08.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022280+0000", 2024-09-17T21:20:08.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022280+0000", 2024-09-17T21:20:08.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:20:08.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022283+0000", 2024-09-17T21:20:08.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4080000000000001e-06 2024-09-17T21:20:08.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022289+0000", 2024-09-17T21:20:08.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.477e-06 2024-09-17T21:20:08.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022295+0000", 2024-09-17T21:20:08.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5439999999999998e-06 2024-09-17T21:20:08.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.069004+0000", 2024-09-17T21:20:08.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046709222000000002 2024-09-17T21:20:08.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.069152+0000", 2024-09-17T21:20:08.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014832300000000001 2024-09-17T21:20:08.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.070070+0000", 2024-09-17T21:20:08.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00091743300000000005 2024-09-17T21:20:08.307 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.307 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.308 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.308 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1553 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:08.308 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022302+0000", 2024-09-17T21:20:08.308 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.249187535000001, 2024-09-17T21:20:08.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048707517999999998, 2024-09-17T21:20:08.308 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.308 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.308 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.308 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.308 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:08.308 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:08.308 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1553 2024-09-17T21:20:08.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.309 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022302+0000", 2024-09-17T21:20:08.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022302+0000", 2024-09-17T21:20:08.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022303+0000", 2024-09-17T21:20:08.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2300000000000001e-07 2024-09-17T21:20:08.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022305+0000", 2024-09-17T21:20:08.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4700000000000001e-06 2024-09-17T21:20:08.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022312+0000", 2024-09-17T21:20:08.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.494e-06 2024-09-17T21:20:08.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022320+0000", 2024-09-17T21:20:08.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1969999999999996e-06 2024-09-17T21:20:08.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.070138+0000", 2024-09-17T21:20:08.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047818685 2024-09-17T21:20:08.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.070323+0000", 2024-09-17T21:20:08.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000184186 2024-09-17T21:20:08.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071010+0000", 2024-09-17T21:20:08.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00068696299999999996 2024-09-17T21:20:08.312 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.312 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.312 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.312 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1554 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:08.312 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022328+0000", 2024-09-17T21:20:08.312 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.249161693, 2024-09-17T21:20:08.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049212936999999998, 2024-09-17T21:20:08.312 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.312 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.312 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.312 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.313 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:08.313 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:08.313 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1554 2024-09-17T21:20:08.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.313 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022328+0000", 2024-09-17T21:20:08.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022328+0000", 2024-09-17T21:20:08.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022328+0000", 2024-09-17T21:20:08.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3000000000000001e-07 2024-09-17T21:20:08.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022331+0000", 2024-09-17T21:20:08.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3539999999999998e-06 2024-09-17T21:20:08.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022338+0000", 2024-09-17T21:20:08.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8800000000000002e-06 2024-09-17T21:20:08.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022343+0000", 2024-09-17T21:20:08.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6149999999999996e-06 2024-09-17T21:20:08.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071051+0000", 2024-09-17T21:20:08.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048707179000000003 2024-09-17T21:20:08.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071171+0000", 2024-09-17T21:20:08.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000120036 2024-09-17T21:20:08.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071541+0000", 2024-09-17T21:20:08.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037034299999999997 2024-09-17T21:20:08.317 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.317 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.317 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.317 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1555 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:08.317 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022352+0000", 2024-09-17T21:20:08.317 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.249137527, 2024-09-17T21:20:08.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049642617, 2024-09-17T21:20:08.317 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.317 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.317 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.317 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.318 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:08.318 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:08.318 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1555 2024-09-17T21:20:08.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.318 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022352+0000", 2024-09-17T21:20:08.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022352+0000", 2024-09-17T21:20:08.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022353+0000", 2024-09-17T21:20:08.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7000000000000005e-07 2024-09-17T21:20:08.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022355+0000", 2024-09-17T21:20:08.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4569999999999999e-06 2024-09-17T21:20:08.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022364+0000", 2024-09-17T21:20:08.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2550000000000003e-06 2024-09-17T21:20:08.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022374+0000", 2024-09-17T21:20:08.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2820000000000002e-06 2024-09-17T21:20:08.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071626+0000", 2024-09-17T21:20:08.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049251892999999998 2024-09-17T21:20:08.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071703+0000", 2024-09-17T21:20:08.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7273000000000004e-05 2024-09-17T21:20:08.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071995+0000", 2024-09-17T21:20:08.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029188700000000001 2024-09-17T21:20:08.321 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.321 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.321 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.321 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1556 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 663552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:08.321 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022388+0000", 2024-09-17T21:20:08.321 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.249101982999999, 2024-09-17T21:20:08.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050001666, 2024-09-17T21:20:08.322 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.322 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.322 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.322 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.322 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:08.322 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:08.322 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1556 2024-09-17T21:20:08.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.322 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022388+0000", 2024-09-17T21:20:08.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022388+0000", 2024-09-17T21:20:08.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022389+0000", 2024-09-17T21:20:08.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.282e-06 2024-09-17T21:20:08.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022394+0000", 2024-09-17T21:20:08.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6779999999999999e-06 2024-09-17T21:20:08.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022407+0000", 2024-09-17T21:20:08.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3309000000000001e-05 2024-09-17T21:20:08.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022417+0000", 2024-09-17T21:20:08.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0152e-05 2024-09-17T21:20:08.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072033+0000", 2024-09-17T21:20:08.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049616014999999999 2024-09-17T21:20:08.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072137+0000", 2024-09-17T21:20:08.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104214 2024-09-17T21:20:08.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072389+0000", 2024-09-17T21:20:08.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025201599999999998 2024-09-17T21:20:08.325 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.325 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.325 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.326 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1557 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:08.326 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022434+0000", 2024-09-17T21:20:08.326 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.249055930000001, 2024-09-17T21:20:08.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050377024999999999, 2024-09-17T21:20:08.326 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.326 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.326 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.326 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.326 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:08.326 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:08.326 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1557 2024-09-17T21:20:08.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.326 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022434+0000", 2024-09-17T21:20:08.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022434+0000", 2024-09-17T21:20:08.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022435+0000", 2024-09-17T21:20:08.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9700000000000005e-07 2024-09-17T21:20:08.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022439+0000", 2024-09-17T21:20:08.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1899999999999997e-06 2024-09-17T21:20:08.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022452+0000", 2024-09-17T21:20:08.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2865e-05 2024-09-17T21:20:08.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022462+0000", 2024-09-17T21:20:08.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.91e-06 2024-09-17T21:20:08.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072452+0000", 2024-09-17T21:20:08.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049990290999999999 2024-09-17T21:20:08.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072541+0000", 2024-09-17T21:20:08.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9297999999999998e-05 2024-09-17T21:20:08.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072811+0000", 2024-09-17T21:20:08.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026957400000000002 2024-09-17T21:20:08.330 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.330 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.330 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.330 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1584 3.e 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:1b [read 753664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:08.330 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.026146+0000", 2024-09-17T21:20:08.330 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.245343739000001, 2024-09-17T21:20:08.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049235534999999997, 2024-09-17T21:20:08.330 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.331 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.331 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.331 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.331 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:08.331 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:08.331 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1584 2024-09-17T21:20:08.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.331 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026146+0000", 2024-09-17T21:20:08.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026146+0000", 2024-09-17T21:20:08.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026147+0000", 2024-09-17T21:20:08.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0249999999999999e-06 2024-09-17T21:20:08.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026151+0000", 2024-09-17T21:20:08.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:20:08.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026169+0000", 2024-09-17T21:20:08.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7294000000000001e-05 2024-09-17T21:20:08.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026179+0000", 2024-09-17T21:20:08.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0084999999999999e-05 2024-09-17T21:20:08.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072874+0000", 2024-09-17T21:20:08.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046695497000000002 2024-09-17T21:20:08.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072976+0000", 2024-09-17T21:20:08.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101358 2024-09-17T21:20:08.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.075381+0000", 2024-09-17T21:20:08.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002405726 2024-09-17T21:20:08.334 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.334 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.334 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.335 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1585 3.e 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:1b [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:08.335 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.026195+0000", 2024-09-17T21:20:08.335 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.245294611, 2024-09-17T21:20:08.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051427650999999998, 2024-09-17T21:20:08.335 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.335 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.335 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.335 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.335 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:08.335 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:08.335 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1585 2024-09-17T21:20:08.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.335 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026195+0000", 2024-09-17T21:20:08.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026195+0000", 2024-09-17T21:20:08.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026196+0000", 2024-09-17T21:20:08.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7300000000000004e-07 2024-09-17T21:20:08.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026200+0000", 2024-09-17T21:20:08.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.425e-06 2024-09-17T21:20:08.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026214+0000", 2024-09-17T21:20:08.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3108999999999999e-05 2024-09-17T21:20:08.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026224+0000", 2024-09-17T21:20:08.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.042e-05 2024-09-17T21:20:08.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.075431+0000", 2024-09-17T21:20:08.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049206543999999998 2024-09-17T21:20:08.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.075489+0000", 2024-09-17T21:20:08.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8053000000000001e-05 2024-09-17T21:20:08.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.077623+0000", 2024-09-17T21:20:08.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002134127 2024-09-17T21:20:08.339 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.339 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.339 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.339 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1586 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:08.339 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.026238+0000", 2024-09-17T21:20:08.339 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.245251852999999, 2024-09-17T21:20:08.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051776152999999998, 2024-09-17T21:20:08.339 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.339 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.339 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.339 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.339 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:08.339 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:08.339 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1586 2024-09-17T21:20:08.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.340 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026238+0000", 2024-09-17T21:20:08.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026238+0000", 2024-09-17T21:20:08.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026239+0000", 2024-09-17T21:20:08.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0380000000000001e-06 2024-09-17T21:20:08.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026243+0000", 2024-09-17T21:20:08.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3499999999999999e-06 2024-09-17T21:20:08.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026259+0000", 2024-09-17T21:20:08.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5928999999999999e-05 2024-09-17T21:20:08.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026270+0000", 2024-09-17T21:20:08.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0702e-05 2024-09-17T21:20:08.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.077682+0000", 2024-09-17T21:20:08.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051411865000000001 2024-09-17T21:20:08.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.077751+0000", 2024-09-17T21:20:08.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9606999999999999e-05 2024-09-17T21:20:08.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078014+0000", 2024-09-17T21:20:08.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026266199999999999 2024-09-17T21:20:08.343 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.343 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.343 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.343 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1587 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 802816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:08.343 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.026286+0000", 2024-09-17T21:20:08.343 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.245204074, 2024-09-17T21:20:08.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.052384495000000003, 2024-09-17T21:20:08.344 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.344 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.344 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.344 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.344 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:08.344 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:08.344 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1587 2024-09-17T21:20:08.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.344 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026286+0000", 2024-09-17T21:20:08.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026286+0000", 2024-09-17T21:20:08.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026287+0000", 2024-09-17T21:20:08.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0049999999999999e-06 2024-09-17T21:20:08.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026291+0000", 2024-09-17T21:20:08.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3479999999999997e-06 2024-09-17T21:20:08.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026304+0000", 2024-09-17T21:20:08.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3288999999999999e-05 2024-09-17T21:20:08.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026314+0000", 2024-09-17T21:20:08.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9920000000000002e-06 2024-09-17T21:20:08.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078052+0000", 2024-09-17T21:20:08.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051737299 2024-09-17T21:20:08.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078118+0000", 2024-09-17T21:20:08.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6180999999999995e-05 2024-09-17T21:20:08.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078670+0000", 2024-09-17T21:20:08.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00055238100000000001 2024-09-17T21:20:08.347 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.348 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.348 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.348 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1618 3.e 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:1b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:08.348 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.031597+0000", 2024-09-17T21:20:08.348 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.239892633, 2024-09-17T21:20:08.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047545707999999999, 2024-09-17T21:20:08.348 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.348 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.348 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.348 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.348 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:08.348 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:08.349 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1618 2024-09-17T21:20:08.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.349 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031597+0000", 2024-09-17T21:20:08.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031597+0000", 2024-09-17T21:20:08.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031598+0000", 2024-09-17T21:20:08.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:20:08.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031602+0000", 2024-09-17T21:20:08.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:20:08.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031630+0000", 2024-09-17T21:20:08.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8058000000000001e-05 2024-09-17T21:20:08.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031643+0000", 2024-09-17T21:20:08.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2628e-05 2024-09-17T21:20:08.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078728+0000", 2024-09-17T21:20:08.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047085044 2024-09-17T21:20:08.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078830+0000", 2024-09-17T21:20:08.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102493 2024-09-17T21:20:08.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079143+0000", 2024-09-17T21:20:08.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031263599999999998 2024-09-17T21:20:08.352 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.352 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.352 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.352 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1619 3.e 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:1b [read 827392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:08.352 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.031661+0000", 2024-09-17T21:20:08.353 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.239829066, 2024-09-17T21:20:08.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047908566999999999, 2024-09-17T21:20:08.353 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.353 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.353 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:08.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:08.353 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1619 2024-09-17T21:20:08.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.353 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031661+0000", 2024-09-17T21:20:08.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031661+0000", 2024-09-17T21:20:08.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031662+0000", 2024-09-17T21:20:08.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.252e-06 2024-09-17T21:20:08.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031666+0000", 2024-09-17T21:20:08.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6299999999999997e-06 2024-09-17T21:20:08.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031683+0000", 2024-09-17T21:20:08.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6035000000000001e-05 2024-09-17T21:20:08.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031693+0000", 2024-09-17T21:20:08.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0492000000000001e-05 2024-09-17T21:20:08.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079186+0000", 2024-09-17T21:20:08.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047493441999999997 2024-09-17T21:20:08.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079315+0000", 2024-09-17T21:20:08.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012860000000000001 2024-09-17T21:20:08.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079569+0000", 2024-09-17T21:20:08.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025411599999999998 2024-09-17T21:20:08.356 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.356 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.356 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.357 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1620 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:08.357 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.031706+0000", 2024-09-17T21:20:08.357 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.239783642000001, 2024-09-17T21:20:08.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048561143000000001, 2024-09-17T21:20:08.357 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.357 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.357 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.357 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.357 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:08.357 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:08.357 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1620 2024-09-17T21:20:08.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.357 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031706+0000", 2024-09-17T21:20:08.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031706+0000", 2024-09-17T21:20:08.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031707+0000", 2024-09-17T21:20:08.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5799999999999998e-07 2024-09-17T21:20:08.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031711+0000", 2024-09-17T21:20:08.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1930000000000003e-06 2024-09-17T21:20:08.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031724+0000", 2024-09-17T21:20:08.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.24e-05 2024-09-17T21:20:08.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031733+0000", 2024-09-17T21:20:08.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6239999999999997e-06 2024-09-17T21:20:08.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079613+0000", 2024-09-17T21:20:08.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047880082999999997 2024-09-17T21:20:08.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079719+0000", 2024-09-17T21:20:08.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010559499999999999 2024-09-17T21:20:08.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.080267+0000", 2024-09-17T21:20:08.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00054829 2024-09-17T21:20:08.360 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.361 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.361 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.361 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1621 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:08.361 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.031746+0000", 2024-09-17T21:20:08.361 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.239743168, 2024-09-17T21:20:08.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048925189000000001, 2024-09-17T21:20:08.361 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.361 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.361 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.361 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.361 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:08.361 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:08.362 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1621 2024-09-17T21:20:08.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.362 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031746+0000", 2024-09-17T21:20:08.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031746+0000", 2024-09-17T21:20:08.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031747+0000", 2024-09-17T21:20:08.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3900000000000003e-07 2024-09-17T21:20:08.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031752+0000", 2024-09-17T21:20:08.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.172e-06 2024-09-17T21:20:08.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031767+0000", 2024-09-17T21:20:08.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5517000000000001e-05 2024-09-17T21:20:08.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031777+0000", 2024-09-17T21:20:08.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8579999999999995e-06 2024-09-17T21:20:08.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.080318+0000", 2024-09-17T21:20:08.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048541009000000003 2024-09-17T21:20:08.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.080415+0000", 2024-09-17T21:20:08.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7034999999999995e-05 2024-09-17T21:20:08.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.080672+0000", 2024-09-17T21:20:08.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025665900000000001 2024-09-17T21:20:08.365 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.365 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.365 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.365 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.365 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.365 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:08.366 INFO:teuthology.orchestra.run.smithi184.stdout:4630 read 0x6682400 thru 0x668b331 (0x8f32 bytes) 2024-09-17T21:20:08.366 INFO:teuthology.orchestra.run.smithi184.stdout:4633 read 0x7076800 thru 0x707a769 (0x3f6a bytes) 2024-09-17T21:20:08.367 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:20:08.368 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:08.359+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:08.467 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:08.463+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:08.537 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:08.537 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:08.537 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:20:08.538 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:08.552 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:20:08.568 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:08.561+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:08.669 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:08.663+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:08.710 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:08.711 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:08.711 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:08.711 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:08.711 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:08.725 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:20:08.769 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:08.763+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:08.870 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:08.863+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:08.894 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:08.894 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:08.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:08.894 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:08.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.894 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:20:08.894 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:08.894 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 107.285927603, 2024-09-17T21:20:08.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:20:08.894 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.894 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.894 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:08.894 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:08.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:08.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:20:08.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:20:08.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:20:08.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:20:08.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:20:08.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:20:08.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:20:08.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:20:08.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.897 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.897 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:20:08.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:20:08.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.897 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.897 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:20:08.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:20:08.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.897 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:08.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:20:08.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:20:08.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:20:08.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:20:08.898 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.898 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.898 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.898 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:20:08.899 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:08.899 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 107.275980967, 2024-09-17T21:20:08.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:20:08.899 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.899 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.899 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:08.899 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:08.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:08.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:20:08.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:20:08.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:20:08.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:20:08.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:20:08.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:20:08.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:20:08.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:20:08.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:20:08.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:20:08.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.902 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:20:08.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:20:08.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:08.902 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:20:08.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:20:08.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:20:08.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:20:08.903 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.903 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.903 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.903 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:20:08.903 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:08.903 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 106.032830127, 2024-09-17T21:20:08.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:20:08.903 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.903 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.903 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:08.903 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.904 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:20:08.904 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:20:08.904 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:20:08.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.904 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:08.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:08.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:20:08.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:20:08.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:20:08.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:20:08.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:20:08.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:20:08.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:20:08.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:20:08.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:20:08.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:20:08.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:20:08.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:20:08.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:08.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:20:08.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:20:08.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:08.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:20:08.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:20:08.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:08.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:20:08.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:20:08.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:08.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:20:08.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:20:08.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:20:08.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:20:08.908 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.908 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.908 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.909 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:20:08.909 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:08.909 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 106.023184641, 2024-09-17T21:20:08.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:20:08.909 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.909 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.909 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:08.909 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.909 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:20:08.909 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:20:08.909 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:20:08.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.909 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.910 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.910 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:08.910 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.910 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.910 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:08.910 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.910 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.910 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:20:08.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:20:08.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.911 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:20:08.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:20:08.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.911 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:20:08.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:20:08.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:20:08.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:20:08.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:20:08.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:20:08.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:20:08.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:20:08.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:08.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:20:08.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:20:08.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.913 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:08.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:20:08.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:20:08.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.913 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:08.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:20:08.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:20:08.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.914 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.914 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:08.914 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:20:08.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:20:08.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.914 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.914 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.914 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:20:08.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:20:08.914 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.914 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.914 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.914 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.915 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:20:08.915 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:08.915 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 92.654248608000003, 2024-09-17T21:20:08.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:20:08.915 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.915 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.915 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.915 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.915 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:20:08.915 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:20:08.915 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:20:08.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.915 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.915 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:08.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:08.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:20:08.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:20:08.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.917 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:20:08.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:20:08.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.917 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:20:08.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:20:08.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.917 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:20:08.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:20:08.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.918 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.918 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:20:08.918 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:20:08.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.918 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.918 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:20:08.918 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:20:08.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.918 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.918 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:20:08.918 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:20:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:819 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5111808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.452676318000002, 2024-09-17T21:20:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024344811000000001, 2024-09-17T21:20:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.919 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 819 2024-09-17T21:20:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437443+0000", 2024-09-17T21:20:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1969999999999999e-06 2024-09-17T21:20:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437448+0000", 2024-09-17T21:20:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9490000000000004e-06 2024-09-17T21:20:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437467+0000", 2024-09-17T21:20:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8862999999999999e-05 2024-09-17T21:20:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437480+0000", 2024-09-17T21:20:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2833000000000001e-05 2024-09-17T21:20:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461552+0000", 2024-09-17T21:20:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024072695000000002 2024-09-17T21:20:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461639+0000", 2024-09-17T21:20:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6996000000000005e-05 2024-09-17T21:20:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461787+0000", 2024-09-17T21:20:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014727800000000001 2024-09-17T21:20:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:08.923 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.452564046000006, 2024-09-17T21:20:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:20:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:20:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:08.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:20:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:20:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:20:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:20:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:20:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:20:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:20:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:20:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:20:08.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:20:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:20:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:20:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:20:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:20:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:833 3.5 3:a772dfb1:::rbd_data.11523651b015.0000000000000001:d [sparse-read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.452129760000005, 2024-09-17T21:20:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024216024999999999, 2024-09-17T21:20:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 833 2024-09-17T21:20:08.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437990+0000", 2024-09-17T21:20:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2500000000000001e-06 2024-09-17T21:20:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437995+0000", 2024-09-17T21:20:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1240000000000004e-06 2024-09-17T21:20:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438012+0000", 2024-09-17T21:20:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7572e-05 2024-09-17T21:20:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438031+0000", 2024-09-17T21:20:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8612000000000001e-05 2024-09-17T21:20:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461830+0000", 2024-09-17T21:20:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023799135999999999 2024-09-17T21:20:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462005+0000", 2024-09-17T21:20:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000174747 2024-09-17T21:20:08.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:20:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019958399999999999 2024-09-17T21:20:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.452069468999994, 2024-09-17T21:20:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:20:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.932 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:20:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:20:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:20:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:20:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:20:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:20:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:20:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:20:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:20:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:20:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:20:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:20:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:20:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:20:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:20:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.936 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.452014087999999, 2024-09-17T21:20:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:20:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:20:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.937 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:20:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:20:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:20:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:20:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:20:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:20:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:20:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:20:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:20:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:20:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:20:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:20:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:20:08.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:20:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.451957809999996, 2024-09-17T21:20:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:20:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:08.941 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:20:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:20:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:20:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:20:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:20:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:20:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:20:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:20:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:20:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:20:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:20:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:20:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:20:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:20:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:20:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.451008170999998, 2024-09-17T21:20:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:20:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.945 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:20:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:20:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:20:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:20:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:20:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:20:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:20:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:20:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:20:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:20:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:20:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:20:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:20:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:20:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:20:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:08.949 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.450929551000002, 2024-09-17T21:20:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:20:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:20:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:08.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:20:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:20:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:20:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:20:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:20:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:20:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:20:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:20:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:20:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:20:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:20:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:20:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:20:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:20:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.450816611999997, 2024-09-17T21:20:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:20:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:20:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.954 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:20:08.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:20:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:20:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:20:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:20:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:20:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.956 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:20:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:20:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:20:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:20:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:20:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:20:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:20:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:20:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:849 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5963776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.450222576000002, 2024-09-17T21:20:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024537514, 2024-09-17T21:20:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.958 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 849 2024-09-17T21:20:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:08.959 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439897+0000", 2024-09-17T21:20:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9099999999999999e-06 2024-09-17T21:20:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439905+0000", 2024-09-17T21:20:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0870000000000007e-06 2024-09-17T21:20:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.960 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439924+0000", 2024-09-17T21:20:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9042000000000002e-05 2024-09-17T21:20:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439941+0000", 2024-09-17T21:20:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.624e-05 2024-09-17T21:20:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464132+0000", 2024-09-17T21:20:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024191579000000001 2024-09-17T21:20:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464291+0000", 2024-09-17T21:20:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015904800000000001 2024-09-17T21:20:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464433+0000", 2024-09-17T21:20:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014160799999999999 2024-09-17T21:20:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.450155558000006, 2024-09-17T21:20:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:20:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:20:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:08.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:20:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:20:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:20:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:20:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:20:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:20:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:20:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:20:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:20:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:20:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:20:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:20:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:20:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:20:08.966 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.450090747000004, 2024-09-17T21:20:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:20:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:08.967 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:20:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:20:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:20:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:20:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:20:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:20:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:20:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:20:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:20:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:20:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:20:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:20:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:20:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:20:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:20:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.45002977, 2024-09-17T21:20:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:20:08.971 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:20:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:20:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:20:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:20:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:20:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:20:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:20:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:20:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:20:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:20:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:20:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:20:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:20:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:20:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:20:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:20:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.552458585, 2024-09-17T21:20:08.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:20:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:20:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:20:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:20:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:20:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:20:08.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:20:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:20:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:20:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:20:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:20:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:20:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:20:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:20:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:20:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:20:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:20:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.979 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:20:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.343800191, 2024-09-17T21:20:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:20:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:20:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:20:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:08.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:20:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:20:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:20:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:20:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:20:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:20:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:20:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:20:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:20:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:20:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:20:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:20:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:08.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:08.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:20:08.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:20:08.984 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.984 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.984 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.984 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:08.984 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:08.984 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:08.984 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:20:08.986 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:08.979+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:08.986 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:20:09.087 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:09.083+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:09.137 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:09.137 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:09.154 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:09.154 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:09.158 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:09.158 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:09.171 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:09.171 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:09.172 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:20:09.192 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:09.193 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:09.193 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:09.189+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:09.213 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:09.214 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:09.234 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:09.235 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:09.250 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-17T21:20:09.295 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:09.295 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:09.295 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:09.289+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:09.309 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:20:09.397 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:09.393+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:09.469 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:09.469 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:09.483 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:20:09.491 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:09.487+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:09.592 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:09.587+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:09.649 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:09.649 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:09.663 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:20:09.693 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:09.687+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:09.789 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:09.789 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:09.793 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:09.787+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:09.803 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:20:09.894 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:09.887+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:09.920 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:09.920 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:09.934 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:20:09.995 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:09.989+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:10.060 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:10.060 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:10.072 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:20:10.096 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:10.091+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:10.197 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:10.191+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:10.234 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:10.234 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:10.248 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:20:10.291 INFO:teuthology.orchestra.run.smithi184.stdout:4634 read 0x941ea00 thru 0x9427649 (0x8c4a bytes) 2024-09-17T21:20:10.297 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:10.291+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:10.304 INFO:teuthology.orchestra.run.smithi184.stdout:4635 read 0x7726800 thru 0x7729bd3 (0x33d4 bytes) 2024-09-17T21:20:10.306 INFO:teuthology.orchestra.run.smithi184.stdout:4636 punch from 0x114f400 to 0x1156600, (0x7200 bytes) 2024-09-17T21:20:10.362 INFO:teuthology.orchestra.run.smithi184.stdout:4637 write 0x5190a00 thru 0x519e7cd (0xddce bytes) 2024-09-17T21:20:10.389 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:10.389 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:10.398 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:10.391+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:10.406 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:20:10.449 INFO:teuthology.orchestra.run.smithi184.stdout:4641 trunc from 0xd120c00 to 0x2b55400 2024-09-17T21:20:10.498 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:10.491+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:10.572 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:10.572 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:10.588 INFO:teuthology.orchestra.run.smithi184.stdout:4643 write 0xe21e000 thru 0xe229a46 (0xba47 bytes) 2024-09-17T21:20:10.590 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:20:10.590 INFO:teuthology.orchestra.run.smithi184.stdout:4644 punch from 0x900800 to 0x908200, (0x7a00 bytes) 2024-09-17T21:20:10.599 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:10.593+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:10.634 INFO:teuthology.orchestra.run.smithi184.stdout:4649 trunc from 0xe229c00 to 0x72ba00 2024-09-17T21:20:10.700 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:10.695+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:10.733 INFO:teuthology.orchestra.run.smithi184.stdout:4651 read 0x326a00 thru 0x32dd65 (0x7366 bytes) 2024-09-17T21:20:10.737 INFO:teuthology.orchestra.run.smithi184.stdout:4653 punch from 0x211e00 to 0x214800, (0x2a00 bytes) 2024-09-17T21:20:10.737 INFO:teuthology.orchestra.run.smithi184.stdout:4654 read 0xe3c00 thru 0xeb581 (0x7982 bytes) 2024-09-17T21:20:10.754 INFO:teuthology.orchestra.run.smithi184.stdout:4655 trunc from 0x72ba00 to 0x9565600 2024-09-17T21:20:10.768 INFO:teuthology.orchestra.run.smithi184.stdout:4656 read 0xe73c00 thru 0xe79670 (0x5a71 bytes) 2024-09-17T21:20:10.770 INFO:teuthology.orchestra.run.smithi184.stdout:4659 write 0x587e200 thru 0x5882169 (0x3f6a bytes) 2024-09-17T21:20:10.770 INFO:teuthology.orchestra.run.smithi184.stdout:4662 read 0x58f6a00 thru 0x58fb5ca (0x4bcb bytes) 2024-09-17T21:20:10.771 INFO:teuthology.orchestra.run.smithi184.stdout:4666 write 0x5123800 thru 0x5126321 (0x2b22 bytes) 2024-09-17T21:20:10.782 INFO:teuthology.orchestra.run.smithi184.stdout:4667 trunc from 0x9565600 to 0x50a3c00 2024-09-17T21:20:10.782 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:10.782 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:10.800 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:20:10.812 INFO:teuthology.orchestra.run.smithi184.stdout:4669 trunc from 0x50a3c00 to 0x4c18a00 2024-09-17T21:20:10.837 INFO:teuthology.orchestra.run.smithi184.stdout:4672 read 0x1146200 thru 0x1154566 (0xe367 bytes) 2024-09-17T21:20:10.843 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:10.839+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:10.843 INFO:teuthology.orchestra.run.smithi184.stdout:4677 trunc from 0x4c18a00 to 0x712e600 2024-09-17T21:20:10.856 INFO:teuthology.orchestra.run.smithi184.stdout:4679 punch from 0x1857400 to 0x1859800, (0x2400 bytes) 2024-09-17T21:20:10.877 INFO:teuthology.orchestra.run.smithi184.stdout:4681 write 0xa3d7600 thru 0xa3d92ff (0x1d00 bytes) 2024-09-17T21:20:10.882 INFO:teuthology.orchestra.run.smithi184.stdout:4682 trunc from 0xa3d9400 to 0xcbefa00 2024-09-17T21:20:10.901 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:10.895+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:10.904 INFO:teuthology.orchestra.run.smithi184.stdout:4685 punch from 0xbfc1c00 to 0xbfc8600, (0x6a00 bytes) 2024-09-17T21:20:10.926 INFO:teuthology.orchestra.run.smithi184.stdout:4686 trunc from 0xcbefa00 to 0xfa0c00 2024-09-17T21:20:10.951 INFO:teuthology.orchestra.run.smithi184.stdout:4688 read 0xbc8600 thru 0xbcc3e2 (0x3de3 bytes) 2024-09-17T21:20:10.953 INFO:teuthology.orchestra.run.smithi184.stdout:4690 clone 31 order 22 su 131072 sc 5 2024-09-17T21:20:10.985 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:10.985 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:11.002 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:20:11.003 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:10.995+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:11.103 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:11.099+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:11.179 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:11.179 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:11.194 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:20:11.235 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:11.231+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:11.254 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:20:11.275 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:20:11.345 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:11.341+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:11.370 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:11.370 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:11.385 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:20:11.429 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:11.429 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:11.435 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:11.431+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:11.450 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:11.451 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:11.470 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:11.470 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:11.488 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:11.489 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:11.505 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:11.501+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:11.507 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:11.507 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:11.509 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:11.510 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:11.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:11.510 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:11.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.510 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5541.0:2643 3.1a 3:5ab2b679:::rbd_data.115266d0b9f2.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 917504~4608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:20:11.510 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.510099+0000", 2024-09-17T21:20:11.510 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 21.996744062000001, 2024-09-17T21:20:11.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.053163717999999999, 2024-09-17T21:20:11.510 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:11.510 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:11.510 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:11.510 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:11.511 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5541", 2024-09-17T21:20:11.511 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:20:11.511 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2643 2024-09-17T21:20:11.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.511 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:11.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:11.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.510099+0000", 2024-09-17T21:20:11.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:11.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.510099+0000", 2024-09-17T21:20:11.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:11.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.510100+0000", 2024-09-17T21:20:11.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.668e-06 2024-09-17T21:20:11.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:11.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527298+0000", 2024-09-17T21:20:11.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017198108 2024-09-17T21:20:11.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:11.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527333+0000", 2024-09-17T21:20:11.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4304999999999998e-05 2024-09-17T21:20:11.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:11.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527414+0000", 2024-09-17T21:20:11.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1009999999999999e-05 2024-09-17T21:20:11.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:11.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527529+0000", 2024-09-17T21:20:11.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000115122 2024-09-17T21:20:11.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:11.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527653+0000", 2024-09-17T21:20:11.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012405900000000001 2024-09-17T21:20:11.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:20:11.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527950+0000", 2024-09-17T21:20:11.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029670599999999999 2024-09-17T21:20:11.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:11.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.553334+0000", 2024-09-17T21:20:11.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025384750000000001 2024-09-17T21:20:11.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:11.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.563128+0000", 2024-09-17T21:20:11.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0097937179999999999 2024-09-17T21:20:11.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:11.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.563201+0000", 2024-09-17T21:20:11.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3128999999999997e-05 2024-09-17T21:20:11.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:11.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.563262+0000", 2024-09-17T21:20:11.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1142999999999994e-05 2024-09-17T21:20:11.516 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.516 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:11.516 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.516 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5541.0:3004 3.a 3:524d69f6:::rbd_data.115266d0b9f2.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T21:20:11.516 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:50.606531+0000", 2024-09-17T21:20:11.516 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 20.900312110000002, 2024-09-17T21:20:11.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049461896999999998, 2024-09-17T21:20:11.516 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:11.516 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:11.516 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:11.516 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:11.517 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5541", 2024-09-17T21:20:11.517 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:20:11.517 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3004 2024-09-17T21:20:11.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.517 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:11.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:11.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.606531+0000", 2024-09-17T21:20:11.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:11.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.606531+0000", 2024-09-17T21:20:11.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:11.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.606533+0000", 2024-09-17T21:20:11.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.03e-06 2024-09-17T21:20:11.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:11.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617600+0000", 2024-09-17T21:20:11.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011067169999999999 2024-09-17T21:20:11.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:11.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617620+0000", 2024-09-17T21:20:11.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0627000000000001e-05 2024-09-17T21:20:11.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:11.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617631+0000", 2024-09-17T21:20:11.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0991e-05 2024-09-17T21:20:11.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:11.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617713+0000", 2024-09-17T21:20:11.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1241000000000004e-05 2024-09-17T21:20:11.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:11.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617903+0000", 2024-09-17T21:20:11.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019016599999999999 2024-09-17T21:20:11.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:11.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.618842+0000", 2024-09-17T21:20:11.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00093906399999999998 2024-09-17T21:20:11.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:11.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.654887+0000", 2024-09-17T21:20:11.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.036045176999999998 2024-09-17T21:20:11.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:11.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.655762+0000", 2024-09-17T21:20:11.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00087534500000000003 2024-09-17T21:20:11.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:11.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.655900+0000", 2024-09-17T21:20:11.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013781700000000001 2024-09-17T21:20:11.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:11.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.655992+0000", 2024-09-17T21:20:11.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2268999999999994e-05 2024-09-17T21:20:11.522 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.522 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:11.522 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.522 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1489 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:11.522 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.014521+0000", 2024-09-17T21:20:11.522 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.492321356000001, 2024-09-17T21:20:11.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047072140999999998, 2024-09-17T21:20:11.522 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:11.522 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:11.522 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:11.522 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:11.522 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:11.523 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:11.523 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1489 2024-09-17T21:20:11.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.523 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:11.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:11.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014521+0000", 2024-09-17T21:20:11.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:11.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014521+0000", 2024-09-17T21:20:11.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:11.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014522+0000", 2024-09-17T21:20:11.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8199999999999999e-07 2024-09-17T21:20:11.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:11.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014525+0000", 2024-09-17T21:20:11.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2799999999999999e-06 2024-09-17T21:20:11.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:11.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014536+0000", 2024-09-17T21:20:11.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0463e-05 2024-09-17T21:20:11.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:11.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014544+0000", 2024-09-17T21:20:11.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:20:11.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:11.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.061185+0000", 2024-09-17T21:20:11.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046641021999999997 2024-09-17T21:20:11.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:11.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.061263+0000", 2024-09-17T21:20:11.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7973999999999996e-05 2024-09-17T21:20:11.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:11.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.061593+0000", 2024-09-17T21:20:11.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033020199999999998 2024-09-17T21:20:11.526 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.526 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:11.526 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.526 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1521 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 638976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:11.527 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.018585+0000", 2024-09-17T21:20:11.527 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.488257678, 2024-09-17T21:20:11.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046751124999999998, 2024-09-17T21:20:11.528 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:11.528 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:11.529 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:11.529 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:11.529 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:11.529 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:11.529 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1521 2024-09-17T21:20:11.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.529 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:11.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:11.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018585+0000", 2024-09-17T21:20:11.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:11.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018585+0000", 2024-09-17T21:20:11.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:11.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018586+0000", 2024-09-17T21:20:11.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7999999999999993e-07 2024-09-17T21:20:11.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:11.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018590+0000", 2024-09-17T21:20:11.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2719999999999999e-06 2024-09-17T21:20:11.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:11.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018605+0000", 2024-09-17T21:20:11.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4637000000000001e-05 2024-09-17T21:20:11.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:11.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018615+0000", 2024-09-17T21:20:11.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.048e-05 2024-09-17T21:20:11.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:11.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065019+0000", 2024-09-17T21:20:11.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046403563000000002 2024-09-17T21:20:11.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:11.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065073+0000", 2024-09-17T21:20:11.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4521000000000002e-05 2024-09-17T21:20:11.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:11.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065336+0000", 2024-09-17T21:20:11.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026267199999999998 2024-09-17T21:20:11.532 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.532 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:11.533 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.533 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1522 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:11.533 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.018637+0000", 2024-09-17T21:20:11.533 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.488205829999998, 2024-09-17T21:20:11.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047019501999999998, 2024-09-17T21:20:11.533 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:11.533 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:11.533 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:11.533 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:11.533 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:11.533 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:11.534 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1522 2024-09-17T21:20:11.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.534 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:11.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:11.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018637+0000", 2024-09-17T21:20:11.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:11.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018637+0000", 2024-09-17T21:20:11.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:11.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018638+0000", 2024-09-17T21:20:11.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3570000000000001e-06 2024-09-17T21:20:11.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:11.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018643+0000", 2024-09-17T21:20:11.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7400000000000004e-06 2024-09-17T21:20:11.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:11.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018657+0000", 2024-09-17T21:20:11.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3617e-05 2024-09-17T21:20:11.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:11.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018667+0000", 2024-09-17T21:20:11.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0392e-05 2024-09-17T21:20:11.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:11.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065370+0000", 2024-09-17T21:20:11.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046703100999999997 2024-09-17T21:20:11.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:11.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065427+0000", 2024-09-17T21:20:11.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6920999999999999e-05 2024-09-17T21:20:11.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:11.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065656+0000", 2024-09-17T21:20:11.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022937399999999999 2024-09-17T21:20:11.537 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.537 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:11.537 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.537 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1523 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 704512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:11.538 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.018682+0000", 2024-09-17T21:20:11.538 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.488160432000001, 2024-09-17T21:20:11.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047547589000000001, 2024-09-17T21:20:11.538 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:11.538 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:11.538 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:11.538 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:11.538 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:11.538 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:11.538 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1523 2024-09-17T21:20:11.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.538 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:11.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:11.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018682+0000", 2024-09-17T21:20:11.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:11.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018682+0000", 2024-09-17T21:20:11.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:11.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018683+0000", 2024-09-17T21:20:11.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1400000000000001e-06 2024-09-17T21:20:11.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:11.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018688+0000", 2024-09-17T21:20:11.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4900000000000002e-06 2024-09-17T21:20:11.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:11.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018703+0000", 2024-09-17T21:20:11.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4837e-05 2024-09-17T21:20:11.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:11.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018714+0000", 2024-09-17T21:20:11.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0882e-05 2024-09-17T21:20:11.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:11.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065769+0000", 2024-09-17T21:20:11.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047055579 2024-09-17T21:20:11.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:11.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065910+0000", 2024-09-17T21:20:11.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014135100000000001 2024-09-17T21:20:11.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:11.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.066230+0000", 2024-09-17T21:20:11.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031931 2024-09-17T21:20:11.542 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.542 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:11.542 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.543 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1552 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:11.543 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022280+0000", 2024-09-17T21:20:11.543 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.484562892, 2024-09-17T21:20:11.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047789951999999997, 2024-09-17T21:20:11.543 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:11.543 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:11.543 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:11.543 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:11.543 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:11.543 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:11.543 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1552 2024-09-17T21:20:11.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.543 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:11.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:11.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022280+0000", 2024-09-17T21:20:11.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:11.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022280+0000", 2024-09-17T21:20:11.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:11.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022280+0000", 2024-09-17T21:20:11.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:20:11.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:11.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022283+0000", 2024-09-17T21:20:11.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4080000000000001e-06 2024-09-17T21:20:11.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:11.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022289+0000", 2024-09-17T21:20:11.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.477e-06 2024-09-17T21:20:11.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:11.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022295+0000", 2024-09-17T21:20:11.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5439999999999998e-06 2024-09-17T21:20:11.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:11.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.069004+0000", 2024-09-17T21:20:11.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046709222000000002 2024-09-17T21:20:11.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:11.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.069152+0000", 2024-09-17T21:20:11.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014832300000000001 2024-09-17T21:20:11.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:11.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.070070+0000", 2024-09-17T21:20:11.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00091743300000000005 2024-09-17T21:20:11.547 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.547 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:11.547 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.547 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1553 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:11.547 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022302+0000", 2024-09-17T21:20:11.547 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.484540504999998, 2024-09-17T21:20:11.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048707517999999998, 2024-09-17T21:20:11.547 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:11.548 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:11.548 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:11.548 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:11.548 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:11.548 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:11.548 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1553 2024-09-17T21:20:11.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.548 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:11.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:11.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022302+0000", 2024-09-17T21:20:11.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:11.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022302+0000", 2024-09-17T21:20:11.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:11.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022303+0000", 2024-09-17T21:20:11.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2300000000000001e-07 2024-09-17T21:20:11.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:11.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022305+0000", 2024-09-17T21:20:11.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4700000000000001e-06 2024-09-17T21:20:11.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:11.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022312+0000", 2024-09-17T21:20:11.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.494e-06 2024-09-17T21:20:11.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:11.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022320+0000", 2024-09-17T21:20:11.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1969999999999996e-06 2024-09-17T21:20:11.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:11.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.070138+0000", 2024-09-17T21:20:11.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047818685 2024-09-17T21:20:11.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:11.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.070323+0000", 2024-09-17T21:20:11.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000184186 2024-09-17T21:20:11.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:11.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071010+0000", 2024-09-17T21:20:11.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00068696299999999996 2024-09-17T21:20:11.551 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.551 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:11.551 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.552 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1554 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:11.552 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022328+0000", 2024-09-17T21:20:11.552 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.484514662999999, 2024-09-17T21:20:11.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049212936999999998, 2024-09-17T21:20:11.552 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:11.552 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:11.552 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:11.552 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:11.552 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:11.552 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:11.552 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1554 2024-09-17T21:20:11.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.553 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:11.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.553 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:11.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022328+0000", 2024-09-17T21:20:11.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.553 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:11.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022328+0000", 2024-09-17T21:20:11.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:11.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022328+0000", 2024-09-17T21:20:11.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3000000000000001e-07 2024-09-17T21:20:11.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:11.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022331+0000", 2024-09-17T21:20:11.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3539999999999998e-06 2024-09-17T21:20:11.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:11.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022338+0000", 2024-09-17T21:20:11.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8800000000000002e-06 2024-09-17T21:20:11.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:11.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022343+0000", 2024-09-17T21:20:11.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6149999999999996e-06 2024-09-17T21:20:11.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:11.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071051+0000", 2024-09-17T21:20:11.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048707179000000003 2024-09-17T21:20:11.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:11.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071171+0000", 2024-09-17T21:20:11.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000120036 2024-09-17T21:20:11.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:11.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071541+0000", 2024-09-17T21:20:11.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037034299999999997 2024-09-17T21:20:11.556 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.556 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:11.556 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.557 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1555 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:11.557 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022352+0000", 2024-09-17T21:20:11.557 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.484490496999999, 2024-09-17T21:20:11.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049642617, 2024-09-17T21:20:11.557 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:11.557 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:11.557 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:11.557 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:11.557 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:11.557 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:11.557 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1555 2024-09-17T21:20:11.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.558 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:11.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:11.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022352+0000", 2024-09-17T21:20:11.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:11.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022352+0000", 2024-09-17T21:20:11.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:11.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022353+0000", 2024-09-17T21:20:11.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7000000000000005e-07 2024-09-17T21:20:11.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:11.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022355+0000", 2024-09-17T21:20:11.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4569999999999999e-06 2024-09-17T21:20:11.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:11.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022364+0000", 2024-09-17T21:20:11.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2550000000000003e-06 2024-09-17T21:20:11.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:11.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022374+0000", 2024-09-17T21:20:11.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2820000000000002e-06 2024-09-17T21:20:11.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:11.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071626+0000", 2024-09-17T21:20:11.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049251892999999998 2024-09-17T21:20:11.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:11.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071703+0000", 2024-09-17T21:20:11.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7273000000000004e-05 2024-09-17T21:20:11.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:11.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071995+0000", 2024-09-17T21:20:11.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029188700000000001 2024-09-17T21:20:11.561 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.561 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:11.561 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.561 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1556 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 663552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:11.561 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022388+0000", 2024-09-17T21:20:11.561 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.484454953, 2024-09-17T21:20:11.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050001666, 2024-09-17T21:20:11.562 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:11.562 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:11.562 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:11.562 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:11.562 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:11.562 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:11.562 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1556 2024-09-17T21:20:11.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.562 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:11.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.562 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:11.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022388+0000", 2024-09-17T21:20:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022388+0000", 2024-09-17T21:20:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022389+0000", 2024-09-17T21:20:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.282e-06 2024-09-17T21:20:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:11.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022394+0000", 2024-09-17T21:20:11.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6779999999999999e-06 2024-09-17T21:20:11.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:11.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022407+0000", 2024-09-17T21:20:11.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3309000000000001e-05 2024-09-17T21:20:11.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:11.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022417+0000", 2024-09-17T21:20:11.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0152e-05 2024-09-17T21:20:11.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:11.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072033+0000", 2024-09-17T21:20:11.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049616014999999999 2024-09-17T21:20:11.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:11.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072137+0000", 2024-09-17T21:20:11.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104214 2024-09-17T21:20:11.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:11.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072389+0000", 2024-09-17T21:20:11.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025201599999999998 2024-09-17T21:20:11.566 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.566 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:11.566 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.566 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1557 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:11.566 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022434+0000", 2024-09-17T21:20:11.566 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.484408899999998, 2024-09-17T21:20:11.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050377024999999999, 2024-09-17T21:20:11.566 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:11.566 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:11.566 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:11.566 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:11.567 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:11.567 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:11.567 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1557 2024-09-17T21:20:11.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.567 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:11.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:11.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022434+0000", 2024-09-17T21:20:11.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:11.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022434+0000", 2024-09-17T21:20:11.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:11.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022435+0000", 2024-09-17T21:20:11.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9700000000000005e-07 2024-09-17T21:20:11.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:11.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022439+0000", 2024-09-17T21:20:11.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1899999999999997e-06 2024-09-17T21:20:11.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:11.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022452+0000", 2024-09-17T21:20:11.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2865e-05 2024-09-17T21:20:11.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:11.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022462+0000", 2024-09-17T21:20:11.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.91e-06 2024-09-17T21:20:11.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:11.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072452+0000", 2024-09-17T21:20:11.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049990290999999999 2024-09-17T21:20:11.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:11.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072541+0000", 2024-09-17T21:20:11.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9297999999999998e-05 2024-09-17T21:20:11.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:11.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072811+0000", 2024-09-17T21:20:11.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026957400000000002 2024-09-17T21:20:11.570 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.570 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:11.570 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.571 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1584 3.e 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:1b [read 753664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:11.571 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.026146+0000", 2024-09-17T21:20:11.571 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.480696709, 2024-09-17T21:20:11.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049235534999999997, 2024-09-17T21:20:11.571 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:11.571 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:11.571 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:11.571 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:11.571 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:11.571 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:11.571 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1584 2024-09-17T21:20:11.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.572 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:11.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:11.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026146+0000", 2024-09-17T21:20:11.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:11.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026146+0000", 2024-09-17T21:20:11.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:11.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026147+0000", 2024-09-17T21:20:11.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0249999999999999e-06 2024-09-17T21:20:11.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:11.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026151+0000", 2024-09-17T21:20:11.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:20:11.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:11.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026169+0000", 2024-09-17T21:20:11.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7294000000000001e-05 2024-09-17T21:20:11.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:11.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026179+0000", 2024-09-17T21:20:11.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0084999999999999e-05 2024-09-17T21:20:11.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:11.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072874+0000", 2024-09-17T21:20:11.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046695497000000002 2024-09-17T21:20:11.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:11.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072976+0000", 2024-09-17T21:20:11.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101358 2024-09-17T21:20:11.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:11.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.075381+0000", 2024-09-17T21:20:11.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002405726 2024-09-17T21:20:11.575 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.575 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:11.575 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.575 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1585 3.e 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:1b [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:11.575 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.026195+0000", 2024-09-17T21:20:11.575 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.480647580999999, 2024-09-17T21:20:11.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051427650999999998, 2024-09-17T21:20:11.576 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:11.576 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:11.576 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:11.576 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:11.576 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:11.576 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:11.576 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1585 2024-09-17T21:20:11.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.576 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:11.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:11.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026195+0000", 2024-09-17T21:20:11.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:11.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026195+0000", 2024-09-17T21:20:11.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:11.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026196+0000", 2024-09-17T21:20:11.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7300000000000004e-07 2024-09-17T21:20:11.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:11.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026200+0000", 2024-09-17T21:20:11.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.425e-06 2024-09-17T21:20:11.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:11.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026214+0000", 2024-09-17T21:20:11.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3108999999999999e-05 2024-09-17T21:20:11.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:11.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026224+0000", 2024-09-17T21:20:11.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.042e-05 2024-09-17T21:20:11.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:11.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.075431+0000", 2024-09-17T21:20:11.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049206543999999998 2024-09-17T21:20:11.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:11.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.075489+0000", 2024-09-17T21:20:11.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8053000000000001e-05 2024-09-17T21:20:11.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:11.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.077623+0000", 2024-09-17T21:20:11.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002134127 2024-09-17T21:20:11.579 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.579 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:11.579 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.580 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1586 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:11.580 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.026238+0000", 2024-09-17T21:20:11.580 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.480604823, 2024-09-17T21:20:11.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051776152999999998, 2024-09-17T21:20:11.580 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:11.580 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:11.580 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:11.580 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:11.580 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:11.580 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:11.580 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1586 2024-09-17T21:20:11.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.581 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:11.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:11.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026238+0000", 2024-09-17T21:20:11.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:11.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026238+0000", 2024-09-17T21:20:11.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:11.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026239+0000", 2024-09-17T21:20:11.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0380000000000001e-06 2024-09-17T21:20:11.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:11.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026243+0000", 2024-09-17T21:20:11.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3499999999999999e-06 2024-09-17T21:20:11.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:11.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026259+0000", 2024-09-17T21:20:11.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5928999999999999e-05 2024-09-17T21:20:11.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:11.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026270+0000", 2024-09-17T21:20:11.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0702e-05 2024-09-17T21:20:11.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:11.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.077682+0000", 2024-09-17T21:20:11.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051411865000000001 2024-09-17T21:20:11.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:11.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.077751+0000", 2024-09-17T21:20:11.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9606999999999999e-05 2024-09-17T21:20:11.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:11.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078014+0000", 2024-09-17T21:20:11.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026266199999999999 2024-09-17T21:20:11.584 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.584 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:11.584 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.584 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1587 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 802816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:11.584 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.026286+0000", 2024-09-17T21:20:11.585 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.480557044000001, 2024-09-17T21:20:11.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.052384495000000003, 2024-09-17T21:20:11.585 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:11.585 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:11.585 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:11.585 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:11.585 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:11.585 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:11.585 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1587 2024-09-17T21:20:11.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.585 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:11.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:11.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026286+0000", 2024-09-17T21:20:11.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.586 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:11.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026286+0000", 2024-09-17T21:20:11.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.586 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:11.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026287+0000", 2024-09-17T21:20:11.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0049999999999999e-06 2024-09-17T21:20:11.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:11.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026291+0000", 2024-09-17T21:20:11.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3479999999999997e-06 2024-09-17T21:20:11.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:11.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026304+0000", 2024-09-17T21:20:11.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3288999999999999e-05 2024-09-17T21:20:11.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:11.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026314+0000", 2024-09-17T21:20:11.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9920000000000002e-06 2024-09-17T21:20:11.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:11.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078052+0000", 2024-09-17T21:20:11.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051737299 2024-09-17T21:20:11.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:11.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078118+0000", 2024-09-17T21:20:11.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6180999999999995e-05 2024-09-17T21:20:11.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:11.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078670+0000", 2024-09-17T21:20:11.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00055238100000000001 2024-09-17T21:20:11.589 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.589 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:11.589 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.589 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1618 3.e 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:1b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:11.589 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.031597+0000", 2024-09-17T21:20:11.589 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.475245603000001, 2024-09-17T21:20:11.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047545707999999999, 2024-09-17T21:20:11.589 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:11.589 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:11.589 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:11.589 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:11.589 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:11.590 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:11.590 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1618 2024-09-17T21:20:11.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.590 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:11.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:11.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031597+0000", 2024-09-17T21:20:11.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:11.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031597+0000", 2024-09-17T21:20:11.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:11.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031598+0000", 2024-09-17T21:20:11.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:20:11.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:11.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031602+0000", 2024-09-17T21:20:11.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:20:11.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:11.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031630+0000", 2024-09-17T21:20:11.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8058000000000001e-05 2024-09-17T21:20:11.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:11.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031643+0000", 2024-09-17T21:20:11.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2628e-05 2024-09-17T21:20:11.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:11.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078728+0000", 2024-09-17T21:20:11.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047085044 2024-09-17T21:20:11.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:11.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078830+0000", 2024-09-17T21:20:11.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102493 2024-09-17T21:20:11.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:11.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079143+0000", 2024-09-17T21:20:11.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031263599999999998 2024-09-17T21:20:11.593 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.593 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:11.593 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.593 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1619 3.e 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:1b [read 827392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:11.593 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.031661+0000", 2024-09-17T21:20:11.594 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.475182036, 2024-09-17T21:20:11.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047908566999999999, 2024-09-17T21:20:11.594 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:11.594 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:11.594 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:11.594 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:11.594 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:11.594 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:11.594 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1619 2024-09-17T21:20:11.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.594 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:11.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:11.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031661+0000", 2024-09-17T21:20:11.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:11.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031661+0000", 2024-09-17T21:20:11.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:11.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031662+0000", 2024-09-17T21:20:11.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.252e-06 2024-09-17T21:20:11.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:11.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031666+0000", 2024-09-17T21:20:11.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6299999999999997e-06 2024-09-17T21:20:11.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:11.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031683+0000", 2024-09-17T21:20:11.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6035000000000001e-05 2024-09-17T21:20:11.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:11.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031693+0000", 2024-09-17T21:20:11.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0492000000000001e-05 2024-09-17T21:20:11.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:11.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079186+0000", 2024-09-17T21:20:11.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047493441999999997 2024-09-17T21:20:11.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:11.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079315+0000", 2024-09-17T21:20:11.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012860000000000001 2024-09-17T21:20:11.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:11.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079569+0000", 2024-09-17T21:20:11.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025411599999999998 2024-09-17T21:20:11.598 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.598 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:11.598 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.598 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1620 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:11.598 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.031706+0000", 2024-09-17T21:20:11.598 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.475136612, 2024-09-17T21:20:11.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048561143000000001, 2024-09-17T21:20:11.598 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:11.598 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:11.598 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:11.599 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:11.599 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:11.599 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:11.599 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1620 2024-09-17T21:20:11.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.599 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:11.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:11.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031706+0000", 2024-09-17T21:20:11.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:11.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031706+0000", 2024-09-17T21:20:11.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:11.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031707+0000", 2024-09-17T21:20:11.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5799999999999998e-07 2024-09-17T21:20:11.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:11.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031711+0000", 2024-09-17T21:20:11.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1930000000000003e-06 2024-09-17T21:20:11.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:11.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031724+0000", 2024-09-17T21:20:11.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.24e-05 2024-09-17T21:20:11.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:11.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031733+0000", 2024-09-17T21:20:11.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6239999999999997e-06 2024-09-17T21:20:11.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:11.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079613+0000", 2024-09-17T21:20:11.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047880082999999997 2024-09-17T21:20:11.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:11.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079719+0000", 2024-09-17T21:20:11.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010559499999999999 2024-09-17T21:20:11.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:11.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.080267+0000", 2024-09-17T21:20:11.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00054829 2024-09-17T21:20:11.602 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.602 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:11.602 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.603 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1621 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:11.603 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.031746+0000", 2024-09-17T21:20:11.603 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.475096138000001, 2024-09-17T21:20:11.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048925189000000001, 2024-09-17T21:20:11.603 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:11.603 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:11.603 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:11.603 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:11.603 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:11.603 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:11.603 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1621 2024-09-17T21:20:11.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.604 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:11.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:11.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031746+0000", 2024-09-17T21:20:11.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:11.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031746+0000", 2024-09-17T21:20:11.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:11.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:11.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031747+0000", 2024-09-17T21:20:11.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3900000000000003e-07 2024-09-17T21:20:11.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:11.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031752+0000", 2024-09-17T21:20:11.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.172e-06 2024-09-17T21:20:11.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:11.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031767+0000", 2024-09-17T21:20:11.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5517000000000001e-05 2024-09-17T21:20:11.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:11.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031777+0000", 2024-09-17T21:20:11.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8579999999999995e-06 2024-09-17T21:20:11.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:11.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.080318+0000", 2024-09-17T21:20:11.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048541009000000003 2024-09-17T21:20:11.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:11.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.080415+0000", 2024-09-17T21:20:11.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7034999999999995e-05 2024-09-17T21:20:11.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:11.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:11.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:11.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.080672+0000", 2024-09-17T21:20:11.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025665900000000001 2024-09-17T21:20:11.607 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.607 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:11.607 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.607 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:11.607 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:11.607 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:11.608 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:11.609 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:20:11.610 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:11.603+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:11.610 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:11.709 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:11.703+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:11.782 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:11.782 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:11.782 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:20:11.782 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:11.796 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:20:11.839 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:11.835+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:11.911 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:11.905+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:11.964 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:11.964 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:11.964 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:11.964 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:11.964 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:11.978 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:20:12.012 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:12.005+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:12.105 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:20:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 110.497186528, 2024-09-17T21:20:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:20:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:20:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:20:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:20:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:20:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:20:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:20:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:20:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:20:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:20:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:20:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:20:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:20:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:20:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:20:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:20:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:20:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:20:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 110.48723989200001, 2024-09-17T21:20:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:20:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:20:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:20:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:20:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:20:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:20:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:20:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:20:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:20:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:20:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:20:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:20:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:20:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:20:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:20:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:20:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:20:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:20:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 109.24408905200001, 2024-09-17T21:20:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:20:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:20:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:20:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:20:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:20:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:20:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:20:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:20:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:20:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:20:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:20:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:20:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:20:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:20:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:20:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:20:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:20:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:20:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:20:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:20:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:20:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:20:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:20:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:20:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:20:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:20:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:20:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 109.234443566, 2024-09-17T21:20:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:20:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:20:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:20:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:20:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:20:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:20:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:20:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:20:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:20:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:20:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:20:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:20:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:20:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:20:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:20:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:20:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:20:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:20:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:20:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:20:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:20:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:20:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:20:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:20:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:20:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:20:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:20:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 95.865507532999999, 2024-09-17T21:20:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:20:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:20:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:20:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:20:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:20:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:20:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:20:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:20:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:20:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:20:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:20:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:20:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:20:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:20:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:20:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:20:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:20:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:20:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:819 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5111808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.663935242999997, 2024-09-17T21:20:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024344811000000001, 2024-09-17T21:20:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 819 2024-09-17T21:20:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437443+0000", 2024-09-17T21:20:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1969999999999999e-06 2024-09-17T21:20:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437448+0000", 2024-09-17T21:20:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9490000000000004e-06 2024-09-17T21:20:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437467+0000", 2024-09-17T21:20:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8862999999999999e-05 2024-09-17T21:20:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437480+0000", 2024-09-17T21:20:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2833000000000001e-05 2024-09-17T21:20:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461552+0000", 2024-09-17T21:20:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024072695000000002 2024-09-17T21:20:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461639+0000", 2024-09-17T21:20:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6996000000000005e-05 2024-09-17T21:20:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461787+0000", 2024-09-17T21:20:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014727800000000001 2024-09-17T21:20:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.663822971000002, 2024-09-17T21:20:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:20:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:20:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:20:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:20:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:20:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:20:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:20:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:20:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:20:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:20:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:12.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:20:12.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:20:12.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:12.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:20:12.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:20:12.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:20:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:20:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:833 3.5 3:a772dfb1:::rbd_data.11523651b015.0000000000000001:d [sparse-read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.663388685000001, 2024-09-17T21:20:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024216024999999999, 2024-09-17T21:20:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:12.140 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 833 2024-09-17T21:20:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437990+0000", 2024-09-17T21:20:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2500000000000001e-06 2024-09-17T21:20:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437995+0000", 2024-09-17T21:20:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1240000000000004e-06 2024-09-17T21:20:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438012+0000", 2024-09-17T21:20:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7572e-05 2024-09-17T21:20:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438031+0000", 2024-09-17T21:20:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8612000000000001e-05 2024-09-17T21:20:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461830+0000", 2024-09-17T21:20:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023799135999999999 2024-09-17T21:20:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462005+0000", 2024-09-17T21:20:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000174747 2024-09-17T21:20:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:20:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019958399999999999 2024-09-17T21:20:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.144 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.663328394000004, 2024-09-17T21:20:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:20:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:20:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:12.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:20:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:20:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:12.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:20:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:20:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:20:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:20:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:20:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:20:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:12.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:20:12.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:20:12.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:12.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:20:12.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:20:12.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:12.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:20:12.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:20:12.148 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.148 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:12.148 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.149 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:12.149 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:12.149 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.663273012999994, 2024-09-17T21:20:12.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:20:12.149 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:12.149 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:12.149 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:12.149 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:12.149 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:12.149 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:12.149 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:20:12.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.149 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:12.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:12.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:12.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:12.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:12.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:12.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:20:12.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:20:12.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:12.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:20:12.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:20:12.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:12.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:20:12.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:20:12.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:20:12.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:20:12.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:12.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:20:12.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:20:12.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:12.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:20:12.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:20:12.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:12.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:20:12.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:20:12.153 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.153 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:12.153 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.153 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:12.153 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:12.153 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.663216735000006, 2024-09-17T21:20:12.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:20:12.153 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:12.153 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:12.153 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:12.153 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:12.153 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:12.154 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:12.154 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:20:12.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.154 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:12.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:12.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:12.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:12.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:12.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:12.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:20:12.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:20:12.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:12.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:20:12.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:20:12.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:12.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:20:12.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:20:12.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:20:12.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:20:12.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:12.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:20:12.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:20:12.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:12.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:20:12.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:20:12.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:12.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:20:12.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:20:12.157 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.157 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:12.157 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.157 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:12.157 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:12.157 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.662267095999994, 2024-09-17T21:20:12.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:20:12.157 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:12.157 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:12.157 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:12.158 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:12.158 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:12.158 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:12.158 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:20:12.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.158 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:12.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:12.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:12.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:12.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:12.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:12.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:20:12.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:20:12.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:12.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:20:12.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:20:12.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:12.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:20:12.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:20:12.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:20:12.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:20:12.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:12.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:20:12.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:20:12.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:12.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:20:12.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:20:12.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:12.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:20:12.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:20:12.161 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.161 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:12.161 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.161 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:12.161 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:12.161 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.662188475999997, 2024-09-17T21:20:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:20:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:20:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:20:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:20:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:20:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:20:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:20:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:20:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:20:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:20:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:20:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:20:12.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:20:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:20:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:20:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:20:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.662075537000007, 2024-09-17T21:20:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:20:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:20:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.166 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:20:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:20:12.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:20:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:20:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:20:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:20:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:20:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:20:12.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:20:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:20:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:20:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:20:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:20:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:20:12.169 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:849 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5963776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.661481500999997, 2024-09-17T21:20:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024537514, 2024-09-17T21:20:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 849 2024-09-17T21:20:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439897+0000", 2024-09-17T21:20:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9099999999999999e-06 2024-09-17T21:20:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439905+0000", 2024-09-17T21:20:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0870000000000007e-06 2024-09-17T21:20:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439924+0000", 2024-09-17T21:20:12.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9042000000000002e-05 2024-09-17T21:20:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439941+0000", 2024-09-17T21:20:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.624e-05 2024-09-17T21:20:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464132+0000", 2024-09-17T21:20:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024191579000000001 2024-09-17T21:20:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464291+0000", 2024-09-17T21:20:12.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015904800000000001 2024-09-17T21:20:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464433+0000", 2024-09-17T21:20:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014160799999999999 2024-09-17T21:20:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.661414483000001, 2024-09-17T21:20:12.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:20:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:20:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:20:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:20:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:20:12.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:20:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:20:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:20:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:20:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:20:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:20:12.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:20:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:20:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:20:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:20:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:20:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.178 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.661349672, 2024-09-17T21:20:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:20:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:20:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:20:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:20:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:20:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:20:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:20:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:20:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:20:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:20:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:20:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:20:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:20:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:20:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:20:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:20:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.661288694999996, 2024-09-17T21:20:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:20:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:20:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:12.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:20:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:20:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:20:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:20:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:20:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:20:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:20:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:20:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:20:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:20:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:20:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:20:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:20:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:20:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:20:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.763717509999999, 2024-09-17T21:20:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:20:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:20:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:20:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:20:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:20:12.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:20:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:20:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:20:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:20:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:20:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:20:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:20:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:20:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:20:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:20:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:20:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:20:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:20:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.555059115999999, 2024-09-17T21:20:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:20:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:20:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:20:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:20:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:20:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:20:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:20:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:20:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:20:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:20:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:20:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:20:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:20:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:20:12.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:20:12.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:12.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:12.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:12.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:20:12.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:20:12.195 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.195 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:12.195 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.195 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:12.195 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:12.195 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:12.197 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:20:12.197 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:12.191+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:12.281 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-17T21:20:12.299 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:12.291+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:12.335 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:12.335 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:12.335 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:12.335 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:12.349 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:20:12.437 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:12.433+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:12.499 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:12.493+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:12.516 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:12.516 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:12.516 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:12.516 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:12.517 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:12.527 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:20:12.600 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:12.595+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:12.689 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:12.689 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:494 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:20:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 106.48811958500001, 2024-09-17T21:20:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046244995999999997, 2024-09-17T21:20:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:12.690 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 494 2024-09-17T21:20:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:20:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197328+0000", 2024-09-17T21:20:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197329+0000", 2024-09-17T21:20:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.173e-06 2024-09-17T21:20:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197334+0000", 2024-09-17T21:20:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0499999999999999e-06 2024-09-17T21:20:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:12.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197352+0000", 2024-09-17T21:20:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7133e-05 2024-09-17T21:20:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.197367+0000", 2024-09-17T21:20:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5123000000000001e-05 2024-09-17T21:20:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242357+0000", 2024-09-17T21:20:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044989829000000002 2024-09-17T21:20:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.242526+0000", 2024-09-17T21:20:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016968199999999999 2024-09-17T21:20:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.243573+0000", 2024-09-17T21:20:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010470060000000001 2024-09-17T21:20:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:524 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:20:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 106.485373557, 2024-09-17T21:20:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045737414999999997, 2024-09-17T21:20:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 524 2024-09-17T21:20:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:20:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200074+0000", 2024-09-17T21:20:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200076+0000", 2024-09-17T21:20:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3799999999999999e-06 2024-09-17T21:20:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200081+0000", 2024-09-17T21:20:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7030000000000003e-06 2024-09-17T21:20:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200099+0000", 2024-09-17T21:20:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7836999999999999e-05 2024-09-17T21:20:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200113+0000", 2024-09-17T21:20:12.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3570000000000001e-05 2024-09-17T21:20:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244866+0000", 2024-09-17T21:20:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044753573999999997 2024-09-17T21:20:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.244972+0000", 2024-09-17T21:20:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106111 2024-09-17T21:20:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245812+0000", 2024-09-17T21:20:12.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083923999999999999 2024-09-17T21:20:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:525 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3670016~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 106.485222175, 2024-09-17T21:20:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046645697, 2024-09-17T21:20:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:12.699 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 525 2024-09-17T21:20:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:20:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:20:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200234+0000", 2024-09-17T21:20:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:20:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200254+0000", 2024-09-17T21:20:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9959999999999999e-05 2024-09-17T21:20:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200293+0000", 2024-09-17T21:20:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9755999999999997e-05 2024-09-17T21:20:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245871+0000", 2024-09-17T21:20:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045577871999999998 2024-09-17T21:20:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246030+0000", 2024-09-17T21:20:12.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158975 2024-09-17T21:20:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246871+0000", 2024-09-17T21:20:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084111699999999995 2024-09-17T21:20:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 106.4849765, 2024-09-17T21:20:12.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:20:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:20:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:12.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:20:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:20:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:20:12.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:20:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:20:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:20:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:20:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:20:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:20:12.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:20:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:20:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:20:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:20:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:20:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 106.484797818, 2024-09-17T21:20:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:20:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:20:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.708 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:20:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:20:12.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:20:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:20:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:20:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:20:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:20:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:20:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:20:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:20:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:20:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:20:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:20:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:20:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 106.484738545, 2024-09-17T21:20:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:20:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:12.712 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:20:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:20:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:20:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:20:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:20:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:20:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:20:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:20:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:20:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:20:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:20:12.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:20:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:20:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:20:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:20:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 106.48459635, 2024-09-17T21:20:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:20:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:20:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:12.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:12.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:12.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:20:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:20:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:20:12.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:20:12.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:12.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:20:12.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:20:12.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:20:12.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:20:12.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:12.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:20:12.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:20:12.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:12.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:20:12.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:20:12.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:12.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:20:12.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:20:12.722 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.722 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:12.723 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.723 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:12.723 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:12.723 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 106.484540427, 2024-09-17T21:20:12.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:20:12.723 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:12.723 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:12.723 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:12.723 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:12.723 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:12.723 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:12.723 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:20:12.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.724 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:12.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:12.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:12.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:12.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:12.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:20:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:20:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:20:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:20:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:20:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:20:12.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:20:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:20:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:20:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:20:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:20:12.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:20:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:20:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:20:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:12.727 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 106.48448681000001, 2024-09-17T21:20:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:20:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:20:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:20:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:20:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:20:12.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:20:12.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:12.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:20:12.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:20:12.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:20:12.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:20:12.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:12.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:20:12.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:20:12.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:12.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:20:12.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:20:12.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:12.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:20:12.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:20:12.731 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.731 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:12.731 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.732 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:20:12.732 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:12.732 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 56.175258567999997, 2024-09-17T21:20:12.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:20:12.732 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:12.732 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:12.732 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:20:12.732 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:12.732 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:20:12.732 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:20:12.732 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:20:12.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.732 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:12.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:12.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:12.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:12.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:12.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:12.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:20:12.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:20:12.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:12.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:20:12.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:20:12.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:12.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:20:12.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:20:12.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:20:12.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:20:12.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:12.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:20:12.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:20:12.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:12.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:20:12.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:20:12.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:12.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:20:12.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:20:12.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:12.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:20:12.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:20:12.736 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.736 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:12.736 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.737 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2597 3.1f 3:fb0e9615:::rbd_data.1152a8ae2019.000000000000000c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:12.737 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:12.737 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.84682214, 2024-09-17T21:20:12.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097064994000000002, 2024-09-17T21:20:12.737 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:12.737 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:12.737 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:12.737 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:12.737 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:12.737 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:12.737 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2597 2024-09-17T21:20:12.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.738 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:12.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:12.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:12.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:12.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:12.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:12.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838628+0000", 2024-09-17T21:20:12.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.187e-06 2024-09-17T21:20:12.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:12.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838654+0000", 2024-09-17T21:20:12.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:20:12.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:12.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838696+0000", 2024-09-17T21:20:12.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1226000000000001e-05 2024-09-17T21:20:12.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838715+0000", 2024-09-17T21:20:12.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9646000000000001e-05 2024-09-17T21:20:12.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:12.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838937+0000", 2024-09-17T21:20:12.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022163 2024-09-17T21:20:12.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:12.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.839284+0000", 2024-09-17T21:20:12.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034753799999999999 2024-09-17T21:20:12.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:12.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935691+0000", 2024-09-17T21:20:12.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.096406360999999996 2024-09-17T21:20:12.741 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.741 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:12.741 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.741 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2641 3.12 3:488a729f:::rbd_data.1152a8ae2019.0000000000000022:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:12.741 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:12.741 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.843365367000001, 2024-09-17T21:20:12.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098280953000000004, 2024-09-17T21:20:12.742 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:12.742 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:12.742 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:12.742 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:12.742 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:12.742 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:12.742 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2641 2024-09-17T21:20:12.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.742 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:12.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:12.742 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:12.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:12.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:12.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:12.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842083+0000", 2024-09-17T21:20:12.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5300000000000002e-07 2024-09-17T21:20:12.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:12.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842088+0000", 2024-09-17T21:20:12.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:20:12.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:12.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842105+0000", 2024-09-17T21:20:12.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6773e-05 2024-09-17T21:20:12.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842119+0000", 2024-09-17T21:20:12.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:20:12.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:12.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842320+0000", 2024-09-17T21:20:12.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020052700000000001 2024-09-17T21:20:12.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:12.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842845+0000", 2024-09-17T21:20:12.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052508600000000002 2024-09-17T21:20:12.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:12.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940363+0000", 2024-09-17T21:20:12.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097518795000000005 2024-09-17T21:20:12.745 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.746 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:12.746 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.746 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2674 3.12 3:4c87e3ac:::rbd_data.1152a8ae2019.0000000000000033:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:12.746 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:12.746 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.843002729999998, 2024-09-17T21:20:12.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098025125000000005, 2024-09-17T21:20:12.746 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:12.746 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:12.746 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:12.746 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:12.746 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:12.747 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:12.747 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2674 2024-09-17T21:20:12.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.747 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:12.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.747 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:12.747 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:12.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.747 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:12.747 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:12.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:12.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842447+0000", 2024-09-17T21:20:12.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3999999999999999e-06 2024-09-17T21:20:12.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:12.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842453+0000", 2024-09-17T21:20:12.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:20:12.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:12.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842472+0000", 2024-09-17T21:20:12.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9006000000000001e-05 2024-09-17T21:20:12.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842487+0000", 2024-09-17T21:20:12.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4866e-05 2024-09-17T21:20:12.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:12.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843594+0000", 2024-09-17T21:20:12.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011073249999999999 2024-09-17T21:20:12.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:12.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843668+0000", 2024-09-17T21:20:12.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3188000000000005e-05 2024-09-17T21:20:12.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:12.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843893+0000", 2024-09-17T21:20:12.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022503399999999999 2024-09-17T21:20:12.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:12.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850439+0000", 2024-09-17T21:20:12.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065467880000000004 2024-09-17T21:20:12.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:12.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940409+0000", 2024-09-17T21:20:12.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089969511000000002 2024-09-17T21:20:12.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:12.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940444+0000", 2024-09-17T21:20:12.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5345999999999997e-05 2024-09-17T21:20:12.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:12.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940470+0000", 2024-09-17T21:20:12.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5964000000000001e-05 2024-09-17T21:20:12.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.752 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:12.752 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.752 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2707 3.1f 3:f805e65e:::rbd_data.1152a8ae2019.0000000000000049:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:12.752 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:12.752 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.837956069000001, 2024-09-17T21:20:12.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088327300999999997, 2024-09-17T21:20:12.752 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:12.752 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:12.752 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:12.752 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:12.752 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:12.752 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:12.753 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2707 2024-09-17T21:20:12.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.753 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:12.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:12.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:12.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:12.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:12.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:12.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847494+0000", 2024-09-17T21:20:12.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:20:12.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:12.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847504+0000", 2024-09-17T21:20:12.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:20:12.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:12.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847526+0000", 2024-09-17T21:20:12.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1976000000000001e-05 2024-09-17T21:20:12.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847541+0000", 2024-09-17T21:20:12.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5277e-05 2024-09-17T21:20:12.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:12.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856142+0000", 2024-09-17T21:20:12.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086006710000000007 2024-09-17T21:20:12.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:12.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856435+0000", 2024-09-17T21:20:12.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029301499999999999 2024-09-17T21:20:12.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:12.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935819+0000", 2024-09-17T21:20:12.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079384448999999996 2024-09-17T21:20:12.756 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.756 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:12.756 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.756 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2730 3.12 3:48b3422a:::rbd_data.1152a8ae2019.0000000000000057:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:12.756 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:12.756 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.828763169999998, 2024-09-17T21:20:12.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083879975999999995, 2024-09-17T21:20:12.757 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:12.757 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:12.757 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:12.757 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:12.757 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:12.757 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:12.757 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2730 2024-09-17T21:20:12.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.757 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:12.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:12.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:12.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:12.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:12.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:12.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856687+0000", 2024-09-17T21:20:12.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:20:12.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:12.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856697+0000", 2024-09-17T21:20:12.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0207000000000001e-05 2024-09-17T21:20:12.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:12.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856808+0000", 2024-09-17T21:20:12.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011071200000000001 2024-09-17T21:20:12.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856828+0000", 2024-09-17T21:20:12.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0219000000000001e-05 2024-09-17T21:20:12.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:12.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856907+0000", 2024-09-17T21:20:12.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9661999999999997e-05 2024-09-17T21:20:12.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:12.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.857019+0000", 2024-09-17T21:20:12.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011148499999999999 2024-09-17T21:20:12.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:12.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940565+0000", 2024-09-17T21:20:12.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083545791999999994 2024-09-17T21:20:12.760 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.760 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:12.760 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.761 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2743 3.12 3:4d036e62:::rbd_data.1152a8ae2019.000000000000005f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:12.761 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:12.761 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.824817669000002, 2024-09-17T21:20:12.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080059962999999998, 2024-09-17T21:20:12.761 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:12.761 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:12.761 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:12.761 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:12.761 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:12.761 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:12.761 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2743 2024-09-17T21:20:12.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.762 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:12.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:12.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:12.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:12.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:12.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:12.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860631+0000", 2024-09-17T21:20:12.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:20:12.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:12.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860636+0000", 2024-09-17T21:20:12.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5279999999999997e-06 2024-09-17T21:20:12.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:12.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860649+0000", 2024-09-17T21:20:12.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-17T21:20:12.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860655+0000", 2024-09-17T21:20:12.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:20:12.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:12.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860731+0000", 2024-09-17T21:20:12.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5920000000000005e-05 2024-09-17T21:20:12.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:12.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.861070+0000", 2024-09-17T21:20:12.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338535 2024-09-17T21:20:12.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:12.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940690+0000", 2024-09-17T21:20:12.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079620503999999995 2024-09-17T21:20:12.765 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.765 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:12.765 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.765 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3378 3.12 3:4832a51b:::rbd_data.1152a8ae2019.000000000000006c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:12.765 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:12.765 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.796453819, 2024-09-17T21:20:12.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051812516000000003, 2024-09-17T21:20:12.765 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:12.765 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:12.765 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:12.765 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:12.766 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:12.766 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:12.766 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3378 2024-09-17T21:20:12.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.766 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:12.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:12.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:12.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:12.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:12.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:12.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888995+0000", 2024-09-17T21:20:12.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:20:12.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:12.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889006+0000", 2024-09-17T21:20:12.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1012999999999999e-05 2024-09-17T21:20:12.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:12.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889032+0000", 2024-09-17T21:20:12.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6441e-05 2024-09-17T21:20:12.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889040+0000", 2024-09-17T21:20:12.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0639999999999994e-06 2024-09-17T21:20:12.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:12.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889278+0000", 2024-09-17T21:20:12.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023785899999999999 2024-09-17T21:20:12.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:12.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889408+0000", 2024-09-17T21:20:12.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129219 2024-09-17T21:20:12.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:12.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889723+0000", 2024-09-17T21:20:12.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000315264 2024-09-17T21:20:12.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:12.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.891972+0000", 2024-09-17T21:20:12.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002249633 2024-09-17T21:20:12.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:12.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940733+0000", 2024-09-17T21:20:12.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048760626000000001 2024-09-17T21:20:12.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:12.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940785+0000", 2024-09-17T21:20:12.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1906000000000001e-05 2024-09-17T21:20:12.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:12.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940807+0000", 2024-09-17T21:20:12.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1554999999999999e-05 2024-09-17T21:20:12.770 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.770 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:12.770 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.771 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3379 3.12 3:4e83fd71:::rbd_data.1152a8ae2019.000000000000006d:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:12.771 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:12.771 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.796369611999999, 2024-09-17T21:20:12.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051804144000000003, 2024-09-17T21:20:12.771 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:12.771 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:12.771 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:12.771 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:12.771 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:12.771 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:12.771 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3379 2024-09-17T21:20:12.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.772 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:12.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:12.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:12.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:12.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:12.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:12.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889079+0000", 2024-09-17T21:20:12.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1800000000000004e-07 2024-09-17T21:20:12.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:12.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889083+0000", 2024-09-17T21:20:12.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:20:12.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:12.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889094+0000", 2024-09-17T21:20:12.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1562999999999999e-05 2024-09-17T21:20:12.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889102+0000", 2024-09-17T21:20:12.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.401e-06 2024-09-17T21:20:12.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:12.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:20:12.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014279259999999999 2024-09-17T21:20:12.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:12.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890609+0000", 2024-09-17T21:20:12.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9566000000000004e-05 2024-09-17T21:20:12.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:12.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890789+0000", 2024-09-17T21:20:12.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017949700000000001 2024-09-17T21:20:12.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:12.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.893929+0000", 2024-09-17T21:20:12.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003140538 2024-09-17T21:20:12.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:12.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940834+0000", 2024-09-17T21:20:12.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046904811999999997 2024-09-17T21:20:12.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:12.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940860+0000", 2024-09-17T21:20:12.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5341999999999999e-05 2024-09-17T21:20:12.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:12.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940882+0000", 2024-09-17T21:20:12.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2878999999999999e-05 2024-09-17T21:20:12.776 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.776 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:12.776 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.776 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:1613 3.1e 3:7d13ef1b:::rbd_data.115266d0b9f2.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 5666816~34816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-17T21:20:12.777 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:48.315514+0000", 2024-09-17T21:20:12.777 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 24.369933833000001, 2024-09-17T21:20:12.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045611719000000002, 2024-09-17T21:20:12.777 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:12.777 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:12.777 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:12.777 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:12.777 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:20:12.777 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:20:12.777 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 1613 2024-09-17T21:20:12.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.777 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:12.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:12.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.315514+0000", 2024-09-17T21:20:12.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:12.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.315514+0000", 2024-09-17T21:20:12.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:12.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.315516+0000", 2024-09-17T21:20:12.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.497e-06 2024-09-17T21:20:12.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:12.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.330451+0000", 2024-09-17T21:20:12.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014935857 2024-09-17T21:20:12.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:12.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.330479+0000", 2024-09-17T21:20:12.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7151000000000001e-05 2024-09-17T21:20:12.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.330490+0000", 2024-09-17T21:20:12.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1250000000000001e-05 2024-09-17T21:20:12.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:12.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.330560+0000", 2024-09-17T21:20:12.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9901999999999996e-05 2024-09-17T21:20:12.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:12.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.330688+0000", 2024-09-17T21:20:12.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000128608 2024-09-17T21:20:12.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:20:12.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.331269+0000", 2024-09-17T21:20:12.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00058052799999999997 2024-09-17T21:20:12.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:12.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.350343+0000", 2024-09-17T21:20:12.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019074457 2024-09-17T21:20:12.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:12.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.361030+0000", 2024-09-17T21:20:12.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010686357 2024-09-17T21:20:12.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:12.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.361084+0000", 2024-09-17T21:20:12.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4097000000000001e-05 2024-09-17T21:20:12.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:12.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:48.361126+0000", 2024-09-17T21:20:12.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2014999999999997e-05 2024-09-17T21:20:12.782 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.782 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:12.782 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.782 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:2644 3.1e 3:79615585:::rbd_data.115266d0b9f2.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2475520~14848,write 3106304~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:20:12.782 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:12.782 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 23.173741208999999, 2024-09-17T21:20:12.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050733484000000002, 2024-09-17T21:20:12.783 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:12.783 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:12.783 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:12.783 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:12.783 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:20:12.783 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:20:12.783 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2644 2024-09-17T21:20:12.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.783 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:12.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:12.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:12.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:12.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:12.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:12.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:12.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511708+0000", 2024-09-17T21:20:12.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-17T21:20:12.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:12.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522196+0000", 2024-09-17T21:20:12.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010487712999999999 2024-09-17T21:20:12.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:12.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522223+0000", 2024-09-17T21:20:12.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7064999999999999e-05 2024-09-17T21:20:12.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:12.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522236+0000", 2024-09-17T21:20:12.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2931e-05 2024-09-17T21:20:12.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:12.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522348+0000", 2024-09-17T21:20:12.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111906 2024-09-17T21:20:12.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:12.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522554+0000", 2024-09-17T21:20:12.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020632699999999999 2024-09-17T21:20:12.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:20:12.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.523185+0000", 2024-09-17T21:20:12.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063081400000000005 2024-09-17T21:20:12.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:12.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.547011+0000", 2024-09-17T21:20:12.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023825570000000001 2024-09-17T21:20:12.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:12.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562268+0000", 2024-09-17T21:20:12.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015257514 2024-09-17T21:20:12.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:12.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562367+0000", 2024-09-17T21:20:12.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8678000000000006e-05 2024-09-17T21:20:12.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:12.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:12.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:12.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562440+0000", 2024-09-17T21:20:12.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3337000000000006e-05 2024-09-17T21:20:12.788 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.788 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:12.788 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.788 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:12.788 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:12.788 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:12.790 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:20:12.791 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:12.783+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:12.891 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:12.885+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:12.918 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:12.919 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:12.919 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:12.919 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:12.932 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:20:13.021 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:13.017+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:13.093 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:13.087+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:13.098 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:13.098 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:13.098 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:13.098 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:13.098 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:13.112 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:20:13.194 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:13.187+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:13.278 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:543 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:20:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 107.074494755, 2024-09-17T21:20:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047564097999999999, 2024-09-17T21:20:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 543 2024-09-17T21:20:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:20:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:20:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:20:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9699999999999995e-07 2024-09-17T21:20:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199677+0000", 2024-09-17T21:20:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:20:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199690+0000", 2024-09-17T21:20:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2977e-05 2024-09-17T21:20:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199705+0000", 2024-09-17T21:20:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4656e-05 2024-09-17T21:20:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246307+0000", 2024-09-17T21:20:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046601780000000002 2024-09-17T21:20:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246475+0000", 2024-09-17T21:20:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016834200000000001 2024-09-17T21:20:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247227+0000", 2024-09-17T21:20:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00075173299999999998 2024-09-17T21:20:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:544 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:20:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 107.074436346, 2024-09-17T21:20:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049283559999999997, 2024-09-17T21:20:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 544 2024-09-17T21:20:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:20:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:20:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199722+0000", 2024-09-17T21:20:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.08e-06 2024-09-17T21:20:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199738+0000", 2024-09-17T21:20:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5943999999999998e-05 2024-09-17T21:20:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199748+0000", 2024-09-17T21:20:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0193e-05 2024-09-17T21:20:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199756+0000", 2024-09-17T21:20:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4200000000000001e-06 2024-09-17T21:20:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247311+0000", 2024-09-17T21:20:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047555300000000002 2024-09-17T21:20:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247571+0000", 2024-09-17T21:20:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025996999999999999 2024-09-17T21:20:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249005+0000", 2024-09-17T21:20:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001433653 2024-09-17T21:20:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:549 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:20:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 107.074390722, 2024-09-17T21:20:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048464528999999999, 2024-09-17T21:20:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 549 2024-09-17T21:20:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:20:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:20:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:20:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:20:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199770+0000", 2024-09-17T21:20:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9969999999999999e-06 2024-09-17T21:20:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199780+0000", 2024-09-17T21:20:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0003000000000001e-05 2024-09-17T21:20:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199793+0000", 2024-09-17T21:20:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2074e-05 2024-09-17T21:20:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247289+0000", 2024-09-17T21:20:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047496691000000001 2024-09-17T21:20:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247437+0000", 2024-09-17T21:20:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014807899999999999 2024-09-17T21:20:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248231+0000", 2024-09-17T21:20:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00079390899999999998 2024-09-17T21:20:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:558 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:20:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 107.074350209, 2024-09-17T21:20:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049709821000000001, 2024-09-17T21:20:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 558 2024-09-17T21:20:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:20:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:20:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199809+0000", 2024-09-17T21:20:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.333e-06 2024-09-17T21:20:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199814+0000", 2024-09-17T21:20:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0930000000000002e-06 2024-09-17T21:20:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199831+0000", 2024-09-17T21:20:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7580000000000001e-05 2024-09-17T21:20:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199860+0000", 2024-09-17T21:20:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8609999999999999e-05 2024-09-17T21:20:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249106+0000", 2024-09-17T21:20:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049246545000000003 2024-09-17T21:20:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249274+0000", 2024-09-17T21:20:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016806699999999999 2024-09-17T21:20:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249517+0000", 2024-09-17T21:20:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024259300000000001 2024-09-17T21:20:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:559 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:20:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 107.074230847, 2024-09-17T21:20:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049990694000000002, 2024-09-17T21:20:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 559 2024-09-17T21:20:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:20:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:20:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199928+0000", 2024-09-17T21:20:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:20:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199935+0000", 2024-09-17T21:20:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5429999999999999e-06 2024-09-17T21:20:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199954+0000", 2024-09-17T21:20:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.912e-05 2024-09-17T21:20:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199969+0000", 2024-09-17T21:20:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5305999999999999e-05 2024-09-17T21:20:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249604+0000", 2024-09-17T21:20:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049634931 2024-09-17T21:20:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249738+0000", 2024-09-17T21:20:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013411599999999999 2024-09-17T21:20:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249917+0000", 2024-09-17T21:20:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179111 2024-09-17T21:20:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:560 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:20:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 107.07416643099999, 2024-09-17T21:20:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050173059999999998, 2024-09-17T21:20:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 560 2024-09-17T21:20:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:20:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:20:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199992+0000", 2024-09-17T21:20:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.517e-06 2024-09-17T21:20:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199998+0000", 2024-09-17T21:20:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1460000000000004e-06 2024-09-17T21:20:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:20:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7407000000000001e-05 2024-09-17T21:20:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200029+0000", 2024-09-17T21:20:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3499999999999999e-05 2024-09-17T21:20:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249968+0000", 2024-09-17T21:20:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049939799 2024-09-17T21:20:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250055+0000", 2024-09-17T21:20:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6731999999999996e-05 2024-09-17T21:20:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250164+0000", 2024-09-17T21:20:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108959 2024-09-17T21:20:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:20:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 93.464376721999997, 2024-09-17T21:20:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:20:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:20:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:20:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:20:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:20:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:20:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:20:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:20:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:20:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:20:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:20:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:20:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 79.909274936000003, 2024-09-17T21:20:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:20:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:20:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:20:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:20:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:20:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:20:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:20:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:20:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:20:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:20:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:20:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:20:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:20:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:20:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:20:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:20:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:20:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:20:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:20:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:20:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:20:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:20:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:20:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:20:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:20:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:20:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:20:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:20:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:20:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3777 3.c 3:302136a0:::rbd_data.1152a8ae2019.000000000000004b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.902125+0000", 2024-09-17T21:20:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 32.372032310000002, 2024-09-17T21:20:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048610548000000003, 2024-09-17T21:20:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3777 2024-09-17T21:20:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902125+0000", 2024-09-17T21:20:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902125+0000", 2024-09-17T21:20:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902126+0000", 2024-09-17T21:20:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:20:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902138+0000", 2024-09-17T21:20:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1994000000000001e-05 2024-09-17T21:20:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902144+0000", 2024-09-17T21:20:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2820000000000002e-06 2024-09-17T21:20:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902149+0000", 2024-09-17T21:20:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.135e-06 2024-09-17T21:20:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.942910+0000", 2024-09-17T21:20:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.040761190000000003 2024-09-17T21:20:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.942986+0000", 2024-09-17T21:20:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6083999999999999e-05 2024-09-17T21:20:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.943319+0000", 2024-09-17T21:20:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00033264500000000001 2024-09-17T21:20:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944690+0000", 2024-09-17T21:20:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0013710549999999999 2024-09-17T21:20:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.950637+0000", 2024-09-17T21:20:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059473190000000004 2024-09-17T21:20:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.950696+0000", 2024-09-17T21:20:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8449999999999998e-05 2024-09-17T21:20:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.950736+0000", 2024-09-17T21:20:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9971000000000001e-05 2024-09-17T21:20:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3778 3.c 3:375c1b23:::rbd_data.1152a8ae2019.0000000000000052:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.902156+0000", 2024-09-17T21:20:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 32.372000956000001, 2024-09-17T21:20:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049052074000000001, 2024-09-17T21:20:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3778 2024-09-17T21:20:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902156+0000", 2024-09-17T21:20:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902156+0000", 2024-09-17T21:20:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902157+0000", 2024-09-17T21:20:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5299999999999999e-07 2024-09-17T21:20:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902160+0000", 2024-09-17T21:20:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6359999999999998e-06 2024-09-17T21:20:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902165+0000", 2024-09-17T21:20:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9440000000000002e-06 2024-09-17T21:20:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902170+0000", 2024-09-17T21:20:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:20:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.943704+0000", 2024-09-17T21:20:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041533516999999999 2024-09-17T21:20:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.943760+0000", 2024-09-17T21:20:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6552000000000003e-05 2024-09-17T21:20:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:13.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.943879+0000", 2024-09-17T21:20:13.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118347 2024-09-17T21:20:13.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:13.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944733+0000", 2024-09-17T21:20:13.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00085437299999999996 2024-09-17T21:20:13.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:13.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951158+0000", 2024-09-17T21:20:13.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064243520000000004 2024-09-17T21:20:13.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951184+0000", 2024-09-17T21:20:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6913000000000001e-05 2024-09-17T21:20:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951209+0000", 2024-09-17T21:20:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4104000000000002e-05 2024-09-17T21:20:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3785 3.c 3:3414a45b:::rbd_data.1152a8ae2019.0000000000000074:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.902292+0000", 2024-09-17T21:20:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 32.371865433000004, 2024-09-17T21:20:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049058004000000002, 2024-09-17T21:20:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3785 2024-09-17T21:20:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902292+0000", 2024-09-17T21:20:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902292+0000", 2024-09-17T21:20:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902292+0000", 2024-09-17T21:20:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:20:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902299+0000", 2024-09-17T21:20:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1210000000000001e-06 2024-09-17T21:20:13.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:13.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902305+0000", 2024-09-17T21:20:13.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6069999999999998e-06 2024-09-17T21:20:13.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:13.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902310+0000", 2024-09-17T21:20:13.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7509999999999999e-06 2024-09-17T21:20:13.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:13.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944093+0000", 2024-09-17T21:20:13.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041783582999999999 2024-09-17T21:20:13.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:13.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944159+0000", 2024-09-17T21:20:13.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5524999999999994e-05 2024-09-17T21:20:13.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:13.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944281+0000", 2024-09-17T21:20:13.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121573 2024-09-17T21:20:13.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:13.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.945107+0000", 2024-09-17T21:20:13.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00082616700000000005 2024-09-17T21:20:13.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:13.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951260+0000", 2024-09-17T21:20:13.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061530789999999997 2024-09-17T21:20:13.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:13.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951311+0000", 2024-09-17T21:20:13.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1678000000000002e-05 2024-09-17T21:20:13.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:13.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951350+0000", 2024-09-17T21:20:13.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8525000000000001e-05 2024-09-17T21:20:13.333 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.333 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:13.333 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.333 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3880 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:13.334 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.912848+0000", 2024-09-17T21:20:13.334 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 32.361309321, 2024-09-17T21:20:13.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046791626000000003, 2024-09-17T21:20:13.334 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:13.334 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:13.334 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:13.334 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:13.334 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:13.334 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:13.334 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3880 2024-09-17T21:20:13.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.334 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:13.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:13.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912848+0000", 2024-09-17T21:20:13.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:13.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912848+0000", 2024-09-17T21:20:13.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:13.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912849+0000", 2024-09-17T21:20:13.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2600000000000002e-07 2024-09-17T21:20:13.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:13.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912851+0000", 2024-09-17T21:20:13.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.475e-06 2024-09-17T21:20:13.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:13.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912857+0000", 2024-09-17T21:20:13.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1009999999999997e-06 2024-09-17T21:20:13.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:13.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912862+0000", 2024-09-17T21:20:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2639999999999999e-06 2024-09-17T21:20:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959555+0000", 2024-09-17T21:20:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046692589 2024-09-17T21:20:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959586+0000", 2024-09-17T21:20:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0978000000000001e-05 2024-09-17T21:20:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:13.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959640+0000", 2024-09-17T21:20:13.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3693000000000002e-05 2024-09-17T21:20:13.338 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.338 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:13.338 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.338 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3881 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:13.338 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.912869+0000", 2024-09-17T21:20:13.338 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 32.361288197999997, 2024-09-17T21:20:13.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046894383999999997, 2024-09-17T21:20:13.338 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:13.338 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3881 2024-09-17T21:20:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912869+0000", 2024-09-17T21:20:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912869+0000", 2024-09-17T21:20:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:13.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912870+0000", 2024-09-17T21:20:13.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:20:13.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:13.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913014+0000", 2024-09-17T21:20:13.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000144473 2024-09-17T21:20:13.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:13.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913025+0000", 2024-09-17T21:20:13.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1341e-05 2024-09-17T21:20:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913033+0000", 2024-09-17T21:20:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.17e-06 2024-09-17T21:20:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959677+0000", 2024-09-17T21:20:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046644536 2024-09-17T21:20:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959709+0000", 2024-09-17T21:20:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2348000000000003e-05 2024-09-17T21:20:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959764+0000", 2024-09-17T21:20:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4120999999999999e-05 2024-09-17T21:20:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:14003 3.4 3.4f4be604 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.912905+0000", 2024-09-17T21:20:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 32.361252358000002, 2024-09-17T21:20:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046758252, 2024-09-17T21:20:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:20:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:20:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 14003 2024-09-17T21:20:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912905+0000", 2024-09-17T21:20:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:13.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912905+0000", 2024-09-17T21:20:13.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:13.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912906+0000", 2024-09-17T21:20:13.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.091e-06 2024-09-17T21:20:13.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:13.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912911+0000", 2024-09-17T21:20:13.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9409999999999998e-06 2024-09-17T21:20:13.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:13.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912920+0000", 2024-09-17T21:20:13.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3530000000000001e-06 2024-09-17T21:20:13.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:13.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912927+0000", 2024-09-17T21:20:13.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9979999999999997e-06 2024-09-17T21:20:13.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:13.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959659+0000", 2024-09-17T21:20:13.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046731111999999998 2024-09-17T21:20:13.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:13.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959663+0000", 2024-09-17T21:20:13.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7570000000000004e-06 2024-09-17T21:20:13.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.346 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:13.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.346 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3882 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:13.347 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913045+0000", 2024-09-17T21:20:13.347 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 32.361112841999997, 2024-09-17T21:20:13.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046829914, 2024-09-17T21:20:13.347 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:13.347 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:13.347 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:13.347 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:13.347 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:13.347 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:13.347 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3882 2024-09-17T21:20:13.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.347 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:13.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:13.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913045+0000", 2024-09-17T21:20:13.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:13.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913045+0000", 2024-09-17T21:20:13.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:13.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913045+0000", 2024-09-17T21:20:13.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1200000000000002e-07 2024-09-17T21:20:13.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913048+0000", 2024-09-17T21:20:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.179e-06 2024-09-17T21:20:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913055+0000", 2024-09-17T21:20:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0269999999999998e-06 2024-09-17T21:20:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913061+0000", 2024-09-17T21:20:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5709999999999997e-06 2024-09-17T21:20:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959782+0000", 2024-09-17T21:20:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046720708 2024-09-17T21:20:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959813+0000", 2024-09-17T21:20:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1358e-05 2024-09-17T21:20:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959874+0000", 2024-09-17T21:20:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1359000000000007e-05 2024-09-17T21:20:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3883 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913069+0000", 2024-09-17T21:20:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 32.361088473999999, 2024-09-17T21:20:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046905401999999999, 2024-09-17T21:20:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:13.352 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3883 2024-09-17T21:20:13.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.352 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:13.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:13.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913069+0000", 2024-09-17T21:20:13.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:13.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913069+0000", 2024-09-17T21:20:13.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913069+0000", 2024-09-17T21:20:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:20:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913072+0000", 2024-09-17T21:20:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6390000000000001e-06 2024-09-17T21:20:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913079+0000", 2024-09-17T21:20:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2640000000000002e-06 2024-09-17T21:20:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:13.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913085+0000", 2024-09-17T21:20:13.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4249999999999997e-06 2024-09-17T21:20:13.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:13.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959893+0000", 2024-09-17T21:20:13.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046808728000000001 2024-09-17T21:20:13.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:13.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959926+0000", 2024-09-17T21:20:13.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2437999999999997e-05 2024-09-17T21:20:13.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959974+0000", 2024-09-17T21:20:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8516000000000001e-05 2024-09-17T21:20:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3884 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913092+0000", 2024-09-17T21:20:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 32.361065830999998, 2024-09-17T21:20:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047065517000000001, 2024-09-17T21:20:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3884 2024-09-17T21:20:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913092+0000", 2024-09-17T21:20:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913092+0000", 2024-09-17T21:20:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913670+0000", 2024-09-17T21:20:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00057830399999999997 2024-09-17T21:20:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913680+0000", 2024-09-17T21:20:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0426e-05 2024-09-17T21:20:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913691+0000", 2024-09-17T21:20:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0309000000000001e-05 2024-09-17T21:20:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:13.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913698+0000", 2024-09-17T21:20:13.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1760000000000004e-06 2024-09-17T21:20:13.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:13.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.960056+0000", 2024-09-17T21:20:13.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046358411000000002 2024-09-17T21:20:13.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:13.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.960089+0000", 2024-09-17T21:20:13.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2588e-05 2024-09-17T21:20:13.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.960157+0000", 2024-09-17T21:20:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8303e-05 2024-09-17T21:20:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:14004 3.4 3.4f4be604 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913136+0000", 2024-09-17T21:20:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 32.361021594999997, 2024-09-17T21:20:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046862804000000001, 2024-09-17T21:20:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:20:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:20:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 14004 2024-09-17T21:20:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913136+0000", 2024-09-17T21:20:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913136+0000", 2024-09-17T21:20:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913447+0000", 2024-09-17T21:20:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031107200000000002 2024-09-17T21:20:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913460+0000", 2024-09-17T21:20:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2996999999999999e-05 2024-09-17T21:20:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913470+0000", 2024-09-17T21:20:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0037999999999999e-05 2024-09-17T21:20:13.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:13.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913477+0000", 2024-09-17T21:20:13.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:20:13.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:13.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959994+0000", 2024-09-17T21:20:13.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046517599999999999 2024-09-17T21:20:13.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:13.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959999+0000", 2024-09-17T21:20:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1400000000000002e-06 2024-09-17T21:20:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3885 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913710+0000", 2024-09-17T21:20:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 32.360447147999999, 2024-09-17T21:20:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047449947999999999, 2024-09-17T21:20:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:13.364 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3885 2024-09-17T21:20:13.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.364 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:13.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:13.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913710+0000", 2024-09-17T21:20:13.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:13.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913710+0000", 2024-09-17T21:20:13.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:13.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.917352+0000", 2024-09-17T21:20:13.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0036412950000000001 2024-09-17T21:20:13.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:13.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.917357+0000", 2024-09-17T21:20:13.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4879999999999998e-06 2024-09-17T21:20:13.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:13.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.918061+0000", 2024-09-17T21:20:13.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00070362899999999999 2024-09-17T21:20:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.918069+0000", 2024-09-17T21:20:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4530000000000002e-06 2024-09-17T21:20:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.960956+0000", 2024-09-17T21:20:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042886951999999999 2024-09-17T21:20:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.961036+0000", 2024-09-17T21:20:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0010000000000001e-05 2024-09-17T21:20:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.961160+0000", 2024-09-17T21:20:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012412100000000001 2024-09-17T21:20:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:13984 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.921124+0000", 2024-09-17T21:20:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 32.353033171, 2024-09-17T21:20:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047129407999999998, 2024-09-17T21:20:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:20:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 13984 2024-09-17T21:20:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.921124+0000", 2024-09-17T21:20:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.921124+0000", 2024-09-17T21:20:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.928309+0000", 2024-09-17T21:20:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0071847619999999999 2024-09-17T21:20:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.928316+0000", 2024-09-17T21:20:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2969999999999998e-06 2024-09-17T21:20:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.928330+0000", 2024-09-17T21:20:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3849000000000001e-05 2024-09-17T21:20:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.928338+0000", 2024-09-17T21:20:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.354e-06 2024-09-17T21:20:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.968167+0000", 2024-09-17T21:20:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.039828298999999998 2024-09-17T21:20:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.968201+0000", 2024-09-17T21:20:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3933000000000002e-05 2024-09-17T21:20:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.968254+0000", 2024-09-17T21:20:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2914000000000002e-05 2024-09-17T21:20:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:13.372 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:13.372 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:13.372 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 0] out_osds: [2, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:20:13.372 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:20:13.372 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-17T21:20:13.372 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-17T21:20:13.374 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:20:13.374 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:13.369+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:13.398 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone30 from 0xfa0c00 (overlap 0x72ba00) to 0x395c00 2024-09-17T21:20:13.474 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:13.467+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:13.552 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:13.552 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:13.552 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:13.552 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:13.570 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:20:13.576 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:13.571+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:13.611 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:20:13.676 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:13.671+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:13.686 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:13.686 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:13.686 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:13.686 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:13.686 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:13.701 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:20:13.740 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-09-17T21:20:13.731+0000 7f33c3358640 -1 mgr.server reply reply (1) Operation not permitted no change 2024-09-17T21:20:13.741 INFO:teuthology.orchestra.run.smithi083.stdout:moved 4 / 82 (4.87805%) 2024-09-17T21:20:13.741 INFO:teuthology.orchestra.run.smithi083.stdout:avg 20.5 2024-09-17T21:20:13.741 INFO:teuthology.orchestra.run.smithi083.stdout:stddev 1.5 -> 2.87228 (expected baseline 3.9211) 2024-09-17T21:20:13.741 INFO:teuthology.orchestra.run.smithi083.stdout:min osd.0 with 19 -> 20 pgs (0.926829 -> 0.97561 * mean) 2024-09-17T21:20:13.741 INFO:teuthology.orchestra.run.smithi083.stdout:max osd.1 with 23 -> 23 pgs (1.12195 -> 1.12195 * mean) 2024-09-17T21:20:13.741 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:20:13.741 INFO:teuthology.orchestra.run.smithi083.stdout:oload 110 2024-09-17T21:20:13.741 INFO:teuthology.orchestra.run.smithi083.stdout:max_change 1 2024-09-17T21:20:13.741 INFO:teuthology.orchestra.run.smithi083.stdout:max_change_osds 4 2024-09-17T21:20:13.741 INFO:teuthology.orchestra.run.smithi083.stdout:average_utilization 0.0012 2024-09-17T21:20:13.741 INFO:teuthology.orchestra.run.smithi083.stdout:overload_utilization 0.0013 2024-09-17T21:20:13.741 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4 weight 1.0000 -> 0.7242 2024-09-17T21:20:13.742 INFO:teuthology.orchestra.run.smithi083.stderr:no change 2024-09-17T21:20:13.755 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:13.756 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:13.765 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-17T21:20:13.765 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-17T21:20:13.773 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:13.774 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:13.791 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:13.791 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:13.792 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:13.785+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:13.812 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:13.813 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:13.827 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:13.828 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:13.835 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:13.835 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:13.842 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:20:13.856 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:13.857 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:13.891 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:13.887+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:13.964 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-17T21:20:13.977 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:13.977 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:13.978 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:13.973+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:13.997 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:20:14.083 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:14.079+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:14.133 INFO:teuthology.orchestra.run.smithi184.stdout:4691 read 0x8e400 thru 0x92b0c (0x470d bytes) 2024-09-17T21:20:14.136 INFO:teuthology.orchestra.run.smithi184.stdout:4692 read 0xf6ec00 thru 0xf725d2 (0x39d3 bytes) 2024-09-17T21:20:14.166 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:14.166 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:14.181 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:20:14.181 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:14.177+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:14.220 INFO:teuthology.orchestra.run.smithi184.stdout:4693 write 0x8164800 thru 0x8164a46 (0x247 bytes) 2024-09-17T21:20:14.223 INFO:teuthology.orchestra.run.smithi184.stdout:4694 read 0x648d200 thru 0x6493baf (0x69b0 bytes) 2024-09-17T21:20:14.223 INFO:teuthology.orchestra.run.smithi184.stdout:skipping zero length punch hole 2024-09-17T21:20:14.223 INFO:teuthology.orchestra.run.smithi184.stdout:4697 read 0x55c000 thru 0x561cd8 (0x5cd9 bytes) 2024-09-17T21:20:14.225 INFO:teuthology.orchestra.run.smithi184.stdout:4699 read 0x532aa00 thru 0x5335bc9 (0xb1ca bytes) 2024-09-17T21:20:14.225 INFO:teuthology.orchestra.run.smithi184.stdout:4701 punch from 0x4605400 to 0x4607000, (0x1c00 bytes) 2024-09-17T21:20:14.230 INFO:teuthology.orchestra.run.smithi184.stdout:4703 trunc from 0x8164c00 to 0x8b07600 2024-09-17T21:20:14.247 INFO:teuthology.orchestra.run.smithi184.stdout:4706 read 0x85b2600 thru 0x85b7a9a (0x549b bytes) 2024-09-17T21:20:14.249 INFO:teuthology.orchestra.run.smithi184.stdout:4707 read 0x215c800 thru 0x216a2c5 (0xdac6 bytes) 2024-09-17T21:20:14.249 INFO:teuthology.orchestra.run.smithi184.stdout:4708 punch from 0x6402e00 to 0x6411600, (0xe800 bytes) 2024-09-17T21:20:14.249 INFO:teuthology.orchestra.run.smithi184.stdout:4709 read 0x2bdb400 thru 0x2bdbaa4 (0x6a5 bytes) 2024-09-17T21:20:14.250 INFO:teuthology.orchestra.run.smithi184.stdout:4713 write 0x3dd4e00 thru 0x3de189b (0xca9c bytes) 2024-09-17T21:20:14.251 INFO:teuthology.orchestra.run.smithi184.stdout:4716 read 0x52d5400 thru 0x52e1688 (0xc289 bytes) 2024-09-17T21:20:14.251 INFO:teuthology.orchestra.run.smithi184.stdout:4717 write 0xfaec00 thru 0xfb31e1 (0x45e2 bytes) 2024-09-17T21:20:14.251 INFO:teuthology.orchestra.run.smithi184.stdout:4718 punch from 0x444c200 to 0x445b800, (0xf600 bytes) 2024-09-17T21:20:14.251 INFO:teuthology.orchestra.run.smithi184.stdout:4720 write 0x4d63e00 thru 0x4d68be1 (0x4de2 bytes) 2024-09-17T21:20:14.252 INFO:teuthology.orchestra.run.smithi184.stdout:4722 write 0x434200 thru 0x43ea81 (0xa882 bytes) 2024-09-17T21:20:14.254 INFO:teuthology.orchestra.run.smithi184.stdout:4723 read 0x5c64000 thru 0x5c6ddd2 (0x9dd3 bytes) 2024-09-17T21:20:14.264 INFO:teuthology.orchestra.run.smithi184.stdout:4726 trunc from 0x8b07600 to 0xa334600 2024-09-17T21:20:14.281 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:14.275+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:14.285 INFO:teuthology.orchestra.run.smithi184.stdout:4727 read 0x6259c00 thru 0x625e491 (0x4892 bytes) 2024-09-17T21:20:14.294 INFO:teuthology.orchestra.run.smithi184.stdout:4732 trunc from 0xa334600 to 0x64db600 2024-09-17T21:20:14.313 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:20:14.324 INFO:teuthology.orchestra.run.smithi083.stdout:moved 1 / 82 (1.21951%) 2024-09-17T21:20:14.324 INFO:teuthology.orchestra.run.smithi083.stdout:avg 20.5 2024-09-17T21:20:14.324 INFO:teuthology.orchestra.run.smithi083.stdout:stddev 1.5 -> 1.65831 (expected baseline 3.9211) 2024-09-17T21:20:14.324 INFO:teuthology.orchestra.run.smithi083.stdout:min osd.0 with 19 -> 19 pgs (0.926829 -> 0.926829 * mean) 2024-09-17T21:20:14.324 INFO:teuthology.orchestra.run.smithi083.stdout:max osd.1 with 23 -> 23 pgs (1.12195 -> 1.12195 * mean) 2024-09-17T21:20:14.324 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:20:14.325 INFO:teuthology.orchestra.run.smithi083.stdout:oload 110 2024-09-17T21:20:14.325 INFO:teuthology.orchestra.run.smithi083.stdout:max_change 0.05 2024-09-17T21:20:14.325 INFO:teuthology.orchestra.run.smithi083.stdout:max_change_osds 4 2024-09-17T21:20:14.325 INFO:teuthology.orchestra.run.smithi083.stdout:average_utilization 0.0012 2024-09-17T21:20:14.325 INFO:teuthology.orchestra.run.smithi083.stdout:overload_utilization 0.0013 2024-09-17T21:20:14.325 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4 weight 1.0000 -> 0.9500 2024-09-17T21:20:14.348 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-17T21:20:14.348 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 0.05 2024-09-17T21:20:14.363 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:14.364 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:14.378 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:20:14.419 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:14.415+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:14.483 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:14.479+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:14.541 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:14.541 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:14.555 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:20:14.572 INFO:teuthology.orchestra.run.smithi184.stdout:4734 punch from 0x18c8800 to 0x18d3400, (0xac00 bytes) 2024-09-17T21:20:14.584 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:14.577+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:14.599 INFO:teuthology.orchestra.run.smithi184.stdout:4735 write 0xa368400 thru 0xa36a581 (0x2182 bytes) 2024-09-17T21:20:14.657 INFO:teuthology.orchestra.run.smithi184.stdout:4736 trunc from 0xa36a600 to 0xe620000 2024-09-17T21:20:14.684 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:14.677+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:14.685 INFO:teuthology.orchestra.run.smithi184.stdout:4737 read 0x7855e00 thru 0x78608d7 (0xaad8 bytes) 2024-09-17T21:20:14.688 INFO:teuthology.orchestra.run.smithi184.stdout:4738 write 0xb6fac00 thru 0xb7054dd (0xa8de bytes) 2024-09-17T21:20:14.688 INFO:teuthology.orchestra.run.smithi184.stdout:4739 write 0x99c0e00 thru 0x99cc75d (0xb95e bytes) 2024-09-17T21:20:14.689 INFO:teuthology.orchestra.run.smithi184.stdout:4741 read 0x115d600 thru 0x1167861 (0xa262 bytes) 2024-09-17T21:20:14.690 INFO:teuthology.orchestra.run.smithi184.stdout:4742 write 0x5b36a00 thru 0x5b3758c (0xb8d bytes) 2024-09-17T21:20:14.690 INFO:teuthology.orchestra.run.smithi184.stdout:4743 read 0xbcf2600 thru 0xbcf6139 (0x3b3a bytes) 2024-09-17T21:20:14.690 INFO:teuthology.orchestra.run.smithi184.stdout:4747 read 0xa804400 thru 0xa809091 (0x4c92 bytes) 2024-09-17T21:20:14.690 INFO:teuthology.orchestra.run.smithi184.stdout:4748 read 0xc2b400 thru 0xc31490 (0x6091 bytes) 2024-09-17T21:20:14.691 INFO:teuthology.orchestra.run.smithi184.stdout:4749 read 0x1b2f600 thru 0x1b356a7 (0x60a8 bytes) 2024-09-17T21:20:14.691 INFO:teuthology.orchestra.run.smithi184.stdout:4750 write 0xdd1d200 thru 0xdd25e3f (0x8c40 bytes) 2024-09-17T21:20:14.691 INFO:teuthology.orchestra.run.smithi184.stdout:4751 read 0x426e000 thru 0x4272969 (0x496a bytes) 2024-09-17T21:20:14.692 INFO:teuthology.orchestra.run.smithi184.stdout:4752 write 0x3bbae00 thru 0x3bc106a (0x626b bytes) 2024-09-17T21:20:14.692 INFO:teuthology.orchestra.run.smithi184.stdout:4754 write 0x2a79200 thru 0x2a880c4 (0xeec5 bytes) 2024-09-17T21:20:14.693 INFO:teuthology.orchestra.run.smithi184.stdout:4755 read 0x1769e00 thru 0x176bbd3 (0x1dd4 bytes) 2024-09-17T21:20:14.699 INFO:teuthology.orchestra.run.smithi083.stdout:moved 0 / 82 (0%) 2024-09-17T21:20:14.699 INFO:teuthology.orchestra.run.smithi083.stdout:avg 20.5 2024-09-17T21:20:14.699 INFO:teuthology.orchestra.run.smithi083.stdout:stddev 1.65831 -> 1.65831 (expected baseline 3.9211) 2024-09-17T21:20:14.699 INFO:teuthology.orchestra.run.smithi083.stdout:min osd.0 with 19 -> 19 pgs (0.926829 -> 0.926829 * mean) 2024-09-17T21:20:14.699 INFO:teuthology.orchestra.run.smithi083.stdout:max osd.1 with 23 -> 23 pgs (1.12195 -> 1.12195 * mean) 2024-09-17T21:20:14.699 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:20:14.699 INFO:teuthology.orchestra.run.smithi083.stdout:oload 1000 2024-09-17T21:20:14.700 INFO:teuthology.orchestra.run.smithi083.stdout:max_change 0.05 2024-09-17T21:20:14.700 INFO:teuthology.orchestra.run.smithi083.stdout:max_change_osds 4 2024-09-17T21:20:14.700 INFO:teuthology.orchestra.run.smithi083.stdout:average_utilization 0.0012 2024-09-17T21:20:14.700 INFO:teuthology.orchestra.run.smithi083.stdout:overload_utilization 0.0118 2024-09-17T21:20:14.700 INFO:teuthology.orchestra.run.smithi083.stderr:no change 2024-09-17T21:20:14.708 INFO:teuthology.orchestra.run.smithi184.stdout:4756 trunc from 0xe620000 to 0x68d0400 2024-09-17T21:20:14.718 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-17T21:20:14.719 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-17T21:20:14.728 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:14.728 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:14.744 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:20:14.760 INFO:teuthology.orchestra.run.smithi184.stdout:4758 punch from 0x5f73600 to 0x5f74200, (0xc00 bytes) 2024-09-17T21:20:14.762 INFO:teuthology.orchestra.run.smithi184.stdout:4759 write 0x542ba00 thru 0x542f91f (0x3f20 bytes) 2024-09-17T21:20:14.764 INFO:teuthology.orchestra.run.smithi184.stdout:4760 write 0x1312600 thru 0x1312ca9 (0x6aa bytes) 2024-09-17T21:20:14.764 INFO:teuthology.orchestra.run.smithi184.stdout:4764 write 0x3ac9000 thru 0x3aca03f (0x1040 bytes) 2024-09-17T21:20:14.764 INFO:teuthology.orchestra.run.smithi184.stdout:4767 punch from 0x4e9400 to 0x4ec200, (0x2e00 bytes) 2024-09-17T21:20:14.791 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:14.787+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:14.797 INFO:teuthology.orchestra.run.smithi184.stdout:4768 write 0x7596600 thru 0x75a147d (0xae7e bytes) 2024-09-17T21:20:14.823 INFO:teuthology.orchestra.run.smithi184.stdout:4771 write 0x9320800 thru 0x932a1e0 (0x99e1 bytes) 2024-09-17T21:20:14.825 INFO:teuthology.orchestra.run.smithi184.stdout:4772 read 0x9ab800 thru 0x9b74bf (0xbcc0 bytes) 2024-09-17T21:20:14.826 INFO:teuthology.orchestra.run.smithi184.stdout:4773 punch from 0x1c6b000 to 0x1c6ca00, (0x1a00 bytes) 2024-09-17T21:20:14.826 INFO:teuthology.orchestra.run.smithi184.stdout:4774 read 0x37c2a00 thru 0x37c81f4 (0x57f5 bytes) 2024-09-17T21:20:14.827 INFO:teuthology.orchestra.run.smithi184.stdout:4777 read 0x7bbbe00 thru 0x7bc07b4 (0x49b5 bytes) 2024-09-17T21:20:14.827 INFO:teuthology.orchestra.run.smithi184.stdout:4779 read 0x68c0400 thru 0x68cfd41 (0xf942 bytes) 2024-09-17T21:20:14.850 INFO:teuthology.orchestra.run.smithi184.stdout:4781 write 0xd076e00 thru 0xd082687 (0xb888 bytes) 2024-09-17T21:20:14.851 INFO:teuthology.orchestra.run.smithi184.stdout:4783 write 0x433c00 thru 0x437d52 (0x4153 bytes) 2024-09-17T21:20:14.853 INFO:teuthology.orchestra.run.smithi184.stdout:4784 punch from 0x2177e00 to 0x217c800, (0x4a00 bytes) 2024-09-17T21:20:14.853 INFO:teuthology.orchestra.run.smithi184.stdout:4785 clone 32 order 18 su 1024 sc 4 2024-09-17T21:20:14.877 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:14.877 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:14.885 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:14.881+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:14.891 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:20:14.986 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:14.979+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:15.060 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:15.060 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:15.075 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:20:15.119 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:15.115+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:15.188 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:15.183+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:15.235 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:15.235 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:15.249 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:20:15.291 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:15.287+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:15.325 INFO:teuthology.orchestra.run.smithi083.stdout:moved 4 / 82 (4.87805%) 2024-09-17T21:20:15.325 INFO:teuthology.orchestra.run.smithi083.stdout:avg 20.5 2024-09-17T21:20:15.325 INFO:teuthology.orchestra.run.smithi083.stdout:stddev 1.65831 -> 3.5 (expected baseline 3.9211) 2024-09-17T21:20:15.325 INFO:teuthology.orchestra.run.smithi083.stdout:min osd.0 with 19 -> 20 pgs (0.926829 -> 0.97561 * mean) 2024-09-17T21:20:15.325 INFO:teuthology.orchestra.run.smithi083.stdout:max osd.1 with 23 -> 23 pgs (1.12195 -> 1.12195 * mean) 2024-09-17T21:20:15.325 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:20:15.325 INFO:teuthology.orchestra.run.smithi083.stdout:oload 110 2024-09-17T21:20:15.325 INFO:teuthology.orchestra.run.smithi083.stdout:max_change 1 2024-09-17T21:20:15.325 INFO:teuthology.orchestra.run.smithi083.stdout:max_change_osds 4 2024-09-17T21:20:15.325 INFO:teuthology.orchestra.run.smithi083.stdout:average_utilization 0.0012 2024-09-17T21:20:15.325 INFO:teuthology.orchestra.run.smithi083.stdout:overload_utilization 0.0013 2024-09-17T21:20:15.325 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4 weight 0.9500 -> 0.6880 2024-09-17T21:20:15.330 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-17T21:20:15.353 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-17T21:20:15.353 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 3.0 2024-09-17T21:20:15.401 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:15.397+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:15.421 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:15.421 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:15.435 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:20:15.491 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:15.487+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:15.564 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:15.564 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:15.579 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:20:15.592 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:15.587+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:15.693 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:15.689+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:15.695 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:15.695 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:15.709 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:20:15.712 INFO:teuthology.orchestra.run.smithi083.stdout:moved 0 / 82 (0%) 2024-09-17T21:20:15.713 INFO:teuthology.orchestra.run.smithi083.stdout:avg 20.5 2024-09-17T21:20:15.713 INFO:teuthology.orchestra.run.smithi083.stdout:stddev 3.5 -> 3.5 (expected baseline 3.9211) 2024-09-17T21:20:15.713 INFO:teuthology.orchestra.run.smithi083.stdout:min osd.4 with 15 -> 15 pgs (0.731707 -> 0.731707 * mean) 2024-09-17T21:20:15.713 INFO:teuthology.orchestra.run.smithi083.stdout:max osd.3 with 24 -> 24 pgs (1.17073 -> 1.17073 * mean) 2024-09-17T21:20:15.713 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:20:15.713 INFO:teuthology.orchestra.run.smithi083.stdout:oload 1000 2024-09-17T21:20:15.713 INFO:teuthology.orchestra.run.smithi083.stdout:max_change 3 2024-09-17T21:20:15.713 INFO:teuthology.orchestra.run.smithi083.stdout:max_change_osds 4 2024-09-17T21:20:15.713 INFO:teuthology.orchestra.run.smithi083.stdout:average_utilization 0.0014 2024-09-17T21:20:15.713 INFO:teuthology.orchestra.run.smithi083.stdout:overload_utilization 0.0136 2024-09-17T21:20:15.714 INFO:teuthology.orchestra.run.smithi083.stderr:no change 2024-09-17T21:20:15.794 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:15.789+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:15.831 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:15.831 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:15.845 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:20:15.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:20:15.933 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:15.929+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:15.995 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:15.991+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:16.018 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:16.018 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:16.029 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:16.029 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:16.033 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:20:16.046 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:16.046 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:16.063 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:16.063 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:16.083 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:16.083 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:16.096 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:16.089+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:16.104 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:16.104 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:16.124 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:16.124 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:16.159 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:16.160 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:16.160 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:16.160 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:16.160 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:16.173 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:20:16.197 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:16.191+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:16.298 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:16.293+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:16.347 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:16.347 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:16.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:16.347 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:16.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.347 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:543 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:16.347 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:20:16.347 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.145007271, 2024-09-17T21:20:16.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047564097999999999, 2024-09-17T21:20:16.348 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.348 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.348 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:16.348 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:16.348 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:16.348 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:16.348 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 543 2024-09-17T21:20:16.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.348 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:20:16.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:20:16.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199663+0000", 2024-09-17T21:20:16.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9699999999999995e-07 2024-09-17T21:20:16.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199677+0000", 2024-09-17T21:20:16.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:20:16.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199690+0000", 2024-09-17T21:20:16.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2977e-05 2024-09-17T21:20:16.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199705+0000", 2024-09-17T21:20:16.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4656e-05 2024-09-17T21:20:16.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246307+0000", 2024-09-17T21:20:16.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046601780000000002 2024-09-17T21:20:16.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246475+0000", 2024-09-17T21:20:16.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016834200000000001 2024-09-17T21:20:16.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247227+0000", 2024-09-17T21:20:16.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00075173299999999998 2024-09-17T21:20:16.352 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.352 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.352 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.352 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:544 3.0 3:011b209d:::rbd_data.1152dfd5dee5.0000000000000000:6 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:16.352 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:20:16.352 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.14494886200001, 2024-09-17T21:20:16.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049283559999999997, 2024-09-17T21:20:16.352 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.352 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.352 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:16.353 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:16.353 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:16.353 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:16.353 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 544 2024-09-17T21:20:16.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.353 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:20:16.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199721+0000", 2024-09-17T21:20:16.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199722+0000", 2024-09-17T21:20:16.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.08e-06 2024-09-17T21:20:16.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199738+0000", 2024-09-17T21:20:16.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5943999999999998e-05 2024-09-17T21:20:16.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199748+0000", 2024-09-17T21:20:16.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0193e-05 2024-09-17T21:20:16.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199756+0000", 2024-09-17T21:20:16.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4200000000000001e-06 2024-09-17T21:20:16.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247311+0000", 2024-09-17T21:20:16.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047555300000000002 2024-09-17T21:20:16.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247571+0000", 2024-09-17T21:20:16.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025996999999999999 2024-09-17T21:20:16.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249005+0000", 2024-09-17T21:20:16.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001433653 2024-09-17T21:20:16.356 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.356 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.356 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.357 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:549 3.8 3:17c5d8e8:::rbd_data.1152dfd5dee5.0000000000000003:6 [sparse-read 1835008~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:16.357 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:20:16.357 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.144903238, 2024-09-17T21:20:16.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048464528999999999, 2024-09-17T21:20:16.357 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.357 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.357 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:16.357 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:16.357 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:16.357 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:16.357 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 549 2024-09-17T21:20:16.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.358 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:20:16.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:20:16.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199767+0000", 2024-09-17T21:20:16.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:20:16.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199770+0000", 2024-09-17T21:20:16.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9969999999999999e-06 2024-09-17T21:20:16.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199780+0000", 2024-09-17T21:20:16.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0003000000000001e-05 2024-09-17T21:20:16.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199793+0000", 2024-09-17T21:20:16.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2074e-05 2024-09-17T21:20:16.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247289+0000", 2024-09-17T21:20:16.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047496691000000001 2024-09-17T21:20:16.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247437+0000", 2024-09-17T21:20:16.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014807899999999999 2024-09-17T21:20:16.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248231+0000", 2024-09-17T21:20:16.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00079390899999999998 2024-09-17T21:20:16.361 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.361 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.361 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.361 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:558 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:16.361 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:20:16.362 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.144862725, 2024-09-17T21:20:16.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049709821000000001, 2024-09-17T21:20:16.362 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.362 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.362 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:16.362 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:16.362 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:16.362 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:16.362 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 558 2024-09-17T21:20:16.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.362 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:20:16.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199807+0000", 2024-09-17T21:20:16.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199809+0000", 2024-09-17T21:20:16.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.333e-06 2024-09-17T21:20:16.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199814+0000", 2024-09-17T21:20:16.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0930000000000002e-06 2024-09-17T21:20:16.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199831+0000", 2024-09-17T21:20:16.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7580000000000001e-05 2024-09-17T21:20:16.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199860+0000", 2024-09-17T21:20:16.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8609999999999999e-05 2024-09-17T21:20:16.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249106+0000", 2024-09-17T21:20:16.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049246545000000003 2024-09-17T21:20:16.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249274+0000", 2024-09-17T21:20:16.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016806699999999999 2024-09-17T21:20:16.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249517+0000", 2024-09-17T21:20:16.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024259300000000001 2024-09-17T21:20:16.366 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.366 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.366 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.366 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:559 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:16.366 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:20:16.366 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.144743363, 2024-09-17T21:20:16.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049990694000000002, 2024-09-17T21:20:16.366 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.366 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.367 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:16.367 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:16.367 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:16.367 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:16.367 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 559 2024-09-17T21:20:16.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.367 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:20:16.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199927+0000", 2024-09-17T21:20:16.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199928+0000", 2024-09-17T21:20:16.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:20:16.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199935+0000", 2024-09-17T21:20:16.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5429999999999999e-06 2024-09-17T21:20:16.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199954+0000", 2024-09-17T21:20:16.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.912e-05 2024-09-17T21:20:16.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199969+0000", 2024-09-17T21:20:16.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5305999999999999e-05 2024-09-17T21:20:16.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249604+0000", 2024-09-17T21:20:16.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049634931 2024-09-17T21:20:16.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249738+0000", 2024-09-17T21:20:16.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013411599999999999 2024-09-17T21:20:16.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249917+0000", 2024-09-17T21:20:16.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179111 2024-09-17T21:20:16.370 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.370 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.370 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.371 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:560 3.0 3:00cc64ba:::rbd_data.1152dfd5dee5.000000000000000b:6 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:16.371 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:20:16.371 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.144678947, 2024-09-17T21:20:16.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050173059999999998, 2024-09-17T21:20:16.371 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.371 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.371 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:16.371 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:16.371 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:16.371 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:16.371 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 560 2024-09-17T21:20:16.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.371 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:20:16.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199991+0000", 2024-09-17T21:20:16.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199992+0000", 2024-09-17T21:20:16.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.517e-06 2024-09-17T21:20:16.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.199998+0000", 2024-09-17T21:20:16.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1460000000000004e-06 2024-09-17T21:20:16.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200015+0000", 2024-09-17T21:20:16.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7407000000000001e-05 2024-09-17T21:20:16.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200029+0000", 2024-09-17T21:20:16.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3499999999999999e-05 2024-09-17T21:20:16.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249968+0000", 2024-09-17T21:20:16.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049939799 2024-09-17T21:20:16.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250055+0000", 2024-09-17T21:20:16.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6731999999999996e-05 2024-09-17T21:20:16.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.250164+0000", 2024-09-17T21:20:16.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108959 2024-09-17T21:20:16.375 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.375 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.375 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.375 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:20:16.375 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:16.375 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 96.534889238000005, 2024-09-17T21:20:16.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:20:16.375 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.376 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.376 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:16.376 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:16.376 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:20:16.376 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:20:16.376 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:20:16.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.376 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:16.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:16.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:20:16.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:20:16.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:20:16.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:20:16.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:20:16.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:20:16.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:20:16.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:20:16.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:20:16.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:20:16.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:16.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:20:16.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:20:16.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:20:16.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:20:16.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:20:16.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:20:16.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:20:16.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:20:16.380 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.380 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.381 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.381 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:20:16.381 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:16.381 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 82.979787451999997, 2024-09-17T21:20:16.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:20:16.381 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.381 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.381 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:16.381 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:16.381 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:20:16.381 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:16.382 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:20:16.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.382 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:16.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:16.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:20:16.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:20:16.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:20:16.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:20:16.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:20:16.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:20:16.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:20:16.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:20:16.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:20:16.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:20:16.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:16.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:20:16.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:20:16.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:20:16.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:20:16.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:20:16.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:20:16.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:16.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:20:16.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:20:16.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:16.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:20:16.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:20:16.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:16.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:20:16.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:20:16.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:16.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:20:16.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:20:16.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:20:16.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:20:16.387 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.387 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.388 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.388 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3777 3.c 3:302136a0:::rbd_data.1152a8ae2019.000000000000004b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:16.388 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.902125+0000", 2024-09-17T21:20:16.388 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 35.442544826000002, 2024-09-17T21:20:16.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048610548000000003, 2024-09-17T21:20:16.388 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.388 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.388 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:16.388 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:16.388 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:16.388 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:16.389 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3777 2024-09-17T21:20:16.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.389 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902125+0000", 2024-09-17T21:20:16.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902125+0000", 2024-09-17T21:20:16.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902126+0000", 2024-09-17T21:20:16.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:20:16.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902138+0000", 2024-09-17T21:20:16.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1994000000000001e-05 2024-09-17T21:20:16.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902144+0000", 2024-09-17T21:20:16.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2820000000000002e-06 2024-09-17T21:20:16.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902149+0000", 2024-09-17T21:20:16.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.135e-06 2024-09-17T21:20:16.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.942910+0000", 2024-09-17T21:20:16.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.040761190000000003 2024-09-17T21:20:16.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.942986+0000", 2024-09-17T21:20:16.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6083999999999999e-05 2024-09-17T21:20:16.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:16.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.943319+0000", 2024-09-17T21:20:16.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00033264500000000001 2024-09-17T21:20:16.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:16.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944690+0000", 2024-09-17T21:20:16.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0013710549999999999 2024-09-17T21:20:16.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:16.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.950637+0000", 2024-09-17T21:20:16.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059473190000000004 2024-09-17T21:20:16.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:16.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.950696+0000", 2024-09-17T21:20:16.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8449999999999998e-05 2024-09-17T21:20:16.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.950736+0000", 2024-09-17T21:20:16.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9971000000000001e-05 2024-09-17T21:20:16.394 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.394 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.394 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.394 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3778 3.c 3:375c1b23:::rbd_data.1152a8ae2019.0000000000000052:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:16.394 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.902156+0000", 2024-09-17T21:20:16.394 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 35.442513472000002, 2024-09-17T21:20:16.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049052074000000001, 2024-09-17T21:20:16.394 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.394 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.394 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:16.394 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:16.394 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:16.395 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:16.395 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3778 2024-09-17T21:20:16.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.395 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902156+0000", 2024-09-17T21:20:16.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902156+0000", 2024-09-17T21:20:16.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902157+0000", 2024-09-17T21:20:16.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5299999999999999e-07 2024-09-17T21:20:16.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902160+0000", 2024-09-17T21:20:16.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6359999999999998e-06 2024-09-17T21:20:16.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902165+0000", 2024-09-17T21:20:16.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9440000000000002e-06 2024-09-17T21:20:16.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902170+0000", 2024-09-17T21:20:16.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:20:16.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.943704+0000", 2024-09-17T21:20:16.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041533516999999999 2024-09-17T21:20:16.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.943760+0000", 2024-09-17T21:20:16.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6552000000000003e-05 2024-09-17T21:20:16.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:16.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.943879+0000", 2024-09-17T21:20:16.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118347 2024-09-17T21:20:16.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:16.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944733+0000", 2024-09-17T21:20:16.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00085437299999999996 2024-09-17T21:20:16.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:16.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951158+0000", 2024-09-17T21:20:16.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064243520000000004 2024-09-17T21:20:16.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:16.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951184+0000", 2024-09-17T21:20:16.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6913000000000001e-05 2024-09-17T21:20:16.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951209+0000", 2024-09-17T21:20:16.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4104000000000002e-05 2024-09-17T21:20:16.399 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3785 3.c 3:3414a45b:::rbd_data.1152a8ae2019.0000000000000074:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.902292+0000", 2024-09-17T21:20:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 35.442377948999997, 2024-09-17T21:20:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049058004000000002, 2024-09-17T21:20:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3785 2024-09-17T21:20:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902292+0000", 2024-09-17T21:20:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902292+0000", 2024-09-17T21:20:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902292+0000", 2024-09-17T21:20:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:20:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902299+0000", 2024-09-17T21:20:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1210000000000001e-06 2024-09-17T21:20:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902305+0000", 2024-09-17T21:20:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6069999999999998e-06 2024-09-17T21:20:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.902310+0000", 2024-09-17T21:20:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7509999999999999e-06 2024-09-17T21:20:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944093+0000", 2024-09-17T21:20:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041783582999999999 2024-09-17T21:20:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944159+0000", 2024-09-17T21:20:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5524999999999994e-05 2024-09-17T21:20:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.944281+0000", 2024-09-17T21:20:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121573 2024-09-17T21:20:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.945107+0000", 2024-09-17T21:20:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00082616700000000005 2024-09-17T21:20:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951260+0000", 2024-09-17T21:20:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061530789999999997 2024-09-17T21:20:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951311+0000", 2024-09-17T21:20:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1678000000000002e-05 2024-09-17T21:20:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.951350+0000", 2024-09-17T21:20:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8525000000000001e-05 2024-09-17T21:20:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3880 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.912848+0000", 2024-09-17T21:20:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 35.431821837000001, 2024-09-17T21:20:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046791626000000003, 2024-09-17T21:20:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3880 2024-09-17T21:20:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912848+0000", 2024-09-17T21:20:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912848+0000", 2024-09-17T21:20:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912849+0000", 2024-09-17T21:20:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2600000000000002e-07 2024-09-17T21:20:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912851+0000", 2024-09-17T21:20:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.475e-06 2024-09-17T21:20:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912857+0000", 2024-09-17T21:20:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1009999999999997e-06 2024-09-17T21:20:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912862+0000", 2024-09-17T21:20:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2639999999999999e-06 2024-09-17T21:20:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959555+0000", 2024-09-17T21:20:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046692589 2024-09-17T21:20:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959586+0000", 2024-09-17T21:20:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0978000000000001e-05 2024-09-17T21:20:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959640+0000", 2024-09-17T21:20:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3693000000000002e-05 2024-09-17T21:20:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3881 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.912869+0000", 2024-09-17T21:20:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 35.431800713999998, 2024-09-17T21:20:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046894383999999997, 2024-09-17T21:20:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3881 2024-09-17T21:20:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912869+0000", 2024-09-17T21:20:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912869+0000", 2024-09-17T21:20:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912870+0000", 2024-09-17T21:20:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:20:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913014+0000", 2024-09-17T21:20:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000144473 2024-09-17T21:20:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913025+0000", 2024-09-17T21:20:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1341e-05 2024-09-17T21:20:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913033+0000", 2024-09-17T21:20:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.17e-06 2024-09-17T21:20:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959677+0000", 2024-09-17T21:20:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046644536 2024-09-17T21:20:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959709+0000", 2024-09-17T21:20:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2348000000000003e-05 2024-09-17T21:20:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959764+0000", 2024-09-17T21:20:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4120999999999999e-05 2024-09-17T21:20:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:14003 3.4 3.4f4be604 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.912905+0000", 2024-09-17T21:20:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 35.431764874000002, 2024-09-17T21:20:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046758252, 2024-09-17T21:20:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:20:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:20:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 14003 2024-09-17T21:20:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912905+0000", 2024-09-17T21:20:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912905+0000", 2024-09-17T21:20:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912906+0000", 2024-09-17T21:20:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.091e-06 2024-09-17T21:20:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912911+0000", 2024-09-17T21:20:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9409999999999998e-06 2024-09-17T21:20:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912920+0000", 2024-09-17T21:20:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3530000000000001e-06 2024-09-17T21:20:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.912927+0000", 2024-09-17T21:20:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9979999999999997e-06 2024-09-17T21:20:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959659+0000", 2024-09-17T21:20:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046731111999999998 2024-09-17T21:20:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959663+0000", 2024-09-17T21:20:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7570000000000004e-06 2024-09-17T21:20:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3882 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913045+0000", 2024-09-17T21:20:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 35.431625357999998, 2024-09-17T21:20:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046829914, 2024-09-17T21:20:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3882 2024-09-17T21:20:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913045+0000", 2024-09-17T21:20:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913045+0000", 2024-09-17T21:20:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913045+0000", 2024-09-17T21:20:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1200000000000002e-07 2024-09-17T21:20:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913048+0000", 2024-09-17T21:20:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.179e-06 2024-09-17T21:20:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913055+0000", 2024-09-17T21:20:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0269999999999998e-06 2024-09-17T21:20:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913061+0000", 2024-09-17T21:20:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5709999999999997e-06 2024-09-17T21:20:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959782+0000", 2024-09-17T21:20:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046720708 2024-09-17T21:20:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959813+0000", 2024-09-17T21:20:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1358e-05 2024-09-17T21:20:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959874+0000", 2024-09-17T21:20:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1359000000000007e-05 2024-09-17T21:20:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3883 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913069+0000", 2024-09-17T21:20:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 35.43160099, 2024-09-17T21:20:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046905401999999999, 2024-09-17T21:20:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3883 2024-09-17T21:20:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913069+0000", 2024-09-17T21:20:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913069+0000", 2024-09-17T21:20:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913069+0000", 2024-09-17T21:20:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:20:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913072+0000", 2024-09-17T21:20:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6390000000000001e-06 2024-09-17T21:20:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913079+0000", 2024-09-17T21:20:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2640000000000002e-06 2024-09-17T21:20:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913085+0000", 2024-09-17T21:20:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4249999999999997e-06 2024-09-17T21:20:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959893+0000", 2024-09-17T21:20:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046808728000000001 2024-09-17T21:20:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959926+0000", 2024-09-17T21:20:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2437999999999997e-05 2024-09-17T21:20:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959974+0000", 2024-09-17T21:20:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8516000000000001e-05 2024-09-17T21:20:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3884 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913092+0000", 2024-09-17T21:20:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 35.431578346999999, 2024-09-17T21:20:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047065517000000001, 2024-09-17T21:20:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3884 2024-09-17T21:20:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913092+0000", 2024-09-17T21:20:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913092+0000", 2024-09-17T21:20:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913670+0000", 2024-09-17T21:20:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00057830399999999997 2024-09-17T21:20:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913680+0000", 2024-09-17T21:20:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0426e-05 2024-09-17T21:20:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913691+0000", 2024-09-17T21:20:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0309000000000001e-05 2024-09-17T21:20:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913698+0000", 2024-09-17T21:20:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1760000000000004e-06 2024-09-17T21:20:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.960056+0000", 2024-09-17T21:20:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046358411000000002 2024-09-17T21:20:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.960089+0000", 2024-09-17T21:20:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2588e-05 2024-09-17T21:20:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.960157+0000", 2024-09-17T21:20:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8303e-05 2024-09-17T21:20:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:14004 3.4 3.4f4be604 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913136+0000", 2024-09-17T21:20:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 35.431534110999998, 2024-09-17T21:20:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046862804000000001, 2024-09-17T21:20:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:20:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:20:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 14004 2024-09-17T21:20:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913136+0000", 2024-09-17T21:20:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913136+0000", 2024-09-17T21:20:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913447+0000", 2024-09-17T21:20:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031107200000000002 2024-09-17T21:20:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913460+0000", 2024-09-17T21:20:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2996999999999999e-05 2024-09-17T21:20:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913470+0000", 2024-09-17T21:20:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0037999999999999e-05 2024-09-17T21:20:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913477+0000", 2024-09-17T21:20:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:20:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959994+0000", 2024-09-17T21:20:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046517599999999999 2024-09-17T21:20:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.959999+0000", 2024-09-17T21:20:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1400000000000002e-06 2024-09-17T21:20:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3885 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.913710+0000", 2024-09-17T21:20:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 35.430959664, 2024-09-17T21:20:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047449947999999999, 2024-09-17T21:20:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3885 2024-09-17T21:20:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913710+0000", 2024-09-17T21:20:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.913710+0000", 2024-09-17T21:20:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.917352+0000", 2024-09-17T21:20:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0036412950000000001 2024-09-17T21:20:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.917357+0000", 2024-09-17T21:20:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4879999999999998e-06 2024-09-17T21:20:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.918061+0000", 2024-09-17T21:20:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00070362899999999999 2024-09-17T21:20:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.918069+0000", 2024-09-17T21:20:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4530000000000002e-06 2024-09-17T21:20:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.960956+0000", 2024-09-17T21:20:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042886951999999999 2024-09-17T21:20:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.961036+0000", 2024-09-17T21:20:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0010000000000001e-05 2024-09-17T21:20:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.961160+0000", 2024-09-17T21:20:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012412100000000001 2024-09-17T21:20:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:13984 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [notify-ack] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.921124+0000", 2024-09-17T21:20:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 35.423545687000001, 2024-09-17T21:20:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047129407999999998, 2024-09-17T21:20:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:20:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 13984 2024-09-17T21:20:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.921124+0000", 2024-09-17T21:20:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.921124+0000", 2024-09-17T21:20:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.928309+0000", 2024-09-17T21:20:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0071847619999999999 2024-09-17T21:20:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.928316+0000", 2024-09-17T21:20:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2969999999999998e-06 2024-09-17T21:20:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.928330+0000", 2024-09-17T21:20:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3849000000000001e-05 2024-09-17T21:20:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.928338+0000", 2024-09-17T21:20:16.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.354e-06 2024-09-17T21:20:16.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.968167+0000", 2024-09-17T21:20:16.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.039828298999999998 2024-09-17T21:20:16.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.968201+0000", 2024-09-17T21:20:16.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3933000000000002e-05 2024-09-17T21:20:16.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.968254+0000", 2024-09-17T21:20:16.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2914000000000002e-05 2024-09-17T21:20:16.445 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.445 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.445 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.445 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.445 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.445 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:16.446 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone31 from 0xd082800 (overlap 0xfa0c00) to 0x960600 2024-09-17T21:20:16.447 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:20:16.447 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:16.439+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:16.548 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:16.543+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:16.619 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:16.620 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:16.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:20:16.648 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:16.643+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:16.749 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:16.743+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:16.756 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:16.756 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:16.756 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:16.756 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:16.757 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:16.770 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:20:16.850 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:16.845+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:16.896 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:16.896 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:16.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:16.897 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:16.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.897 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:20:16.897 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:16.897 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 118.675639123, 2024-09-17T21:20:16.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:20:16.897 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.897 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.897 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:16.897 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:16.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:16.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:20:16.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:20:16.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:20:16.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:20:16.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:20:16.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:20:16.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:20:16.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:20:16.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:20:16.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:20:16.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:20:16.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:20:16.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:16.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:20:16.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:20:16.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:20:16.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:20:16.901 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.901 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.901 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.901 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:20:16.901 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:16.901 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 114.025338459, 2024-09-17T21:20:16.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:20:16.901 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.902 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.902 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:16.902 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:16.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:16.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:20:16.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:20:16.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:20:16.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:20:16.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:20:16.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:20:16.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:20:16.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:20:16.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:20:16.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:20:16.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:20:16.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:20:16.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:16.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:20:16.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:20:16.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:20:16.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:20:16.905 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.905 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.905 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.906 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:20:16.906 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:16.906 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 114.024073376, 2024-09-17T21:20:16.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:20:16.906 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.906 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.906 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:16.906 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:16.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:16.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:20:16.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:20:16.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:20:16.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:20:16.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:20:16.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:20:16.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:20:16.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:20:16.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:20:16.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:20:16.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:20:16.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:20:16.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:16.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:20:16.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:20:16.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:20:16.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:20:16.910 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.910 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.910 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.910 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:20:16.910 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:16.910 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 114.014293966, 2024-09-17T21:20:16.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:20:16.910 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.910 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.911 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:16.911 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:16.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:16.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:20:16.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:20:16.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:20:16.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:20:16.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:20:16.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:20:16.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:20:16.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:20:16.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:20:16.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:20:16.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:20:16.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:20:16.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:16.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:20:16.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:20:16.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:20:16.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:20:16.914 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.914 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.914 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.915 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:20:16.915 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:16.915 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 114.005571563, 2024-09-17T21:20:16.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:20:16.915 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.915 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.915 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:16.915 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:16.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:16.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:20:16.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:20:16.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:20:16.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:20:16.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:20:16.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:20:16.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:20:16.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:20:16.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:20:16.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:20:16.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:20:16.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:20:16.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:16.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:20:16.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:20:16.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:20:16.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:20:16.919 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.919 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.919 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.920 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:20:16.920 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:16.920 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 100.62048099899999, 2024-09-17T21:20:16.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:20:16.920 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.920 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.920 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:16.920 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:16.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:16.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:20:16.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:20:16.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:20:16.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:20:16.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:20:16.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:20:16.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:20:16.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:20:16.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:20:16.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:20:16.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:20:16.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:20:16.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:16.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:20:16.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:20:16.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:20:16.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:20:16.924 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.924 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.924 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.924 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1780 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'195, mlcod=69'190)", 2024-09-17T21:20:16.924 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:16.924 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.642625395000003, 2024-09-17T21:20:16.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075761769999999999, 2024-09-17T21:20:16.924 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.925 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.925 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:16.925 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:16.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:16.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250375+0000", 2024-09-17T21:20:16.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:20:16.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250383+0000", 2024-09-17T21:20:16.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7870000000000002e-06 2024-09-17T21:20:16.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250402+0000", 2024-09-17T21:20:16.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9346000000000001e-05 2024-09-17T21:20:16.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250413+0000", 2024-09-17T21:20:16.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.039e-05 2024-09-17T21:20:16.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251353+0000", 2024-09-17T21:20:16.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000940091 2024-09-17T21:20:16.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251407+0000", 2024-09-17T21:20:16.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4701999999999999e-05 2024-09-17T21:20:16.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:16.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257829+0000", 2024-09-17T21:20:16.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064212779999999999 2024-09-17T21:20:16.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257950+0000", 2024-09-17T21:20:16.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121623 2024-09-17T21:20:16.928 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.928 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.928 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.929 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1781 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'196, mlcod=69'190)", 2024-09-17T21:20:16.929 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:16.929 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.641542444000002, 2024-09-17T21:20:16.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065632939999999999, 2024-09-17T21:20:16.929 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.929 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.929 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:16.929 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:16.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:16.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251459+0000", 2024-09-17T21:20:16.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.624e-06 2024-09-17T21:20:16.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251471+0000", 2024-09-17T21:20:16.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2636000000000001e-05 2024-09-17T21:20:16.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251498+0000", 2024-09-17T21:20:16.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:20:16.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251515+0000", 2024-09-17T21:20:16.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6939000000000002e-05 2024-09-17T21:20:16.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252938+0000", 2024-09-17T21:20:16.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001423082 2024-09-17T21:20:16.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252999+0000", 2024-09-17T21:20:16.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1138999999999999e-05 2024-09-17T21:20:16.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:16.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257955+0000", 2024-09-17T21:20:16.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049559640000000002 2024-09-17T21:20:16.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.258020+0000", 2024-09-17T21:20:16.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5504000000000002e-05 2024-09-17T21:20:16.933 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.933 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.933 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.933 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1788 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'198, mlcod=69'191)", 2024-09-17T21:20:16.933 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:16.933 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.639516954999998, 2024-09-17T21:20:16.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073690370000000002, 2024-09-17T21:20:16.933 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.933 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.933 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:16.934 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:16.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:16.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253484+0000", 2024-09-17T21:20:16.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2470000000000001e-06 2024-09-17T21:20:16.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253492+0000", 2024-09-17T21:20:16.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7159999999999992e-06 2024-09-17T21:20:16.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253509+0000", 2024-09-17T21:20:16.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.666e-05 2024-09-17T21:20:16.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253522+0000", 2024-09-17T21:20:16.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2643000000000001e-05 2024-09-17T21:20:16.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254853+0000", 2024-09-17T21:20:16.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331238 2024-09-17T21:20:16.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254885+0000", 2024-09-17T21:20:16.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1562e-05 2024-09-17T21:20:16.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:16.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260788+0000", 2024-09-17T21:20:16.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059030749999999998 2024-09-17T21:20:16.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260852+0000", 2024-09-17T21:20:16.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3895999999999999e-05 2024-09-17T21:20:16.937 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.937 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.937 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.938 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5746 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'477, mlcod=71'473)", 2024-09-17T21:20:16.938 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:16.938 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.634089396999997, 2024-09-17T21:20:16.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074182709999999997, 2024-09-17T21:20:16.938 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.938 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.938 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:16.938 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:16.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:16.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258912+0000", 2024-09-17T21:20:16.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:20:16.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258926+0000", 2024-09-17T21:20:16.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4249e-05 2024-09-17T21:20:16.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258946+0000", 2024-09-17T21:20:16.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9879999999999999e-05 2024-09-17T21:20:16.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258956+0000", 2024-09-17T21:20:16.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:20:16.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259380+0000", 2024-09-17T21:20:16.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042337599999999999 2024-09-17T21:20:16.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259412+0000", 2024-09-17T21:20:16.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:20:16.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:16.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266191+0000", 2024-09-17T21:20:16.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067793269999999999 2024-09-17T21:20:16.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266328+0000", 2024-09-17T21:20:16.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013690999999999999 2024-09-17T21:20:16.942 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.942 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.942 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.942 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5747 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'478, mlcod=71'473)", 2024-09-17T21:20:16.942 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:16.942 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.633383047999999, 2024-09-17T21:20:16.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068779440000000004, 2024-09-17T21:20:16.942 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.942 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.942 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:16.942 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:16.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:16.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259618+0000", 2024-09-17T21:20:16.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6199999999999999e-06 2024-09-17T21:20:16.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259630+0000", 2024-09-17T21:20:16.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1884e-05 2024-09-17T21:20:16.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259657+0000", 2024-09-17T21:20:16.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7339e-05 2024-09-17T21:20:16.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259677+0000", 2024-09-17T21:20:16.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0156000000000001e-05 2024-09-17T21:20:16.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260521+0000", 2024-09-17T21:20:16.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084312899999999997 2024-09-17T21:20:16.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260585+0000", 2024-09-17T21:20:16.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4558000000000001e-05 2024-09-17T21:20:16.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:16.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266336+0000", 2024-09-17T21:20:16.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057505020000000002 2024-09-17T21:20:16.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266494+0000", 2024-09-17T21:20:16.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158756 2024-09-17T21:20:16.946 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.946 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.946 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.946 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5750 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'479, mlcod=71'473)", 2024-09-17T21:20:16.947 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:16.947 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.632670855000001, 2024-09-17T21:20:16.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006302874, 2024-09-17T21:20:16.947 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.947 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.947 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:16.947 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:16.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:16.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260330+0000", 2024-09-17T21:20:16.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:20:16.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260343+0000", 2024-09-17T21:20:16.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2862e-05 2024-09-17T21:20:16.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260367+0000", 2024-09-17T21:20:16.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3723e-05 2024-09-17T21:20:16.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260386+0000", 2024-09-17T21:20:16.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8879999999999999e-05 2024-09-17T21:20:16.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261579+0000", 2024-09-17T21:20:16.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001193431 2024-09-17T21:20:16.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261608+0000", 2024-09-17T21:20:16.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8858999999999999e-05 2024-09-17T21:20:16.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:16.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266505+0000", 2024-09-17T21:20:16.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004896999 2024-09-17T21:20:16.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266631+0000", 2024-09-17T21:20:16.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012655300000000001 2024-09-17T21:20:16.951 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.951 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.951 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.951 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5751 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'480, mlcod=71'473)", 2024-09-17T21:20:16.951 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:16.951 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.631963642999999, 2024-09-17T21:20:16.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075756299999999999, 2024-09-17T21:20:16.951 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.951 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.951 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:16.952 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:16.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:16.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:16.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261037+0000", 2024-09-17T21:20:16.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:20:16.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:16.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261049+0000", 2024-09-17T21:20:16.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1063999999999999e-05 2024-09-17T21:20:16.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:16.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261072+0000", 2024-09-17T21:20:16.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3102e-05 2024-09-17T21:20:16.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:16.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261133+0000", 2024-09-17T21:20:16.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1276000000000001e-05 2024-09-17T21:20:16.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:16.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:20:16.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015334319999999999 2024-09-17T21:20:16.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:16.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262702+0000", 2024-09-17T21:20:16.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5281999999999999e-05 2024-09-17T21:20:16.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:16.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268527+0000", 2024-09-17T21:20:16.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058250389999999997 2024-09-17T21:20:16.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:16.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268611+0000", 2024-09-17T21:20:16.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4724999999999999e-05 2024-09-17T21:20:16.955 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.955 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:16.955 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:16.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.956 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5756 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'481, mlcod=71'473)", 2024-09-17T21:20:16.956 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:16.956 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.631102787000003, 2024-09-17T21:20:16.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006772386, 2024-09-17T21:20:16.956 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:16.956 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:16.956 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:16.956 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:16.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:16.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:16.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:16.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:16.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:16.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:16.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:16.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:17.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:17.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261899+0000", 2024-09-17T21:20:17.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.903e-06 2024-09-17T21:20:17.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261912+0000", 2024-09-17T21:20:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-05 2024-09-17T21:20:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261938+0000", 2024-09-17T21:20:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6098999999999999e-05 2024-09-17T21:20:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261955+0000", 2024-09-17T21:20:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.696e-05 2024-09-17T21:20:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263845+0000", 2024-09-17T21:20:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001890187 2024-09-17T21:20:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263907+0000", 2024-09-17T21:20:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1600999999999995e-05 2024-09-17T21:20:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268615+0000", 2024-09-17T21:20:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047086070000000001 2024-09-17T21:20:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268669+0000", 2024-09-17T21:20:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3588999999999998e-05 2024-09-17T21:20:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4978.0:598 3.9 e80/23 3:9701cc17:::rbd_data.1152724ff2a3.0000000000000019:head v 80'202, mlcod=80'198)", 2024-09-17T21:20:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.129528403999998, 2024-09-17T21:20:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066473340000000004, 2024-09-17T21:20:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763472+0000", 2024-09-17T21:20:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6899999999999997e-07 2024-09-17T21:20:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763478+0000", 2024-09-17T21:20:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:20:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763489+0000", 2024-09-17T21:20:17.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0628999999999999e-05 2024-09-17T21:20:17.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763495+0000", 2024-09-17T21:20:17.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6649999999999997e-06 2024-09-17T21:20:17.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:17.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763907+0000", 2024-09-17T21:20:17.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041163899999999999 2024-09-17T21:20:17.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:17.101 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763976+0000", 2024-09-17T21:20:17.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8534999999999993e-05 2024-09-17T21:20:17.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.101 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:17.101 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.769900+0000", 2024-09-17T21:20:17.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059243270000000001 2024-09-17T21:20:17.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.101 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:17.101 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.770118+0000", 2024-09-17T21:20:17.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021842500000000001 2024-09-17T21:20:17.101 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:17.101 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:17.102 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:17.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.102 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:468 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'94, mlcod=89'87)", 2024-09-17T21:20:17.102 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:17.102 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.206783105, 2024-09-17T21:20:17.102 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066528790000000004, 2024-09-17T21:20:17.102 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:17.102 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:17.102 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:17.102 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:17.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.102 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:17.102 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:17.102 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:17.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:17.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:17.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:17.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:17.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686218+0000", 2024-09-17T21:20:17.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3060000000000001e-06 2024-09-17T21:20:17.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:17.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686256+0000", 2024-09-17T21:20:17.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8333e-05 2024-09-17T21:20:17.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:17.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686272+0000", 2024-09-17T21:20:17.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:20:17.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686283+0000", 2024-09-17T21:20:17.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0903e-05 2024-09-17T21:20:17.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:17.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686682+0000", 2024-09-17T21:20:17.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039935099999999999 2024-09-17T21:20:17.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:17.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686785+0000", 2024-09-17T21:20:17.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010275199999999999 2024-09-17T21:20:17.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:17.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692816+0000", 2024-09-17T21:20:17.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060317579999999999 2024-09-17T21:20:17.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.106 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:17.106 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692869+0000", 2024-09-17T21:20:17.106 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2775999999999998e-05 2024-09-17T21:20:17.106 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:17.106 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:17.106 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:17.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.106 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:755 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'127, mlcod=89'122)", 2024-09-17T21:20:17.106 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:17.106 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.070327890999998, 2024-09-17T21:20:17.106 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084014080000000008, 2024-09-17T21:20:17.107 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:17.107 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:17.107 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:17.107 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:17.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:17.107 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:17.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:17.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:17.107 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:17.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:17.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:17.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:17.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:20:17.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:17.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822683+0000", 2024-09-17T21:20:17.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0674e-05 2024-09-17T21:20:17.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:17.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822698+0000", 2024-09-17T21:20:17.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5189e-05 2024-09-17T21:20:17.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822708+0000", 2024-09-17T21:20:17.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9119999999999993e-06 2024-09-17T21:20:17.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:17.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822748+0000", 2024-09-17T21:20:17.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9727000000000001e-05 2024-09-17T21:20:17.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:17.110 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822759+0000", 2024-09-17T21:20:17.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0865e-05 2024-09-17T21:20:17.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.110 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:17.110 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831034+0000", 2024-09-17T21:20:17.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082750710000000002 2024-09-17T21:20:17.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.110 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:17.110 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831073+0000", 2024-09-17T21:20:17.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9063999999999997e-05 2024-09-17T21:20:17.110 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:17.110 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:17.111 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:17.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.111 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:756 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'128, mlcod=89'124)", 2024-09-17T21:20:17.111 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:17.111 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.068405259000002, 2024-09-17T21:20:17.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0076300270000000002, 2024-09-17T21:20:17.111 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:17.111 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:17.111 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:17.111 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:17.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:17.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:17.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:17.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:17.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:17.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:17.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:17.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824595+0000", 2024-09-17T21:20:17.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-17T21:20:17.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:17.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824600+0000", 2024-09-17T21:20:17.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:20:17.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:17.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824612+0000", 2024-09-17T21:20:17.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.181e-05 2024-09-17T21:20:17.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824619+0000", 2024-09-17T21:20:17.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:20:17.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:17.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824671+0000", 2024-09-17T21:20:17.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2262999999999998e-05 2024-09-17T21:20:17.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:17.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824695+0000", 2024-09-17T21:20:17.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3883000000000002e-05 2024-09-17T21:20:17.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:17.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832185+0000", 2024-09-17T21:20:17.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007490048 2024-09-17T21:20:17.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:17.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832224+0000", 2024-09-17T21:20:17.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8957e-05 2024-09-17T21:20:17.115 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:17.115 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:17.115 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:17.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.115 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2617 3.d e90/23 3:b0143718:::rbd_data.1152a8ae2019.0000000000000016:head v 90'249, mlcod=90'246)", 2024-09-17T21:20:17.115 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:17.115 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 36.048554258999999, 2024-09-17T21:20:17.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063715559999999996, 2024-09-17T21:20:17.115 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:17.116 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:17.116 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:17.116 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:17.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:17.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:17.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:17.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:17.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:17.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:17.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:17.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844455+0000", 2024-09-17T21:20:17.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4259999999999992e-06 2024-09-17T21:20:17.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:17.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844470+0000", 2024-09-17T21:20:17.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5386000000000001e-05 2024-09-17T21:20:17.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:17.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844499+0000", 2024-09-17T21:20:17.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.883e-05 2024-09-17T21:20:17.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844518+0000", 2024-09-17T21:20:17.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8830000000000001e-05 2024-09-17T21:20:17.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:17.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844649+0000", 2024-09-17T21:20:17.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013143399999999999 2024-09-17T21:20:17.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:17.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844702+0000", 2024-09-17T21:20:17.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2876e-05 2024-09-17T21:20:17.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.119 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:17.119 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850696+0000", 2024-09-17T21:20:17.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059943729999999999 2024-09-17T21:20:17.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.119 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:17.119 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850817+0000", 2024-09-17T21:20:17.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120401 2024-09-17T21:20:17.119 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:17.119 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:17.119 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:17.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.119 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2635 3.d e90/23 3:b3d1e920:::rbd_data.1152a8ae2019.000000000000001f:head v 90'252, mlcod=90'246)", 2024-09-17T21:20:17.120 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:17.120 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 36.045846189000002, 2024-09-17T21:20:17.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0077265190000000003, 2024-09-17T21:20:17.120 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:17.120 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:17.120 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:17.120 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:17.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:17.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:17.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:17.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:17.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:17.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:17.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:17.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847155+0000", 2024-09-17T21:20:17.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.544e-06 2024-09-17T21:20:17.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:17.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847165+0000", 2024-09-17T21:20:17.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.043e-05 2024-09-17T21:20:17.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:17.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847189+0000", 2024-09-17T21:20:17.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3659000000000002e-05 2024-09-17T21:20:17.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847210+0000", 2024-09-17T21:20:17.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1163e-05 2024-09-17T21:20:17.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:17.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848856+0000", 2024-09-17T21:20:17.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001645521 2024-09-17T21:20:17.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.123 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:17.123 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848905+0000", 2024-09-17T21:20:17.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9048999999999997e-05 2024-09-17T21:20:17.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.123 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:17.123 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854770+0000", 2024-09-17T21:20:17.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058652879999999998 2024-09-17T21:20:17.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:17.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:17.123 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:17.124 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854880+0000", 2024-09-17T21:20:17.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010986500000000001 2024-09-17T21:20:17.124 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:17.124 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:17.124 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:17.124 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:17.124 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:17.124 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:17.126 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:20:17.126 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:17.118+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:17.227 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:17.225+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:17.259 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:17.259 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:17.259 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:20:17.259 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:17.273 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:20:17.327 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:17.321+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:17.357 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:20:17.410 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:17.410 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:17.410 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:17.410 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:17.410 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:17.425 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:20:17.428 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:17.422+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:17.528 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:17.525+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:17.566 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:17.566 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:17.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:17.566 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:17.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.566 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:20:17.566 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:20:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 140.54801398399999, 2024-09-17T21:20:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:20:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:20:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:20:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:20:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:20:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:20:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:20:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:20:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:20:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:20:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:20:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:20:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:20:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:20:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:20:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:20:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:20:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:20:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:20:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:20:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:20:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:20:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:20:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:20:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:20:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:20:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:20:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:20:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:20:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:20:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:20:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:20:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:20:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:20:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:20:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 120.967849704, 2024-09-17T21:20:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:20:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:20:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:20:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:20:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:20:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:20:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:20:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:20:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:20:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:20:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:20:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:20:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:20:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:20:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:20:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:20:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:20:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:20:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:20:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:20:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:20:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:20:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:20:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:20:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:20:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:20:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:20:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:20:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:20:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:20:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:20:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 120.96775239199999, 2024-09-17T21:20:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:20:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:20:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:20:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:20:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:20:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:20:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:20:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:20:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:20:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:20:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:20:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:20:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:20:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:20:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:20:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:20:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:20:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:20:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:20:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:20:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:20:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:20:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:20:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:20:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:20:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:20:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:20:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:20:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:20:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:20:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:20:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:20:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:20:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:20:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 120.965665591, 2024-09-17T21:20:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:20:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:20:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:20:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:20:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:20:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:20:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:20:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:20:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:20:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:20:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:20:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:20:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:20:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:20:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:20:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:20:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:20:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:20:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:20:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:20:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:20:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:20:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:20:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:20:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:20:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:20:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:20:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:20:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:20:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:20:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:20:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:20:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:20:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:20:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:20:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:20:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:20:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:20:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 88.281766809999993, 2024-09-17T21:20:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:20:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:20:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:20:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:20:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:20:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:20:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:20:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:20:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:20:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:20:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:20:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:20:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:20:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:20:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:20:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:20:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:20:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:20:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:20:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:20:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:20:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:20:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:20:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:20:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:20:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:20:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:20:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:20:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:20:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:20:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 88.272675098999997, 2024-09-17T21:20:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:20:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:20:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:20:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:20:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:20:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:20:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:20:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:20:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:20:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:20:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:20:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:20:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:20:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:20:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:20:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:20:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:20:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:20:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:20:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:20:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:20:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:20:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:20:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:20:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:20:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 88.272503048000004, 2024-09-17T21:20:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:20:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:20:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:20:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:20:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:20:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:20:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:20:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:20:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:20:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:20:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:20:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:20:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:20:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:20:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:20:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:20:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:20:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:20:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:20:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:20:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:20:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:20:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:20:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:20:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:20:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 88.097570301000005, 2024-09-17T21:20:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:20:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:20:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:20:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:20:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:20:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:20:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:20:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:20:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:20:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:20:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:20:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:20:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:20:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:20:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:20:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:20:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:20:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:20:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:20:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:20:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:20:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:20:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 88.097485743999997, 2024-09-17T21:20:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:20:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:20:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:20:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:20:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:20:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:20:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:20:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:20:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:20:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:20:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:20:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:20:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:20:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:20:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:20:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:20:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:20:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:20:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:20:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:20:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 88.097420107000005, 2024-09-17T21:20:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:20:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:20:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:20:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:20:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:20:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:20:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:20:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:20:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:20:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:20:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:20:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:20:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:20:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:20:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:20:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:20:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:20:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:20:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:20:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:20:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:20:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:20:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:20:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.723845009999998, 2024-09-17T21:20:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:20:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:20:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:20:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:20:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:20:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:20:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:20:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:20:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:20:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:20:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:20:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:20:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:20:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:20:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:20:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:20:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:20:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:20:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:20:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.723725498, 2024-09-17T21:20:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:20:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:20:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:20:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:20:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:20:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:20:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:20:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:20:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:20:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:20:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:20:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:20:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:20:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:20:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:20:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:20:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:20:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:20:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2695 3.15 3:aedfd032:::rbd_data.1152a8ae2019.0000000000000041:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.843199+0000", 2024-09-17T21:20:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.718580193999998, 2024-09-17T21:20:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079730569000000001, 2024-09-17T21:20:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2695 2024-09-17T21:20:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843199+0000", 2024-09-17T21:20:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843199+0000", 2024-09-17T21:20:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843201+0000", 2024-09-17T21:20:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0899999999999999e-06 2024-09-17T21:20:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843214+0000", 2024-09-17T21:20:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2233e-05 2024-09-17T21:20:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843245+0000", 2024-09-17T21:20:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1464000000000002e-05 2024-09-17T21:20:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843266+0000", 2024-09-17T21:20:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0879000000000001e-05 2024-09-17T21:20:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847920+0000", 2024-09-17T21:20:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.004654292 2024-09-17T21:20:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.848240+0000", 2024-09-17T21:20:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031994499999999997 2024-09-17T21:20:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922930+0000", 2024-09-17T21:20:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.074689666000000002 2024-09-17T21:20:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2732 3.15 3:ab9a4731:::rbd_data.1152a8ae2019.0000000000000058:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.846909+0000", 2024-09-17T21:20:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.714870148000003, 2024-09-17T21:20:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076145476000000004, 2024-09-17T21:20:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2732 2024-09-17T21:20:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846909+0000", 2024-09-17T21:20:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846909+0000", 2024-09-17T21:20:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846911+0000", 2024-09-17T21:20:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.31e-06 2024-09-17T21:20:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846916+0000", 2024-09-17T21:20:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.023e-06 2024-09-17T21:20:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846931+0000", 2024-09-17T21:20:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5549999999999999e-05 2024-09-17T21:20:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.846943+0000", 2024-09-17T21:20:17.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1783000000000001e-05 2024-09-17T21:20:17.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.850592+0000", 2024-09-17T21:20:17.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036486159999999999 2024-09-17T21:20:17.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:17.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.850870+0000", 2024-09-17T21:20:17.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027864200000000001 2024-09-17T21:20:17.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.923055+0000", 2024-09-17T21:20:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.072184551999999999 2024-09-17T21:20:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2737 3.15 3:a8c4e696:::rbd_data.1152a8ae2019.000000000000005b:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.847014+0000", 2024-09-17T21:20:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.714765012999997, 2024-09-17T21:20:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076158271, 2024-09-17T21:20:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2737 2024-09-17T21:20:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847014+0000", 2024-09-17T21:20:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847014+0000", 2024-09-17T21:20:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847015+0000", 2024-09-17T21:20:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0699999999999999e-06 2024-09-17T21:20:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847021+0000", 2024-09-17T21:20:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.446e-06 2024-09-17T21:20:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847038+0000", 2024-09-17T21:20:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7257000000000001e-05 2024-09-17T21:20:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.847051+0000", 2024-09-17T21:20:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2649999999999999e-05 2024-09-17T21:20:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.852127+0000", 2024-09-17T21:20:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050757420000000003 2024-09-17T21:20:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.852423+0000", 2024-09-17T21:20:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029608100000000001 2024-09-17T21:20:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.923173+0000", 2024-09-17T21:20:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.070750024999999994 2024-09-17T21:20:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3397 3.8 3:174f9d43:::rbd_data.1152a8ae2019.000000000000006e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.897797+0000", 2024-09-17T21:20:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.663982068000003, 2024-09-17T21:20:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045478192000000001, 2024-09-17T21:20:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3397 2024-09-17T21:20:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897797+0000", 2024-09-17T21:20:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897797+0000", 2024-09-17T21:20:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897799+0000", 2024-09-17T21:20:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.238e-06 2024-09-17T21:20:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897806+0000", 2024-09-17T21:20:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5900000000000002e-06 2024-09-17T21:20:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897822+0000", 2024-09-17T21:20:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5659e-05 2024-09-17T21:20:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.897832+0000", 2024-09-17T21:20:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0034000000000001e-05 2024-09-17T21:20:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.931249+0000", 2024-09-17T21:20:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033416700000000001 2024-09-17T21:20:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.931425+0000", 2024-09-17T21:20:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017629500000000001 2024-09-17T21:20:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943276+0000", 2024-09-17T21:20:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011850675999999999 2024-09-17T21:20:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3524 3.8 3:119ba024:::rbd_data.1152c6ce51b2.0000000000000005:17 [read 1261568~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.901789+0000", 2024-09-17T21:20:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.659990141000002, 2024-09-17T21:20:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.042862082000000003, 2024-09-17T21:20:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3524 2024-09-17T21:20:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901789+0000", 2024-09-17T21:20:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901789+0000", 2024-09-17T21:20:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901790+0000", 2024-09-17T21:20:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0400000000000005e-07 2024-09-17T21:20:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901795+0000", 2024-09-17T21:20:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3839999999999999e-06 2024-09-17T21:20:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901807+0000", 2024-09-17T21:20:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2532e-05 2024-09-17T21:20:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.901816+0000", 2024-09-17T21:20:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2220000000000008e-06 2024-09-17T21:20:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.944407+0000", 2024-09-17T21:20:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.042591088999999999 2024-09-17T21:20:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.944498+0000", 2024-09-17T21:20:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0883000000000004e-05 2024-09-17T21:20:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.944651+0000", 2024-09-17T21:20:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015306799999999999 2024-09-17T21:20:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3679 3.8 3:129b2dec:::rbd_data.1152a8ae2019.0000000000000012:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.908354+0000", 2024-09-17T21:20:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.653425779999999, 2024-09-17T21:20:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046195880000000002, 2024-09-17T21:20:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3679 2024-09-17T21:20:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908354+0000", 2024-09-17T21:20:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908354+0000", 2024-09-17T21:20:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908355+0000", 2024-09-17T21:20:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9999999999999995e-07 2024-09-17T21:20:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908361+0000", 2024-09-17T21:20:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7399999999999998e-06 2024-09-17T21:20:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908375+0000", 2024-09-17T21:20:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.363e-05 2024-09-17T21:20:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908386+0000", 2024-09-17T21:20:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.06e-05 2024-09-17T21:20:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.945600+0000", 2024-09-17T21:20:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.037214199000000003 2024-09-17T21:20:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.945650+0000", 2024-09-17T21:20:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9737999999999997e-05 2024-09-17T21:20:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.945799+0000", 2024-09-17T21:20:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014921699999999999 2024-09-17T21:20:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954371+0000", 2024-09-17T21:20:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085719730000000001 2024-09-17T21:20:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954460+0000", 2024-09-17T21:20:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9596000000000001e-05 2024-09-17T21:20:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954511+0000", 2024-09-17T21:20:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0455999999999999e-05 2024-09-17T21:20:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954550+0000", 2024-09-17T21:20:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8730999999999999e-05 2024-09-17T21:20:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3703 3.13 3:cdb4d88a:::rbd_data.1152a8ae2019.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.908402+0000", 2024-09-17T21:20:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.653377740000003, 2024-09-17T21:20:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046393446999999997, 2024-09-17T21:20:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3703 2024-09-17T21:20:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908402+0000", 2024-09-17T21:20:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908402+0000", 2024-09-17T21:20:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908403+0000", 2024-09-17T21:20:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0529999999999999e-06 2024-09-17T21:20:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908409+0000", 2024-09-17T21:20:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2219999999999999e-06 2024-09-17T21:20:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908422+0000", 2024-09-17T21:20:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3147e-05 2024-09-17T21:20:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908432+0000", 2024-09-17T21:20:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.039e-05 2024-09-17T21:20:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.942217+0000", 2024-09-17T21:20:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033784994999999998 2024-09-17T21:20:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.942296+0000", 2024-09-17T21:20:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8218000000000002e-05 2024-09-17T21:20:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943003+0000", 2024-09-17T21:20:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000707403 2024-09-17T21:20:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954635+0000", 2024-09-17T21:20:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011631786999999999 2024-09-17T21:20:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954706+0000", 2024-09-17T21:20:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1064999999999998e-05 2024-09-17T21:20:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954757+0000", 2024-09-17T21:20:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0760000000000002e-05 2024-09-17T21:20:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954795+0000", 2024-09-17T21:20:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8407e-05 2024-09-17T21:20:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:3711 3.13 3:ca5a0e5e:::rbd_data.1152a8ae2019.0000000000000040:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.908514+0000", 2024-09-17T21:20:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.653265576999999, 2024-09-17T21:20:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046370871000000001, 2024-09-17T21:20:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3711 2024-09-17T21:20:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908514+0000", 2024-09-17T21:20:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908514+0000", 2024-09-17T21:20:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908515+0000", 2024-09-17T21:20:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0449999999999999e-06 2024-09-17T21:20:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908521+0000", 2024-09-17T21:20:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2179999999999996e-06 2024-09-17T21:20:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908535+0000", 2024-09-17T21:20:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4124999999999999e-05 2024-09-17T21:20:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.908548+0000", 2024-09-17T21:20:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2680000000000001e-05 2024-09-17T21:20:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943530+0000", 2024-09-17T21:20:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034981655 2024-09-17T21:20:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943636+0000", 2024-09-17T21:20:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106086 2024-09-17T21:20:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.943837+0000", 2024-09-17T21:20:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201826 2024-09-17T21:20:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954688+0000", 2024-09-17T21:20:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010850963 2024-09-17T21:20:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954805+0000", 2024-09-17T21:20:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000117025 2024-09-17T21:20:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954851+0000", 2024-09-17T21:20:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5065000000000003e-05 2024-09-17T21:20:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.954885+0000", 2024-09-17T21:20:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4183000000000001e-05 2024-09-17T21:20:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:17.673 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:17.675 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:20:17.675 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:17.669+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:17.776 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:17.770+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:17.819 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:17.819 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:17.819 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:20:17.819 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:17.834 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:20:17.879 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:17.874+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:17.977 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:17.973+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:18.015 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:18.015 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:18.015 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:18.015 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:18.015 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:18.032 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:20:18.079 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:18.074+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:18.112 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-17T21:20:18.142 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:20:18.163 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:18.163 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:18.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5541.0:2643 3.1a 3:5ab2b679:::rbd_data.115266d0b9f2.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 917504~4608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:20:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.510099+0000", 2024-09-17T21:20:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.65045933, 2024-09-17T21:20:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.053163717999999999, 2024-09-17T21:20:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5541", 2024-09-17T21:20:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:20:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2643 2024-09-17T21:20:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.510099+0000", 2024-09-17T21:20:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.510099+0000", 2024-09-17T21:20:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.510100+0000", 2024-09-17T21:20:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.668e-06 2024-09-17T21:20:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527298+0000", 2024-09-17T21:20:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017198108 2024-09-17T21:20:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527333+0000", 2024-09-17T21:20:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4304999999999998e-05 2024-09-17T21:20:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527414+0000", 2024-09-17T21:20:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1009999999999999e-05 2024-09-17T21:20:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527529+0000", 2024-09-17T21:20:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000115122 2024-09-17T21:20:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527653+0000", 2024-09-17T21:20:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012405900000000001 2024-09-17T21:20:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:20:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527950+0000", 2024-09-17T21:20:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029670599999999999 2024-09-17T21:20:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.553334+0000", 2024-09-17T21:20:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025384750000000001 2024-09-17T21:20:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.563128+0000", 2024-09-17T21:20:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0097937179999999999 2024-09-17T21:20:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.563201+0000", 2024-09-17T21:20:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3128999999999997e-05 2024-09-17T21:20:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.563262+0000", 2024-09-17T21:20:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1142999999999994e-05 2024-09-17T21:20:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5541.0:3004 3.a 3:524d69f6:::rbd_data.115266d0b9f2.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T21:20:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:50.606531+0000", 2024-09-17T21:20:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.554027378000001, 2024-09-17T21:20:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049461896999999998, 2024-09-17T21:20:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5541", 2024-09-17T21:20:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:20:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3004 2024-09-17T21:20:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.606531+0000", 2024-09-17T21:20:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.606531+0000", 2024-09-17T21:20:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.606533+0000", 2024-09-17T21:20:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.03e-06 2024-09-17T21:20:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617600+0000", 2024-09-17T21:20:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011067169999999999 2024-09-17T21:20:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617620+0000", 2024-09-17T21:20:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0627000000000001e-05 2024-09-17T21:20:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617631+0000", 2024-09-17T21:20:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0991e-05 2024-09-17T21:20:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617713+0000", 2024-09-17T21:20:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1241000000000004e-05 2024-09-17T21:20:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617903+0000", 2024-09-17T21:20:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019016599999999999 2024-09-17T21:20:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.618842+0000", 2024-09-17T21:20:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00093906399999999998 2024-09-17T21:20:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.654887+0000", 2024-09-17T21:20:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.036045176999999998 2024-09-17T21:20:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.655762+0000", 2024-09-17T21:20:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00087534500000000003 2024-09-17T21:20:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.655900+0000", 2024-09-17T21:20:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013781700000000001 2024-09-17T21:20:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.655992+0000", 2024-09-17T21:20:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2268999999999994e-05 2024-09-17T21:20:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1489 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.014521+0000", 2024-09-17T21:20:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.146036624000001, 2024-09-17T21:20:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047072140999999998, 2024-09-17T21:20:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1489 2024-09-17T21:20:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014521+0000", 2024-09-17T21:20:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014521+0000", 2024-09-17T21:20:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014522+0000", 2024-09-17T21:20:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8199999999999999e-07 2024-09-17T21:20:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014525+0000", 2024-09-17T21:20:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2799999999999999e-06 2024-09-17T21:20:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014536+0000", 2024-09-17T21:20:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0463e-05 2024-09-17T21:20:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014544+0000", 2024-09-17T21:20:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:20:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.061185+0000", 2024-09-17T21:20:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046641021999999997 2024-09-17T21:20:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.061263+0000", 2024-09-17T21:20:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7973999999999996e-05 2024-09-17T21:20:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.061593+0000", 2024-09-17T21:20:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033020199999999998 2024-09-17T21:20:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.180 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1522 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:18.180 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.018637+0000", 2024-09-17T21:20:18.180 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.141921098000001, 2024-09-17T21:20:18.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047019501999999998, 2024-09-17T21:20:18.180 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:18.180 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:18.180 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:18.180 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:18.180 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:18.180 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:18.180 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1522 2024-09-17T21:20:18.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.181 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:18.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:18.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018637+0000", 2024-09-17T21:20:18.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:18.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018637+0000", 2024-09-17T21:20:18.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018638+0000", 2024-09-17T21:20:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3570000000000001e-06 2024-09-17T21:20:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018643+0000", 2024-09-17T21:20:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7400000000000004e-06 2024-09-17T21:20:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018657+0000", 2024-09-17T21:20:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3617e-05 2024-09-17T21:20:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018667+0000", 2024-09-17T21:20:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0392e-05 2024-09-17T21:20:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065370+0000", 2024-09-17T21:20:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046703100999999997 2024-09-17T21:20:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065427+0000", 2024-09-17T21:20:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6920999999999999e-05 2024-09-17T21:20:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065656+0000", 2024-09-17T21:20:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022937399999999999 2024-09-17T21:20:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1523 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 704512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.018682+0000", 2024-09-17T21:20:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.1418757, 2024-09-17T21:20:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047547589000000001, 2024-09-17T21:20:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1523 2024-09-17T21:20:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018682+0000", 2024-09-17T21:20:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018682+0000", 2024-09-17T21:20:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018683+0000", 2024-09-17T21:20:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1400000000000001e-06 2024-09-17T21:20:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018688+0000", 2024-09-17T21:20:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4900000000000002e-06 2024-09-17T21:20:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018703+0000", 2024-09-17T21:20:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4837e-05 2024-09-17T21:20:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018714+0000", 2024-09-17T21:20:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0882e-05 2024-09-17T21:20:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065769+0000", 2024-09-17T21:20:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047055579 2024-09-17T21:20:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065910+0000", 2024-09-17T21:20:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014135100000000001 2024-09-17T21:20:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.066230+0000", 2024-09-17T21:20:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031931 2024-09-17T21:20:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1552 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022280+0000", 2024-09-17T21:20:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.138278159999999, 2024-09-17T21:20:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047789951999999997, 2024-09-17T21:20:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1552 2024-09-17T21:20:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022280+0000", 2024-09-17T21:20:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022280+0000", 2024-09-17T21:20:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022280+0000", 2024-09-17T21:20:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:20:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022283+0000", 2024-09-17T21:20:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4080000000000001e-06 2024-09-17T21:20:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022289+0000", 2024-09-17T21:20:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.477e-06 2024-09-17T21:20:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022295+0000", 2024-09-17T21:20:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5439999999999998e-06 2024-09-17T21:20:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.069004+0000", 2024-09-17T21:20:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046709222000000002 2024-09-17T21:20:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.069152+0000", 2024-09-17T21:20:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014832300000000001 2024-09-17T21:20:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.070070+0000", 2024-09-17T21:20:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00091743300000000005 2024-09-17T21:20:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1553 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022302+0000", 2024-09-17T21:20:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.138255773000001, 2024-09-17T21:20:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048707517999999998, 2024-09-17T21:20:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1553 2024-09-17T21:20:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022302+0000", 2024-09-17T21:20:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022302+0000", 2024-09-17T21:20:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022303+0000", 2024-09-17T21:20:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2300000000000001e-07 2024-09-17T21:20:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022305+0000", 2024-09-17T21:20:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4700000000000001e-06 2024-09-17T21:20:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022312+0000", 2024-09-17T21:20:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.494e-06 2024-09-17T21:20:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022320+0000", 2024-09-17T21:20:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1969999999999996e-06 2024-09-17T21:20:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.070138+0000", 2024-09-17T21:20:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047818685 2024-09-17T21:20:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.070323+0000", 2024-09-17T21:20:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000184186 2024-09-17T21:20:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071010+0000", 2024-09-17T21:20:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00068696299999999996 2024-09-17T21:20:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1554 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022328+0000", 2024-09-17T21:20:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.138229931000001, 2024-09-17T21:20:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049212936999999998, 2024-09-17T21:20:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1554 2024-09-17T21:20:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022328+0000", 2024-09-17T21:20:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022328+0000", 2024-09-17T21:20:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022328+0000", 2024-09-17T21:20:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3000000000000001e-07 2024-09-17T21:20:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022331+0000", 2024-09-17T21:20:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3539999999999998e-06 2024-09-17T21:20:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022338+0000", 2024-09-17T21:20:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8800000000000002e-06 2024-09-17T21:20:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022343+0000", 2024-09-17T21:20:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6149999999999996e-06 2024-09-17T21:20:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071051+0000", 2024-09-17T21:20:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048707179000000003 2024-09-17T21:20:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071171+0000", 2024-09-17T21:20:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000120036 2024-09-17T21:20:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071541+0000", 2024-09-17T21:20:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037034299999999997 2024-09-17T21:20:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1555 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022352+0000", 2024-09-17T21:20:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.138205764999999, 2024-09-17T21:20:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049642617, 2024-09-17T21:20:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1555 2024-09-17T21:20:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022352+0000", 2024-09-17T21:20:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022352+0000", 2024-09-17T21:20:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022353+0000", 2024-09-17T21:20:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7000000000000005e-07 2024-09-17T21:20:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022355+0000", 2024-09-17T21:20:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4569999999999999e-06 2024-09-17T21:20:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022364+0000", 2024-09-17T21:20:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2550000000000003e-06 2024-09-17T21:20:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022374+0000", 2024-09-17T21:20:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2820000000000002e-06 2024-09-17T21:20:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071626+0000", 2024-09-17T21:20:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049251892999999998 2024-09-17T21:20:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071703+0000", 2024-09-17T21:20:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7273000000000004e-05 2024-09-17T21:20:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071995+0000", 2024-09-17T21:20:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029188700000000001 2024-09-17T21:20:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1556 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 663552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022388+0000", 2024-09-17T21:20:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.138170220999999, 2024-09-17T21:20:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050001666, 2024-09-17T21:20:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1556 2024-09-17T21:20:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022388+0000", 2024-09-17T21:20:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022388+0000", 2024-09-17T21:20:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022389+0000", 2024-09-17T21:20:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.282e-06 2024-09-17T21:20:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022394+0000", 2024-09-17T21:20:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6779999999999999e-06 2024-09-17T21:20:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022407+0000", 2024-09-17T21:20:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3309000000000001e-05 2024-09-17T21:20:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022417+0000", 2024-09-17T21:20:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0152e-05 2024-09-17T21:20:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072033+0000", 2024-09-17T21:20:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049616014999999999 2024-09-17T21:20:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072137+0000", 2024-09-17T21:20:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104214 2024-09-17T21:20:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072389+0000", 2024-09-17T21:20:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025201599999999998 2024-09-17T21:20:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1557 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022434+0000", 2024-09-17T21:20:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.138124168000001, 2024-09-17T21:20:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050377024999999999, 2024-09-17T21:20:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1557 2024-09-17T21:20:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022434+0000", 2024-09-17T21:20:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022434+0000", 2024-09-17T21:20:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022435+0000", 2024-09-17T21:20:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9700000000000005e-07 2024-09-17T21:20:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022439+0000", 2024-09-17T21:20:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1899999999999997e-06 2024-09-17T21:20:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022452+0000", 2024-09-17T21:20:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2865e-05 2024-09-17T21:20:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022462+0000", 2024-09-17T21:20:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.91e-06 2024-09-17T21:20:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072452+0000", 2024-09-17T21:20:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049990290999999999 2024-09-17T21:20:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072541+0000", 2024-09-17T21:20:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9297999999999998e-05 2024-09-17T21:20:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072811+0000", 2024-09-17T21:20:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026957400000000002 2024-09-17T21:20:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1584 3.e 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:1b [read 753664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.026146+0000", 2024-09-17T21:20:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.134411976999999, 2024-09-17T21:20:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049235534999999997, 2024-09-17T21:20:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1584 2024-09-17T21:20:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026146+0000", 2024-09-17T21:20:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026146+0000", 2024-09-17T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026147+0000", 2024-09-17T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0249999999999999e-06 2024-09-17T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026151+0000", 2024-09-17T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026169+0000", 2024-09-17T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7294000000000001e-05 2024-09-17T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026179+0000", 2024-09-17T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0084999999999999e-05 2024-09-17T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072874+0000", 2024-09-17T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046695497000000002 2024-09-17T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072976+0000", 2024-09-17T21:20:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101358 2024-09-17T21:20:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.075381+0000", 2024-09-17T21:20:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002405726 2024-09-17T21:20:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1585 3.e 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:1b [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.026195+0000", 2024-09-17T21:20:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.134362848999999, 2024-09-17T21:20:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051427650999999998, 2024-09-17T21:20:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1585 2024-09-17T21:20:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026195+0000", 2024-09-17T21:20:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026195+0000", 2024-09-17T21:20:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026196+0000", 2024-09-17T21:20:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7300000000000004e-07 2024-09-17T21:20:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026200+0000", 2024-09-17T21:20:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.425e-06 2024-09-17T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026214+0000", 2024-09-17T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3108999999999999e-05 2024-09-17T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026224+0000", 2024-09-17T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.042e-05 2024-09-17T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.075431+0000", 2024-09-17T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049206543999999998 2024-09-17T21:20:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.075489+0000", 2024-09-17T21:20:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8053000000000001e-05 2024-09-17T21:20:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.077623+0000", 2024-09-17T21:20:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002134127 2024-09-17T21:20:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1586 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.026238+0000", 2024-09-17T21:20:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.134320090999999, 2024-09-17T21:20:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051776152999999998, 2024-09-17T21:20:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1586 2024-09-17T21:20:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026238+0000", 2024-09-17T21:20:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026238+0000", 2024-09-17T21:20:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026239+0000", 2024-09-17T21:20:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0380000000000001e-06 2024-09-17T21:20:18.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:18.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026243+0000", 2024-09-17T21:20:18.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3499999999999999e-06 2024-09-17T21:20:18.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:18.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026259+0000", 2024-09-17T21:20:18.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5928999999999999e-05 2024-09-17T21:20:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026270+0000", 2024-09-17T21:20:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0702e-05 2024-09-17T21:20:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.077682+0000", 2024-09-17T21:20:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051411865000000001 2024-09-17T21:20:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.077751+0000", 2024-09-17T21:20:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9606999999999999e-05 2024-09-17T21:20:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078014+0000", 2024-09-17T21:20:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026266199999999999 2024-09-17T21:20:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1587 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 802816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.026286+0000", 2024-09-17T21:20:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.134272312, 2024-09-17T21:20:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.052384495000000003, 2024-09-17T21:20:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1587 2024-09-17T21:20:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026286+0000", 2024-09-17T21:20:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:18.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026286+0000", 2024-09-17T21:20:18.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:18.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026287+0000", 2024-09-17T21:20:18.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0049999999999999e-06 2024-09-17T21:20:18.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:18.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026291+0000", 2024-09-17T21:20:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3479999999999997e-06 2024-09-17T21:20:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026304+0000", 2024-09-17T21:20:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3288999999999999e-05 2024-09-17T21:20:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026314+0000", 2024-09-17T21:20:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9920000000000002e-06 2024-09-17T21:20:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078052+0000", 2024-09-17T21:20:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051737299 2024-09-17T21:20:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078118+0000", 2024-09-17T21:20:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6180999999999995e-05 2024-09-17T21:20:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078670+0000", 2024-09-17T21:20:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00055238100000000001 2024-09-17T21:20:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1618 3.e 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:1b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.031597+0000", 2024-09-17T21:20:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.128960871, 2024-09-17T21:20:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047545707999999999, 2024-09-17T21:20:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1618 2024-09-17T21:20:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031597+0000", 2024-09-17T21:20:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031597+0000", 2024-09-17T21:20:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031598+0000", 2024-09-17T21:20:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:20:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031602+0000", 2024-09-17T21:20:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:20:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031630+0000", 2024-09-17T21:20:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8058000000000001e-05 2024-09-17T21:20:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031643+0000", 2024-09-17T21:20:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2628e-05 2024-09-17T21:20:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078728+0000", 2024-09-17T21:20:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047085044 2024-09-17T21:20:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078830+0000", 2024-09-17T21:20:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102493 2024-09-17T21:20:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079143+0000", 2024-09-17T21:20:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031263599999999998 2024-09-17T21:20:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1619 3.e 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:1b [read 827392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.031661+0000", 2024-09-17T21:20:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.128897303999999, 2024-09-17T21:20:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047908566999999999, 2024-09-17T21:20:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1619 2024-09-17T21:20:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031661+0000", 2024-09-17T21:20:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031661+0000", 2024-09-17T21:20:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031662+0000", 2024-09-17T21:20:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.252e-06 2024-09-17T21:20:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031666+0000", 2024-09-17T21:20:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6299999999999997e-06 2024-09-17T21:20:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031683+0000", 2024-09-17T21:20:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6035000000000001e-05 2024-09-17T21:20:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031693+0000", 2024-09-17T21:20:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0492000000000001e-05 2024-09-17T21:20:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079186+0000", 2024-09-17T21:20:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047493441999999997 2024-09-17T21:20:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079315+0000", 2024-09-17T21:20:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012860000000000001 2024-09-17T21:20:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079569+0000", 2024-09-17T21:20:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025411599999999998 2024-09-17T21:20:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1620 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.031706+0000", 2024-09-17T21:20:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.128851879999999, 2024-09-17T21:20:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048561143000000001, 2024-09-17T21:20:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1620 2024-09-17T21:20:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031706+0000", 2024-09-17T21:20:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031706+0000", 2024-09-17T21:20:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031707+0000", 2024-09-17T21:20:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5799999999999998e-07 2024-09-17T21:20:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031711+0000", 2024-09-17T21:20:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1930000000000003e-06 2024-09-17T21:20:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031724+0000", 2024-09-17T21:20:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.24e-05 2024-09-17T21:20:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031733+0000", 2024-09-17T21:20:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6239999999999997e-06 2024-09-17T21:20:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079613+0000", 2024-09-17T21:20:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047880082999999997 2024-09-17T21:20:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079719+0000", 2024-09-17T21:20:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010559499999999999 2024-09-17T21:20:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.080267+0000", 2024-09-17T21:20:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00054829 2024-09-17T21:20:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1621 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.031746+0000", 2024-09-17T21:20:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.128811406000001, 2024-09-17T21:20:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048925189000000001, 2024-09-17T21:20:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1621 2024-09-17T21:20:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031746+0000", 2024-09-17T21:20:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031746+0000", 2024-09-17T21:20:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031747+0000", 2024-09-17T21:20:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3900000000000003e-07 2024-09-17T21:20:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031752+0000", 2024-09-17T21:20:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.172e-06 2024-09-17T21:20:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031767+0000", 2024-09-17T21:20:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5517000000000001e-05 2024-09-17T21:20:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031777+0000", 2024-09-17T21:20:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8579999999999995e-06 2024-09-17T21:20:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.080318+0000", 2024-09-17T21:20:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048541009000000003 2024-09-17T21:20:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.080415+0000", 2024-09-17T21:20:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7034999999999995e-05 2024-09-17T21:20:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.080672+0000", 2024-09-17T21:20:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025665900000000001 2024-09-17T21:20:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:18003 3.1d 3:b9765580:::rbd_header.11527d2b15f6:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T21:20:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:20:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.8026791800000002, 2024-09-17T21:20:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94165346900000002, 2024-09-17T21:20:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18003 2024-09-17T21:20:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:20:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:20:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357880+0000", 2024-09-17T21:20:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:20:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357891+0000", 2024-09-17T21:20:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0417999999999999e-05 2024-09-17T21:20:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357913+0000", 2024-09-17T21:20:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2215e-05 2024-09-17T21:20:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357922+0000", 2024-09-17T21:20:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4779999999999997e-06 2024-09-17T21:20:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358023+0000", 2024-09-17T21:20:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101025 2024-09-17T21:20:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358040+0000", 2024-09-17T21:20:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6291999999999999e-05 2024-09-17T21:20:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298203+0000", 2024-09-17T21:20:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94016297500000001 2024-09-17T21:20:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298626+0000", 2024-09-17T21:20:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000422853 2024-09-17T21:20:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:18.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.299532+0000", 2024-09-17T21:20:18.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00090657300000000004 2024-09-17T21:20:18.255 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.255 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:18.255 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.255 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:18.255 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:18.255 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:18.256 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:20:18.257 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:18.249+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:18.284 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-17T21:20:18.315 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:18.316 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:18.335 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:18.336 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:18.357 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:18.350+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:18.357 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:18.357 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:18.377 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:18.377 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:18.399 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:18.399 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:18.421 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:18.421 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:18.428 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:18.428 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:18.442 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:20:18.491 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:18.486+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:18.559 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:18.557+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:18.615 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:18.615 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:18.629 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:20:18.675 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:18.670+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:18.752 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:18.752 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:18.761 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:18.757+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:18.765 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:20:18.863 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:18.858+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:18.899 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:18.899 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:18.913 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:20:18.963 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:18.957+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:19.034 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:19.034 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:19.048 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:20:19.063 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:19.058+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:19.165 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:19.158+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:19.213 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:19.214 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:19.228 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:20:19.266 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:19.258+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:19.350 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:19.351 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:19.365 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:20:19.367 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:19.362+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:19.450 INFO:teuthology.orchestra.run.smithi184.stdout:4788 write 0xadc3600 thru 0xadc4570 (0xf71 bytes) 2024-09-17T21:20:19.452 INFO:teuthology.orchestra.run.smithi184.stdout:4790 read 0x3787800 thru 0x37890b0 (0x18b1 bytes) 2024-09-17T21:20:19.467 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:19.462+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:19.489 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:19.490 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:19.505 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:20:19.568 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:19.561+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:19.636 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:19.636 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:19.651 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:20:19.669 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:19.662+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:19.769 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:19.762+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:19.815 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:19.815 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:19.830 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:20:19.870 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:19.866+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:19.955 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:19.955 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:19.969 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:20:19.971 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:19.962+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:20.071 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:20.066+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:20.092 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:20.092 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:20.108 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:20:20.172 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:20.165+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:20.240 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:20.240 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:20.255 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:20:20.299 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:20.294+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:20.312 INFO:teuthology.orchestra.run.smithi184.stdout:4792 read 0x4935400 thru 0x493ed40 (0x9941 bytes) 2024-09-17T21:20:20.316 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:20:20.325 INFO:teuthology.orchestra.run.smithi184.stdout:4794 punch from 0xc68fa00 to 0xc694c00, (0x5200 bytes) 2024-09-17T21:20:20.374 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:20.369+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:20.389 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:20.390 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:20.404 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:20:20.429 INFO:teuthology.orchestra.run.smithi184.stdout:4795 trunc from 0xd082800 to 0x85fd200 2024-09-17T21:20:20.438 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:20:20.477 INFO:teuthology.orchestra.run.smithi184.stdout:4796 write 0x45c1400 thru 0x45d0b35 (0xf736 bytes) 2024-09-17T21:20:20.485 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:20.481+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:20.508 INFO:teuthology.orchestra.run.smithi184.stdout:4798 trunc from 0x85fd200 to 0xe631600 2024-09-17T21:20:20.576 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:20.569+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:20.580 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:20.581 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:20.582 INFO:teuthology.orchestra.run.smithi184.stdout:4799 write 0xae34000 thru 0xae34dec (0xded bytes) 2024-09-17T21:20:20.584 INFO:teuthology.orchestra.run.smithi184.stdout:4800 punch from 0x6b37a00 to 0x6b44400, (0xca00 bytes) 2024-09-17T21:20:20.599 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:20:20.600 INFO:teuthology.orchestra.run.smithi184.stdout:4801 read 0xa44be00 thru 0xa457df1 (0xbff2 bytes) 2024-09-17T21:20:20.601 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:20.601 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:20.602 INFO:teuthology.orchestra.run.smithi184.stdout:4802 read 0x3e0a600 thru 0x3e183d8 (0xddd9 bytes) 2024-09-17T21:20:20.610 INFO:teuthology.orchestra.run.smithi184.stdout:4804 write 0xa4c3a00 thru 0xa4ce2d7 (0xa8d8 bytes) 2024-09-17T21:20:20.611 INFO:teuthology.orchestra.run.smithi184.stdout:4806 punch from 0x8e34800 to 0x8e3dc00, (0x9400 bytes) 2024-09-17T21:20:20.632 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:20.638 INFO:teuthology.orchestra.run.smithi184.stdout:4810 trunc from 0xe631600 to 0x64f6600 2024-09-17T21:20:20.662 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:20.663 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:20.677 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:20.670+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:20.682 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:20.682 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:20.707 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:20.708 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:20.726 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:20.727 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:20.733 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 0] out_osds: [2, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:20:20.733 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:20:20.733 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-17T21:20:20.734 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-17T21:20:20.751 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:20.751 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:20.751 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:20:20.751 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:20.769 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:20:20.815 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:20.810+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:20.850 INFO:teuthology.orchestra.run.smithi184.stdout:4812 write 0xdae3800 thru 0xdae98af (0x60b0 bytes) 2024-09-17T21:20:20.878 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:20.873+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-17T21:20:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-17T21:20:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-17T21:20:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-17T21:20:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-17T21:20:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-17T21:20:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:bench [] [] [] [] 2024-09-17T21:20:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs debug_inject_read_zeros 2024-09-17T21:20:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs files list 2024-09-17T21:20:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs stats 2024-09-17T21:20:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator dump block 2024-09-17T21:20:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:Error EINVAL: invalid command 2024-09-17T21:20:20.891 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T21:20:20.891 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T21:20:20.891 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-17T21:20:20.900 INFO:teuthology.orchestra.run.smithi184.stdout:4815 trunc from 0xdae9a00 to 0x4107600 2024-09-17T21:20:20.960 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:20.960 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:20.960 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:20.960 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:20.961 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:20.979 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:20:20.979 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:20.974+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:21.080 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:21.074+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:21.101 INFO:teuthology.orchestra.run.smithi184.stdout:4818 write 0xd4d8600 thru 0xd4df445 (0x6e46 bytes) 2024-09-17T21:20:21.101 INFO:teuthology.orchestra.run.smithi184.stdout:4821 write 0x72c7200 thru 0x72c8b86 (0x1987 bytes) 2024-09-17T21:20:21.103 INFO:teuthology.orchestra.run.smithi184.stdout:4822 trunc from 0xd4df600 to 0xdd96c00 2024-09-17T21:20:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-17T21:20:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-17T21:20:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-17T21:20:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-17T21:20:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-17T21:20:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-17T21:20:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:bench [] [] [] [] 2024-09-17T21:20:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs debug_inject_read_zeros 2024-09-17T21:20:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs files list 2024-09-17T21:20:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs stats 2024-09-17T21:20:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator dump block 2024-09-17T21:20:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:Error EINVAL: invalid command 2024-09-17T21:20:21.113 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T21:20:21.114 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T21:20:21.114 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-17T21:20:21.143 INFO:teuthology.orchestra.run.smithi184.stdout:4823 read 0x4d71600 thru 0x4d7d29e (0xbc9f bytes) 2024-09-17T21:20:21.145 INFO:teuthology.orchestra.run.smithi184.stdout:4825 trunc from 0xdd96c00 to 0xda16400 2024-09-17T21:20:21.164 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:21.164 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:21.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:21.164 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:21.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.164 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5541.0:2643 3.1a 3:5ab2b679:::rbd_data.115266d0b9f2.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 917504~4608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:20:21.165 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.510099+0000", 2024-09-17T21:20:21.165 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.650906528, 2024-09-17T21:20:21.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.053163717999999999, 2024-09-17T21:20:21.165 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.165 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.165 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:21.165 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.165 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5541", 2024-09-17T21:20:21.165 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:20:21.165 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2643 2024-09-17T21:20:21.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.165 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.510099+0000", 2024-09-17T21:20:21.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.510099+0000", 2024-09-17T21:20:21.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.510100+0000", 2024-09-17T21:20:21.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.668e-06 2024-09-17T21:20:21.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527298+0000", 2024-09-17T21:20:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017198108 2024-09-17T21:20:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527333+0000", 2024-09-17T21:20:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4304999999999998e-05 2024-09-17T21:20:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527414+0000", 2024-09-17T21:20:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1009999999999999e-05 2024-09-17T21:20:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527529+0000", 2024-09-17T21:20:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000115122 2024-09-17T21:20:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527653+0000", 2024-09-17T21:20:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012405900000000001 2024-09-17T21:20:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:20:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.527950+0000", 2024-09-17T21:20:21.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029670599999999999 2024-09-17T21:20:21.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:21.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.553334+0000", 2024-09-17T21:20:21.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025384750000000001 2024-09-17T21:20:21.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:21.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.563128+0000", 2024-09-17T21:20:21.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0097937179999999999 2024-09-17T21:20:21.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.563201+0000", 2024-09-17T21:20:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3128999999999997e-05 2024-09-17T21:20:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.563262+0000", 2024-09-17T21:20:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1142999999999994e-05 2024-09-17T21:20:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5541.0:3004 3.a 3:524d69f6:::rbd_data.115266d0b9f2.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T21:20:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:50.606531+0000", 2024-09-17T21:20:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 30.554474576, 2024-09-17T21:20:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049461896999999998, 2024-09-17T21:20:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5541", 2024-09-17T21:20:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:20:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3004 2024-09-17T21:20:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.606531+0000", 2024-09-17T21:20:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.606531+0000", 2024-09-17T21:20:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.606533+0000", 2024-09-17T21:20:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.03e-06 2024-09-17T21:20:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617600+0000", 2024-09-17T21:20:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011067169999999999 2024-09-17T21:20:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617620+0000", 2024-09-17T21:20:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0627000000000001e-05 2024-09-17T21:20:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617631+0000", 2024-09-17T21:20:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0991e-05 2024-09-17T21:20:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617713+0000", 2024-09-17T21:20:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1241000000000004e-05 2024-09-17T21:20:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.617903+0000", 2024-09-17T21:20:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019016599999999999 2024-09-17T21:20:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.618842+0000", 2024-09-17T21:20:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00093906399999999998 2024-09-17T21:20:21.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:21.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.654887+0000", 2024-09-17T21:20:21.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.036045176999999998 2024-09-17T21:20:21.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:21.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.655762+0000", 2024-09-17T21:20:21.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00087534500000000003 2024-09-17T21:20:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.655900+0000", 2024-09-17T21:20:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013781700000000001 2024-09-17T21:20:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:50.655992+0000", 2024-09-17T21:20:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2268999999999994e-05 2024-09-17T21:20:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1489 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.014521+0000", 2024-09-17T21:20:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.146483822, 2024-09-17T21:20:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047072140999999998, 2024-09-17T21:20:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1489 2024-09-17T21:20:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014521+0000", 2024-09-17T21:20:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014521+0000", 2024-09-17T21:20:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014522+0000", 2024-09-17T21:20:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8199999999999999e-07 2024-09-17T21:20:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014525+0000", 2024-09-17T21:20:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2799999999999999e-06 2024-09-17T21:20:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014536+0000", 2024-09-17T21:20:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0463e-05 2024-09-17T21:20:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.014544+0000", 2024-09-17T21:20:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:20:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.061185+0000", 2024-09-17T21:20:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046641021999999997 2024-09-17T21:20:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.061263+0000", 2024-09-17T21:20:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7973999999999996e-05 2024-09-17T21:20:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.061593+0000", 2024-09-17T21:20:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033020199999999998 2024-09-17T21:20:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1522 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.018637+0000", 2024-09-17T21:20:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.142368296000001, 2024-09-17T21:20:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047019501999999998, 2024-09-17T21:20:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1522 2024-09-17T21:20:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018637+0000", 2024-09-17T21:20:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018637+0000", 2024-09-17T21:20:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018638+0000", 2024-09-17T21:20:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3570000000000001e-06 2024-09-17T21:20:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018643+0000", 2024-09-17T21:20:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7400000000000004e-06 2024-09-17T21:20:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018657+0000", 2024-09-17T21:20:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3617e-05 2024-09-17T21:20:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018667+0000", 2024-09-17T21:20:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0392e-05 2024-09-17T21:20:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065370+0000", 2024-09-17T21:20:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046703100999999997 2024-09-17T21:20:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065427+0000", 2024-09-17T21:20:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6920999999999999e-05 2024-09-17T21:20:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065656+0000", 2024-09-17T21:20:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022937399999999999 2024-09-17T21:20:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1523 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 704512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.018682+0000", 2024-09-17T21:20:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.142322898, 2024-09-17T21:20:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047547589000000001, 2024-09-17T21:20:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1523 2024-09-17T21:20:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018682+0000", 2024-09-17T21:20:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018682+0000", 2024-09-17T21:20:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018683+0000", 2024-09-17T21:20:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1400000000000001e-06 2024-09-17T21:20:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018688+0000", 2024-09-17T21:20:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4900000000000002e-06 2024-09-17T21:20:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018703+0000", 2024-09-17T21:20:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4837e-05 2024-09-17T21:20:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.018714+0000", 2024-09-17T21:20:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0882e-05 2024-09-17T21:20:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065769+0000", 2024-09-17T21:20:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047055579 2024-09-17T21:20:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.065910+0000", 2024-09-17T21:20:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014135100000000001 2024-09-17T21:20:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.066230+0000", 2024-09-17T21:20:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031931 2024-09-17T21:20:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1552 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022280+0000", 2024-09-17T21:20:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.138725357999999, 2024-09-17T21:20:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047789951999999997, 2024-09-17T21:20:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1552 2024-09-17T21:20:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022280+0000", 2024-09-17T21:20:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022280+0000", 2024-09-17T21:20:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022280+0000", 2024-09-17T21:20:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:20:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022283+0000", 2024-09-17T21:20:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4080000000000001e-06 2024-09-17T21:20:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022289+0000", 2024-09-17T21:20:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.477e-06 2024-09-17T21:20:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022295+0000", 2024-09-17T21:20:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5439999999999998e-06 2024-09-17T21:20:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.069004+0000", 2024-09-17T21:20:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046709222000000002 2024-09-17T21:20:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.069152+0000", 2024-09-17T21:20:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014832300000000001 2024-09-17T21:20:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.070070+0000", 2024-09-17T21:20:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00091743300000000005 2024-09-17T21:20:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1553 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022302+0000", 2024-09-17T21:20:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.138702971000001, 2024-09-17T21:20:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048707517999999998, 2024-09-17T21:20:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1553 2024-09-17T21:20:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022302+0000", 2024-09-17T21:20:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022302+0000", 2024-09-17T21:20:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022303+0000", 2024-09-17T21:20:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2300000000000001e-07 2024-09-17T21:20:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022305+0000", 2024-09-17T21:20:21.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4700000000000001e-06 2024-09-17T21:20:21.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022312+0000", 2024-09-17T21:20:21.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.494e-06 2024-09-17T21:20:21.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022320+0000", 2024-09-17T21:20:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1969999999999996e-06 2024-09-17T21:20:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.070138+0000", 2024-09-17T21:20:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047818685 2024-09-17T21:20:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.070323+0000", 2024-09-17T21:20:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000184186 2024-09-17T21:20:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071010+0000", 2024-09-17T21:20:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00068696299999999996 2024-09-17T21:20:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1554 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022328+0000", 2024-09-17T21:20:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.138677129000001, 2024-09-17T21:20:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049212936999999998, 2024-09-17T21:20:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1554 2024-09-17T21:20:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022328+0000", 2024-09-17T21:20:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022328+0000", 2024-09-17T21:20:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022328+0000", 2024-09-17T21:20:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3000000000000001e-07 2024-09-17T21:20:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022331+0000", 2024-09-17T21:20:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3539999999999998e-06 2024-09-17T21:20:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022338+0000", 2024-09-17T21:20:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8800000000000002e-06 2024-09-17T21:20:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022343+0000", 2024-09-17T21:20:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6149999999999996e-06 2024-09-17T21:20:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071051+0000", 2024-09-17T21:20:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048707179000000003 2024-09-17T21:20:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071171+0000", 2024-09-17T21:20:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000120036 2024-09-17T21:20:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071541+0000", 2024-09-17T21:20:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037034299999999997 2024-09-17T21:20:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1555 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022352+0000", 2024-09-17T21:20:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.138652962999998, 2024-09-17T21:20:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049642617, 2024-09-17T21:20:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1555 2024-09-17T21:20:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022352+0000", 2024-09-17T21:20:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022352+0000", 2024-09-17T21:20:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022353+0000", 2024-09-17T21:20:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7000000000000005e-07 2024-09-17T21:20:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022355+0000", 2024-09-17T21:20:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4569999999999999e-06 2024-09-17T21:20:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022364+0000", 2024-09-17T21:20:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2550000000000003e-06 2024-09-17T21:20:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022374+0000", 2024-09-17T21:20:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2820000000000002e-06 2024-09-17T21:20:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071626+0000", 2024-09-17T21:20:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049251892999999998 2024-09-17T21:20:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071703+0000", 2024-09-17T21:20:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7273000000000004e-05 2024-09-17T21:20:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.071995+0000", 2024-09-17T21:20:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029188700000000001 2024-09-17T21:20:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1556 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 663552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022388+0000", 2024-09-17T21:20:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.138617418999999, 2024-09-17T21:20:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050001666, 2024-09-17T21:20:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1556 2024-09-17T21:20:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022388+0000", 2024-09-17T21:20:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022388+0000", 2024-09-17T21:20:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022389+0000", 2024-09-17T21:20:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.282e-06 2024-09-17T21:20:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022394+0000", 2024-09-17T21:20:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6779999999999999e-06 2024-09-17T21:20:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022407+0000", 2024-09-17T21:20:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3309000000000001e-05 2024-09-17T21:20:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022417+0000", 2024-09-17T21:20:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0152e-05 2024-09-17T21:20:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072033+0000", 2024-09-17T21:20:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049616014999999999 2024-09-17T21:20:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072137+0000", 2024-09-17T21:20:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104214 2024-09-17T21:20:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072389+0000", 2024-09-17T21:20:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025201599999999998 2024-09-17T21:20:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1557 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.022434+0000", 2024-09-17T21:20:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.138571366000001, 2024-09-17T21:20:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050377024999999999, 2024-09-17T21:20:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1557 2024-09-17T21:20:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022434+0000", 2024-09-17T21:20:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022434+0000", 2024-09-17T21:20:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022435+0000", 2024-09-17T21:20:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9700000000000005e-07 2024-09-17T21:20:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022439+0000", 2024-09-17T21:20:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1899999999999997e-06 2024-09-17T21:20:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022452+0000", 2024-09-17T21:20:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2865e-05 2024-09-17T21:20:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.022462+0000", 2024-09-17T21:20:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.91e-06 2024-09-17T21:20:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072452+0000", 2024-09-17T21:20:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049990290999999999 2024-09-17T21:20:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072541+0000", 2024-09-17T21:20:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9297999999999998e-05 2024-09-17T21:20:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072811+0000", 2024-09-17T21:20:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026957400000000002 2024-09-17T21:20:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1584 3.e 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:1b [read 753664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.026146+0000", 2024-09-17T21:20:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.134859174999999, 2024-09-17T21:20:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049235534999999997, 2024-09-17T21:20:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1584 2024-09-17T21:20:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026146+0000", 2024-09-17T21:20:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026146+0000", 2024-09-17T21:20:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026147+0000", 2024-09-17T21:20:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0249999999999999e-06 2024-09-17T21:20:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026151+0000", 2024-09-17T21:20:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:20:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026169+0000", 2024-09-17T21:20:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7294000000000001e-05 2024-09-17T21:20:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026179+0000", 2024-09-17T21:20:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0084999999999999e-05 2024-09-17T21:20:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072874+0000", 2024-09-17T21:20:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046695497000000002 2024-09-17T21:20:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.072976+0000", 2024-09-17T21:20:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101358 2024-09-17T21:20:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.075381+0000", 2024-09-17T21:20:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002405726 2024-09-17T21:20:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1585 3.e 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:1b [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.026195+0000", 2024-09-17T21:20:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.134810046999998, 2024-09-17T21:20:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051427650999999998, 2024-09-17T21:20:21.222 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.222 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.222 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.222 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.222 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:21.222 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:21.222 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1585 2024-09-17T21:20:21.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.222 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026195+0000", 2024-09-17T21:20:21.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026195+0000", 2024-09-17T21:20:21.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026196+0000", 2024-09-17T21:20:21.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7300000000000004e-07 2024-09-17T21:20:21.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026200+0000", 2024-09-17T21:20:21.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.425e-06 2024-09-17T21:20:21.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026214+0000", 2024-09-17T21:20:21.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3108999999999999e-05 2024-09-17T21:20:21.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026224+0000", 2024-09-17T21:20:21.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.042e-05 2024-09-17T21:20:21.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.075431+0000", 2024-09-17T21:20:21.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049206543999999998 2024-09-17T21:20:21.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.075489+0000", 2024-09-17T21:20:21.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8053000000000001e-05 2024-09-17T21:20:21.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.077623+0000", 2024-09-17T21:20:21.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002134127 2024-09-17T21:20:21.225 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.225 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.225 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.226 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1586 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:21.226 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.026238+0000", 2024-09-17T21:20:21.226 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.134767288999999, 2024-09-17T21:20:21.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051776152999999998, 2024-09-17T21:20:21.226 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.226 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.226 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.226 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.226 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:21.226 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:21.226 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1586 2024-09-17T21:20:21.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.226 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026238+0000", 2024-09-17T21:20:21.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026238+0000", 2024-09-17T21:20:21.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026239+0000", 2024-09-17T21:20:21.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0380000000000001e-06 2024-09-17T21:20:21.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026243+0000", 2024-09-17T21:20:21.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3499999999999999e-06 2024-09-17T21:20:21.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026259+0000", 2024-09-17T21:20:21.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5928999999999999e-05 2024-09-17T21:20:21.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026270+0000", 2024-09-17T21:20:21.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0702e-05 2024-09-17T21:20:21.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.077682+0000", 2024-09-17T21:20:21.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051411865000000001 2024-09-17T21:20:21.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.077751+0000", 2024-09-17T21:20:21.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9606999999999999e-05 2024-09-17T21:20:21.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078014+0000", 2024-09-17T21:20:21.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026266199999999999 2024-09-17T21:20:21.230 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.230 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.230 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.230 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1587 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 802816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:21.230 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.026286+0000", 2024-09-17T21:20:21.230 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.13471951, 2024-09-17T21:20:21.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.052384495000000003, 2024-09-17T21:20:21.230 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.230 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.230 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.231 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.231 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:21.231 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:21.231 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1587 2024-09-17T21:20:21.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.231 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026286+0000", 2024-09-17T21:20:21.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026286+0000", 2024-09-17T21:20:21.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026287+0000", 2024-09-17T21:20:21.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0049999999999999e-06 2024-09-17T21:20:21.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026291+0000", 2024-09-17T21:20:21.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3479999999999997e-06 2024-09-17T21:20:21.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026304+0000", 2024-09-17T21:20:21.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3288999999999999e-05 2024-09-17T21:20:21.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.026314+0000", 2024-09-17T21:20:21.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9920000000000002e-06 2024-09-17T21:20:21.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078052+0000", 2024-09-17T21:20:21.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051737299 2024-09-17T21:20:21.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078118+0000", 2024-09-17T21:20:21.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6180999999999995e-05 2024-09-17T21:20:21.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078670+0000", 2024-09-17T21:20:21.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00055238100000000001 2024-09-17T21:20:21.234 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.234 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.234 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.234 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1618 3.e 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:1b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:21.235 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.031597+0000", 2024-09-17T21:20:21.235 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.129408069, 2024-09-17T21:20:21.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047545707999999999, 2024-09-17T21:20:21.235 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.235 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.235 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.235 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.235 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:21.235 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:21.235 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1618 2024-09-17T21:20:21.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.235 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031597+0000", 2024-09-17T21:20:21.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031597+0000", 2024-09-17T21:20:21.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031598+0000", 2024-09-17T21:20:21.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:20:21.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031602+0000", 2024-09-17T21:20:21.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:20:21.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031630+0000", 2024-09-17T21:20:21.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8058000000000001e-05 2024-09-17T21:20:21.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031643+0000", 2024-09-17T21:20:21.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2628e-05 2024-09-17T21:20:21.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078728+0000", 2024-09-17T21:20:21.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047085044 2024-09-17T21:20:21.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.078830+0000", 2024-09-17T21:20:21.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102493 2024-09-17T21:20:21.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079143+0000", 2024-09-17T21:20:21.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031263599999999998 2024-09-17T21:20:21.238 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.239 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.239 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.239 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1619 3.e 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:1b [read 827392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:21.239 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.031661+0000", 2024-09-17T21:20:21.239 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.129344501999999, 2024-09-17T21:20:21.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047908566999999999, 2024-09-17T21:20:21.239 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.239 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.239 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.239 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.239 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:21.239 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:21.240 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1619 2024-09-17T21:20:21.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.240 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031661+0000", 2024-09-17T21:20:21.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031661+0000", 2024-09-17T21:20:21.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031662+0000", 2024-09-17T21:20:21.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.252e-06 2024-09-17T21:20:21.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031666+0000", 2024-09-17T21:20:21.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6299999999999997e-06 2024-09-17T21:20:21.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031683+0000", 2024-09-17T21:20:21.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6035000000000001e-05 2024-09-17T21:20:21.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031693+0000", 2024-09-17T21:20:21.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0492000000000001e-05 2024-09-17T21:20:21.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079186+0000", 2024-09-17T21:20:21.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047493441999999997 2024-09-17T21:20:21.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079315+0000", 2024-09-17T21:20:21.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012860000000000001 2024-09-17T21:20:21.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079569+0000", 2024-09-17T21:20:21.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025411599999999998 2024-09-17T21:20:21.243 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.243 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.243 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.243 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1620 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:21.243 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.031706+0000", 2024-09-17T21:20:21.243 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.129299077999999, 2024-09-17T21:20:21.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048561143000000001, 2024-09-17T21:20:21.244 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.244 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.244 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.244 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.244 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:21.244 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:21.244 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1620 2024-09-17T21:20:21.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.244 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031706+0000", 2024-09-17T21:20:21.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031706+0000", 2024-09-17T21:20:21.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031707+0000", 2024-09-17T21:20:21.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5799999999999998e-07 2024-09-17T21:20:21.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031711+0000", 2024-09-17T21:20:21.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1930000000000003e-06 2024-09-17T21:20:21.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031724+0000", 2024-09-17T21:20:21.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.24e-05 2024-09-17T21:20:21.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031733+0000", 2024-09-17T21:20:21.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6239999999999997e-06 2024-09-17T21:20:21.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079613+0000", 2024-09-17T21:20:21.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047880082999999997 2024-09-17T21:20:21.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.079719+0000", 2024-09-17T21:20:21.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010559499999999999 2024-09-17T21:20:21.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.080267+0000", 2024-09-17T21:20:21.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00054829 2024-09-17T21:20:21.247 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.247 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.248 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.248 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5243.0:1621 3.e 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:1b [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T21:20:21.248 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:54.031746+0000", 2024-09-17T21:20:21.248 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.129258604, 2024-09-17T21:20:21.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048925189000000001, 2024-09-17T21:20:21.248 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.248 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.248 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.248 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.248 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5243", 2024-09-17T21:20:21.248 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/2378702756", 2024-09-17T21:20:21.249 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1621 2024-09-17T21:20:21.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.249 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031746+0000", 2024-09-17T21:20:21.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031746+0000", 2024-09-17T21:20:21.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031747+0000", 2024-09-17T21:20:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3900000000000003e-07 2024-09-17T21:20:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031752+0000", 2024-09-17T21:20:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.172e-06 2024-09-17T21:20:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031767+0000", 2024-09-17T21:20:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5517000000000001e-05 2024-09-17T21:20:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.031777+0000", 2024-09-17T21:20:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8579999999999995e-06 2024-09-17T21:20:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.080318+0000", 2024-09-17T21:20:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048541009000000003 2024-09-17T21:20:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.080415+0000", 2024-09-17T21:20:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7034999999999995e-05 2024-09-17T21:20:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:54.080672+0000", 2024-09-17T21:20:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025665900000000001 2024-09-17T21:20:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:18003 3.1d 3:b9765580:::rbd_header.11527d2b15f6:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T21:20:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:20:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.803126378, 2024-09-17T21:20:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94165346900000002, 2024-09-17T21:20:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18003 2024-09-17T21:20:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:20:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:20:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357880+0000", 2024-09-17T21:20:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:20:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357891+0000", 2024-09-17T21:20:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0417999999999999e-05 2024-09-17T21:20:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357913+0000", 2024-09-17T21:20:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2215e-05 2024-09-17T21:20:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357922+0000", 2024-09-17T21:20:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4779999999999997e-06 2024-09-17T21:20:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358023+0000", 2024-09-17T21:20:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101025 2024-09-17T21:20:21.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358040+0000", 2024-09-17T21:20:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6291999999999999e-05 2024-09-17T21:20:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298203+0000", 2024-09-17T21:20:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94016297500000001 2024-09-17T21:20:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298626+0000", 2024-09-17T21:20:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000422853 2024-09-17T21:20:21.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.299532+0000", 2024-09-17T21:20:21.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00090657300000000004 2024-09-17T21:20:21.257 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.257 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.257 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.257 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.257 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.257 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:21.259 INFO:teuthology.orchestra.run.smithi184.stdout:4826 read 0xa710c00 thru 0xa714486 (0x3887 bytes) 2024-09-17T21:20:21.259 INFO:teuthology.orchestra.run.smithi184.stdout:4828 write 0x37d9e00 thru 0x37e862a (0xe82b bytes) 2024-09-17T21:20:21.259 INFO:teuthology.orchestra.run.smithi184.stdout:4829 write 0xa9c0200 thru 0xa9ce3a6 (0xe1a7 bytes) 2024-09-17T21:20:21.259 INFO:teuthology.orchestra.run.smithi184.stdout:4832 trunc from 0xda16400 to 0x3a1f200 2024-09-17T21:20:21.259 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:20:21.260 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:21.253+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-17T21:20:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-17T21:20:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-17T21:20:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-17T21:20:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-17T21:20:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-17T21:20:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:bench [] [] [] [] 2024-09-17T21:20:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs debug_inject_read_zeros 2024-09-17T21:20:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs files list 2024-09-17T21:20:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs stats 2024-09-17T21:20:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator dump block 2024-09-17T21:20:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:Error EINVAL: invalid command 2024-09-17T21:20:21.290 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T21:20:21.291 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T21:20:21.291 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-17T21:20:21.299 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-17T21:20:21.369 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:21.365+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:21.450 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:21.451 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:21.451 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:20:21.451 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:21.460 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:21.453+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:21.467 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:20:21.469 INFO:teuthology.orchestra.run.smithi184.stdout:4833 punch from 0x32e4e00 to 0x32ee200, (0x9400 bytes) 2024-09-17T21:20:21.487 INFO:teuthology.orchestra.run.smithi184.stdout:4836 write 0x2c62e00 thru 0x2c67e82 (0x5083 bytes) 2024-09-17T21:20:21.489 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-17T21:20:21.489 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-17T21:20:21.489 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-17T21:20:21.489 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-17T21:20:21.489 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-17T21:20:21.489 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-17T21:20:21.489 INFO:teuthology.orchestra.run.smithi083.stderr:bench [] [] [] [] 2024-09-17T21:20:21.489 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs debug_inject_read_zeros 2024-09-17T21:20:21.489 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs files list 2024-09-17T21:20:21.489 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs stats 2024-09-17T21:20:21.489 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator dump block 2024-09-17T21:20:21.490 INFO:teuthology.orchestra.run.smithi083.stderr:Error EINVAL: invalid command 2024-09-17T21:20:21.494 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T21:20:21.494 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T21:20:21.561 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:21.557+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:21.608 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:21.609 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:21.628 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:20:21.647 INFO:teuthology.orchestra.run.smithi184.stdout:4837 write 0x7753400 thru 0x775c2b9 (0x8eba bytes) 2024-09-17T21:20:21.663 INFO:teuthology.orchestra.run.smithi184.stdout:4839 trunc from 0x775c400 to 0xcb7b600 2024-09-17T21:20:21.675 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:21.670+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:21.705 INFO:teuthology.orchestra.run.smithi184.stdout:4840 write 0x24c4800 thru 0x24cf86f (0xb070 bytes) 2024-09-17T21:20:21.707 INFO:teuthology.orchestra.run.smithi184.stdout:4842 read 0x43ef600 thru 0x43f9692 (0xa093 bytes) 2024-09-17T21:20:21.708 INFO:teuthology.orchestra.run.smithi184.stdout:4844 write 0xec6a00 thru 0xecf923 (0x8f24 bytes) 2024-09-17T21:20:21.717 INFO:teuthology.orchestra.run.smithi184.stdout:4845 trunc from 0xcb7b600 to 0xa644a00 2024-09-17T21:20:21.762 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:21.754+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:21.769 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:21.769 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:21.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:21.769 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:21.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.769 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:20:21.769 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:21.769 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 120.161406955, 2024-09-17T21:20:21.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:20:21.769 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.769 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.769 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:21.770 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:21.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:21.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:20:21.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:20:21.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:20:21.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:20:21.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:20:21.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:20:21.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:20:21.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:20:21.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:20:21.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:20:21.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:20:21.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:20:21.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:21.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:20:21.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:20:21.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:20:21.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:20:21.773 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.773 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.773 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.774 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:20:21.774 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:21.774 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 120.15146031899999, 2024-09-17T21:20:21.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:20:21.774 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.774 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.774 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:21.774 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:21.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:21.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:20:21.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:20:21.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:20:21.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:20:21.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:20:21.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:20:21.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:20:21.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:20:21.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:20:21.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:20:21.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:20:21.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:20:21.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:21.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:20:21.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:20:21.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:20:21.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:20:21.778 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.778 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.778 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.778 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:20:21.779 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:21.779 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 118.908309479, 2024-09-17T21:20:21.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:20:21.779 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.779 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.779 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:21.779 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.779 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:20:21.779 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:20:21.779 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:20:21.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.779 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:21.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:21.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:20:21.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:20:21.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:20:21.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:20:21.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:20:21.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:20:21.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:20:21.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:20:21.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:20:21.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:20:21.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:20:21.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:20:21.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:21.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:20:21.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:20:21.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:21.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:20:21.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:20:21.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:21.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:20:21.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:20:21.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:21.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:20:21.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:20:21.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:20:21.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:20:21.784 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.784 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.784 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.784 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:20:21.784 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:21.784 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 118.898663993, 2024-09-17T21:20:21.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:20:21.784 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.785 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.785 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:21.785 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.785 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:20:21.785 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:20:21.785 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:20:21.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.785 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:21.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:21.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:20:21.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:20:21.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:20:21.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:20:21.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:20:21.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:20:21.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:20:21.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:20:21.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:20:21.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:20:21.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:20:21.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:20:21.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:21.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:20:21.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:20:21.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:21.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:20:21.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:20:21.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:21.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:20:21.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:20:21.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:21.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:20:21.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:20:21.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:20:21.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:20:21.790 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.790 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.790 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.790 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:20:21.790 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:21.790 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 105.52972796, 2024-09-17T21:20:21.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:20:21.790 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.790 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.791 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.791 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.791 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:20:21.791 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:20:21.791 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:20:21.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.791 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:21.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:21.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:20:21.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:20:21.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:20:21.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:20:21.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:20:21.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:20:21.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:20:21.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:20:21.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:20:21.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:20:21.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:20:21.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:20:21.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:20:21.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:20:21.794 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.794 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.794 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.795 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:819 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5111808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:21.795 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:21.795 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 87.328155670000001, 2024-09-17T21:20:21.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024344811000000001, 2024-09-17T21:20:21.795 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.795 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.795 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.795 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.795 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:21.795 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:21.795 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 819 2024-09-17T21:20:21.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.796 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:21.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:21.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437443+0000", 2024-09-17T21:20:21.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1969999999999999e-06 2024-09-17T21:20:21.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437448+0000", 2024-09-17T21:20:21.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9490000000000004e-06 2024-09-17T21:20:21.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437467+0000", 2024-09-17T21:20:21.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8862999999999999e-05 2024-09-17T21:20:21.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437480+0000", 2024-09-17T21:20:21.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2833000000000001e-05 2024-09-17T21:20:21.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461552+0000", 2024-09-17T21:20:21.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024072695000000002 2024-09-17T21:20:21.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461639+0000", 2024-09-17T21:20:21.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6996000000000005e-05 2024-09-17T21:20:21.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461787+0000", 2024-09-17T21:20:21.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014727800000000001 2024-09-17T21:20:21.798 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.799 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.799 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.799 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:21.799 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:21.799 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 87.328043398000005, 2024-09-17T21:20:21.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:20:21.799 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.799 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.799 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.799 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.799 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:21.799 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:21.799 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:20:21.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.800 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:21.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:21.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:20:21.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:20:21.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:20:21.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:20:21.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:20:21.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:20:21.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:20:21.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:20:21.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:20:21.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:20:21.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:20:21.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:20:21.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:20:21.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:20:21.803 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.803 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.803 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.803 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:833 3.5 3:a772dfb1:::rbd_data.11523651b015.0000000000000001:d [sparse-read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:21.803 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:21.803 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 87.327609112000005, 2024-09-17T21:20:21.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024216024999999999, 2024-09-17T21:20:21.803 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.803 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.803 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.803 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.804 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:21.804 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:21.804 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 833 2024-09-17T21:20:21.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.804 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:21.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:21.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437990+0000", 2024-09-17T21:20:21.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2500000000000001e-06 2024-09-17T21:20:21.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437995+0000", 2024-09-17T21:20:21.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1240000000000004e-06 2024-09-17T21:20:21.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438012+0000", 2024-09-17T21:20:21.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7572e-05 2024-09-17T21:20:21.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438031+0000", 2024-09-17T21:20:21.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8612000000000001e-05 2024-09-17T21:20:21.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461830+0000", 2024-09-17T21:20:21.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023799135999999999 2024-09-17T21:20:21.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462005+0000", 2024-09-17T21:20:21.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000174747 2024-09-17T21:20:21.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:20:21.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019958399999999999 2024-09-17T21:20:21.807 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.807 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.807 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.807 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:21.807 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:21.807 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 87.327548820999993, 2024-09-17T21:20:21.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:20:21.807 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.807 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.807 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.807 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.808 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:21.808 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:21.808 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:20:21.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.808 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:21.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:21.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:20:21.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:20:21.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:20:21.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:20:21.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:20:21.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:20:21.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:20:21.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:20:21.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:20:21.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:20:21.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 87.327493439999998, 2024-09-17T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:20:21.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:20:21.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:20:21.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:20:21.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:20:21.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:20:21.815 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.815 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.815 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.816 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:21.816 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:21.816 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 87.327437161999995, 2024-09-17T21:20:21.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:20:21.816 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.816 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.816 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.816 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.816 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:21.816 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:21.816 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:20:21.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.816 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:21.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:21.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:20:21.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:20:21.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:20:21.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:20:21.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:20:21.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:20:21.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:20:21.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:20:21.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:20:21.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:20:21.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:20:21.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:20:21.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:20:21.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:20:21.819 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.820 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.820 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.820 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:21.820 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:21.820 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 87.326487522999997, 2024-09-17T21:20:21.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:20:21.820 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.820 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.820 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.820 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.820 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:21.820 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:21.820 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:20:21.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.821 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:21.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:21.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:20:21.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:20:21.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:20:21.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:20:21.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:20:21.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:20:21.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:20:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:20:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:20:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:20:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:20:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:20:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:20:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:20:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 87.326408903000001, 2024-09-17T21:20:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:20:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.825 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.825 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.825 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:21.825 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:21.825 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:20:21.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.825 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:21.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:21.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:20:21.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:20:21.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:20:21.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:20:21.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:20:21.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:20:21.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:20:21.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:20:21.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:20:21.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:20:21.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:20:21.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:20:21.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:20:21.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:20:21.828 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.828 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.828 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.829 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:21.829 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:21.829 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 87.326295963999996, 2024-09-17T21:20:21.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:20:21.829 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.829 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.829 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.829 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.829 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:21.829 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:21.829 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:20:21.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.829 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:21.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:21.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:20:21.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:20:21.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:20:21.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:20:21.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:20:21.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:20:21.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:20:21.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:20:21.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:20:21.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:20:21.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:20:21.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:20:21.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:20:21.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:20:21.832 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.833 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.833 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.833 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:849 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5963776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:21.833 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:21.833 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 87.325701928000001, 2024-09-17T21:20:21.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024537514, 2024-09-17T21:20:21.833 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.833 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.833 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.833 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.833 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:21.833 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:21.834 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 849 2024-09-17T21:20:21.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.834 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:21.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:21.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439897+0000", 2024-09-17T21:20:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9099999999999999e-06 2024-09-17T21:20:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439905+0000", 2024-09-17T21:20:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0870000000000007e-06 2024-09-17T21:20:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439924+0000", 2024-09-17T21:20:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9042000000000002e-05 2024-09-17T21:20:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439941+0000", 2024-09-17T21:20:21.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.624e-05 2024-09-17T21:20:21.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464132+0000", 2024-09-17T21:20:21.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024191579000000001 2024-09-17T21:20:21.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464291+0000", 2024-09-17T21:20:21.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015904800000000001 2024-09-17T21:20:21.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464433+0000", 2024-09-17T21:20:21.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014160799999999999 2024-09-17T21:20:21.837 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.837 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.837 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.837 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:21.837 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:21.837 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 87.325634910000005, 2024-09-17T21:20:21.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:20:21.838 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.838 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.838 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.838 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.838 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:21.838 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:21.838 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:20:21.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.838 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:21.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:21.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:20:21.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:20:21.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:20:21.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:20:21.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:20:21.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:20:21.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:20:21.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:20:21.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:20:21.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:20:21.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:20:21.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:20:21.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:20:21.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:20:21.841 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.842 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.842 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.842 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:21.842 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:21.842 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 87.325570099000004, 2024-09-17T21:20:21.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:20:21.842 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.842 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.842 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.842 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.843 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:21.843 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:21.843 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:20:21.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.843 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:21.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:21.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:20:21.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:20:21.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:20:21.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:20:21.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:20:21.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:20:21.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:20:21.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:20:21.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:20:21.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:20:21.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:20:21.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:20:21.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:20:21.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:20:21.846 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.846 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.846 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.846 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:21.846 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:21.846 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 87.325509122, 2024-09-17T21:20:21.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:20:21.847 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.847 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.847 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:21.847 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:21.847 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:21.847 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:21.847 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:20:21.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.847 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:21.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:21.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:20:21.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:20:21.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:20:21.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:20:21.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:20:21.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:20:21.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:20:21.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:20:21.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:20:21.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:20:21.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:20:21.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:20:21.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:20:21.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:20:21.850 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.850 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.850 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.850 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:20:21.850 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:21.851 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.427937937000003, 2024-09-17T21:20:21.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:20:21.851 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.851 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.851 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:21.851 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:21.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:21.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:20:21.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:20:21.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:20:21.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:20:21.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:20:21.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:20:21.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:20:21.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:20:21.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:20:21.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:20:21.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:20:21.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:20:21.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:21.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:20:21.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:20:21.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:20:21.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:20:21.854 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.854 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.854 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.855 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:20:21.855 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:21.855 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.219279542999999, 2024-09-17T21:20:21.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:20:21.855 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:21.855 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:21.855 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:21.855 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:21.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:21.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:21.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:21.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:21.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:21.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:21.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:20:21.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:20:21.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:21.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:20:21.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:20:21.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:21.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:20:21.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:20:21.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:21.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:20:21.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:20:21.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:21.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:20:21.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:20:21.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:21.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:20:21.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:20:21.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:21.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:20:21.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:20:21.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:21.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:21.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:21.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:20:21.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:20:21.859 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.859 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.859 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.859 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:21.859 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:21.859 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:21.860 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:20:21.863 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:21.857+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:21.920 INFO:teuthology.orchestra.run.smithi184.stdout:4847 trunc from 0xa644a00 to 0x274bc00 2024-09-17T21:20:21.964 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:21.958+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:21.991 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:21.991 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:21.991 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:21.991 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:22.007 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:20:22.065 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:22.061+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:22.082 INFO:teuthology.orchestra.run.smithi184.stdout:4848 read 0x255fa00 thru 0x256e6cc (0xeccd bytes) 2024-09-17T21:20:22.087 INFO:teuthology.orchestra.run.smithi184.stdout:4851 read 0x7d1000 thru 0x7d5e5c (0x4e5d bytes) 2024-09-17T21:20:22.092 INFO:teuthology.orchestra.run.smithi184.stdout:4852 read 0x208b200 thru 0x209a61b (0xf41c bytes) 2024-09-17T21:20:22.105 INFO:teuthology.orchestra.run.smithi184.stdout:4853 trunc from 0x274bc00 to 0x7001800 2024-09-17T21:20:22.122 INFO:teuthology.orchestra.run.smithi184.stdout:4855 read 0x502a00 thru 0x507458 (0x4a59 bytes) 2024-09-17T21:20:22.138 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:22.138 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:22.138 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:22.138 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:22.139 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:22.143 INFO:teuthology.orchestra.run.smithi184.stdout:4856 trunc from 0x7001800 to 0x3d35c00 2024-09-17T21:20:22.153 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:20:22.165 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:22.158+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:22.240 INFO:teuthology.orchestra.run.smithi184.stdout:4858 write 0x3c6aa00 thru 0x3c72f59 (0x855a bytes) 2024-09-17T21:20:22.242 INFO:teuthology.orchestra.run.smithi184.stdout:4859 read 0x3b72800 thru 0x3b7c5f9 (0x9dfa bytes) 2024-09-17T21:20:22.266 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:22.261+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:22.332 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:22.332 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:22.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:22.332 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:22.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.332 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:525 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3670016~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:22.332 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:22.332 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 116.127536558, 2024-09-17T21:20:22.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046645697, 2024-09-17T21:20:22.333 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:22.333 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:22.333 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:22.333 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:22.333 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:22.333 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:22.333 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 525 2024-09-17T21:20:22.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.333 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:22.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:22.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:22.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:22.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:22.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:22.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:20:22.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:20:22.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:22.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200234+0000", 2024-09-17T21:20:22.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:20:22.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:22.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200254+0000", 2024-09-17T21:20:22.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9959999999999999e-05 2024-09-17T21:20:22.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:22.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200293+0000", 2024-09-17T21:20:22.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9755999999999997e-05 2024-09-17T21:20:22.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245871+0000", 2024-09-17T21:20:22.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045577871999999998 2024-09-17T21:20:22.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:22.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246030+0000", 2024-09-17T21:20:22.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158975 2024-09-17T21:20:22.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:22.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246871+0000", 2024-09-17T21:20:22.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084111699999999995 2024-09-17T21:20:22.337 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.337 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:22.337 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.337 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:22.337 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:22.337 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 116.127290883, 2024-09-17T21:20:22.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:20:22.338 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:22.338 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:22.338 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:22.338 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:22.338 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:22.338 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:22.338 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:20:22.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.338 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:22.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:22.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:22.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:22.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:22.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:22.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:20:22.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:20:22.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:22.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:20:22.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:20:22.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:22.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:20:22.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:20:22.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:22.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:20:22.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:20:22.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:20:22.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:20:22.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:22.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:20:22.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:20:22.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:22.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:20:22.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:20:22.342 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.342 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:22.342 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.342 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:22.342 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:22.342 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 116.127112201, 2024-09-17T21:20:22.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:20:22.342 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:22.343 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:22.343 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:22.343 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:22.343 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:22.343 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:22.343 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:20:22.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.343 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:22.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:22.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:22.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:22.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:22.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:22.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:20:22.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:20:22.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:22.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:20:22.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:20:22.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:22.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:20:22.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:20:22.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:22.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:20:22.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:20:22.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:20:22.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:20:22.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:22.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:20:22.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:20:22.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:22.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:20:22.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:20:22.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.347 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:22.347 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.347 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:22.347 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:22.347 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 116.127052928, 2024-09-17T21:20:22.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:20:22.347 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:22.347 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:22.347 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:22.347 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:22.347 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:22.348 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:22.348 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:20:22.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.348 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:22.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:22.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:22.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:22.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:22.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:22.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:20:22.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:20:22.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:22.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:20:22.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:20:22.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:22.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:20:22.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:20:22.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:22.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:20:22.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:20:22.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:20:22.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:20:22.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:22.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:20:22.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:20:22.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:22.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:20:22.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:20:22.351 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.351 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:22.351 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.352 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:22.352 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:22.352 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 116.126910733, 2024-09-17T21:20:22.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:20:22.352 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:22.352 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:22.352 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:22.352 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:22.352 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:22.352 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:22.352 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:20:22.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.352 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:22.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:22.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:22.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:22.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:22.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:22.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:20:22.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:20:22.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:22.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:20:22.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:20:22.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:22.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:20:22.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:20:22.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:22.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:20:22.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:20:22.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:20:22.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:20:22.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:22.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:20:22.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:20:22.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:22.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:20:22.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:20:22.356 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.356 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:22.356 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.356 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:22.356 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:22.356 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 116.12685481, 2024-09-17T21:20:22.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:20:22.357 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:22.357 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:22.357 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:22.357 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:22.357 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:22.357 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:22.357 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:20:22.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.357 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:22.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:22.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:22.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:22.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:22.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:22.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:20:22.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:20:22.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:22.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:20:22.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:20:22.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:22.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:20:22.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:20:22.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:22.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:20:22.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:20:22.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:20:22.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:20:22.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:22.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:20:22.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:20:22.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:22.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:20:22.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:20:22.361 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.361 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:22.361 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.361 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:22.361 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:22.361 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 116.12680119300001, 2024-09-17T21:20:22.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:20:22.361 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:22.362 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:22.362 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:22.362 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:22.362 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:22.362 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:22.362 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:20:22.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.362 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:22.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:22.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:22.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:22.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:22.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:22.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:20:22.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:20:22.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:22.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:20:22.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:20:22.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:22.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:20:22.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:20:22.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:22.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:20:22.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:20:22.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:20:22.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:20:22.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:22.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:20:22.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:20:22.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:22.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:20:22.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:20:22.366 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.366 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:22.366 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.366 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:20:22.366 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:22.366 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 65.817572951000002, 2024-09-17T21:20:22.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:20:22.366 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:22.366 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:22.366 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:20:22.366 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:22.367 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:20:22.367 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:20:22.367 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:20:22.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.367 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:22.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:22.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:22.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:22.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:22.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:22.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:20:22.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:20:22.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:22.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:20:22.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:20:22.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:22.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:20:22.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:20:22.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:22.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:20:22.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:20:22.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:20:22.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:20:22.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:22.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:20:22.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:20:22.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:20:22.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:20:22.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:22.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:20:22.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:20:22.371 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.371 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:22.371 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.371 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2597 3.1f 3:fb0e9615:::rbd_data.1152a8ae2019.000000000000000c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:22.371 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:22.372 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.489136522999999, 2024-09-17T21:20:22.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097064994000000002, 2024-09-17T21:20:22.372 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:22.372 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:22.372 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:22.372 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:22.372 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:22.372 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:22.372 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2597 2024-09-17T21:20:22.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.372 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:22.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:22.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:22.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:22.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:22.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:22.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838628+0000", 2024-09-17T21:20:22.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.187e-06 2024-09-17T21:20:22.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:22.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838654+0000", 2024-09-17T21:20:22.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:20:22.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:22.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838696+0000", 2024-09-17T21:20:22.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1226000000000001e-05 2024-09-17T21:20:22.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:22.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838715+0000", 2024-09-17T21:20:22.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9646000000000001e-05 2024-09-17T21:20:22.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838937+0000", 2024-09-17T21:20:22.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022163 2024-09-17T21:20:22.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:22.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.839284+0000", 2024-09-17T21:20:22.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034753799999999999 2024-09-17T21:20:22.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:22.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935691+0000", 2024-09-17T21:20:22.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.096406360999999996 2024-09-17T21:20:22.376 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.376 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:22.376 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.377 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2641 3.12 3:488a729f:::rbd_data.1152a8ae2019.0000000000000022:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:22.377 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:22.377 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.485679750000003, 2024-09-17T21:20:22.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098280953000000004, 2024-09-17T21:20:22.377 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:22.377 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:22.377 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:22.377 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:22.377 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:22.377 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:22.377 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2641 2024-09-17T21:20:22.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.378 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:22.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:22.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:22.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:22.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842083+0000", 2024-09-17T21:20:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5300000000000002e-07 2024-09-17T21:20:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842088+0000", 2024-09-17T21:20:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:20:22.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:22.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842105+0000", 2024-09-17T21:20:22.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6773e-05 2024-09-17T21:20:22.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:22.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842119+0000", 2024-09-17T21:20:22.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:20:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842320+0000", 2024-09-17T21:20:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020052700000000001 2024-09-17T21:20:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842845+0000", 2024-09-17T21:20:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052508600000000002 2024-09-17T21:20:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940363+0000", 2024-09-17T21:20:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097518795000000005 2024-09-17T21:20:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2674 3.12 3:4c87e3ac:::rbd_data.1152a8ae2019.0000000000000033:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.485317113000001, 2024-09-17T21:20:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098025125000000005, 2024-09-17T21:20:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:22.382 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2674 2024-09-17T21:20:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842447+0000", 2024-09-17T21:20:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3999999999999999e-06 2024-09-17T21:20:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842453+0000", 2024-09-17T21:20:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:20:22.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842472+0000", 2024-09-17T21:20:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9006000000000001e-05 2024-09-17T21:20:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842487+0000", 2024-09-17T21:20:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4866e-05 2024-09-17T21:20:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843594+0000", 2024-09-17T21:20:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011073249999999999 2024-09-17T21:20:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843668+0000", 2024-09-17T21:20:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3188000000000005e-05 2024-09-17T21:20:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843893+0000", 2024-09-17T21:20:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022503399999999999 2024-09-17T21:20:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850439+0000", 2024-09-17T21:20:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065467880000000004 2024-09-17T21:20:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940409+0000", 2024-09-17T21:20:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089969511000000002 2024-09-17T21:20:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940444+0000", 2024-09-17T21:20:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5345999999999997e-05 2024-09-17T21:20:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940470+0000", 2024-09-17T21:20:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5964000000000001e-05 2024-09-17T21:20:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2707 3.1f 3:f805e65e:::rbd_data.1152a8ae2019.0000000000000049:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.480270451999999, 2024-09-17T21:20:22.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088327300999999997, 2024-09-17T21:20:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2707 2024-09-17T21:20:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847494+0000", 2024-09-17T21:20:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:20:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847504+0000", 2024-09-17T21:20:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:20:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847526+0000", 2024-09-17T21:20:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1976000000000001e-05 2024-09-17T21:20:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847541+0000", 2024-09-17T21:20:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5277e-05 2024-09-17T21:20:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856142+0000", 2024-09-17T21:20:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086006710000000007 2024-09-17T21:20:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856435+0000", 2024-09-17T21:20:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029301499999999999 2024-09-17T21:20:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935819+0000", 2024-09-17T21:20:22.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079384448999999996 2024-09-17T21:20:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2730 3.12 3:48b3422a:::rbd_data.1152a8ae2019.0000000000000057:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.471077553000001, 2024-09-17T21:20:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083879975999999995, 2024-09-17T21:20:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:22.393 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2730 2024-09-17T21:20:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:22.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856687+0000", 2024-09-17T21:20:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:20:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856697+0000", 2024-09-17T21:20:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0207000000000001e-05 2024-09-17T21:20:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856808+0000", 2024-09-17T21:20:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011071200000000001 2024-09-17T21:20:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856828+0000", 2024-09-17T21:20:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0219000000000001e-05 2024-09-17T21:20:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856907+0000", 2024-09-17T21:20:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9661999999999997e-05 2024-09-17T21:20:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.857019+0000", 2024-09-17T21:20:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011148499999999999 2024-09-17T21:20:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940565+0000", 2024-09-17T21:20:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083545791999999994 2024-09-17T21:20:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2743 3.12 3:4d036e62:::rbd_data.1152a8ae2019.000000000000005f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.467132051999997, 2024-09-17T21:20:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080059962999999998, 2024-09-17T21:20:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2743 2024-09-17T21:20:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860631+0000", 2024-09-17T21:20:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:20:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860636+0000", 2024-09-17T21:20:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5279999999999997e-06 2024-09-17T21:20:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860649+0000", 2024-09-17T21:20:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-17T21:20:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:22.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860655+0000", 2024-09-17T21:20:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:20:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860731+0000", 2024-09-17T21:20:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5920000000000005e-05 2024-09-17T21:20:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.861070+0000", 2024-09-17T21:20:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338535 2024-09-17T21:20:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940690+0000", 2024-09-17T21:20:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079620503999999995 2024-09-17T21:20:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3378 3.12 3:4832a51b:::rbd_data.1152a8ae2019.000000000000006c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.438768201999999, 2024-09-17T21:20:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051812516000000003, 2024-09-17T21:20:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:22.402 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3378 2024-09-17T21:20:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:22.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:22.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:22.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888995+0000", 2024-09-17T21:20:22.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:20:22.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889006+0000", 2024-09-17T21:20:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1012999999999999e-05 2024-09-17T21:20:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889032+0000", 2024-09-17T21:20:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6441e-05 2024-09-17T21:20:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889040+0000", 2024-09-17T21:20:22.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0639999999999994e-06 2024-09-17T21:20:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889278+0000", 2024-09-17T21:20:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023785899999999999 2024-09-17T21:20:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889408+0000", 2024-09-17T21:20:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129219 2024-09-17T21:20:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:22.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889723+0000", 2024-09-17T21:20:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000315264 2024-09-17T21:20:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.891972+0000", 2024-09-17T21:20:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002249633 2024-09-17T21:20:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940733+0000", 2024-09-17T21:20:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048760626000000001 2024-09-17T21:20:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940785+0000", 2024-09-17T21:20:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1906000000000001e-05 2024-09-17T21:20:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940807+0000", 2024-09-17T21:20:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1554999999999999e-05 2024-09-17T21:20:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3379 3.12 3:4e83fd71:::rbd_data.1152a8ae2019.000000000000006d:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.438683994999998, 2024-09-17T21:20:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051804144000000003, 2024-09-17T21:20:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3379 2024-09-17T21:20:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889079+0000", 2024-09-17T21:20:22.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1800000000000004e-07 2024-09-17T21:20:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889083+0000", 2024-09-17T21:20:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:20:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889094+0000", 2024-09-17T21:20:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1562999999999999e-05 2024-09-17T21:20:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889102+0000", 2024-09-17T21:20:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.401e-06 2024-09-17T21:20:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:20:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014279259999999999 2024-09-17T21:20:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890609+0000", 2024-09-17T21:20:22.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9566000000000004e-05 2024-09-17T21:20:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890789+0000", 2024-09-17T21:20:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017949700000000001 2024-09-17T21:20:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.893929+0000", 2024-09-17T21:20:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003140538 2024-09-17T21:20:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.413 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940834+0000", 2024-09-17T21:20:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046904811999999997 2024-09-17T21:20:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940860+0000", 2024-09-17T21:20:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5341999999999999e-05 2024-09-17T21:20:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940882+0000", 2024-09-17T21:20:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2878999999999999e-05 2024-09-17T21:20:22.414 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:2644 3.1e 3:79615585:::rbd_data.115266d0b9f2.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2475520~14848,write 3106304~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:20:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 32.816055591999998, 2024-09-17T21:20:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050733484000000002, 2024-09-17T21:20:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:22.415 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:20:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:20:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2644 2024-09-17T21:20:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511708+0000", 2024-09-17T21:20:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-17T21:20:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522196+0000", 2024-09-17T21:20:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010487712999999999 2024-09-17T21:20:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522223+0000", 2024-09-17T21:20:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7064999999999999e-05 2024-09-17T21:20:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522236+0000", 2024-09-17T21:20:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2931e-05 2024-09-17T21:20:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522348+0000", 2024-09-17T21:20:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111906 2024-09-17T21:20:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522554+0000", 2024-09-17T21:20:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020632699999999999 2024-09-17T21:20:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:20:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.523185+0000", 2024-09-17T21:20:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063081400000000005 2024-09-17T21:20:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.547011+0000", 2024-09-17T21:20:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023825570000000001 2024-09-17T21:20:22.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562268+0000", 2024-09-17T21:20:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015257514 2024-09-17T21:20:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562367+0000", 2024-09-17T21:20:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8678000000000006e-05 2024-09-17T21:20:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562440+0000", 2024-09-17T21:20:22.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3337000000000006e-05 2024-09-17T21:20:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:231 3.1 3:856fed78:::rbd_data.115220dd393e.000000000000000b:head [read 0~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:20:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:20:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.3733003229999996, 2024-09-17T21:20:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34254251699999999, 2024-09-17T21:20:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:20:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:22.421 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 231 2024-09-17T21:20:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:20:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:22.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:20:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954464+0000", 2024-09-17T21:20:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:20:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954475+0000", 2024-09-17T21:20:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1146e-05 2024-09-17T21:20:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954506+0000", 2024-09-17T21:20:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0292999999999999e-05 2024-09-17T21:20:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954524+0000", 2024-09-17T21:20:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8430000000000001e-05 2024-09-17T21:20:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954661+0000", 2024-09-17T21:20:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136403 2024-09-17T21:20:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954680+0000", 2024-09-17T21:20:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9743000000000001e-05 2024-09-17T21:20:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.296896+0000", 2024-09-17T21:20:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34221550499999998 2024-09-17T21:20:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297004+0000", 2024-09-17T21:20:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010865399999999999 2024-09-17T21:20:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:56 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:20:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:20:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.2176666259999998, 2024-09-17T21:20:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18718773899999999, 2024-09-17T21:20:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:22.426 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:20:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:20:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 56 2024-09-17T21:20:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:20:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:20:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110098+0000", 2024-09-17T21:20:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:20:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110111+0000", 2024-09-17T21:20:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2289999999999999e-05 2024-09-17T21:20:22.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110143+0000", 2024-09-17T21:20:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1979e-05 2024-09-17T21:20:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110161+0000", 2024-09-17T21:20:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7839e-05 2024-09-17T21:20:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110319+0000", 2024-09-17T21:20:22.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015835400000000001 2024-09-17T21:20:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110353+0000", 2024-09-17T21:20:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3849000000000001e-05 2024-09-17T21:20:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297041+0000", 2024-09-17T21:20:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18668842299999999 2024-09-17T21:20:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297169+0000", 2024-09-17T21:20:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012786199999999999 2024-09-17T21:20:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297283+0000", 2024-09-17T21:20:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011430300000000001 2024-09-17T21:20:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:68 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:20:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:20:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.209125695, 2024-09-17T21:20:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17885582999999999, 2024-09-17T21:20:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:20:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:20:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 68 2024-09-17T21:20:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:20:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:20:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118639+0000", 2024-09-17T21:20:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.306e-06 2024-09-17T21:20:22.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118651+0000", 2024-09-17T21:20:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2567e-05 2024-09-17T21:20:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118682+0000", 2024-09-17T21:20:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0502e-05 2024-09-17T21:20:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118707+0000", 2024-09-17T21:20:22.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5463e-05 2024-09-17T21:20:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118889+0000", 2024-09-17T21:20:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018150999999999999 2024-09-17T21:20:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118910+0000", 2024-09-17T21:20:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:20:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:22.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297313+0000", 2024-09-17T21:20:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17840366699999999 2024-09-17T21:20:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297389+0000", 2024-09-17T21:20:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-17T21:20:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297492+0000", 2024-09-17T21:20:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103019 2024-09-17T21:20:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:22.436 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.437 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:22.437 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:22.437 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:22.439 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:20:22.440 INFO:teuthology.orchestra.run.smithi184.stdout:4860 write 0x894f400 thru 0x895d347 (0xdf48 bytes) 2024-09-17T21:20:22.440 INFO:teuthology.orchestra.run.smithi184.stdout:4867 read 0x5ac5200 thru 0x5ac5d2e (0xb2f bytes) 2024-09-17T21:20:22.440 INFO:teuthology.orchestra.run.smithi184.stdout:4868 punch from 0x676aa00 to 0x6779600, (0xec00 bytes) 2024-09-17T21:20:22.440 INFO:teuthology.orchestra.run.smithi184.stdout:4870 punch from 0x1c36e00 to 0x1c43c00, (0xce00 bytes) 2024-09-17T21:20:22.440 INFO:teuthology.orchestra.run.smithi184.stdout:4871 write 0x67cbe00 thru 0x67cec64 (0x2e65 bytes) 2024-09-17T21:20:22.440 INFO:teuthology.orchestra.run.smithi184.stdout:4872 read 0x40b7400 thru 0x40b9657 (0x2258 bytes) 2024-09-17T21:20:22.440 INFO:teuthology.orchestra.run.smithi184.stdout:4875 read 0x7253400 thru 0x7258a46 (0x5647 bytes) 2024-09-17T21:20:22.440 INFO:teuthology.orchestra.run.smithi184.stdout:4877 read 0x2703e00 thru 0x270417d (0x37e bytes) 2024-09-17T21:20:22.440 INFO:teuthology.orchestra.run.smithi184.stdout:4878 read 0x8289e00 thru 0x8291d6d (0x7f6e bytes) 2024-09-17T21:20:22.441 INFO:teuthology.orchestra.run.smithi184.stdout:4880 read 0x1cc00 thru 0x2756d (0xa96e bytes) 2024-09-17T21:20:22.441 INFO:teuthology.orchestra.run.smithi184.stdout:4885 write 0x418a00 thru 0x4226fe (0x9cff bytes) 2024-09-17T21:20:22.441 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:22.430+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:22.504 INFO:teuthology.orchestra.run.smithi184.stdout:4887 trunc from 0x895d400 to 0x1b24000 2024-09-17T21:20:22.539 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:22.533+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:22.637 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:22.637 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:22.639 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:22.634+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:22.654 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:20:22.744 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:20:22.745 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:22.741+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:22.761 INFO:teuthology.orchestra.run.smithi184.stdout:4893 trunc from 0x1b24000 to 0x43ff600 2024-09-17T21:20:22.774 INFO:teuthology.orchestra.run.smithi184.stdout:4895 read 0x314ba00 thru 0x314cfd0 (0x15d1 bytes) 2024-09-17T21:20:22.777 INFO:teuthology.orchestra.run.smithi184.stdout:4896 read 0x164a00 thru 0x17485b (0xfe5c bytes) 2024-09-17T21:20:22.818 INFO:teuthology.orchestra.run.smithi184.stdout:4897 read 0x6a4200 thru 0x6a55f0 (0x13f1 bytes) 2024-09-17T21:20:22.832 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:22.832 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:22.832 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:22.832 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:22.832 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:22.842 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:22.834+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:22.848 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:20:22.886 INFO:teuthology.orchestra.run.smithi184.stdout:4898 write 0x8741000 thru 0x874c040 (0xb041 bytes) 2024-09-17T21:20:22.887 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:22.889 INFO:teuthology.orchestra.run.smithi184.stdout:4902 read 0x5857600 thru 0x58621f7 (0xabf8 bytes) 2024-09-17T21:20:22.889 INFO:teuthology.orchestra.run.smithi184.stdout:4903 read 0x2064200 thru 0x206efc8 (0xadc9 bytes) 2024-09-17T21:20:22.904 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:22.904 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:22.922 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:22.922 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:22.940 INFO:teuthology.orchestra.run.smithi184.stdout:4904 write 0xe9f3800 thru 0xe9f502b (0x182c bytes) 2024-09-17T21:20:22.942 INFO:teuthology.orchestra.run.smithi184.stdout:4905 write 0x6937400 thru 0x693fe80 (0x8a81 bytes) 2024-09-17T21:20:22.943 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:22.934+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:22.943 INFO:teuthology.orchestra.run.smithi184.stdout:4908 read 0x6721600 thru 0x672999c (0x839d bytes) 2024-09-17T21:20:22.943 INFO:teuthology.orchestra.run.smithi184.stdout:4909 read 0x78f1400 thru 0x78ffff8 (0xebf9 bytes) 2024-09-17T21:20:22.943 INFO:teuthology.orchestra.run.smithi184.stdout:4911 write 0x4102e00 thru 0x4103a01 (0xc02 bytes) 2024-09-17T21:20:22.944 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:22.944 INFO:teuthology.orchestra.run.smithi184.stdout:4913 read 0x7f52000 thru 0x7f59967 (0x7968 bytes) 2024-09-17T21:20:22.944 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:22.965 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:22.965 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:22.966 INFO:teuthology.orchestra.run.smithi184.stdout:4914 trunc from 0xe9f5200 to 0x38a0400 2024-09-17T21:20:22.987 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:22.987 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:22.993 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:22.993 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:23.003 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:20:23.054 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:23.049+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:23.087 INFO:teuthology.orchestra.run.smithi184.stdout:4917 clone 33 order 23 su 1048576 sc 5 2024-09-17T21:20:23.145 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:23.138+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:23.170 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:23.170 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:23.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:20:23.245 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:23.241+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:23.306 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:23.306 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:23.320 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:20:23.328 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:20:23.347 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:23.342+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:23.448 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:23.442+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:23.487 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:23.487 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:23.503 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:20:23.541 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone32 from 0x38a0400 (overlap 0x1b24000) to 0xd92000 2024-09-17T21:20:23.548 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:23.541+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:23.649 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:23.642+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:23.658 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:23.659 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:23.673 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:20:23.750 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:23.745+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:23.846 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:23.847 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:23.851 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:23.846+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:23.863 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:20:23.952 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:23.945+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:24.053 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:24.049+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:24.076 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:24.076 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:24.094 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:20:24.153 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:24.146+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:24.254 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:24.246+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:24.257 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:24.258 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:24.277 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:20:24.339 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-17T21:20:24.364 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:24.358+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:24.457 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:24.453+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:24.460 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:24.460 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:24.475 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:20:24.558 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:24.553+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:24.644 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:24.644 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:24.658 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:20:24.659 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:24.650+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:24.759 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:24.754+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:24.826 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:24.827 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:24.841 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:20:24.859 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:24.853+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:24.961 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:24.954+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:25.009 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:25.009 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:25.012 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:20:25.023 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:20:25.061 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:25.057+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:25.100 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-17T21:20:25.162 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:25.157+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:25.165 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:25.165 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:25.182 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:25.182 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:25.195 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:25.195 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:25.198 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:25.199 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:25.209 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:20:25.220 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:25.220 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:25.239 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:25.239 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:25.258 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:25.258 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:25.263 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:25.258+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:25.326 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:25.326 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:25.326 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:25.326 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:25.340 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:20:25.364 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:25.358+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:25.364 INFO:teuthology.orchestra.run.smithi184.stdout:4922 write 0xb14e600 thru 0xb1579d7 (0x93d8 bytes) 2024-09-17T21:20:25.365 INFO:teuthology.orchestra.run.smithi184.stdout:4924 punch from 0x1bd4600 to 0x1bdf000, (0xaa00 bytes) 2024-09-17T21:20:25.465 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:25.461+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:25.468 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:25.468 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:25.468 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:25.468 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:25.468 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:25.482 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:20:25.565 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:25.558+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:25.656 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:25.656 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:25.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:25.656 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:25.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.656 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:525 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3670016~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:25.657 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:25.657 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 119.450852922, 2024-09-17T21:20:25.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046645697, 2024-09-17T21:20:25.657 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.657 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.657 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:25.657 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.657 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:25.657 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:25.657 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 525 2024-09-17T21:20:25.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.657 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:25.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:25.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:20:25.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:20:25.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200234+0000", 2024-09-17T21:20:25.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:20:25.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200254+0000", 2024-09-17T21:20:25.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9959999999999999e-05 2024-09-17T21:20:25.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200293+0000", 2024-09-17T21:20:25.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9755999999999997e-05 2024-09-17T21:20:25.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245871+0000", 2024-09-17T21:20:25.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045577871999999998 2024-09-17T21:20:25.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:25.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246030+0000", 2024-09-17T21:20:25.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158975 2024-09-17T21:20:25.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:25.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246871+0000", 2024-09-17T21:20:25.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084111699999999995 2024-09-17T21:20:25.661 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.661 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.661 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.661 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:25.661 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:25.661 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 119.45060724699999, 2024-09-17T21:20:25.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:20:25.661 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.661 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.661 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:25.662 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.662 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:25.662 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:25.662 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:20:25.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.662 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:25.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:25.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:20:25.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:20:25.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:20:25.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:20:25.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:20:25.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:20:25.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:20:25.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:20:25.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:20:25.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:20:25.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:25.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:20:25.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:20:25.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:25.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:20:25.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:20:25.665 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.665 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.665 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.666 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:25.666 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:25.666 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 119.450428565, 2024-09-17T21:20:25.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:20:25.666 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.666 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.666 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:25.666 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.666 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:25.666 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:25.666 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:20:25.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.666 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:25.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:25.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:20:25.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:20:25.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:20:25.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:20:25.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:20:25.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:20:25.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:20:25.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:20:25.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:20:25.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:20:25.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:25.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:20:25.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:20:25.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:25.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:20:25.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:20:25.669 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.670 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.670 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.670 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:25.670 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:25.670 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 119.450369292, 2024-09-17T21:20:25.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:20:25.670 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.670 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.670 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:25.670 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.670 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:25.671 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:25.671 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:20:25.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.671 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:25.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:25.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:20:25.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:20:25.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.672 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:20:25.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:20:25.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.672 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:20:25.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:20:25.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:20:25.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:20:25.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:20:25.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:20:25.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:25.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:20:25.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:20:25.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:25.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:20:25.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:20:25.674 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.674 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.674 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.674 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:25.674 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:25.674 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 119.450227097, 2024-09-17T21:20:25.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:20:25.674 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.674 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.674 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:25.675 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.675 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:25.675 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:25.675 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:20:25.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.675 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:25.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:25.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:20:25.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:20:25.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:20:25.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:20:25.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:20:25.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:20:25.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:20:25.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:20:25.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:20:25.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:20:25.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:25.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:20:25.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:20:25.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:25.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:20:25.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:20:25.678 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.678 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.678 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.678 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:25.678 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:25.678 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 119.450171174, 2024-09-17T21:20:25.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:20:25.679 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.679 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.679 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:25.679 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.679 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:25.679 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:25.679 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:20:25.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.679 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:25.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:25.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:20:25.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:20:25.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:20:25.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:20:25.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:20:25.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:20:25.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:20:25.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:20:25.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:20:25.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:20:25.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:25.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:20:25.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:20:25.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:25.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:20:25.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:20:25.682 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.682 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.682 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.683 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:25.683 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:25.683 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 119.450117557, 2024-09-17T21:20:25.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:20:25.683 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.683 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.683 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:25.683 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.683 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:25.683 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:25.683 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:20:25.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.684 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:25.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:25.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:20:25.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:20:25.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:20:25.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:20:25.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:20:25.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:20:25.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:20:25.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:20:25.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:20:25.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:20:25.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:25.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:20:25.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:20:25.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:25.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:20:25.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:20:25.687 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.687 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.687 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.687 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:20:25.687 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:25.687 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 69.140889314999995, 2024-09-17T21:20:25.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:20:25.687 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.687 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.687 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:20:25.687 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.687 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:20:25.688 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:20:25.688 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:20:25.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.688 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:25.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:25.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:20:25.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:20:25.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:20:25.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:20:25.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:20:25.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:20:25.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:20:25.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:20:25.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:20:25.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:20:25.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:25.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:20:25.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:20:25.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:20:25.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:20:25.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:25.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:20:25.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:20:25.691 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.691 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.691 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.692 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2597 3.1f 3:fb0e9615:::rbd_data.1152a8ae2019.000000000000000c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:25.692 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:25.692 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 44.812452886999999, 2024-09-17T21:20:25.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097064994000000002, 2024-09-17T21:20:25.692 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.692 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.692 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:25.692 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.692 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:25.692 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:25.692 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2597 2024-09-17T21:20:25.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.693 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:25.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:25.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838628+0000", 2024-09-17T21:20:25.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.187e-06 2024-09-17T21:20:25.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838654+0000", 2024-09-17T21:20:25.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:20:25.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838696+0000", 2024-09-17T21:20:25.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1226000000000001e-05 2024-09-17T21:20:25.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838715+0000", 2024-09-17T21:20:25.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9646000000000001e-05 2024-09-17T21:20:25.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838937+0000", 2024-09-17T21:20:25.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022163 2024-09-17T21:20:25.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:25.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.839284+0000", 2024-09-17T21:20:25.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034753799999999999 2024-09-17T21:20:25.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:25.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935691+0000", 2024-09-17T21:20:25.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.096406360999999996 2024-09-17T21:20:25.696 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.696 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.696 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.696 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2641 3.12 3:488a729f:::rbd_data.1152a8ae2019.0000000000000022:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:25.696 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:25.696 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 44.808996114000003, 2024-09-17T21:20:25.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098280953000000004, 2024-09-17T21:20:25.696 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.696 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.696 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:25.697 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.697 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:25.697 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:25.697 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2641 2024-09-17T21:20:25.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.697 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:25.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:25.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842083+0000", 2024-09-17T21:20:25.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5300000000000002e-07 2024-09-17T21:20:25.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842088+0000", 2024-09-17T21:20:25.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:20:25.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842105+0000", 2024-09-17T21:20:25.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6773e-05 2024-09-17T21:20:25.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842119+0000", 2024-09-17T21:20:25.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:20:25.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842320+0000", 2024-09-17T21:20:25.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020052700000000001 2024-09-17T21:20:25.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:25.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842845+0000", 2024-09-17T21:20:25.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052508600000000002 2024-09-17T21:20:25.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:25.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940363+0000", 2024-09-17T21:20:25.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097518795000000005 2024-09-17T21:20:25.700 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.700 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.700 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.701 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2674 3.12 3:4c87e3ac:::rbd_data.1152a8ae2019.0000000000000033:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:25.701 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:25.701 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 44.808633477000001, 2024-09-17T21:20:25.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098025125000000005, 2024-09-17T21:20:25.701 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.701 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.701 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:25.701 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.701 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:25.701 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:25.701 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2674 2024-09-17T21:20:25.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.701 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:25.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:25.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842447+0000", 2024-09-17T21:20:25.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3999999999999999e-06 2024-09-17T21:20:25.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842453+0000", 2024-09-17T21:20:25.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:20:25.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842472+0000", 2024-09-17T21:20:25.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9006000000000001e-05 2024-09-17T21:20:25.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842487+0000", 2024-09-17T21:20:25.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4866e-05 2024-09-17T21:20:25.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843594+0000", 2024-09-17T21:20:25.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011073249999999999 2024-09-17T21:20:25.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:25.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843668+0000", 2024-09-17T21:20:25.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3188000000000005e-05 2024-09-17T21:20:25.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:25.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843893+0000", 2024-09-17T21:20:25.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022503399999999999 2024-09-17T21:20:25.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:25.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850439+0000", 2024-09-17T21:20:25.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065467880000000004 2024-09-17T21:20:25.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:25.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940409+0000", 2024-09-17T21:20:25.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089969511000000002 2024-09-17T21:20:25.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:25.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940444+0000", 2024-09-17T21:20:25.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5345999999999997e-05 2024-09-17T21:20:25.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:25.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940470+0000", 2024-09-17T21:20:25.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5964000000000001e-05 2024-09-17T21:20:25.706 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.706 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.706 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.706 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2707 3.1f 3:f805e65e:::rbd_data.1152a8ae2019.0000000000000049:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:25.706 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:25.706 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 44.803586815999999, 2024-09-17T21:20:25.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088327300999999997, 2024-09-17T21:20:25.706 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.707 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.707 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:25.707 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.707 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:25.707 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:25.707 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2707 2024-09-17T21:20:25.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.707 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:25.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:25.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847494+0000", 2024-09-17T21:20:25.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:20:25.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847504+0000", 2024-09-17T21:20:25.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:20:25.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847526+0000", 2024-09-17T21:20:25.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1976000000000001e-05 2024-09-17T21:20:25.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847541+0000", 2024-09-17T21:20:25.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5277e-05 2024-09-17T21:20:25.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856142+0000", 2024-09-17T21:20:25.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086006710000000007 2024-09-17T21:20:25.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:25.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856435+0000", 2024-09-17T21:20:25.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029301499999999999 2024-09-17T21:20:25.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:25.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935819+0000", 2024-09-17T21:20:25.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079384448999999996 2024-09-17T21:20:25.710 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.710 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.710 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.710 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2730 3.12 3:48b3422a:::rbd_data.1152a8ae2019.0000000000000057:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:25.711 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:25.711 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 44.794393917000001, 2024-09-17T21:20:25.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083879975999999995, 2024-09-17T21:20:25.711 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.711 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.711 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:25.711 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.711 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:25.711 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:25.711 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2730 2024-09-17T21:20:25.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.711 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:25.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:25.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856687+0000", 2024-09-17T21:20:25.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:20:25.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856697+0000", 2024-09-17T21:20:25.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0207000000000001e-05 2024-09-17T21:20:25.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856808+0000", 2024-09-17T21:20:25.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011071200000000001 2024-09-17T21:20:25.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856828+0000", 2024-09-17T21:20:25.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0219000000000001e-05 2024-09-17T21:20:25.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856907+0000", 2024-09-17T21:20:25.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9661999999999997e-05 2024-09-17T21:20:25.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:25.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.857019+0000", 2024-09-17T21:20:25.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011148499999999999 2024-09-17T21:20:25.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:25.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940565+0000", 2024-09-17T21:20:25.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083545791999999994 2024-09-17T21:20:25.714 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.714 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.714 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.715 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2743 3.12 3:4d036e62:::rbd_data.1152a8ae2019.000000000000005f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:25.715 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:25.715 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 44.790448415999997, 2024-09-17T21:20:25.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080059962999999998, 2024-09-17T21:20:25.715 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.715 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.715 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:25.715 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.715 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:25.715 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:25.715 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2743 2024-09-17T21:20:25.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.715 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:25.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:25.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860631+0000", 2024-09-17T21:20:25.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:20:25.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860636+0000", 2024-09-17T21:20:25.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5279999999999997e-06 2024-09-17T21:20:25.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860649+0000", 2024-09-17T21:20:25.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-17T21:20:25.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860655+0000", 2024-09-17T21:20:25.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:20:25.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860731+0000", 2024-09-17T21:20:25.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5920000000000005e-05 2024-09-17T21:20:25.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:25.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.861070+0000", 2024-09-17T21:20:25.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338535 2024-09-17T21:20:25.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:25.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940690+0000", 2024-09-17T21:20:25.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079620503999999995 2024-09-17T21:20:25.719 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.719 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.719 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.719 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3378 3.12 3:4832a51b:::rbd_data.1152a8ae2019.000000000000006c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:25.719 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:25.719 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 44.762084565999999, 2024-09-17T21:20:25.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051812516000000003, 2024-09-17T21:20:25.719 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.720 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.720 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:25.720 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.720 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:25.720 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:25.720 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3378 2024-09-17T21:20:25.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.720 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.720 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:25.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888995+0000", 2024-09-17T21:20:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:20:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889006+0000", 2024-09-17T21:20:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1012999999999999e-05 2024-09-17T21:20:25.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889032+0000", 2024-09-17T21:20:25.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6441e-05 2024-09-17T21:20:25.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889040+0000", 2024-09-17T21:20:25.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0639999999999994e-06 2024-09-17T21:20:25.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889278+0000", 2024-09-17T21:20:25.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023785899999999999 2024-09-17T21:20:25.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:25.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889408+0000", 2024-09-17T21:20:25.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129219 2024-09-17T21:20:25.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:25.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889723+0000", 2024-09-17T21:20:25.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000315264 2024-09-17T21:20:25.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:25.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.891972+0000", 2024-09-17T21:20:25.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002249633 2024-09-17T21:20:25.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:25.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940733+0000", 2024-09-17T21:20:25.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048760626000000001 2024-09-17T21:20:25.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:25.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940785+0000", 2024-09-17T21:20:25.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1906000000000001e-05 2024-09-17T21:20:25.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:25.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940807+0000", 2024-09-17T21:20:25.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1554999999999999e-05 2024-09-17T21:20:25.725 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.725 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.725 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.725 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3379 3.12 3:4e83fd71:::rbd_data.1152a8ae2019.000000000000006d:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:25.725 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:25.725 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 44.762000358999998, 2024-09-17T21:20:25.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051804144000000003, 2024-09-17T21:20:25.725 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.725 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.726 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:25.726 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.726 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:25.726 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:25.726 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3379 2024-09-17T21:20:25.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.726 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:25.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:25.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889079+0000", 2024-09-17T21:20:25.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1800000000000004e-07 2024-09-17T21:20:25.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889083+0000", 2024-09-17T21:20:25.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:20:25.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889094+0000", 2024-09-17T21:20:25.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1562999999999999e-05 2024-09-17T21:20:25.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889102+0000", 2024-09-17T21:20:25.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.401e-06 2024-09-17T21:20:25.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:20:25.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014279259999999999 2024-09-17T21:20:25.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:25.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890609+0000", 2024-09-17T21:20:25.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9566000000000004e-05 2024-09-17T21:20:25.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:25.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890789+0000", 2024-09-17T21:20:25.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017949700000000001 2024-09-17T21:20:25.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:25.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.893929+0000", 2024-09-17T21:20:25.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003140538 2024-09-17T21:20:25.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:25.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940834+0000", 2024-09-17T21:20:25.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046904811999999997 2024-09-17T21:20:25.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:25.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940860+0000", 2024-09-17T21:20:25.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5341999999999999e-05 2024-09-17T21:20:25.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:25.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940882+0000", 2024-09-17T21:20:25.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2878999999999999e-05 2024-09-17T21:20:25.731 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.731 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.731 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.731 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:2644 3.1e 3:79615585:::rbd_data.115266d0b9f2.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2475520~14848,write 3106304~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:20:25.731 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:25.731 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 36.139371955999998, 2024-09-17T21:20:25.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050733484000000002, 2024-09-17T21:20:25.732 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.732 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.732 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:25.732 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.732 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:20:25.732 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:20:25.732 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2644 2024-09-17T21:20:25.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.732 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:25.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:25.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511708+0000", 2024-09-17T21:20:25.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-17T21:20:25.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522196+0000", 2024-09-17T21:20:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010487712999999999 2024-09-17T21:20:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522223+0000", 2024-09-17T21:20:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7064999999999999e-05 2024-09-17T21:20:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522236+0000", 2024-09-17T21:20:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2931e-05 2024-09-17T21:20:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522348+0000", 2024-09-17T21:20:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111906 2024-09-17T21:20:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522554+0000", 2024-09-17T21:20:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020632699999999999 2024-09-17T21:20:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:20:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.523185+0000", 2024-09-17T21:20:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063081400000000005 2024-09-17T21:20:25.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.547011+0000", 2024-09-17T21:20:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023825570000000001 2024-09-17T21:20:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562268+0000", 2024-09-17T21:20:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015257514 2024-09-17T21:20:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562367+0000", 2024-09-17T21:20:25.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8678000000000006e-05 2024-09-17T21:20:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562440+0000", 2024-09-17T21:20:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3337000000000006e-05 2024-09-17T21:20:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:231 3.1 3:856fed78:::rbd_data.115220dd393e.000000000000000b:head [read 0~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:20:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:20:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 7.6966166869999997, 2024-09-17T21:20:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34254251699999999, 2024-09-17T21:20:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.737 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:20:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 231 2024-09-17T21:20:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:20:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:20:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954464+0000", 2024-09-17T21:20:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:20:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954475+0000", 2024-09-17T21:20:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1146e-05 2024-09-17T21:20:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954506+0000", 2024-09-17T21:20:25.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0292999999999999e-05 2024-09-17T21:20:25.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954524+0000", 2024-09-17T21:20:25.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8430000000000001e-05 2024-09-17T21:20:25.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954661+0000", 2024-09-17T21:20:25.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136403 2024-09-17T21:20:25.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954680+0000", 2024-09-17T21:20:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9743000000000001e-05 2024-09-17T21:20:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.296896+0000", 2024-09-17T21:20:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34221550499999998 2024-09-17T21:20:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297004+0000", 2024-09-17T21:20:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010865399999999999 2024-09-17T21:20:25.742 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.742 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.742 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.742 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:56 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:20:25.742 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:20:25.742 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 7.5409829899999998, 2024-09-17T21:20:25.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18718773899999999, 2024-09-17T21:20:25.742 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.742 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.742 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:25.742 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.742 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:20:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:20:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 56 2024-09-17T21:20:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:20:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:20:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110098+0000", 2024-09-17T21:20:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:20:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110111+0000", 2024-09-17T21:20:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2289999999999999e-05 2024-09-17T21:20:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110143+0000", 2024-09-17T21:20:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1979e-05 2024-09-17T21:20:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110161+0000", 2024-09-17T21:20:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7839e-05 2024-09-17T21:20:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110319+0000", 2024-09-17T21:20:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015835400000000001 2024-09-17T21:20:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110353+0000", 2024-09-17T21:20:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3849000000000001e-05 2024-09-17T21:20:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297041+0000", 2024-09-17T21:20:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18668842299999999 2024-09-17T21:20:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297169+0000", 2024-09-17T21:20:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012786199999999999 2024-09-17T21:20:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297283+0000", 2024-09-17T21:20:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011430300000000001 2024-09-17T21:20:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.746 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:68 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:20:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:20:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 7.5324420590000001, 2024-09-17T21:20:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17885582999999999, 2024-09-17T21:20:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:20:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:20:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 68 2024-09-17T21:20:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.747 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:20:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:20:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118639+0000", 2024-09-17T21:20:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.306e-06 2024-09-17T21:20:25.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118651+0000", 2024-09-17T21:20:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2567e-05 2024-09-17T21:20:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118682+0000", 2024-09-17T21:20:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0502e-05 2024-09-17T21:20:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118707+0000", 2024-09-17T21:20:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5463e-05 2024-09-17T21:20:25.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118889+0000", 2024-09-17T21:20:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018150999999999999 2024-09-17T21:20:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118910+0000", 2024-09-17T21:20:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:20:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297313+0000", 2024-09-17T21:20:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17840366699999999 2024-09-17T21:20:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297389+0000", 2024-09-17T21:20:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-17T21:20:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297492+0000", 2024-09-17T21:20:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103019 2024-09-17T21:20:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.751 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.751 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:25.753 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:20:25.753 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:25.749+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:25.796 INFO:teuthology.orchestra.run.smithi184.stdout:4925 write 0xb010e00 thru 0xb016569 (0x576a bytes) 2024-09-17T21:20:25.808 INFO:teuthology.orchestra.run.smithi184.stdout:4926 write 0x8470200 thru 0x847bc7e (0xba7f bytes) 2024-09-17T21:20:25.808 INFO:teuthology.orchestra.run.smithi184.stdout:4929 punch from 0x9274800 to 0x927f000, (0xa800 bytes) 2024-09-17T21:20:25.809 INFO:teuthology.orchestra.run.smithi184.stdout:4931 read 0x45c8800 thru 0x45cc98a (0x418b bytes) 2024-09-17T21:20:25.809 INFO:teuthology.orchestra.run.smithi184.stdout:4932 punch from 0x267d600 to 0x267fe00, (0x2800 bytes) 2024-09-17T21:20:25.854 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:25.849+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:25.929 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5590.0:5941 3.2 3.c185cfc2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:25.910773+0000", 2024-09-17T21:20:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.015101292000000001, 2024-09-17T21:20:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015138795, 2024-09-17T21:20:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5590", 2024-09-17T21:20:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 5941 2024-09-17T21:20:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.910773+0000", 2024-09-17T21:20:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.910773+0000", 2024-09-17T21:20:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.910775+0000", 2024-09-17T21:20:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7770000000000001e-06 2024-09-17T21:20:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.910783+0000", 2024-09-17T21:20:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000004e-06 2024-09-17T21:20:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.910802+0000", 2024-09-17T21:20:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9816000000000001e-05 2024-09-17T21:20:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.910822+0000", 2024-09-17T21:20:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9188999999999999e-05 2024-09-17T21:20:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5590.0:5943 3.2 3.c185cfc2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:25.911093+0000", 2024-09-17T21:20:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.014781107999999999, 2024-09-17T21:20:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014913938999999999, 2024-09-17T21:20:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5590", 2024-09-17T21:20:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 5943 2024-09-17T21:20:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.911093+0000", 2024-09-17T21:20:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.911093+0000", 2024-09-17T21:20:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.911094+0000", 2024-09-17T21:20:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.127e-06 2024-09-17T21:20:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.911100+0000", 2024-09-17T21:20:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1660000000000002e-06 2024-09-17T21:20:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.911114+0000", 2024-09-17T21:20:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4796e-05 2024-09-17T21:20:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.911124+0000", 2024-09-17T21:20:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8099999999999992e-06 2024-09-17T21:20:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5590.0:5944 3.2 3.c185cfc2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:25.911639+0000", 2024-09-17T21:20:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.014235209, 2024-09-17T21:20:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014440662, 2024-09-17T21:20:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5590", 2024-09-17T21:20:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 5944 2024-09-17T21:20:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.911639+0000", 2024-09-17T21:20:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.911639+0000", 2024-09-17T21:20:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.911641+0000", 2024-09-17T21:20:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0470000000000001e-06 2024-09-17T21:20:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.911650+0000", 2024-09-17T21:20:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7430000000000008e-06 2024-09-17T21:20:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.911677+0000", 2024-09-17T21:20:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6669000000000002e-05 2024-09-17T21:20:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.911693+0000", 2024-09-17T21:20:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6520000000000001e-05 2024-09-17T21:20:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5590.0:5946 3.2 3.c185cfc2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:25.911886+0000", 2024-09-17T21:20:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.013988674, 2024-09-17T21:20:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014265475999999999, 2024-09-17T21:20:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5590", 2024-09-17T21:20:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 5946 2024-09-17T21:20:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.911886+0000", 2024-09-17T21:20:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.911886+0000", 2024-09-17T21:20:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.911887+0000", 2024-09-17T21:20:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.373e-06 2024-09-17T21:20:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.911893+0000", 2024-09-17T21:20:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6869999999999999e-06 2024-09-17T21:20:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.911906+0000", 2024-09-17T21:20:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3243e-05 2024-09-17T21:20:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.911916+0000", 2024-09-17T21:20:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6090000000000007e-06 2024-09-17T21:20:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5590.0:5948 3.2 3.c185cfc2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:25.912334+0000", 2024-09-17T21:20:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.013540653, 2024-09-17T21:20:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013886526, 2024-09-17T21:20:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5590", 2024-09-17T21:20:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 5948 2024-09-17T21:20:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.912334+0000", 2024-09-17T21:20:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.912334+0000", 2024-09-17T21:20:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.912335+0000", 2024-09-17T21:20:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6169999999999999e-06 2024-09-17T21:20:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.912342+0000", 2024-09-17T21:20:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.99e-06 2024-09-17T21:20:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.912363+0000", 2024-09-17T21:20:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0696e-05 2024-09-17T21:20:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.912382+0000", 2024-09-17T21:20:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8502999999999999e-05 2024-09-17T21:20:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5590.0:5950 3.2 3.c185cfc2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:25.912681+0000", 2024-09-17T21:20:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.013193613, 2024-09-17T21:20:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013612828, 2024-09-17T21:20:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5590", 2024-09-17T21:20:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 5950 2024-09-17T21:20:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.912681+0000", 2024-09-17T21:20:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.912681+0000", 2024-09-17T21:20:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.912683+0000", 2024-09-17T21:20:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.95e-06 2024-09-17T21:20:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.912691+0000", 2024-09-17T21:20:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.926e-06 2024-09-17T21:20:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.912711+0000", 2024-09-17T21:20:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9894e-05 2024-09-17T21:20:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.912727+0000", 2024-09-17T21:20:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6382e-05 2024-09-17T21:20:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5590.0:5952 3.2 3.c185cfc2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:25.912828+0000", 2024-09-17T21:20:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.01304638, 2024-09-17T21:20:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013534581, 2024-09-17T21:20:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5590", 2024-09-17T21:20:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 5952 2024-09-17T21:20:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.912828+0000", 2024-09-17T21:20:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.912828+0000", 2024-09-17T21:20:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.912829+0000", 2024-09-17T21:20:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.164e-06 2024-09-17T21:20:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.912834+0000", 2024-09-17T21:20:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2299999999999999e-06 2024-09-17T21:20:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.912848+0000", 2024-09-17T21:20:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3406e-05 2024-09-17T21:20:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.912857+0000", 2024-09-17T21:20:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3840000000000003e-06 2024-09-17T21:20:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5590.0:5955 3.2 3.c185cfc2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:25.912986+0000", 2024-09-17T21:20:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.012888359, 2024-09-17T21:20:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013468160999999999, 2024-09-17T21:20:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5590", 2024-09-17T21:20:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 5955 2024-09-17T21:20:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.912986+0000", 2024-09-17T21:20:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.912986+0000", 2024-09-17T21:20:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.912987+0000", 2024-09-17T21:20:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0529999999999999e-06 2024-09-17T21:20:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.912991+0000", 2024-09-17T21:20:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8430000000000003e-06 2024-09-17T21:20:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913002+0000", 2024-09-17T21:20:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.116e-05 2024-09-17T21:20:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913010+0000", 2024-09-17T21:20:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.354e-06 2024-09-17T21:20:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5590.0:5956 3.2 3.c185cfc2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:25.913027+0000", 2024-09-17T21:20:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.012847137, 2024-09-17T21:20:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013499041, 2024-09-17T21:20:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5590", 2024-09-17T21:20:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 5956 2024-09-17T21:20:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913027+0000", 2024-09-17T21:20:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913027+0000", 2024-09-17T21:20:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913028+0000", 2024-09-17T21:20:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2699999999999999e-07 2024-09-17T21:20:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913031+0000", 2024-09-17T21:20:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3459999999999998e-06 2024-09-17T21:20:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913041+0000", 2024-09-17T21:20:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2900000000000008e-06 2024-09-17T21:20:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913050+0000", 2024-09-17T21:20:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9439999999999994e-06 2024-09-17T21:20:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5590.0:5958 3.2 3.c185cfc2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:25.913060+0000", 2024-09-17T21:20:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.01281449, 2024-09-17T21:20:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013539933000000001, 2024-09-17T21:20:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5590", 2024-09-17T21:20:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 5958 2024-09-17T21:20:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913060+0000", 2024-09-17T21:20:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913060+0000", 2024-09-17T21:20:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913061+0000", 2024-09-17T21:20:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5600000000000005e-07 2024-09-17T21:20:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913063+0000", 2024-09-17T21:20:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8040000000000002e-06 2024-09-17T21:20:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913072+0000", 2024-09-17T21:20:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5960000000000004e-06 2024-09-17T21:20:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913079+0000", 2024-09-17T21:20:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.046e-06 2024-09-17T21:20:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5590.0:5960 3.2 3.c185cfc2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:25.913099+0000", 2024-09-17T21:20:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.012775135, 2024-09-17T21:20:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013569975999999999, 2024-09-17T21:20:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5590", 2024-09-17T21:20:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 5960 2024-09-17T21:20:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913099+0000", 2024-09-17T21:20:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913099+0000", 2024-09-17T21:20:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913100+0000", 2024-09-17T21:20:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2399999999999997e-07 2024-09-17T21:20:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913104+0000", 2024-09-17T21:20:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6260000000000001e-06 2024-09-17T21:20:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913113+0000", 2024-09-17T21:20:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2839999999999995e-06 2024-09-17T21:20:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913120+0000", 2024-09-17T21:20:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1790000000000002e-06 2024-09-17T21:20:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5590.0:5962 3.2 3.c185cfc2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:25.913141+0000", 2024-09-17T21:20:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.012733642, 2024-09-17T21:20:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013596898, 2024-09-17T21:20:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5590", 2024-09-17T21:20:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 5962 2024-09-17T21:20:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913141+0000", 2024-09-17T21:20:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913141+0000", 2024-09-17T21:20:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913141+0000", 2024-09-17T21:20:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1299999999999999e-07 2024-09-17T21:20:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913145+0000", 2024-09-17T21:20:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6069999999999999e-06 2024-09-17T21:20:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913154+0000", 2024-09-17T21:20:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9730000000000003e-06 2024-09-17T21:20:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913161+0000", 2024-09-17T21:20:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0230000000000004e-06 2024-09-17T21:20:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5590.0:5964 3.2 3.c185cfc2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:25.913234+0000", 2024-09-17T21:20:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.012640874, 2024-09-17T21:20:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013572669000000001, 2024-09-17T21:20:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5590", 2024-09-17T21:20:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 5964 2024-09-17T21:20:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913234+0000", 2024-09-17T21:20:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913234+0000", 2024-09-17T21:20:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913235+0000", 2024-09-17T21:20:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.59e-06 2024-09-17T21:20:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913242+0000", 2024-09-17T21:20:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3729999999999998e-06 2024-09-17T21:20:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913262+0000", 2024-09-17T21:20:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0227000000000001e-05 2024-09-17T21:20:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913282+0000", 2024-09-17T21:20:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0591999999999999e-05 2024-09-17T21:20:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5590.0:5966 3.2 3.c185cfc2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:25.913323+0000", 2024-09-17T21:20:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.012551292, 2024-09-17T21:20:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013556322000000001, 2024-09-17T21:20:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5590", 2024-09-17T21:20:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 5966 2024-09-17T21:20:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913323+0000", 2024-09-17T21:20:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913323+0000", 2024-09-17T21:20:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913324+0000", 2024-09-17T21:20:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.22e-06 2024-09-17T21:20:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913330+0000", 2024-09-17T21:20:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0290000000000002e-06 2024-09-17T21:20:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913367+0000", 2024-09-17T21:20:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6687000000000003e-05 2024-09-17T21:20:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913382+0000", 2024-09-17T21:20:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4752e-05 2024-09-17T21:20:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5590.0:5968 3.2 3.c185cfc2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:25.913417+0000", 2024-09-17T21:20:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.012457074, 2024-09-17T21:20:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013533096, 2024-09-17T21:20:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5590", 2024-09-17T21:20:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 5968 2024-09-17T21:20:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913417+0000", 2024-09-17T21:20:25.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913417+0000", 2024-09-17T21:20:25.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913419+0000", 2024-09-17T21:20:25.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.257e-06 2024-09-17T21:20:25.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913424+0000", 2024-09-17T21:20:25.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7089999999999999e-06 2024-09-17T21:20:25.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913445+0000", 2024-09-17T21:20:25.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0602999999999999e-05 2024-09-17T21:20:25.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913461+0000", 2024-09-17T21:20:25.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5999999999999999e-05 2024-09-17T21:20:25.977 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.977 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.977 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.978 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5590.0:5970 3.2 3.c185cfc2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:25.978 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:25.913500+0000", 2024-09-17T21:20:25.978 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.012374039, 2024-09-17T21:20:25.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013518789999999999, 2024-09-17T21:20:25.978 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.978 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.978 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:25.978 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.978 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5590", 2024-09-17T21:20:25.978 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:25.978 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 5970 2024-09-17T21:20:25.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.978 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913500+0000", 2024-09-17T21:20:25.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913500+0000", 2024-09-17T21:20:25.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913502+0000", 2024-09-17T21:20:25.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6130000000000001e-06 2024-09-17T21:20:25.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913507+0000", 2024-09-17T21:20:25.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4970000000000001e-06 2024-09-17T21:20:25.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913525+0000", 2024-09-17T21:20:25.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7793000000000001e-05 2024-09-17T21:20:25.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913539+0000", 2024-09-17T21:20:25.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3509000000000001e-05 2024-09-17T21:20:25.981 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.981 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.981 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.981 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5590.0:5972 3.2 3.c185cfc2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:25.981 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:25.913567+0000", 2024-09-17T21:20:25.981 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.012307624, 2024-09-17T21:20:25.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013520953, 2024-09-17T21:20:25.981 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.981 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.981 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:25.981 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.981 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5590", 2024-09-17T21:20:25.982 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:25.982 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 5972 2024-09-17T21:20:25.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.982 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913567+0000", 2024-09-17T21:20:25.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913567+0000", 2024-09-17T21:20:25.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913568+0000", 2024-09-17T21:20:25.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.697e-06 2024-09-17T21:20:25.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913574+0000", 2024-09-17T21:20:25.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8599999999999998e-06 2024-09-17T21:20:25.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913592+0000", 2024-09-17T21:20:25.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7561999999999999e-05 2024-09-17T21:20:25.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913611+0000", 2024-09-17T21:20:25.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9074000000000001e-05 2024-09-17T21:20:25.984 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.984 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.984 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.984 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5590.0:5974 3.2 3.c185cfc2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:25.984 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:25.913649+0000", 2024-09-17T21:20:25.985 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.012225596, 2024-09-17T21:20:25.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013508841000000001, 2024-09-17T21:20:25.985 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.985 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.985 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:25.985 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.985 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5590", 2024-09-17T21:20:25.985 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:25.985 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 5974 2024-09-17T21:20:25.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.985 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913649+0000", 2024-09-17T21:20:25.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913649+0000", 2024-09-17T21:20:25.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913650+0000", 2024-09-17T21:20:25.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-06 2024-09-17T21:20:25.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913656+0000", 2024-09-17T21:20:25.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2439999999999998e-06 2024-09-17T21:20:25.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913675+0000", 2024-09-17T21:20:25.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8433000000000001e-05 2024-09-17T21:20:25.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913691+0000", 2024-09-17T21:20:25.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6239000000000002e-05 2024-09-17T21:20:25.987 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.987 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.987 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.987 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5590.0:5976 3.2 3.c185cfc2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:25.988 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:25.913714+0000", 2024-09-17T21:20:25.988 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.012160260000000001, 2024-09-17T21:20:25.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013517817, 2024-09-17T21:20:25.988 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.988 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.988 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:25.988 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.988 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5590", 2024-09-17T21:20:25.988 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:25.988 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 5976 2024-09-17T21:20:25.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.988 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913714+0000", 2024-09-17T21:20:25.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913714+0000", 2024-09-17T21:20:25.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913715+0000", 2024-09-17T21:20:25.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.11e-06 2024-09-17T21:20:25.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913721+0000", 2024-09-17T21:20:25.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4870000000000002e-06 2024-09-17T21:20:25.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913739+0000", 2024-09-17T21:20:25.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8099000000000001e-05 2024-09-17T21:20:25.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913755+0000", 2024-09-17T21:20:25.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6520000000000001e-05 2024-09-17T21:20:25.990 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.990 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.991 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.991 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5590.0:5978 3.2 3.c185cfc2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:25.991 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:25.913789+0000", 2024-09-17T21:20:25.991 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.012085065000000001, 2024-09-17T21:20:25.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013515597000000001, 2024-09-17T21:20:25.991 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.991 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.991 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:25.991 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.991 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5590", 2024-09-17T21:20:25.991 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:25.991 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 5978 2024-09-17T21:20:25.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.992 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913789+0000", 2024-09-17T21:20:25.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913789+0000", 2024-09-17T21:20:25.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913791+0000", 2024-09-17T21:20:25.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3030000000000001e-06 2024-09-17T21:20:25.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913797+0000", 2024-09-17T21:20:25.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0070000000000003e-06 2024-09-17T21:20:25.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913819+0000", 2024-09-17T21:20:25.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2382999999999999e-05 2024-09-17T21:20:25.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913836+0000", 2024-09-17T21:20:25.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7136e-05 2024-09-17T21:20:25.994 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.994 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.994 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.994 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5590.0:5980 3.2 3.c185cfc2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:25.994 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:25.913889+0000", 2024-09-17T21:20:25.994 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.011985060000000001, 2024-09-17T21:20:25.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.01349822, 2024-09-17T21:20:25.994 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:25.994 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:25.994 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:25.994 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:25.995 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5590", 2024-09-17T21:20:25.995 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:25.995 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 5980 2024-09-17T21:20:25.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.995 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:25.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:25.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913889+0000", 2024-09-17T21:20:25.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:25.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913889+0000", 2024-09-17T21:20:25.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:25.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:25.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913890+0000", 2024-09-17T21:20:25.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5600000000000005e-07 2024-09-17T21:20:25.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:25.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913894+0000", 2024-09-17T21:20:25.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8340000000000001e-06 2024-09-17T21:20:25.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:25.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913905+0000", 2024-09-17T21:20:25.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0993e-05 2024-09-17T21:20:25.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:25.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:25.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:25.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:25.913913+0000", 2024-09-17T21:20:25.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0299999999999994e-06 2024-09-17T21:20:25.997 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.997 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:25.997 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.997 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:25.997 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:20:25.997 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 21 2024-09-17T21:20:25.997 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:25.998 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:20:25.999 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:25.990+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:26.100 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:26.093+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:26.184 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:26.184 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:26.184 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:26.185 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:26.185 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:26.201 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:20:26.201 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:26.194+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:26.302 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:26.297+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:26.369 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:20:26.371 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:26.371 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:26.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:26.371 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:26.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.371 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:20:26.371 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:26.371 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 106.55659598699999, 2024-09-17T21:20:26.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:20:26.371 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.371 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.371 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:26.372 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:26.372 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:20:26.372 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:20:26.372 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:20:26.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.372 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:26.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:26.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:20:26.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:20:26.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:20:26.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:20:26.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:20:26.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:20:26.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:20:26.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:20:26.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:20:26.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:20:26.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:26.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:20:26.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:20:26.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:20:26.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:20:26.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:20:26.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:20:26.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:20:26.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:20:26.376 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.376 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.376 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.376 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:20:26.377 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:26.377 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 93.001494201, 2024-09-17T21:20:26.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:20:26.377 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.377 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.377 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:26.377 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:26.377 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:20:26.377 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:26.377 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:20:26.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.377 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:26.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:26.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:20:26.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:20:26.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:20:26.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:20:26.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:20:26.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:20:26.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:20:26.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:20:26.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:20:26.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:20:26.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:26.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:20:26.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:20:26.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:20:26.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:20:26.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:20:26.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:20:26.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:26.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:20:26.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:20:26.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:26.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:20:26.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:20:26.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:26.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:20:26.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:20:26.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:26.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:20:26.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:20:26.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:20:26.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:20:26.383 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.383 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.383 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.383 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:793 3.9 3:95f63599:::rbd_data.1152eb799094.0000000000000007:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:20:26.383 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:20:26.383 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.9344380460000004, 2024-09-17T21:20:26.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88245766400000003, 2024-09-17T21:20:26.383 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.383 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.384 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:26.384 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:26.384 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:20:26.384 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:20:26.384 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 793 2024-09-17T21:20:26.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.384 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:20:26.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:20:26.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431941+0000", 2024-09-17T21:20:26.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.864e-06 2024-09-17T21:20:26.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431953+0000", 2024-09-17T21:20:26.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1486e-05 2024-09-17T21:20:26.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431984+0000", 2024-09-17T21:20:26.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1075999999999999e-05 2024-09-17T21:20:26.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431999+0000", 2024-09-17T21:20:26.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.503e-05 2024-09-17T21:20:26.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432095+0000", 2024-09-17T21:20:26.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5698000000000004e-05 2024-09-17T21:20:26.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:26.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432109+0000", 2024-09-17T21:20:26.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4188e-05 2024-09-17T21:20:26.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.309642+0000", 2024-09-17T21:20:26.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87753353000000001 2024-09-17T21:20:26.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310104+0000", 2024-09-17T21:20:26.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046103600000000001 2024-09-17T21:20:26.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:26.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310630+0000", 2024-09-17T21:20:26.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052682800000000002 2024-09-17T21:20:26.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:26.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313100+0000", 2024-09-17T21:20:26.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024693369999999998 2024-09-17T21:20:26.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:26.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313935+0000", 2024-09-17T21:20:26.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083540100000000005 2024-09-17T21:20:26.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:26.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314086+0000", 2024-09-17T21:20:26.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015053399999999999 2024-09-17T21:20:26.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314396+0000", 2024-09-17T21:20:26.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031065599999999997 2024-09-17T21:20:26.390 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.390 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.390 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.390 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:800 3.19 3:9dc17f12:::rbd_data.1152eb799094.000000000000000e:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:20:26.390 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:20:26.390 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.9341908009999997, 2024-09-17T21:20:26.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88154663799999999, 2024-09-17T21:20:26.390 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.390 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.390 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:26.390 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:26.391 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:20:26.391 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:20:26.391 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 800 2024-09-17T21:20:26.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.391 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:20:26.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:20:26.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432187+0000", 2024-09-17T21:20:26.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4100000000000001e-06 2024-09-17T21:20:26.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432194+0000", 2024-09-17T21:20:26.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0389999999999999e-06 2024-09-17T21:20:26.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432216+0000", 2024-09-17T21:20:26.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1614e-05 2024-09-17T21:20:26.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432232+0000", 2024-09-17T21:20:26.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5739e-05 2024-09-17T21:20:26.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432405+0000", 2024-09-17T21:20:26.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017355300000000001 2024-09-17T21:20:26.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:26.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432432+0000", 2024-09-17T21:20:26.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6979e-05 2024-09-17T21:20:26.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.307767+0000", 2024-09-17T21:20:26.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87533472899999998 2024-09-17T21:20:26.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308192+0000", 2024-09-17T21:20:26.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042485999999999999 2024-09-17T21:20:26.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:26.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308727+0000", 2024-09-17T21:20:26.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00053550400000000001 2024-09-17T21:20:26.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:26.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310347+0000", 2024-09-17T21:20:26.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016198670000000001 2024-09-17T21:20:26.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:26.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313424+0000", 2024-09-17T21:20:26.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030764640000000001 2024-09-17T21:20:26.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:26.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313510+0000", 2024-09-17T21:20:26.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5938000000000003e-05 2024-09-17T21:20:26.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313733+0000", 2024-09-17T21:20:26.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022294200000000001 2024-09-17T21:20:26.396 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.396 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.396 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.397 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:805 3.19 3:9cab2ed6:::rbd_data.1152eb799094.0000000000000013:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:20:26.397 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:20:26.397 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.9339867819999998, 2024-09-17T21:20:26.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88274355699999996, 2024-09-17T21:20:26.397 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.397 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.397 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:26.397 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:26.397 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:20:26.397 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:20:26.397 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 805 2024-09-17T21:20:26.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.398 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:20:26.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:20:26.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432391+0000", 2024-09-17T21:20:26.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3e-06 2024-09-17T21:20:26.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432398+0000", 2024-09-17T21:20:26.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4489999999999996e-06 2024-09-17T21:20:26.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432415+0000", 2024-09-17T21:20:26.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7750000000000001e-05 2024-09-17T21:20:26.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432430+0000", 2024-09-17T21:20:26.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.449e-05 2024-09-17T21:20:26.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432635+0000", 2024-09-17T21:20:26.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205031 2024-09-17T21:20:26.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:26.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432657+0000", 2024-09-17T21:20:26.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2064000000000001e-05 2024-09-17T21:20:26.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310444+0000", 2024-09-17T21:20:26.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87778699199999999 2024-09-17T21:20:26.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310849+0000", 2024-09-17T21:20:26.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00040497800000000001 2024-09-17T21:20:26.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.311396+0000", 2024-09-17T21:20:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00054729099999999999 2024-09-17T21:20:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313847+0000", 2024-09-17T21:20:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024504940000000001 2024-09-17T21:20:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314879+0000", 2024-09-17T21:20:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001031817 2024-09-17T21:20:26.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:26.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314964+0000", 2024-09-17T21:20:26.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5685000000000004e-05 2024-09-17T21:20:26.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.315133+0000", 2024-09-17T21:20:26.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000169216 2024-09-17T21:20:26.403 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.403 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.403 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.404 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12087 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 52224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:26.404 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:20:26.404 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.5968981040000001, 2024-09-17T21:20:26.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087684460000000006, 2024-09-17T21:20:26.404 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.404 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.404 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:26.404 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:26.404 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:26.404 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:26.404 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12087 2024-09-17T21:20:26.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.405 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:20:26.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:20:26.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769480+0000", 2024-09-17T21:20:26.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:20:26.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769485+0000", 2024-09-17T21:20:26.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8880000000000002e-06 2024-09-17T21:20:26.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769509+0000", 2024-09-17T21:20:26.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3343e-05 2024-09-17T21:20:26.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769521+0000", 2024-09-17T21:20:26.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2497000000000001e-05 2024-09-17T21:20:26.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.856883+0000", 2024-09-17T21:20:26.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087362191000000006 2024-09-17T21:20:26.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857014+0000", 2024-09-17T21:20:26.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013090399999999999 2024-09-17T21:20:26.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857163+0000", 2024-09-17T21:20:26.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014864000000000001 2024-09-17T21:20:26.408 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.408 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.408 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.408 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12088 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 56320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:26.408 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:20:26.408 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.596814449, 2024-09-17T21:20:26.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087917772000000005, 2024-09-17T21:20:26.408 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.408 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.408 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:26.408 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:26.409 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:26.409 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:26.409 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12088 2024-09-17T21:20:26.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.409 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:20:26.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:20:26.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769569+0000", 2024-09-17T21:20:26.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:20:26.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769574+0000", 2024-09-17T21:20:26.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4659999999999998e-06 2024-09-17T21:20:26.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769586+0000", 2024-09-17T21:20:26.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2311e-05 2024-09-17T21:20:26.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769597+0000", 2024-09-17T21:20:26.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0548e-05 2024-09-17T21:20:26.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857217+0000", 2024-09-17T21:20:26.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087619896000000003 2024-09-17T21:20:26.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857335+0000", 2024-09-17T21:20:26.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118538 2024-09-17T21:20:26.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857480+0000", 2024-09-17T21:20:26.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014470999999999999 2024-09-17T21:20:26.412 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.412 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.412 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.413 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12089 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 60416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:26.413 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:20:26.413 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.5967617499999998, 2024-09-17T21:20:26.413 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088293233999999998, 2024-09-17T21:20:26.413 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.413 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.413 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:26.413 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:26.413 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:26.413 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:26.413 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12089 2024-09-17T21:20:26.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.413 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:20:26.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:20:26.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769616+0000", 2024-09-17T21:20:26.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:20:26.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769620+0000", 2024-09-17T21:20:26.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5140000000000002e-06 2024-09-17T21:20:26.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769633+0000", 2024-09-17T21:20:26.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2849e-05 2024-09-17T21:20:26.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769644+0000", 2024-09-17T21:20:26.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1471000000000001e-05 2024-09-17T21:20:26.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857564+0000", 2024-09-17T21:20:26.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087919445999999998 2024-09-17T21:20:26.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857716+0000", 2024-09-17T21:20:26.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000152065 2024-09-17T21:20:26.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857908+0000", 2024-09-17T21:20:26.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019252599999999999 2024-09-17T21:20:26.417 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.417 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.417 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.417 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12090 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 64512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:26.417 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:20:26.417 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.5967153649999997, 2024-09-17T21:20:26.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088582672000000001, 2024-09-17T21:20:26.417 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.417 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.417 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:26.417 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:26.417 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:26.418 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:26.418 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12090 2024-09-17T21:20:26.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.418 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:20:26.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:20:26.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769662+0000", 2024-09-17T21:20:26.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:20:26.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769683+0000", 2024-09-17T21:20:26.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0979e-05 2024-09-17T21:20:26.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769699+0000", 2024-09-17T21:20:26.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5446000000000001e-05 2024-09-17T21:20:26.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769708+0000", 2024-09-17T21:20:26.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1679999999999993e-06 2024-09-17T21:20:26.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857976+0000", 2024-09-17T21:20:26.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088267795999999996 2024-09-17T21:20:26.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858095+0000", 2024-09-17T21:20:26.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119147 2024-09-17T21:20:26.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.421 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858244+0000", 2024-09-17T21:20:26.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014917000000000001 2024-09-17T21:20:26.421 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.421 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.421 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.421 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12091 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 68608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:26.421 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:20:26.421 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.5966439809999997, 2024-09-17T21:20:26.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088832752000000001, 2024-09-17T21:20:26.421 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.421 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.421 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:26.421 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:26.422 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:26.422 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:26.422 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12091 2024-09-17T21:20:26.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.422 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:20:26.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:20:26.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769734+0000", 2024-09-17T21:20:26.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:20:26.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769739+0000", 2024-09-17T21:20:26.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8080000000000004e-06 2024-09-17T21:20:26.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769764+0000", 2024-09-17T21:20:26.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4839999999999999e-05 2024-09-17T21:20:26.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769777+0000", 2024-09-17T21:20:26.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3037000000000001e-05 2024-09-17T21:20:26.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858295+0000", 2024-09-17T21:20:26.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088518475999999999 2024-09-17T21:20:26.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858419+0000", 2024-09-17T21:20:26.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012378700000000001 2024-09-17T21:20:26.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858565+0000", 2024-09-17T21:20:26.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014630099999999999 2024-09-17T21:20:26.425 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.425 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.425 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.425 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12092 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:26.425 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:20:26.425 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.5965616569999996, 2024-09-17T21:20:26.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089191555000000006, 2024-09-17T21:20:26.426 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.426 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.426 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:26.426 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:26.426 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:26.426 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:26.426 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12092 2024-09-17T21:20:26.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.426 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.426 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.426 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:20:26.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:20:26.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769816+0000", 2024-09-17T21:20:26.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7199999999999997e-07 2024-09-17T21:20:26.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769822+0000", 2024-09-17T21:20:26.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:20:26.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769838+0000", 2024-09-17T21:20:26.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6674e-05 2024-09-17T21:20:26.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769851+0000", 2024-09-17T21:20:26.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2527999999999999e-05 2024-09-17T21:20:26.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858660+0000", 2024-09-17T21:20:26.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088808556999999996 2024-09-17T21:20:26.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858817+0000", 2024-09-17T21:20:26.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015730400000000001 2024-09-17T21:20:26.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859007+0000", 2024-09-17T21:20:26.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018974899999999999 2024-09-17T21:20:26.429 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.429 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.429 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.430 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12093 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:26.430 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:20:26.430 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.5964535529999999, 2024-09-17T21:20:26.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089429710999999995, 2024-09-17T21:20:26.430 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.430 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.430 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:26.430 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:26.430 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:26.430 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:26.430 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12093 2024-09-17T21:20:26.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.430 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.431 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.431 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:20:26.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.431 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.431 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:20:26.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.431 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.431 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769924+0000", 2024-09-17T21:20:26.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.342e-06 2024-09-17T21:20:26.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.432 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769929+0000", 2024-09-17T21:20:26.432 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0350000000000001e-06 2024-09-17T21:20:26.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.432 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769946+0000", 2024-09-17T21:20:26.432 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6793999999999999e-05 2024-09-17T21:20:26.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769959+0000", 2024-09-17T21:20:26.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3062e-05 2024-09-17T21:20:26.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859075+0000", 2024-09-17T21:20:26.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089115696999999994 2024-09-17T21:20:26.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859202+0000", 2024-09-17T21:20:26.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012657 2024-09-17T21:20:26.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859353+0000", 2024-09-17T21:20:26.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151211 2024-09-17T21:20:26.434 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.434 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.434 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.434 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12094 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:26.434 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:20:26.434 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.596399377, 2024-09-17T21:20:26.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089695001999999996, 2024-09-17T21:20:26.434 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.435 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.435 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:26.435 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:26.435 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:26.435 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:26.435 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12094 2024-09-17T21:20:26.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.435 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:20:26.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.436 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:20:26.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.436 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.436 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769978+0000", 2024-09-17T21:20:26.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.142e-06 2024-09-17T21:20:26.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.436 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.436 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769983+0000", 2024-09-17T21:20:26.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.386e-06 2024-09-17T21:20:26.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769997+0000", 2024-09-17T21:20:26.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.416e-05 2024-09-17T21:20:26.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770012+0000", 2024-09-17T21:20:26.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5384999999999999e-05 2024-09-17T21:20:26.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859405+0000", 2024-09-17T21:20:26.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089392376999999995 2024-09-17T21:20:26.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859525+0000", 2024-09-17T21:20:26.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120555 2024-09-17T21:20:26.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859672+0000", 2024-09-17T21:20:26.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000146997 2024-09-17T21:20:26.438 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.438 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.438 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.439 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12095 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:26.439 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:20:26.439 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.5963468899999995, 2024-09-17T21:20:26.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090073476, 2024-09-17T21:20:26.439 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.439 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.439 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:26.439 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:26.439 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:26.439 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:26.439 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12095 2024-09-17T21:20:26.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.439 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:20:26.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:20:26.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770031+0000", 2024-09-17T21:20:26.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1699999999999997e-07 2024-09-17T21:20:26.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.441 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770036+0000", 2024-09-17T21:20:26.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.023e-06 2024-09-17T21:20:26.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.441 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770051+0000", 2024-09-17T21:20:26.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4953999999999999e-05 2024-09-17T21:20:26.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.441 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770063+0000", 2024-09-17T21:20:26.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:20:26.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859756+0000", 2024-09-17T21:20:26.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089692980000000005 2024-09-17T21:20:26.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859891+0000", 2024-09-17T21:20:26.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013459499999999999 2024-09-17T21:20:26.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860103+0000", 2024-09-17T21:20:26.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021274400000000001 2024-09-17T21:20:26.443 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.443 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.443 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.443 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12096 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:26.443 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:20:26.443 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.5962972049999999, 2024-09-17T21:20:26.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090358274000000002, 2024-09-17T21:20:26.443 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.443 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.443 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:26.443 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:26.444 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:26.444 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:26.444 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12096 2024-09-17T21:20:26.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.444 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:20:26.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:20:26.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770080+0000", 2024-09-17T21:20:26.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9400000000000004e-07 2024-09-17T21:20:26.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770085+0000", 2024-09-17T21:20:26.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2169999999999996e-06 2024-09-17T21:20:26.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770100+0000", 2024-09-17T21:20:26.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5077e-05 2024-09-17T21:20:26.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770112+0000", 2024-09-17T21:20:26.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1951e-05 2024-09-17T21:20:26.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860160+0000", 2024-09-17T21:20:26.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090048413999999993 2024-09-17T21:20:26.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860285+0000", 2024-09-17T21:20:26.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012466699999999999 2024-09-17T21:20:26.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860438+0000", 2024-09-17T21:20:26.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015305399999999999 2024-09-17T21:20:26.447 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.447 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.447 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.447 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12097 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:26.447 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:20:26.447 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.596249458, 2024-09-17T21:20:26.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090702375000000002, 2024-09-17T21:20:26.448 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.448 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.448 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:26.448 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:26.448 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:26.448 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:26.448 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12097 2024-09-17T21:20:26.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.448 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:20:26.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:20:26.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770128+0000", 2024-09-17T21:20:26.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.04e-06 2024-09-17T21:20:26.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770133+0000", 2024-09-17T21:20:26.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8999999999999997e-06 2024-09-17T21:20:26.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770152+0000", 2024-09-17T21:20:26.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8890999999999999e-05 2024-09-17T21:20:26.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770164+0000", 2024-09-17T21:20:26.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1769e-05 2024-09-17T21:20:26.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860511+0000", 2024-09-17T21:20:26.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090347527999999996 2024-09-17T21:20:26.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860652+0000", 2024-09-17T21:20:26.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140894 2024-09-17T21:20:26.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860830+0000", 2024-09-17T21:20:26.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017735299999999999 2024-09-17T21:20:26.451 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.451 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.451 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.451 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12098 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:26.452 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:20:26.452 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.59619599, 2024-09-17T21:20:26.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090997262999999995, 2024-09-17T21:20:26.452 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.452 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.452 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:26.452 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:26.452 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:26.452 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:26.452 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12098 2024-09-17T21:20:26.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.452 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:20:26.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:20:26.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775465+0000", 2024-09-17T21:20:26.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0052843350000000002 2024-09-17T21:20:26.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.454 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775476+0000", 2024-09-17T21:20:26.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0797e-05 2024-09-17T21:20:26.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.454 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.454 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775506+0000", 2024-09-17T21:20:26.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9709999999999998e-05 2024-09-17T21:20:26.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.454 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.454 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775530+0000", 2024-09-17T21:20:26.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4528999999999998e-05 2024-09-17T21:20:26.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.455 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.455 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860894+0000", 2024-09-17T21:20:26.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.085364195000000004 2024-09-17T21:20:26.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.455 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.455 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861019+0000", 2024-09-17T21:20:26.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012467399999999999 2024-09-17T21:20:26.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.455 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.455 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861178+0000", 2024-09-17T21:20:26.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159023 2024-09-17T21:20:26.455 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.455 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.456 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.456 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12104 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:26.456 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:20:26.456 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.5904948809999997, 2024-09-17T21:20:26.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087969628999999994, 2024-09-17T21:20:26.456 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.456 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.456 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:26.456 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:26.456 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:26.456 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:26.457 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12104 2024-09-17T21:20:26.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.457 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:20:26.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:20:26.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775883+0000", 2024-09-17T21:20:26.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3399999999999998e-07 2024-09-17T21:20:26.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775886+0000", 2024-09-17T21:20:26.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2119999999999999e-06 2024-09-17T21:20:26.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775895+0000", 2024-09-17T21:20:26.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8710000000000003e-06 2024-09-17T21:20:26.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775901+0000", 2024-09-17T21:20:26.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:20:26.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863544+0000", 2024-09-17T21:20:26.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087642642000000007 2024-09-17T21:20:26.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863694+0000", 2024-09-17T21:20:26.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014952300000000001 2024-09-17T21:20:26.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.460 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.460 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863851+0000", 2024-09-17T21:20:26.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000157804 2024-09-17T21:20:26.460 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.460 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.460 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.460 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12105 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:26.460 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:26.460 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.590465987, 2024-09-17T21:20:26.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088330400000000003, 2024-09-17T21:20:26.460 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.460 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.461 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:26.461 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:26.461 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:26.461 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:26.461 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12105 2024-09-17T21:20:26.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.461 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.461 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.461 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:26.461 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:26.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:26.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9999999999999997e-07 2024-09-17T21:20:26.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775914+0000", 2024-09-17T21:20:26.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4710000000000001e-06 2024-09-17T21:20:26.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775921+0000", 2024-09-17T21:20:26.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:20:26.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775928+0000", 2024-09-17T21:20:26.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:20:26.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863923+0000", 2024-09-17T21:20:26.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087995126000000007 2024-09-17T21:20:26.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864059+0000", 2024-09-17T21:20:26.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135797 2024-09-17T21:20:26.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864241+0000", 2024-09-17T21:20:26.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018252599999999999 2024-09-17T21:20:26.464 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.464 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.465 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.465 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12106 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:26.465 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:20:26.465 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.5903768090000003, 2024-09-17T21:20:26.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088586394999999998, 2024-09-17T21:20:26.465 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.465 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.465 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:26.465 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:26.465 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:26.465 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:26.466 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12106 2024-09-17T21:20:26.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.466 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.466 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.466 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:20:26.466 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.466 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.466 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:20:26.466 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.467 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.467 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776001+0000", 2024-09-17T21:20:26.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4300000000000001e-07 2024-09-17T21:20:26.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.467 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.467 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776004+0000", 2024-09-17T21:20:26.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8049999999999998e-06 2024-09-17T21:20:26.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.467 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.467 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776018+0000", 2024-09-17T21:20:26.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4613999999999999e-05 2024-09-17T21:20:26.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.468 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.468 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776025+0000", 2024-09-17T21:20:26.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:20:26.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.468 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.468 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864279+0000", 2024-09-17T21:20:26.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088253570000000003 2024-09-17T21:20:26.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.468 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.468 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864433+0000", 2024-09-17T21:20:26.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015469400000000001 2024-09-17T21:20:26.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864586+0000", 2024-09-17T21:20:26.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000153003 2024-09-17T21:20:26.469 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.469 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.469 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.469 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.469 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.469 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:26.471 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:20:26.472 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:26.465+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:26.495 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 0] out_osds: [2, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:20:26.495 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:20:26.495 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-17T21:20:26.495 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T21:20:26.495 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T21:20:26.496 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-17T21:20:26.548 INFO:teuthology.orchestra.run.smithi184.stdout:4934 write 0xdd43200 thru 0xdd462cc (0x30cd bytes) 2024-09-17T21:20:26.551 INFO:teuthology.orchestra.run.smithi184.stdout:4936 read 0x460d400 thru 0x460e87a (0x147b bytes) 2024-09-17T21:20:26.551 INFO:teuthology.orchestra.run.smithi184.stdout:4940 read 0x27d4a00 thru 0x27dd620 (0x8c21 bytes) 2024-09-17T21:20:26.551 INFO:teuthology.orchestra.run.smithi184.stdout:4942 read 0x1726e00 thru 0x17354e9 (0xe6ea bytes) 2024-09-17T21:20:26.552 INFO:teuthology.orchestra.run.smithi184.stdout:4943 write 0x1f96400 thru 0x1f97428 (0x1029 bytes) 2024-09-17T21:20:26.552 INFO:teuthology.orchestra.run.smithi184.stdout:4946 write 0xa468600 thru 0xa475b6c (0xd56d bytes) 2024-09-17T21:20:26.553 INFO:teuthology.orchestra.run.smithi184.stdout:4947 write 0xa0f800 thru 0xa11d7a (0x257b bytes) 2024-09-17T21:20:26.571 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:26.566+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:26.645 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:26.645 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:26.645 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:26.645 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:26.661 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:20:26.670 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:26.670 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T21:20:26.670 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:26.671 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:26.666+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:26.772 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:26.765+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:26.784 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:26.784 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:26.784 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:26.784 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:26.784 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:26.799 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:20:26.873 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:26.866+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:26.967 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:26.967 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:26.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:20:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 128.74741987600001, 2024-09-17T21:20:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:20:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:20:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:20:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:20:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:20:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:20:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:20:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:20:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:20:26.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:20:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:20:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:20:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:20:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:26.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:20:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:20:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:20:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:20:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:20:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:26.972 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 124.097119212, 2024-09-17T21:20:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:20:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:20:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:20:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:20:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:20:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:20:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:20:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:20:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:20:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:20:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:20:26.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:20:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:20:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:20:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:20:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:20:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:20:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:20:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 124.095854129, 2024-09-17T21:20:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:20:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:26.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:26.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:20:26.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:20:26.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:20:26.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:20:26.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:20:26.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:20:26.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:20:26.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:20:26.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:20:26.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:20:26.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:20:26.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:20:26.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:26.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:20:26.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:20:26.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:20:26.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:20:26.981 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.981 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.981 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.981 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:20:26.982 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:26.982 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 124.086074719, 2024-09-17T21:20:26.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:20:26.982 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.982 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.982 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:26.982 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:26.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:26.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:20:26.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:20:26.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:20:26.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:20:26.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:20:26.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:20:26.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:20:26.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:20:26.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:20:26.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:20:26.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:20:26.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:20:26.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:26.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:20:26.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:20:26.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:20:26.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:20:26.986 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.986 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.986 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.986 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:20:26.986 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:26.986 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 124.077352316, 2024-09-17T21:20:26.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:20:26.986 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.987 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.987 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:26.987 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:26.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:26.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:20:26.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:20:26.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:20:26.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:20:26.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:20:26.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:20:26.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:20:26.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:20:26.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:20:26.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:20:26.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:20:26.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:20:26.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:26.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:20:26.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:20:26.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:20:26.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:20:26.990 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.991 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.991 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.991 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:20:26.991 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:26.991 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.69226175199999, 2024-09-17T21:20:26.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:20:26.991 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.991 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.991 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:26.991 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:26.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:26.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:20:26.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:20:26.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:20:26.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:20:26.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:20:26.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:20:26.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:20:26.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:20:26.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.994 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.994 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:20:26.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:20:26.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.994 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.994 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:20:26.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:20:26.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.994 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:26.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:20:26.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:20:26.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:26.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:20:26.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:20:26.995 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.995 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:26.995 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:26.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.996 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1780 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'195, mlcod=69'190)", 2024-09-17T21:20:26.996 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:26.996 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.714406147999995, 2024-09-17T21:20:26.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075761769999999999, 2024-09-17T21:20:26.996 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:26.996 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:26.996 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:26.996 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:26.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:26.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:26.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:26.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:26.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:26.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:26.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250375+0000", 2024-09-17T21:20:26.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:20:26.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:26.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250383+0000", 2024-09-17T21:20:26.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7870000000000002e-06 2024-09-17T21:20:26.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:26.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250402+0000", 2024-09-17T21:20:26.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9346000000000001e-05 2024-09-17T21:20:26.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:26.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250413+0000", 2024-09-17T21:20:26.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.039e-05 2024-09-17T21:20:26.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:26.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251353+0000", 2024-09-17T21:20:26.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000940091 2024-09-17T21:20:26.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:26.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251407+0000", 2024-09-17T21:20:26.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4701999999999999e-05 2024-09-17T21:20:26.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:26.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:26.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:26.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257829+0000", 2024-09-17T21:20:26.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064212779999999999 2024-09-17T21:20:26.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:27.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257950+0000", 2024-09-17T21:20:27.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121623 2024-09-17T21:20:27.000 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.000 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:27.000 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.000 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1781 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'196, mlcod=69'190)", 2024-09-17T21:20:27.000 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:27.000 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.713323196999994, 2024-09-17T21:20:27.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065632939999999999, 2024-09-17T21:20:27.000 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:27.001 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:27.001 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:27.001 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:27.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:27.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:27.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:27.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:27.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:27.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251459+0000", 2024-09-17T21:20:27.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.624e-06 2024-09-17T21:20:27.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:27.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251471+0000", 2024-09-17T21:20:27.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2636000000000001e-05 2024-09-17T21:20:27.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:27.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251498+0000", 2024-09-17T21:20:27.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:20:27.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251515+0000", 2024-09-17T21:20:27.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6939000000000002e-05 2024-09-17T21:20:27.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:27.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252938+0000", 2024-09-17T21:20:27.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001423082 2024-09-17T21:20:27.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:27.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252999+0000", 2024-09-17T21:20:27.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1138999999999999e-05 2024-09-17T21:20:27.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:27.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257955+0000", 2024-09-17T21:20:27.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049559640000000002 2024-09-17T21:20:27.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:27.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.258020+0000", 2024-09-17T21:20:27.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5504000000000002e-05 2024-09-17T21:20:27.005 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.005 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:27.005 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.005 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1788 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'198, mlcod=69'191)", 2024-09-17T21:20:27.005 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:27.005 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.711297708000004, 2024-09-17T21:20:27.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073690370000000002, 2024-09-17T21:20:27.005 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:27.005 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:27.005 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:27.005 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:27.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:27.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:27.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:27.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:27.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:27.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253484+0000", 2024-09-17T21:20:27.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2470000000000001e-06 2024-09-17T21:20:27.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:27.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253492+0000", 2024-09-17T21:20:27.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7159999999999992e-06 2024-09-17T21:20:27.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:27.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253509+0000", 2024-09-17T21:20:27.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.666e-05 2024-09-17T21:20:27.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253522+0000", 2024-09-17T21:20:27.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2643000000000001e-05 2024-09-17T21:20:27.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.008 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:27.008 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254853+0000", 2024-09-17T21:20:27.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331238 2024-09-17T21:20:27.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.008 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:27.008 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254885+0000", 2024-09-17T21:20:27.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1562e-05 2024-09-17T21:20:27.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:27.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260788+0000", 2024-09-17T21:20:27.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059030749999999998 2024-09-17T21:20:27.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:27.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260852+0000", 2024-09-17T21:20:27.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3895999999999999e-05 2024-09-17T21:20:27.009 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.009 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:27.009 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.009 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5746 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'477, mlcod=71'473)", 2024-09-17T21:20:27.010 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:27.010 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.705870149999996, 2024-09-17T21:20:27.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074182709999999997, 2024-09-17T21:20:27.010 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:27.010 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:27.010 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:27.010 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:27.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:27.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:27.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:27.011 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:27.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.011 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:27.011 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258912+0000", 2024-09-17T21:20:27.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:20:27.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.011 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:27.011 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258926+0000", 2024-09-17T21:20:27.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4249e-05 2024-09-17T21:20:27.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.012 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:27.012 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258946+0000", 2024-09-17T21:20:27.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9879999999999999e-05 2024-09-17T21:20:27.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.012 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.012 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258956+0000", 2024-09-17T21:20:27.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:20:27.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.012 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:27.012 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259380+0000", 2024-09-17T21:20:27.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042337599999999999 2024-09-17T21:20:27.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:27.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259412+0000", 2024-09-17T21:20:27.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:20:27.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:27.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266191+0000", 2024-09-17T21:20:27.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067793269999999999 2024-09-17T21:20:27.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:27.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266328+0000", 2024-09-17T21:20:27.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013690999999999999 2024-09-17T21:20:27.014 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.014 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:27.014 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.014 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5747 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'478, mlcod=71'473)", 2024-09-17T21:20:27.014 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:27.014 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.705163800999998, 2024-09-17T21:20:27.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068779440000000004, 2024-09-17T21:20:27.015 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:27.015 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:27.015 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:27.015 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:27.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:27.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:27.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:27.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:27.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.016 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:27.016 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259618+0000", 2024-09-17T21:20:27.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6199999999999999e-06 2024-09-17T21:20:27.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.016 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:27.016 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259630+0000", 2024-09-17T21:20:27.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1884e-05 2024-09-17T21:20:27.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.016 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:27.017 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259657+0000", 2024-09-17T21:20:27.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7339e-05 2024-09-17T21:20:27.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.017 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.017 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259677+0000", 2024-09-17T21:20:27.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0156000000000001e-05 2024-09-17T21:20:27.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.017 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:27.017 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260521+0000", 2024-09-17T21:20:27.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084312899999999997 2024-09-17T21:20:27.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:27.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260585+0000", 2024-09-17T21:20:27.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4558000000000001e-05 2024-09-17T21:20:27.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:27.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266336+0000", 2024-09-17T21:20:27.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057505020000000002 2024-09-17T21:20:27.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:27.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266494+0000", 2024-09-17T21:20:27.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158756 2024-09-17T21:20:27.018 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.019 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:27.019 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.019 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5750 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'479, mlcod=71'473)", 2024-09-17T21:20:27.019 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:27.019 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.704451607999999, 2024-09-17T21:20:27.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006302874, 2024-09-17T21:20:27.019 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:27.019 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:27.019 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:27.019 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:27.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:27.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:27.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:27.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:27.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:27.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260330+0000", 2024-09-17T21:20:27.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:20:27.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.021 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:27.021 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260343+0000", 2024-09-17T21:20:27.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2862e-05 2024-09-17T21:20:27.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.021 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:27.021 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260367+0000", 2024-09-17T21:20:27.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3723e-05 2024-09-17T21:20:27.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.021 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.021 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260386+0000", 2024-09-17T21:20:27.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8879999999999999e-05 2024-09-17T21:20:27.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.022 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:27.022 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261579+0000", 2024-09-17T21:20:27.022 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001193431 2024-09-17T21:20:27.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.022 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:27.022 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261608+0000", 2024-09-17T21:20:27.022 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8858999999999999e-05 2024-09-17T21:20:27.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.022 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:27.022 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266505+0000", 2024-09-17T21:20:27.023 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004896999 2024-09-17T21:20:27.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.023 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:27.023 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266631+0000", 2024-09-17T21:20:27.023 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012655300000000001 2024-09-17T21:20:27.023 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.023 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:27.023 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.023 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5751 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'480, mlcod=71'473)", 2024-09-17T21:20:27.023 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:27.023 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.703744396000005, 2024-09-17T21:20:27.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075756299999999999, 2024-09-17T21:20:27.024 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:27.024 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:27.024 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:27.024 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:27.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:27.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:27.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:27.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:27.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:27.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261037+0000", 2024-09-17T21:20:27.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:20:27.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:27.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261049+0000", 2024-09-17T21:20:27.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1063999999999999e-05 2024-09-17T21:20:27.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:27.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261072+0000", 2024-09-17T21:20:27.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3102e-05 2024-09-17T21:20:27.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.026 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.026 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261133+0000", 2024-09-17T21:20:27.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1276000000000001e-05 2024-09-17T21:20:27.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.026 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:27.026 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:20:27.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015334319999999999 2024-09-17T21:20:27.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.027 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:27.027 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262702+0000", 2024-09-17T21:20:27.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5281999999999999e-05 2024-09-17T21:20:27.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.027 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:27.027 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268527+0000", 2024-09-17T21:20:27.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058250389999999997 2024-09-17T21:20:27.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.027 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:27.027 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268611+0000", 2024-09-17T21:20:27.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4724999999999999e-05 2024-09-17T21:20:27.028 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.028 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:27.028 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.028 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5756 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'481, mlcod=71'473)", 2024-09-17T21:20:27.028 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:27.028 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.702883540000002, 2024-09-17T21:20:27.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006772386, 2024-09-17T21:20:27.028 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:27.028 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:27.028 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:27.028 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:27.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:27.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:27.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:27.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:27.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:27.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261899+0000", 2024-09-17T21:20:27.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.903e-06 2024-09-17T21:20:27.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:27.030 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261912+0000", 2024-09-17T21:20:27.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-05 2024-09-17T21:20:27.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:27.030 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261938+0000", 2024-09-17T21:20:27.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6098999999999999e-05 2024-09-17T21:20:27.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.030 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261955+0000", 2024-09-17T21:20:27.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.696e-05 2024-09-17T21:20:27.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:27.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263845+0000", 2024-09-17T21:20:27.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001890187 2024-09-17T21:20:27.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:27.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263907+0000", 2024-09-17T21:20:27.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1600999999999995e-05 2024-09-17T21:20:27.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:27.032 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268615+0000", 2024-09-17T21:20:27.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047086070000000001 2024-09-17T21:20:27.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:27.032 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268669+0000", 2024-09-17T21:20:27.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3588999999999998e-05 2024-09-17T21:20:27.032 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.032 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:27.032 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.032 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4978.0:598 3.9 e80/23 3:9701cc17:::rbd_data.1152724ff2a3.0000000000000019:head v 80'202, mlcod=80'198)", 2024-09-17T21:20:27.033 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:27.033 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.201309156999997, 2024-09-17T21:20:27.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066473340000000004, 2024-09-17T21:20:27.033 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:27.033 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:27.033 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:27.033 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:27.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:27.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:27.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:27.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:27.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.034 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:27.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763472+0000", 2024-09-17T21:20:27.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6899999999999997e-07 2024-09-17T21:20:27.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.034 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:27.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763478+0000", 2024-09-17T21:20:27.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:20:27.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:27.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763489+0000", 2024-09-17T21:20:27.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0628999999999999e-05 2024-09-17T21:20:27.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763495+0000", 2024-09-17T21:20:27.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6649999999999997e-06 2024-09-17T21:20:27.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:27.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763907+0000", 2024-09-17T21:20:27.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041163899999999999 2024-09-17T21:20:27.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:27.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763976+0000", 2024-09-17T21:20:27.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8534999999999993e-05 2024-09-17T21:20:27.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:27.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.769900+0000", 2024-09-17T21:20:27.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059243270000000001 2024-09-17T21:20:27.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:27.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.770118+0000", 2024-09-17T21:20:27.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021842500000000001 2024-09-17T21:20:27.037 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.037 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:27.037 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.037 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:468 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'94, mlcod=89'87)", 2024-09-17T21:20:27.037 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:27.037 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 47.278563857999998, 2024-09-17T21:20:27.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066528790000000004, 2024-09-17T21:20:27.037 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:27.037 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:27.037 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:27.038 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:27.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.038 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:27.038 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:27.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.038 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:27.038 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:27.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.038 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:27.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686218+0000", 2024-09-17T21:20:27.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3060000000000001e-06 2024-09-17T21:20:27.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.039 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:27.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686256+0000", 2024-09-17T21:20:27.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8333e-05 2024-09-17T21:20:27.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.039 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:27.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686272+0000", 2024-09-17T21:20:27.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:20:27.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686283+0000", 2024-09-17T21:20:27.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0903e-05 2024-09-17T21:20:27.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:27.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686682+0000", 2024-09-17T21:20:27.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039935099999999999 2024-09-17T21:20:27.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:27.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686785+0000", 2024-09-17T21:20:27.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010275199999999999 2024-09-17T21:20:27.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:27.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692816+0000", 2024-09-17T21:20:27.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060317579999999999 2024-09-17T21:20:27.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:27.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692869+0000", 2024-09-17T21:20:27.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2775999999999998e-05 2024-09-17T21:20:27.041 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.041 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:27.042 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.042 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:755 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'127, mlcod=89'122)", 2024-09-17T21:20:27.042 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:27.042 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 47.142108643999997, 2024-09-17T21:20:27.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084014080000000008, 2024-09-17T21:20:27.042 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:27.042 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:27.042 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:27.042 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:27.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:27.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:27.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:27.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:27.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:27.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:27.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:20:27.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:27.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822683+0000", 2024-09-17T21:20:27.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0674e-05 2024-09-17T21:20:27.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:27.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822698+0000", 2024-09-17T21:20:27.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5189e-05 2024-09-17T21:20:27.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822708+0000", 2024-09-17T21:20:27.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9119999999999993e-06 2024-09-17T21:20:27.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:27.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822748+0000", 2024-09-17T21:20:27.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9727000000000001e-05 2024-09-17T21:20:27.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:27.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822759+0000", 2024-09-17T21:20:27.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0865e-05 2024-09-17T21:20:27.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:27.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831034+0000", 2024-09-17T21:20:27.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082750710000000002 2024-09-17T21:20:27.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:27.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831073+0000", 2024-09-17T21:20:27.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9063999999999997e-05 2024-09-17T21:20:27.046 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.046 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:27.046 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.046 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:756 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'128, mlcod=89'124)", 2024-09-17T21:20:27.047 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:27.047 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 47.140186012000001, 2024-09-17T21:20:27.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0076300270000000002, 2024-09-17T21:20:27.047 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:27.047 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:27.047 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:27.047 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:27.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:27.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:27.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:27.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:27.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:27.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824595+0000", 2024-09-17T21:20:27.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-17T21:20:27.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:27.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824600+0000", 2024-09-17T21:20:27.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:20:27.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.049 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:27.049 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824612+0000", 2024-09-17T21:20:27.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.181e-05 2024-09-17T21:20:27.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.049 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.049 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824619+0000", 2024-09-17T21:20:27.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:20:27.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.049 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:27.049 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824671+0000", 2024-09-17T21:20:27.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2262999999999998e-05 2024-09-17T21:20:27.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.050 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:27.050 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824695+0000", 2024-09-17T21:20:27.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3883000000000002e-05 2024-09-17T21:20:27.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.050 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:27.050 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832185+0000", 2024-09-17T21:20:27.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007490048 2024-09-17T21:20:27.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:27.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832224+0000", 2024-09-17T21:20:27.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8957e-05 2024-09-17T21:20:27.051 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.051 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:27.051 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.051 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2617 3.d e90/23 3:b0143718:::rbd_data.1152a8ae2019.0000000000000016:head v 90'249, mlcod=90'246)", 2024-09-17T21:20:27.051 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:27.051 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 46.120335011999998, 2024-09-17T21:20:27.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063715559999999996, 2024-09-17T21:20:27.051 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:27.052 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:27.052 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:27.052 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:27.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:27.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:27.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:27.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:27.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:27.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844455+0000", 2024-09-17T21:20:27.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4259999999999992e-06 2024-09-17T21:20:27.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:27.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844470+0000", 2024-09-17T21:20:27.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5386000000000001e-05 2024-09-17T21:20:27.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:27.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844499+0000", 2024-09-17T21:20:27.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.883e-05 2024-09-17T21:20:27.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844518+0000", 2024-09-17T21:20:27.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8830000000000001e-05 2024-09-17T21:20:27.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:27.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844649+0000", 2024-09-17T21:20:27.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013143399999999999 2024-09-17T21:20:27.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:27.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844702+0000", 2024-09-17T21:20:27.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2876e-05 2024-09-17T21:20:27.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.055 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:27.055 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850696+0000", 2024-09-17T21:20:27.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059943729999999999 2024-09-17T21:20:27.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.055 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:27.055 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850817+0000", 2024-09-17T21:20:27.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120401 2024-09-17T21:20:27.055 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.055 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:27.055 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.056 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2635 3.d e90/23 3:b3d1e920:::rbd_data.1152a8ae2019.000000000000001f:head v 90'252, mlcod=90'246)", 2024-09-17T21:20:27.056 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:27.056 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 46.117626942000001, 2024-09-17T21:20:27.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0077265190000000003, 2024-09-17T21:20:27.056 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:27.056 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:27.056 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:27.056 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:27.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:27.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:27.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.057 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:27.057 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:27.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:27.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.057 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:27.057 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847155+0000", 2024-09-17T21:20:27.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.544e-06 2024-09-17T21:20:27.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.057 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:27.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847165+0000", 2024-09-17T21:20:27.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.043e-05 2024-09-17T21:20:27.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:27.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847189+0000", 2024-09-17T21:20:27.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3659000000000002e-05 2024-09-17T21:20:27.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847210+0000", 2024-09-17T21:20:27.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1163e-05 2024-09-17T21:20:27.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:27.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848856+0000", 2024-09-17T21:20:27.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001645521 2024-09-17T21:20:27.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:27.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848905+0000", 2024-09-17T21:20:27.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9048999999999997e-05 2024-09-17T21:20:27.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:27.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854770+0000", 2024-09-17T21:20:27.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058652879999999998 2024-09-17T21:20:27.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:27.060 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:27.060 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:27.060 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854880+0000", 2024-09-17T21:20:27.060 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010986500000000001 2024-09-17T21:20:27.060 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.060 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:27.060 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.060 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:27.060 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:27.060 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:27.062 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:20:27.062 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:27.057+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:27.162 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:27.157+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:27.224 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:27.224 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:27.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.225 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18649 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.225 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.214611+0000", 2024-09-17T21:20:27.225 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0042891020000000004, 2024-09-17T21:20:27.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.004303215, 2024-09-17T21:20:27.225 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.225 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.225 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.225 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.225 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.225 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.225 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18649 2024-09-17T21:20:27.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.226 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214611+0000", 2024-09-17T21:20:27.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214611+0000", 2024-09-17T21:20:27.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214612+0000", 2024-09-17T21:20:27.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8299999999999997e-07 2024-09-17T21:20:27.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214625+0000", 2024-09-17T21:20:27.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3427999999999999e-05 2024-09-17T21:20:27.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214632+0000", 2024-09-17T21:20:27.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3710000000000002e-06 2024-09-17T21:20:27.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214640+0000", 2024-09-17T21:20:27.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2450000000000001e-06 2024-09-17T21:20:27.228 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.228 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.228 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.228 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18681 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.228 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215524+0000", 2024-09-17T21:20:27.228 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.003376272, 2024-09-17T21:20:27.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0034242640000000002, 2024-09-17T21:20:27.229 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.229 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.229 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.229 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.229 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.229 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.229 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18681 2024-09-17T21:20:27.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.229 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215524+0000", 2024-09-17T21:20:27.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215524+0000", 2024-09-17T21:20:27.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215524+0000", 2024-09-17T21:20:27.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9900000000000001e-07 2024-09-17T21:20:27.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215527+0000", 2024-09-17T21:20:27.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3649999999999998e-06 2024-09-17T21:20:27.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215534+0000", 2024-09-17T21:20:27.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9890000000000003e-06 2024-09-17T21:20:27.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215539+0000", 2024-09-17T21:20:27.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4750000000000001e-06 2024-09-17T21:20:27.231 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.231 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.231 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.232 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18714 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.232 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.216331+0000", 2024-09-17T21:20:27.232 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.002569287, 2024-09-17T21:20:27.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002645887, 2024-09-17T21:20:27.232 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.232 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.232 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.232 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.232 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.232 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.232 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18714 2024-09-17T21:20:27.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.233 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216331+0000", 2024-09-17T21:20:27.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216331+0000", 2024-09-17T21:20:27.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216331+0000", 2024-09-17T21:20:27.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:20:27.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216334+0000", 2024-09-17T21:20:27.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.266e-06 2024-09-17T21:20:27.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216340+0000", 2024-09-17T21:20:27.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5819999999999998e-06 2024-09-17T21:20:27.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216345+0000", 2024-09-17T21:20:27.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3159999999999996e-06 2024-09-17T21:20:27.235 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.235 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.235 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.235 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18650 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.235 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.214657+0000", 2024-09-17T21:20:27.235 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0042435820000000001, 2024-09-17T21:20:27.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043457879999999997, 2024-09-17T21:20:27.235 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.235 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.235 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.236 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.236 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.236 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.236 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18650 2024-09-17T21:20:27.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.236 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214657+0000", 2024-09-17T21:20:27.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214657+0000", 2024-09-17T21:20:27.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214657+0000", 2024-09-17T21:20:27.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8899999999999999e-07 2024-09-17T21:20:27.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214660+0000", 2024-09-17T21:20:27.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8940000000000002e-06 2024-09-17T21:20:27.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214668+0000", 2024-09-17T21:20:27.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6820000000000003e-06 2024-09-17T21:20:27.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214674+0000", 2024-09-17T21:20:27.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7450000000000001e-06 2024-09-17T21:20:27.238 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.238 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.238 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.238 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18682 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.239 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215547+0000", 2024-09-17T21:20:27.239 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0033526329999999998, 2024-09-17T21:20:27.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0034972929999999998, 2024-09-17T21:20:27.239 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.239 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.239 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.239 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.239 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.239 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.239 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18682 2024-09-17T21:20:27.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.239 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215547+0000", 2024-09-17T21:20:27.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215547+0000", 2024-09-17T21:20:27.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215548+0000", 2024-09-17T21:20:27.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1399999999999997e-07 2024-09-17T21:20:27.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215551+0000", 2024-09-17T21:20:27.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4679999999999999e-06 2024-09-17T21:20:27.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215557+0000", 2024-09-17T21:20:27.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7970000000000003e-06 2024-09-17T21:20:27.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215563+0000", 2024-09-17T21:20:27.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1819999999999997e-06 2024-09-17T21:20:27.242 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.242 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.242 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.242 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18715 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.242 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.216353+0000", 2024-09-17T21:20:27.242 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0025474669999999999, 2024-09-17T21:20:27.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002718945, 2024-09-17T21:20:27.242 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.242 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.243 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.243 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.243 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.243 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.243 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18715 2024-09-17T21:20:27.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.243 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216353+0000", 2024-09-17T21:20:27.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216353+0000", 2024-09-17T21:20:27.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216353+0000", 2024-09-17T21:20:27.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8899999999999999e-07 2024-09-17T21:20:27.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216355+0000", 2024-09-17T21:20:27.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.131e-06 2024-09-17T21:20:27.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216362+0000", 2024-09-17T21:20:27.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5339999999999996e-06 2024-09-17T21:20:27.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216367+0000", 2024-09-17T21:20:27.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3419999999999998e-06 2024-09-17T21:20:27.245 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.245 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.245 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.246 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18651 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.246 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.214682+0000", 2024-09-17T21:20:27.246 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0042179080000000002, 2024-09-17T21:20:27.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0044195179999999999, 2024-09-17T21:20:27.246 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.246 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.246 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.246 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.246 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.246 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.246 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18651 2024-09-17T21:20:27.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.246 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214682+0000", 2024-09-17T21:20:27.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214682+0000", 2024-09-17T21:20:27.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214683+0000", 2024-09-17T21:20:27.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4099999999999998e-07 2024-09-17T21:20:27.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214685+0000", 2024-09-17T21:20:27.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.232e-06 2024-09-17T21:20:27.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214692+0000", 2024-09-17T21:20:27.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.72e-06 2024-09-17T21:20:27.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214698+0000", 2024-09-17T21:20:27.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:20:27.249 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.249 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.249 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.249 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18683 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.249 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215570+0000", 2024-09-17T21:20:27.249 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.00333008, 2024-09-17T21:20:27.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035648860000000002, 2024-09-17T21:20:27.249 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.249 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.249 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.249 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.250 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.250 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.250 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18683 2024-09-17T21:20:27.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.250 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215570+0000", 2024-09-17T21:20:27.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215570+0000", 2024-09-17T21:20:27.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215571+0000", 2024-09-17T21:20:27.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:20:27.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215573+0000", 2024-09-17T21:20:27.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3659999999999999e-06 2024-09-17T21:20:27.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215579+0000", 2024-09-17T21:20:27.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3720000000000001e-06 2024-09-17T21:20:27.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215585+0000", 2024-09-17T21:20:27.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3399999999999997e-06 2024-09-17T21:20:27.252 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.252 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.252 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.252 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18716 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.252 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.216375+0000", 2024-09-17T21:20:27.253 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.002525575, 2024-09-17T21:20:27.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002784623, 2024-09-17T21:20:27.253 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.253 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.253 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.253 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.253 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.253 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.253 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18716 2024-09-17T21:20:27.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.253 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216375+0000", 2024-09-17T21:20:27.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216375+0000", 2024-09-17T21:20:27.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216375+0000", 2024-09-17T21:20:27.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2399999999999998e-07 2024-09-17T21:20:27.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216377+0000", 2024-09-17T21:20:27.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1129999999999999e-06 2024-09-17T21:20:27.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216383+0000", 2024-09-17T21:20:27.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2280000000000003e-06 2024-09-17T21:20:27.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216389+0000", 2024-09-17T21:20:27.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.254e-06 2024-09-17T21:20:27.255 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.256 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.256 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.256 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18652 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.256 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.214706+0000", 2024-09-17T21:20:27.256 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0041942539999999997, 2024-09-17T21:20:27.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0044789620000000004, 2024-09-17T21:20:27.256 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.256 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.256 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.256 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.256 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.257 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.257 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18652 2024-09-17T21:20:27.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.257 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214706+0000", 2024-09-17T21:20:27.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214706+0000", 2024-09-17T21:20:27.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214707+0000", 2024-09-17T21:20:27.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5579999999999999e-06 2024-09-17T21:20:27.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214710+0000", 2024-09-17T21:20:27.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3690000000000001e-06 2024-09-17T21:20:27.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214724+0000", 2024-09-17T21:20:27.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4358e-05 2024-09-17T21:20:27.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214730+0000", 2024-09-17T21:20:27.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9070000000000004e-06 2024-09-17T21:20:27.259 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.259 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.259 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.259 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18684 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.259 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215593+0000", 2024-09-17T21:20:27.259 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0033072100000000001, 2024-09-17T21:20:27.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036402909999999999, 2024-09-17T21:20:27.259 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.260 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.260 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.260 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.260 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.260 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.260 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18684 2024-09-17T21:20:27.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.260 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215593+0000", 2024-09-17T21:20:27.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215593+0000", 2024-09-17T21:20:27.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215593+0000", 2024-09-17T21:20:27.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2599999999999998e-07 2024-09-17T21:20:27.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215596+0000", 2024-09-17T21:20:27.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2189999999999998e-06 2024-09-17T21:20:27.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215602+0000", 2024-09-17T21:20:27.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6279999999999999e-06 2024-09-17T21:20:27.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215608+0000", 2024-09-17T21:20:27.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.874e-06 2024-09-17T21:20:27.262 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.262 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.262 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.263 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18717 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.263 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.216397+0000", 2024-09-17T21:20:27.263 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0025029060000000001, 2024-09-17T21:20:27.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002861667, 2024-09-17T21:20:27.263 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.263 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.263 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.263 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.263 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.263 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.263 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18717 2024-09-17T21:20:27.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.263 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216397+0000", 2024-09-17T21:20:27.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216397+0000", 2024-09-17T21:20:27.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216398+0000", 2024-09-17T21:20:27.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6199999999999998e-07 2024-09-17T21:20:27.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216400+0000", 2024-09-17T21:20:27.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1390000000000002e-06 2024-09-17T21:20:27.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216406+0000", 2024-09-17T21:20:27.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5359999999999998e-06 2024-09-17T21:20:27.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216413+0000", 2024-09-17T21:20:27.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0650000000000001e-06 2024-09-17T21:20:27.266 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.266 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.266 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.266 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18653 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.266 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.214748+0000", 2024-09-17T21:20:27.266 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0041522870000000002, 2024-09-17T21:20:27.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045346880000000003, 2024-09-17T21:20:27.266 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.266 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.266 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.266 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.267 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.267 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.267 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18653 2024-09-17T21:20:27.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.267 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214748+0000", 2024-09-17T21:20:27.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214748+0000", 2024-09-17T21:20:27.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214749+0000", 2024-09-17T21:20:27.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2099999999999996e-07 2024-09-17T21:20:27.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214751+0000", 2024-09-17T21:20:27.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5229999999999998e-06 2024-09-17T21:20:27.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214758+0000", 2024-09-17T21:20:27.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8959999999999997e-06 2024-09-17T21:20:27.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214763+0000", 2024-09-17T21:20:27.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3879999999999999e-06 2024-09-17T21:20:27.269 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.269 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.269 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.269 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18685 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.270 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215616+0000", 2024-09-17T21:20:27.270 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0032843630000000002, 2024-09-17T21:20:27.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00369046, 2024-09-17T21:20:27.270 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.270 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.270 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.270 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.270 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.270 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.270 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18685 2024-09-17T21:20:27.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.270 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215616+0000", 2024-09-17T21:20:27.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215616+0000", 2024-09-17T21:20:27.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215616+0000", 2024-09-17T21:20:27.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9299999999999998e-07 2024-09-17T21:20:27.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215622+0000", 2024-09-17T21:20:27.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6960000000000002e-06 2024-09-17T21:20:27.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215629+0000", 2024-09-17T21:20:27.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5629999999999997e-06 2024-09-17T21:20:27.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215634+0000", 2024-09-17T21:20:27.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3109999999999996e-06 2024-09-17T21:20:27.273 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.273 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.273 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.273 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18718 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.273 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.216422+0000", 2024-09-17T21:20:27.273 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0024784030000000001, 2024-09-17T21:20:27.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0029089839999999999, 2024-09-17T21:20:27.273 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.273 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.273 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.273 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.274 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.274 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.274 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18718 2024-09-17T21:20:27.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.274 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216422+0000", 2024-09-17T21:20:27.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216422+0000", 2024-09-17T21:20:27.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216422+0000", 2024-09-17T21:20:27.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0900000000000002e-07 2024-09-17T21:20:27.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216425+0000", 2024-09-17T21:20:27.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.385e-06 2024-09-17T21:20:27.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216431+0000", 2024-09-17T21:20:27.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6640000000000001e-06 2024-09-17T21:20:27.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216436+0000", 2024-09-17T21:20:27.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0499999999999999e-06 2024-09-17T21:20:27.276 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.276 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.276 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.277 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18654 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.277 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.214772+0000", 2024-09-17T21:20:27.277 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0041282869999999996, 2024-09-17T21:20:27.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045828650000000002, 2024-09-17T21:20:27.277 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.277 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.277 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.277 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.277 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.277 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.277 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18654 2024-09-17T21:20:27.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.277 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214772+0000", 2024-09-17T21:20:27.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214772+0000", 2024-09-17T21:20:27.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214773+0000", 2024-09-17T21:20:27.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1699999999999997e-07 2024-09-17T21:20:27.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214775+0000", 2024-09-17T21:20:27.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.266e-06 2024-09-17T21:20:27.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214782+0000", 2024-09-17T21:20:27.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6889999999999998e-06 2024-09-17T21:20:27.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214787+0000", 2024-09-17T21:20:27.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2650000000000004e-06 2024-09-17T21:20:27.280 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.280 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.280 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.280 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18686 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.280 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215642+0000", 2024-09-17T21:20:27.280 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0032582510000000002, 2024-09-17T21:20:27.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0037382280000000001, 2024-09-17T21:20:27.280 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.280 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.280 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.281 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.281 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.281 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.281 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18686 2024-09-17T21:20:27.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.281 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215642+0000", 2024-09-17T21:20:27.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215642+0000", 2024-09-17T21:20:27.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215643+0000", 2024-09-17T21:20:27.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2900000000000003e-07 2024-09-17T21:20:27.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215645+0000", 2024-09-17T21:20:27.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2800000000000002e-06 2024-09-17T21:20:27.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215651+0000", 2024-09-17T21:20:27.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5570000000000001e-06 2024-09-17T21:20:27.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215658+0000", 2024-09-17T21:20:27.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1569999999999997e-06 2024-09-17T21:20:27.283 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.283 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.283 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.283 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18719 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.284 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.216463+0000", 2024-09-17T21:20:27.284 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.002437247, 2024-09-17T21:20:27.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0029589210000000002, 2024-09-17T21:20:27.284 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.284 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.284 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.284 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.284 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.284 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.284 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18719 2024-09-17T21:20:27.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.284 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216463+0000", 2024-09-17T21:20:27.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216463+0000", 2024-09-17T21:20:27.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216464+0000", 2024-09-17T21:20:27.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8899999999999999e-07 2024-09-17T21:20:27.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216466+0000", 2024-09-17T21:20:27.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.695e-06 2024-09-17T21:20:27.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216475+0000", 2024-09-17T21:20:27.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5830000000000007e-06 2024-09-17T21:20:27.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216481+0000", 2024-09-17T21:20:27.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8710000000000002e-06 2024-09-17T21:20:27.287 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.287 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.287 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.287 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18655 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.287 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.214796+0000", 2024-09-17T21:20:27.287 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.004103654, 2024-09-17T21:20:27.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.004650174, 2024-09-17T21:20:27.287 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.287 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.287 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.287 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.288 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.288 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.288 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18655 2024-09-17T21:20:27.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.288 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214796+0000", 2024-09-17T21:20:27.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214796+0000", 2024-09-17T21:20:27.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214797+0000", 2024-09-17T21:20:27.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4300000000000003e-07 2024-09-17T21:20:27.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214799+0000", 2024-09-17T21:20:27.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3269999999999999e-06 2024-09-17T21:20:27.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214808+0000", 2024-09-17T21:20:27.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2009999999999999e-06 2024-09-17T21:20:27.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214813+0000", 2024-09-17T21:20:27.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3510000000000001e-06 2024-09-17T21:20:27.290 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.290 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.290 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.290 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18687 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.290 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215666+0000", 2024-09-17T21:20:27.291 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0032339140000000001, 2024-09-17T21:20:27.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038049870000000001, 2024-09-17T21:20:27.291 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.291 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.291 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.291 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.291 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.291 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.291 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18687 2024-09-17T21:20:27.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.291 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215666+0000", 2024-09-17T21:20:27.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215666+0000", 2024-09-17T21:20:27.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215667+0000", 2024-09-17T21:20:27.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8599999999999998e-07 2024-09-17T21:20:27.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215669+0000", 2024-09-17T21:20:27.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.469e-06 2024-09-17T21:20:27.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215676+0000", 2024-09-17T21:20:27.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8079999999999999e-06 2024-09-17T21:20:27.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215681+0000", 2024-09-17T21:20:27.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0590000000000002e-06 2024-09-17T21:20:27.293 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.294 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.294 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.294 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18720 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.294 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.216490+0000", 2024-09-17T21:20:27.294 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.002410517, 2024-09-17T21:20:27.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003005452, 2024-09-17T21:20:27.294 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.294 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.294 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.294 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.294 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.294 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.295 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18720 2024-09-17T21:20:27.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.295 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216490+0000", 2024-09-17T21:20:27.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216490+0000", 2024-09-17T21:20:27.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216490+0000", 2024-09-17T21:20:27.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9699999999999996e-07 2024-09-17T21:20:27.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216493+0000", 2024-09-17T21:20:27.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3420000000000002e-06 2024-09-17T21:20:27.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216499+0000", 2024-09-17T21:20:27.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6930000000000001e-06 2024-09-17T21:20:27.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216505+0000", 2024-09-17T21:20:27.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3630000000000002e-06 2024-09-17T21:20:27.297 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.297 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.297 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.297 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18656 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.297 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.214829+0000", 2024-09-17T21:20:27.297 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0040713540000000001, 2024-09-17T21:20:27.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0046902910000000001, 2024-09-17T21:20:27.298 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.298 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.298 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.298 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.298 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.298 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.298 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18656 2024-09-17T21:20:27.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.298 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214829+0000", 2024-09-17T21:20:27.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214829+0000", 2024-09-17T21:20:27.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214829+0000", 2024-09-17T21:20:27.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9800000000000003e-07 2024-09-17T21:20:27.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214832+0000", 2024-09-17T21:20:27.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5770000000000001e-06 2024-09-17T21:20:27.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214839+0000", 2024-09-17T21:20:27.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.64e-06 2024-09-17T21:20:27.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214844+0000", 2024-09-17T21:20:27.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.113e-06 2024-09-17T21:20:27.300 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.300 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.300 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.301 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18688 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.301 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215689+0000", 2024-09-17T21:20:27.301 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0032114209999999999, 2024-09-17T21:20:27.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038542139999999999, 2024-09-17T21:20:27.301 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.301 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.301 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.301 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.301 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.301 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.301 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18688 2024-09-17T21:20:27.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.301 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215689+0000", 2024-09-17T21:20:27.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215689+0000", 2024-09-17T21:20:27.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215689+0000", 2024-09-17T21:20:27.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:20:27.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215692+0000", 2024-09-17T21:20:27.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4370000000000001e-06 2024-09-17T21:20:27.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215700+0000", 2024-09-17T21:20:27.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2959999999999999e-06 2024-09-17T21:20:27.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215707+0000", 2024-09-17T21:20:27.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7599999999999997e-06 2024-09-17T21:20:27.304 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.304 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.304 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.304 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18725 3.16 3.8696fb6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.304 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.216513+0000", 2024-09-17T21:20:27.304 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0023871600000000001, 2024-09-17T21:20:27.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0030578480000000002, 2024-09-17T21:20:27.304 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.304 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.304 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.304 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.305 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.305 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.305 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18725 2024-09-17T21:20:27.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.305 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216513+0000", 2024-09-17T21:20:27.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216513+0000", 2024-09-17T21:20:27.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216513+0000", 2024-09-17T21:20:27.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:20:27.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216516+0000", 2024-09-17T21:20:27.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4389999999999999e-06 2024-09-17T21:20:27.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216523+0000", 2024-09-17T21:20:27.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2969999999999998e-06 2024-09-17T21:20:27.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216529+0000", 2024-09-17T21:20:27.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.485e-06 2024-09-17T21:20:27.307 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.307 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.307 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.307 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18657 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.307 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.214851+0000", 2024-09-17T21:20:27.308 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0040490400000000003, 2024-09-17T21:20:27.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0047547199999999996, 2024-09-17T21:20:27.308 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.308 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.308 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.308 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.308 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.308 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.308 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18657 2024-09-17T21:20:27.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.308 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214851+0000", 2024-09-17T21:20:27.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214851+0000", 2024-09-17T21:20:27.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214852+0000", 2024-09-17T21:20:27.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1900000000000002e-07 2024-09-17T21:20:27.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214854+0000", 2024-09-17T21:20:27.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:20:27.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214860+0000", 2024-09-17T21:20:27.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7390000000000002e-06 2024-09-17T21:20:27.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214866+0000", 2024-09-17T21:20:27.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.806e-06 2024-09-17T21:20:27.310 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.310 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.311 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.311 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18689 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.311 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215715+0000", 2024-09-17T21:20:27.311 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0031854489999999999, 2024-09-17T21:20:27.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0039155329999999997, 2024-09-17T21:20:27.311 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.311 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.311 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.311 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.311 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.311 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.311 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18689 2024-09-17T21:20:27.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.312 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215715+0000", 2024-09-17T21:20:27.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215715+0000", 2024-09-17T21:20:27.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215715+0000", 2024-09-17T21:20:27.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6199999999999998e-07 2024-09-17T21:20:27.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215717+0000", 2024-09-17T21:20:27.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3420000000000002e-06 2024-09-17T21:20:27.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215727+0000", 2024-09-17T21:20:27.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6500000000000008e-06 2024-09-17T21:20:27.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215733+0000", 2024-09-17T21:20:27.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4210000000000003e-06 2024-09-17T21:20:27.314 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.314 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.314 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.314 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18726 3.16 3.8696fb6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.314 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.216536+0000", 2024-09-17T21:20:27.314 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0023641560000000001, 2024-09-17T21:20:27.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031194159999999999, 2024-09-17T21:20:27.315 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.315 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.315 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.315 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.315 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.315 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.315 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18726 2024-09-17T21:20:27.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.315 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216536+0000", 2024-09-17T21:20:27.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216536+0000", 2024-09-17T21:20:27.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216536+0000", 2024-09-17T21:20:27.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.63e-07 2024-09-17T21:20:27.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216543+0000", 2024-09-17T21:20:27.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2650000000000002e-06 2024-09-17T21:20:27.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216552+0000", 2024-09-17T21:20:27.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.251e-06 2024-09-17T21:20:27.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216557+0000", 2024-09-17T21:20:27.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2809999999999999e-06 2024-09-17T21:20:27.317 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.317 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.317 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.318 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18658 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.318 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.214874+0000", 2024-09-17T21:20:27.318 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.004026127, 2024-09-17T21:20:27.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0048053080000000003, 2024-09-17T21:20:27.318 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.318 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.318 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.318 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.318 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.318 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.318 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18658 2024-09-17T21:20:27.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.319 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214874+0000", 2024-09-17T21:20:27.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214874+0000", 2024-09-17T21:20:27.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214875+0000", 2024-09-17T21:20:27.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7299999999999995e-07 2024-09-17T21:20:27.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214877+0000", 2024-09-17T21:20:27.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0669999999999999e-06 2024-09-17T21:20:27.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214883+0000", 2024-09-17T21:20:27.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3969999999999999e-06 2024-09-17T21:20:27.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214888+0000", 2024-09-17T21:20:27.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9969999999999998e-06 2024-09-17T21:20:27.321 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.321 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.321 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.321 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18690 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.321 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215749+0000", 2024-09-17T21:20:27.321 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0031514099999999999, 2024-09-17T21:20:27.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0039540770000000003, 2024-09-17T21:20:27.321 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.322 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.322 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.322 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.322 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.322 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.322 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18690 2024-09-17T21:20:27.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.322 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215749+0000", 2024-09-17T21:20:27.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215749+0000", 2024-09-17T21:20:27.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215749+0000", 2024-09-17T21:20:27.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0900000000000001e-07 2024-09-17T21:20:27.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215752+0000", 2024-09-17T21:20:27.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5000000000000002e-06 2024-09-17T21:20:27.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215759+0000", 2024-09-17T21:20:27.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2420000000000003e-06 2024-09-17T21:20:27.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215764+0000", 2024-09-17T21:20:27.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1460000000000004e-06 2024-09-17T21:20:27.324 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.324 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.324 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.325 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18727 3.16 3.8696fb6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.325 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.216566+0000", 2024-09-17T21:20:27.325 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0023343539999999999, 2024-09-17T21:20:27.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031606659999999999, 2024-09-17T21:20:27.325 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.325 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.325 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.325 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.325 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.325 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.325 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18727 2024-09-17T21:20:27.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.326 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216566+0000", 2024-09-17T21:20:27.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216566+0000", 2024-09-17T21:20:27.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216566+0000", 2024-09-17T21:20:27.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1500000000000004e-07 2024-09-17T21:20:27.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216568+0000", 2024-09-17T21:20:27.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0509999999999999e-06 2024-09-17T21:20:27.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216575+0000", 2024-09-17T21:20:27.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3369999999999996e-06 2024-09-17T21:20:27.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216581+0000", 2024-09-17T21:20:27.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7590000000000003e-06 2024-09-17T21:20:27.328 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.328 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.328 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.328 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18659 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.328 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.214896+0000", 2024-09-17T21:20:27.328 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0040046279999999997, 2024-09-17T21:20:27.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0048547859999999998, 2024-09-17T21:20:27.328 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.329 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.329 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.329 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.329 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.329 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.329 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18659 2024-09-17T21:20:27.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.329 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214896+0000", 2024-09-17T21:20:27.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214896+0000", 2024-09-17T21:20:27.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214896+0000", 2024-09-17T21:20:27.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7299999999999996e-07 2024-09-17T21:20:27.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214898+0000", 2024-09-17T21:20:27.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3020000000000002e-06 2024-09-17T21:20:27.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214912+0000", 2024-09-17T21:20:27.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3745e-05 2024-09-17T21:20:27.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214918+0000", 2024-09-17T21:20:27.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6720000000000001e-06 2024-09-17T21:20:27.331 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.331 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.331 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.332 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18691 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.332 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215773+0000", 2024-09-17T21:20:27.332 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.003127534, 2024-09-17T21:20:27.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.004016759, 2024-09-17T21:20:27.332 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.332 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.332 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.332 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18691 2024-09-17T21:20:27.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.332 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215773+0000", 2024-09-17T21:20:27.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215773+0000", 2024-09-17T21:20:27.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215773+0000", 2024-09-17T21:20:27.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9599999999999999e-07 2024-09-17T21:20:27.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215776+0000", 2024-09-17T21:20:27.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4600000000000002e-06 2024-09-17T21:20:27.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215784+0000", 2024-09-17T21:20:27.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8779999999999998e-06 2024-09-17T21:20:27.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215789+0000", 2024-09-17T21:20:27.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2530000000000004e-06 2024-09-17T21:20:27.335 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.335 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.335 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.335 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18728 3.16 3.8696fb6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.335 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.216587+0000", 2024-09-17T21:20:27.335 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0023131459999999999, 2024-09-17T21:20:27.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003227097, 2024-09-17T21:20:27.335 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.335 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.336 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.336 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.336 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.336 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.336 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18728 2024-09-17T21:20:27.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.336 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216587+0000", 2024-09-17T21:20:27.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216587+0000", 2024-09-17T21:20:27.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216587+0000", 2024-09-17T21:20:27.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:20:27.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216589+0000", 2024-09-17T21:20:27.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.928e-06 2024-09-17T21:20:27.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216595+0000", 2024-09-17T21:20:27.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.801e-06 2024-09-17T21:20:27.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216600+0000", 2024-09-17T21:20:27.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9479999999999999e-06 2024-09-17T21:20:27.338 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.338 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.338 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.339 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18660 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.339 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.214927+0000", 2024-09-17T21:20:27.339 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0039729960000000003, 2024-09-17T21:20:27.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0049120819999999999, 2024-09-17T21:20:27.339 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.339 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.339 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.339 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.339 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.339 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.339 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18660 2024-09-17T21:20:27.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.339 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214927+0000", 2024-09-17T21:20:27.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214927+0000", 2024-09-17T21:20:27.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214928+0000", 2024-09-17T21:20:27.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9900000000000001e-07 2024-09-17T21:20:27.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214930+0000", 2024-09-17T21:20:27.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6570000000000001e-06 2024-09-17T21:20:27.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214937+0000", 2024-09-17T21:20:27.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5930000000000002e-06 2024-09-17T21:20:27.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214943+0000", 2024-09-17T21:20:27.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9739999999999999e-06 2024-09-17T21:20:27.342 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.342 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.342 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.342 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18692 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.342 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215796+0000", 2024-09-17T21:20:27.342 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0031037769999999998, 2024-09-17T21:20:27.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040741029999999999, 2024-09-17T21:20:27.342 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.343 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.343 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.343 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.343 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.343 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.343 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18692 2024-09-17T21:20:27.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.343 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215796+0000", 2024-09-17T21:20:27.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215796+0000", 2024-09-17T21:20:27.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215797+0000", 2024-09-17T21:20:27.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5299999999999999e-07 2024-09-17T21:20:27.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215799+0000", 2024-09-17T21:20:27.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.187e-06 2024-09-17T21:20:27.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215806+0000", 2024-09-17T21:20:27.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:20:27.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215812+0000", 2024-09-17T21:20:27.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1709999999999999e-06 2024-09-17T21:20:27.345 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.345 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.346 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.346 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18661 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.346 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.214949+0000", 2024-09-17T21:20:27.346 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0039507570000000001, 2024-09-17T21:20:27.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0049452810000000002, 2024-09-17T21:20:27.346 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.346 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.346 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.346 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.346 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.346 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.347 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18661 2024-09-17T21:20:27.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.347 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214949+0000", 2024-09-17T21:20:27.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214949+0000", 2024-09-17T21:20:27.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214950+0000", 2024-09-17T21:20:27.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:20:27.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214956+0000", 2024-09-17T21:20:27.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8499999999999999e-06 2024-09-17T21:20:27.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214970+0000", 2024-09-17T21:20:27.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4836e-05 2024-09-17T21:20:27.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214976+0000", 2024-09-17T21:20:27.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6960000000000002e-06 2024-09-17T21:20:27.349 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.349 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.349 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.349 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18693 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.349 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215821+0000", 2024-09-17T21:20:27.349 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0030794289999999998, 2024-09-17T21:20:27.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040974569999999997, 2024-09-17T21:20:27.350 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.350 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.350 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.350 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.350 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.350 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.350 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18693 2024-09-17T21:20:27.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.350 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215821+0000", 2024-09-17T21:20:27.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215821+0000", 2024-09-17T21:20:27.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215821+0000", 2024-09-17T21:20:27.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2900000000000004e-07 2024-09-17T21:20:27.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215824+0000", 2024-09-17T21:20:27.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2989999999999999e-06 2024-09-17T21:20:27.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215832+0000", 2024-09-17T21:20:27.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5329999999999994e-06 2024-09-17T21:20:27.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215838+0000", 2024-09-17T21:20:27.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4689999999999996e-06 2024-09-17T21:20:27.352 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.352 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.352 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.353 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18662 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.353 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.214984+0000", 2024-09-17T21:20:27.353 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0039164680000000002, 2024-09-17T21:20:27.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0049627250000000003, 2024-09-17T21:20:27.353 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.353 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.353 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.353 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18662 2024-09-17T21:20:27.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.353 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214984+0000", 2024-09-17T21:20:27.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214984+0000", 2024-09-17T21:20:27.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214984+0000", 2024-09-17T21:20:27.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0200000000000002e-07 2024-09-17T21:20:27.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214987+0000", 2024-09-17T21:20:27.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.26e-06 2024-09-17T21:20:27.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214993+0000", 2024-09-17T21:20:27.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1340000000000001e-06 2024-09-17T21:20:27.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.214997+0000", 2024-09-17T21:20:27.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7010000000000004e-06 2024-09-17T21:20:27.356 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.356 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.356 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.356 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18694 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.356 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215846+0000", 2024-09-17T21:20:27.356 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0030544840000000001, 2024-09-17T21:20:27.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041348519999999996, 2024-09-17T21:20:27.356 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.356 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.357 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.357 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.357 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.357 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.357 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18694 2024-09-17T21:20:27.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.357 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215846+0000", 2024-09-17T21:20:27.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215846+0000", 2024-09-17T21:20:27.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215846+0000", 2024-09-17T21:20:27.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4000000000000001e-07 2024-09-17T21:20:27.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215849+0000", 2024-09-17T21:20:27.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.368e-06 2024-09-17T21:20:27.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215858+0000", 2024-09-17T21:20:27.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2900000000000008e-06 2024-09-17T21:20:27.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215863+0000", 2024-09-17T21:20:27.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1460000000000004e-06 2024-09-17T21:20:27.359 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.359 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.359 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.359 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18663 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.359 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215006+0000", 2024-09-17T21:20:27.359 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0038936769999999999, 2024-09-17T21:20:27.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050130569999999996, 2024-09-17T21:20:27.360 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.360 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.360 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.360 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.360 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.360 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.360 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18663 2024-09-17T21:20:27.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.360 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215006+0000", 2024-09-17T21:20:27.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215006+0000", 2024-09-17T21:20:27.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215007+0000", 2024-09-17T21:20:27.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:20:27.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215009+0000", 2024-09-17T21:20:27.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1770000000000001e-06 2024-09-17T21:20:27.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215016+0000", 2024-09-17T21:20:27.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:20:27.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215022+0000", 2024-09-17T21:20:27.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1319999999999999e-06 2024-09-17T21:20:27.362 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.362 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.362 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.362 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18695 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.363 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215871+0000", 2024-09-17T21:20:27.363 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0030293249999999998, 2024-09-17T21:20:27.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041728759999999998, 2024-09-17T21:20:27.363 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.363 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.363 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.363 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.363 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.363 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.363 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18695 2024-09-17T21:20:27.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.363 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215871+0000", 2024-09-17T21:20:27.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215871+0000", 2024-09-17T21:20:27.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215871+0000", 2024-09-17T21:20:27.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1500000000000004e-07 2024-09-17T21:20:27.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215874+0000", 2024-09-17T21:20:27.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.283e-06 2024-09-17T21:20:27.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215888+0000", 2024-09-17T21:20:27.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4061e-05 2024-09-17T21:20:27.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215893+0000", 2024-09-17T21:20:27.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6749999999999999e-06 2024-09-17T21:20:27.365 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.365 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.365 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.366 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18664 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.366 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215029+0000", 2024-09-17T21:20:27.366 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0038706830000000002, 2024-09-17T21:20:27.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050399329999999999, 2024-09-17T21:20:27.366 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.366 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.366 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.366 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.366 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.366 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.366 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18664 2024-09-17T21:20:27.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.366 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215029+0000", 2024-09-17T21:20:27.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215029+0000", 2024-09-17T21:20:27.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215030+0000", 2024-09-17T21:20:27.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.37e-07 2024-09-17T21:20:27.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215032+0000", 2024-09-17T21:20:27.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3929999999999998e-06 2024-09-17T21:20:27.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215040+0000", 2024-09-17T21:20:27.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.712e-06 2024-09-17T21:20:27.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215046+0000", 2024-09-17T21:20:27.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0120000000000002e-06 2024-09-17T21:20:27.369 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.369 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.369 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.369 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18697 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.369 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215902+0000", 2024-09-17T21:20:27.369 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.002997698, 2024-09-17T21:20:27.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041907400000000001, 2024-09-17T21:20:27.369 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.369 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.369 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.369 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.370 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.370 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.370 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18697 2024-09-17T21:20:27.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.370 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215902+0000", 2024-09-17T21:20:27.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215902+0000", 2024-09-17T21:20:27.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215903+0000", 2024-09-17T21:20:27.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4799999999999998e-07 2024-09-17T21:20:27.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215906+0000", 2024-09-17T21:20:27.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5349999999999999e-06 2024-09-17T21:20:27.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215912+0000", 2024-09-17T21:20:27.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6980000000000001e-06 2024-09-17T21:20:27.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215918+0000", 2024-09-17T21:20:27.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.344e-06 2024-09-17T21:20:27.372 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.372 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.372 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.372 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18665 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.372 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215054+0000", 2024-09-17T21:20:27.372 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0038458939999999999, 2024-09-17T21:20:27.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050641180000000003, 2024-09-17T21:20:27.373 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.373 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.373 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.373 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.373 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.373 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.373 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18665 2024-09-17T21:20:27.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.373 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215054+0000", 2024-09-17T21:20:27.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215054+0000", 2024-09-17T21:20:27.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215055+0000", 2024-09-17T21:20:27.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:20:27.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215057+0000", 2024-09-17T21:20:27.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1890000000000001e-06 2024-09-17T21:20:27.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215063+0000", 2024-09-17T21:20:27.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4869999999999999e-06 2024-09-17T21:20:27.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215069+0000", 2024-09-17T21:20:27.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:20:27.375 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.376 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.376 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.376 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18698 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.376 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215925+0000", 2024-09-17T21:20:27.376 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0029746949999999999, 2024-09-17T21:20:27.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042324279999999999, 2024-09-17T21:20:27.376 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.376 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.376 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.376 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.376 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.376 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.377 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18698 2024-09-17T21:20:27.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.377 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215925+0000", 2024-09-17T21:20:27.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215925+0000", 2024-09-17T21:20:27.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215926+0000", 2024-09-17T21:20:27.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7e-07 2024-09-17T21:20:27.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215932+0000", 2024-09-17T21:20:27.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.947e-06 2024-09-17T21:20:27.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215938+0000", 2024-09-17T21:20:27.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5690000000000001e-06 2024-09-17T21:20:27.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215946+0000", 2024-09-17T21:20:27.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4379999999999998e-06 2024-09-17T21:20:27.379 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.379 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.379 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.379 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18666 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.379 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215100+0000", 2024-09-17T21:20:27.379 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0038003139999999999, 2024-09-17T21:20:27.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050858350000000004, 2024-09-17T21:20:27.380 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.380 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.380 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.380 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.380 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.380 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.380 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18666 2024-09-17T21:20:27.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.380 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215100+0000", 2024-09-17T21:20:27.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215100+0000", 2024-09-17T21:20:27.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215101+0000", 2024-09-17T21:20:27.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3300000000000001e-07 2024-09-17T21:20:27.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215103+0000", 2024-09-17T21:20:27.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8380000000000002e-06 2024-09-17T21:20:27.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215112+0000", 2024-09-17T21:20:27.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0380000000000005e-06 2024-09-17T21:20:27.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215118+0000", 2024-09-17T21:20:27.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7699999999999998e-06 2024-09-17T21:20:27.382 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.383 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.383 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.383 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18699 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.383 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215954+0000", 2024-09-17T21:20:27.383 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0029463390000000001, 2024-09-17T21:20:27.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.004255778, 2024-09-17T21:20:27.383 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.383 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.383 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.383 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.383 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.383 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.384 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18699 2024-09-17T21:20:27.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.384 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215954+0000", 2024-09-17T21:20:27.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215954+0000", 2024-09-17T21:20:27.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215954+0000", 2024-09-17T21:20:27.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8999999999999996e-07 2024-09-17T21:20:27.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215957+0000", 2024-09-17T21:20:27.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2970000000000002e-06 2024-09-17T21:20:27.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215963+0000", 2024-09-17T21:20:27.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1040000000000003e-06 2024-09-17T21:20:27.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215968+0000", 2024-09-17T21:20:27.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6169999999999998e-06 2024-09-17T21:20:27.386 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.386 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.386 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.386 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18667 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.386 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215126+0000", 2024-09-17T21:20:27.386 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0037736660000000002, 2024-09-17T21:20:27.387 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0051084820000000001, 2024-09-17T21:20:27.387 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.387 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.387 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.387 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.387 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.387 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.387 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18667 2024-09-17T21:20:27.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.387 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.387 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.387 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.387 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215126+0000", 2024-09-17T21:20:27.387 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.388 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.388 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.388 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215126+0000", 2024-09-17T21:20:27.388 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.388 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.388 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.388 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215127+0000", 2024-09-17T21:20:27.388 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2600000000000002e-07 2024-09-17T21:20:27.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.388 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.388 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.388 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215129+0000", 2024-09-17T21:20:27.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.357e-06 2024-09-17T21:20:27.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.389 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.389 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215136+0000", 2024-09-17T21:20:27.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6340000000000003e-06 2024-09-17T21:20:27.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.389 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.389 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215141+0000", 2024-09-17T21:20:27.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:20:27.389 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.389 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18700 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215975+0000", 2024-09-17T21:20:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.002925048, 2024-09-17T21:20:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042835540000000002, 2024-09-17T21:20:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18700 2024-09-17T21:20:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215975+0000", 2024-09-17T21:20:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215975+0000", 2024-09-17T21:20:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215976+0000", 2024-09-17T21:20:27.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:20:27.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.392 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215978+0000", 2024-09-17T21:20:27.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.995e-06 2024-09-17T21:20:27.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.392 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215984+0000", 2024-09-17T21:20:27.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1290000000000001e-06 2024-09-17T21:20:27.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215988+0000", 2024-09-17T21:20:27.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7010000000000004e-06 2024-09-17T21:20:27.393 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.393 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.393 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.393 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18668 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.393 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215149+0000", 2024-09-17T21:20:27.393 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0037511839999999999, 2024-09-17T21:20:27.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0051351849999999996, 2024-09-17T21:20:27.394 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.394 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.394 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.394 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.394 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.394 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.394 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18668 2024-09-17T21:20:27.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.394 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215149+0000", 2024-09-17T21:20:27.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215149+0000", 2024-09-17T21:20:27.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215149+0000", 2024-09-17T21:20:27.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0900000000000002e-07 2024-09-17T21:20:27.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215152+0000", 2024-09-17T21:20:27.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4559999999999999e-06 2024-09-17T21:20:27.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215160+0000", 2024-09-17T21:20:27.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.824e-06 2024-09-17T21:20:27.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215165+0000", 2024-09-17T21:20:27.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4500000000000003e-06 2024-09-17T21:20:27.396 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.396 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.396 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.397 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18701 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.397 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215996+0000", 2024-09-17T21:20:27.397 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0029040429999999998, 2024-09-17T21:20:27.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043116320000000001, 2024-09-17T21:20:27.397 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.397 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.397 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.397 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.397 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.397 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.397 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18701 2024-09-17T21:20:27.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.398 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215996+0000", 2024-09-17T21:20:27.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215996+0000", 2024-09-17T21:20:27.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215997+0000", 2024-09-17T21:20:27.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2399999999999999e-07 2024-09-17T21:20:27.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215999+0000", 2024-09-17T21:20:27.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.357e-06 2024-09-17T21:20:27.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216005+0000", 2024-09-17T21:20:27.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2369999999999997e-06 2024-09-17T21:20:27.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.400 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216012+0000", 2024-09-17T21:20:27.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4910000000000002e-06 2024-09-17T21:20:27.400 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.400 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.400 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.400 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18669 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.400 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215184+0000", 2024-09-17T21:20:27.400 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0037159739999999999, 2024-09-17T21:20:27.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0051645720000000001, 2024-09-17T21:20:27.401 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.401 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.401 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.401 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.401 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.401 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.401 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18669 2024-09-17T21:20:27.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.401 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215184+0000", 2024-09-17T21:20:27.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215184+0000", 2024-09-17T21:20:27.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215185+0000", 2024-09-17T21:20:27.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2900000000000003e-07 2024-09-17T21:20:27.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215187+0000", 2024-09-17T21:20:27.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.695e-06 2024-09-17T21:20:27.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215202+0000", 2024-09-17T21:20:27.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4997999999999999e-05 2024-09-17T21:20:27.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215209+0000", 2024-09-17T21:20:27.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0329999999999997e-06 2024-09-17T21:20:27.403 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.403 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.404 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.404 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18702 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.404 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.216019+0000", 2024-09-17T21:20:27.404 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0028807559999999999, 2024-09-17T21:20:27.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.004354269, 2024-09-17T21:20:27.404 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.404 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.404 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.404 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.404 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.404 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.404 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18702 2024-09-17T21:20:27.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.405 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216019+0000", 2024-09-17T21:20:27.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216019+0000", 2024-09-17T21:20:27.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216020+0000", 2024-09-17T21:20:27.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2800000000000002e-07 2024-09-17T21:20:27.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216022+0000", 2024-09-17T21:20:27.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4820000000000001e-06 2024-09-17T21:20:27.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216030+0000", 2024-09-17T21:20:27.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7719999999999994e-06 2024-09-17T21:20:27.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216036+0000", 2024-09-17T21:20:27.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5029999999999997e-06 2024-09-17T21:20:27.407 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.407 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.407 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.407 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18670 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.407 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215229+0000", 2024-09-17T21:20:27.407 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0036709249999999998, 2024-09-17T21:20:27.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0051794240000000002, 2024-09-17T21:20:27.408 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.408 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.408 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.408 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.408 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.408 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.408 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18670 2024-09-17T21:20:27.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.408 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.408 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215229+0000", 2024-09-17T21:20:27.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215229+0000", 2024-09-17T21:20:27.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215230+0000", 2024-09-17T21:20:27.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7400000000000003e-07 2024-09-17T21:20:27.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215233+0000", 2024-09-17T21:20:27.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7630000000000001e-06 2024-09-17T21:20:27.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215240+0000", 2024-09-17T21:20:27.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4540000000000001e-06 2024-09-17T21:20:27.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215246+0000", 2024-09-17T21:20:27.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4410000000000001e-06 2024-09-17T21:20:27.410 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.410 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.410 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.411 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18703 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.411 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.216043+0000", 2024-09-17T21:20:27.411 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0028568410000000002, 2024-09-17T21:20:27.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043895089999999998, 2024-09-17T21:20:27.411 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.411 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.411 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.411 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.411 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.411 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.411 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18703 2024-09-17T21:20:27.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.412 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.412 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216043+0000", 2024-09-17T21:20:27.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.412 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216043+0000", 2024-09-17T21:20:27.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.412 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216044+0000", 2024-09-17T21:20:27.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3799999999999997e-07 2024-09-17T21:20:27.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216046+0000", 2024-09-17T21:20:27.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1059999999999998e-06 2024-09-17T21:20:27.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216053+0000", 2024-09-17T21:20:27.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.562e-06 2024-09-17T21:20:27.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216058+0000", 2024-09-17T21:20:27.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1590000000000001e-06 2024-09-17T21:20:27.414 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.414 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.414 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.414 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18671 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.414 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215254+0000", 2024-09-17T21:20:27.414 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.003646088, 2024-09-17T21:20:27.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052058440000000003, 2024-09-17T21:20:27.414 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.415 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.415 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.415 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.415 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.415 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.415 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18671 2024-09-17T21:20:27.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.415 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215254+0000", 2024-09-17T21:20:27.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215254+0000", 2024-09-17T21:20:27.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215255+0000", 2024-09-17T21:20:27.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5300000000000004e-07 2024-09-17T21:20:27.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215257+0000", 2024-09-17T21:20:27.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.345e-06 2024-09-17T21:20:27.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215265+0000", 2024-09-17T21:20:27.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5730000000000002e-06 2024-09-17T21:20:27.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215270+0000", 2024-09-17T21:20:27.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2789999999999998e-06 2024-09-17T21:20:27.417 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.417 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.417 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.418 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18704 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.418 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.216065+0000", 2024-09-17T21:20:27.418 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0028349930000000001, 2024-09-17T21:20:27.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0044185520000000001, 2024-09-17T21:20:27.418 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.418 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.418 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.418 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.418 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.418 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.418 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18704 2024-09-17T21:20:27.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.418 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216065+0000", 2024-09-17T21:20:27.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216065+0000", 2024-09-17T21:20:27.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216066+0000", 2024-09-17T21:20:27.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6400000000000002e-07 2024-09-17T21:20:27.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216068+0000", 2024-09-17T21:20:27.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.311e-06 2024-09-17T21:20:27.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216083+0000", 2024-09-17T21:20:27.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4514e-05 2024-09-17T21:20:27.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216088+0000", 2024-09-17T21:20:27.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6690000000000003e-06 2024-09-17T21:20:27.421 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.421 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.421 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.421 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18672 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.421 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215277+0000", 2024-09-17T21:20:27.421 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0036230519999999999, 2024-09-17T21:20:27.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052363069999999999, 2024-09-17T21:20:27.421 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.421 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.422 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.422 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.422 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.422 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.422 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18672 2024-09-17T21:20:27.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.422 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.422 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215277+0000", 2024-09-17T21:20:27.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215277+0000", 2024-09-17T21:20:27.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215277+0000", 2024-09-17T21:20:27.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0900000000000002e-07 2024-09-17T21:20:27.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215280+0000", 2024-09-17T21:20:27.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0590000000000001e-06 2024-09-17T21:20:27.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215286+0000", 2024-09-17T21:20:27.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.229e-06 2024-09-17T21:20:27.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215291+0000", 2024-09-17T21:20:27.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5740000000000003e-06 2024-09-17T21:20:27.424 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.424 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.424 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.425 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18705 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.425 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.216096+0000", 2024-09-17T21:20:27.425 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0028037560000000001, 2024-09-17T21:20:27.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0044529579999999999, 2024-09-17T21:20:27.425 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.425 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.425 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.425 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.425 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.425 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.425 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18705 2024-09-17T21:20:27.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.425 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.426 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.426 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216096+0000", 2024-09-17T21:20:27.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.426 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.426 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216096+0000", 2024-09-17T21:20:27.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.426 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.426 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216097+0000", 2024-09-17T21:20:27.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:20:27.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216099+0000", 2024-09-17T21:20:27.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4169999999999999e-06 2024-09-17T21:20:27.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216107+0000", 2024-09-17T21:20:27.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.469e-06 2024-09-17T21:20:27.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216112+0000", 2024-09-17T21:20:27.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7450000000000001e-06 2024-09-17T21:20:27.428 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.428 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.428 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.428 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18673 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.428 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215298+0000", 2024-09-17T21:20:27.428 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0036022409999999999, 2024-09-17T21:20:27.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052756560000000001, 2024-09-17T21:20:27.428 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.428 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.428 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.428 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.429 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.429 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.429 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18673 2024-09-17T21:20:27.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.429 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215298+0000", 2024-09-17T21:20:27.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215298+0000", 2024-09-17T21:20:27.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215298+0000", 2024-09-17T21:20:27.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9999999999999998e-07 2024-09-17T21:20:27.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215306+0000", 2024-09-17T21:20:27.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2359999999999998e-06 2024-09-17T21:20:27.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215312+0000", 2024-09-17T21:20:27.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5610000000000004e-06 2024-09-17T21:20:27.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215317+0000", 2024-09-17T21:20:27.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1569999999999999e-06 2024-09-17T21:20:27.431 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.431 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.431 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.431 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18706 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.431 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.216121+0000", 2024-09-17T21:20:27.432 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0027795160000000001, 2024-09-17T21:20:27.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.004479111, 2024-09-17T21:20:27.432 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.432 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.432 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.432 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.432 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.432 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.432 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18706 2024-09-17T21:20:27.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.432 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216121+0000", 2024-09-17T21:20:27.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216121+0000", 2024-09-17T21:20:27.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216121+0000", 2024-09-17T21:20:27.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2800000000000002e-07 2024-09-17T21:20:27.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216123+0000", 2024-09-17T21:20:27.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.373e-06 2024-09-17T21:20:27.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216130+0000", 2024-09-17T21:20:27.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6409999999999996e-06 2024-09-17T21:20:27.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216135+0000", 2024-09-17T21:20:27.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2719999999999997e-06 2024-09-17T21:20:27.434 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.435 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.435 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.435 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18674 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.435 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215324+0000", 2024-09-17T21:20:27.435 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.003576011, 2024-09-17T21:20:27.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052994950000000004, 2024-09-17T21:20:27.435 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.435 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.435 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.435 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.435 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.436 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.436 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18674 2024-09-17T21:20:27.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.436 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215324+0000", 2024-09-17T21:20:27.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215324+0000", 2024-09-17T21:20:27.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215325+0000", 2024-09-17T21:20:27.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4000000000000001e-07 2024-09-17T21:20:27.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215327+0000", 2024-09-17T21:20:27.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9479999999999998e-06 2024-09-17T21:20:27.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215333+0000", 2024-09-17T21:20:27.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8989999999999998e-06 2024-09-17T21:20:27.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215347+0000", 2024-09-17T21:20:27.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4769999999999999e-05 2024-09-17T21:20:27.438 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.438 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.438 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.438 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18707 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.438 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.216143+0000", 2024-09-17T21:20:27.438 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0027571129999999998, 2024-09-17T21:20:27.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045058720000000002, 2024-09-17T21:20:27.439 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.439 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.439 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.439 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.439 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.439 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.439 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18707 2024-09-17T21:20:27.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.439 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216143+0000", 2024-09-17T21:20:27.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216143+0000", 2024-09-17T21:20:27.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216144+0000", 2024-09-17T21:20:27.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0500000000000003e-07 2024-09-17T21:20:27.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216146+0000", 2024-09-17T21:20:27.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2110000000000001e-06 2024-09-17T21:20:27.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216152+0000", 2024-09-17T21:20:27.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3369999999999996e-06 2024-09-17T21:20:27.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216157+0000", 2024-09-17T21:20:27.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2100000000000001e-06 2024-09-17T21:20:27.441 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.441 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.441 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.442 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18675 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.442 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215357+0000", 2024-09-17T21:20:27.442 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0035432319999999999, 2024-09-17T21:20:27.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0053156100000000001, 2024-09-17T21:20:27.442 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.442 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.442 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.442 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.442 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.442 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.442 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18675 2024-09-17T21:20:27.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.443 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215357+0000", 2024-09-17T21:20:27.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215357+0000", 2024-09-17T21:20:27.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215357+0000", 2024-09-17T21:20:27.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7100000000000002e-07 2024-09-17T21:20:27.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215360+0000", 2024-09-17T21:20:27.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6259999999999999e-06 2024-09-17T21:20:27.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215367+0000", 2024-09-17T21:20:27.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9399999999999996e-06 2024-09-17T21:20:27.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215373+0000", 2024-09-17T21:20:27.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6030000000000004e-06 2024-09-17T21:20:27.445 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.445 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.445 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.445 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18708 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.445 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.216166+0000", 2024-09-17T21:20:27.445 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0027340860000000002, 2024-09-17T21:20:27.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045383130000000004, 2024-09-17T21:20:27.446 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.446 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.446 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.446 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.446 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.446 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.446 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18708 2024-09-17T21:20:27.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.446 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216166+0000", 2024-09-17T21:20:27.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216166+0000", 2024-09-17T21:20:27.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216167+0000", 2024-09-17T21:20:27.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9200000000000001e-07 2024-09-17T21:20:27.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216169+0000", 2024-09-17T21:20:27.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4200000000000001e-06 2024-09-17T21:20:27.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216175+0000", 2024-09-17T21:20:27.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:20:27.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216182+0000", 2024-09-17T21:20:27.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1260000000000003e-06 2024-09-17T21:20:27.448 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.448 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.449 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.449 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18676 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.449 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215381+0000", 2024-09-17T21:20:27.449 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.003519599, 2024-09-17T21:20:27.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0053580720000000002, 2024-09-17T21:20:27.449 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.449 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.449 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.449 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.449 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.449 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.449 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18676 2024-09-17T21:20:27.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.450 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215381+0000", 2024-09-17T21:20:27.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215381+0000", 2024-09-17T21:20:27.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215381+0000", 2024-09-17T21:20:27.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.51e-07 2024-09-17T21:20:27.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215383+0000", 2024-09-17T21:20:27.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4250000000000001e-06 2024-09-17T21:20:27.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215390+0000", 2024-09-17T21:20:27.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.365e-06 2024-09-17T21:20:27.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215396+0000", 2024-09-17T21:20:27.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:20:27.452 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.452 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.452 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.452 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18709 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.453 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.216190+0000", 2024-09-17T21:20:27.453 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.002710367, 2024-09-17T21:20:27.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.004582872, 2024-09-17T21:20:27.453 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.453 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.453 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.453 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.453 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.453 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.453 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18709 2024-09-17T21:20:27.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.453 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216190+0000", 2024-09-17T21:20:27.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216190+0000", 2024-09-17T21:20:27.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216190+0000", 2024-09-17T21:20:27.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.32e-07 2024-09-17T21:20:27.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216193+0000", 2024-09-17T21:20:27.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.418e-06 2024-09-17T21:20:27.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216202+0000", 2024-09-17T21:20:27.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8199999999999992e-06 2024-09-17T21:20:27.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216208+0000", 2024-09-17T21:20:27.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.254e-06 2024-09-17T21:20:27.456 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.456 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.456 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.456 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18677 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.456 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215403+0000", 2024-09-17T21:20:27.456 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0034971479999999998, 2024-09-17T21:20:27.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005394261, 2024-09-17T21:20:27.456 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.456 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.456 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.456 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.456 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.457 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.457 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18677 2024-09-17T21:20:27.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.457 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215403+0000", 2024-09-17T21:20:27.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215403+0000", 2024-09-17T21:20:27.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215403+0000", 2024-09-17T21:20:27.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:20:27.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215406+0000", 2024-09-17T21:20:27.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4080000000000001e-06 2024-09-17T21:20:27.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215413+0000", 2024-09-17T21:20:27.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7109999999999998e-06 2024-09-17T21:20:27.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215418+0000", 2024-09-17T21:20:27.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1150000000000002e-06 2024-09-17T21:20:27.459 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.459 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.459 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.459 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18710 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.459 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.216215+0000", 2024-09-17T21:20:27.459 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.002684882, 2024-09-17T21:20:27.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0046691450000000004, 2024-09-17T21:20:27.460 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.460 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.460 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.460 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.460 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.460 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.460 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18710 2024-09-17T21:20:27.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.460 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216215+0000", 2024-09-17T21:20:27.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216215+0000", 2024-09-17T21:20:27.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216216+0000", 2024-09-17T21:20:27.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:20:27.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216222+0000", 2024-09-17T21:20:27.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3559999999999998e-06 2024-09-17T21:20:27.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216229+0000", 2024-09-17T21:20:27.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2960000000000001e-06 2024-09-17T21:20:27.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216238+0000", 2024-09-17T21:20:27.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3909999999999998e-06 2024-09-17T21:20:27.463 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.463 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.463 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.463 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18678 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.463 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215448+0000", 2024-09-17T21:20:27.463 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.003452063, 2024-09-17T21:20:27.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0054736480000000002, 2024-09-17T21:20:27.463 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.463 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.463 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.463 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.464 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.464 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.464 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18678 2024-09-17T21:20:27.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.464 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215448+0000", 2024-09-17T21:20:27.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215448+0000", 2024-09-17T21:20:27.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215449+0000", 2024-09-17T21:20:27.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0399999999999995e-07 2024-09-17T21:20:27.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215452+0000", 2024-09-17T21:20:27.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8899999999999999e-06 2024-09-17T21:20:27.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215460+0000", 2024-09-17T21:20:27.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1210000000000007e-06 2024-09-17T21:20:27.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215466+0000", 2024-09-17T21:20:27.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7559999999999996e-06 2024-09-17T21:20:27.466 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.466 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.466 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.466 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18711 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.466 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.216262+0000", 2024-09-17T21:20:27.467 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0026384540000000001, 2024-09-17T21:20:27.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.004687236, 2024-09-17T21:20:27.467 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.467 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.467 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.467 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.467 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.467 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.467 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18711 2024-09-17T21:20:27.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.467 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216262+0000", 2024-09-17T21:20:27.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216262+0000", 2024-09-17T21:20:27.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216262+0000", 2024-09-17T21:20:27.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:20:27.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216265+0000", 2024-09-17T21:20:27.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6840000000000001e-06 2024-09-17T21:20:27.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216272+0000", 2024-09-17T21:20:27.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3690000000000001e-06 2024-09-17T21:20:27.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216278+0000", 2024-09-17T21:20:27.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5079999999999996e-06 2024-09-17T21:20:27.469 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.470 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.470 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.470 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18679 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.470 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215475+0000", 2024-09-17T21:20:27.470 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0034246540000000001, 2024-09-17T21:20:27.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0054975029999999999, 2024-09-17T21:20:27.470 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.470 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.470 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.470 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.470 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.471 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.471 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18679 2024-09-17T21:20:27.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.471 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215475+0000", 2024-09-17T21:20:27.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215475+0000", 2024-09-17T21:20:27.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215476+0000", 2024-09-17T21:20:27.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5599999999999995e-07 2024-09-17T21:20:27.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215479+0000", 2024-09-17T21:20:27.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5110000000000002e-06 2024-09-17T21:20:27.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215485+0000", 2024-09-17T21:20:27.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6220000000000003e-06 2024-09-17T21:20:27.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215491+0000", 2024-09-17T21:20:27.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5999999999999997e-06 2024-09-17T21:20:27.473 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.473 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.473 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.473 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18712 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.473 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.216286+0000", 2024-09-17T21:20:27.473 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.002613864, 2024-09-17T21:20:27.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0047118469999999999, 2024-09-17T21:20:27.474 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.474 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.474 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.474 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.474 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.474 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.474 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18712 2024-09-17T21:20:27.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.474 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216286+0000", 2024-09-17T21:20:27.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216286+0000", 2024-09-17T21:20:27.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216287+0000", 2024-09-17T21:20:27.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6499999999999999e-07 2024-09-17T21:20:27.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216289+0000", 2024-09-17T21:20:27.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.43e-06 2024-09-17T21:20:27.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216296+0000", 2024-09-17T21:20:27.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4269999999999996e-06 2024-09-17T21:20:27.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216302+0000", 2024-09-17T21:20:27.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8629999999999996e-06 2024-09-17T21:20:27.476 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.476 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.477 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.477 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18680 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.477 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.215500+0000", 2024-09-17T21:20:27.477 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0033998380000000001, 2024-09-17T21:20:27.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055216370000000002, 2024-09-17T21:20:27.477 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.477 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.477 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.477 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.477 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.477 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.477 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18680 2024-09-17T21:20:27.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.478 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215500+0000", 2024-09-17T21:20:27.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215500+0000", 2024-09-17T21:20:27.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215501+0000", 2024-09-17T21:20:27.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8100000000000003e-07 2024-09-17T21:20:27.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215503+0000", 2024-09-17T21:20:27.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.52e-06 2024-09-17T21:20:27.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215510+0000", 2024-09-17T21:20:27.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8580000000000002e-06 2024-09-17T21:20:27.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.215516+0000", 2024-09-17T21:20:27.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8370000000000002e-06 2024-09-17T21:20:27.480 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.480 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.480 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.480 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:18713 3.16 3.9e675336 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:27.480 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:27.216309+0000", 2024-09-17T21:20:27.480 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0025906520000000001, 2024-09-17T21:20:27.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0047423889999999996, 2024-09-17T21:20:27.480 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:27.481 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:27.481 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:27.481 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:27.481 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:27.481 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:27.481 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18713 2024-09-17T21:20:27.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.481 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:27.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:27.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216309+0000", 2024-09-17T21:20:27.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:27.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216309+0000", 2024-09-17T21:20:27.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:27.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:27.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216310+0000", 2024-09-17T21:20:27.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9699999999999996e-07 2024-09-17T21:20:27.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:27.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216312+0000", 2024-09-17T21:20:27.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3010000000000001e-06 2024-09-17T21:20:27.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:27.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216319+0000", 2024-09-17T21:20:27.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2500000000000003e-06 2024-09-17T21:20:27.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:27.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:27.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:27.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:27.216324+0000", 2024-09-17T21:20:27.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1830000000000002e-06 2024-09-17T21:20:27.483 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.483 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:27.483 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.483 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:27.484 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-09-17T21:20:27.484 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 75 2024-09-17T21:20:27.484 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:27.484 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:20:27.486 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-17T21:20:27.487 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:20:27.487 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:27.482+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:27.586 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:27.578+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:27.586 INFO:teuthology.orchestra.run.smithi184.stdout:4948 write 0xe73fe00 thru 0xe7460da (0x62db bytes) 2024-09-17T21:20:27.588 INFO:teuthology.orchestra.run.smithi184.stdout:4949 read 0xd8e8c00 thru 0xd8eae56 (0x2257 bytes) 2024-09-17T21:20:27.589 INFO:teuthology.orchestra.run.smithi184.stdout:4950 read 0x84e3400 thru 0x84f1463 (0xe064 bytes) 2024-09-17T21:20:27.590 INFO:teuthology.orchestra.run.smithi184.stdout:4953 read 0x5789000 thru 0x5793f9c (0xaf9d bytes) 2024-09-17T21:20:27.590 INFO:teuthology.orchestra.run.smithi184.stdout:4954 write 0xc9bb000 thru 0xc9c2858 (0x7859 bytes) 2024-09-17T21:20:27.591 INFO:teuthology.orchestra.run.smithi184.stdout:4956 punch from 0xcca400 to 0xccd000, (0x2c00 bytes) 2024-09-17T21:20:27.591 INFO:teuthology.orchestra.run.smithi184.stdout:4958 punch from 0xdb4e200 to 0xdb54400, (0x6200 bytes) 2024-09-17T21:20:27.592 INFO:teuthology.orchestra.run.smithi184.stdout:4959 read 0xefa600 thru 0xf06aab (0xc4ac bytes) 2024-09-17T21:20:27.594 INFO:teuthology.orchestra.run.smithi184.stdout:4960 write 0x51ca00 thru 0x524c47 (0x8248 bytes) 2024-09-17T21:20:27.599 INFO:teuthology.orchestra.run.smithi184.stdout:4962 write 0x539d000 thru 0x53a50d9 (0x80da bytes) 2024-09-17T21:20:27.599 INFO:teuthology.orchestra.run.smithi184.stdout:4963 read 0x58c1000 thru 0x58cc21d (0xb21e bytes) 2024-09-17T21:20:27.599 INFO:teuthology.orchestra.run.smithi184.stdout:4964 write 0x48ce800 thru 0x48d1ac4 (0x32c5 bytes) 2024-09-17T21:20:27.599 INFO:teuthology.orchestra.run.smithi184.stdout:4966 write 0xe5bf200 thru 0xe5c2988 (0x3789 bytes) 2024-09-17T21:20:27.601 INFO:teuthology.orchestra.run.smithi184.stdout:4968 read 0x657aa00 thru 0x6589432 (0xea33 bytes) 2024-09-17T21:20:27.601 INFO:teuthology.orchestra.run.smithi184.stdout:4969 write 0xcbcce00 thru 0xcbda69b (0xd89c bytes) 2024-09-17T21:20:27.601 INFO:teuthology.orchestra.run.smithi184.stdout:4973 write 0x8bc1c00 thru 0x8bd10c9 (0xf4ca bytes) 2024-09-17T21:20:27.601 INFO:teuthology.orchestra.run.smithi184.stdout:4974 read 0xe2e7a00 thru 0xe2ec96c (0x4f6d bytes) 2024-09-17T21:20:27.633 INFO:teuthology.orchestra.run.smithi184.stdout:4975 trunc from 0xe746200 to 0x1404a00 2024-09-17T21:20:27.664 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:27.664 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:27.664 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:27.664 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:27.664 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:27.683 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:27.684 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:20:27.686 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:27.681+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:27.705 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:27.706 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:27.727 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:27.727 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:27.757 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:27.757 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:27.775 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:27.775 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:27.787 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:27.782+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:27.793 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:27.793 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:27.859 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:27.859 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:27.873 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:20:27.907 INFO:teuthology.orchestra.run.smithi184.stdout:4976 punch from 0xd09c00 to 0xd14400, (0xa800 bytes) 2024-09-17T21:20:27.909 INFO:teuthology.orchestra.run.smithi184.stdout:4978 clone 34 order 24 su 131072 sc 12 2024-09-17T21:20:27.919 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:27.914+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:27.989 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:27.985+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:28.036 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:28.036 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:28.051 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:20:28.090 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:28.085+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:28.175 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:28.175 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:28.189 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:20:28.190 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:28.182+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:28.291 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:28.285+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:28.307 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:28.307 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:28.321 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:20:28.392 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:28.386+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:28.443 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:28.443 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:28.458 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:20:28.492 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:28.485+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:28.583 INFO:teuthology.orchestra.run.smithi184.stdout:flattening image image_client.0-clone33 2024-09-17T21:20:28.593 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:28.589+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:28.630 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:28.630 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:28.644 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:20:28.693 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:28.689+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:28.769 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:28.769 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:28.783 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:20:28.833 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:28.829+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:28.895 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:28.889+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:28.919 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:28.919 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:28.933 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:20:28.996 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:28.990+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:29.097 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:29.090+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:29.110 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:29.110 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:29.125 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:20:29.198 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:29.193+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:29.243 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:29.243 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:29.257 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:20:29.300 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:29.294+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:29.398 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-17T21:20:29.401 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:29.394+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:29.425 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:29.425 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:29.439 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:20:29.488 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:20:29.501 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:29.494+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:29.558 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:29.559 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:29.576 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:20:29.602 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:29.594+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:29.694 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:29.694 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:29.703 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:29.697+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:29.710 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:20:29.803 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:29.797+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:29.809 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:20:29.876 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:29.876 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:29.893 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:20:29.904 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:29.898+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:29.965 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:29.965 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:29.997 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:29.997 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:30.004 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:30.001+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:30.014 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:30.014 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:30.032 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:30.032 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:30.032 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:30.032 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:30.032 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:30.041 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:30.042 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:30.046 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:20:30.063 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:30.063 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:30.085 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:30.086 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:30.106 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:30.098+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:30.185 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:30.185 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:30.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:30.185 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:20:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 131.96568637799999, 2024-09-17T21:20:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:20:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:20:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:20:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:20:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:20:30.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:20:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:20:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:20:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:20:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:30.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:20:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:20:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:20:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:20:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:20:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:20:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:20:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:20:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:20:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 127.315385714, 2024-09-17T21:20:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:20:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:30.190 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:20:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:20:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:20:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:20:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:20:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:20:30.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:20:30.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:20:30.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:30.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:20:30.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:20:30.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:30.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:20:30.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:20:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:20:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:20:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:20:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:20:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:30.194 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:20:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 127.31412063099999, 2024-09-17T21:20:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:20:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:30.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:20:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:20:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:20:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:20:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:20:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:20:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:20:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:20:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:20:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:20:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:20:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:20:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:20:30.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:20:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:20:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:20:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:20:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 127.304341221, 2024-09-17T21:20:30.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:20:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:20:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:20:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:20:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:20:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:20:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:20:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:20:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:20:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:20:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:20:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:20:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:20:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:20:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:20:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:20:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:20:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.203 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:20:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 127.29561881799999, 2024-09-17T21:20:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:20:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:30.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:20:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:20:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:30.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:20:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:20:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:20:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:20:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:20:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:20:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:20:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:20:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:20:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:20:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:20:30.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:20:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:20:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:20:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:20:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:30.208 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 113.910528254, 2024-09-17T21:20:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:20:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:30.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:20:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:20:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:20:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:20:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:30.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:20:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:20:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:20:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:20:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:20:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:20:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:20:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:20:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:20:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:20:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:20:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:20:30.212 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1780 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'195, mlcod=69'190)", 2024-09-17T21:20:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 76.932672650000001, 2024-09-17T21:20:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075761769999999999, 2024-09-17T21:20:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:30.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:30.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:30.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:30.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:30.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250375+0000", 2024-09-17T21:20:30.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:20:30.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:30.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250383+0000", 2024-09-17T21:20:30.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7870000000000002e-06 2024-09-17T21:20:30.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:30.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250402+0000", 2024-09-17T21:20:30.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9346000000000001e-05 2024-09-17T21:20:30.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250413+0000", 2024-09-17T21:20:30.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.039e-05 2024-09-17T21:20:30.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:30.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251353+0000", 2024-09-17T21:20:30.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000940091 2024-09-17T21:20:30.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:30.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251407+0000", 2024-09-17T21:20:30.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4701999999999999e-05 2024-09-17T21:20:30.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:30.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257829+0000", 2024-09-17T21:20:30.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064212779999999999 2024-09-17T21:20:30.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:30.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257950+0000", 2024-09-17T21:20:30.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121623 2024-09-17T21:20:30.217 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.217 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:30.217 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.217 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1781 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'196, mlcod=69'190)", 2024-09-17T21:20:30.217 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:30.217 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 76.931589699, 2024-09-17T21:20:30.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065632939999999999, 2024-09-17T21:20:30.218 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:30.218 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:30.218 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:30.218 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:30.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:30.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:30.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:30.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:30.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:30.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251459+0000", 2024-09-17T21:20:30.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.624e-06 2024-09-17T21:20:30.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:30.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251471+0000", 2024-09-17T21:20:30.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2636000000000001e-05 2024-09-17T21:20:30.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:30.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251498+0000", 2024-09-17T21:20:30.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:20:30.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251515+0000", 2024-09-17T21:20:30.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6939000000000002e-05 2024-09-17T21:20:30.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:30.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252938+0000", 2024-09-17T21:20:30.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001423082 2024-09-17T21:20:30.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:30.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252999+0000", 2024-09-17T21:20:30.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1138999999999999e-05 2024-09-17T21:20:30.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:30.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257955+0000", 2024-09-17T21:20:30.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049559640000000002 2024-09-17T21:20:30.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:30.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.258020+0000", 2024-09-17T21:20:30.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5504000000000002e-05 2024-09-17T21:20:30.221 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.221 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:30.221 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.222 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1788 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'198, mlcod=69'191)", 2024-09-17T21:20:30.222 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:30.222 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 76.929564209999995, 2024-09-17T21:20:30.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073690370000000002, 2024-09-17T21:20:30.222 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:30.222 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:30.222 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:30.222 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:30.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:30.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:30.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:30.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:30.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:30.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253484+0000", 2024-09-17T21:20:30.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2470000000000001e-06 2024-09-17T21:20:30.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:30.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253492+0000", 2024-09-17T21:20:30.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7159999999999992e-06 2024-09-17T21:20:30.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:30.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253509+0000", 2024-09-17T21:20:30.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.666e-05 2024-09-17T21:20:30.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253522+0000", 2024-09-17T21:20:30.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2643000000000001e-05 2024-09-17T21:20:30.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:30.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254853+0000", 2024-09-17T21:20:30.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331238 2024-09-17T21:20:30.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:30.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254885+0000", 2024-09-17T21:20:30.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1562e-05 2024-09-17T21:20:30.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:30.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260788+0000", 2024-09-17T21:20:30.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059030749999999998 2024-09-17T21:20:30.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:30.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260852+0000", 2024-09-17T21:20:30.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3895999999999999e-05 2024-09-17T21:20:30.226 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.226 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:30.226 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.226 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5746 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'477, mlcod=71'473)", 2024-09-17T21:20:30.226 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:30.226 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.924136652000001, 2024-09-17T21:20:30.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074182709999999997, 2024-09-17T21:20:30.226 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:30.227 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:30.227 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:30.227 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:30.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:30.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:30.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:30.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:30.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:30.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258912+0000", 2024-09-17T21:20:30.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:20:30.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:30.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258926+0000", 2024-09-17T21:20:30.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4249e-05 2024-09-17T21:20:30.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:30.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258946+0000", 2024-09-17T21:20:30.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9879999999999999e-05 2024-09-17T21:20:30.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258956+0000", 2024-09-17T21:20:30.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:20:30.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:30.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259380+0000", 2024-09-17T21:20:30.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042337599999999999 2024-09-17T21:20:30.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:30.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259412+0000", 2024-09-17T21:20:30.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:20:30.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:30.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266191+0000", 2024-09-17T21:20:30.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067793269999999999 2024-09-17T21:20:30.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:30.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266328+0000", 2024-09-17T21:20:30.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013690999999999999 2024-09-17T21:20:30.230 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.230 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:30.231 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.231 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5747 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'478, mlcod=71'473)", 2024-09-17T21:20:30.231 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:30.231 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.923430303000004, 2024-09-17T21:20:30.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068779440000000004, 2024-09-17T21:20:30.231 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:30.231 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:30.231 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:30.231 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:30.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:30.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:30.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:30.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:30.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:30.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259618+0000", 2024-09-17T21:20:30.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6199999999999999e-06 2024-09-17T21:20:30.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:30.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259630+0000", 2024-09-17T21:20:30.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1884e-05 2024-09-17T21:20:30.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:30.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259657+0000", 2024-09-17T21:20:30.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7339e-05 2024-09-17T21:20:30.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259677+0000", 2024-09-17T21:20:30.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0156000000000001e-05 2024-09-17T21:20:30.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:30.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260521+0000", 2024-09-17T21:20:30.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084312899999999997 2024-09-17T21:20:30.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:30.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260585+0000", 2024-09-17T21:20:30.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4558000000000001e-05 2024-09-17T21:20:30.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:30.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266336+0000", 2024-09-17T21:20:30.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057505020000000002 2024-09-17T21:20:30.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:30.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266494+0000", 2024-09-17T21:20:30.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158756 2024-09-17T21:20:30.235 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.235 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:30.235 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.235 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5750 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'479, mlcod=71'473)", 2024-09-17T21:20:30.235 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:30.236 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.922718110000005, 2024-09-17T21:20:30.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006302874, 2024-09-17T21:20:30.236 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:30.236 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:30.236 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:30.236 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:30.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:30.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:30.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:30.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:30.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:30.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260330+0000", 2024-09-17T21:20:30.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:20:30.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:30.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260343+0000", 2024-09-17T21:20:30.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2862e-05 2024-09-17T21:20:30.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:30.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260367+0000", 2024-09-17T21:20:30.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3723e-05 2024-09-17T21:20:30.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260386+0000", 2024-09-17T21:20:30.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8879999999999999e-05 2024-09-17T21:20:30.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:30.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261579+0000", 2024-09-17T21:20:30.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001193431 2024-09-17T21:20:30.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:30.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261608+0000", 2024-09-17T21:20:30.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8858999999999999e-05 2024-09-17T21:20:30.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:30.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266505+0000", 2024-09-17T21:20:30.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004896999 2024-09-17T21:20:30.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:30.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266631+0000", 2024-09-17T21:20:30.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012655300000000001 2024-09-17T21:20:30.239 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.240 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:30.240 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.240 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5751 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'480, mlcod=71'473)", 2024-09-17T21:20:30.240 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:30.240 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.922010897999996, 2024-09-17T21:20:30.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075756299999999999, 2024-09-17T21:20:30.240 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:30.240 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:30.240 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:30.240 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:30.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:30.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:30.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:30.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:30.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:30.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261037+0000", 2024-09-17T21:20:30.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:20:30.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:30.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261049+0000", 2024-09-17T21:20:30.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1063999999999999e-05 2024-09-17T21:20:30.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:30.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261072+0000", 2024-09-17T21:20:30.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3102e-05 2024-09-17T21:20:30.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261133+0000", 2024-09-17T21:20:30.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1276000000000001e-05 2024-09-17T21:20:30.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:30.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:20:30.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015334319999999999 2024-09-17T21:20:30.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:30.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262702+0000", 2024-09-17T21:20:30.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5281999999999999e-05 2024-09-17T21:20:30.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:30.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268527+0000", 2024-09-17T21:20:30.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058250389999999997 2024-09-17T21:20:30.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.244 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:30.244 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268611+0000", 2024-09-17T21:20:30.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4724999999999999e-05 2024-09-17T21:20:30.244 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.244 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:30.244 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.244 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5756 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'481, mlcod=71'473)", 2024-09-17T21:20:30.244 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:30.244 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.921150041999994, 2024-09-17T21:20:30.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006772386, 2024-09-17T21:20:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261899+0000", 2024-09-17T21:20:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.903e-06 2024-09-17T21:20:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261912+0000", 2024-09-17T21:20:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-05 2024-09-17T21:20:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261938+0000", 2024-09-17T21:20:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6098999999999999e-05 2024-09-17T21:20:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261955+0000", 2024-09-17T21:20:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.696e-05 2024-09-17T21:20:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263845+0000", 2024-09-17T21:20:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001890187 2024-09-17T21:20:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263907+0000", 2024-09-17T21:20:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1600999999999995e-05 2024-09-17T21:20:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268615+0000", 2024-09-17T21:20:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047086070000000001 2024-09-17T21:20:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268669+0000", 2024-09-17T21:20:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3588999999999998e-05 2024-09-17T21:20:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4978.0:598 3.9 e80/23 3:9701cc17:::rbd_data.1152724ff2a3.0000000000000019:head v 80'202, mlcod=80'198)", 2024-09-17T21:20:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.419575659000003, 2024-09-17T21:20:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066473340000000004, 2024-09-17T21:20:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763472+0000", 2024-09-17T21:20:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6899999999999997e-07 2024-09-17T21:20:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763478+0000", 2024-09-17T21:20:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:20:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763489+0000", 2024-09-17T21:20:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0628999999999999e-05 2024-09-17T21:20:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763495+0000", 2024-09-17T21:20:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6649999999999997e-06 2024-09-17T21:20:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763907+0000", 2024-09-17T21:20:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041163899999999999 2024-09-17T21:20:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763976+0000", 2024-09-17T21:20:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8534999999999993e-05 2024-09-17T21:20:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.769900+0000", 2024-09-17T21:20:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059243270000000001 2024-09-17T21:20:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.770118+0000", 2024-09-17T21:20:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021842500000000001 2024-09-17T21:20:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:468 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'94, mlcod=89'87)", 2024-09-17T21:20:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.496830359999997, 2024-09-17T21:20:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066528790000000004, 2024-09-17T21:20:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686218+0000", 2024-09-17T21:20:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3060000000000001e-06 2024-09-17T21:20:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686256+0000", 2024-09-17T21:20:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8333e-05 2024-09-17T21:20:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686272+0000", 2024-09-17T21:20:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:20:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686283+0000", 2024-09-17T21:20:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0903e-05 2024-09-17T21:20:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686682+0000", 2024-09-17T21:20:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039935099999999999 2024-09-17T21:20:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686785+0000", 2024-09-17T21:20:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010275199999999999 2024-09-17T21:20:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692816+0000", 2024-09-17T21:20:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060317579999999999 2024-09-17T21:20:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692869+0000", 2024-09-17T21:20:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2775999999999998e-05 2024-09-17T21:20:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:755 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'127, mlcod=89'122)", 2024-09-17T21:20:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.360375146000003, 2024-09-17T21:20:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084014080000000008, 2024-09-17T21:20:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:20:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822683+0000", 2024-09-17T21:20:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0674e-05 2024-09-17T21:20:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822698+0000", 2024-09-17T21:20:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5189e-05 2024-09-17T21:20:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822708+0000", 2024-09-17T21:20:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9119999999999993e-06 2024-09-17T21:20:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:30.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822748+0000", 2024-09-17T21:20:30.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9727000000000001e-05 2024-09-17T21:20:30.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:30.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822759+0000", 2024-09-17T21:20:30.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0865e-05 2024-09-17T21:20:30.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:30.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831034+0000", 2024-09-17T21:20:30.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082750710000000002 2024-09-17T21:20:30.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:30.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831073+0000", 2024-09-17T21:20:30.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9063999999999997e-05 2024-09-17T21:20:30.262 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.262 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:30.262 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.262 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:756 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'128, mlcod=89'124)", 2024-09-17T21:20:30.262 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:30.262 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.358452514, 2024-09-17T21:20:30.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0076300270000000002, 2024-09-17T21:20:30.262 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:30.263 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:30.263 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:30.263 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:30.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:30.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:30.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:30.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:30.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:30.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824595+0000", 2024-09-17T21:20:30.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-17T21:20:30.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:30.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824600+0000", 2024-09-17T21:20:30.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:20:30.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:30.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824612+0000", 2024-09-17T21:20:30.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.181e-05 2024-09-17T21:20:30.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824619+0000", 2024-09-17T21:20:30.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:20:30.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:30.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824671+0000", 2024-09-17T21:20:30.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2262999999999998e-05 2024-09-17T21:20:30.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:30.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824695+0000", 2024-09-17T21:20:30.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3883000000000002e-05 2024-09-17T21:20:30.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:30.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832185+0000", 2024-09-17T21:20:30.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007490048 2024-09-17T21:20:30.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:30.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832224+0000", 2024-09-17T21:20:30.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8957e-05 2024-09-17T21:20:30.266 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.266 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:30.266 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.267 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2617 3.d e90/23 3:b0143718:::rbd_data.1152a8ae2019.0000000000000016:head v 90'249, mlcod=90'246)", 2024-09-17T21:20:30.267 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:30.267 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 49.338601513999997, 2024-09-17T21:20:30.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063715559999999996, 2024-09-17T21:20:30.267 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:30.267 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:30.267 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:30.267 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:30.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:30.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:30.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:30.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:30.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:30.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844455+0000", 2024-09-17T21:20:30.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4259999999999992e-06 2024-09-17T21:20:30.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:30.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844470+0000", 2024-09-17T21:20:30.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5386000000000001e-05 2024-09-17T21:20:30.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:30.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844499+0000", 2024-09-17T21:20:30.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.883e-05 2024-09-17T21:20:30.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844518+0000", 2024-09-17T21:20:30.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8830000000000001e-05 2024-09-17T21:20:30.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:30.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844649+0000", 2024-09-17T21:20:30.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013143399999999999 2024-09-17T21:20:30.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:30.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844702+0000", 2024-09-17T21:20:30.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2876e-05 2024-09-17T21:20:30.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:30.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850696+0000", 2024-09-17T21:20:30.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059943729999999999 2024-09-17T21:20:30.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:30.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850817+0000", 2024-09-17T21:20:30.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120401 2024-09-17T21:20:30.271 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.271 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:30.271 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.271 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2635 3.d e90/23 3:b3d1e920:::rbd_data.1152a8ae2019.000000000000001f:head v 90'252, mlcod=90'246)", 2024-09-17T21:20:30.271 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:30.271 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 49.335893444, 2024-09-17T21:20:30.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0077265190000000003, 2024-09-17T21:20:30.271 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:30.272 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:30.272 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:30.272 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:30.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:30.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:30.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:30.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:30.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:30.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:30.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847155+0000", 2024-09-17T21:20:30.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.544e-06 2024-09-17T21:20:30.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:30.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847165+0000", 2024-09-17T21:20:30.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.043e-05 2024-09-17T21:20:30.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:30.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847189+0000", 2024-09-17T21:20:30.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3659000000000002e-05 2024-09-17T21:20:30.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847210+0000", 2024-09-17T21:20:30.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1163e-05 2024-09-17T21:20:30.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:30.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848856+0000", 2024-09-17T21:20:30.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001645521 2024-09-17T21:20:30.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:30.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848905+0000", 2024-09-17T21:20:30.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9048999999999997e-05 2024-09-17T21:20:30.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:30.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854770+0000", 2024-09-17T21:20:30.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058652879999999998 2024-09-17T21:20:30.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:30.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:30.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:30.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854880+0000", 2024-09-17T21:20:30.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010986500000000001 2024-09-17T21:20:30.275 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.275 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:30.275 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.275 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:30.275 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:30.275 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:30.277 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:20:30.278 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:30.270+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:30.378 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:30.373+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:30.384 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-17T21:20:30.422 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:30.422 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:30.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.422 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:4851 3.5 3.d300445 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:30.422 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:30.412768+0000", 2024-09-17T21:20:30.423 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0083363029999999998, 2024-09-17T21:20:30.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083543980000000007, 2024-09-17T21:20:30.423 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.423 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.423 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "reached pg", 2024-09-17T21:20:30.423 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.423 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:30.423 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:30.423 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4851 2024-09-17T21:20:30.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.423 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412768+0000", 2024-09-17T21:20:30.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412768+0000", 2024-09-17T21:20:30.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412768+0000", 2024-09-17T21:20:30.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6800000000000001e-07 2024-09-17T21:20:30.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412770+0000", 2024-09-17T21:20:30.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.142e-06 2024-09-17T21:20:30.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412778+0000", 2024-09-17T21:20:30.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6699999999999994e-06 2024-09-17T21:20:30.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412783+0000", 2024-09-17T21:20:30.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1919999999999996e-06 2024-09-17T21:20:30.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.421111+0000", 2024-09-17T21:20:30.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083284569999999992 2024-09-17T21:20:30.426 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.426 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.426 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.426 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:4852 3.5 3.d300445 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:30.426 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:30.412794+0000", 2024-09-17T21:20:30.426 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0083102520000000006, 2024-09-17T21:20:30.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083674909999999995, 2024-09-17T21:20:30.426 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.426 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.426 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:30.426 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.426 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:30.426 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:30.427 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4852 2024-09-17T21:20:30.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.427 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412794+0000", 2024-09-17T21:20:30.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412794+0000", 2024-09-17T21:20:30.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412794+0000", 2024-09-17T21:20:30.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6000000000000003e-07 2024-09-17T21:20:30.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412796+0000", 2024-09-17T21:20:30.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:20:30.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412803+0000", 2024-09-17T21:20:30.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5400000000000001e-06 2024-09-17T21:20:30.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412808+0000", 2024-09-17T21:20:30.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9339999999999997e-06 2024-09-17T21:20:30.429 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.429 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.429 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.429 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:4853 3.5 3.d300445 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:30.429 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:30.412819+0000", 2024-09-17T21:20:30.429 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0082851699999999997, 2024-09-17T21:20:30.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083706189999999993, 2024-09-17T21:20:30.429 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.430 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.430 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:30.430 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.430 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:30.430 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:30.430 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4853 2024-09-17T21:20:30.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.430 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412819+0000", 2024-09-17T21:20:30.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412819+0000", 2024-09-17T21:20:30.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412819+0000", 2024-09-17T21:20:30.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:20:30.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412821+0000", 2024-09-17T21:20:30.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.373e-06 2024-09-17T21:20:30.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412828+0000", 2024-09-17T21:20:30.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4930000000000003e-06 2024-09-17T21:20:30.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412835+0000", 2024-09-17T21:20:30.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4220000000000003e-06 2024-09-17T21:20:30.432 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.432 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.432 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.432 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:4854 3.5 3.d300445 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:30.432 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:30.412843+0000", 2024-09-17T21:20:30.432 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0082607329999999993, 2024-09-17T21:20:30.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083731389999999999, 2024-09-17T21:20:30.433 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.433 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.433 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:30.433 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.433 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:30.433 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:30.433 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4854 2024-09-17T21:20:30.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.433 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412843+0000", 2024-09-17T21:20:30.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412843+0000", 2024-09-17T21:20:30.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412844+0000", 2024-09-17T21:20:30.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.46e-07 2024-09-17T21:20:30.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412846+0000", 2024-09-17T21:20:30.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1900000000000002e-06 2024-09-17T21:20:30.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412853+0000", 2024-09-17T21:20:30.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2400000000000001e-06 2024-09-17T21:20:30.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:30.412858+0000", 2024-09-17T21:20:30.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9830000000000004e-06 2024-09-17T21:20:30.435 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.436 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.436 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.436 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.436 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-09-17T21:20:30.436 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 4 2024-09-17T21:20:30.436 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:30.443 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:20:30.491 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:30.486+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:30.580 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:30.573+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:30.588 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:30.588 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:30.588 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:30.588 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:30.588 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:30.606 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:20:30.681 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:30.677+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:30.782 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:30.777+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:30.798 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:30.798 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:30.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:30.798 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:20:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:20:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 153.781624658, 2024-09-17T21:20:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:20:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:20:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:20:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:20:30.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:20:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:20:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:20:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:20:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:20:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:20:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:20:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:20:30.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:20:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:20:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:20:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:20:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:20:30.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:20:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:20:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:20:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:20:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:20:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:20:30.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:20:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:20:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:20:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:20:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:20:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:20:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:20:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:20:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:20:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:20:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:20:30.805 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:20:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 134.20146037800001, 2024-09-17T21:20:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:20:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:20:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:20:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:20:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:20:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:20:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:20:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:20:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:20:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:20:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:20:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:20:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:20:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:20:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:20:30.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:20:30.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:20:30.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:20:30.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:20:30.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:20:30.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:20:30.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:30.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:20:30.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:20:30.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:20:30.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:20:30.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:30.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:20:30.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:20:30.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.811 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:30.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:20:30.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:20:30.811 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.811 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.811 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.811 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:20:30.812 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:20:30.812 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 134.201363066, 2024-09-17T21:20:30.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:20:30.812 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.812 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.812 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:30.812 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.812 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:30.812 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:30.812 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:20:30.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.812 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:20:30.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:20:30.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:20:30.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:20:30.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:20:30.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:20:30.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:20:30.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:20:30.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:20:30.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:20:30.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:20:30.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:20:30.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:20:30.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:20:30.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:20:30.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:20:30.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:20:30.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:30.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:20:30.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:20:30.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:20:30.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:20:30.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:20:30.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:20:30.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:20:30.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:20:30.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:20:30.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:30.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:20:30.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:20:30.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:30.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:20:30.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:20:30.818 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.818 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.818 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.818 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:20:30.819 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:20:30.819 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 134.19927626500001, 2024-09-17T21:20:30.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:20:30.819 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.819 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.819 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:30.819 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.819 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:20:30.819 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:20:30.819 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:20:30.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.819 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:20:30.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:20:30.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:20:30.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:20:30.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:20:30.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:20:30.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:20:30.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:20:30.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:20:30.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:20:30.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:20:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:20:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:20:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:20:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:20:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:20:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:20:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:20:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:20:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:20:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:20:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:20:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:20:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:20:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:20:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:20:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:20:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:20:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:20:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:20:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:20:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:20:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:20:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:20:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.515377484, 2024-09-17T21:20:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:20:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:20:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:20:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:20:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:20:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:20:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:20:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:20:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:20:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:20:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:20:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:20:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:20:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:20:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:20:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:20:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:20:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:20:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:20:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:20:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:20:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:20:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:20:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:20:30.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:20:30.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:30.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:20:30.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:20:30.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:20:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:20:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:20:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.506285773, 2024-09-17T21:20:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:20:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:20:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:20:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:20:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:20:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:20:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:20:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:20:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:20:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:20:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:20:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:20:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:20:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:20:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:20:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:20:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:20:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:20:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:20:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:20:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:20:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:20:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:20:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:20:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:20:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.50611372199999, 2024-09-17T21:20:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:20:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:20:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:20:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:20:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:20:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:20:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:20:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:20:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:20:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:20:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:20:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:20:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:20:30.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:20:30.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:20:30.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:20:30.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:20:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:20:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:20:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:20:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:20:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:20:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:20:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:20:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:20:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.331180975, 2024-09-17T21:20:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:20:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:20:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:20:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:20:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:20:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:20:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:20:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:20:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:20:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:20:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:20:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:20:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:20:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:20:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:20:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:20:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:20:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:20:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:20:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:20:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:20:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:20:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.331096418, 2024-09-17T21:20:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:20:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:20:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:20:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:20:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:20:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:20:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:20:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:20:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:20:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:20:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:20:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:20:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:20:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:20:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:20:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:20:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:20:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:20:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:20:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:20:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.331030781, 2024-09-17T21:20:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:20:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:20:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:20:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:20:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:20:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:20:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:20:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:20:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:20:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:20:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:20:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:20:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:20:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:20:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:20:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:20:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:20:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:20:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:20:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:20:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:20:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:20:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:20:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.957455684000003, 2024-09-17T21:20:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:20:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:20:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:20:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:20:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:20:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:20:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:20:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:20:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:20:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:20:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:20:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:20:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:20:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:20:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:20:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:20:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:20:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:20:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:20:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.957336171999998, 2024-09-17T21:20:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:20:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:20:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:20:30.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:20:30.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:20:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:20:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:20:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:20:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:20:30.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:20:30.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:20:30.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:20:30.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:20:30.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:20:30.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:20:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:20:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:20:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:20:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:992 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 567296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431004+0000", 2024-09-17T21:20:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.3643858680000003, 2024-09-17T21:20:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083642674, 2024-09-17T21:20:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 992 2024-09-17T21:20:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431004+0000", 2024-09-17T21:20:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431004+0000", 2024-09-17T21:20:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431005+0000", 2024-09-17T21:20:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:20:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431007+0000", 2024-09-17T21:20:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.283e-06 2024-09-17T21:20:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431013+0000", 2024-09-17T21:20:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.387e-06 2024-09-17T21:20:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431019+0000", 2024-09-17T21:20:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1780000000000002e-06 2024-09-17T21:20:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514341+0000", 2024-09-17T21:20:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083321939999999997 2024-09-17T21:20:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514439+0000", 2024-09-17T21:20:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8267e-05 2024-09-17T21:20:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514647+0000", 2024-09-17T21:20:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000208109 2024-09-17T21:20:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:994 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 571392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:20:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.3643641979999996, 2024-09-17T21:20:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083898465000000005, 2024-09-17T21:20:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 994 2024-09-17T21:20:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:20:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:20:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:20:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:20:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431058+0000", 2024-09-17T21:20:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1667e-05 2024-09-17T21:20:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431067+0000", 2024-09-17T21:20:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0720000000000006e-06 2024-09-17T21:20:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431075+0000", 2024-09-17T21:20:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9200000000000004e-06 2024-09-17T21:20:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514671+0000", 2024-09-17T21:20:30.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083596229999999994 2024-09-17T21:20:30.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:30.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514741+0000", 2024-09-17T21:20:30.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9929000000000001e-05 2024-09-17T21:20:30.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:30.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514924+0000", 2024-09-17T21:20:30.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000183122 2024-09-17T21:20:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:995 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 575488~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:20:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.3643051819999998, 2024-09-17T21:20:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084214699000000004, 2024-09-17T21:20:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 995 2024-09-17T21:20:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:20:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:20:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431086+0000", 2024-09-17T21:20:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7999999999999995e-07 2024-09-17T21:20:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431088+0000", 2024-09-17T21:20:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4729999999999999e-06 2024-09-17T21:20:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431126+0000", 2024-09-17T21:20:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7583999999999998e-05 2024-09-17T21:20:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431137+0000", 2024-09-17T21:20:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1768e-05 2024-09-17T21:20:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515003+0000", 2024-09-17T21:20:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083865097 2024-09-17T21:20:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:30.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515115+0000", 2024-09-17T21:20:30.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112561 2024-09-17T21:20:30.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:30.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515300+0000", 2024-09-17T21:20:30.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018453600000000001 2024-09-17T21:20:30.882 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.882 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.882 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.882 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:996 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:20:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.3642228010000004, 2024-09-17T21:20:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084493736999999999, 2024-09-17T21:20:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 996 2024-09-17T21:20:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:20:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:20:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431168+0000", 2024-09-17T21:20:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4e-07 2024-09-17T21:20:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431172+0000", 2024-09-17T21:20:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.608e-06 2024-09-17T21:20:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431180+0000", 2024-09-17T21:20:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5650000000000002e-06 2024-09-17T21:20:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431188+0000", 2024-09-17T21:20:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:20:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515354+0000", 2024-09-17T21:20:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084165977000000003 2024-09-17T21:20:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515451+0000", 2024-09-17T21:20:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6669000000000006e-05 2024-09-17T21:20:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515661+0000", 2024-09-17T21:20:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021021599999999999 2024-09-17T21:20:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:997 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 583680~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:20:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.3641914059999998, 2024-09-17T21:20:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084772419000000002, 2024-09-17T21:20:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 997 2024-09-17T21:20:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:20:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:20:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:20:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0400000000000005e-07 2024-09-17T21:20:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431202+0000", 2024-09-17T21:20:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:20:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431208+0000", 2024-09-17T21:20:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4740000000000002e-06 2024-09-17T21:20:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431214+0000", 2024-09-17T21:20:30.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6550000000000001e-06 2024-09-17T21:20:30.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515741+0000", 2024-09-17T21:20:30.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084526904 2024-09-17T21:20:30.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515826+0000", 2024-09-17T21:20:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4643999999999998e-05 2024-09-17T21:20:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515971+0000", 2024-09-17T21:20:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145444 2024-09-17T21:20:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:998 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 587776~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:20:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.3641684339999998, 2024-09-17T21:20:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085108565999999997, 2024-09-17T21:20:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 998 2024-09-17T21:20:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:20:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:20:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:20:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:20:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431224+0000", 2024-09-17T21:20:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:20:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431231+0000", 2024-09-17T21:20:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000002e-06 2024-09-17T21:20:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431236+0000", 2024-09-17T21:20:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.648e-06 2024-09-17T21:20:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516024+0000", 2024-09-17T21:20:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084787584999999999 2024-09-17T21:20:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516122+0000", 2024-09-17T21:20:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8076000000000001e-05 2024-09-17T21:20:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516330+0000", 2024-09-17T21:20:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020808 2024-09-17T21:20:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1000 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 591872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:20:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.3641463189999996, 2024-09-17T21:20:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085389065, 2024-09-17T21:20:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1000 2024-09-17T21:20:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:20:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:20:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:20:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:20:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431246+0000", 2024-09-17T21:20:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1749999999999999e-06 2024-09-17T21:20:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431253+0000", 2024-09-17T21:20:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2790000000000004e-06 2024-09-17T21:20:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431258+0000", 2024-09-17T21:20:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3299999999999998e-06 2024-09-17T21:20:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516384+0000", 2024-09-17T21:20:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085125549999999994 2024-09-17T21:20:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516460+0000", 2024-09-17T21:20:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6359999999999994e-05 2024-09-17T21:20:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516633+0000", 2024-09-17T21:20:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017285900000000001 2024-09-17T21:20:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1001 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 595968~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:20:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.3641229130000001, 2024-09-17T21:20:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085614715999999993, 2024-09-17T21:20:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1001 2024-09-17T21:20:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:20:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:20:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438066+0000", 2024-09-17T21:20:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067989490000000003 2024-09-17T21:20:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438073+0000", 2024-09-17T21:20:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8419999999999999e-06 2024-09-17T21:20:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438090+0000", 2024-09-17T21:20:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6657e-05 2024-09-17T21:20:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438099+0000", 2024-09-17T21:20:30.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0529999999999996e-06 2024-09-17T21:20:30.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:30.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516678+0000", 2024-09-17T21:20:30.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078579422999999995 2024-09-17T21:20:30.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:30.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516739+0000", 2024-09-17T21:20:30.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1315000000000005e-05 2024-09-17T21:20:30.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:30.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:30.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:30.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516882+0000", 2024-09-17T21:20:30.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014247700000000001 2024-09-17T21:20:30.906 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.906 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.906 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.906 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:30.906 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:30.906 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:30.908 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:20:30.908 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:30.901+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:31.008 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:31.001+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:31.060 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:31.060 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:31.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.061 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10357 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.061 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041284+0000", 2024-09-17T21:20:31.061 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012726929999999999, 2024-09-17T21:20:31.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012750707, 2024-09-17T21:20:31.061 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.061 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.061 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.061 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.061 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.061 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.061 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10357 2024-09-17T21:20:31.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.062 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041284+0000", 2024-09-17T21:20:31.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041284+0000", 2024-09-17T21:20:31.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041284+0000", 2024-09-17T21:20:31.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7700000000000001e-07 2024-09-17T21:20:31.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041285+0000", 2024-09-17T21:20:31.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.252e-06 2024-09-17T21:20:31.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041289+0000", 2024-09-17T21:20:31.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4419999999999998e-06 2024-09-17T21:20:31.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041292+0000", 2024-09-17T21:20:31.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8260000000000001e-06 2024-09-17T21:20:31.064 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.064 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.064 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.064 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10391 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.064 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041899+0000", 2024-09-17T21:20:31.064 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012111363999999999, 2024-09-17T21:20:31.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012194851, 2024-09-17T21:20:31.064 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.064 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.064 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.065 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.065 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.065 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.065 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10391 2024-09-17T21:20:31.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.065 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041899+0000", 2024-09-17T21:20:31.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041899+0000", 2024-09-17T21:20:31.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041900+0000", 2024-09-17T21:20:31.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9799999999999999e-07 2024-09-17T21:20:31.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041901+0000", 2024-09-17T21:20:31.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3179999999999999e-06 2024-09-17T21:20:31.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041905+0000", 2024-09-17T21:20:31.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.619e-06 2024-09-17T21:20:31.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041908+0000", 2024-09-17T21:20:31.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.824e-06 2024-09-17T21:20:31.067 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.067 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.067 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.067 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10358 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.067 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041295+0000", 2024-09-17T21:20:31.067 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012715376, 2024-09-17T21:20:31.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012849282, 2024-09-17T21:20:31.068 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.068 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.068 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.068 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.068 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.068 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.068 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10358 2024-09-17T21:20:31.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.068 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041295+0000", 2024-09-17T21:20:31.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041295+0000", 2024-09-17T21:20:31.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041296+0000", 2024-09-17T21:20:31.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.36e-07 2024-09-17T21:20:31.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041297+0000", 2024-09-17T21:20:31.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1680000000000001e-06 2024-09-17T21:20:31.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041300+0000", 2024-09-17T21:20:31.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0869999999999998e-06 2024-09-17T21:20:31.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041303+0000", 2024-09-17T21:20:31.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6989999999999999e-06 2024-09-17T21:20:31.070 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.070 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.070 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.071 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10392 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.071 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041922+0000", 2024-09-17T21:20:31.071 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012088999, 2024-09-17T21:20:31.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012261996000000001, 2024-09-17T21:20:31.071 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.071 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.071 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.071 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10392 2024-09-17T21:20:31.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.071 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041922+0000", 2024-09-17T21:20:31.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041922+0000", 2024-09-17T21:20:31.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041922+0000", 2024-09-17T21:20:31.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3300000000000001e-07 2024-09-17T21:20:31.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041923+0000", 2024-09-17T21:20:31.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2890000000000001e-06 2024-09-17T21:20:31.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041927+0000", 2024-09-17T21:20:31.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3289999999999998e-06 2024-09-17T21:20:31.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041929+0000", 2024-09-17T21:20:31.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7379999999999999e-06 2024-09-17T21:20:31.073 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.073 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.073 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.074 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10359 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.074 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041306+0000", 2024-09-17T21:20:31.074 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012704768999999999, 2024-09-17T21:20:31.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012902848, 2024-09-17T21:20:31.074 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.074 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.074 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.074 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.074 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.074 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.074 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10359 2024-09-17T21:20:31.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.074 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041306+0000", 2024-09-17T21:20:31.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041306+0000", 2024-09-17T21:20:31.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041306+0000", 2024-09-17T21:20:31.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5400000000000002e-07 2024-09-17T21:20:31.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041308+0000", 2024-09-17T21:20:31.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1620000000000001e-06 2024-09-17T21:20:31.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041311+0000", 2024-09-17T21:20:31.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9909999999999998e-06 2024-09-17T21:20:31.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041313+0000", 2024-09-17T21:20:31.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.796e-06 2024-09-17T21:20:31.077 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.077 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.077 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.077 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10393 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.077 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041933+0000", 2024-09-17T21:20:31.077 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012077674, 2024-09-17T21:20:31.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012319337, 2024-09-17T21:20:31.077 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.077 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.077 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.077 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.078 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.078 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.078 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10393 2024-09-17T21:20:31.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.078 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041933+0000", 2024-09-17T21:20:31.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041933+0000", 2024-09-17T21:20:31.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041933+0000", 2024-09-17T21:20:31.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4600000000000001e-07 2024-09-17T21:20:31.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041935+0000", 2024-09-17T21:20:31.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1820000000000001e-06 2024-09-17T21:20:31.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041938+0000", 2024-09-17T21:20:31.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0529999999999998e-06 2024-09-17T21:20:31.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041940+0000", 2024-09-17T21:20:31.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6740000000000001e-06 2024-09-17T21:20:31.080 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.080 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.080 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.080 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10360 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.080 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041338+0000", 2024-09-17T21:20:31.080 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012672449000000001, 2024-09-17T21:20:31.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012963633, 2024-09-17T21:20:31.081 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.081 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.081 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.081 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.081 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.081 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.081 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10360 2024-09-17T21:20:31.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.081 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041338+0000", 2024-09-17T21:20:31.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041338+0000", 2024-09-17T21:20:31.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041339+0000", 2024-09-17T21:20:31.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.46e-07 2024-09-17T21:20:31.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041340+0000", 2024-09-17T21:20:31.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2780000000000001e-06 2024-09-17T21:20:31.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041344+0000", 2024-09-17T21:20:31.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5839999999999999e-06 2024-09-17T21:20:31.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041346+0000", 2024-09-17T21:20:31.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.92e-06 2024-09-17T21:20:31.083 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.083 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.083 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.084 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10394 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.084 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041944+0000", 2024-09-17T21:20:31.084 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012067213, 2024-09-17T21:20:31.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012412029, 2024-09-17T21:20:31.084 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.084 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.084 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.084 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.084 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.084 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.085 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10394 2024-09-17T21:20:31.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.085 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041944+0000", 2024-09-17T21:20:31.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041944+0000", 2024-09-17T21:20:31.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041944+0000", 2024-09-17T21:20:31.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0900000000000001e-07 2024-09-17T21:20:31.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041945+0000", 2024-09-17T21:20:31.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1170000000000001e-06 2024-09-17T21:20:31.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041948+0000", 2024-09-17T21:20:31.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1669999999999999e-06 2024-09-17T21:20:31.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041951+0000", 2024-09-17T21:20:31.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6240000000000002e-06 2024-09-17T21:20:31.087 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.087 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.087 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.087 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10361 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.087 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041351+0000", 2024-09-17T21:20:31.087 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012659365000000001, 2024-09-17T21:20:31.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013053735, 2024-09-17T21:20:31.087 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.087 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.087 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.087 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.088 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.088 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.088 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10361 2024-09-17T21:20:31.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.088 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041351+0000", 2024-09-17T21:20:31.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041351+0000", 2024-09-17T21:20:31.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041352+0000", 2024-09-17T21:20:31.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3799999999999999e-07 2024-09-17T21:20:31.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041353+0000", 2024-09-17T21:20:31.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3179999999999999e-06 2024-09-17T21:20:31.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041356+0000", 2024-09-17T21:20:31.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1540000000000002e-06 2024-09-17T21:20:31.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041359+0000", 2024-09-17T21:20:31.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8729999999999999e-06 2024-09-17T21:20:31.090 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.090 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.090 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.090 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10395 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.090 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041977+0000", 2024-09-17T21:20:31.090 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012033749999999999, 2024-09-17T21:20:31.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012479264, 2024-09-17T21:20:31.090 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.091 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.091 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.091 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.091 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.091 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.091 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10395 2024-09-17T21:20:31.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.091 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041977+0000", 2024-09-17T21:20:31.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041977+0000", 2024-09-17T21:20:31.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041977+0000", 2024-09-17T21:20:31.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.53e-07 2024-09-17T21:20:31.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041979+0000", 2024-09-17T21:20:31.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.296e-06 2024-09-17T21:20:31.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041982+0000", 2024-09-17T21:20:31.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4460000000000001e-06 2024-09-17T21:20:31.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041985+0000", 2024-09-17T21:20:31.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9720000000000001e-06 2024-09-17T21:20:31.093 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.093 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.093 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.094 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10362 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.094 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041386+0000", 2024-09-17T21:20:31.094 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012624497, 2024-09-17T21:20:31.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013104545, 2024-09-17T21:20:31.094 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.094 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.094 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.094 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.094 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.094 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.094 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10362 2024-09-17T21:20:31.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.094 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041386+0000", 2024-09-17T21:20:31.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041386+0000", 2024-09-17T21:20:31.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041387+0000", 2024-09-17T21:20:31.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.84e-07 2024-09-17T21:20:31.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041388+0000", 2024-09-17T21:20:31.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2610000000000001e-06 2024-09-17T21:20:31.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041392+0000", 2024-09-17T21:20:31.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2030000000000002e-06 2024-09-17T21:20:31.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041395+0000", 2024-09-17T21:20:31.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8880000000000001e-06 2024-09-17T21:20:31.096 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.096 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.097 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.097 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10396 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.097 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041989+0000", 2024-09-17T21:20:31.097 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012021641, 2024-09-17T21:20:31.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012526041, 2024-09-17T21:20:31.097 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.097 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.097 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.097 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.097 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.097 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.097 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10396 2024-09-17T21:20:31.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.098 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041989+0000", 2024-09-17T21:20:31.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041989+0000", 2024-09-17T21:20:31.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041989+0000", 2024-09-17T21:20:31.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.58e-07 2024-09-17T21:20:31.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041991+0000", 2024-09-17T21:20:31.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.311e-06 2024-09-17T21:20:31.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041994+0000", 2024-09-17T21:20:31.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9749999999999999e-06 2024-09-17T21:20:31.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041996+0000", 2024-09-17T21:20:31.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7030000000000002e-06 2024-09-17T21:20:31.100 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.100 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.100 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.100 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10363 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.100 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041399+0000", 2024-09-17T21:20:31.100 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.01261206, 2024-09-17T21:20:31.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01314355, 2024-09-17T21:20:31.100 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.100 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.100 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.101 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.101 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.101 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.101 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10363 2024-09-17T21:20:31.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.101 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041399+0000", 2024-09-17T21:20:31.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041399+0000", 2024-09-17T21:20:31.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041399+0000", 2024-09-17T21:20:31.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.35e-07 2024-09-17T21:20:31.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041400+0000", 2024-09-17T21:20:31.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.203e-06 2024-09-17T21:20:31.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041403+0000", 2024-09-17T21:20:31.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.174e-06 2024-09-17T21:20:31.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041406+0000", 2024-09-17T21:20:31.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7269999999999999e-06 2024-09-17T21:20:31.103 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.103 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.103 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.104 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10397 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.104 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042010+0000", 2024-09-17T21:20:31.104 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012000693, 2024-09-17T21:20:31.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012556397, 2024-09-17T21:20:31.104 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.104 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.104 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.104 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.104 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.105 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.105 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10397 2024-09-17T21:20:31.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.105 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042010+0000", 2024-09-17T21:20:31.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042010+0000", 2024-09-17T21:20:31.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042010+0000", 2024-09-17T21:20:31.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4600000000000001e-07 2024-09-17T21:20:31.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042012+0000", 2024-09-17T21:20:31.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.22e-06 2024-09-17T21:20:31.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042015+0000", 2024-09-17T21:20:31.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4800000000000001e-06 2024-09-17T21:20:31.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042018+0000", 2024-09-17T21:20:31.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.937e-06 2024-09-17T21:20:31.107 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.107 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.107 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.107 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10364 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.107 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041409+0000", 2024-09-17T21:20:31.107 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012601328, 2024-09-17T21:20:31.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013180841, 2024-09-17T21:20:31.108 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.108 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.108 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.108 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.108 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.108 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.108 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10364 2024-09-17T21:20:31.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.108 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041409+0000", 2024-09-17T21:20:31.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041409+0000", 2024-09-17T21:20:31.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041410+0000", 2024-09-17T21:20:31.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6800000000000001e-07 2024-09-17T21:20:31.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041415+0000", 2024-09-17T21:20:31.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0000000000000004e-06 2024-09-17T21:20:31.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041418+0000", 2024-09-17T21:20:31.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4359999999999998e-06 2024-09-17T21:20:31.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041421+0000", 2024-09-17T21:20:31.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7549999999999999e-06 2024-09-17T21:20:31.110 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.110 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.110 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.111 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10398 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.111 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042022+0000", 2024-09-17T21:20:31.111 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.01198888, 2024-09-17T21:20:31.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012592186, 2024-09-17T21:20:31.111 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.111 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.112 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.112 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.112 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.112 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.112 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10398 2024-09-17T21:20:31.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.112 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042022+0000", 2024-09-17T21:20:31.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042022+0000", 2024-09-17T21:20:31.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042022+0000", 2024-09-17T21:20:31.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:20:31.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042023+0000", 2024-09-17T21:20:31.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.305e-06 2024-09-17T21:20:31.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042027+0000", 2024-09-17T21:20:31.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7060000000000002e-06 2024-09-17T21:20:31.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042030+0000", 2024-09-17T21:20:31.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.818e-06 2024-09-17T21:20:31.114 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.114 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.114 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.115 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10365 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.115 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041425+0000", 2024-09-17T21:20:31.115 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012585860000000001, 2024-09-17T21:20:31.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013217276999999999, 2024-09-17T21:20:31.115 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.115 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.115 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.115 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.115 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.115 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.115 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10365 2024-09-17T21:20:31.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.115 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041425+0000", 2024-09-17T21:20:31.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041425+0000", 2024-09-17T21:20:31.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041425+0000", 2024-09-17T21:20:31.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4799999999999999e-07 2024-09-17T21:20:31.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041426+0000", 2024-09-17T21:20:31.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1999999999999999e-06 2024-09-17T21:20:31.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041429+0000", 2024-09-17T21:20:31.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9799999999999998e-06 2024-09-17T21:20:31.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041432+0000", 2024-09-17T21:20:31.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7010000000000001e-06 2024-09-17T21:20:31.118 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.118 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.118 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.118 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10399 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.118 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042034+0000", 2024-09-17T21:20:31.118 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.01197721, 2024-09-17T21:20:31.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012657684000000001, 2024-09-17T21:20:31.118 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.118 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.118 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.118 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.118 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.119 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.119 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10399 2024-09-17T21:20:31.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.119 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042034+0000", 2024-09-17T21:20:31.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042034+0000", 2024-09-17T21:20:31.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042034+0000", 2024-09-17T21:20:31.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.17e-07 2024-09-17T21:20:31.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042035+0000", 2024-09-17T21:20:31.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0979999999999999e-06 2024-09-17T21:20:31.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042038+0000", 2024-09-17T21:20:31.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.163e-06 2024-09-17T21:20:31.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042041+0000", 2024-09-17T21:20:31.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.835e-06 2024-09-17T21:20:31.121 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.121 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.121 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.121 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10366 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.121 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041441+0000", 2024-09-17T21:20:31.121 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012570122, 2024-09-17T21:20:31.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013301175, 2024-09-17T21:20:31.122 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.122 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.122 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.122 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.122 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.122 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.122 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10366 2024-09-17T21:20:31.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.122 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041441+0000", 2024-09-17T21:20:31.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041441+0000", 2024-09-17T21:20:31.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041441+0000", 2024-09-17T21:20:31.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3200000000000001e-07 2024-09-17T21:20:31.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041442+0000", 2024-09-17T21:20:31.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.299e-06 2024-09-17T21:20:31.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041446+0000", 2024-09-17T21:20:31.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3979999999999999e-06 2024-09-17T21:20:31.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041448+0000", 2024-09-17T21:20:31.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.92e-06 2024-09-17T21:20:31.124 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.124 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.124 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.124 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10400 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.125 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042066+0000", 2024-09-17T21:20:31.125 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.011944966, 2024-09-17T21:20:31.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012725082, 2024-09-17T21:20:31.125 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.125 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.125 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.125 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.125 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.125 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.125 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10400 2024-09-17T21:20:31.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.125 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042066+0000", 2024-09-17T21:20:31.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042066+0000", 2024-09-17T21:20:31.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042066+0000", 2024-09-17T21:20:31.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1399999999999997e-07 2024-09-17T21:20:31.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042067+0000", 2024-09-17T21:20:31.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2020000000000001e-06 2024-09-17T21:20:31.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042071+0000", 2024-09-17T21:20:31.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8859999999999997e-06 2024-09-17T21:20:31.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042074+0000", 2024-09-17T21:20:31.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8760000000000001e-06 2024-09-17T21:20:31.127 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.127 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10367 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041455+0000", 2024-09-17T21:20:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.01255566, 2024-09-17T21:20:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013389386, 2024-09-17T21:20:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10367 2024-09-17T21:20:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.129 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041455+0000", 2024-09-17T21:20:31.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041455+0000", 2024-09-17T21:20:31.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041455+0000", 2024-09-17T21:20:31.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2600000000000001e-07 2024-09-17T21:20:31.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041457+0000", 2024-09-17T21:20:31.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2720000000000001e-06 2024-09-17T21:20:31.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041460+0000", 2024-09-17T21:20:31.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2540000000000001e-06 2024-09-17T21:20:31.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041463+0000", 2024-09-17T21:20:31.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9220000000000001e-06 2024-09-17T21:20:31.131 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.131 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.131 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.131 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10401 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.131 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042079+0000", 2024-09-17T21:20:31.131 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.011931973, 2024-09-17T21:20:31.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012814298, 2024-09-17T21:20:31.131 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.131 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.131 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.131 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.131 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.131 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.132 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10401 2024-09-17T21:20:31.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.132 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042079+0000", 2024-09-17T21:20:31.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042079+0000", 2024-09-17T21:20:31.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042079+0000", 2024-09-17T21:20:31.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:20:31.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042080+0000", 2024-09-17T21:20:31.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3060000000000001e-06 2024-09-17T21:20:31.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042084+0000", 2024-09-17T21:20:31.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1839999999999999e-06 2024-09-17T21:20:31.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042086+0000", 2024-09-17T21:20:31.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6800000000000002e-06 2024-09-17T21:20:31.134 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.134 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.134 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.134 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10368 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.134 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041489+0000", 2024-09-17T21:20:31.134 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012521361999999999, 2024-09-17T21:20:31.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013431792, 2024-09-17T21:20:31.135 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.135 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.135 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.135 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.135 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.135 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.135 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10368 2024-09-17T21:20:31.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.135 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041489+0000", 2024-09-17T21:20:31.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041489+0000", 2024-09-17T21:20:31.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041490+0000", 2024-09-17T21:20:31.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9799999999999999e-07 2024-09-17T21:20:31.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041491+0000", 2024-09-17T21:20:31.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:20:31.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041494+0000", 2024-09-17T21:20:31.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3689999999999998e-06 2024-09-17T21:20:31.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041497+0000", 2024-09-17T21:20:31.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9919999999999999e-06 2024-09-17T21:20:31.137 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.137 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.137 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.138 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10402 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.138 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042115+0000", 2024-09-17T21:20:31.138 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.011895476, 2024-09-17T21:20:31.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012830391999999999, 2024-09-17T21:20:31.138 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.138 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.138 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.138 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.138 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.138 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.138 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10402 2024-09-17T21:20:31.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.138 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042115+0000", 2024-09-17T21:20:31.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042115+0000", 2024-09-17T21:20:31.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042116+0000", 2024-09-17T21:20:31.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3000000000000002e-07 2024-09-17T21:20:31.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042120+0000", 2024-09-17T21:20:31.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8959999999999996e-06 2024-09-17T21:20:31.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042127+0000", 2024-09-17T21:20:31.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4229999999999999e-06 2024-09-17T21:20:31.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042131+0000", 2024-09-17T21:20:31.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6739999999999999e-06 2024-09-17T21:20:31.140 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.140 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.141 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.141 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10369 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.141 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041501+0000", 2024-09-17T21:20:31.141 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012509621, 2024-09-17T21:20:31.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013468513999999999, 2024-09-17T21:20:31.141 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.141 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.141 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.141 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.141 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.141 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.141 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10369 2024-09-17T21:20:31.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.142 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041501+0000", 2024-09-17T21:20:31.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041501+0000", 2024-09-17T21:20:31.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041501+0000", 2024-09-17T21:20:31.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3799999999999999e-07 2024-09-17T21:20:31.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041502+0000", 2024-09-17T21:20:31.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.119e-06 2024-09-17T21:20:31.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041506+0000", 2024-09-17T21:20:31.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0359999999999998e-06 2024-09-17T21:20:31.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041509+0000", 2024-09-17T21:20:31.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.106e-06 2024-09-17T21:20:31.144 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.144 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.144 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.144 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10403 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.144 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042135+0000", 2024-09-17T21:20:31.144 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.011875594999999999, 2024-09-17T21:20:31.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012858273999999999, 2024-09-17T21:20:31.144 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.144 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.144 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.144 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.144 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.145 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.145 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10403 2024-09-17T21:20:31.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.145 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042135+0000", 2024-09-17T21:20:31.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042135+0000", 2024-09-17T21:20:31.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042135+0000", 2024-09-17T21:20:31.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:20:31.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042137+0000", 2024-09-17T21:20:31.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.254e-06 2024-09-17T21:20:31.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042140+0000", 2024-09-17T21:20:31.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0680000000000001e-06 2024-09-17T21:20:31.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042143+0000", 2024-09-17T21:20:31.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1070000000000001e-06 2024-09-17T21:20:31.147 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.147 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.147 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.147 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10371 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.147 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041512+0000", 2024-09-17T21:20:31.147 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012498818, 2024-09-17T21:20:31.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013511749, 2024-09-17T21:20:31.147 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.147 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.147 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.147 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.148 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.148 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.148 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10371 2024-09-17T21:20:31.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.148 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041512+0000", 2024-09-17T21:20:31.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041512+0000", 2024-09-17T21:20:31.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041512+0000", 2024-09-17T21:20:31.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:20:31.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041513+0000", 2024-09-17T21:20:31.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1149999999999999e-06 2024-09-17T21:20:31.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041516+0000", 2024-09-17T21:20:31.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0199999999999999e-06 2024-09-17T21:20:31.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041519+0000", 2024-09-17T21:20:31.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7130000000000001e-06 2024-09-17T21:20:31.150 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.150 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.150 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.150 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10404 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.150 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042146+0000", 2024-09-17T21:20:31.150 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.011864473, 2024-09-17T21:20:31.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01291724, 2024-09-17T21:20:31.150 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.151 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.151 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.151 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.151 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.151 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.151 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10404 2024-09-17T21:20:31.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.151 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042146+0000", 2024-09-17T21:20:31.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042146+0000", 2024-09-17T21:20:31.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042147+0000", 2024-09-17T21:20:31.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3200000000000001e-07 2024-09-17T21:20:31.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042148+0000", 2024-09-17T21:20:31.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.105e-06 2024-09-17T21:20:31.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042151+0000", 2024-09-17T21:20:31.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0639999999999998e-06 2024-09-17T21:20:31.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042153+0000", 2024-09-17T21:20:31.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7269999999999999e-06 2024-09-17T21:20:31.153 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.153 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.153 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.154 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10372 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.154 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041546+0000", 2024-09-17T21:20:31.154 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012464494, 2024-09-17T21:20:31.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013542388000000001, 2024-09-17T21:20:31.154 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.154 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.154 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.154 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.154 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.154 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.154 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10372 2024-09-17T21:20:31.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.154 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041546+0000", 2024-09-17T21:20:31.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041546+0000", 2024-09-17T21:20:31.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041547+0000", 2024-09-17T21:20:31.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:20:31.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041548+0000", 2024-09-17T21:20:31.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.449e-06 2024-09-17T21:20:31.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041557+0000", 2024-09-17T21:20:31.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8079999999999994e-06 2024-09-17T21:20:31.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041561+0000", 2024-09-17T21:20:31.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2420000000000002e-06 2024-09-17T21:20:31.156 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.156 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.156 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.157 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10405 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.157 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042180+0000", 2024-09-17T21:20:31.157 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.011831168, 2024-09-17T21:20:31.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012933027999999999, 2024-09-17T21:20:31.157 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.157 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.157 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.157 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.157 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.157 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.157 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10405 2024-09-17T21:20:31.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.158 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042180+0000", 2024-09-17T21:20:31.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042180+0000", 2024-09-17T21:20:31.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042180+0000", 2024-09-17T21:20:31.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.01e-07 2024-09-17T21:20:31.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042181+0000", 2024-09-17T21:20:31.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3549999999999999e-06 2024-09-17T21:20:31.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042190+0000", 2024-09-17T21:20:31.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5299999999999996e-06 2024-09-17T21:20:31.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042193+0000", 2024-09-17T21:20:31.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0869999999999998e-06 2024-09-17T21:20:31.160 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.160 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.160 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.160 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10374 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.160 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041565+0000", 2024-09-17T21:20:31.160 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012445543999999999, 2024-09-17T21:20:31.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013571121, 2024-09-17T21:20:31.160 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.160 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.160 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.160 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.161 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.161 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.161 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10374 2024-09-17T21:20:31.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.161 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041565+0000", 2024-09-17T21:20:31.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041565+0000", 2024-09-17T21:20:31.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041566+0000", 2024-09-17T21:20:31.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7000000000000005e-07 2024-09-17T21:20:31.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041567+0000", 2024-09-17T21:20:31.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.288e-06 2024-09-17T21:20:31.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041570+0000", 2024-09-17T21:20:31.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1980000000000001e-06 2024-09-17T21:20:31.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041573+0000", 2024-09-17T21:20:31.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:20:31.163 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.163 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.163 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.163 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10406 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.163 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042197+0000", 2024-09-17T21:20:31.163 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.011813726, 2024-09-17T21:20:31.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012963282, 2024-09-17T21:20:31.164 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.164 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.164 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.164 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.164 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.164 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.164 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10406 2024-09-17T21:20:31.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.164 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042197+0000", 2024-09-17T21:20:31.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042197+0000", 2024-09-17T21:20:31.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042197+0000", 2024-09-17T21:20:31.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.35e-07 2024-09-17T21:20:31.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042199+0000", 2024-09-17T21:20:31.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2550000000000001e-06 2024-09-17T21:20:31.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042202+0000", 2024-09-17T21:20:31.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.224e-06 2024-09-17T21:20:31.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042205+0000", 2024-09-17T21:20:31.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8219999999999998e-06 2024-09-17T21:20:31.166 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.166 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.166 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.166 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10339 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 2199552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.166 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.040917+0000", 2024-09-17T21:20:31.167 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.013093449, 2024-09-17T21:20:31.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014271862, 2024-09-17T21:20:31.167 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.167 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.167 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:31.167 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.167 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.167 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.167 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10339 2024-09-17T21:20:31.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.167 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040917+0000", 2024-09-17T21:20:31.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040917+0000", 2024-09-17T21:20:31.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040918+0000", 2024-09-17T21:20:31.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:20:31.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040919+0000", 2024-09-17T21:20:31.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.497e-06 2024-09-17T21:20:31.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040923+0000", 2024-09-17T21:20:31.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4470000000000002e-06 2024-09-17T21:20:31.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040927+0000", 2024-09-17T21:20:31.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.946e-06 2024-09-17T21:20:31.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.055051+0000", 2024-09-17T21:20:31.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014123935000000001 2024-09-17T21:20:31.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:31.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.055152+0000", 2024-09-17T21:20:31.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010145999999999999 2024-09-17T21:20:31.170 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.170 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.170 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.170 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10375 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.170 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041577+0000", 2024-09-17T21:20:31.171 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012433964, 2024-09-17T21:20:31.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013641464000000001, 2024-09-17T21:20:31.171 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.171 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.171 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.171 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.171 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.171 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.171 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10375 2024-09-17T21:20:31.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.171 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041577+0000", 2024-09-17T21:20:31.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041577+0000", 2024-09-17T21:20:31.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041577+0000", 2024-09-17T21:20:31.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.72e-07 2024-09-17T21:20:31.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041578+0000", 2024-09-17T21:20:31.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1519999999999999e-06 2024-09-17T21:20:31.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041581+0000", 2024-09-17T21:20:31.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.977e-06 2024-09-17T21:20:31.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041584+0000", 2024-09-17T21:20:31.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6560000000000001e-06 2024-09-17T21:20:31.173 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.173 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.174 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.174 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10407 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.174 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042208+0000", 2024-09-17T21:20:31.174 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.011802523000000001, 2024-09-17T21:20:31.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013083032, 2024-09-17T21:20:31.174 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.174 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.174 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.174 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.174 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.174 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.174 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10407 2024-09-17T21:20:31.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.175 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042208+0000", 2024-09-17T21:20:31.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042208+0000", 2024-09-17T21:20:31.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042209+0000", 2024-09-17T21:20:31.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:20:31.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042210+0000", 2024-09-17T21:20:31.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1000000000000001e-06 2024-09-17T21:20:31.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042213+0000", 2024-09-17T21:20:31.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.999e-06 2024-09-17T21:20:31.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042215+0000", 2024-09-17T21:20:31.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.712e-06 2024-09-17T21:20:31.177 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.177 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.177 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.177 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10340 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 2176000~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.177 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.040937+0000", 2024-09-17T21:20:31.177 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.013074194000000001, 2024-09-17T21:20:31.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014409768999999999, 2024-09-17T21:20:31.177 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.177 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.177 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "reached pg", 2024-09-17T21:20:31.178 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.178 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.178 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.178 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10340 2024-09-17T21:20:31.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.178 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040937+0000", 2024-09-17T21:20:31.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040937+0000", 2024-09-17T21:20:31.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040937+0000", 2024-09-17T21:20:31.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7099999999999998e-07 2024-09-17T21:20:31.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040938+0000", 2024-09-17T21:20:31.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3060000000000001e-06 2024-09-17T21:20:31.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040946+0000", 2024-09-17T21:20:31.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2069999999999995e-06 2024-09-17T21:20:31.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040950+0000", 2024-09-17T21:20:31.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:20:31.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.055285+0000", 2024-09-17T21:20:31.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014334700000000001 2024-09-17T21:20:31.180 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.180 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.180 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.181 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10376 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.181 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041597+0000", 2024-09-17T21:20:31.181 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012413588999999999, 2024-09-17T21:20:31.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013807565000000001, 2024-09-17T21:20:31.181 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.181 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.181 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.181 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.181 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.181 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.181 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10376 2024-09-17T21:20:31.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.182 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041597+0000", 2024-09-17T21:20:31.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041597+0000", 2024-09-17T21:20:31.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041597+0000", 2024-09-17T21:20:31.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5199999999999998e-07 2024-09-17T21:20:31.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041599+0000", 2024-09-17T21:20:31.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.232e-06 2024-09-17T21:20:31.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041602+0000", 2024-09-17T21:20:31.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4520000000000002e-06 2024-09-17T21:20:31.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041605+0000", 2024-09-17T21:20:31.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8490000000000002e-06 2024-09-17T21:20:31.184 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.184 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.184 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.184 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10408 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.184 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042231+0000", 2024-09-17T21:20:31.184 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.011779853999999999, 2024-09-17T21:20:31.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013224252000000001, 2024-09-17T21:20:31.184 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.184 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.184 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.184 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.184 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.185 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.185 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10408 2024-09-17T21:20:31.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.185 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042231+0000", 2024-09-17T21:20:31.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042231+0000", 2024-09-17T21:20:31.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042231+0000", 2024-09-17T21:20:31.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3099999999999999e-07 2024-09-17T21:20:31.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042232+0000", 2024-09-17T21:20:31.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2410000000000001e-06 2024-09-17T21:20:31.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042236+0000", 2024-09-17T21:20:31.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3220000000000001e-06 2024-09-17T21:20:31.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042239+0000", 2024-09-17T21:20:31.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.819e-06 2024-09-17T21:20:31.187 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.187 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.187 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.187 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10341 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.187 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.040956+0000", 2024-09-17T21:20:31.187 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.013054787999999999, 2024-09-17T21:20:31.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014550416, 2024-09-17T21:20:31.187 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.187 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.187 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.188 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.188 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.188 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.188 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10341 2024-09-17T21:20:31.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.188 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040956+0000", 2024-09-17T21:20:31.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040956+0000", 2024-09-17T21:20:31.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040957+0000", 2024-09-17T21:20:31.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9599999999999999e-07 2024-09-17T21:20:31.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040959+0000", 2024-09-17T21:20:31.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2570000000000002e-06 2024-09-17T21:20:31.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040964+0000", 2024-09-17T21:20:31.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5679999999999999e-06 2024-09-17T21:20:31.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040968+0000", 2024-09-17T21:20:31.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1889999999999999e-06 2024-09-17T21:20:31.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.055511+0000", 2024-09-17T21:20:31.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014542955999999999 2024-09-17T21:20:31.190 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.190 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.190 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.191 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10377 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.191 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041609+0000", 2024-09-17T21:20:31.191 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012401977999999999, 2024-09-17T21:20:31.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013929405000000001, 2024-09-17T21:20:31.191 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.191 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.191 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.191 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.191 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.191 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.191 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10377 2024-09-17T21:20:31.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.191 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041609+0000", 2024-09-17T21:20:31.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041609+0000", 2024-09-17T21:20:31.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041609+0000", 2024-09-17T21:20:31.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3799999999999999e-07 2024-09-17T21:20:31.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041610+0000", 2024-09-17T21:20:31.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.108e-06 2024-09-17T21:20:31.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041613+0000", 2024-09-17T21:20:31.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0970000000000002e-06 2024-09-17T21:20:31.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041616+0000", 2024-09-17T21:20:31.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6579999999999998e-06 2024-09-17T21:20:31.194 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.194 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.194 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.194 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10409 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.194 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042243+0000", 2024-09-17T21:20:31.194 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.011768104, 2024-09-17T21:20:31.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013321872, 2024-09-17T21:20:31.194 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.194 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.194 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.194 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.194 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.194 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.195 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10409 2024-09-17T21:20:31.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.195 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042243+0000", 2024-09-17T21:20:31.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042243+0000", 2024-09-17T21:20:31.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042243+0000", 2024-09-17T21:20:31.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3999999999999998e-07 2024-09-17T21:20:31.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042244+0000", 2024-09-17T21:20:31.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1489999999999999e-06 2024-09-17T21:20:31.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042247+0000", 2024-09-17T21:20:31.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0139999999999999e-06 2024-09-17T21:20:31.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042250+0000", 2024-09-17T21:20:31.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8550000000000002e-06 2024-09-17T21:20:31.197 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.197 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.197 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.197 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10342 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.197 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.040989+0000", 2024-09-17T21:20:31.197 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.013021809000000001, 2024-09-17T21:20:31.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014599974999999999, 2024-09-17T21:20:31.197 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.197 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.197 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.197 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.198 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.198 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.198 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10342 2024-09-17T21:20:31.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.198 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040989+0000", 2024-09-17T21:20:31.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040989+0000", 2024-09-17T21:20:31.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040989+0000", 2024-09-17T21:20:31.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.79e-07 2024-09-17T21:20:31.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040991+0000", 2024-09-17T21:20:31.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3659999999999999e-06 2024-09-17T21:20:31.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040994+0000", 2024-09-17T21:20:31.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.467e-06 2024-09-17T21:20:31.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.040997+0000", 2024-09-17T21:20:31.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7690000000000001e-06 2024-09-17T21:20:31.200 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.200 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.200 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.200 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10378 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.200 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041619+0000", 2024-09-17T21:20:31.200 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012391512, 2024-09-17T21:20:31.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013993675000000001, 2024-09-17T21:20:31.200 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.200 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.201 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.201 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.201 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.201 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.201 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10378 2024-09-17T21:20:31.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.201 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041619+0000", 2024-09-17T21:20:31.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041619+0000", 2024-09-17T21:20:31.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041619+0000", 2024-09-17T21:20:31.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:20:31.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041623+0000", 2024-09-17T21:20:31.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4470000000000002e-06 2024-09-17T21:20:31.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041626+0000", 2024-09-17T21:20:31.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2339999999999999e-06 2024-09-17T21:20:31.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041629+0000", 2024-09-17T21:20:31.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9560000000000002e-06 2024-09-17T21:20:31.203 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.203 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.203 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.203 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10410 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.204 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042253+0000", 2024-09-17T21:20:31.204 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.011757365000000001, 2024-09-17T21:20:31.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013387217, 2024-09-17T21:20:31.204 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.204 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.204 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.204 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.204 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.204 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.204 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10410 2024-09-17T21:20:31.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.204 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042253+0000", 2024-09-17T21:20:31.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042253+0000", 2024-09-17T21:20:31.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042255+0000", 2024-09-17T21:20:31.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:20:31.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042256+0000", 2024-09-17T21:20:31.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.192e-06 2024-09-17T21:20:31.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042260+0000", 2024-09-17T21:20:31.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1159999999999999e-06 2024-09-17T21:20:31.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042262+0000", 2024-09-17T21:20:31.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7659999999999999e-06 2024-09-17T21:20:31.207 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.207 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.207 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.207 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10343 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.207 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041002+0000", 2024-09-17T21:20:31.207 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.013008985000000001, 2024-09-17T21:20:31.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014665522, 2024-09-17T21:20:31.207 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.207 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.207 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.208 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.208 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.208 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.208 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10343 2024-09-17T21:20:31.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.208 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041002+0000", 2024-09-17T21:20:31.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041002+0000", 2024-09-17T21:20:31.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041002+0000", 2024-09-17T21:20:31.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8799999999999998e-07 2024-09-17T21:20:31.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041004+0000", 2024-09-17T21:20:31.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.764e-06 2024-09-17T21:20:31.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041009+0000", 2024-09-17T21:20:31.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6970000000000001e-06 2024-09-17T21:20:31.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041012+0000", 2024-09-17T21:20:31.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0570000000000001e-06 2024-09-17T21:20:31.210 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.210 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.210 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.210 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10379 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.210 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041652+0000", 2024-09-17T21:20:31.210 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012358319, 2024-09-17T21:20:31.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014041448999999999, 2024-09-17T21:20:31.211 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.211 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.211 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.211 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.211 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.211 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.211 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10379 2024-09-17T21:20:31.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.211 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041652+0000", 2024-09-17T21:20:31.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041652+0000", 2024-09-17T21:20:31.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041653+0000", 2024-09-17T21:20:31.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0600000000000001e-07 2024-09-17T21:20:31.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041654+0000", 2024-09-17T21:20:31.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2780000000000001e-06 2024-09-17T21:20:31.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041658+0000", 2024-09-17T21:20:31.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4860000000000002e-06 2024-09-17T21:20:31.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041661+0000", 2024-09-17T21:20:31.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0369999999999999e-06 2024-09-17T21:20:31.214 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.214 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.214 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.214 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10411 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.214 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042266+0000", 2024-09-17T21:20:31.214 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.011744843, 2024-09-17T21:20:31.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013467128, 2024-09-17T21:20:31.214 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.214 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.214 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.214 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.215 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.215 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.215 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10411 2024-09-17T21:20:31.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.215 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042266+0000", 2024-09-17T21:20:31.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042266+0000", 2024-09-17T21:20:31.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042266+0000", 2024-09-17T21:20:31.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:20:31.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042267+0000", 2024-09-17T21:20:31.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1379999999999999e-06 2024-09-17T21:20:31.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042270+0000", 2024-09-17T21:20:31.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0359999999999998e-06 2024-09-17T21:20:31.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042274+0000", 2024-09-17T21:20:31.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.546e-06 2024-09-17T21:20:31.217 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.217 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.217 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.217 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10344 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.217 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041043+0000", 2024-09-17T21:20:31.217 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012967533999999999, 2024-09-17T21:20:31.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014741381, 2024-09-17T21:20:31.217 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.218 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.218 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.218 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.218 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.218 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.218 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10344 2024-09-17T21:20:31.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.218 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041043+0000", 2024-09-17T21:20:31.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041043+0000", 2024-09-17T21:20:31.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041044+0000", 2024-09-17T21:20:31.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:20:31.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041045+0000", 2024-09-17T21:20:31.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.37e-06 2024-09-17T21:20:31.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041051+0000", 2024-09-17T21:20:31.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:20:31.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041055+0000", 2024-09-17T21:20:31.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:20:31.220 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.220 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.221 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.221 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10380 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.221 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041664+0000", 2024-09-17T21:20:31.221 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012346332, 2024-09-17T21:20:31.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014168703, 2024-09-17T21:20:31.221 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.221 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.221 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.221 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.221 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.221 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.221 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10380 2024-09-17T21:20:31.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.222 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041664+0000", 2024-09-17T21:20:31.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041664+0000", 2024-09-17T21:20:31.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041665+0000", 2024-09-17T21:20:31.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.28e-07 2024-09-17T21:20:31.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041666+0000", 2024-09-17T21:20:31.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.142e-06 2024-09-17T21:20:31.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041669+0000", 2024-09-17T21:20:31.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.134e-06 2024-09-17T21:20:31.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041672+0000", 2024-09-17T21:20:31.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6970000000000002e-06 2024-09-17T21:20:31.224 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.224 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.224 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.224 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10412 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.224 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042283+0000", 2024-09-17T21:20:31.224 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.011728216, 2024-09-17T21:20:31.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013602506, 2024-09-17T21:20:31.224 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.224 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.224 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.224 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.224 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.225 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.225 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10412 2024-09-17T21:20:31.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.225 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042283+0000", 2024-09-17T21:20:31.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042283+0000", 2024-09-17T21:20:31.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042283+0000", 2024-09-17T21:20:31.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:20:31.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042284+0000", 2024-09-17T21:20:31.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.207e-06 2024-09-17T21:20:31.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042287+0000", 2024-09-17T21:20:31.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3689999999999998e-06 2024-09-17T21:20:31.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042290+0000", 2024-09-17T21:20:31.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8870000000000001e-06 2024-09-17T21:20:31.227 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.227 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.227 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.227 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10345 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.227 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041060+0000", 2024-09-17T21:20:31.227 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012950844, 2024-09-17T21:20:31.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014873005999999999, 2024-09-17T21:20:31.227 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.227 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.228 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.228 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.228 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.228 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.228 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10345 2024-09-17T21:20:31.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.228 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041060+0000", 2024-09-17T21:20:31.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041060+0000", 2024-09-17T21:20:31.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041060+0000", 2024-09-17T21:20:31.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:20:31.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041062+0000", 2024-09-17T21:20:31.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3319999999999999e-06 2024-09-17T21:20:31.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041065+0000", 2024-09-17T21:20:31.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3799999999999998e-06 2024-09-17T21:20:31.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041068+0000", 2024-09-17T21:20:31.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9799999999999998e-06 2024-09-17T21:20:31.231 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.231 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.231 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.231 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10381 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.231 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041699+0000", 2024-09-17T21:20:31.231 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012311456, 2024-09-17T21:20:31.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014266905, 2024-09-17T21:20:31.231 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.231 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.232 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.232 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.232 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.232 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.232 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10381 2024-09-17T21:20:31.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.232 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041699+0000", 2024-09-17T21:20:31.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041699+0000", 2024-09-17T21:20:31.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041700+0000", 2024-09-17T21:20:31.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4400000000000001e-07 2024-09-17T21:20:31.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041701+0000", 2024-09-17T21:20:31.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.237e-06 2024-09-17T21:20:31.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041704+0000", 2024-09-17T21:20:31.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3419999999999999e-06 2024-09-17T21:20:31.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041707+0000", 2024-09-17T21:20:31.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8930000000000001e-06 2024-09-17T21:20:31.234 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.234 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.234 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.234 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10413 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.234 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042295+0000", 2024-09-17T21:20:31.235 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.011716234000000001, 2024-09-17T21:20:31.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013695761000000001, 2024-09-17T21:20:31.235 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.235 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.235 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.235 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.235 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.235 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.235 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10413 2024-09-17T21:20:31.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.235 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042295+0000", 2024-09-17T21:20:31.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042295+0000", 2024-09-17T21:20:31.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042295+0000", 2024-09-17T21:20:31.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1300000000000001e-07 2024-09-17T21:20:31.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042296+0000", 2024-09-17T21:20:31.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1790000000000001e-06 2024-09-17T21:20:31.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042299+0000", 2024-09-17T21:20:31.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0350000000000002e-06 2024-09-17T21:20:31.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042302+0000", 2024-09-17T21:20:31.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6840000000000001e-06 2024-09-17T21:20:31.237 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.237 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.238 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.238 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10346 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.238 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041072+0000", 2024-09-17T21:20:31.238 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012939127999999999, 2024-09-17T21:20:31.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014945346, 2024-09-17T21:20:31.238 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.238 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.238 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.238 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.238 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.238 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.238 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10346 2024-09-17T21:20:31.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.239 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041072+0000", 2024-09-17T21:20:31.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041072+0000", 2024-09-17T21:20:31.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041072+0000", 2024-09-17T21:20:31.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:20:31.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041073+0000", 2024-09-17T21:20:31.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1379999999999999e-06 2024-09-17T21:20:31.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041076+0000", 2024-09-17T21:20:31.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0189999999999998e-06 2024-09-17T21:20:31.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041079+0000", 2024-09-17T21:20:31.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7489999999999999e-06 2024-09-17T21:20:31.241 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.241 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.241 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.241 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10382 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.241 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041711+0000", 2024-09-17T21:20:31.241 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012299905999999999, 2024-09-17T21:20:31.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014330099000000001, 2024-09-17T21:20:31.241 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.241 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.241 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.241 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.241 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.242 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.242 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10382 2024-09-17T21:20:31.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.242 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041711+0000", 2024-09-17T21:20:31.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041711+0000", 2024-09-17T21:20:31.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041711+0000", 2024-09-17T21:20:31.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3900000000000001e-07 2024-09-17T21:20:31.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041712+0000", 2024-09-17T21:20:31.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.184e-06 2024-09-17T21:20:31.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041715+0000", 2024-09-17T21:20:31.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0489999999999999e-06 2024-09-17T21:20:31.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041718+0000", 2024-09-17T21:20:31.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6630000000000002e-06 2024-09-17T21:20:31.244 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.244 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.244 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.245 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10414 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.245 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042315+0000", 2024-09-17T21:20:31.245 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.011695916000000001, 2024-09-17T21:20:31.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013757289000000001, 2024-09-17T21:20:31.245 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.245 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.245 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.245 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.245 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.245 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.245 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10414 2024-09-17T21:20:31.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.246 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042315+0000", 2024-09-17T21:20:31.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042315+0000", 2024-09-17T21:20:31.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042315+0000", 2024-09-17T21:20:31.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4499999999999998e-07 2024-09-17T21:20:31.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042320+0000", 2024-09-17T21:20:31.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4150000000000001e-06 2024-09-17T21:20:31.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042323+0000", 2024-09-17T21:20:31.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7840000000000001e-06 2024-09-17T21:20:31.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042326+0000", 2024-09-17T21:20:31.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0690000000000002e-06 2024-09-17T21:20:31.248 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.248 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.248 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.248 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10347 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.248 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041092+0000", 2024-09-17T21:20:31.248 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012918816, 2024-09-17T21:20:31.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015004959, 2024-09-17T21:20:31.248 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.249 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.249 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.249 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.249 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.249 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.249 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10347 2024-09-17T21:20:31.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.249 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041092+0000", 2024-09-17T21:20:31.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041092+0000", 2024-09-17T21:20:31.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041092+0000", 2024-09-17T21:20:31.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.53e-07 2024-09-17T21:20:31.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041093+0000", 2024-09-17T21:20:31.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.26e-06 2024-09-17T21:20:31.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041099+0000", 2024-09-17T21:20:31.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1379999999999998e-06 2024-09-17T21:20:31.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041104+0000", 2024-09-17T21:20:31.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9139999999999999e-06 2024-09-17T21:20:31.251 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.251 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.251 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.252 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10383 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.252 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041721+0000", 2024-09-17T21:20:31.252 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012289385, 2024-09-17T21:20:31.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014401116, 2024-09-17T21:20:31.252 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.252 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.252 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.252 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.252 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.252 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.252 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10383 2024-09-17T21:20:31.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.253 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041721+0000", 2024-09-17T21:20:31.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041721+0000", 2024-09-17T21:20:31.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041722+0000", 2024-09-17T21:20:31.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.28e-07 2024-09-17T21:20:31.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041723+0000", 2024-09-17T21:20:31.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.02e-06 2024-09-17T21:20:31.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041726+0000", 2024-09-17T21:20:31.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.016e-06 2024-09-17T21:20:31.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041729+0000", 2024-09-17T21:20:31.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1070000000000001e-06 2024-09-17T21:20:31.255 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.255 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.255 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.255 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10415 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.255 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042331+0000", 2024-09-17T21:20:31.255 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.011679991000000001, 2024-09-17T21:20:31.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013816286000000001, 2024-09-17T21:20:31.255 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.255 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.255 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.255 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.255 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.255 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.255 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10415 2024-09-17T21:20:31.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.256 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042331+0000", 2024-09-17T21:20:31.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042331+0000", 2024-09-17T21:20:31.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042331+0000", 2024-09-17T21:20:31.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3799999999999999e-07 2024-09-17T21:20:31.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042332+0000", 2024-09-17T21:20:31.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1570000000000001e-06 2024-09-17T21:20:31.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042335+0000", 2024-09-17T21:20:31.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.072e-06 2024-09-17T21:20:31.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042338+0000", 2024-09-17T21:20:31.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7530000000000002e-06 2024-09-17T21:20:31.258 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.258 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.258 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.258 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10348 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.258 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041109+0000", 2024-09-17T21:20:31.258 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012901763, 2024-09-17T21:20:31.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015077775999999999, 2024-09-17T21:20:31.258 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.258 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.258 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.258 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.259 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.259 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.259 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10348 2024-09-17T21:20:31.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.259 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041109+0000", 2024-09-17T21:20:31.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041109+0000", 2024-09-17T21:20:31.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041109+0000", 2024-09-17T21:20:31.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5499999999999999e-07 2024-09-17T21:20:31.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041111+0000", 2024-09-17T21:20:31.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.432e-06 2024-09-17T21:20:31.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041114+0000", 2024-09-17T21:20:31.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.422e-06 2024-09-17T21:20:31.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041117+0000", 2024-09-17T21:20:31.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9040000000000001e-06 2024-09-17T21:20:31.261 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.261 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.261 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.261 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10384 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.261 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041757+0000", 2024-09-17T21:20:31.262 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012253328000000001, 2024-09-17T21:20:31.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014481064, 2024-09-17T21:20:31.262 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.262 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.262 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.262 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.262 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.262 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.262 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10384 2024-09-17T21:20:31.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.262 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041757+0000", 2024-09-17T21:20:31.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041757+0000", 2024-09-17T21:20:31.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041758+0000", 2024-09-17T21:20:31.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.32e-07 2024-09-17T21:20:31.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041759+0000", 2024-09-17T21:20:31.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.238e-06 2024-09-17T21:20:31.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041767+0000", 2024-09-17T21:20:31.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3429999999999996e-06 2024-09-17T21:20:31.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041772+0000", 2024-09-17T21:20:31.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2989999999999998e-06 2024-09-17T21:20:31.264 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.264 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.264 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.265 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10416 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.265 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042367+0000", 2024-09-17T21:20:31.265 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.011643389000000001, 2024-09-17T21:20:31.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013919564000000001, 2024-09-17T21:20:31.265 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.265 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.265 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.265 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.265 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.265 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.265 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10416 2024-09-17T21:20:31.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.265 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042367+0000", 2024-09-17T21:20:31.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042367+0000", 2024-09-17T21:20:31.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042368+0000", 2024-09-17T21:20:31.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4799999999999999e-07 2024-09-17T21:20:31.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042369+0000", 2024-09-17T21:20:31.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2419999999999999e-06 2024-09-17T21:20:31.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042372+0000", 2024-09-17T21:20:31.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4640000000000002e-06 2024-09-17T21:20:31.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042375+0000", 2024-09-17T21:20:31.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.977e-06 2024-09-17T21:20:31.267 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.267 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.268 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.268 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10349 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.268 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041146+0000", 2024-09-17T21:20:31.268 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012864792999999999, 2024-09-17T21:20:31.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015192505, 2024-09-17T21:20:31.268 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.268 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.268 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.268 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.268 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.268 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.268 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10349 2024-09-17T21:20:31.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.269 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041146+0000", 2024-09-17T21:20:31.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041146+0000", 2024-09-17T21:20:31.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041147+0000", 2024-09-17T21:20:31.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5600000000000005e-07 2024-09-17T21:20:31.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041149+0000", 2024-09-17T21:20:31.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9139999999999998e-06 2024-09-17T21:20:31.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041154+0000", 2024-09-17T21:20:31.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8579999999999996e-06 2024-09-17T21:20:31.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041157+0000", 2024-09-17T21:20:31.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.993e-06 2024-09-17T21:20:31.271 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.271 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.271 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.271 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10385 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.271 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041776+0000", 2024-09-17T21:20:31.271 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012235074, 2024-09-17T21:20:31.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014605325000000001, 2024-09-17T21:20:31.271 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.271 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.271 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.272 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.272 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.272 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.272 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10385 2024-09-17T21:20:31.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.272 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041776+0000", 2024-09-17T21:20:31.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041776+0000", 2024-09-17T21:20:31.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041776+0000", 2024-09-17T21:20:31.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2100000000000002e-07 2024-09-17T21:20:31.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041777+0000", 2024-09-17T21:20:31.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2950000000000001e-06 2024-09-17T21:20:31.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041781+0000", 2024-09-17T21:20:31.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.18e-06 2024-09-17T21:20:31.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041783+0000", 2024-09-17T21:20:31.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8329999999999998e-06 2024-09-17T21:20:31.274 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.274 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.274 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.274 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10417 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.275 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042381+0000", 2024-09-17T21:20:31.275 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.011630191, 2024-09-17T21:20:31.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014036507, 2024-09-17T21:20:31.275 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.275 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.275 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.275 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.275 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.275 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.275 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10417 2024-09-17T21:20:31.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.275 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042381+0000", 2024-09-17T21:20:31.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042381+0000", 2024-09-17T21:20:31.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042381+0000", 2024-09-17T21:20:31.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2000000000000001e-07 2024-09-17T21:20:31.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042382+0000", 2024-09-17T21:20:31.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.206e-06 2024-09-17T21:20:31.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042385+0000", 2024-09-17T21:20:31.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.208e-06 2024-09-17T21:20:31.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042388+0000", 2024-09-17T21:20:31.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.756e-06 2024-09-17T21:20:31.278 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.278 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.278 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.278 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10350 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.278 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041163+0000", 2024-09-17T21:20:31.278 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012847356000000001, 2024-09-17T21:20:31.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015282241, 2024-09-17T21:20:31.278 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.278 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.278 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.279 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.279 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10350 2024-09-17T21:20:31.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.279 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041163+0000", 2024-09-17T21:20:31.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041163+0000", 2024-09-17T21:20:31.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041164+0000", 2024-09-17T21:20:31.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5400000000000002e-07 2024-09-17T21:20:31.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041172+0000", 2024-09-17T21:20:31.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5949999999999999e-06 2024-09-17T21:20:31.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041176+0000", 2024-09-17T21:20:31.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9079999999999997e-06 2024-09-17T21:20:31.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041179+0000", 2024-09-17T21:20:31.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0960000000000001e-06 2024-09-17T21:20:31.281 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.281 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.281 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.281 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10386 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.281 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041787+0000", 2024-09-17T21:20:31.281 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012223645999999999, 2024-09-17T21:20:31.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014684969000000001, 2024-09-17T21:20:31.281 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.282 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.282 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.282 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10386 2024-09-17T21:20:31.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.282 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041787+0000", 2024-09-17T21:20:31.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041787+0000", 2024-09-17T21:20:31.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041787+0000", 2024-09-17T21:20:31.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:20:31.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041789+0000", 2024-09-17T21:20:31.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1230000000000001e-06 2024-09-17T21:20:31.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041791+0000", 2024-09-17T21:20:31.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9000000000000002e-06 2024-09-17T21:20:31.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041794+0000", 2024-09-17T21:20:31.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.694e-06 2024-09-17T21:20:31.284 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.284 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.285 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.285 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10418 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.285 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042415+0000", 2024-09-17T21:20:31.285 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.011595503, 2024-09-17T21:20:31.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014081945, 2024-09-17T21:20:31.285 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.285 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.285 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.285 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.285 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.285 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.286 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10418 2024-09-17T21:20:31.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.286 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042415+0000", 2024-09-17T21:20:31.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042415+0000", 2024-09-17T21:20:31.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042416+0000", 2024-09-17T21:20:31.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:20:31.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042417+0000", 2024-09-17T21:20:31.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2720000000000001e-06 2024-09-17T21:20:31.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042420+0000", 2024-09-17T21:20:31.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3189999999999999e-06 2024-09-17T21:20:31.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042423+0000", 2024-09-17T21:20:31.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9179999999999998e-06 2024-09-17T21:20:31.288 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.288 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.288 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.288 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10351 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.288 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041197+0000", 2024-09-17T21:20:31.288 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012813849, 2024-09-17T21:20:31.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015382817, 2024-09-17T21:20:31.289 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.289 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.289 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.289 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.289 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.289 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.289 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10351 2024-09-17T21:20:31.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.289 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041197+0000", 2024-09-17T21:20:31.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041197+0000", 2024-09-17T21:20:31.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041197+0000", 2024-09-17T21:20:31.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1699999999999999e-07 2024-09-17T21:20:31.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041199+0000", 2024-09-17T21:20:31.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.325e-06 2024-09-17T21:20:31.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041204+0000", 2024-09-17T21:20:31.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4700000000000001e-06 2024-09-17T21:20:31.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041209+0000", 2024-09-17T21:20:31.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9339999999999997e-06 2024-09-17T21:20:31.291 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.291 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.291 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.292 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10387 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.292 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041807+0000", 2024-09-17T21:20:31.292 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0122033, 2024-09-17T21:20:31.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014816213999999999, 2024-09-17T21:20:31.292 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.292 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.292 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.292 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.292 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.292 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.292 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10387 2024-09-17T21:20:31.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.292 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041807+0000", 2024-09-17T21:20:31.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041807+0000", 2024-09-17T21:20:31.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041808+0000", 2024-09-17T21:20:31.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3099999999999999e-07 2024-09-17T21:20:31.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041809+0000", 2024-09-17T21:20:31.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.139e-06 2024-09-17T21:20:31.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041812+0000", 2024-09-17T21:20:31.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3960000000000002e-06 2024-09-17T21:20:31.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041815+0000", 2024-09-17T21:20:31.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7669999999999999e-06 2024-09-17T21:20:31.294 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.295 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.295 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.295 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10419 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.295 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042427+0000", 2024-09-17T21:20:31.295 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.011583812000000001, 2024-09-17T21:20:31.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014226525, 2024-09-17T21:20:31.295 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.295 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.295 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.295 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.295 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.295 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.296 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10419 2024-09-17T21:20:31.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.296 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042427+0000", 2024-09-17T21:20:31.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042427+0000", 2024-09-17T21:20:31.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042427+0000", 2024-09-17T21:20:31.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:20:31.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042428+0000", 2024-09-17T21:20:31.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.02e-06 2024-09-17T21:20:31.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042431+0000", 2024-09-17T21:20:31.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0589999999999998e-06 2024-09-17T21:20:31.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042434+0000", 2024-09-17T21:20:31.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7010000000000001e-06 2024-09-17T21:20:31.298 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.298 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.298 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.298 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10352 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.298 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041216+0000", 2024-09-17T21:20:31.298 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012795045999999999, 2024-09-17T21:20:31.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015462442999999999, 2024-09-17T21:20:31.298 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.298 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.298 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.299 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10352 2024-09-17T21:20:31.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.299 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041216+0000", 2024-09-17T21:20:31.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041216+0000", 2024-09-17T21:20:31.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041216+0000", 2024-09-17T21:20:31.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8099999999999999e-07 2024-09-17T21:20:31.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041218+0000", 2024-09-17T21:20:31.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.649e-06 2024-09-17T21:20:31.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041221+0000", 2024-09-17T21:20:31.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.675e-06 2024-09-17T21:20:31.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041224+0000", 2024-09-17T21:20:31.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9969999999999999e-06 2024-09-17T21:20:31.301 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.301 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.301 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.301 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10388 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.301 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041819+0000", 2024-09-17T21:20:31.301 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012192075, 2024-09-17T21:20:31.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014886900999999999, 2024-09-17T21:20:31.302 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.302 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.302 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.302 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.302 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.302 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.302 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10388 2024-09-17T21:20:31.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.302 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041819+0000", 2024-09-17T21:20:31.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041819+0000", 2024-09-17T21:20:31.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041819+0000", 2024-09-17T21:20:31.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3999999999999998e-07 2024-09-17T21:20:31.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041820+0000", 2024-09-17T21:20:31.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.313e-06 2024-09-17T21:20:31.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041824+0000", 2024-09-17T21:20:31.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6899999999999998e-06 2024-09-17T21:20:31.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041827+0000", 2024-09-17T21:20:31.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7669999999999999e-06 2024-09-17T21:20:31.304 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.304 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.304 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.305 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10420 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.305 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042438+0000", 2024-09-17T21:20:31.305 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.011573221999999999, 2024-09-17T21:20:31.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014294543999999999, 2024-09-17T21:20:31.305 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.305 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.305 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.305 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.305 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.305 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.305 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10420 2024-09-17T21:20:31.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.305 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042438+0000", 2024-09-17T21:20:31.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042438+0000", 2024-09-17T21:20:31.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042438+0000", 2024-09-17T21:20:31.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2000000000000001e-07 2024-09-17T21:20:31.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042439+0000", 2024-09-17T21:20:31.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.308e-06 2024-09-17T21:20:31.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042442+0000", 2024-09-17T21:20:31.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8779999999999998e-06 2024-09-17T21:20:31.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042445+0000", 2024-09-17T21:20:31.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7240000000000001e-06 2024-09-17T21:20:31.307 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.307 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.308 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.308 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10354 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.308 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041251+0000", 2024-09-17T21:20:31.308 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012759283, 2024-09-17T21:20:31.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015516594999999999, 2024-09-17T21:20:31.308 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.308 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.308 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.308 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.308 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.308 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.309 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10354 2024-09-17T21:20:31.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.309 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041251+0000", 2024-09-17T21:20:31.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041251+0000", 2024-09-17T21:20:31.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041252+0000", 2024-09-17T21:20:31.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4999999999999998e-07 2024-09-17T21:20:31.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041253+0000", 2024-09-17T21:20:31.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.305e-06 2024-09-17T21:20:31.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041257+0000", 2024-09-17T21:20:31.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.444e-06 2024-09-17T21:20:31.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041260+0000", 2024-09-17T21:20:31.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1429999999999998e-06 2024-09-17T21:20:31.311 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.311 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.311 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.311 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10389 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.311 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041830+0000", 2024-09-17T21:20:31.311 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012180478, 2024-09-17T21:20:31.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014971498999999999, 2024-09-17T21:20:31.311 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.311 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.311 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.311 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.311 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.311 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.312 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10389 2024-09-17T21:20:31.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.312 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041830+0000", 2024-09-17T21:20:31.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041830+0000", 2024-09-17T21:20:31.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041831+0000", 2024-09-17T21:20:31.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5200000000000002e-07 2024-09-17T21:20:31.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041832+0000", 2024-09-17T21:20:31.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0899999999999999e-06 2024-09-17T21:20:31.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041835+0000", 2024-09-17T21:20:31.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0070000000000002e-06 2024-09-17T21:20:31.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041838+0000", 2024-09-17T21:20:31.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.712e-06 2024-09-17T21:20:31.314 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.314 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.314 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.316 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10421 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.316 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042470+0000", 2024-09-17T21:20:31.316 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.011540646, 2024-09-17T21:20:31.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014358142000000001, 2024-09-17T21:20:31.316 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.316 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.316 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.316 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.316 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.317 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.317 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10421 2024-09-17T21:20:31.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.317 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042470+0000", 2024-09-17T21:20:31.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042470+0000", 2024-09-17T21:20:31.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042471+0000", 2024-09-17T21:20:31.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1699999999999999e-07 2024-09-17T21:20:31.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042472+0000", 2024-09-17T21:20:31.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4079999999999999e-06 2024-09-17T21:20:31.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042475+0000", 2024-09-17T21:20:31.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3670000000000001e-06 2024-09-17T21:20:31.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042478+0000", 2024-09-17T21:20:31.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.976e-06 2024-09-17T21:20:31.319 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.319 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.319 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.319 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10355 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.319 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041264+0000", 2024-09-17T21:20:31.320 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012746993, 2024-09-17T21:20:31.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015592508999999999, 2024-09-17T21:20:31.320 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.320 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.320 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.320 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.320 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.320 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.320 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10355 2024-09-17T21:20:31.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.320 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041264+0000", 2024-09-17T21:20:31.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041264+0000", 2024-09-17T21:20:31.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041264+0000", 2024-09-17T21:20:31.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5199999999999998e-07 2024-09-17T21:20:31.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041265+0000", 2024-09-17T21:20:31.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1260000000000001e-06 2024-09-17T21:20:31.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041268+0000", 2024-09-17T21:20:31.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.044e-06 2024-09-17T21:20:31.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041271+0000", 2024-09-17T21:20:31.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8739999999999999e-06 2024-09-17T21:20:31.323 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.323 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.323 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.323 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10390 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.323 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.041870+0000", 2024-09-17T21:20:31.323 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.012140860999999999, 2024-09-17T21:20:31.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015010783999999999, 2024-09-17T21:20:31.323 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.323 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.323 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.323 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.323 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.324 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.324 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10390 2024-09-17T21:20:31.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.324 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041870+0000", 2024-09-17T21:20:31.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041870+0000", 2024-09-17T21:20:31.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041870+0000", 2024-09-17T21:20:31.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1699999999999999e-07 2024-09-17T21:20:31.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041875+0000", 2024-09-17T21:20:31.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3850000000000004e-06 2024-09-17T21:20:31.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041879+0000", 2024-09-17T21:20:31.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5290000000000002e-06 2024-09-17T21:20:31.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.041883+0000", 2024-09-17T21:20:31.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6480000000000001e-06 2024-09-17T21:20:31.326 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.326 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.326 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.326 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.6084.0:10422 3.d 3.35939e8d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e127)", 2024-09-17T21:20:31.326 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:31.042482+0000", 2024-09-17T21:20:31.326 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.011528511, 2024-09-17T21:20:31.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014425383, 2024-09-17T21:20:31.327 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.327 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.327 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:31.327 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.327 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.6084", 2024-09-17T21:20:31.327 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/4206333406", 2024-09-17T21:20:31.327 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10422 2024-09-17T21:20:31.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.327 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042482+0000", 2024-09-17T21:20:31.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042482+0000", 2024-09-17T21:20:31.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042482+0000", 2024-09-17T21:20:31.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3999999999999998e-07 2024-09-17T21:20:31.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042484+0000", 2024-09-17T21:20:31.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.15e-06 2024-09-17T21:20:31.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042487+0000", 2024-09-17T21:20:31.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0359999999999998e-06 2024-09-17T21:20:31.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:31.042490+0000", 2024-09-17T21:20:31.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9160000000000001e-06 2024-09-17T21:20:31.329 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.329 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.329 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.329 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.330 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-09-17T21:20:31.330 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 80 2024-09-17T21:20:31.330 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:31.331 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:20:31.332 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:31.329+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:31.432 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:31.425+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:31.457 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:31.457 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:31.457 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:31.457 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:31.457 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:31.471 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:20:31.533 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:31.529+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:31.558 INFO:teuthology.orchestra.run.smithi184.stdout:4979 punch from 0x910600 to 0x912000, (0x1a00 bytes) 2024-09-17T21:20:31.597 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:31.597 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:31.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:31.597 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:31.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.597 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:18003 3.1d 3:b9765580:::rbd_header.11527d2b15f6:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T21:20:31.597 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:20:31.597 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.234336969000001, 2024-09-17T21:20:31.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94165346900000002, 2024-09-17T21:20:31.597 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.598 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.598 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:31.598 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.598 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:31.598 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:31.598 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18003 2024-09-17T21:20:31.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.598 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:20:31.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:20:31.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357880+0000", 2024-09-17T21:20:31.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:20:31.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357891+0000", 2024-09-17T21:20:31.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0417999999999999e-05 2024-09-17T21:20:31.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357913+0000", 2024-09-17T21:20:31.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2215e-05 2024-09-17T21:20:31.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357922+0000", 2024-09-17T21:20:31.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4779999999999997e-06 2024-09-17T21:20:31.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358023+0000", 2024-09-17T21:20:31.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101025 2024-09-17T21:20:31.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:31.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358040+0000", 2024-09-17T21:20:31.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6291999999999999e-05 2024-09-17T21:20:31.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298203+0000", 2024-09-17T21:20:31.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94016297500000001 2024-09-17T21:20:31.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:31.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298626+0000", 2024-09-17T21:20:31.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000422853 2024-09-17T21:20:31.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:31.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.299532+0000", 2024-09-17T21:20:31.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00090657300000000004 2024-09-17T21:20:31.602 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.602 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.602 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.602 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15705 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 973824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:31.602 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.520882+0000", 2024-09-17T21:20:31.602 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.071333546, 2024-09-17T21:20:31.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091691083000000007, 2024-09-17T21:20:31.603 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.603 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.603 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:31.603 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.603 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:31.603 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:31.603 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15705 2024-09-17T21:20:31.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.603 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520882+0000", 2024-09-17T21:20:31.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520882+0000", 2024-09-17T21:20:31.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520884+0000", 2024-09-17T21:20:31.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:20:31.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520889+0000", 2024-09-17T21:20:31.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:20:31.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520907+0000", 2024-09-17T21:20:31.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7750000000000001e-05 2024-09-17T21:20:31.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520920+0000", 2024-09-17T21:20:31.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2826000000000001e-05 2024-09-17T21:20:31.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612293+0000", 2024-09-17T21:20:31.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091373290999999995 2024-09-17T21:20:31.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:31.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612386+0000", 2024-09-17T21:20:31.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3261e-05 2024-09-17T21:20:31.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:31.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612573+0000", 2024-09-17T21:20:31.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000186725 2024-09-17T21:20:31.606 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.606 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.606 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.607 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15706 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 977920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:31.607 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:20:31.607 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0712351780000002, 2024-09-17T21:20:31.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091952328, 2024-09-17T21:20:31.607 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.607 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.607 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:31.607 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.607 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:31.607 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:31.607 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15706 2024-09-17T21:20:31.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.607 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:20:31.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:20:31.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520982+0000", 2024-09-17T21:20:31.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.277e-06 2024-09-17T21:20:31.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520988+0000", 2024-09-17T21:20:31.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4470000000000003e-06 2024-09-17T21:20:31.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521008+0000", 2024-09-17T21:20:31.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9309000000000001e-05 2024-09-17T21:20:31.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521023+0000", 2024-09-17T21:20:31.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5119999999999999e-05 2024-09-17T21:20:31.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.610 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612638+0000", 2024-09-17T21:20:31.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091615328999999995 2024-09-17T21:20:31.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.610 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:31.610 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612738+0000", 2024-09-17T21:20:31.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100466 2024-09-17T21:20:31.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.610 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:31.610 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612933+0000", 2024-09-17T21:20:31.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019437999999999999 2024-09-17T21:20:31.610 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.610 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.611 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.611 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15707 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 982016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:31.611 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:20:31.611 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0711717220000008, 2024-09-17T21:20:31.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092305228000000003, 2024-09-17T21:20:31.611 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.611 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.611 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:31.611 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.611 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:31.611 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:31.611 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15707 2024-09-17T21:20:31.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.612 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:20:31.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:20:31.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521045+0000", 2024-09-17T21:20:31.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4899999999999999e-06 2024-09-17T21:20:31.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521066+0000", 2024-09-17T21:20:31.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0236000000000001e-05 2024-09-17T21:20:31.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.614 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521085+0000", 2024-09-17T21:20:31.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8819999999999999e-05 2024-09-17T21:20:31.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.614 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521100+0000", 2024-09-17T21:20:31.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5146e-05 2024-09-17T21:20:31.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.614 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612978+0000", 2024-09-17T21:20:31.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091878847 2024-09-17T21:20:31.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:31.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613079+0000", 2024-09-17T21:20:31.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100503 2024-09-17T21:20:31.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:31.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613349+0000", 2024-09-17T21:20:31.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000270186 2024-09-17T21:20:31.615 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.615 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.615 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.615 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15708 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 986112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:31.615 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:20:31.616 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0710935470000003, 2024-09-17T21:20:31.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092547217000000001, 2024-09-17T21:20:31.616 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.616 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.616 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:31.616 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.616 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:31.616 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:31.616 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15708 2024-09-17T21:20:31.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.616 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:20:31.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:20:31.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521123+0000", 2024-09-17T21:20:31.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.353e-06 2024-09-17T21:20:31.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521129+0000", 2024-09-17T21:20:31.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:20:31.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521145+0000", 2024-09-17T21:20:31.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.611e-05 2024-09-17T21:20:31.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521154+0000", 2024-09-17T21:20:31.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4500000000000004e-06 2024-09-17T21:20:31.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613397+0000", 2024-09-17T21:20:31.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092243275 2024-09-17T21:20:31.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:31.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613503+0000", 2024-09-17T21:20:31.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106013 2024-09-17T21:20:31.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:31.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613669+0000", 2024-09-17T21:20:31.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016642299999999999 2024-09-17T21:20:31.619 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.619 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.620 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.620 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15709 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 990208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:31.620 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:20:31.620 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0710505309999991, 2024-09-17T21:20:31.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092728678999999994, 2024-09-17T21:20:31.620 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.620 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.620 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:31.620 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.620 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:31.620 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:31.620 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15709 2024-09-17T21:20:31.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.621 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:20:31.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:20:31.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521166+0000", 2024-09-17T21:20:31.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7000000000000004e-07 2024-09-17T21:20:31.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521169+0000", 2024-09-17T21:20:31.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2660000000000002e-06 2024-09-17T21:20:31.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521181+0000", 2024-09-17T21:20:31.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.208e-05 2024-09-17T21:20:31.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521194+0000", 2024-09-17T21:20:31.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2707e-05 2024-09-17T21:20:31.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613695+0000", 2024-09-17T21:20:31.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092500709 2024-09-17T21:20:31.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:31.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613750+0000", 2024-09-17T21:20:31.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.503e-05 2024-09-17T21:20:31.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:31.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613894+0000", 2024-09-17T21:20:31.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144117 2024-09-17T21:20:31.624 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.624 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.624 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.624 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15710 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 994304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:31.624 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:20:31.624 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0710059649999994, 2024-09-17T21:20:31.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092957729000000003, 2024-09-17T21:20:31.624 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.624 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.625 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:31.625 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.625 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:31.625 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:31.625 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15710 2024-09-17T21:20:31.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.625 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:20:31.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:20:31.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:20:31.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8000000000000005e-07 2024-09-17T21:20:31.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521216+0000", 2024-09-17T21:20:31.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3859999999999998e-06 2024-09-17T21:20:31.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521234+0000", 2024-09-17T21:20:31.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:20:31.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521248+0000", 2024-09-17T21:20:31.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:20:31.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613926+0000", 2024-09-17T21:20:31.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092678324000000006 2024-09-17T21:20:31.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:31.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613974+0000", 2024-09-17T21:20:31.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7754999999999999e-05 2024-09-17T21:20:31.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:31.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614167+0000", 2024-09-17T21:20:31.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193508 2024-09-17T21:20:31.628 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.628 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.628 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.628 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15711 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 998400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:31.628 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:20:31.629 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0709485829999998, 2024-09-17T21:20:31.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093268000000000004, 2024-09-17T21:20:31.629 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.629 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.629 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:31.629 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.629 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:31.629 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:31.629 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15711 2024-09-17T21:20:31.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.629 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:20:31.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:20:31.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521268+0000", 2024-09-17T21:20:31.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-17T21:20:31.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521273+0000", 2024-09-17T21:20:31.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:20:31.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521290+0000", 2024-09-17T21:20:31.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7269999999999999e-05 2024-09-17T21:20:31.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521304+0000", 2024-09-17T21:20:31.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3626e-05 2024-09-17T21:20:31.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614220+0000", 2024-09-17T21:20:31.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092916367 2024-09-17T21:20:31.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:31.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614322+0000", 2024-09-17T21:20:31.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102212 2024-09-17T21:20:31.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:31.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614535+0000", 2024-09-17T21:20:31.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021266199999999999 2024-09-17T21:20:31.633 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.633 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.633 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.633 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15712 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1002496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:31.633 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:20:31.633 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.070893088, 2024-09-17T21:20:31.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093568707000000001, 2024-09-17T21:20:31.634 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.634 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.634 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:31.634 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.634 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:31.634 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:31.634 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15712 2024-09-17T21:20:31.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.634 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:20:31.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:20:31.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521324+0000", 2024-09-17T21:20:31.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.17e-06 2024-09-17T21:20:31.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521334+0000", 2024-09-17T21:20:31.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9369999999999999e-06 2024-09-17T21:20:31.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521356+0000", 2024-09-17T21:20:31.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2045999999999999e-05 2024-09-17T21:20:31.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521370+0000", 2024-09-17T21:20:31.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4270000000000001e-05 2024-09-17T21:20:31.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614583+0000", 2024-09-17T21:20:31.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093213220999999999 2024-09-17T21:20:31.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:31.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614680+0000", 2024-09-17T21:20:31.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6900999999999999e-05 2024-09-17T21:20:31.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:31.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614891+0000", 2024-09-17T21:20:31.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021116200000000001 2024-09-17T21:20:31.638 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.638 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.638 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.638 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15713 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1006592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:31.638 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:20:31.638 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0708263860000002, 2024-09-17T21:20:31.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093834641999999996, 2024-09-17T21:20:31.638 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.638 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.638 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:31.638 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.638 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:31.638 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:31.639 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15713 2024-09-17T21:20:31.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.639 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:20:31.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:20:31.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521390+0000", 2024-09-17T21:20:31.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:20:31.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521396+0000", 2024-09-17T21:20:31.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:20:31.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521417+0000", 2024-09-17T21:20:31.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1236000000000001e-05 2024-09-17T21:20:31.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521431+0000", 2024-09-17T21:20:31.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3873000000000001e-05 2024-09-17T21:20:31.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614938+0000", 2024-09-17T21:20:31.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093506522999999994 2024-09-17T21:20:31.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:31.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615025+0000", 2024-09-17T21:20:31.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7635999999999999e-05 2024-09-17T21:20:31.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:31.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615224+0000", 2024-09-17T21:20:31.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019844700000000001 2024-09-17T21:20:31.642 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.642 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.642 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.642 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15714 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:31.642 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:20:31.642 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0707631299999996, 2024-09-17T21:20:31.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094071925000000001, 2024-09-17T21:20:31.642 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.643 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.643 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:31.643 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.643 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:31.643 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:31.643 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15714 2024-09-17T21:20:31.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.643 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:20:31.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:20:31.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521454+0000", 2024-09-17T21:20:31.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.097e-06 2024-09-17T21:20:31.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521459+0000", 2024-09-17T21:20:31.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2660000000000001e-06 2024-09-17T21:20:31.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521481+0000", 2024-09-17T21:20:31.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1878999999999998e-05 2024-09-17T21:20:31.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521498+0000", 2024-09-17T21:20:31.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6807e-05 2024-09-17T21:20:31.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615271+0000", 2024-09-17T21:20:31.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093773286999999997 2024-09-17T21:20:31.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:31.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615366+0000", 2024-09-17T21:20:31.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5322000000000007e-05 2024-09-17T21:20:31.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:31.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615524+0000", 2024-09-17T21:20:31.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015826699999999999 2024-09-17T21:20:31.646 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.646 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.646 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.647 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15715 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1014784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:31.647 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:20:31.647 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0706987950000002, 2024-09-17T21:20:31.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094243840999999995, 2024-09-17T21:20:31.647 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.647 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.647 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:31.647 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.647 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:31.647 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:31.647 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15715 2024-09-17T21:20:31.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.647 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:20:31.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:20:31.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521518+0000", 2024-09-17T21:20:31.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4300000000000001e-06 2024-09-17T21:20:31.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521524+0000", 2024-09-17T21:20:31.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5860000000000004e-06 2024-09-17T21:20:31.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521541+0000", 2024-09-17T21:20:31.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7574000000000001e-05 2024-09-17T21:20:31.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521555+0000", 2024-09-17T21:20:31.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3540000000000001e-05 2024-09-17T21:20:31.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615560+0000", 2024-09-17T21:20:31.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.09400472 2024-09-17T21:20:31.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:31.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615614+0000", 2024-09-17T21:20:31.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4693e-05 2024-09-17T21:20:31.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:31.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615761+0000", 2024-09-17T21:20:31.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000146298 2024-09-17T21:20:31.650 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.651 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.651 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.651 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15716 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1018880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:31.651 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:20:31.651 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0706424499999994, 2024-09-17T21:20:31.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094400676000000003, 2024-09-17T21:20:31.651 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.651 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.651 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:31.651 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.651 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:31.651 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:31.651 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15716 2024-09-17T21:20:31.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.652 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:20:31.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:20:31.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521574+0000", 2024-09-17T21:20:31.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1939999999999999e-06 2024-09-17T21:20:31.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521579+0000", 2024-09-17T21:20:31.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:20:31.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521597+0000", 2024-09-17T21:20:31.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8074e-05 2024-09-17T21:20:31.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521613+0000", 2024-09-17T21:20:31.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5316e-05 2024-09-17T21:20:31.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615780+0000", 2024-09-17T21:20:31.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094166874999999997 2024-09-17T21:20:31.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:31.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615836+0000", 2024-09-17T21:20:31.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6926999999999999e-05 2024-09-17T21:20:31.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:31.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615974+0000", 2024-09-17T21:20:31.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013741400000000001 2024-09-17T21:20:31.655 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.655 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.655 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.655 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15717 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1022976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:31.655 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:20:31.655 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0705837539999994, 2024-09-17T21:20:31.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094586723999999997, 2024-09-17T21:20:31.655 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.655 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.655 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:31.655 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.656 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:31.656 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:31.656 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15717 2024-09-17T21:20:31.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.656 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:20:31.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:20:31.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521633+0000", 2024-09-17T21:20:31.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:20:31.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521638+0000", 2024-09-17T21:20:31.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:20:31.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521659+0000", 2024-09-17T21:20:31.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-05 2024-09-17T21:20:31.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521675+0000", 2024-09-17T21:20:31.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5688999999999998e-05 2024-09-17T21:20:31.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616008+0000", 2024-09-17T21:20:31.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094333314000000001 2024-09-17T21:20:31.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:31.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616066+0000", 2024-09-17T21:20:31.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7797000000000002e-05 2024-09-17T21:20:31.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:31.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616219+0000", 2024-09-17T21:20:31.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015281100000000001 2024-09-17T21:20:31.659 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.659 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.659 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.659 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15718 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:31.659 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:20:31.659 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0704836790000005, 2024-09-17T21:20:31.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094703674000000002, 2024-09-17T21:20:31.659 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.659 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.660 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:31.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:31.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:31.660 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15718 2024-09-17T21:20:31.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.660 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:20:31.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:20:31.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521734+0000", 2024-09-17T21:20:31.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.61e-06 2024-09-17T21:20:31.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521740+0000", 2024-09-17T21:20:31.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:20:31.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521754+0000", 2024-09-17T21:20:31.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3793e-05 2024-09-17T21:20:31.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521763+0000", 2024-09-17T21:20:31.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5129999999999996e-06 2024-09-17T21:20:31.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616256+0000", 2024-09-17T21:20:31.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094492919999999994 2024-09-17T21:20:31.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:31.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616312+0000", 2024-09-17T21:20:31.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6737000000000002e-05 2024-09-17T21:20:31.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:31.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616436+0000", 2024-09-17T21:20:31.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012335800000000001 2024-09-17T21:20:31.663 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.663 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.663 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.664 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15719 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1031168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:31.664 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:20:31.664 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0704302739999996, 2024-09-17T21:20:31.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094888963000000007, 2024-09-17T21:20:31.664 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.664 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.664 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:31.664 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.664 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:31.664 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:31.664 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15719 2024-09-17T21:20:31.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.665 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:20:31.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:20:31.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521786+0000", 2024-09-17T21:20:31.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.074e-06 2024-09-17T21:20:31.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521791+0000", 2024-09-17T21:20:31.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3030000000000001e-06 2024-09-17T21:20:31.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521804+0000", 2024-09-17T21:20:31.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3027000000000001e-05 2024-09-17T21:20:31.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521811+0000", 2024-09-17T21:20:31.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6090000000000003e-06 2024-09-17T21:20:31.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616466+0000", 2024-09-17T21:20:31.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094654294 2024-09-17T21:20:31.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:31.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616523+0000", 2024-09-17T21:20:31.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6953999999999997e-05 2024-09-17T21:20:31.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:31.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616674+0000", 2024-09-17T21:20:31.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015170200000000001 2024-09-17T21:20:31.668 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.668 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.668 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.669 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15720 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1035264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:31.669 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:20:31.669 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0703934339999996, 2024-09-17T21:20:31.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095067061999999994, 2024-09-17T21:20:31.669 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.669 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.669 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:31.669 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.669 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:31.669 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:31.669 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15720 2024-09-17T21:20:31.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.670 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:20:31.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:20:31.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521823+0000", 2024-09-17T21:20:31.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:20:31.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521826+0000", 2024-09-17T21:20:31.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2100000000000002e-06 2024-09-17T21:20:31.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521835+0000", 2024-09-17T21:20:31.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1630000000000002e-06 2024-09-17T21:20:31.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521842+0000", 2024-09-17T21:20:31.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:20:31.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616711+0000", 2024-09-17T21:20:31.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094868859 2024-09-17T21:20:31.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:31.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616764+0000", 2024-09-17T21:20:31.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2778999999999997e-05 2024-09-17T21:20:31.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:31.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616889+0000", 2024-09-17T21:20:31.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012533100000000001 2024-09-17T21:20:31.673 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.673 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.673 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.673 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15721 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1039360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:31.673 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:20:31.673 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0703463249999992, 2024-09-17T21:20:31.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095277219999999996, 2024-09-17T21:20:31.674 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.674 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.674 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:31.674 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.674 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:31.674 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:31.674 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15721 2024-09-17T21:20:31.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.674 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:20:31.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:20:31.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521871+0000", 2024-09-17T21:20:31.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.443e-06 2024-09-17T21:20:31.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521876+0000", 2024-09-17T21:20:31.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0300000000000001e-06 2024-09-17T21:20:31.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521886+0000", 2024-09-17T21:20:31.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186e-05 2024-09-17T21:20:31.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521893+0000", 2024-09-17T21:20:31.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.08e-06 2024-09-17T21:20:31.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616918+0000", 2024-09-17T21:20:31.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095025223000000006 2024-09-17T21:20:31.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:31.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616977+0000", 2024-09-17T21:20:31.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8403999999999999e-05 2024-09-17T21:20:31.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:31.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617147+0000", 2024-09-17T21:20:31.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016985399999999999 2024-09-17T21:20:31.677 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.677 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.677 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.678 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15722 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:31.678 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:20:31.678 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0703118759999999, 2024-09-17T21:20:31.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095459783000000006, 2024-09-17T21:20:31.678 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.678 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.678 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:31.678 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.678 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:31.678 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:31.678 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15722 2024-09-17T21:20:31.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.679 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:20:31.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:20:31.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:20:31.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7700000000000004e-07 2024-09-17T21:20:31.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521907+0000", 2024-09-17T21:20:31.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.813e-06 2024-09-17T21:20:31.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521922+0000", 2024-09-17T21:20:31.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4533e-05 2024-09-17T21:20:31.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521934+0000", 2024-09-17T21:20:31.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2469999999999999e-05 2024-09-17T21:20:31.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617187+0000", 2024-09-17T21:20:31.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095252311000000006 2024-09-17T21:20:31.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:31.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617240+0000", 2024-09-17T21:20:31.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3740999999999999e-05 2024-09-17T21:20:31.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:31.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617364+0000", 2024-09-17T21:20:31.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-17T21:20:31.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.682 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.683 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15723 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:31.683 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:20:31.683 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0702680099999995, 2024-09-17T21:20:31.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095653563999999996, 2024-09-17T21:20:31.683 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:31.683 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:31.683 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:31.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:31.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:31.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:31.683 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15723 2024-09-17T21:20:31.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.683 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:31.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:31.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:20:31.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:31.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:20:31.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:31.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:31.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521949+0000", 2024-09-17T21:20:31.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:20:31.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:31.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521953+0000", 2024-09-17T21:20:31.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5870000000000003e-06 2024-09-17T21:20:31.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:31.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521970+0000", 2024-09-17T21:20:31.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7282000000000002e-05 2024-09-17T21:20:31.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:31.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521984+0000", 2024-09-17T21:20:31.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:20:31.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:31.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617383+0000", 2024-09-17T21:20:31.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095399063000000006 2024-09-17T21:20:31.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:31.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617437+0000", 2024-09-17T21:20:31.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3931999999999998e-05 2024-09-17T21:20:31.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:31.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:31.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:31.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617601+0000", 2024-09-17T21:20:31.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016407300000000001 2024-09-17T21:20:31.686 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.686 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.686 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.686 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:31.687 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:31.687 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:31.687 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 0] out_osds: [2, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:20:31.687 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:20:31.687 INFO:tasks.thrashosds.thrasher:Adding osd 5 2024-09-17T21:20:31.688 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 5 2024-09-17T21:20:31.689 INFO:teuthology.orchestra.run.smithi184.stdout:4980 write 0x8259600 thru 0x825c431 (0x2e32 bytes) 2024-09-17T21:20:31.689 INFO:teuthology.orchestra.run.smithi184.stdout:4981 punch from 0x5ff7600 to 0x5ffc000, (0x4a00 bytes) 2024-09-17T21:20:31.690 INFO:teuthology.orchestra.run.smithi184.stdout:4982 write 0x30c0000 thru 0x30c30a6 (0x30a7 bytes) 2024-09-17T21:20:31.690 INFO:teuthology.orchestra.run.smithi184.stdout:4983 read 0x18e9400 thru 0x18e9697 (0x298 bytes) 2024-09-17T21:20:31.690 INFO:teuthology.orchestra.run.smithi184.stdout:4985 trunc from 0x825c600 to 0x2384a00 2024-09-17T21:20:31.690 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:20:31.690 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:31.682+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:31.773 INFO:teuthology.orchestra.run.smithi184.stdout:4991 read 0x2305e00 thru 0x23150a2 (0xf2a3 bytes) 2024-09-17T21:20:31.776 INFO:teuthology.orchestra.run.smithi184.stdout:4992 read 0x1747400 thru 0x1752842 (0xb443 bytes) 2024-09-17T21:20:31.777 INFO:teuthology.orchestra.run.smithi184.stdout:4993 punch from 0xf61a00 to 0xf6f400, (0xda00 bytes) 2024-09-17T21:20:31.778 INFO:teuthology.orchestra.run.smithi184.stdout:4996 read 0x719600 thru 0x725b59 (0xc55a bytes) 2024-09-17T21:20:31.779 INFO:teuthology.orchestra.run.smithi184.stdout:4997 write 0xaeaa00 thru 0xaecd35 (0x2336 bytes) 2024-09-17T21:20:31.781 INFO:teuthology.orchestra.run.smithi184.stdout:4999 read 0x626200 thru 0x631da1 (0xbba2 bytes) 2024-09-17T21:20:31.781 INFO:teuthology.orchestra.run.smithi184.stdout:5000 read 0x14ffe00 thru 0x15039b6 (0x3bb7 bytes) 2024-09-17T21:20:31.782 INFO:teuthology.orchestra.run.smithi184.stdout:5002 read 0x184a800 thru 0x18537e2 (0x8fe3 bytes) 2024-09-17T21:20:31.783 INFO:teuthology.orchestra.run.smithi184.stdout:5004 read 0x100d400 thru 0x10146fb (0x72fc bytes) 2024-09-17T21:20:31.789 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:31.782+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:31.808 INFO:teuthology.orchestra.run.smithi184.stdout:5006 write 0x3eede00 thru 0x3ef337b (0x557c bytes) 2024-09-17T21:20:31.811 INFO:teuthology.orchestra.run.smithi184.stdout:5007 read 0x1edac00 thru 0x1ee6890 (0xbc91 bytes) 2024-09-17T21:20:31.812 INFO:teuthology.orchestra.run.smithi184.stdout:5008 punch from 0x1d38800 to 0x1d43a00, (0xb200 bytes) 2024-09-17T21:20:31.821 INFO:teuthology.orchestra.run.smithi184.stdout:5009 trunc from 0x3ef3400 to 0xaf9200 2024-09-17T21:20:31.875 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:31.875 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:31.875 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:20:31.875 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:31.889 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:20:31.890 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:31.882+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:31.921 INFO:teuthology.orchestra.run.smithi184.stdout:5011 write 0xd472600 thru 0xd475aa2 (0x34a3 bytes) 2024-09-17T21:20:31.923 INFO:teuthology.orchestra.run.smithi184.stdout:5012 read 0x806f800 thru 0x80757be (0x5fbf bytes) 2024-09-17T21:20:31.925 INFO:teuthology.orchestra.run.smithi184.stdout:5013 trunc from 0xd475c00 to 0xcc8ee00 2024-09-17T21:20:31.952 INFO:teuthology.orchestra.run.smithi184.stdout:5015 write 0xab2a200 thru 0xab394f5 (0xf2f6 bytes) 2024-09-17T21:20:31.955 INFO:teuthology.orchestra.run.smithi184.stdout:5019 punch from 0x926ca00 to 0x9271e00, (0x5400 bytes) 2024-09-17T21:20:31.955 INFO:teuthology.orchestra.run.smithi184.stdout:5020 write 0x1410a00 thru 0x14159e8 (0x4fe9 bytes) 2024-09-17T21:20:31.956 INFO:teuthology.orchestra.run.smithi184.stdout:5022 read 0x9920e00 thru 0x992581e (0x4a1f bytes) 2024-09-17T21:20:31.960 INFO:teuthology.orchestra.run.smithi184.stdout:5023 punch from 0xbe09400 to 0xbe13800, (0xa400 bytes) 2024-09-17T21:20:31.961 INFO:teuthology.orchestra.run.smithi184.stdout:5025 read 0x2225c00 thru 0x223592c (0xfd2d bytes) 2024-09-17T21:20:31.970 INFO:teuthology.orchestra.run.smithi184.stdout:5030 trunc from 0xcc8ee00 to 0x9642a00 2024-09-17T21:20:31.990 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:31.982+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:32.006 INFO:teuthology.orchestra.run.smithi184.stdout:5031 punch from 0x514e00 to 0x51fc00, (0xae00 bytes) 2024-09-17T21:20:32.013 INFO:teuthology.orchestra.run.smithi184.stdout:5033 trunc from 0x9642a00 to 0xac3be00 2024-09-17T21:20:32.045 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-09-17T21:20:32.041+0000 7fe37768f640 -1 mon.a@0(leader).osd e127 definitely_dead 0 2024-09-17T21:20:32.070 INFO:teuthology.orchestra.run.smithi184.stdout:5034 write 0xe823000 thru 0xe82ad25 (0x7d26 bytes) 2024-09-17T21:20:32.070 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:32.070 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:32.070 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:32.070 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:32.070 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:32.072 INFO:teuthology.orchestra.run.smithi184.stdout:5036 read 0x41d6000 thru 0x41d96f7 (0x36f8 bytes) 2024-09-17T21:20:32.085 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:20:32.090 INFO:teuthology.orchestra.run.smithi184.stdout:5038 trunc from 0xe82ae00 to 0x7b70400 2024-09-17T21:20:32.103 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:20:32.127 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:32.122+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:32.163 INFO:teuthology.orchestra.run.smithi184.stdout:5039 write 0xdebda00 thru 0xdec060a (0x2c0b bytes) 2024-09-17T21:20:32.165 INFO:teuthology.orchestra.run.smithi184.stdout:5040 write 0x2609c00 thru 0x2618800 (0xec01 bytes) 2024-09-17T21:20:32.165 INFO:teuthology.orchestra.run.smithi184.stdout:5049 write 0x3c17200 thru 0x3c1e0d8 (0x6ed9 bytes) 2024-09-17T21:20:32.167 INFO:teuthology.orchestra.run.smithi184.stdout:5050 write 0x6352800 thru 0x6359247 (0x6a48 bytes) 2024-09-17T21:20:32.179 INFO:teuthology.orchestra.run.smithi184.stdout:5052 trunc from 0xdec0800 to 0x9082800 2024-09-17T21:20:32.191 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:32.186+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:32.223 INFO:teuthology.orchestra.run.smithi184.stdout:5053 punch from 0x306d600 to 0x3078800, (0xb200 bytes) 2024-09-17T21:20:32.225 INFO:teuthology.orchestra.run.smithi184.stdout:5054 read 0x325fc00 thru 0x3269136 (0x9537 bytes) 2024-09-17T21:20:32.227 INFO:teuthology.orchestra.run.smithi184.stdout:5055 trunc from 0x9082800 to 0x98d2e00 2024-09-17T21:20:32.247 INFO:teuthology.orchestra.run.smithi184.stdout:5058 read 0x22b0800 thru 0x22bbf12 (0xb713 bytes) 2024-09-17T21:20:32.250 INFO:teuthology.orchestra.run.smithi184.stdout:5060 read 0x524ba00 thru 0x5255230 (0x9831 bytes) 2024-09-17T21:20:32.257 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:32.257 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:32.269 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:32.269 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:32.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:32.269 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:32.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.269 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:20:32.269 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:32.270 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 130.662113269, 2024-09-17T21:20:32.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:20:32.270 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:32.270 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:32.270 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:32.270 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:32.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:32.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:32.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:32.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:32.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:32.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:20:32.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:20:32.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:32.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:20:32.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:20:32.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:32.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:20:32.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:20:32.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:32.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:20:32.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:20:32.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:32.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:20:32.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:20:32.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:32.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:20:32.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:20:32.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:32.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:20:32.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:20:32.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:32.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:20:32.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:20:32.274 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.274 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:32.274 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.274 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:20:32.274 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:32.274 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 130.65216663300001, 2024-09-17T21:20:32.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:20:32.275 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:32.275 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:32.275 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:32.275 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:32.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:32.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:32.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:32.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:32.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:32.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:20:32.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:20:32.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:32.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:20:32.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:20:32.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:32.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:20:32.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:20:32.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:32.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:20:32.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:20:32.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:32.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:20:32.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:20:32.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:32.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:20:32.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:20:32.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:32.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:20:32.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:20:32.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:32.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:20:32.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:20:32.278 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.278 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:32.278 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.279 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:20:32.279 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:32.279 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 129.40901579300001, 2024-09-17T21:20:32.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:20:32.279 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:32.279 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:32.279 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:32.279 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:32.279 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:20:32.279 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:20:32.279 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:20:32.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.280 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:32.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:32.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:32.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:32.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:32.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:32.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:20:32.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:20:32.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:32.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:20:32.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:20:32.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:32.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:20:32.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:20:32.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:32.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:20:32.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:20:32.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:32.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:20:32.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:20:32.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:32.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:20:32.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:20:32.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:32.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:20:32.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:20:32.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:32.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:20:32.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:20:32.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:32.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:20:32.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:20:32.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:32.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:20:32.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:20:32.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:32.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:20:32.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:20:32.284 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.285 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:32.285 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.285 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:20:32.285 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:32.285 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 129.399370307, 2024-09-17T21:20:32.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:20:32.285 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:32.285 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:32.285 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:32.285 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:32.285 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:20:32.286 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:20:32.286 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:20:32.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.286 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:32.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:32.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:32.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:32.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:32.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:32.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:20:32.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:20:32.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:32.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:20:32.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:20:32.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:32.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:20:32.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:20:32.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:32.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:20:32.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:20:32.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:32.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:20:32.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:20:32.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:32.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:20:32.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:20:32.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:32.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:20:32.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:20:32.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:32.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:20:32.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:20:32.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:32.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:20:32.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:20:32.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:32.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:20:32.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:20:32.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:32.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:20:32.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:20:32.290 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.291 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:32.291 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.291 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:20:32.291 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:32.291 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 116.030434274, 2024-09-17T21:20:32.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:20:32.291 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:32.291 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:32.291 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:32.291 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:32.292 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:20:32.292 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:20:32.292 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:20:32.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.292 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:32.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:32.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:32.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:32.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:32.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:32.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:20:32.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:20:32.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:32.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:20:32.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:20:32.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:32.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:20:32.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:20:32.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:32.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:20:32.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:20:32.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:32.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:20:32.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:20:32.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:32.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:20:32.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:20:32.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:32.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:20:32.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:20:32.295 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.295 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:32.295 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.295 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:819 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5111808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:32.296 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:32.296 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 97.828861984, 2024-09-17T21:20:32.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024344811000000001, 2024-09-17T21:20:32.296 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:32.296 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:32.296 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:32.296 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:32.296 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:32.296 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:32.296 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 819 2024-09-17T21:20:32.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.296 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:32.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:32.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:32.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:32.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:32.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:32.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437443+0000", 2024-09-17T21:20:32.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1969999999999999e-06 2024-09-17T21:20:32.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:32.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437448+0000", 2024-09-17T21:20:32.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9490000000000004e-06 2024-09-17T21:20:32.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:32.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437467+0000", 2024-09-17T21:20:32.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8862999999999999e-05 2024-09-17T21:20:32.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:32.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437480+0000", 2024-09-17T21:20:32.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2833000000000001e-05 2024-09-17T21:20:32.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:32.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461552+0000", 2024-09-17T21:20:32.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024072695000000002 2024-09-17T21:20:32.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:32.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461639+0000", 2024-09-17T21:20:32.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6996000000000005e-05 2024-09-17T21:20:32.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:32.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461787+0000", 2024-09-17T21:20:32.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014727800000000001 2024-09-17T21:20:32.300 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.300 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:32.300 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.300 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:32.300 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:32.300 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 97.828749712000004, 2024-09-17T21:20:32.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:20:32.300 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:32.300 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:32.301 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:32.301 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:32.301 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:32.301 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:32.301 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:20:32.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.301 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:32.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:32.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:32.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:32.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:32.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:32.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:20:32.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:20:32.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:32.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:20:32.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:20:32.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:32.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:20:32.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:20:32.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:32.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:20:32.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:20:32.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:32.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:20:32.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:20:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:20:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:20:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:20:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:20:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.305 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:833 3.5 3:a772dfb1:::rbd_data.11523651b015.0000000000000001:d [sparse-read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:32.305 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:32.305 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 97.828315426000003, 2024-09-17T21:20:32.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024216024999999999, 2024-09-17T21:20:32.305 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:32.305 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:32.305 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:32.305 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:32.305 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:32.305 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:32.305 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 833 2024-09-17T21:20:32.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.305 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:32.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:32.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:32.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:32.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:32.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:32.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437990+0000", 2024-09-17T21:20:32.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2500000000000001e-06 2024-09-17T21:20:32.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:32.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437995+0000", 2024-09-17T21:20:32.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1240000000000004e-06 2024-09-17T21:20:32.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:32.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438012+0000", 2024-09-17T21:20:32.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7572e-05 2024-09-17T21:20:32.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:32.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438031+0000", 2024-09-17T21:20:32.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8612000000000001e-05 2024-09-17T21:20:32.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:32.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461830+0000", 2024-09-17T21:20:32.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023799135999999999 2024-09-17T21:20:32.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:32.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462005+0000", 2024-09-17T21:20:32.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000174747 2024-09-17T21:20:32.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:32.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:20:32.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019958399999999999 2024-09-17T21:20:32.309 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.309 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:32.309 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.309 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:32.309 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:32.309 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 97.828255135000006, 2024-09-17T21:20:32.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:20:32.309 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:32.309 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:32.309 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:32.310 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:32.310 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:32.310 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:32.310 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:20:32.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.310 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:32.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:32.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:32.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:32.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:32.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:32.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:20:32.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:20:32.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:32.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:20:32.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:20:32.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:32.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:20:32.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:20:32.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:32.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:20:32.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:20:32.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:32.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:20:32.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:20:32.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:32.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:20:32.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:20:32.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:32.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:20:32.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:20:32.313 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.313 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:32.313 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.313 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:32.314 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:32.314 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 97.828199753999996, 2024-09-17T21:20:32.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:20:32.314 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:32.314 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:32.314 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:32.314 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:32.314 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:32.314 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:32.314 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:20:32.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.314 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:32.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:32.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:32.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:32.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:32.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:32.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:20:32.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:20:32.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:32.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:20:32.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:20:32.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:32.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:20:32.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:20:32.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:32.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:20:32.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:20:32.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:32.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:20:32.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:20:32.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:32.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:20:32.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:20:32.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:32.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:20:32.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:20:32.318 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.318 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:32.318 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.318 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:32.318 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:32.318 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 97.828143475999994, 2024-09-17T21:20:32.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:20:32.319 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:32.319 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:32.319 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:32.319 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:32.319 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:32.319 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:32.319 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:20:32.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.319 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:32.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:32.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:32.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:32.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:32.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:32.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:20:32.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:20:32.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:32.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:20:32.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:20:32.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:32.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:20:32.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:20:32.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:32.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:20:32.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:20:32.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:32.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:20:32.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:20:32.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:32.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:20:32.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:20:32.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:32.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:20:32.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:20:32.323 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.323 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:32.323 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.323 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:32.323 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:32.323 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 97.827193836999996, 2024-09-17T21:20:32.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:20:32.323 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:32.323 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:32.323 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:32.323 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:32.323 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:32.324 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:32.324 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:20:32.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.324 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:32.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:32.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:32.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:32.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:32.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:32.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:20:32.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:20:32.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:32.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:20:32.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:20:32.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:32.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:20:32.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:20:32.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:32.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:20:32.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:20:32.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:32.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:20:32.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:20:32.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:32.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:20:32.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:20:32.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:32.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:20:32.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:20:32.327 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.327 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:32.327 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.328 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:32.328 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:32.328 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 97.827115216999999, 2024-09-17T21:20:32.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:20:32.328 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:32.328 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:32.328 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:32.328 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:32.328 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:32.328 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:32.328 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:20:32.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.328 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:32.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:32.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:32.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:32.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:32.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:32.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:20:32.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:20:32.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:32.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:20:32.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:20:32.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:32.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:20:32.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:20:32.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:32.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:20:32.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:20:32.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:32.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:20:32.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:20:32.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:32.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:20:32.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:20:32.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:32.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:20:32.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:20:32.332 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.332 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:32.332 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.332 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:32.332 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:32.332 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 97.827002277999995, 2024-09-17T21:20:32.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:20:32.332 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:32.332 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:32.333 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:32.333 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:32.333 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:32.333 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:32.333 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:20:32.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.333 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:32.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:32.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:32.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:32.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:32.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:32.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:20:32.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:20:32.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:32.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:20:32.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:20:32.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:32.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:20:32.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:20:32.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:32.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:20:32.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:20:32.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:32.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:20:32.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:20:32.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:32.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:20:32.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:20:32.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:32.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:20:32.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:20:32.336 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.336 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:32.337 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.337 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:849 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5963776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:32.340 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:32.340 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 97.826408241999999, 2024-09-17T21:20:32.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024537514, 2024-09-17T21:20:32.340 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:32.340 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:32.340 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:32.340 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:32.340 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:32.340 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:32.340 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 849 2024-09-17T21:20:32.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.340 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:32.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:32.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:32.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:32.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:32.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:32.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439897+0000", 2024-09-17T21:20:32.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9099999999999999e-06 2024-09-17T21:20:32.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:32.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439905+0000", 2024-09-17T21:20:32.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0870000000000007e-06 2024-09-17T21:20:32.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:32.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439924+0000", 2024-09-17T21:20:32.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9042000000000002e-05 2024-09-17T21:20:32.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:32.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439941+0000", 2024-09-17T21:20:32.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.624e-05 2024-09-17T21:20:32.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:32.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464132+0000", 2024-09-17T21:20:32.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024191579000000001 2024-09-17T21:20:32.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:32.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464291+0000", 2024-09-17T21:20:32.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015904800000000001 2024-09-17T21:20:32.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:32.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464433+0000", 2024-09-17T21:20:32.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014160799999999999 2024-09-17T21:20:32.344 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.344 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:32.344 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.344 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:32.344 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:32.344 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 97.826341224000004, 2024-09-17T21:20:32.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:20:32.344 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:32.344 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:32.345 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:32.345 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:32.345 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:32.345 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:32.345 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:20:32.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.345 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:32.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:32.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:32.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:32.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:32.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:32.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:20:32.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:20:32.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:32.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:20:32.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:20:32.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:32.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:20:32.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:20:32.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:32.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:20:32.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:20:32.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:32.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:20:32.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:20:32.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:32.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:20:32.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:20:32.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:32.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:20:32.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:20:32.348 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.348 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:32.348 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.349 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:32.349 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:32.349 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 97.826276413000002, 2024-09-17T21:20:32.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:20:32.349 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:32.349 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:32.349 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:32.349 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:32.349 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:32.350 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:32.350 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:20:32.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.350 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:32.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:32.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:32.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:32.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:32.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:32.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:20:32.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:20:32.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:32.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:20:32.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:20:32.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:32.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:20:32.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:20:32.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:32.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:20:32.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:20:32.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:32.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:20:32.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:20:32.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:32.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:20:32.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:20:32.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:32.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:20:32.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:20:32.353 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.353 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:32.353 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.353 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:32.353 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:32.353 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 97.826215435999998, 2024-09-17T21:20:32.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:20:32.354 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:32.354 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:32.354 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:32.354 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:32.354 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:32.354 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:32.354 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:20:32.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.354 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:32.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:32.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:32.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:32.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:32.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:32.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:20:32.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:20:32.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:32.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:20:32.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:20:32.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:32.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:20:32.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:20:32.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:32.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:20:32.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:20:32.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:32.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:20:32.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:20:32.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:32.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:20:32.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:20:32.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:32.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:20:32.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:20:32.357 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.357 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:32.357 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.357 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:20:32.358 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:32.358 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 42.928644251000001, 2024-09-17T21:20:32.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:20:32.358 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:32.358 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:32.358 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:32.358 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:32.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:32.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:32.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:32.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:32.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:32.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:20:32.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:20:32.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:32.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:20:32.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:20:32.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:32.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:20:32.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:20:32.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:32.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:20:32.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:20:32.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:32.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:20:32.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:20:32.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:32.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:20:32.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:20:32.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:32.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:20:32.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:20:32.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:32.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:20:32.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:20:32.362 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.362 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:32.362 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.362 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:20:32.362 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:32.362 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 42.719985856999998, 2024-09-17T21:20:32.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:20:32.362 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:32.363 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:32.363 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:32.363 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:32.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:32.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:32.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:32.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:32.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:32.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:32.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:20:32.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:20:32.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:32.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:20:32.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:20:32.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:32.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:20:32.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:20:32.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:32.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:20:32.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:20:32.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:32.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:20:32.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:20:32.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:32.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:20:32.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:20:32.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:32.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:20:32.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:20:32.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:32.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:32.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:32.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:20:32.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:20:32.366 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.366 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:32.366 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.366 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:32.366 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:32.366 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:32.367 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:32.368 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:32.369 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:32.369 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:32.370 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:32.362+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:32.370 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:20:32.371 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:32.371 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:32.372 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:32.372 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:32.373 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:32.373 INFO:teuthology.orchestra.run.smithi184.stdout:5062 write 0xb50be00 thru 0xb5148d3 (0x8ad4 bytes) 2024-09-17T21:20:32.373 INFO:teuthology.orchestra.run.smithi184.stdout:5064 punch from 0x9571200 to 0x9572600, (0x1400 bytes) 2024-09-17T21:20:32.373 INFO:teuthology.orchestra.run.smithi184.stdout:5065 write 0x2b02400 thru 0x2b02a3c (0x63d bytes) 2024-09-17T21:20:32.373 INFO:teuthology.orchestra.run.smithi184.stdout:5066 write 0xad89200 thru 0xad918bb (0x86bc bytes) 2024-09-17T21:20:32.373 INFO:teuthology.orchestra.run.smithi184.stdout:5068 read 0xb67000 thru 0xb75f76 (0xef77 bytes) 2024-09-17T21:20:32.373 INFO:teuthology.orchestra.run.smithi184.stdout:5069 write 0xa201600 thru 0xa203ab3 (0x24b4 bytes) 2024-09-17T21:20:32.373 INFO:teuthology.orchestra.run.smithi184.stdout:5076 punch from 0x1ff9400 to 0x2005200, (0xbe00 bytes) 2024-09-17T21:20:32.373 INFO:teuthology.orchestra.run.smithi184.stdout:5078 trunc from 0xb514a00 to 0xc1d0000 2024-09-17T21:20:32.374 INFO:teuthology.orchestra.run.smithi184.stdout:5080 write 0x4d75c00 thru 0x4d76236 (0x637 bytes) 2024-09-17T21:20:32.374 INFO:teuthology.orchestra.run.smithi184.stdout:5081 punch from 0xa469400 to 0xa475000, (0xbc00 bytes) 2024-09-17T21:20:32.374 INFO:teuthology.orchestra.run.smithi184.stdout:5082 punch from 0x5c21600 to 0x5c30800, (0xf200 bytes) 2024-09-17T21:20:32.374 INFO:teuthology.orchestra.run.smithi184.stdout:5083 read 0x1d64400 thru 0x1d68fc5 (0x4bc6 bytes) 2024-09-17T21:20:32.374 INFO:teuthology.orchestra.run.smithi184.stdout:5085 trunc from 0xc1d0000 to 0x6ec5a00 2024-09-17T21:20:32.374 INFO:teuthology.orchestra.run.smithi184.stdout:5087 write 0x3159400 thru 0x3159421 (0x22 bytes) 2024-09-17T21:20:32.374 INFO:teuthology.orchestra.run.smithi184.stdout:5089 punch from 0x418ce00 to 0x4193200, (0x6400 bytes) 2024-09-17T21:20:32.374 INFO:teuthology.orchestra.run.smithi184.stdout:5090 write 0x5c7e000 thru 0x5c82d84 (0x4d85 bytes) 2024-09-17T21:20:32.374 INFO:teuthology.orchestra.run.smithi184.stdout:5091 punch from 0x11c2c00 to 0x11c4800, (0x1c00 bytes) 2024-09-17T21:20:32.374 INFO:teuthology.orchestra.run.smithi184.stdout:5092 read 0x805200 thru 0x80d4ab (0x82ac bytes) 2024-09-17T21:20:32.375 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:32.380 INFO:teuthology.orchestra.run.smithi184.stdout:5094 write 0x7c13600 thru 0x7c178f8 (0x42f9 bytes) 2024-09-17T21:20:32.382 INFO:teuthology.orchestra.run.smithi184.stdout:5095 trunc from 0x7c17a00 to 0x7d11600 2024-09-17T21:20:32.385 INFO:teuthology.orchestra.run.smithi083.stderr:marked in osd.5. 2024-09-17T21:20:32.410 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:20:32.411 INFO:tasks.thrashosds.thrasher:Added osd 5 2024-09-17T21:20:32.503 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:32.503 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:32.504 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:32.497+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:32.518 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:20:32.569 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:32.562+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:32.670 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:32.665+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:32.681 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:32.681 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:32.695 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:20:32.771 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:32.762+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:32.864 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:32.865 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:32.871 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:32.866+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:32.878 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:20:32.972 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:32.970+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:33.001 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:33.001 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:33.015 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:20:33.073 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:33.069+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:33.137 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:33.137 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:33.150 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:20:33.174 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:33.166+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:33.275 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:33.266+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:33.318 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:33.318 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:33.333 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:20:33.377 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:33.373+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-17T21:20:33.465 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:33.465 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:33.476 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:33.473+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:33.479 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:20:33.578 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:33.573+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:33.637 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:33.637 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:33.651 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:20:33.679 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:33.674+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:33.780 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:33.773+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:33.832 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:33.833 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:33.847 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:20:33.889 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:33.885+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:33.977 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:33.977 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:33.980 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:33.973+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:33.991 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:20:34.081 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:34.074+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:34.108 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:34.108 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:34.121 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:20:34.182 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:34.177+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:34.283 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:34.278+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:34.284 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:34.284 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:34.298 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:20:34.376 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:20:34.387 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:34.382+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:34.472 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:34.472 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:34.484 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:34.481+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:34.485 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:20:34.530 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:34.530 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:34.546 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:34.547 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:34.563 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:34.563 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:34.581 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:34.581 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:34.585 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:34.578+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:34.600 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:34.600 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:34.620 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:34.620 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:34.649 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:34.650 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:34.650 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:34.650 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:34.650 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:34.664 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:20:34.686 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:34.681+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:34.787 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:34.785+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:34.827 INFO:teuthology.orchestra.run.smithi184.stdout:5096 read 0x2ceb800 thru 0x2cec84b (0x104c bytes) 2024-09-17T21:20:34.835 INFO:teuthology.orchestra.run.smithi184.stdout:5097 trunc from 0x7d11600 to 0x651c400 2024-09-17T21:20:34.858 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:34.858 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:34.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:34.858 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:34.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.859 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:18003 3.1d 3:b9765580:::rbd_header.11527d2b15f6:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T21:20:34.859 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:20:34.859 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.497291066999999, 2024-09-17T21:20:34.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94165346900000002, 2024-09-17T21:20:34.859 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:34.859 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:34.859 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:34.859 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:34.859 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:34.859 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:34.859 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18003 2024-09-17T21:20:34.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.860 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:34.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:34.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:20:34.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:34.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:20:34.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:34.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357880+0000", 2024-09-17T21:20:34.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:20:34.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:34.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357891+0000", 2024-09-17T21:20:34.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0417999999999999e-05 2024-09-17T21:20:34.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:34.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357913+0000", 2024-09-17T21:20:34.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2215e-05 2024-09-17T21:20:34.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:34.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357922+0000", 2024-09-17T21:20:34.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4779999999999997e-06 2024-09-17T21:20:34.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:34.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358023+0000", 2024-09-17T21:20:34.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101025 2024-09-17T21:20:34.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:34.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358040+0000", 2024-09-17T21:20:34.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6291999999999999e-05 2024-09-17T21:20:34.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:34.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298203+0000", 2024-09-17T21:20:34.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94016297500000001 2024-09-17T21:20:34.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:34.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298626+0000", 2024-09-17T21:20:34.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000422853 2024-09-17T21:20:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.299532+0000", 2024-09-17T21:20:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00090657300000000004 2024-09-17T21:20:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15705 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 973824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.520882+0000", 2024-09-17T21:20:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.334287644, 2024-09-17T21:20:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091691083000000007, 2024-09-17T21:20:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15705 2024-09-17T21:20:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520882+0000", 2024-09-17T21:20:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520882+0000", 2024-09-17T21:20:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520884+0000", 2024-09-17T21:20:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:20:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520889+0000", 2024-09-17T21:20:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:20:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520907+0000", 2024-09-17T21:20:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7750000000000001e-05 2024-09-17T21:20:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520920+0000", 2024-09-17T21:20:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2826000000000001e-05 2024-09-17T21:20:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612293+0000", 2024-09-17T21:20:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091373290999999995 2024-09-17T21:20:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612386+0000", 2024-09-17T21:20:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3261e-05 2024-09-17T21:20:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612573+0000", 2024-09-17T21:20:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000186725 2024-09-17T21:20:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15706 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 977920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:20:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.334189276, 2024-09-17T21:20:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091952328, 2024-09-17T21:20:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15706 2024-09-17T21:20:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:20:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:20:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520982+0000", 2024-09-17T21:20:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.277e-06 2024-09-17T21:20:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520988+0000", 2024-09-17T21:20:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4470000000000003e-06 2024-09-17T21:20:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521008+0000", 2024-09-17T21:20:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9309000000000001e-05 2024-09-17T21:20:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521023+0000", 2024-09-17T21:20:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5119999999999999e-05 2024-09-17T21:20:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612638+0000", 2024-09-17T21:20:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091615328999999995 2024-09-17T21:20:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612738+0000", 2024-09-17T21:20:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100466 2024-09-17T21:20:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612933+0000", 2024-09-17T21:20:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019437999999999999 2024-09-17T21:20:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15707 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 982016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:20:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.334125820000001, 2024-09-17T21:20:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092305228000000003, 2024-09-17T21:20:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15707 2024-09-17T21:20:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:20:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:20:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521045+0000", 2024-09-17T21:20:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4899999999999999e-06 2024-09-17T21:20:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521066+0000", 2024-09-17T21:20:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0236000000000001e-05 2024-09-17T21:20:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521085+0000", 2024-09-17T21:20:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8819999999999999e-05 2024-09-17T21:20:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521100+0000", 2024-09-17T21:20:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5146e-05 2024-09-17T21:20:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612978+0000", 2024-09-17T21:20:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091878847 2024-09-17T21:20:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613079+0000", 2024-09-17T21:20:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100503 2024-09-17T21:20:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613349+0000", 2024-09-17T21:20:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000270186 2024-09-17T21:20:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15708 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 986112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:20:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.334047645, 2024-09-17T21:20:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092547217000000001, 2024-09-17T21:20:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15708 2024-09-17T21:20:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:20:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:20:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521123+0000", 2024-09-17T21:20:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.353e-06 2024-09-17T21:20:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521129+0000", 2024-09-17T21:20:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:20:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521145+0000", 2024-09-17T21:20:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.611e-05 2024-09-17T21:20:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521154+0000", 2024-09-17T21:20:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4500000000000004e-06 2024-09-17T21:20:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613397+0000", 2024-09-17T21:20:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092243275 2024-09-17T21:20:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613503+0000", 2024-09-17T21:20:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106013 2024-09-17T21:20:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613669+0000", 2024-09-17T21:20:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016642299999999999 2024-09-17T21:20:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15709 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 990208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:34.884 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:20:34.884 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.334004629000001, 2024-09-17T21:20:34.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092728678999999994, 2024-09-17T21:20:34.884 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:34.884 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:34.884 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:34.884 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:34.884 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:34.884 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:34.884 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15709 2024-09-17T21:20:34.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.884 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:34.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:20:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:20:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521166+0000", 2024-09-17T21:20:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7000000000000004e-07 2024-09-17T21:20:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521169+0000", 2024-09-17T21:20:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2660000000000002e-06 2024-09-17T21:20:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521181+0000", 2024-09-17T21:20:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.208e-05 2024-09-17T21:20:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521194+0000", 2024-09-17T21:20:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2707e-05 2024-09-17T21:20:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613695+0000", 2024-09-17T21:20:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092500709 2024-09-17T21:20:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613750+0000", 2024-09-17T21:20:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.503e-05 2024-09-17T21:20:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613894+0000", 2024-09-17T21:20:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144117 2024-09-17T21:20:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15710 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 994304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:20:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.333960062999999, 2024-09-17T21:20:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092957729000000003, 2024-09-17T21:20:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15710 2024-09-17T21:20:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:20:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:20:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:20:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8000000000000005e-07 2024-09-17T21:20:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521216+0000", 2024-09-17T21:20:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3859999999999998e-06 2024-09-17T21:20:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521234+0000", 2024-09-17T21:20:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:20:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521248+0000", 2024-09-17T21:20:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:20:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613926+0000", 2024-09-17T21:20:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092678324000000006 2024-09-17T21:20:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613974+0000", 2024-09-17T21:20:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7754999999999999e-05 2024-09-17T21:20:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614167+0000", 2024-09-17T21:20:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193508 2024-09-17T21:20:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15711 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 998400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:20:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.333902681, 2024-09-17T21:20:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093268000000000004, 2024-09-17T21:20:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15711 2024-09-17T21:20:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:20:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:20:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521268+0000", 2024-09-17T21:20:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-17T21:20:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521273+0000", 2024-09-17T21:20:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:20:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521290+0000", 2024-09-17T21:20:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7269999999999999e-05 2024-09-17T21:20:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521304+0000", 2024-09-17T21:20:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3626e-05 2024-09-17T21:20:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614220+0000", 2024-09-17T21:20:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092916367 2024-09-17T21:20:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614322+0000", 2024-09-17T21:20:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102212 2024-09-17T21:20:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614535+0000", 2024-09-17T21:20:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021266199999999999 2024-09-17T21:20:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15712 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1002496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:20:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.333847186, 2024-09-17T21:20:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093568707000000001, 2024-09-17T21:20:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15712 2024-09-17T21:20:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:20:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:20:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521324+0000", 2024-09-17T21:20:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.17e-06 2024-09-17T21:20:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521334+0000", 2024-09-17T21:20:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9369999999999999e-06 2024-09-17T21:20:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521356+0000", 2024-09-17T21:20:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2045999999999999e-05 2024-09-17T21:20:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521370+0000", 2024-09-17T21:20:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4270000000000001e-05 2024-09-17T21:20:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614583+0000", 2024-09-17T21:20:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093213220999999999 2024-09-17T21:20:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614680+0000", 2024-09-17T21:20:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6900999999999999e-05 2024-09-17T21:20:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614891+0000", 2024-09-17T21:20:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021116200000000001 2024-09-17T21:20:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15713 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1006592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:20:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.333780484, 2024-09-17T21:20:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093834641999999996, 2024-09-17T21:20:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15713 2024-09-17T21:20:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:20:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:20:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521390+0000", 2024-09-17T21:20:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:20:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521396+0000", 2024-09-17T21:20:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:20:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521417+0000", 2024-09-17T21:20:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1236000000000001e-05 2024-09-17T21:20:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521431+0000", 2024-09-17T21:20:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3873000000000001e-05 2024-09-17T21:20:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614938+0000", 2024-09-17T21:20:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093506522999999994 2024-09-17T21:20:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615025+0000", 2024-09-17T21:20:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7635999999999999e-05 2024-09-17T21:20:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615224+0000", 2024-09-17T21:20:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019844700000000001 2024-09-17T21:20:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15714 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:20:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.333717227999999, 2024-09-17T21:20:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094071925000000001, 2024-09-17T21:20:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15714 2024-09-17T21:20:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:20:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:20:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521454+0000", 2024-09-17T21:20:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.097e-06 2024-09-17T21:20:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521459+0000", 2024-09-17T21:20:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2660000000000001e-06 2024-09-17T21:20:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521481+0000", 2024-09-17T21:20:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1878999999999998e-05 2024-09-17T21:20:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521498+0000", 2024-09-17T21:20:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6807e-05 2024-09-17T21:20:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615271+0000", 2024-09-17T21:20:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093773286999999997 2024-09-17T21:20:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615366+0000", 2024-09-17T21:20:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5322000000000007e-05 2024-09-17T21:20:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615524+0000", 2024-09-17T21:20:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015826699999999999 2024-09-17T21:20:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15715 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1014784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:20:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.333652893, 2024-09-17T21:20:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094243840999999995, 2024-09-17T21:20:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15715 2024-09-17T21:20:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:20:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:20:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521518+0000", 2024-09-17T21:20:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4300000000000001e-06 2024-09-17T21:20:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521524+0000", 2024-09-17T21:20:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5860000000000004e-06 2024-09-17T21:20:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521541+0000", 2024-09-17T21:20:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7574000000000001e-05 2024-09-17T21:20:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521555+0000", 2024-09-17T21:20:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3540000000000001e-05 2024-09-17T21:20:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615560+0000", 2024-09-17T21:20:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.09400472 2024-09-17T21:20:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615614+0000", 2024-09-17T21:20:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4693e-05 2024-09-17T21:20:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615761+0000", 2024-09-17T21:20:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000146298 2024-09-17T21:20:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15716 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1018880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:20:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.333596547999999, 2024-09-17T21:20:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094400676000000003, 2024-09-17T21:20:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15716 2024-09-17T21:20:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:20:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:20:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521574+0000", 2024-09-17T21:20:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1939999999999999e-06 2024-09-17T21:20:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521579+0000", 2024-09-17T21:20:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:20:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521597+0000", 2024-09-17T21:20:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8074e-05 2024-09-17T21:20:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521613+0000", 2024-09-17T21:20:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5316e-05 2024-09-17T21:20:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615780+0000", 2024-09-17T21:20:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094166874999999997 2024-09-17T21:20:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615836+0000", 2024-09-17T21:20:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6926999999999999e-05 2024-09-17T21:20:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615974+0000", 2024-09-17T21:20:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013741400000000001 2024-09-17T21:20:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15717 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1022976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:20:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.333537851999999, 2024-09-17T21:20:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094586723999999997, 2024-09-17T21:20:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15717 2024-09-17T21:20:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:20:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:20:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521633+0000", 2024-09-17T21:20:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:20:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521638+0000", 2024-09-17T21:20:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:20:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521659+0000", 2024-09-17T21:20:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-05 2024-09-17T21:20:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521675+0000", 2024-09-17T21:20:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5688999999999998e-05 2024-09-17T21:20:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:34.923 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616008+0000", 2024-09-17T21:20:34.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094333314000000001 2024-09-17T21:20:34.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.923 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:34.923 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616066+0000", 2024-09-17T21:20:34.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7797000000000002e-05 2024-09-17T21:20:34.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616219+0000", 2024-09-17T21:20:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015281100000000001 2024-09-17T21:20:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15718 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:20:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.333437777, 2024-09-17T21:20:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094703674000000002, 2024-09-17T21:20:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:34.925 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:34.925 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:34.925 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:34.925 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:34.925 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:34.925 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15718 2024-09-17T21:20:34.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:20:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:20:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521734+0000", 2024-09-17T21:20:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.61e-06 2024-09-17T21:20:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521740+0000", 2024-09-17T21:20:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:20:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521754+0000", 2024-09-17T21:20:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3793e-05 2024-09-17T21:20:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521763+0000", 2024-09-17T21:20:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5129999999999996e-06 2024-09-17T21:20:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616256+0000", 2024-09-17T21:20:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094492919999999994 2024-09-17T21:20:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616312+0000", 2024-09-17T21:20:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6737000000000002e-05 2024-09-17T21:20:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616436+0000", 2024-09-17T21:20:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012335800000000001 2024-09-17T21:20:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15719 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1031168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:20:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.333384371999999, 2024-09-17T21:20:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094888963000000007, 2024-09-17T21:20:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15719 2024-09-17T21:20:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:20:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:20:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521786+0000", 2024-09-17T21:20:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.074e-06 2024-09-17T21:20:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521791+0000", 2024-09-17T21:20:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3030000000000001e-06 2024-09-17T21:20:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521804+0000", 2024-09-17T21:20:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3027000000000001e-05 2024-09-17T21:20:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521811+0000", 2024-09-17T21:20:34.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6090000000000003e-06 2024-09-17T21:20:34.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:34.933 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616466+0000", 2024-09-17T21:20:34.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094654294 2024-09-17T21:20:34.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616523+0000", 2024-09-17T21:20:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6953999999999997e-05 2024-09-17T21:20:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616674+0000", 2024-09-17T21:20:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015170200000000001 2024-09-17T21:20:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15720 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1035264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:20:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.333347531999999, 2024-09-17T21:20:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095067061999999994, 2024-09-17T21:20:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15720 2024-09-17T21:20:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:20:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:20:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521823+0000", 2024-09-17T21:20:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:20:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521826+0000", 2024-09-17T21:20:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2100000000000002e-06 2024-09-17T21:20:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521835+0000", 2024-09-17T21:20:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1630000000000002e-06 2024-09-17T21:20:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521842+0000", 2024-09-17T21:20:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:20:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616711+0000", 2024-09-17T21:20:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094868859 2024-09-17T21:20:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616764+0000", 2024-09-17T21:20:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2778999999999997e-05 2024-09-17T21:20:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616889+0000", 2024-09-17T21:20:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012533100000000001 2024-09-17T21:20:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15721 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1039360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:20:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.333300423000001, 2024-09-17T21:20:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095277219999999996, 2024-09-17T21:20:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:34.940 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:34.940 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15721 2024-09-17T21:20:34.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.940 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:34.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.940 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:34.940 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:20:34.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.940 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:34.940 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:20:34.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.941 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:34.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521871+0000", 2024-09-17T21:20:34.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.443e-06 2024-09-17T21:20:34.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.941 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:34.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521876+0000", 2024-09-17T21:20:34.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0300000000000001e-06 2024-09-17T21:20:34.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.941 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:34.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521886+0000", 2024-09-17T21:20:34.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186e-05 2024-09-17T21:20:34.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:34.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521893+0000", 2024-09-17T21:20:34.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.08e-06 2024-09-17T21:20:34.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:34.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616918+0000", 2024-09-17T21:20:34.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095025223000000006 2024-09-17T21:20:34.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:34.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616977+0000", 2024-09-17T21:20:34.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8403999999999999e-05 2024-09-17T21:20:34.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:34.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617147+0000", 2024-09-17T21:20:34.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016985399999999999 2024-09-17T21:20:34.943 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.943 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:34.943 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.943 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15722 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:34.944 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:20:34.944 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.333265974, 2024-09-17T21:20:34.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095459783000000006, 2024-09-17T21:20:34.944 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:34.944 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:34.944 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:34.944 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:34.944 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:34.944 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:34.944 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15722 2024-09-17T21:20:34.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.944 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:34.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.944 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:34.945 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:20:34.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.945 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.945 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:34.945 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:20:34.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.945 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.945 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:34.945 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:20:34.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7700000000000004e-07 2024-09-17T21:20:34.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:34.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521907+0000", 2024-09-17T21:20:34.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.813e-06 2024-09-17T21:20:34.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:34.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521922+0000", 2024-09-17T21:20:34.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4533e-05 2024-09-17T21:20:34.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:34.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521934+0000", 2024-09-17T21:20:34.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2469999999999999e-05 2024-09-17T21:20:34.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:34.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617187+0000", 2024-09-17T21:20:34.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095252311000000006 2024-09-17T21:20:34.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:34.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617240+0000", 2024-09-17T21:20:34.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3740999999999999e-05 2024-09-17T21:20:34.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:34.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617364+0000", 2024-09-17T21:20:34.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-17T21:20:34.948 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.948 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:34.948 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.948 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15723 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:34.948 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:20:34.948 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.333222107999999, 2024-09-17T21:20:34.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095653563999999996, 2024-09-17T21:20:34.948 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:34.948 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:34.948 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:34.949 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:34.949 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:34.949 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:34.949 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15723 2024-09-17T21:20:34.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.949 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:34.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:34.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:20:34.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:34.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:20:34.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:34.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.950 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.950 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:34.950 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521949+0000", 2024-09-17T21:20:34.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:20:34.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.950 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.950 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:34.950 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521953+0000", 2024-09-17T21:20:34.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5870000000000003e-06 2024-09-17T21:20:34.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.950 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:34.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521970+0000", 2024-09-17T21:20:34.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7282000000000002e-05 2024-09-17T21:20:34.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:34.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521984+0000", 2024-09-17T21:20:34.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:20:34.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:34.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617383+0000", 2024-09-17T21:20:34.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095399063000000006 2024-09-17T21:20:34.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:34.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617437+0000", 2024-09-17T21:20:34.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3931999999999998e-05 2024-09-17T21:20:34.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:34.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:34.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:34.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617601+0000", 2024-09-17T21:20:34.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016407300000000001 2024-09-17T21:20:34.952 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.952 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:34.952 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.952 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:34.953 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:34.953 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:34.954 INFO:teuthology.orchestra.run.smithi184.stdout:5098 read 0x6453400 thru 0x645fc6f (0xc870 bytes) 2024-09-17T21:20:34.954 INFO:teuthology.orchestra.run.smithi184.stdout:5100 trunc from 0x651c400 to 0x168d400 2024-09-17T21:20:34.955 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:20:34.955 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:34.946+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:34.964 INFO:teuthology.orchestra.run.smithi184.stdout:5101 write 0xe8a5c00 thru 0xe8a7793 (0x1b94 bytes) 2024-09-17T21:20:34.994 INFO:teuthology.orchestra.run.smithi184.stdout:5104 trunc from 0xe8a7800 to 0x3430c00 2024-09-17T21:20:35.055 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:35.049+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:35.061 INFO:teuthology.orchestra.run.smithi184.stdout:5105 punch from 0x2773a00 to 0x2777400, (0x3a00 bytes) 2024-09-17T21:20:35.111 INFO:teuthology.orchestra.run.smithi184.stdout:5106 write 0xd9ff200 thru 0xda0eb69 (0xf96a bytes) 2024-09-17T21:20:35.114 INFO:teuthology.orchestra.run.smithi184.stdout:5107 punch from 0xbc89200 to 0xbc96a00, (0xd800 bytes) 2024-09-17T21:20:35.115 INFO:teuthology.orchestra.run.smithi184.stdout:5108 trunc from 0xda0ec00 to 0xdcc3000 2024-09-17T21:20:35.143 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:35.143 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:35.143 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:20:35.143 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:35.148 INFO:teuthology.orchestra.run.smithi184.stdout:5110 write 0x7e51c00 thru 0x7e5d370 (0xb771 bytes) 2024-09-17T21:20:35.151 INFO:teuthology.orchestra.run.smithi184.stdout:5112 read 0xc2e5c00 thru 0xc2f227d (0xc67e bytes) 2024-09-17T21:20:35.151 INFO:teuthology.orchestra.run.smithi184.stdout:5115 write 0x550a00 thru 0x5600e3 (0xf6e4 bytes) 2024-09-17T21:20:35.153 INFO:teuthology.orchestra.run.smithi184.stdout:5116 write 0x4138800 thru 0x4142f93 (0xa794 bytes) 2024-09-17T21:20:35.154 INFO:teuthology.orchestra.run.smithi184.stdout:5117 trunc from 0xdcc3000 to 0xdc3a000 2024-09-17T21:20:35.155 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:35.149+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:35.160 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:20:35.184 INFO:teuthology.orchestra.run.smithi184.stdout:5119 write 0x8328400 thru 0x8328aaa (0x6ab bytes) 2024-09-17T21:20:35.186 INFO:teuthology.orchestra.run.smithi184.stdout:5120 read 0xb9d3400 thru 0xb9d771d (0x431e bytes) 2024-09-17T21:20:35.187 INFO:teuthology.orchestra.run.smithi184.stdout:5122 read 0xbb08400 thru 0xbb12d3b (0xa93c bytes) 2024-09-17T21:20:35.189 INFO:teuthology.orchestra.run.smithi184.stdout:5123 write 0x7aeb800 thru 0x7aecbcf (0x13d0 bytes) 2024-09-17T21:20:35.189 INFO:teuthology.orchestra.run.smithi184.stdout:5126 read 0xb914600 thru 0xb918270 (0x3c71 bytes) 2024-09-17T21:20:35.190 INFO:teuthology.orchestra.run.smithi184.stdout:5127 read 0x296a200 thru 0x2975ccc (0xbacd bytes) 2024-09-17T21:20:35.192 INFO:teuthology.orchestra.run.smithi184.stdout:5130 write 0x60f5600 thru 0x60fc1f9 (0x6bfa bytes) 2024-09-17T21:20:35.214 INFO:teuthology.orchestra.run.smithi184.stdout:5131 trunc from 0xdc3a000 to 0x620a600 2024-09-17T21:20:35.256 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:35.250+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:35.297 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:35.297 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:35.297 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:35.297 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:35.297 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:35.308 INFO:teuthology.orchestra.run.smithi184.stdout:5132 write 0xc2ae200 thru 0xc2bc9af (0xe7b0 bytes) 2024-09-17T21:20:35.310 INFO:teuthology.orchestra.run.smithi184.stdout:5134 write 0x6f9da00 thru 0x6face81 (0xf482 bytes) 2024-09-17T21:20:35.311 INFO:teuthology.orchestra.run.smithi184.stdout:5135 read 0x7f37200 thru 0x7f3e4e7 (0x72e8 bytes) 2024-09-17T21:20:35.312 INFO:teuthology.orchestra.run.smithi184.stdout:5136 read 0x1908e00 thru 0x191652e (0xd72f bytes) 2024-09-17T21:20:35.313 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:20:35.314 INFO:teuthology.orchestra.run.smithi184.stdout:5139 punch from 0x4d70a00 to 0x4d7cc00, (0xc200 bytes) 2024-09-17T21:20:35.315 INFO:teuthology.orchestra.run.smithi184.stdout:5141 read 0x4d44a00 thru 0x4d4fe3a (0xb43b bytes) 2024-09-17T21:20:35.340 INFO:teuthology.orchestra.run.smithi184.stdout:5144 write 0xed2dc00 thru 0xed31aee (0x3eef bytes) 2024-09-17T21:20:35.341 INFO:teuthology.orchestra.run.smithi184.stdout:5145 read 0x3eda200 thru 0x3edd209 (0x300a bytes) 2024-09-17T21:20:35.343 INFO:teuthology.orchestra.run.smithi184.stdout:5146 read 0x5489000 thru 0x549835c (0xf35d bytes) 2024-09-17T21:20:35.344 INFO:teuthology.orchestra.run.smithi184.stdout:5148 punch from 0x77c800 to 0x78a400, (0xdc00 bytes) 2024-09-17T21:20:35.345 INFO:teuthology.orchestra.run.smithi184.stdout:5153 punch from 0x7423600 to 0x7423e00, (0x800 bytes) 2024-09-17T21:20:35.345 INFO:teuthology.orchestra.run.smithi184.stdout:5155 write 0xe24d000 thru 0xe25c2e9 (0xf2ea bytes) 2024-09-17T21:20:35.357 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:35.353+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:35.360 INFO:teuthology.orchestra.run.smithi184.stdout:5156 trunc from 0xed31c00 to 0x86fe200 2024-09-17T21:20:35.414 INFO:teuthology.orchestra.run.smithi184.stdout:5158 write 0xd20800 thru 0xd26f2f (0x6730 bytes) 2024-09-17T21:20:35.415 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:20:35.417 INFO:teuthology.orchestra.run.smithi184.stdout:5161 read 0x4afa200 thru 0x4b0a18c (0xff8d bytes) 2024-09-17T21:20:35.418 INFO:teuthology.orchestra.run.smithi184.stdout:5164 read 0x6159000 thru 0x6161712 (0x8713 bytes) 2024-09-17T21:20:35.435 INFO:teuthology.orchestra.run.smithi184.stdout:5169 trunc from 0x86fe200 to 0xdfcf600 2024-09-17T21:20:35.465 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:35.461+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:35.485 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:35.486 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:35.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:35.486 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:35.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.486 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:20:35.486 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:35.486 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 133.87869368899999, 2024-09-17T21:20:35.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:20:35.486 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:35.486 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:35.486 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:35.486 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:35.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:35.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:35.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:35.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:35.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:35.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:20:35.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:20:35.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:35.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:20:35.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:20:35.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:35.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:20:35.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:20:35.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:35.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:20:35.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:20:35.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:35.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:20:35.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:20:35.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:35.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:20:35.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:20:35.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:35.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:20:35.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:20:35.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:35.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:20:35.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:20:35.490 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.490 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:35.490 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.490 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:20:35.490 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:35.490 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 133.86874705299999, 2024-09-17T21:20:35.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:20:35.490 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:35.490 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:35.491 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:35.491 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:35.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:35.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:35.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:35.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:35.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:35.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:20:35.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:20:35.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:35.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:20:35.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:20:35.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:35.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:20:35.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:20:35.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:35.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:20:35.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:20:35.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:35.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:20:35.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:20:35.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:35.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:20:35.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:20:35.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:35.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:20:35.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:20:35.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:35.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:20:35.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:20:35.494 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.494 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:35.494 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.494 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:20:35.494 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:35.494 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 132.62559621299999, 2024-09-17T21:20:35.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:20:35.495 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:35.495 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:35.495 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:35.495 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:35.495 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:20:35.495 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:20:35.495 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:20:35.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.495 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:35.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:35.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:35.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:35.496 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:35.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:35.496 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:20:35.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:20:35.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:35.496 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:20:35.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:20:35.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:35.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:20:35.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:20:35.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:35.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:20:35.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:20:35.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:35.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:20:35.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:20:35.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:35.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:20:35.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:20:35.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:35.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:20:35.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:20:35.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:35.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:20:35.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:20:35.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:35.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:20:35.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:20:35.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:35.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:20:35.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:20:35.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:35.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:20:35.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:20:35.500 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.500 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:35.500 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.500 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:20:35.500 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:35.500 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 132.61595072700001, 2024-09-17T21:20:35.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:20:35.501 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:35.501 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:35.501 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:35.501 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:35.501 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:20:35.501 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:20:35.501 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:20:35.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.501 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:35.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:35.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:35.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:35.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:35.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:35.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:20:35.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:20:35.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:35.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:20:35.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:20:35.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:35.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:20:35.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:20:35.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:35.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:20:35.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:20:35.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:35.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:20:35.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:20:35.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:35.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:20:35.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:20:35.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:35.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:20:35.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:20:35.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:35.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:20:35.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:20:35.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:35.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:20:35.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:20:35.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:35.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:20:35.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:20:35.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:35.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:20:35.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:20:35.506 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.506 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:35.506 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.506 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:20:35.506 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:35.506 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 119.247014694, 2024-09-17T21:20:35.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:20:35.507 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:35.507 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:35.507 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:35.507 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:35.507 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:20:35.507 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:20:35.507 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:20:35.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.507 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:35.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:35.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:35.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:35.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:35.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:35.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:20:35.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:20:35.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:35.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:20:35.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:20:35.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:35.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:20:35.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:20:35.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:35.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:20:35.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:20:35.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:35.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:20:35.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:20:35.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:35.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:20:35.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:20:35.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:35.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:20:35.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:20:35.510 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.510 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:35.510 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.511 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:819 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5111808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:35.511 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:35.511 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.045442404, 2024-09-17T21:20:35.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024344811000000001, 2024-09-17T21:20:35.511 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:35.511 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:35.511 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:35.511 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:35.511 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:35.511 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:35.511 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 819 2024-09-17T21:20:35.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.512 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:35.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:35.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:35.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:35.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:35.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:35.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437443+0000", 2024-09-17T21:20:35.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1969999999999999e-06 2024-09-17T21:20:35.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:35.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437448+0000", 2024-09-17T21:20:35.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9490000000000004e-06 2024-09-17T21:20:35.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:35.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437467+0000", 2024-09-17T21:20:35.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8862999999999999e-05 2024-09-17T21:20:35.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:35.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437480+0000", 2024-09-17T21:20:35.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2833000000000001e-05 2024-09-17T21:20:35.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:35.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461552+0000", 2024-09-17T21:20:35.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024072695000000002 2024-09-17T21:20:35.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:35.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461639+0000", 2024-09-17T21:20:35.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6996000000000005e-05 2024-09-17T21:20:35.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:35.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461787+0000", 2024-09-17T21:20:35.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014727800000000001 2024-09-17T21:20:35.515 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.515 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:35.516 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.516 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:35.516 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:35.516 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.045330132, 2024-09-17T21:20:35.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:20:35.516 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:35.516 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:35.516 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:35.516 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:35.516 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:35.516 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:35.517 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:20:35.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.517 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:35.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:35.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:35.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:35.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:35.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:35.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:20:35.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:20:35.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:35.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:20:35.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:20:35.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:35.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:20:35.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:20:35.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:35.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:20:35.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:20:35.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:35.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:20:35.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:20:35.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:35.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:20:35.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:20:35.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:35.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:20:35.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:20:35.520 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.520 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:35.520 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.520 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:833 3.5 3:a772dfb1:::rbd_data.11523651b015.0000000000000001:d [sparse-read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:35.520 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:35.521 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.044895846, 2024-09-17T21:20:35.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024216024999999999, 2024-09-17T21:20:35.521 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:35.521 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:35.521 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:35.521 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:35.521 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:35.521 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:35.521 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 833 2024-09-17T21:20:35.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.521 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:35.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:35.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:35.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:35.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:35.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:35.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437990+0000", 2024-09-17T21:20:35.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2500000000000001e-06 2024-09-17T21:20:35.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:35.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437995+0000", 2024-09-17T21:20:35.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1240000000000004e-06 2024-09-17T21:20:35.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:35.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438012+0000", 2024-09-17T21:20:35.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7572e-05 2024-09-17T21:20:35.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:35.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438031+0000", 2024-09-17T21:20:35.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8612000000000001e-05 2024-09-17T21:20:35.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:35.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461830+0000", 2024-09-17T21:20:35.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023799135999999999 2024-09-17T21:20:35.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:35.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462005+0000", 2024-09-17T21:20:35.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000174747 2024-09-17T21:20:35.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:35.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:20:35.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019958399999999999 2024-09-17T21:20:35.524 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.525 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:35.525 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.525 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:35.525 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:35.525 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.04483555500001, 2024-09-17T21:20:35.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:20:35.525 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:35.525 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:35.525 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:35.525 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:35.525 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:35.525 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:35.526 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:20:35.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.526 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:35.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:35.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:35.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:35.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:35.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:35.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:20:35.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:20:35.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:35.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:20:35.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:20:35.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:35.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:20:35.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:20:35.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:35.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:20:35.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:20:35.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:35.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:20:35.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:20:35.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:35.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:20:35.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:20:35.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:35.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:20:35.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:20:35.529 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.529 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:35.529 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.530 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:35.530 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:35.530 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.044780174, 2024-09-17T21:20:35.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:20:35.530 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:35.530 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:35.530 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:35.530 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:35.530 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:35.530 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:35.530 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:20:35.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.530 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:35.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:35.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:35.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:35.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:35.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:35.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:20:35.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:20:35.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:35.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:20:35.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:20:35.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:35.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:20:35.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:20:35.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:35.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:20:35.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:20:35.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:35.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:20:35.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:20:35.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:35.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:20:35.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:20:35.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:35.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:20:35.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:20:35.534 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.534 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:35.534 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.534 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:35.534 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:35.534 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.04472389599999, 2024-09-17T21:20:35.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:20:35.534 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:35.534 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:35.534 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:35.534 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:35.534 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:35.534 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:35.535 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:20:35.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.535 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:35.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:35.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:35.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:35.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:35.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:35.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:20:35.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:20:35.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:35.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:20:35.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:20:35.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:35.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:20:35.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:20:35.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:35.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:20:35.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:20:35.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:35.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:20:35.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:20:35.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:35.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:20:35.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:20:35.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:35.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:20:35.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:20:35.538 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.538 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:35.538 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.538 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:35.538 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:35.538 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.043774257, 2024-09-17T21:20:35.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:20:35.539 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:35.539 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:35.539 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:35.539 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:35.539 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:35.539 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:35.539 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:20:35.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.539 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:35.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:35.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:35.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:35.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:35.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:35.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:20:35.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:20:35.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:35.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:20:35.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:20:35.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:35.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:20:35.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:20:35.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:35.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:20:35.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:20:35.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:35.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:20:35.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:20:35.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:35.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:20:35.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:20:35.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:35.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:20:35.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:20:35.543 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.543 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:35.543 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.543 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:35.543 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:35.543 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.043695637, 2024-09-17T21:20:35.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:20:35.543 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:35.543 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:35.543 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:35.543 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:35.544 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:35.544 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:35.544 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:20:35.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.544 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:35.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:35.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:35.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:35.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:35.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:35.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:20:35.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:20:35.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:35.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:20:35.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:20:35.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:35.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:20:35.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:20:35.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:35.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:20:35.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:20:35.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:35.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:20:35.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:20:35.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:35.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:20:35.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:20:35.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:35.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:20:35.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:20:35.547 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.547 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:35.547 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.548 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:35.548 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:35.548 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.04358269799999, 2024-09-17T21:20:35.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:20:35.548 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:35.548 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:35.548 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:35.548 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:35.548 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:35.548 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:35.548 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:20:35.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.548 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:35.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:35.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:35.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:35.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:35.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:35.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:20:35.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:20:35.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:35.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:20:35.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:20:35.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:35.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:20:35.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:20:35.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:35.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:20:35.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:20:35.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:35.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:20:35.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:20:35.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:35.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:20:35.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:20:35.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:35.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:20:35.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:20:35.552 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.552 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:35.552 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.552 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:849 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5963776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:35.552 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:35.552 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.042988662, 2024-09-17T21:20:35.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024537514, 2024-09-17T21:20:35.552 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:35.552 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:35.552 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:35.552 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:35.552 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:35.553 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:35.553 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 849 2024-09-17T21:20:35.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.553 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:35.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.553 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:35.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:35.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.553 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:35.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:35.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:35.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439897+0000", 2024-09-17T21:20:35.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9099999999999999e-06 2024-09-17T21:20:35.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:35.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439905+0000", 2024-09-17T21:20:35.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0870000000000007e-06 2024-09-17T21:20:35.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:35.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439924+0000", 2024-09-17T21:20:35.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9042000000000002e-05 2024-09-17T21:20:35.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:35.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439941+0000", 2024-09-17T21:20:35.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.624e-05 2024-09-17T21:20:35.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:35.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464132+0000", 2024-09-17T21:20:35.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024191579000000001 2024-09-17T21:20:35.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:35.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464291+0000", 2024-09-17T21:20:35.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015904800000000001 2024-09-17T21:20:35.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:35.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464433+0000", 2024-09-17T21:20:35.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014160799999999999 2024-09-17T21:20:35.556 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.556 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:35.556 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.556 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:35.556 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:35.556 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.042921644, 2024-09-17T21:20:35.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:20:35.557 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:35.557 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:35.557 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:35.557 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:35.557 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:35.557 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:35.557 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:20:35.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.557 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:35.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.557 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:35.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:35.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:35.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:35.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:35.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:20:35.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:20:35.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:35.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:20:35.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:20:35.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:35.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:20:35.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:20:35.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:35.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:20:35.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:20:35.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:35.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:20:35.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:20:35.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:35.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:20:35.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:20:35.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:35.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:20:35.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:20:35.560 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.560 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:35.560 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.561 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:35.561 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:35.561 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.042856833, 2024-09-17T21:20:35.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:20:35.561 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:35.561 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:35.561 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:35.561 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:35.561 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:35.561 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:35.561 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:20:35.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.562 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:35.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.562 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:35.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:35.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.562 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:35.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:35.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.562 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:35.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:20:35.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:20:35.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:35.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:20:35.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:20:35.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:35.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:20:35.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:20:35.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:35.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:20:35.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:20:35.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:35.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:20:35.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:20:35.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:35.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:20:35.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:20:35.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:35.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:20:35.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:20:35.565 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.565 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:35.565 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.565 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:35.565 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:35.565 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.042795856, 2024-09-17T21:20:35.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:20:35.565 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:35.565 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:35.566 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:35.566 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:35.566 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:35.566 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:35.566 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:20:35.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.566 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:35.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:35.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:35.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:35.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:35.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:35.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:20:35.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:20:35.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:35.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:20:35.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:20:35.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:35.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:20:35.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:20:35.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:35.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:20:35.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:20:35.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:35.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:20:35.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:20:35.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:35.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:20:35.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:20:35.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:35.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:20:35.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:20:35.569 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.569 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:35.569 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.569 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:20:35.570 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:35.570 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.145224671000001, 2024-09-17T21:20:35.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:20:35.570 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:35.570 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:35.570 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:35.570 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:35.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:35.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:35.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:35.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:35.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:35.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:20:35.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:20:35.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:35.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:20:35.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:20:35.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:35.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:20:35.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:20:35.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:35.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:20:35.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:20:35.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:35.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:20:35.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:20:35.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:35.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:20:35.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:20:35.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:35.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:20:35.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:20:35.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:35.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:20:35.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:20:35.573 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.574 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:35.574 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.574 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:20:35.574 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:35.574 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.936566276999997, 2024-09-17T21:20:35.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:20:35.574 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:35.574 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:35.574 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:35.574 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:35.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:35.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:35.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:35.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:35.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:35.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:35.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:20:35.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:20:35.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:35.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:20:35.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:20:35.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:35.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:20:35.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:20:35.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:35.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:20:35.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:20:35.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:35.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:20:35.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:20:35.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:35.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:20:35.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:20:35.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:35.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:20:35.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:20:35.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:35.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:35.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:35.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:20:35.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:20:35.578 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.578 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:35.578 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.578 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:35.578 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:35.578 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:35.579 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:20:35.580 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:35.573+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:35.681 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:35.677+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:35.695 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:35.696 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:35.696 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:35.696 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:35.710 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:20:35.797 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:35.793+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:35.867 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:35.867 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:35.867 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:35.867 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:35.867 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:35.881 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:20:35.925 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:35.921+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:35.983 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:35.977+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:36.052 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:36.052 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:36.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:36.053 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:36.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.053 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:525 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3670016~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:36.053 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:36.053 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 129.84825368700001, 2024-09-17T21:20:36.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046645697, 2024-09-17T21:20:36.053 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.053 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.053 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.053 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.053 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:36.053 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:36.053 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 525 2024-09-17T21:20:36.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.054 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:36.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:36.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:20:36.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:20:36.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.055 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.055 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200234+0000", 2024-09-17T21:20:36.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:20:36.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.055 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.055 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200254+0000", 2024-09-17T21:20:36.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9959999999999999e-05 2024-09-17T21:20:36.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.055 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200293+0000", 2024-09-17T21:20:36.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9755999999999997e-05 2024-09-17T21:20:36.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245871+0000", 2024-09-17T21:20:36.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045577871999999998 2024-09-17T21:20:36.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246030+0000", 2024-09-17T21:20:36.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158975 2024-09-17T21:20:36.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.057 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246871+0000", 2024-09-17T21:20:36.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084111699999999995 2024-09-17T21:20:36.057 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.057 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.057 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.057 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:36.057 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:36.057 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 129.84800801200001, 2024-09-17T21:20:36.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:20:36.057 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.057 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.057 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.058 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.058 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:36.058 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:36.058 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:20:36.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.058 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:36.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:36.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:20:36.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:20:36.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:20:36.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:20:36.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:20:36.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:20:36.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.060 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.060 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.060 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:20:36.060 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:20:36.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.060 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.060 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.060 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:20:36.060 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:20:36.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.060 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.060 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.060 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:20:36.060 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:20:36.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.061 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.061 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:20:36.061 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:20:36.061 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.061 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.061 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.061 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:36.061 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:36.061 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 129.84782933, 2024-09-17T21:20:36.061 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:20:36.061 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.062 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.062 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.062 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.062 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:36.062 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:36.062 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:20:36.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.062 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.062 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.062 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.062 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:36.062 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.062 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:36.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:20:36.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:20:36.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:20:36.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:20:36.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.064 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.064 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:20:36.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:20:36.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.064 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.064 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:20:36.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:20:36.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.064 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.064 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:20:36.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:20:36.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.065 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.065 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.065 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:20:36.065 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:20:36.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.065 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.065 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.065 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:20:36.065 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:20:36.065 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.065 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.065 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.065 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.065 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:36.066 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:36.066 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 129.84777005699999, 2024-09-17T21:20:36.066 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:20:36.066 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.066 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.066 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.066 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.066 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:36.066 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:36.066 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:20:36.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.066 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.066 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:36.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:36.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:20:36.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:20:36.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.068 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.068 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:20:36.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:20:36.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.068 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.068 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:20:36.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:20:36.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.068 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.068 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:20:36.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:20:36.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.069 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.069 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:20:36.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:20:36.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.069 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.069 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:20:36.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:20:36.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.069 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.069 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:20:36.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:20:36.070 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.070 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.070 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.070 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.070 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:36.070 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:36.070 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 129.847627862, 2024-09-17T21:20:36.070 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:20:36.070 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.070 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.070 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.070 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.071 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:36.071 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:36.071 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:20:36.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.071 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.071 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:36.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.071 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:36.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:20:36.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:20:36.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:20:36.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:20:36.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:20:36.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:20:36.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:20:36.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:20:36.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:20:36.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:20:36.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:20:36.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:20:36.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:20:36.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:20:36.074 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.074 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.074 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.074 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:36.074 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:36.074 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 129.84757193900001, 2024-09-17T21:20:36.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:20:36.075 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.075 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.075 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.075 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.075 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:36.075 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:36.075 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:20:36.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.075 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.075 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.075 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:36.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.076 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.076 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:36.076 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.076 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.076 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.076 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:20:36.076 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:20:36.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.076 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.076 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.076 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:20:36.076 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:20:36.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.076 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.076 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:20:36.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:20:36.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:20:36.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:20:36.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:20:36.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:20:36.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:20:36.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:20:36.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:20:36.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:20:36.078 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.078 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.078 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.078 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:36.078 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:36.079 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 129.84751832200001, 2024-09-17T21:20:36.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:20:36.079 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.079 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.079 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.079 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.079 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:36.079 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:36.079 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:20:36.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.079 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.079 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.079 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:36.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:36.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:20:36.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:20:36.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:20:36.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:20:36.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.081 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.081 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:20:36.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:20:36.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.081 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.081 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:20:36.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:20:36.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.081 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:20:36.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:20:36.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.082 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:20:36.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:20:36.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.082 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:20:36.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:20:36.082 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.082 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.083 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.083 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:20:36.083 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:36.083 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 79.538290079999996, 2024-09-17T21:20:36.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:20:36.083 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.083 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.083 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:20:36.083 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.083 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:20:36.083 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:20:36.083 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:20:36.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.084 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.084 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:36.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.084 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:36.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:20:36.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:20:36.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:20:36.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:20:36.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:20:36.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:20:36.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:20:36.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:20:36.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:20:36.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:20:36.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:36.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:20:36.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:20:36.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.087 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:20:36.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:20:36.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.087 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.087 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:20:36.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:20:36.087 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.087 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.087 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.087 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2597 3.1f 3:fb0e9615:::rbd_data.1152a8ae2019.000000000000000c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:36.088 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:36.088 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 55.209853652, 2024-09-17T21:20:36.088 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097064994000000002, 2024-09-17T21:20:36.088 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.088 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.088 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.088 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.088 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:36.088 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:36.088 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2597 2024-09-17T21:20:36.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.088 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.088 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.088 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:36.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:36.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838628+0000", 2024-09-17T21:20:36.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.187e-06 2024-09-17T21:20:36.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838654+0000", 2024-09-17T21:20:36.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:20:36.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838696+0000", 2024-09-17T21:20:36.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1226000000000001e-05 2024-09-17T21:20:36.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838715+0000", 2024-09-17T21:20:36.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9646000000000001e-05 2024-09-17T21:20:36.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.091 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838937+0000", 2024-09-17T21:20:36.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022163 2024-09-17T21:20:36.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.091 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.091 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.839284+0000", 2024-09-17T21:20:36.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034753799999999999 2024-09-17T21:20:36.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.091 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.091 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935691+0000", 2024-09-17T21:20:36.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.096406360999999996 2024-09-17T21:20:36.091 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.091 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.092 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.092 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.092 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2641 3.12 3:488a729f:::rbd_data.1152a8ae2019.0000000000000022:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:36.092 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:36.092 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 55.206396879000003, 2024-09-17T21:20:36.092 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098280953000000004, 2024-09-17T21:20:36.092 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.092 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.092 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.092 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.092 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:36.092 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:36.092 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2641 2024-09-17T21:20:36.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.093 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.093 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.093 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:36.093 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.093 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.093 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:36.093 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842083+0000", 2024-09-17T21:20:36.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5300000000000002e-07 2024-09-17T21:20:36.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842088+0000", 2024-09-17T21:20:36.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:20:36.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842105+0000", 2024-09-17T21:20:36.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6773e-05 2024-09-17T21:20:36.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842119+0000", 2024-09-17T21:20:36.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:20:36.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842320+0000", 2024-09-17T21:20:36.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020052700000000001 2024-09-17T21:20:36.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842845+0000", 2024-09-17T21:20:36.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052508600000000002 2024-09-17T21:20:36.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.096 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.096 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940363+0000", 2024-09-17T21:20:36.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097518795000000005 2024-09-17T21:20:36.096 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.096 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.096 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.096 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2674 3.12 3:4c87e3ac:::rbd_data.1152a8ae2019.0000000000000033:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:36.096 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:36.096 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 55.206034242000001, 2024-09-17T21:20:36.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098025125000000005, 2024-09-17T21:20:36.096 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.096 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.097 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:36.097 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.097 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:36.097 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:36.097 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2674 2024-09-17T21:20:36.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.097 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.097 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:36.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:36.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842447+0000", 2024-09-17T21:20:36.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3999999999999999e-06 2024-09-17T21:20:36.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842453+0000", 2024-09-17T21:20:36.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:20:36.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842472+0000", 2024-09-17T21:20:36.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9006000000000001e-05 2024-09-17T21:20:36.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842487+0000", 2024-09-17T21:20:36.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4866e-05 2024-09-17T21:20:36.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843594+0000", 2024-09-17T21:20:36.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011073249999999999 2024-09-17T21:20:36.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843668+0000", 2024-09-17T21:20:36.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3188000000000005e-05 2024-09-17T21:20:36.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:36.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843893+0000", 2024-09-17T21:20:36.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022503399999999999 2024-09-17T21:20:36.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:36.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850439+0000", 2024-09-17T21:20:36.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065467880000000004 2024-09-17T21:20:36.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:36.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940409+0000", 2024-09-17T21:20:36.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089969511000000002 2024-09-17T21:20:36.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.101 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:36.101 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940444+0000", 2024-09-17T21:20:36.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5345999999999997e-05 2024-09-17T21:20:36.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.101 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.101 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940470+0000", 2024-09-17T21:20:36.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5964000000000001e-05 2024-09-17T21:20:36.101 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.101 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.101 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.102 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2707 3.1f 3:f805e65e:::rbd_data.1152a8ae2019.0000000000000049:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:36.102 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:36.102 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 55.200987581, 2024-09-17T21:20:36.102 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088327300999999997, 2024-09-17T21:20:36.102 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.102 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.102 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.102 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.102 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:36.102 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:36.102 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2707 2024-09-17T21:20:36.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.102 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:36.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:36.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847494+0000", 2024-09-17T21:20:36.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:20:36.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847504+0000", 2024-09-17T21:20:36.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:20:36.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847526+0000", 2024-09-17T21:20:36.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1976000000000001e-05 2024-09-17T21:20:36.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847541+0000", 2024-09-17T21:20:36.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5277e-05 2024-09-17T21:20:36.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856142+0000", 2024-09-17T21:20:36.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086006710000000007 2024-09-17T21:20:36.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856435+0000", 2024-09-17T21:20:36.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029301499999999999 2024-09-17T21:20:36.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935819+0000", 2024-09-17T21:20:36.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079384448999999996 2024-09-17T21:20:36.106 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.106 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.106 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.106 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2730 3.12 3:48b3422a:::rbd_data.1152a8ae2019.0000000000000057:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:36.106 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:36.106 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 55.191794682000001, 2024-09-17T21:20:36.106 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083879975999999995, 2024-09-17T21:20:36.106 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.106 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.106 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.106 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.106 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:36.106 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:36.107 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2730 2024-09-17T21:20:36.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.107 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.107 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:36.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.107 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:36.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856687+0000", 2024-09-17T21:20:36.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:20:36.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856697+0000", 2024-09-17T21:20:36.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0207000000000001e-05 2024-09-17T21:20:36.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856808+0000", 2024-09-17T21:20:36.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011071200000000001 2024-09-17T21:20:36.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856828+0000", 2024-09-17T21:20:36.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0219000000000001e-05 2024-09-17T21:20:36.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856907+0000", 2024-09-17T21:20:36.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9661999999999997e-05 2024-09-17T21:20:36.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.857019+0000", 2024-09-17T21:20:36.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011148499999999999 2024-09-17T21:20:36.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.110 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.110 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940565+0000", 2024-09-17T21:20:36.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083545791999999994 2024-09-17T21:20:36.110 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.110 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.110 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.110 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2743 3.12 3:4d036e62:::rbd_data.1152a8ae2019.000000000000005f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:36.110 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:36.110 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 55.187849180999997, 2024-09-17T21:20:36.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080059962999999998, 2024-09-17T21:20:36.110 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.110 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.110 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.111 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.111 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:36.111 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:36.111 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2743 2024-09-17T21:20:36.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.111 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:36.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:36.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860631+0000", 2024-09-17T21:20:36.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:20:36.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860636+0000", 2024-09-17T21:20:36.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5279999999999997e-06 2024-09-17T21:20:36.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860649+0000", 2024-09-17T21:20:36.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-17T21:20:36.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860655+0000", 2024-09-17T21:20:36.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:20:36.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860731+0000", 2024-09-17T21:20:36.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5920000000000005e-05 2024-09-17T21:20:36.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.861070+0000", 2024-09-17T21:20:36.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338535 2024-09-17T21:20:36.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940690+0000", 2024-09-17T21:20:36.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079620503999999995 2024-09-17T21:20:36.114 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.114 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.114 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.114 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3378 3.12 3:4832a51b:::rbd_data.1152a8ae2019.000000000000006c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:36.114 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:36.115 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 55.159485330999999, 2024-09-17T21:20:36.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051812516000000003, 2024-09-17T21:20:36.115 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.115 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.115 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:36.115 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.115 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:36.115 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:36.115 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3378 2024-09-17T21:20:36.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.115 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:36.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:36.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888995+0000", 2024-09-17T21:20:36.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:20:36.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889006+0000", 2024-09-17T21:20:36.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1012999999999999e-05 2024-09-17T21:20:36.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889032+0000", 2024-09-17T21:20:36.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6441e-05 2024-09-17T21:20:36.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889040+0000", 2024-09-17T21:20:36.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0639999999999994e-06 2024-09-17T21:20:36.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889278+0000", 2024-09-17T21:20:36.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023785899999999999 2024-09-17T21:20:36.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889408+0000", 2024-09-17T21:20:36.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129219 2024-09-17T21:20:36.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:36.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889723+0000", 2024-09-17T21:20:36.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000315264 2024-09-17T21:20:36.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:36.119 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.891972+0000", 2024-09-17T21:20:36.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002249633 2024-09-17T21:20:36.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.119 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:36.119 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940733+0000", 2024-09-17T21:20:36.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048760626000000001 2024-09-17T21:20:36.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.119 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:36.119 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940785+0000", 2024-09-17T21:20:36.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1906000000000001e-05 2024-09-17T21:20:36.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940807+0000", 2024-09-17T21:20:36.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1554999999999999e-05 2024-09-17T21:20:36.120 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.120 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.120 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.121 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3379 3.12 3:4e83fd71:::rbd_data.1152a8ae2019.000000000000006d:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:36.121 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:36.121 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 55.159401123999999, 2024-09-17T21:20:36.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051804144000000003, 2024-09-17T21:20:36.121 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.121 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.121 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:36.121 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.121 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:36.121 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:36.121 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3379 2024-09-17T21:20:36.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.121 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:36.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:36.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889079+0000", 2024-09-17T21:20:36.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1800000000000004e-07 2024-09-17T21:20:36.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.123 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.123 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889083+0000", 2024-09-17T21:20:36.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:20:36.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.123 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.123 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889094+0000", 2024-09-17T21:20:36.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1562999999999999e-05 2024-09-17T21:20:36.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.123 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.123 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889102+0000", 2024-09-17T21:20:36.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.401e-06 2024-09-17T21:20:36.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.124 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.124 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:20:36.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014279259999999999 2024-09-17T21:20:36.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.124 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.124 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890609+0000", 2024-09-17T21:20:36.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9566000000000004e-05 2024-09-17T21:20:36.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.124 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:36.124 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890789+0000", 2024-09-17T21:20:36.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017949700000000001 2024-09-17T21:20:36.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:36.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.893929+0000", 2024-09-17T21:20:36.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003140538 2024-09-17T21:20:36.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:36.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940834+0000", 2024-09-17T21:20:36.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046904811999999997 2024-09-17T21:20:36.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:36.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940860+0000", 2024-09-17T21:20:36.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5341999999999999e-05 2024-09-17T21:20:36.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940882+0000", 2024-09-17T21:20:36.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2878999999999999e-05 2024-09-17T21:20:36.126 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.126 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.126 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.126 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:2644 3.1e 3:79615585:::rbd_data.115266d0b9f2.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2475520~14848,write 3106304~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:20:36.127 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:36.127 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 46.536772720999998, 2024-09-17T21:20:36.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050733484000000002, 2024-09-17T21:20:36.127 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.127 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.127 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:36.127 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.127 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:20:36.127 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:20:36.127 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2644 2024-09-17T21:20:36.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.127 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:36.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:36.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511708+0000", 2024-09-17T21:20:36.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-17T21:20:36.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.129 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.129 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522196+0000", 2024-09-17T21:20:36.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010487712999999999 2024-09-17T21:20:36.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.129 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.129 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522223+0000", 2024-09-17T21:20:36.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7064999999999999e-05 2024-09-17T21:20:36.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522236+0000", 2024-09-17T21:20:36.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2931e-05 2024-09-17T21:20:36.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522348+0000", 2024-09-17T21:20:36.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111906 2024-09-17T21:20:36.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522554+0000", 2024-09-17T21:20:36.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020632699999999999 2024-09-17T21:20:36.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:20:36.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.523185+0000", 2024-09-17T21:20:36.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063081400000000005 2024-09-17T21:20:36.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:36.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.547011+0000", 2024-09-17T21:20:36.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023825570000000001 2024-09-17T21:20:36.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:36.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562268+0000", 2024-09-17T21:20:36.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015257514 2024-09-17T21:20:36.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:36.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562367+0000", 2024-09-17T21:20:36.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8678000000000006e-05 2024-09-17T21:20:36.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562440+0000", 2024-09-17T21:20:36.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3337000000000006e-05 2024-09-17T21:20:36.133 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.133 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.133 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.133 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:231 3.1 3:856fed78:::rbd_data.115220dd393e.000000000000000b:head [read 0~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:20:36.133 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:20:36.133 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.094017451999999, 2024-09-17T21:20:36.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34254251699999999, 2024-09-17T21:20:36.133 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.133 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.133 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:20:36.133 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.133 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:36.134 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:36.134 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 231 2024-09-17T21:20:36.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.134 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:20:36.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:20:36.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954464+0000", 2024-09-17T21:20:36.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:20:36.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954475+0000", 2024-09-17T21:20:36.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1146e-05 2024-09-17T21:20:36.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954506+0000", 2024-09-17T21:20:36.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0292999999999999e-05 2024-09-17T21:20:36.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954524+0000", 2024-09-17T21:20:36.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8430000000000001e-05 2024-09-17T21:20:36.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954661+0000", 2024-09-17T21:20:36.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136403 2024-09-17T21:20:36.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:36.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954680+0000", 2024-09-17T21:20:36.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9743000000000001e-05 2024-09-17T21:20:36.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.137 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.137 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.296896+0000", 2024-09-17T21:20:36.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34221550499999998 2024-09-17T21:20:36.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.137 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.137 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297004+0000", 2024-09-17T21:20:36.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010865399999999999 2024-09-17T21:20:36.137 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.137 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.137 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.138 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.138 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:56 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:20:36.138 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:20:36.138 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 17.938383755, 2024-09-17T21:20:36.138 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18718773899999999, 2024-09-17T21:20:36.138 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.138 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.138 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.138 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.138 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:20:36.138 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:20:36.138 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 56 2024-09-17T21:20:36.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.138 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:20:36.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:20:36.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110098+0000", 2024-09-17T21:20:36.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:20:36.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.140 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110111+0000", 2024-09-17T21:20:36.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2289999999999999e-05 2024-09-17T21:20:36.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.140 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110143+0000", 2024-09-17T21:20:36.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1979e-05 2024-09-17T21:20:36.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.140 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110161+0000", 2024-09-17T21:20:36.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7839e-05 2024-09-17T21:20:36.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110319+0000", 2024-09-17T21:20:36.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015835400000000001 2024-09-17T21:20:36.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:36.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110353+0000", 2024-09-17T21:20:36.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3849000000000001e-05 2024-09-17T21:20:36.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297041+0000", 2024-09-17T21:20:36.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18668842299999999 2024-09-17T21:20:36.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297169+0000", 2024-09-17T21:20:36.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012786199999999999 2024-09-17T21:20:36.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297283+0000", 2024-09-17T21:20:36.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011430300000000001 2024-09-17T21:20:36.142 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.142 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.142 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.143 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:68 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:20:36.143 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:20:36.143 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 17.929842824000001, 2024-09-17T21:20:36.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17885582999999999, 2024-09-17T21:20:36.143 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.143 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.143 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.143 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.143 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:20:36.143 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:20:36.143 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 68 2024-09-17T21:20:36.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.144 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:20:36.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:20:36.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118639+0000", 2024-09-17T21:20:36.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.306e-06 2024-09-17T21:20:36.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118651+0000", 2024-09-17T21:20:36.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2567e-05 2024-09-17T21:20:36.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118682+0000", 2024-09-17T21:20:36.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0502e-05 2024-09-17T21:20:36.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118707+0000", 2024-09-17T21:20:36.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5463e-05 2024-09-17T21:20:36.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.146 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118889+0000", 2024-09-17T21:20:36.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018150999999999999 2024-09-17T21:20:36.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.146 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:36.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118910+0000", 2024-09-17T21:20:36.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:20:36.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.146 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297313+0000", 2024-09-17T21:20:36.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17840366699999999 2024-09-17T21:20:36.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.147 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297389+0000", 2024-09-17T21:20:36.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-17T21:20:36.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.147 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297492+0000", 2024-09-17T21:20:36.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103019 2024-09-17T21:20:36.147 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.147 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.147 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.148 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.148 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.148 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:36.149 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:20:36.150 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:36.144+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:36.250 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:36.242+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:36.330 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:36.330 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:36.330 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:36.330 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:36.344 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:20:36.351 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:36.346+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:36.436 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-17T21:20:36.452 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:36.446+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:36.477 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:36.477 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:36.477 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:36.478 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:36.478 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:36.478 INFO:teuthology.orchestra.run.smithi184.stdout:5170 read 0x56bb600 thru 0x56c5f75 (0xa976 bytes) 2024-09-17T21:20:36.482 INFO:teuthology.orchestra.run.smithi184.stdout:5171 write 0x1748800 thru 0x17495d2 (0xdd3 bytes) 2024-09-17T21:20:36.483 INFO:teuthology.orchestra.run.smithi184.stdout:5173 write 0x6c19200 thru 0x6c27854 (0xe655 bytes) 2024-09-17T21:20:36.485 INFO:teuthology.orchestra.run.smithi184.stdout:5174 clone 35 order 19 su 32768 sc 7 2024-09-17T21:20:36.492 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:20:36.553 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:36.546+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:36.637 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:20:36.652 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:36.652 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:36.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:36.653 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:36.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.653 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:20:36.653 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:36.653 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 116.839178624, 2024-09-17T21:20:36.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:20:36.653 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.653 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.653 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.653 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.653 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:20:36.653 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:20:36.654 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:20:36.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.654 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:36.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:36.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:20:36.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:20:36.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:20:36.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:20:36.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:20:36.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:20:36.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:20:36.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:20:36.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:20:36.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:20:36.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:36.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:20:36.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:20:36.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:20:36.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:20:36.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:20:36.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:20:36.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:20:36.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:20:36.657 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.657 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.658 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.658 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:20:36.658 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:36.658 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 103.284076838, 2024-09-17T21:20:36.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:20:36.658 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.658 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.658 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:36.658 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.658 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:20:36.658 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:36.659 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:20:36.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.659 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:36.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:36.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:20:36.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:20:36.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:20:36.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:20:36.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:20:36.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:20:36.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:20:36.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:20:36.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:20:36.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:20:36.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:36.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:20:36.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:20:36.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:20:36.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:20:36.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:20:36.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:20:36.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:36.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:20:36.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:20:36.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:36.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:20:36.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:20:36.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:36.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:20:36.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:20:36.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:36.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:20:36.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:20:36.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:20:36.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:20:36.664 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.664 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.664 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.664 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:793 3.9 3:95f63599:::rbd_data.1152eb799094.0000000000000007:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:20:36.665 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:20:36.665 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 20.217020683000001, 2024-09-17T21:20:36.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88245766400000003, 2024-09-17T21:20:36.665 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.665 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.665 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:36.665 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.665 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:20:36.665 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:20:36.665 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 793 2024-09-17T21:20:36.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.665 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:20:36.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:20:36.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431941+0000", 2024-09-17T21:20:36.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.864e-06 2024-09-17T21:20:36.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431953+0000", 2024-09-17T21:20:36.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1486e-05 2024-09-17T21:20:36.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431984+0000", 2024-09-17T21:20:36.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1075999999999999e-05 2024-09-17T21:20:36.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431999+0000", 2024-09-17T21:20:36.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.503e-05 2024-09-17T21:20:36.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432095+0000", 2024-09-17T21:20:36.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5698000000000004e-05 2024-09-17T21:20:36.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:36.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432109+0000", 2024-09-17T21:20:36.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4188e-05 2024-09-17T21:20:36.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.309642+0000", 2024-09-17T21:20:36.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87753353000000001 2024-09-17T21:20:36.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310104+0000", 2024-09-17T21:20:36.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046103600000000001 2024-09-17T21:20:36.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:36.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310630+0000", 2024-09-17T21:20:36.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052682800000000002 2024-09-17T21:20:36.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:36.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313100+0000", 2024-09-17T21:20:36.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024693369999999998 2024-09-17T21:20:36.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:36.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313935+0000", 2024-09-17T21:20:36.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083540100000000005 2024-09-17T21:20:36.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:36.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314086+0000", 2024-09-17T21:20:36.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015053399999999999 2024-09-17T21:20:36.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314396+0000", 2024-09-17T21:20:36.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031065599999999997 2024-09-17T21:20:36.671 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.671 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.671 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.671 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:800 3.19 3:9dc17f12:::rbd_data.1152eb799094.000000000000000e:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:20:36.671 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:20:36.671 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 20.216773438000001, 2024-09-17T21:20:36.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88154663799999999, 2024-09-17T21:20:36.671 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.671 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.672 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:36.672 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.672 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:20:36.672 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:20:36.672 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 800 2024-09-17T21:20:36.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.672 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.672 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:20:36.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:20:36.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432187+0000", 2024-09-17T21:20:36.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4100000000000001e-06 2024-09-17T21:20:36.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432194+0000", 2024-09-17T21:20:36.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0389999999999999e-06 2024-09-17T21:20:36.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432216+0000", 2024-09-17T21:20:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1614e-05 2024-09-17T21:20:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432232+0000", 2024-09-17T21:20:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5739e-05 2024-09-17T21:20:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432405+0000", 2024-09-17T21:20:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017355300000000001 2024-09-17T21:20:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432432+0000", 2024-09-17T21:20:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6979e-05 2024-09-17T21:20:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.307767+0000", 2024-09-17T21:20:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87533472899999998 2024-09-17T21:20:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308192+0000", 2024-09-17T21:20:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042485999999999999 2024-09-17T21:20:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308727+0000", 2024-09-17T21:20:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00053550400000000001 2024-09-17T21:20:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310347+0000", 2024-09-17T21:20:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016198670000000001 2024-09-17T21:20:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313424+0000", 2024-09-17T21:20:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030764640000000001 2024-09-17T21:20:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313510+0000", 2024-09-17T21:20:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5938000000000003e-05 2024-09-17T21:20:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313733+0000", 2024-09-17T21:20:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022294200000000001 2024-09-17T21:20:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:805 3.19 3:9cab2ed6:::rbd_data.1152eb799094.0000000000000013:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:20:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:20:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 20.216569418999999, 2024-09-17T21:20:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88274355699999996, 2024-09-17T21:20:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:20:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:20:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 805 2024-09-17T21:20:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:20:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:20:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432391+0000", 2024-09-17T21:20:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3e-06 2024-09-17T21:20:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432398+0000", 2024-09-17T21:20:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4489999999999996e-06 2024-09-17T21:20:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432415+0000", 2024-09-17T21:20:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7750000000000001e-05 2024-09-17T21:20:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432430+0000", 2024-09-17T21:20:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.449e-05 2024-09-17T21:20:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432635+0000", 2024-09-17T21:20:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205031 2024-09-17T21:20:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432657+0000", 2024-09-17T21:20:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2064000000000001e-05 2024-09-17T21:20:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310444+0000", 2024-09-17T21:20:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87778699199999999 2024-09-17T21:20:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310849+0000", 2024-09-17T21:20:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00040497800000000001 2024-09-17T21:20:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.311396+0000", 2024-09-17T21:20:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00054729099999999999 2024-09-17T21:20:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313847+0000", 2024-09-17T21:20:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024504940000000001 2024-09-17T21:20:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314879+0000", 2024-09-17T21:20:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001031817 2024-09-17T21:20:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314964+0000", 2024-09-17T21:20:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5685000000000004e-05 2024-09-17T21:20:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.315133+0000", 2024-09-17T21:20:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000169216 2024-09-17T21:20:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12087 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 52224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:20:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.879480741, 2024-09-17T21:20:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087684460000000006, 2024-09-17T21:20:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12087 2024-09-17T21:20:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:20:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:20:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769480+0000", 2024-09-17T21:20:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:20:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769485+0000", 2024-09-17T21:20:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8880000000000002e-06 2024-09-17T21:20:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769509+0000", 2024-09-17T21:20:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3343e-05 2024-09-17T21:20:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769521+0000", 2024-09-17T21:20:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2497000000000001e-05 2024-09-17T21:20:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.856883+0000", 2024-09-17T21:20:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087362191000000006 2024-09-17T21:20:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857014+0000", 2024-09-17T21:20:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013090399999999999 2024-09-17T21:20:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857163+0000", 2024-09-17T21:20:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014864000000000001 2024-09-17T21:20:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12088 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 56320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:20:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.879397086000001, 2024-09-17T21:20:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087917772000000005, 2024-09-17T21:20:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12088 2024-09-17T21:20:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:20:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:20:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769569+0000", 2024-09-17T21:20:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:20:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769574+0000", 2024-09-17T21:20:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4659999999999998e-06 2024-09-17T21:20:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769586+0000", 2024-09-17T21:20:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2311e-05 2024-09-17T21:20:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769597+0000", 2024-09-17T21:20:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0548e-05 2024-09-17T21:20:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857217+0000", 2024-09-17T21:20:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087619896000000003 2024-09-17T21:20:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857335+0000", 2024-09-17T21:20:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118538 2024-09-17T21:20:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857480+0000", 2024-09-17T21:20:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014470999999999999 2024-09-17T21:20:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12089 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 60416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:20:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.879344387, 2024-09-17T21:20:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088293233999999998, 2024-09-17T21:20:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12089 2024-09-17T21:20:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:20:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:20:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769616+0000", 2024-09-17T21:20:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:20:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769620+0000", 2024-09-17T21:20:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5140000000000002e-06 2024-09-17T21:20:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769633+0000", 2024-09-17T21:20:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2849e-05 2024-09-17T21:20:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769644+0000", 2024-09-17T21:20:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1471000000000001e-05 2024-09-17T21:20:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857564+0000", 2024-09-17T21:20:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087919445999999998 2024-09-17T21:20:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857716+0000", 2024-09-17T21:20:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000152065 2024-09-17T21:20:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857908+0000", 2024-09-17T21:20:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019252599999999999 2024-09-17T21:20:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12090 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 64512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:20:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.879298002000001, 2024-09-17T21:20:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088582672000000001, 2024-09-17T21:20:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12090 2024-09-17T21:20:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:20:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:20:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769662+0000", 2024-09-17T21:20:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:20:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769683+0000", 2024-09-17T21:20:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0979e-05 2024-09-17T21:20:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769699+0000", 2024-09-17T21:20:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5446000000000001e-05 2024-09-17T21:20:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769708+0000", 2024-09-17T21:20:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1679999999999993e-06 2024-09-17T21:20:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857976+0000", 2024-09-17T21:20:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088267795999999996 2024-09-17T21:20:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858095+0000", 2024-09-17T21:20:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119147 2024-09-17T21:20:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858244+0000", 2024-09-17T21:20:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014917000000000001 2024-09-17T21:20:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12091 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 68608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:20:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.879226618000001, 2024-09-17T21:20:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088832752000000001, 2024-09-17T21:20:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12091 2024-09-17T21:20:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:20:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:20:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769734+0000", 2024-09-17T21:20:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:20:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769739+0000", 2024-09-17T21:20:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8080000000000004e-06 2024-09-17T21:20:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769764+0000", 2024-09-17T21:20:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4839999999999999e-05 2024-09-17T21:20:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769777+0000", 2024-09-17T21:20:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3037000000000001e-05 2024-09-17T21:20:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858295+0000", 2024-09-17T21:20:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088518475999999999 2024-09-17T21:20:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858419+0000", 2024-09-17T21:20:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012378700000000001 2024-09-17T21:20:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858565+0000", 2024-09-17T21:20:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014630099999999999 2024-09-17T21:20:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12092 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:20:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.879144294, 2024-09-17T21:20:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089191555000000006, 2024-09-17T21:20:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12092 2024-09-17T21:20:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:20:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:20:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769816+0000", 2024-09-17T21:20:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7199999999999997e-07 2024-09-17T21:20:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769822+0000", 2024-09-17T21:20:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:20:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769838+0000", 2024-09-17T21:20:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6674e-05 2024-09-17T21:20:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769851+0000", 2024-09-17T21:20:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2527999999999999e-05 2024-09-17T21:20:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858660+0000", 2024-09-17T21:20:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088808556999999996 2024-09-17T21:20:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858817+0000", 2024-09-17T21:20:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015730400000000001 2024-09-17T21:20:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859007+0000", 2024-09-17T21:20:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018974899999999999 2024-09-17T21:20:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12093 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:20:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.879036190000001, 2024-09-17T21:20:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089429710999999995, 2024-09-17T21:20:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12093 2024-09-17T21:20:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:20:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:20:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769924+0000", 2024-09-17T21:20:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.342e-06 2024-09-17T21:20:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769929+0000", 2024-09-17T21:20:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0350000000000001e-06 2024-09-17T21:20:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769946+0000", 2024-09-17T21:20:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6793999999999999e-05 2024-09-17T21:20:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769959+0000", 2024-09-17T21:20:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3062e-05 2024-09-17T21:20:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859075+0000", 2024-09-17T21:20:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089115696999999994 2024-09-17T21:20:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859202+0000", 2024-09-17T21:20:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012657 2024-09-17T21:20:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859353+0000", 2024-09-17T21:20:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151211 2024-09-17T21:20:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12094 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:20:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.878982014, 2024-09-17T21:20:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089695001999999996, 2024-09-17T21:20:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12094 2024-09-17T21:20:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:20:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:20:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769978+0000", 2024-09-17T21:20:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.142e-06 2024-09-17T21:20:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769983+0000", 2024-09-17T21:20:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.386e-06 2024-09-17T21:20:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769997+0000", 2024-09-17T21:20:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.416e-05 2024-09-17T21:20:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770012+0000", 2024-09-17T21:20:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5384999999999999e-05 2024-09-17T21:20:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859405+0000", 2024-09-17T21:20:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089392376999999995 2024-09-17T21:20:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859525+0000", 2024-09-17T21:20:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120555 2024-09-17T21:20:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859672+0000", 2024-09-17T21:20:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000146997 2024-09-17T21:20:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12095 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:20:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.878929527, 2024-09-17T21:20:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090073476, 2024-09-17T21:20:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12095 2024-09-17T21:20:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:20:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:20:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770031+0000", 2024-09-17T21:20:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1699999999999997e-07 2024-09-17T21:20:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770036+0000", 2024-09-17T21:20:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.023e-06 2024-09-17T21:20:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770051+0000", 2024-09-17T21:20:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4953999999999999e-05 2024-09-17T21:20:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770063+0000", 2024-09-17T21:20:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:20:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859756+0000", 2024-09-17T21:20:36.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089692980000000005 2024-09-17T21:20:36.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859891+0000", 2024-09-17T21:20:36.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013459499999999999 2024-09-17T21:20:36.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860103+0000", 2024-09-17T21:20:36.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021274400000000001 2024-09-17T21:20:36.724 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.724 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.725 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.725 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12096 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:36.725 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:20:36.725 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.878879842, 2024-09-17T21:20:36.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090358274000000002, 2024-09-17T21:20:36.725 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.725 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.725 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.725 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.725 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:36.725 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:36.726 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12096 2024-09-17T21:20:36.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.726 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:20:36.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:20:36.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770080+0000", 2024-09-17T21:20:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9400000000000004e-07 2024-09-17T21:20:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770085+0000", 2024-09-17T21:20:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2169999999999996e-06 2024-09-17T21:20:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770100+0000", 2024-09-17T21:20:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5077e-05 2024-09-17T21:20:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770112+0000", 2024-09-17T21:20:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1951e-05 2024-09-17T21:20:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860160+0000", 2024-09-17T21:20:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090048413999999993 2024-09-17T21:20:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860285+0000", 2024-09-17T21:20:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012466699999999999 2024-09-17T21:20:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860438+0000", 2024-09-17T21:20:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015305399999999999 2024-09-17T21:20:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12097 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:20:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.878832095, 2024-09-17T21:20:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090702375000000002, 2024-09-17T21:20:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12097 2024-09-17T21:20:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:20:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:20:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770128+0000", 2024-09-17T21:20:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.04e-06 2024-09-17T21:20:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770133+0000", 2024-09-17T21:20:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8999999999999997e-06 2024-09-17T21:20:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770152+0000", 2024-09-17T21:20:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8890999999999999e-05 2024-09-17T21:20:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770164+0000", 2024-09-17T21:20:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1769e-05 2024-09-17T21:20:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860511+0000", 2024-09-17T21:20:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090347527999999996 2024-09-17T21:20:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860652+0000", 2024-09-17T21:20:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140894 2024-09-17T21:20:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860830+0000", 2024-09-17T21:20:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017735299999999999 2024-09-17T21:20:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12098 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:20:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.878778626999999, 2024-09-17T21:20:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090997262999999995, 2024-09-17T21:20:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12098 2024-09-17T21:20:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:20:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:20:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775465+0000", 2024-09-17T21:20:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0052843350000000002 2024-09-17T21:20:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775476+0000", 2024-09-17T21:20:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0797e-05 2024-09-17T21:20:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775506+0000", 2024-09-17T21:20:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9709999999999998e-05 2024-09-17T21:20:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775530+0000", 2024-09-17T21:20:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4528999999999998e-05 2024-09-17T21:20:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860894+0000", 2024-09-17T21:20:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.085364195000000004 2024-09-17T21:20:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861019+0000", 2024-09-17T21:20:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012467399999999999 2024-09-17T21:20:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861178+0000", 2024-09-17T21:20:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159023 2024-09-17T21:20:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12104 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:20:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.873077518000001, 2024-09-17T21:20:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087969628999999994, 2024-09-17T21:20:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12104 2024-09-17T21:20:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:20:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:20:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775883+0000", 2024-09-17T21:20:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3399999999999998e-07 2024-09-17T21:20:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775886+0000", 2024-09-17T21:20:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2119999999999999e-06 2024-09-17T21:20:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775895+0000", 2024-09-17T21:20:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8710000000000003e-06 2024-09-17T21:20:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775901+0000", 2024-09-17T21:20:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:20:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863544+0000", 2024-09-17T21:20:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087642642000000007 2024-09-17T21:20:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863694+0000", 2024-09-17T21:20:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014952300000000001 2024-09-17T21:20:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863851+0000", 2024-09-17T21:20:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000157804 2024-09-17T21:20:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12105 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.873048624000001, 2024-09-17T21:20:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088330400000000003, 2024-09-17T21:20:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12105 2024-09-17T21:20:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9999999999999997e-07 2024-09-17T21:20:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775914+0000", 2024-09-17T21:20:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4710000000000001e-06 2024-09-17T21:20:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775921+0000", 2024-09-17T21:20:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:20:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775928+0000", 2024-09-17T21:20:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:20:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863923+0000", 2024-09-17T21:20:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087995126000000007 2024-09-17T21:20:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864059+0000", 2024-09-17T21:20:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135797 2024-09-17T21:20:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864241+0000", 2024-09-17T21:20:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018252599999999999 2024-09-17T21:20:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12106 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:20:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.872959445999999, 2024-09-17T21:20:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088586394999999998, 2024-09-17T21:20:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12106 2024-09-17T21:20:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:20:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:20:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776001+0000", 2024-09-17T21:20:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4300000000000001e-07 2024-09-17T21:20:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776004+0000", 2024-09-17T21:20:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8049999999999998e-06 2024-09-17T21:20:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776018+0000", 2024-09-17T21:20:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4613999999999999e-05 2024-09-17T21:20:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776025+0000", 2024-09-17T21:20:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:20:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864279+0000", 2024-09-17T21:20:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088253570000000003 2024-09-17T21:20:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864433+0000", 2024-09-17T21:20:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015469400000000001 2024-09-17T21:20:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864586+0000", 2024-09-17T21:20:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000153003 2024-09-17T21:20:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:36.751 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:36.753 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:20:36.753 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:36.748+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:36.789 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:36.789 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:36.808 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:36.808 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:36.825 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:36.826 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:36.847 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:36.847 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:36.854 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:36.846+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:36.869 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:36.869 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:36.890 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:36.890 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:36.907 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:36.908 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:36.922 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:20:36.965 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:36.960+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:37.056 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:37.048+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:37.094 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:37.094 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:37.108 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:20:37.197 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:37.192+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:37.257 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:37.252+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:37.281 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:37.281 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:37.299 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:20:37.358 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:37.352+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:37.412 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 0, 5] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:20:37.412 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:20:37.413 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:20:37.449 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:37.525 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:37.526 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:20:37.527 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:37.520+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:37.628 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:37.620+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:37.661 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:37.661 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:37.673 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:20:37.728 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:37.722+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:37.734 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:20:37.734 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T21:20:37.753 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-17T21:20:37.575151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1358337871,"num_objects":1509,"num_object_clones":406,"num_object_copies":3018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1146,"num_objects_misplaced":67,"num_objects_unfound":0,"num_objects_dirty":1509,"num_whiteouts":536,"num_read":234312,"num_read_kb":872447,"num_write":14309,"num_write_kb":323375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2146,"num_bytes_recovered":170064687,"num_keys_recovered":457,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12165,"ondisk_log_size":12165,"up":82,"acting":68,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":590492,"kb_used_data":398352,"kb_used_omap":7,"kb_used_meta":191928,"kb_avail":471268708,"statfs":{"total":483183820800,"available":482579156992,"internally_reserved":0,"allocated":407912448,"data_stored":613969379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7978,"internal_metadata":196534486},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":117298176,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1146,"num_objects_misplaced":67,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":14477,"num_read_kb":20858,"num_write":287,"num_write_kb":5539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000989"},"pg_stats":[{"pgid":"3.1f","version":"124'407","reported_seq":1282,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.393093+0000","last_change":"2024-09-17T21:19:17.489680+0000","last_active":"2024-09-17T21:20:34.393093+0000","last_peered":"2024-09-17T21:20:34.393093+0000","last_clean":"2024-09-17T21:20:34.393093+0000","last_became_active":"2024-09-17T21:19:17.290274+0000","last_became_peered":"2024-09-17T21:19:17.290274+0000","last_unstale":"2024-09-17T21:20:34.393093+0000","last_undegraded":"2024-09-17T21:20:34.393093+0000","last_fullsized":"2024-09-17T21:20:34.393093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:28:43.625090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088525599999999996,"stat_sum":{"num_bytes":10514247,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":14,"num_read":2736,"num_read_kb":12872,"num_write":573,"num_write_kb":2867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":568483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"127'370","reported_seq":6636,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.393465+0000","last_change":"2024-09-17T21:19:20.291890+0000","last_active":"2024-09-17T21:20:34.393465+0000","last_peered":"2024-09-17T21:20:34.393465+0000","last_clean":"2024-09-17T21:20:34.393465+0000","last_became_active":"2024-09-17T21:19:20.291104+0000","last_became_peered":"2024-09-17T21:19:20.291104+0000","last_unstale":"2024-09-17T21:20:34.393465+0000","last_undegraded":"2024-09-17T21:20:34.393465+0000","last_fullsized":"2024-09-17T21:20:34.393465+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:01:01.191295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013256229999999999,"stat_sum":{"num_bytes":48552606,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":12,"num_read":7122,"num_read_kb":43878,"num_write":355,"num_write_kb":17655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":390798,"num_keys_recovered":27,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"126'532","reported_seq":6632,"reported_epoch":130,"state":"activating+undersized+remapped","last_fresh":"2024-09-17T21:20:34.398919+0000","last_change":"2024-09-17T21:20:34.400846+0000","last_active":"2024-09-17T21:20:32.393431+0000","last_peered":"2024-09-17T21:20:30.384332+0000","last_clean":"2024-09-17T21:20:30.384332+0000","last_became_active":"2024-09-17T21:20:13.296605+0000","last_became_peered":"2024-09-17T21:20:13.296605+0000","last_unstale":"2024-09-17T21:20:34.398919+0000","last_undegraded":"2024-09-17T21:20:34.398919+0000","last_fullsized":"2024-09-17T21:20:34.398919+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018973079999999999,"stat_sum":{"num_bytes":36596915,"num_objects":57,"num_object_clones":15,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":90,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":25,"num_read":8303,"num_read_kb":22785,"num_write":647,"num_write_kb":9899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4085903,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4],"avail_no_missing":["4","1"],"object_location_counts":[{"shards":"4","objects":57}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"127'187","reported_seq":2279,"reported_epoch":130,"state":"activating+undersized+remapped","last_fresh":"2024-09-17T21:20:34.398689+0000","last_change":"2024-09-17T21:20:34.399640+0000","last_active":"2024-09-17T21:20:32.392875+0000","last_peered":"2024-09-17T21:20:32.341012+0000","last_clean":"2024-09-17T21:20:32.341012+0000","last_became_active":"2024-09-17T21:20:12.300882+0000","last_became_peered":"2024-09-17T21:20:12.300882+0000","last_unstale":"2024-09-17T21:20:34.398689+0000","last_undegraded":"2024-09-17T21:20:34.398689+0000","last_fullsized":"2024-09-17T21:20:34.398689+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034186630000000002,"stat_sum":{"num_bytes":22275600,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":72,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":12,"num_read":1815,"num_read_kb":18816,"num_write":184,"num_write_kb":4657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":2451984,"num_keys_recovered":62,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4],"avail_no_missing":["4","1"],"object_location_counts":[{"shards":"4","objects":39}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"124'472","reported_seq":6612,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.392734+0000","last_change":"2024-09-17T21:19:20.291675+0000","last_active":"2024-09-17T21:20:34.392734+0000","last_peered":"2024-09-17T21:20:34.392734+0000","last_clean":"2024-09-17T21:20:34.392734+0000","last_became_active":"2024-09-17T21:19:20.290844+0000","last_became_peered":"2024-09-17T21:19:20.290844+0000","last_unstale":"2024-09-17T21:20:34.392734+0000","last_undegraded":"2024-09-17T21:20:34.392734+0000","last_fullsized":"2024-09-17T21:20:34.392734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:26:48.312422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089504999999999999,"stat_sum":{"num_bytes":27793936,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":20,"num_read":9972,"num_read_kb":19354,"num_write":365,"num_write_kb":4440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":1443328,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"127'417","reported_seq":9377,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.398575+0000","last_change":"2024-09-17T21:18:13.314063+0000","last_active":"2024-09-17T21:20:34.398575+0000","last_peered":"2024-09-17T21:20:34.398575+0000","last_clean":"2024-09-17T21:20:34.398575+0000","last_became_active":"2024-09-17T21:18:11.301426+0000","last_became_peered":"2024-09-17T21:18:11.301426+0000","last_unstale":"2024-09-17T21:20:34.398575+0000","last_undegraded":"2024-09-17T21:20:34.398575+0000","last_fullsized":"2024-09-17T21:20:34.398575+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:27:34.644657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027366090000000001,"stat_sum":{"num_bytes":51602504,"num_objects":54,"num_object_clones":18,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":14,"num_read":11404,"num_read_kb":43410,"num_write":360,"num_write_kb":12879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"127'222","reported_seq":4623,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.400030+0000","last_change":"2024-09-17T21:20:19.296133+0000","last_active":"2024-09-17T21:20:34.400030+0000","last_peered":"2024-09-17T21:20:34.400030+0000","last_clean":"2024-09-17T21:20:34.400030+0000","last_became_active":"2024-09-17T21:20:19.295432+0000","last_became_peered":"2024-09-17T21:20:19.295432+0000","last_unstale":"2024-09-17T21:20:34.400030+0000","last_undegraded":"2024-09-17T21:20:34.400030+0000","last_fullsized":"2024-09-17T21:20:34.400030+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:55:48.131607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047673000000000002,"stat_sum":{"num_bytes":32541743,"num_objects":44,"num_object_clones":14,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":15,"num_read":4081,"num_read_kb":16634,"num_write":238,"num_write_kb":4692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":3100207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"125'229","reported_seq":1873,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.392013+0000","last_change":"2024-09-17T21:19:48.295801+0000","last_active":"2024-09-17T21:20:34.392013+0000","last_peered":"2024-09-17T21:20:34.392013+0000","last_clean":"2024-09-17T21:20:34.392013+0000","last_became_active":"2024-09-17T21:19:48.294966+0000","last_became_peered":"2024-09-17T21:19:48.294966+0000","last_unstale":"2024-09-17T21:20:34.392013+0000","last_undegraded":"2024-09-17T21:20:34.392013+0000","last_fullsized":"2024-09-17T21:20:34.392013+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:42.681615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25414467,"num_objects":51,"num_object_clones":14,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":23,"num_read":2124,"num_read_kb":8424,"num_write":248,"num_write_kb":2827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":1971619,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"123'269","reported_seq":1613,"reported_epoch":130,"state":"activating+undersized+remapped","last_fresh":"2024-09-17T21:20:34.393353+0000","last_change":"2024-09-17T21:20:34.396356+0000","last_active":"2024-09-17T21:20:32.388258+0000","last_peered":"2024-09-17T21:20:30.381849+0000","last_clean":"2024-09-17T21:20:30.381849+0000","last_became_active":"2024-09-17T21:20:12.301912+0000","last_became_peered":"2024-09-17T21:20:12.301912+0000","last_unstale":"2024-09-17T21:20:34.393353+0000","last_undegraded":"2024-09-17T21:20:34.393353+0000","last_fullsized":"2024-09-17T21:20:34.393353+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088962999999999998,"stat_sum":{"num_bytes":43926682,"num_objects":49,"num_object_clones":15,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":82,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":20,"num_read":2411,"num_read_kb":15246,"num_write":392,"num_write_kb":6055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":6599322,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0],"avail_no_missing":["0","4"],"object_location_counts":[{"shards":"0","objects":49}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"127'485","reported_seq":16915,"reported_epoch":130,"state":"activating+undersized+remapped","last_fresh":"2024-09-17T21:20:34.394578+0000","last_change":"2024-09-17T21:20:34.396551+0000","last_active":"2024-09-17T21:20:32.387762+0000","last_peered":"2024-09-17T21:20:31.980751+0000","last_clean":"2024-09-17T21:20:31.980751+0000","last_became_active":"2024-09-17T21:20:12.292429+0000","last_became_peered":"2024-09-17T21:20:12.292429+0000","last_unstale":"2024-09-17T21:20:34.394578+0000","last_undegraded":"2024-09-17T21:20:34.394578+0000","last_fullsized":"2024-09-17T21:20:34.394578+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001825658,"stat_sum":{"num_bytes":22711664,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":70,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":13,"num_read":17893,"num_read_kb":22045,"num_write":754,"num_write_kb":12145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":1089392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"3","objects":36}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"126'605","reported_seq":4749,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.394238+0000","last_change":"2024-09-17T21:19:20.288548+0000","last_active":"2024-09-17T21:20:34.394238+0000","last_peered":"2024-09-17T21:20:34.394238+0000","last_clean":"2024-09-17T21:20:34.394238+0000","last_became_active":"2024-09-17T21:19:20.287118+0000","last_became_peered":"2024-09-17T21:19:20.287118+0000","last_unstale":"2024-09-17T21:20:34.394238+0000","last_undegraded":"2024-09-17T21:20:34.394238+0000","last_fullsized":"2024-09-17T21:20:34.394238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:11:23.776150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027112149999999999,"stat_sum":{"num_bytes":58948720,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":15,"num_read":6007,"num_read_kb":23285,"num_write":850,"num_write_kb":4631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":3174038,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"124'310","reported_seq":3515,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.393895+0000","last_change":"2024-09-17T21:18:13.307234+0000","last_active":"2024-09-17T21:20:34.393895+0000","last_peered":"2024-09-17T21:20:34.393895+0000","last_clean":"2024-09-17T21:20:34.393895+0000","last_became_active":"2024-09-17T21:18:11.299817+0000","last_became_peered":"2024-09-17T21:18:11.299817+0000","last_unstale":"2024-09-17T21:20:34.393895+0000","last_undegraded":"2024-09-17T21:20:34.393895+0000","last_fullsized":"2024-09-17T21:20:34.393895+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:48:29.641143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002281113,"stat_sum":{"num_bytes":44187963,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":18,"num_read":4441,"num_read_kb":19184,"num_write":335,"num_write_kb":14554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":239,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.395449+0000","last_change":"2024-09-17T21:18:03.059715+0000","last_active":"2024-09-17T21:20:34.395449+0000","last_peered":"2024-09-17T21:20:34.395449+0000","last_clean":"2024-09-17T21:20:34.395449+0000","last_became_active":"2024-09-17T21:18:01.213047+0000","last_became_peered":"2024-09-17T21:18:01.213047+0000","last_unstale":"2024-09-17T21:20:34.395449+0000","last_undegraded":"2024-09-17T21:20:34.395449+0000","last_fullsized":"2024-09-17T21:20:34.395449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:46:35.222373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"127'409","reported_seq":5227,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.393620+0000","last_change":"2024-09-17T21:19:20.290612+0000","last_active":"2024-09-17T21:20:34.393620+0000","last_peered":"2024-09-17T21:20:34.393620+0000","last_clean":"2024-09-17T21:20:34.393620+0000","last_became_active":"2024-09-17T21:19:20.289862+0000","last_became_peered":"2024-09-17T21:19:20.289862+0000","last_unstale":"2024-09-17T21:20:34.393620+0000","last_undegraded":"2024-09-17T21:20:34.393620+0000","last_fullsized":"2024-09-17T21:20:34.393620+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:43:45.672528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084327200000000005,"stat_sum":{"num_bytes":49060194,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":15,"num_read":6956,"num_read_kb":44284,"num_write":471,"num_write_kb":22557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":17516408,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":239,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.402346+0000","last_change":"2024-09-17T21:18:03.073440+0000","last_active":"2024-09-17T21:20:34.402346+0000","last_peered":"2024-09-17T21:20:34.402346+0000","last_clean":"2024-09-17T21:20:34.402346+0000","last_became_active":"2024-09-17T21:18:01.048806+0000","last_became_peered":"2024-09-17T21:18:01.048806+0000","last_unstale":"2024-09-17T21:20:34.402346+0000","last_undegraded":"2024-09-17T21:20:34.402346+0000","last_fullsized":"2024-09-17T21:20:34.402346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:26.487163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094161400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"124'204","reported_seq":3902,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.398980+0000","last_change":"2024-09-17T21:19:48.301684+0000","last_active":"2024-09-17T21:20:34.398980+0000","last_peered":"2024-09-17T21:20:34.398980+0000","last_clean":"2024-09-17T21:20:34.398980+0000","last_became_active":"2024-09-17T21:19:48.301027+0000","last_became_peered":"2024-09-17T21:19:48.301027+0000","last_unstale":"2024-09-17T21:20:34.398980+0000","last_undegraded":"2024-09-17T21:20:34.398980+0000","last_fullsized":"2024-09-17T21:20:34.398980+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:02:36.260557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39619183,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":3561,"num_read_kb":32845,"num_write":218,"num_write_kb":12986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":995423,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":251,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.394598+0000","last_change":"2024-09-17T21:19:17.294067+0000","last_active":"2024-09-17T21:20:34.394598+0000","last_peered":"2024-09-17T21:20:34.394598+0000","last_clean":"2024-09-17T21:20:34.394598+0000","last_became_active":"2024-09-17T21:19:17.293357+0000","last_became_peered":"2024-09-17T21:19:17.293357+0000","last_unstale":"2024-09-17T21:20:34.394598+0000","last_undegraded":"2024-09-17T21:20:34.394598+0000","last_fullsized":"2024-09-17T21:20:34.394598+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:13:10.070543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099219900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"125'515","reported_seq":9607,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.410929+0000","last_change":"2024-09-17T21:19:48.296853+0000","last_active":"2024-09-17T21:20:34.410929+0000","last_peered":"2024-09-17T21:20:34.410929+0000","last_clean":"2024-09-17T21:20:34.410929+0000","last_became_active":"2024-09-17T21:19:48.296024+0000","last_became_peered":"2024-09-17T21:19:48.296024+0000","last_unstale":"2024-09-17T21:20:34.410929+0000","last_undegraded":"2024-09-17T21:20:34.410929+0000","last_fullsized":"2024-09-17T21:20:34.410929+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:18:43.112086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078465700000000002,"stat_sum":{"num_bytes":40275522,"num_objects":59,"num_object_clones":17,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":23,"num_read":9133,"num_read_kb":24713,"num_write":468,"num_write_kb":9977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":5833764,"num_keys_recovered":24,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":251,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.411065+0000","last_change":"2024-09-17T21:19:17.294907+0000","last_active":"2024-09-17T21:20:34.411065+0000","last_peered":"2024-09-17T21:20:34.411065+0000","last_clean":"2024-09-17T21:20:34.411065+0000","last_became_active":"2024-09-17T21:19:17.293525+0000","last_became_peered":"2024-09-17T21:19:17.293525+0000","last_unstale":"2024-09-17T21:20:34.411065+0000","last_undegraded":"2024-09-17T21:20:34.411065+0000","last_fullsized":"2024-09-17T21:20:34.411065+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:32:46.627308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006222,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"127'756","reported_seq":31191,"reported_epoch":130,"state":"activating+undersized+remapped","last_fresh":"2024-09-17T21:20:34.393289+0000","last_change":"2024-09-17T21:20:34.396568+0000","last_active":"2024-09-17T21:20:32.389470+0000","last_peered":"2024-09-17T21:20:32.375089+0000","last_clean":"2024-09-17T21:20:32.375089+0000","last_became_active":"2024-09-17T21:20:13.291558+0000","last_became_peered":"2024-09-17T21:20:13.291558+0000","last_unstale":"2024-09-17T21:20:34.393289+0000","last_undegraded":"2024-09-17T21:20:34.393289+0000","last_fullsized":"2024-09-17T21:20:34.393289+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036493720000000001,"stat_sum":{"num_bytes":99611225,"num_objects":71,"num_object_clones":22,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":19,"num_read":35247,"num_read_kb":52255,"num_write":564,"num_write_kb":21501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":158,"num_bytes_recovered":13123192,"num_keys_recovered":68,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"3","objects":71}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"126'216","reported_seq":1853,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.401606+0000","last_change":"2024-09-17T21:19:17.416920+0000","last_active":"2024-09-17T21:20:34.401606+0000","last_peered":"2024-09-17T21:20:34.401606+0000","last_clean":"2024-09-17T21:20:34.401606+0000","last_became_active":"2024-09-17T21:19:17.294385+0000","last_became_peered":"2024-09-17T21:19:17.294385+0000","last_unstale":"2024-09-17T21:20:34.401606+0000","last_undegraded":"2024-09-17T21:20:34.401606+0000","last_fullsized":"2024-09-17T21:20:34.401606+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:37:09.167522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084988700000000002,"stat_sum":{"num_bytes":20513954,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":15,"num_read":1505,"num_read_kb":11688,"num_write":235,"num_write_kb":4639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":2000896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":180,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.400811+0000","last_change":"2024-09-17T21:20:33.408746+0000","last_active":"2024-09-17T21:20:34.400811+0000","last_peered":"2024-09-17T21:20:34.400811+0000","last_clean":"2024-09-17T21:20:34.400811+0000","last_became_active":"2024-09-17T21:20:33.401029+0000","last_became_peered":"2024-09-17T21:20:33.401029+0000","last_unstale":"2024-09-17T21:20:34.400811+0000","last_undegraded":"2024-09-17T21:20:34.400811+0000","last_fullsized":"2024-09-17T21:20:34.400811+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:23:16.392612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"127'145","reported_seq":2014,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.398145+0000","last_change":"2024-09-17T21:20:19.297215+0000","last_active":"2024-09-17T21:20:34.398145+0000","last_peered":"2024-09-17T21:20:34.398145+0000","last_clean":"2024-09-17T21:20:34.398145+0000","last_became_active":"2024-09-17T21:20:19.296229+0000","last_became_peered":"2024-09-17T21:20:19.296229+0000","last_unstale":"2024-09-17T21:20:34.398145+0000","last_undegraded":"2024-09-17T21:20:34.398145+0000","last_fullsized":"2024-09-17T21:20:34.398145+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:15:52.834395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049125000000000004,"stat_sum":{"num_bytes":43285024,"num_objects":41,"num_object_clones":14,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":11,"num_read":1423,"num_read_kb":14082,"num_write":155,"num_write_kb":6276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":6536768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":223,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.400204+0000","last_change":"2024-09-17T21:19:45.702432+0000","last_active":"2024-09-17T21:20:34.400204+0000","last_peered":"2024-09-17T21:20:34.400204+0000","last_clean":"2024-09-17T21:20:34.400204+0000","last_became_active":"2024-09-17T21:19:45.701525+0000","last_became_peered":"2024-09-17T21:19:45.701525+0000","last_unstale":"2024-09-17T21:20:34.400204+0000","last_undegraded":"2024-09-17T21:20:34.400204+0000","last_fullsized":"2024-09-17T21:20:34.400204+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:39:24.504928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"124'892","reported_seq":3121,"reported_epoch":130,"state":"activating+undersized+remapped","last_fresh":"2024-09-17T21:20:34.392245+0000","last_change":"2024-09-17T21:20:34.400517+0000","last_active":"2024-09-17T21:20:32.387486+0000","last_peered":"2024-09-17T21:20:31.459937+0000","last_clean":"2024-09-17T21:20:31.459937+0000","last_became_active":"2024-09-17T21:20:18.295908+0000","last_became_peered":"2024-09-17T21:20:18.295908+0000","last_unstale":"2024-09-17T21:20:34.392245+0000","last_undegraded":"2024-09-17T21:20:34.392245+0000","last_fullsized":"2024-09-17T21:20:34.392245+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":892,"log_dups_size":0,"ondisk_log_size":892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066398399999999995,"stat_sum":{"num_bytes":18265234,"num_objects":48,"num_object_clones":14,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":81,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":12,"num_read":5633,"num_read_kb":31556,"num_write":1442,"num_write_kb":8236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":13794544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0","objects":48}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":239,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.393213+0000","last_change":"2024-09-17T21:18:03.059156+0000","last_active":"2024-09-17T21:20:34.393213+0000","last_peered":"2024-09-17T21:20:34.393213+0000","last_clean":"2024-09-17T21:20:34.393213+0000","last_became_active":"2024-09-17T21:18:01.049180+0000","last_became_peered":"2024-09-17T21:18:01.049180+0000","last_unstale":"2024-09-17T21:20:34.393213+0000","last_undegraded":"2024-09-17T21:20:34.393213+0000","last_fullsized":"2024-09-17T21:20:34.393213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:27.069794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077408,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"127'452","reported_seq":11865,"reported_epoch":130,"state":"activating+undersized+remapped","last_fresh":"2024-09-17T21:20:34.395234+0000","last_change":"2024-09-17T21:20:34.400244+0000","last_active":"2024-09-17T21:20:32.390588+0000","last_peered":"2024-09-17T21:20:32.382902+0000","last_clean":"2024-09-17T21:20:32.382902+0000","last_became_active":"2024-09-17T21:20:12.289188+0000","last_became_peered":"2024-09-17T21:20:12.289188+0000","last_unstale":"2024-09-17T21:20:34.395234+0000","last_undegraded":"2024-09-17T21:20:34.395234+0000","last_fullsized":"2024-09-17T21:20:34.395234+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049408099999999995,"stat_sum":{"num_bytes":63075193,"num_objects":47,"num_object_clones":13,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":84,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":21,"num_read":12937,"num_read_kb":26746,"num_write":556,"num_write_kb":10108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":9004409,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1","objects":47}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":109,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.397738+0000","last_change":"2024-09-17T21:20:33.413969+0000","last_active":"2024-09-17T21:20:34.397738+0000","last_peered":"2024-09-17T21:20:34.397738+0000","last_clean":"2024-09-17T21:20:34.397738+0000","last_became_active":"2024-09-17T21:20:33.403467+0000","last_became_peered":"2024-09-17T21:20:33.403467+0000","last_unstale":"2024-09-17T21:20:34.397738+0000","last_undegraded":"2024-09-17T21:20:34.397738+0000","last_fullsized":"2024-09-17T21:20:34.397738+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:59:38.585465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":354,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.394004+0000","last_change":"2024-09-17T21:19:17.427717+0000","last_active":"2024-09-17T21:20:34.394004+0000","last_peered":"2024-09-17T21:20:34.394004+0000","last_clean":"2024-09-17T21:20:34.394004+0000","last_became_active":"2024-09-17T21:19:17.297187+0000","last_became_peered":"2024-09-17T21:19:17.297187+0000","last_unstale":"2024-09-17T21:20:34.394004+0000","last_undegraded":"2024-09-17T21:20:34.394004+0000","last_fullsized":"2024-09-17T21:20:34.394004+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_clean_scrub_stamp":"2024-09-17T21:17:55.982339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:19:07.636190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"127'381","reported_seq":7040,"reported_epoch":130,"state":"activating+undersized+remapped","last_fresh":"2024-09-17T21:20:34.395385+0000","last_change":"2024-09-17T21:20:34.401042+0000","last_active":"2024-09-17T21:20:32.391061+0000","last_peered":"2024-09-17T21:20:32.373572+0000","last_clean":"2024-09-17T21:20:32.373572+0000","last_became_active":"2024-09-17T21:20:12.302006+0000","last_became_peered":"2024-09-17T21:20:12.302006+0000","last_unstale":"2024-09-17T21:20:34.395385+0000","last_undegraded":"2024-09-17T21:20:34.395385+0000","last_fullsized":"2024-09-17T21:20:34.395385+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50402901,"num_objects":54,"num_object_clones":15,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":95,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":18,"num_read":7831,"num_read_kb":50562,"num_write":369,"num_write_kb":19023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":13288544,"num_keys_recovered":66,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1],"avail_no_missing":["1","4"],"object_location_counts":[{"shards":"1","objects":54}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"127'558","reported_seq":9181,"reported_epoch":130,"state":"activating+undersized+remapped","last_fresh":"2024-09-17T21:20:34.395034+0000","last_change":"2024-09-17T21:20:34.397802+0000","last_active":"2024-09-17T21:20:32.390728+0000","last_peered":"2024-09-17T21:20:32.374126+0000","last_clean":"2024-09-17T21:20:32.374126+0000","last_became_active":"2024-09-17T21:20:19.296912+0000","last_became_peered":"2024-09-17T21:20:19.296912+0000","last_unstale":"2024-09-17T21:20:34.395034+0000","last_undegraded":"2024-09-17T21:20:34.395034+0000","last_fullsized":"2024-09-17T21:20:34.395034+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023749890000000001,"stat_sum":{"num_bytes":52859208,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":80,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":9981,"num_read_kb":39665,"num_write":988,"num_write_kb":7208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":10779469,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1","objects":43}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"126'241","reported_seq":3371,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.398364+0000","last_change":"2024-09-17T21:19:34.613370+0000","last_active":"2024-09-17T21:20:34.398364+0000","last_peered":"2024-09-17T21:20:34.398364+0000","last_clean":"2024-09-17T21:20:34.398364+0000","last_became_active":"2024-09-17T21:19:34.611967+0000","last_became_peered":"2024-09-17T21:19:34.611967+0000","last_unstale":"2024-09-17T21:20:34.398364+0000","last_undegraded":"2024-09-17T21:20:34.398364+0000","last_fullsized":"2024-09-17T21:20:34.398364+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:13:43.409891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021951829999999999,"stat_sum":{"num_bytes":58398866,"num_objects":52,"num_object_clones":15,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":20,"num_read":2972,"num_read_kb":31447,"num_write":220,"num_write_kb":15999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"127'266","reported_seq":3465,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.397982+0000","last_change":"2024-09-17T21:19:20.292113+0000","last_active":"2024-09-17T21:20:34.397982+0000","last_peered":"2024-09-17T21:20:34.397982+0000","last_clean":"2024-09-17T21:20:34.397982+0000","last_became_active":"2024-09-17T21:19:20.290717+0000","last_became_peered":"2024-09-17T21:19:20.290717+0000","last_unstale":"2024-09-17T21:20:34.397982+0000","last_undegraded":"2024-09-17T21:20:34.397982+0000","last_fullsized":"2024-09-17T21:20:34.397982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:18:50.513258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048704800000000002,"stat_sum":{"num_bytes":46412448,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":13,"num_read":3923,"num_read_kb":22378,"num_write":239,"num_write_kb":13367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":1215535,"num_keys_recovered":36,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"125'230","reported_seq":1281,"reported_epoch":130,"state":"activating+undersized+degraded+remapped","last_fresh":"2024-09-17T21:20:34.400443+0000","last_change":"2024-09-17T21:20:34.411837+0000","last_active":"2024-09-17T21:20:33.389495+0000","last_peered":"2024-09-17T21:20:31.748637+0000","last_clean":"2024-09-17T21:20:31.748637+0000","last_became_active":"2024-09-17T21:20:13.293448+0000","last_became_peered":"2024-09-17T21:20:13.293448+0000","last_unstale":"2024-09-17T21:20:34.400443+0000","last_undegraded":"2024-09-17T21:20:34.400443+0000","last_fullsized":"2024-09-17T21:20:34.392173+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015319789999999999,"stat_sum":{"num_bytes":33081856,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":67,"num_objects_misplaced":67,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":20,"num_read":3527,"num_read_kb":9282,"num_write":412,"num_write_kb":3625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":550912,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[0],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":39}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"125'375","reported_seq":11122,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.398749+0000","last_change":"2024-09-17T21:19:49.303045+0000","last_active":"2024-09-17T21:20:34.398749+0000","last_peered":"2024-09-17T21:20:34.398749+0000","last_clean":"2024-09-17T21:20:34.398749+0000","last_became_active":"2024-09-17T21:19:49.298871+0000","last_became_peered":"2024-09-17T21:19:49.298871+0000","last_unstale":"2024-09-17T21:20:34.398749+0000","last_undegraded":"2024-09-17T21:20:34.398749+0000","last_fullsized":"2024-09-17T21:20:34.398749+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:31:59.751254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015238949999999999,"stat_sum":{"num_bytes":29073529,"num_objects":43,"num_object_clones":15,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":12,"num_read":12283,"num_read_kb":34100,"num_write":529,"num_write_kb":13574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":162846,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"124'379","reported_seq":11158,"reported_epoch":130,"state":"activating+undersized+remapped","last_fresh":"2024-09-17T21:20:34.394814+0000","last_change":"2024-09-17T21:20:34.397446+0000","last_active":"2024-09-17T21:20:32.391075+0000","last_peered":"2024-09-17T21:20:31.434131+0000","last_clean":"2024-09-17T21:20:31.434131+0000","last_became_active":"2024-09-17T21:20:20.308461+0000","last_became_peered":"2024-09-17T21:20:20.308461+0000","last_unstale":"2024-09-17T21:20:34.394814+0000","last_undegraded":"2024-09-17T21:20:34.394814+0000","last_fullsized":"2024-09-17T21:20:34.394814+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003632209,"stat_sum":{"num_bytes":59239099,"num_objects":54,"num_object_clones":17,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":79,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":16,"num_read":11537,"num_read_kb":65001,"num_write":373,"num_write_kb":35158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":41051835,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"1","objects":54}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"126'253","reported_seq":3846,"reported_epoch":130,"state":"activating+undersized+remapped","last_fresh":"2024-09-17T21:20:34.393116+0000","last_change":"2024-09-17T21:20:34.394417+0000","last_active":"2024-09-17T21:20:32.387846+0000","last_peered":"2024-09-17T21:20:30.378295+0000","last_clean":"2024-09-17T21:20:30.378295+0000","last_became_active":"2024-09-17T21:20:12.296084+0000","last_became_peered":"2024-09-17T21:20:12.296084+0000","last_unstale":"2024-09-17T21:20:34.393116+0000","last_undegraded":"2024-09-17T21:20:34.393116+0000","last_fullsized":"2024-09-17T21:20:34.393116+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002454758,"stat_sum":{"num_bytes":37514373,"num_objects":50,"num_object_clones":14,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":86,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":20,"num_read":3972,"num_read_kb":19659,"num_write":276,"num_write_kb":2346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":1546869,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"3","objects":50}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"124'167","reported_seq":1165,"reported_epoch":130,"state":"activating+undersized+remapped","last_fresh":"2024-09-17T21:20:34.392204+0000","last_change":"2024-09-17T21:20:34.393868+0000","last_active":"2024-09-17T21:20:32.386995+0000","last_peered":"2024-09-17T21:20:31.434217+0000","last_clean":"2024-09-17T21:20:31.434217+0000","last_became_active":"2024-09-17T21:20:12.288878+0000","last_became_peered":"2024-09-17T21:20:12.288878+0000","last_unstale":"2024-09-17T21:20:34.392204+0000","last_undegraded":"2024-09-17T21:20:34.392204+0000","last_fullsized":"2024-09-17T21:20:34.392204+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001519261,"stat_sum":{"num_bytes":11071637,"num_objects":29,"num_object_clones":4,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":62,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":10,"num_read":2305,"num_read_kb":10143,"num_write":132,"num_write_kb":2507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":2254633,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0","objects":29}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"127'290","reported_seq":3706,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.393818+0000","last_change":"2024-09-17T21:18:13.305680+0000","last_active":"2024-09-17T21:20:34.393818+0000","last_peered":"2024-09-17T21:20:34.393818+0000","last_clean":"2024-09-17T21:20:34.393818+0000","last_became_active":"2024-09-17T21:18:11.299946+0000","last_became_peered":"2024-09-17T21:18:11.299946+0000","last_unstale":"2024-09-17T21:20:34.393818+0000","last_undegraded":"2024-09-17T21:20:34.393818+0000","last_fullsized":"2024-09-17T21:20:34.393818+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:55:03.420163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225884,"stat_sum":{"num_bytes":53611357,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":15,"num_read":3570,"num_read_kb":14093,"num_write":299,"num_write_kb":3886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"127'358","reported_seq":4570,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T21:20:34.392224+0000","last_change":"2024-09-17T21:19:17.399377+0000","last_active":"2024-09-17T21:20:34.392224+0000","last_peered":"2024-09-17T21:20:34.392224+0000","last_clean":"2024-09-17T21:20:34.392224+0000","last_became_active":"2024-09-17T21:19:17.290182+0000","last_became_peered":"2024-09-17T21:19:17.290182+0000","last_unstale":"2024-09-17T21:20:34.392224+0000","last_undegraded":"2024-09-17T21:20:34.392224+0000","last_fullsized":"2024-09-17T21:20:34.392224+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:23:29.661355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013045489999999999,"stat_sum":{"num_bytes":67581130,"num_objects":62,"num_object_clones":18,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":22,"num_read":6904,"num_read_kb":24026,"num_write":450,"num_write_kb":6754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":2706590,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"124'498","reported_seq":10008,"reported_epoch":130,"state":"activating+undersized+degraded+remapped","last_fresh":"2024-09-17T21:20:34.396456+0000","last_change":"2024-09-17T21:20:34.517705+0000","last_active":"2024-09-17T21:20:32.389430+0000","last_peered":"2024-09-17T21:20:30.378018+0000","last_clean":"2024-09-17T21:20:30.378018+0000","last_became_active":"2024-09-17T21:20:13.297148+0000","last_became_peered":"2024-09-17T21:20:13.297148+0000","last_unstale":"2024-09-17T21:20:34.396456+0000","last_undegraded":"2024-09-17T21:20:34.396456+0000","last_fullsized":"2024-09-17T21:20:34.393421+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019790290000000002,"stat_sum":{"num_bytes":59728504,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":91,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":22,"num_read":10737,"num_read_kb":47934,"num_write":509,"num_write_kb":4717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":1646202,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3],"avail_no_missing":["3","4"],"object_location_counts":[{"shards":"3,4","objects":54}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1357747484,"num_objects":1505,"num_object_clones":406,"num_object_copies":3010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1146,"num_objects_misplaced":67,"num_objects_unfound":0,"num_objects_dirty":1505,"num_whiteouts":536,"num_read":234246,"num_read_kb":872392,"num_write":14206,"num_write_kb":321745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2137,"num_bytes_recovered":168883913,"num_keys_recovered":457,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":482037760,"data_stored":737894615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12090,"ondisk_log_size":12090,"up":64,"acting":50,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28528,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343312,"statfs":{"total":96636764160,"available":96607551488,"internally_reserved":0,"allocated":1527808,"data_stored":625236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":138900,"kb_used_data":85500,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94232940,"statfs":{"total":96636764160,"available":96494530560,"internally_reserved":0,"allocated":87552000,"data_stored":138477077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145332,"kb_used_data":91892,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94226508,"statfs":{"total":96636764160,"available":96487944192,"internally_reserved":0,"allocated":94097408,"data_stored":146534191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":142988,"kb_used_data":111796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94228852,"statfs":{"total":96636764160,"available":96490344448,"internally_reserved":0,"allocated":114479104,"data_stored":179821011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":134744,"kb_used_data":107672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94237096,"statfs":{"total":96636764160,"available":96498786304,"internally_reserved":0,"allocated":110256128,"data_stored":148511864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":51539607588,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":108138496,"data_stored":147296279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112951296,"data_stored":179195775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13430784,"data_stored":18128957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":91979776,"data_stored":145318606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86028288,"data_stored":137851860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":69509120,"data_stored":110103138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:20:37.755 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.c 2024-09-17T21:20:37.798 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:37.798 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:37.813 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:20:37.859 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:37.854+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:37.929 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:37.922+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:38.005 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:38.006 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:38.022 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:20:38.030 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:38.024+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:38.112 INFO:teuthology.orchestra.run.smithi083.stderr:instructing pg(s) [3.c] on osd.0 to force-recovery; 2024-09-17T21:20:38.112 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-09-17T21:20:38.131 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:38.126+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:38.194 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:38.195 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:38.211 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:20:38.259 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:38.254+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:38.334 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:38.328+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:38.401 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:38.401 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:38.418 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:20:38.435 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:38.428+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:38.462 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:20:38.535 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:38.530+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:38.539 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:38.539 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:38.556 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:20:38.647 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:38.642+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:38.724 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:38.724 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:38.738 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:38.732+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:38.740 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:20:38.839 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:38.834+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:38.908 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:20:38.912 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:38.912 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:38.926 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:20:38.967 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:38.962+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:39.041 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:39.034+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:39.057 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:39.057 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:39.074 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:39.074 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:39.076 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:39.076 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:39.076 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:39.076 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:39.091 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:20:39.091 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:39.092 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:39.112 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:39.112 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:39.133 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:39.133 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:39.142 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:39.134+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:39.154 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:39.154 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:39.206 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:39.206 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:39.206 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:39.206 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:39.206 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:39.220 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:20:39.261 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:39.256+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:39.295 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-17T21:20:39.365 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:39.360+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:39.393 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:39.393 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:39.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:39.393 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:39.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.393 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:525 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3670016~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:39.394 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:39.394 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 133.190636697, 2024-09-17T21:20:39.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046645697, 2024-09-17T21:20:39.394 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.394 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.394 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:39.394 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.394 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:39.394 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:39.394 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 525 2024-09-17T21:20:39.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.394 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:39.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:39.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:20:39.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:20:39.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200234+0000", 2024-09-17T21:20:39.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:20:39.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200254+0000", 2024-09-17T21:20:39.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9959999999999999e-05 2024-09-17T21:20:39.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200293+0000", 2024-09-17T21:20:39.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9755999999999997e-05 2024-09-17T21:20:39.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245871+0000", 2024-09-17T21:20:39.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045577871999999998 2024-09-17T21:20:39.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246030+0000", 2024-09-17T21:20:39.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158975 2024-09-17T21:20:39.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246871+0000", 2024-09-17T21:20:39.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084111699999999995 2024-09-17T21:20:39.398 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.398 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.398 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.398 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:39.398 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:39.398 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 133.190391022, 2024-09-17T21:20:39.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:20:39.398 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.398 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.398 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:39.398 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.399 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:39.399 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:39.399 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:20:39.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.399 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:39.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:39.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:20:39.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:20:39.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:20:39.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:20:39.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:20:39.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:20:39.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:20:39.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:20:39.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:20:39.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:20:39.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:20:39.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:20:39.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:20:39.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:20:39.402 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.402 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.402 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.402 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:39.403 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:39.403 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 133.19021233999999, 2024-09-17T21:20:39.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:20:39.403 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.403 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.403 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:39.403 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.403 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:39.403 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:39.403 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:20:39.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.403 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:39.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:39.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:20:39.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:20:39.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:20:39.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:20:39.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:20:39.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:20:39.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:20:39.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:20:39.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:20:39.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:20:39.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:20:39.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:20:39.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:20:39.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:20:39.407 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.407 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.407 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.407 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:39.407 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:39.407 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 133.19015306700001, 2024-09-17T21:20:39.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:20:39.407 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.407 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.407 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:39.408 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.408 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:39.408 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:39.408 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:20:39.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.408 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:39.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:39.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:20:39.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:20:39.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:20:39.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:20:39.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:20:39.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:20:39.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:20:39.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:20:39.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:20:39.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:20:39.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:20:39.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:20:39.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:20:39.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:20:39.411 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.411 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.412 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.412 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:39.412 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:39.412 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 133.19001087199999, 2024-09-17T21:20:39.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:20:39.412 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.412 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.412 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:39.412 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.412 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:39.412 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:39.413 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:20:39.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.413 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.413 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.413 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:39.413 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.413 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.413 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:39.413 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:20:39.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:20:39.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:20:39.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:20:39.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:20:39.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:20:39.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:20:39.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:20:39.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:20:39.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:20:39.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:20:39.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:20:39.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:20:39.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:20:39.416 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.416 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.416 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.416 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:39.416 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:39.417 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 133.189954949, 2024-09-17T21:20:39.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:20:39.417 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.417 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.417 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:39.417 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.417 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:39.417 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:39.417 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:20:39.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.417 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:39.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:39.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:20:39.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:20:39.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:20:39.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:20:39.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:20:39.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:20:39.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:20:39.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:20:39.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:20:39.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:20:39.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:20:39.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:20:39.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:20:39.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:20:39.421 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.421 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.421 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.421 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:39.421 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:39.421 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 133.18990133200001, 2024-09-17T21:20:39.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:20:39.421 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.421 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.421 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:39.422 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.422 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:39.422 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:39.422 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:20:39.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.422 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:39.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:39.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:20:39.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:20:39.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:20:39.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:20:39.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:20:39.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:20:39.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:20:39.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:20:39.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:20:39.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:20:39.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:20:39.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:20:39.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:20:39.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:20:39.425 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.425 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.425 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:20:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 82.880673090000002, 2024-09-17T21:20:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:20:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:20:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:20:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:20:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:20:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:20:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:20:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:20:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:20:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:20:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:20:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:20:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:20:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:20:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:20:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:20:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:20:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:20:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:20:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:20:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:20:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2597 3.1f 3:fb0e9615:::rbd_data.1152a8ae2019.000000000000000c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 58.552236661999999, 2024-09-17T21:20:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097064994000000002, 2024-09-17T21:20:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2597 2024-09-17T21:20:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838628+0000", 2024-09-17T21:20:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.187e-06 2024-09-17T21:20:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838654+0000", 2024-09-17T21:20:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:20:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838696+0000", 2024-09-17T21:20:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1226000000000001e-05 2024-09-17T21:20:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838715+0000", 2024-09-17T21:20:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9646000000000001e-05 2024-09-17T21:20:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838937+0000", 2024-09-17T21:20:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022163 2024-09-17T21:20:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.839284+0000", 2024-09-17T21:20:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034753799999999999 2024-09-17T21:20:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935691+0000", 2024-09-17T21:20:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.096406360999999996 2024-09-17T21:20:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2641 3.12 3:488a729f:::rbd_data.1152a8ae2019.0000000000000022:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 58.548779889000002, 2024-09-17T21:20:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098280953000000004, 2024-09-17T21:20:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2641 2024-09-17T21:20:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842083+0000", 2024-09-17T21:20:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5300000000000002e-07 2024-09-17T21:20:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842088+0000", 2024-09-17T21:20:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:20:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842105+0000", 2024-09-17T21:20:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6773e-05 2024-09-17T21:20:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842119+0000", 2024-09-17T21:20:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:20:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842320+0000", 2024-09-17T21:20:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020052700000000001 2024-09-17T21:20:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842845+0000", 2024-09-17T21:20:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052508600000000002 2024-09-17T21:20:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940363+0000", 2024-09-17T21:20:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097518795000000005 2024-09-17T21:20:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2674 3.12 3:4c87e3ac:::rbd_data.1152a8ae2019.0000000000000033:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 58.548417252, 2024-09-17T21:20:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098025125000000005, 2024-09-17T21:20:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2674 2024-09-17T21:20:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842447+0000", 2024-09-17T21:20:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3999999999999999e-06 2024-09-17T21:20:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842453+0000", 2024-09-17T21:20:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:20:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842472+0000", 2024-09-17T21:20:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9006000000000001e-05 2024-09-17T21:20:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842487+0000", 2024-09-17T21:20:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4866e-05 2024-09-17T21:20:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843594+0000", 2024-09-17T21:20:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011073249999999999 2024-09-17T21:20:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843668+0000", 2024-09-17T21:20:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3188000000000005e-05 2024-09-17T21:20:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843893+0000", 2024-09-17T21:20:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022503399999999999 2024-09-17T21:20:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850439+0000", 2024-09-17T21:20:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065467880000000004 2024-09-17T21:20:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940409+0000", 2024-09-17T21:20:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089969511000000002 2024-09-17T21:20:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940444+0000", 2024-09-17T21:20:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5345999999999997e-05 2024-09-17T21:20:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940470+0000", 2024-09-17T21:20:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5964000000000001e-05 2024-09-17T21:20:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2707 3.1f 3:f805e65e:::rbd_data.1152a8ae2019.0000000000000049:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 58.543370590999999, 2024-09-17T21:20:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088327300999999997, 2024-09-17T21:20:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2707 2024-09-17T21:20:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847494+0000", 2024-09-17T21:20:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:20:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847504+0000", 2024-09-17T21:20:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:20:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847526+0000", 2024-09-17T21:20:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1976000000000001e-05 2024-09-17T21:20:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847541+0000", 2024-09-17T21:20:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5277e-05 2024-09-17T21:20:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856142+0000", 2024-09-17T21:20:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086006710000000007 2024-09-17T21:20:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856435+0000", 2024-09-17T21:20:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029301499999999999 2024-09-17T21:20:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935819+0000", 2024-09-17T21:20:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079384448999999996 2024-09-17T21:20:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2730 3.12 3:48b3422a:::rbd_data.1152a8ae2019.0000000000000057:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 58.534177692, 2024-09-17T21:20:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083879975999999995, 2024-09-17T21:20:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2730 2024-09-17T21:20:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856687+0000", 2024-09-17T21:20:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:20:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856697+0000", 2024-09-17T21:20:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0207000000000001e-05 2024-09-17T21:20:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856808+0000", 2024-09-17T21:20:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011071200000000001 2024-09-17T21:20:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856828+0000", 2024-09-17T21:20:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0219000000000001e-05 2024-09-17T21:20:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856907+0000", 2024-09-17T21:20:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9661999999999997e-05 2024-09-17T21:20:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.857019+0000", 2024-09-17T21:20:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011148499999999999 2024-09-17T21:20:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940565+0000", 2024-09-17T21:20:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083545791999999994 2024-09-17T21:20:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2743 3.12 3:4d036e62:::rbd_data.1152a8ae2019.000000000000005f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 58.530232191000003, 2024-09-17T21:20:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080059962999999998, 2024-09-17T21:20:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2743 2024-09-17T21:20:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860631+0000", 2024-09-17T21:20:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:20:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860636+0000", 2024-09-17T21:20:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5279999999999997e-06 2024-09-17T21:20:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860649+0000", 2024-09-17T21:20:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-17T21:20:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860655+0000", 2024-09-17T21:20:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:20:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860731+0000", 2024-09-17T21:20:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5920000000000005e-05 2024-09-17T21:20:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.861070+0000", 2024-09-17T21:20:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338535 2024-09-17T21:20:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940690+0000", 2024-09-17T21:20:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079620503999999995 2024-09-17T21:20:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3378 3.12 3:4832a51b:::rbd_data.1152a8ae2019.000000000000006c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 58.501868340999998, 2024-09-17T21:20:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051812516000000003, 2024-09-17T21:20:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3378 2024-09-17T21:20:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888995+0000", 2024-09-17T21:20:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:20:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889006+0000", 2024-09-17T21:20:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1012999999999999e-05 2024-09-17T21:20:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889032+0000", 2024-09-17T21:20:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6441e-05 2024-09-17T21:20:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889040+0000", 2024-09-17T21:20:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0639999999999994e-06 2024-09-17T21:20:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889278+0000", 2024-09-17T21:20:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023785899999999999 2024-09-17T21:20:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889408+0000", 2024-09-17T21:20:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129219 2024-09-17T21:20:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889723+0000", 2024-09-17T21:20:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000315264 2024-09-17T21:20:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.891972+0000", 2024-09-17T21:20:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002249633 2024-09-17T21:20:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940733+0000", 2024-09-17T21:20:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048760626000000001 2024-09-17T21:20:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940785+0000", 2024-09-17T21:20:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1906000000000001e-05 2024-09-17T21:20:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940807+0000", 2024-09-17T21:20:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1554999999999999e-05 2024-09-17T21:20:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3379 3.12 3:4e83fd71:::rbd_data.1152a8ae2019.000000000000006d:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 58.501784133999998, 2024-09-17T21:20:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051804144000000003, 2024-09-17T21:20:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3379 2024-09-17T21:20:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889079+0000", 2024-09-17T21:20:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1800000000000004e-07 2024-09-17T21:20:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889083+0000", 2024-09-17T21:20:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:20:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889094+0000", 2024-09-17T21:20:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1562999999999999e-05 2024-09-17T21:20:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889102+0000", 2024-09-17T21:20:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.401e-06 2024-09-17T21:20:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:20:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014279259999999999 2024-09-17T21:20:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890609+0000", 2024-09-17T21:20:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9566000000000004e-05 2024-09-17T21:20:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890789+0000", 2024-09-17T21:20:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017949700000000001 2024-09-17T21:20:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.893929+0000", 2024-09-17T21:20:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003140538 2024-09-17T21:20:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940834+0000", 2024-09-17T21:20:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046904811999999997 2024-09-17T21:20:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940860+0000", 2024-09-17T21:20:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5341999999999999e-05 2024-09-17T21:20:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940882+0000", 2024-09-17T21:20:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2878999999999999e-05 2024-09-17T21:20:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:2644 3.1e 3:79615585:::rbd_data.115266d0b9f2.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2475520~14848,write 3106304~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:20:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 49.879155730999997, 2024-09-17T21:20:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050733484000000002, 2024-09-17T21:20:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:20:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:20:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2644 2024-09-17T21:20:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511708+0000", 2024-09-17T21:20:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-17T21:20:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522196+0000", 2024-09-17T21:20:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010487712999999999 2024-09-17T21:20:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522223+0000", 2024-09-17T21:20:39.476 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7064999999999999e-05 2024-09-17T21:20:39.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.476 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.476 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522236+0000", 2024-09-17T21:20:39.476 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2931e-05 2024-09-17T21:20:39.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.476 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.476 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522348+0000", 2024-09-17T21:20:39.476 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111906 2024-09-17T21:20:39.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.476 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522554+0000", 2024-09-17T21:20:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020632699999999999 2024-09-17T21:20:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:20:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.523185+0000", 2024-09-17T21:20:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063081400000000005 2024-09-17T21:20:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.547011+0000", 2024-09-17T21:20:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023825570000000001 2024-09-17T21:20:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562268+0000", 2024-09-17T21:20:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015257514 2024-09-17T21:20:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562367+0000", 2024-09-17T21:20:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8678000000000006e-05 2024-09-17T21:20:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562440+0000", 2024-09-17T21:20:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3337000000000006e-05 2024-09-17T21:20:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:231 3.1 3:856fed78:::rbd_data.115220dd393e.000000000000000b:head [read 0~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:20:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:20:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 21.436400462000002, 2024-09-17T21:20:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34254251699999999, 2024-09-17T21:20:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:20:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 231 2024-09-17T21:20:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:20:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:20:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954464+0000", 2024-09-17T21:20:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:20:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954475+0000", 2024-09-17T21:20:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1146e-05 2024-09-17T21:20:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954506+0000", 2024-09-17T21:20:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0292999999999999e-05 2024-09-17T21:20:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954524+0000", 2024-09-17T21:20:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8430000000000001e-05 2024-09-17T21:20:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954661+0000", 2024-09-17T21:20:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136403 2024-09-17T21:20:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954680+0000", 2024-09-17T21:20:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9743000000000001e-05 2024-09-17T21:20:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.296896+0000", 2024-09-17T21:20:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34221550499999998 2024-09-17T21:20:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297004+0000", 2024-09-17T21:20:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010865399999999999 2024-09-17T21:20:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:56 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:20:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:20:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 21.280766764999999, 2024-09-17T21:20:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18718773899999999, 2024-09-17T21:20:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:20:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:20:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 56 2024-09-17T21:20:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:20:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:20:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110098+0000", 2024-09-17T21:20:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:20:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110111+0000", 2024-09-17T21:20:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2289999999999999e-05 2024-09-17T21:20:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110143+0000", 2024-09-17T21:20:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1979e-05 2024-09-17T21:20:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110161+0000", 2024-09-17T21:20:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7839e-05 2024-09-17T21:20:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110319+0000", 2024-09-17T21:20:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015835400000000001 2024-09-17T21:20:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110353+0000", 2024-09-17T21:20:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3849000000000001e-05 2024-09-17T21:20:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297041+0000", 2024-09-17T21:20:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18668842299999999 2024-09-17T21:20:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297169+0000", 2024-09-17T21:20:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012786199999999999 2024-09-17T21:20:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297283+0000", 2024-09-17T21:20:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011430300000000001 2024-09-17T21:20:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:68 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:20:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:20:39.490 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 21.272225834, 2024-09-17T21:20:39.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17885582999999999, 2024-09-17T21:20:39.490 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.490 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.490 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:39.490 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.490 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:20:39.490 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:20:39.490 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 68 2024-09-17T21:20:39.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.490 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:20:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:20:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118639+0000", 2024-09-17T21:20:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.306e-06 2024-09-17T21:20:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118651+0000", 2024-09-17T21:20:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2567e-05 2024-09-17T21:20:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118682+0000", 2024-09-17T21:20:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0502e-05 2024-09-17T21:20:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118707+0000", 2024-09-17T21:20:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5463e-05 2024-09-17T21:20:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118889+0000", 2024-09-17T21:20:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018150999999999999 2024-09-17T21:20:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118910+0000", 2024-09-17T21:20:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:20:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297313+0000", 2024-09-17T21:20:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17840366699999999 2024-09-17T21:20:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297389+0000", 2024-09-17T21:20:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-17T21:20:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297492+0000", 2024-09-17T21:20:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103019 2024-09-17T21:20:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.495 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.495 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.495 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.495 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.495 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:39.496 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:20:39.497 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:39.490+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:39.597 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:39.590+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:39.670 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:39.671 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:39.671 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:39.671 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:39.685 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:20:39.698 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:39.694+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:39.798 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:39.794+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:39.805 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:39.805 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:39.806 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:39.806 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:39.806 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:39.819 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:20:39.899 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:39.894+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:39.945 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:39.945 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:39.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:39.945 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:20:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 120.131785838, 2024-09-17T21:20:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:20:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:20:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:20:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:20:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:20:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:20:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:20:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:20:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:20:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:20:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:20:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:20:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:20:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:20:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:20:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:20:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:20:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:20:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:20:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:20:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:20:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:20:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.951 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.951 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.951 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:20:39.951 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:39.951 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 106.576684052, 2024-09-17T21:20:39.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:20:39.951 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.951 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.951 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:39.951 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.951 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:20:39.951 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:39.952 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:20:39.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.952 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:39.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:39.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:20:39.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:20:39.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:20:39.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:20:39.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:20:39.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:20:39.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:20:39.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:20:39.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:20:39.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:20:39.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:39.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:20:39.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:20:39.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:20:39.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:20:39.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:20:39.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:20:39.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:39.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:20:39.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:20:39.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:39.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:20:39.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:20:39.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:39.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:20:39.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:20:39.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:39.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:20:39.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:20:39.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:20:39.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:20:39.957 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.957 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.957 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.957 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:793 3.9 3:95f63599:::rbd_data.1152eb799094.0000000000000007:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:20:39.957 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:20:39.957 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 23.509627897000001, 2024-09-17T21:20:39.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88245766400000003, 2024-09-17T21:20:39.957 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.958 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.958 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:39.958 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.958 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:20:39.958 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:20:39.958 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 793 2024-09-17T21:20:39.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.958 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:20:39.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:20:39.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431941+0000", 2024-09-17T21:20:39.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.864e-06 2024-09-17T21:20:39.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431953+0000", 2024-09-17T21:20:39.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1486e-05 2024-09-17T21:20:39.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431984+0000", 2024-09-17T21:20:39.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1075999999999999e-05 2024-09-17T21:20:39.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431999+0000", 2024-09-17T21:20:39.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.503e-05 2024-09-17T21:20:39.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432095+0000", 2024-09-17T21:20:39.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5698000000000004e-05 2024-09-17T21:20:39.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:39.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432109+0000", 2024-09-17T21:20:39.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4188e-05 2024-09-17T21:20:39.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.309642+0000", 2024-09-17T21:20:39.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87753353000000001 2024-09-17T21:20:39.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310104+0000", 2024-09-17T21:20:39.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046103600000000001 2024-09-17T21:20:39.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:39.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310630+0000", 2024-09-17T21:20:39.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052682800000000002 2024-09-17T21:20:39.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:39.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313100+0000", 2024-09-17T21:20:39.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024693369999999998 2024-09-17T21:20:39.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:39.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313935+0000", 2024-09-17T21:20:39.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083540100000000005 2024-09-17T21:20:39.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:39.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314086+0000", 2024-09-17T21:20:39.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015053399999999999 2024-09-17T21:20:39.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314396+0000", 2024-09-17T21:20:39.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031065599999999997 2024-09-17T21:20:39.964 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.964 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.964 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.964 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:800 3.19 3:9dc17f12:::rbd_data.1152eb799094.000000000000000e:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:20:39.964 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:20:39.964 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 23.509380652000001, 2024-09-17T21:20:39.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88154663799999999, 2024-09-17T21:20:39.964 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.964 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.964 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:39.964 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.965 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:20:39.965 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:20:39.965 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 800 2024-09-17T21:20:39.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.965 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:20:39.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:20:39.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432187+0000", 2024-09-17T21:20:39.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4100000000000001e-06 2024-09-17T21:20:39.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432194+0000", 2024-09-17T21:20:39.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0389999999999999e-06 2024-09-17T21:20:39.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432216+0000", 2024-09-17T21:20:39.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1614e-05 2024-09-17T21:20:39.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432232+0000", 2024-09-17T21:20:39.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5739e-05 2024-09-17T21:20:39.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432405+0000", 2024-09-17T21:20:39.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017355300000000001 2024-09-17T21:20:39.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:39.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432432+0000", 2024-09-17T21:20:39.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6979e-05 2024-09-17T21:20:39.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.307767+0000", 2024-09-17T21:20:39.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87533472899999998 2024-09-17T21:20:39.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308192+0000", 2024-09-17T21:20:39.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042485999999999999 2024-09-17T21:20:39.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:39.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308727+0000", 2024-09-17T21:20:39.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00053550400000000001 2024-09-17T21:20:39.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:39.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310347+0000", 2024-09-17T21:20:39.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016198670000000001 2024-09-17T21:20:39.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:39.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313424+0000", 2024-09-17T21:20:39.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030764640000000001 2024-09-17T21:20:39.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:39.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313510+0000", 2024-09-17T21:20:39.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5938000000000003e-05 2024-09-17T21:20:39.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313733+0000", 2024-09-17T21:20:39.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022294200000000001 2024-09-17T21:20:39.970 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.970 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.970 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.970 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:805 3.19 3:9cab2ed6:::rbd_data.1152eb799094.0000000000000013:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:20:39.971 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:20:39.971 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 23.509176632999999, 2024-09-17T21:20:39.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88274355699999996, 2024-09-17T21:20:39.971 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.971 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.971 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:39.971 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.971 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:20:39.971 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:20:39.971 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 805 2024-09-17T21:20:39.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.971 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:20:39.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:20:39.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432391+0000", 2024-09-17T21:20:39.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3e-06 2024-09-17T21:20:39.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432398+0000", 2024-09-17T21:20:39.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4489999999999996e-06 2024-09-17T21:20:39.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432415+0000", 2024-09-17T21:20:39.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7750000000000001e-05 2024-09-17T21:20:39.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432430+0000", 2024-09-17T21:20:39.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.449e-05 2024-09-17T21:20:39.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432635+0000", 2024-09-17T21:20:39.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205031 2024-09-17T21:20:39.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:39.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432657+0000", 2024-09-17T21:20:39.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2064000000000001e-05 2024-09-17T21:20:39.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310444+0000", 2024-09-17T21:20:39.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87778699199999999 2024-09-17T21:20:39.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310849+0000", 2024-09-17T21:20:39.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00040497800000000001 2024-09-17T21:20:39.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:39.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.311396+0000", 2024-09-17T21:20:39.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00054729099999999999 2024-09-17T21:20:39.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:39.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313847+0000", 2024-09-17T21:20:39.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024504940000000001 2024-09-17T21:20:39.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:39.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314879+0000", 2024-09-17T21:20:39.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001031817 2024-09-17T21:20:39.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:39.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314964+0000", 2024-09-17T21:20:39.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5685000000000004e-05 2024-09-17T21:20:39.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.315133+0000", 2024-09-17T21:20:39.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000169216 2024-09-17T21:20:39.976 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.976 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.976 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.977 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12087 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 52224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:39.977 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:20:39.977 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.172087954999999, 2024-09-17T21:20:39.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087684460000000006, 2024-09-17T21:20:39.977 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.977 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.977 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:39.977 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.977 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:39.977 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:39.977 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12087 2024-09-17T21:20:39.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.978 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:20:39.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:20:39.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769480+0000", 2024-09-17T21:20:39.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:20:39.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769485+0000", 2024-09-17T21:20:39.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8880000000000002e-06 2024-09-17T21:20:39.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769509+0000", 2024-09-17T21:20:39.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3343e-05 2024-09-17T21:20:39.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769521+0000", 2024-09-17T21:20:39.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2497000000000001e-05 2024-09-17T21:20:39.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.856883+0000", 2024-09-17T21:20:39.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087362191000000006 2024-09-17T21:20:39.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857014+0000", 2024-09-17T21:20:39.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013090399999999999 2024-09-17T21:20:39.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857163+0000", 2024-09-17T21:20:39.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014864000000000001 2024-09-17T21:20:39.981 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.981 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.981 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.981 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12088 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 56320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:39.981 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:20:39.981 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.172004300000001, 2024-09-17T21:20:39.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087917772000000005, 2024-09-17T21:20:39.981 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.981 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.982 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:39.982 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.982 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:39.982 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:39.982 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12088 2024-09-17T21:20:39.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.982 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:20:39.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:20:39.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769569+0000", 2024-09-17T21:20:39.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:20:39.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769574+0000", 2024-09-17T21:20:39.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4659999999999998e-06 2024-09-17T21:20:39.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769586+0000", 2024-09-17T21:20:39.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2311e-05 2024-09-17T21:20:39.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769597+0000", 2024-09-17T21:20:39.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0548e-05 2024-09-17T21:20:39.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857217+0000", 2024-09-17T21:20:39.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087619896000000003 2024-09-17T21:20:39.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857335+0000", 2024-09-17T21:20:39.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118538 2024-09-17T21:20:39.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857480+0000", 2024-09-17T21:20:39.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014470999999999999 2024-09-17T21:20:39.985 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.985 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.985 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.985 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12089 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 60416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:39.986 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:20:39.986 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.171951601, 2024-09-17T21:20:39.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088293233999999998, 2024-09-17T21:20:39.986 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.986 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.986 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:39.986 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.986 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:39.986 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:39.986 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12089 2024-09-17T21:20:39.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.986 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:20:39.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:20:39.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769616+0000", 2024-09-17T21:20:39.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:20:39.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769620+0000", 2024-09-17T21:20:39.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5140000000000002e-06 2024-09-17T21:20:39.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769633+0000", 2024-09-17T21:20:39.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2849e-05 2024-09-17T21:20:39.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769644+0000", 2024-09-17T21:20:39.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1471000000000001e-05 2024-09-17T21:20:39.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857564+0000", 2024-09-17T21:20:39.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087919445999999998 2024-09-17T21:20:39.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857716+0000", 2024-09-17T21:20:39.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000152065 2024-09-17T21:20:39.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857908+0000", 2024-09-17T21:20:39.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019252599999999999 2024-09-17T21:20:39.989 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.990 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.990 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.990 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12090 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 64512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:39.990 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:20:39.990 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.171905215999999, 2024-09-17T21:20:39.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088582672000000001, 2024-09-17T21:20:39.990 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.990 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.990 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:39.990 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.990 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:39.990 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:39.991 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12090 2024-09-17T21:20:39.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.991 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:20:39.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:20:39.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769662+0000", 2024-09-17T21:20:39.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:20:39.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769683+0000", 2024-09-17T21:20:39.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0979e-05 2024-09-17T21:20:39.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769699+0000", 2024-09-17T21:20:39.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5446000000000001e-05 2024-09-17T21:20:39.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769708+0000", 2024-09-17T21:20:39.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1679999999999993e-06 2024-09-17T21:20:39.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857976+0000", 2024-09-17T21:20:39.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088267795999999996 2024-09-17T21:20:39.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858095+0000", 2024-09-17T21:20:39.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119147 2024-09-17T21:20:39.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.994 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.994 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858244+0000", 2024-09-17T21:20:39.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014917000000000001 2024-09-17T21:20:39.994 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.994 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.994 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.994 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12091 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 68608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:39.994 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:20:39.994 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.171833832000001, 2024-09-17T21:20:39.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088832752000000001, 2024-09-17T21:20:39.995 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.995 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.995 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:39.995 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.995 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:39.995 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:39.995 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12091 2024-09-17T21:20:39.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.995 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:39.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:39.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:20:39.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:39.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:20:39.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:39.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:39.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769734+0000", 2024-09-17T21:20:39.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:20:39.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:39.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769739+0000", 2024-09-17T21:20:39.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8080000000000004e-06 2024-09-17T21:20:39.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:39.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769764+0000", 2024-09-17T21:20:39.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4839999999999999e-05 2024-09-17T21:20:39.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:39.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769777+0000", 2024-09-17T21:20:39.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3037000000000001e-05 2024-09-17T21:20:39.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:39.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858295+0000", 2024-09-17T21:20:39.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088518475999999999 2024-09-17T21:20:39.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:39.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858419+0000", 2024-09-17T21:20:39.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012378700000000001 2024-09-17T21:20:39.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:39.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858565+0000", 2024-09-17T21:20:39.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014630099999999999 2024-09-17T21:20:39.998 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.998 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:39.998 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:39.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:39.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:39.999 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12092 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:39.999 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:20:39.999 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.171751508, 2024-09-17T21:20:39.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089191555000000006, 2024-09-17T21:20:39.999 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:39.999 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:39.999 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:39.999 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:39.999 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:39.999 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:39.999 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12092 2024-09-17T21:20:39.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.000 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:20:40.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:20:40.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769816+0000", 2024-09-17T21:20:40.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7199999999999997e-07 2024-09-17T21:20:40.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769822+0000", 2024-09-17T21:20:40.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:20:40.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769838+0000", 2024-09-17T21:20:40.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6674e-05 2024-09-17T21:20:40.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769851+0000", 2024-09-17T21:20:40.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2527999999999999e-05 2024-09-17T21:20:40.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858660+0000", 2024-09-17T21:20:40.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088808556999999996 2024-09-17T21:20:40.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858817+0000", 2024-09-17T21:20:40.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015730400000000001 2024-09-17T21:20:40.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859007+0000", 2024-09-17T21:20:40.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018974899999999999 2024-09-17T21:20:40.003 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.003 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.003 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.003 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12093 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:40.003 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:20:40.003 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.171643404000001, 2024-09-17T21:20:40.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089429710999999995, 2024-09-17T21:20:40.003 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.004 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.004 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:40.004 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:40.004 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:40.004 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:40.004 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12093 2024-09-17T21:20:40.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.004 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:20:40.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:20:40.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769924+0000", 2024-09-17T21:20:40.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.342e-06 2024-09-17T21:20:40.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769929+0000", 2024-09-17T21:20:40.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0350000000000001e-06 2024-09-17T21:20:40.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769946+0000", 2024-09-17T21:20:40.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6793999999999999e-05 2024-09-17T21:20:40.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769959+0000", 2024-09-17T21:20:40.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3062e-05 2024-09-17T21:20:40.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859075+0000", 2024-09-17T21:20:40.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089115696999999994 2024-09-17T21:20:40.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859202+0000", 2024-09-17T21:20:40.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012657 2024-09-17T21:20:40.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859353+0000", 2024-09-17T21:20:40.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151211 2024-09-17T21:20:40.007 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.007 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.007 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.008 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12094 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:40.008 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:20:40.008 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.171589227999998, 2024-09-17T21:20:40.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089695001999999996, 2024-09-17T21:20:40.008 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.008 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.008 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:40.008 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:40.008 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:40.008 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:40.008 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12094 2024-09-17T21:20:40.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.008 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:20:40.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:20:40.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769978+0000", 2024-09-17T21:20:40.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.142e-06 2024-09-17T21:20:40.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769983+0000", 2024-09-17T21:20:40.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.386e-06 2024-09-17T21:20:40.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769997+0000", 2024-09-17T21:20:40.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.416e-05 2024-09-17T21:20:40.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770012+0000", 2024-09-17T21:20:40.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5384999999999999e-05 2024-09-17T21:20:40.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.011 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.011 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859405+0000", 2024-09-17T21:20:40.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089392376999999995 2024-09-17T21:20:40.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.011 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.011 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859525+0000", 2024-09-17T21:20:40.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120555 2024-09-17T21:20:40.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.011 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.012 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859672+0000", 2024-09-17T21:20:40.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000146997 2024-09-17T21:20:40.012 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.012 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.012 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.012 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12095 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:40.012 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:20:40.012 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.171536741000001, 2024-09-17T21:20:40.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090073476, 2024-09-17T21:20:40.012 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.012 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.012 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:40.013 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:40.013 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:40.013 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:40.013 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12095 2024-09-17T21:20:40.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.013 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:20:40.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:20:40.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770031+0000", 2024-09-17T21:20:40.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1699999999999997e-07 2024-09-17T21:20:40.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770036+0000", 2024-09-17T21:20:40.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.023e-06 2024-09-17T21:20:40.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770051+0000", 2024-09-17T21:20:40.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4953999999999999e-05 2024-09-17T21:20:40.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770063+0000", 2024-09-17T21:20:40.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:20:40.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859756+0000", 2024-09-17T21:20:40.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089692980000000005 2024-09-17T21:20:40.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.016 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.016 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859891+0000", 2024-09-17T21:20:40.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013459499999999999 2024-09-17T21:20:40.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.016 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.016 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860103+0000", 2024-09-17T21:20:40.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021274400000000001 2024-09-17T21:20:40.016 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.016 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.016 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.016 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12096 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:40.017 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:20:40.017 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.171487056, 2024-09-17T21:20:40.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090358274000000002, 2024-09-17T21:20:40.017 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.017 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.017 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:40.017 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:40.017 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:40.017 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:40.017 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12096 2024-09-17T21:20:40.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.017 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:20:40.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:20:40.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770080+0000", 2024-09-17T21:20:40.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9400000000000004e-07 2024-09-17T21:20:40.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770085+0000", 2024-09-17T21:20:40.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2169999999999996e-06 2024-09-17T21:20:40.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770100+0000", 2024-09-17T21:20:40.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5077e-05 2024-09-17T21:20:40.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770112+0000", 2024-09-17T21:20:40.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1951e-05 2024-09-17T21:20:40.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860160+0000", 2024-09-17T21:20:40.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090048413999999993 2024-09-17T21:20:40.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860285+0000", 2024-09-17T21:20:40.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012466699999999999 2024-09-17T21:20:40.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860438+0000", 2024-09-17T21:20:40.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015305399999999999 2024-09-17T21:20:40.021 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.021 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.021 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.021 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12097 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:40.021 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:20:40.021 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.171439309, 2024-09-17T21:20:40.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090702375000000002, 2024-09-17T21:20:40.021 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.021 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.021 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:40.021 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:40.021 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:40.022 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:40.022 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12097 2024-09-17T21:20:40.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.022 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.022 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.022 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:20:40.022 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.022 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.022 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:20:40.022 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.023 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.023 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770128+0000", 2024-09-17T21:20:40.023 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.04e-06 2024-09-17T21:20:40.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.023 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.023 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770133+0000", 2024-09-17T21:20:40.023 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8999999999999997e-06 2024-09-17T21:20:40.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.023 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.023 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770152+0000", 2024-09-17T21:20:40.023 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8890999999999999e-05 2024-09-17T21:20:40.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770164+0000", 2024-09-17T21:20:40.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1769e-05 2024-09-17T21:20:40.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860511+0000", 2024-09-17T21:20:40.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090347527999999996 2024-09-17T21:20:40.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860652+0000", 2024-09-17T21:20:40.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140894 2024-09-17T21:20:40.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860830+0000", 2024-09-17T21:20:40.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017735299999999999 2024-09-17T21:20:40.025 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.025 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.025 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.025 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12098 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:40.025 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:20:40.025 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.171385840999999, 2024-09-17T21:20:40.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090997262999999995, 2024-09-17T21:20:40.026 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.026 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.026 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:40.026 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:40.026 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:40.026 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:40.026 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12098 2024-09-17T21:20:40.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.026 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.026 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.026 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:20:40.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.027 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.027 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:20:40.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.027 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.027 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775465+0000", 2024-09-17T21:20:40.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0052843350000000002 2024-09-17T21:20:40.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.027 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775476+0000", 2024-09-17T21:20:40.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0797e-05 2024-09-17T21:20:40.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775506+0000", 2024-09-17T21:20:40.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9709999999999998e-05 2024-09-17T21:20:40.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775530+0000", 2024-09-17T21:20:40.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4528999999999998e-05 2024-09-17T21:20:40.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860894+0000", 2024-09-17T21:20:40.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.085364195000000004 2024-09-17T21:20:40.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861019+0000", 2024-09-17T21:20:40.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012467399999999999 2024-09-17T21:20:40.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861178+0000", 2024-09-17T21:20:40.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159023 2024-09-17T21:20:40.030 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.030 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.030 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.030 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12104 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:40.030 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:20:40.030 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.165684731999999, 2024-09-17T21:20:40.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087969628999999994, 2024-09-17T21:20:40.030 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.030 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.030 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:40.030 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:40.031 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:40.031 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:40.031 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12104 2024-09-17T21:20:40.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.031 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:20:40.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:20:40.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.032 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775883+0000", 2024-09-17T21:20:40.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3399999999999998e-07 2024-09-17T21:20:40.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.032 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775886+0000", 2024-09-17T21:20:40.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2119999999999999e-06 2024-09-17T21:20:40.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775895+0000", 2024-09-17T21:20:40.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8710000000000003e-06 2024-09-17T21:20:40.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775901+0000", 2024-09-17T21:20:40.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:20:40.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.034 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863544+0000", 2024-09-17T21:20:40.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087642642000000007 2024-09-17T21:20:40.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.034 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863694+0000", 2024-09-17T21:20:40.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014952300000000001 2024-09-17T21:20:40.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.034 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863851+0000", 2024-09-17T21:20:40.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000157804 2024-09-17T21:20:40.035 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.035 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.035 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.035 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12105 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:40.035 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:40.035 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.165655837999999, 2024-09-17T21:20:40.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088330400000000003, 2024-09-17T21:20:40.035 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.035 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.035 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:40.035 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:40.035 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:40.036 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:40.036 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12105 2024-09-17T21:20:40.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.036 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:40.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:40.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:40.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9999999999999997e-07 2024-09-17T21:20:40.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775914+0000", 2024-09-17T21:20:40.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4710000000000001e-06 2024-09-17T21:20:40.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775921+0000", 2024-09-17T21:20:40.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:20:40.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.038 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.038 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775928+0000", 2024-09-17T21:20:40.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:20:40.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.038 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.038 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863923+0000", 2024-09-17T21:20:40.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087995126000000007 2024-09-17T21:20:40.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.038 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864059+0000", 2024-09-17T21:20:40.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135797 2024-09-17T21:20:40.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.039 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864241+0000", 2024-09-17T21:20:40.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018252599999999999 2024-09-17T21:20:40.039 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.039 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.039 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.039 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12106 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:40.039 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:20:40.040 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.16556666, 2024-09-17T21:20:40.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088586394999999998, 2024-09-17T21:20:40.040 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.040 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.040 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:40.040 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:40.040 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:40.040 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:40.040 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12106 2024-09-17T21:20:40.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.040 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:20:40.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:20:40.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776001+0000", 2024-09-17T21:20:40.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4300000000000001e-07 2024-09-17T21:20:40.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776004+0000", 2024-09-17T21:20:40.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8049999999999998e-06 2024-09-17T21:20:40.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776018+0000", 2024-09-17T21:20:40.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4613999999999999e-05 2024-09-17T21:20:40.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776025+0000", 2024-09-17T21:20:40.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:20:40.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864279+0000", 2024-09-17T21:20:40.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088253570000000003 2024-09-17T21:20:40.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864433+0000", 2024-09-17T21:20:40.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015469400000000001 2024-09-17T21:20:40.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864586+0000", 2024-09-17T21:20:40.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000153003 2024-09-17T21:20:40.044 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.044 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.044 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.044 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.044 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.044 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:40.045 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:20:40.046 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:40.040+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:40.147 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:40.138+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:40.219 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:40.219 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:40.219 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:40.219 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:40.233 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:20:40.247 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:40.242+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:40.349 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:40.342+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:40.358 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:40.359 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:40.359 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:40.359 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:40.359 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:40.370 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone34 from 0xdfcf600 (overlap 0xaf9200) to 0x7ae600 2024-09-17T21:20:40.373 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:20:40.449 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:40.442+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:40.500 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:40.500 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:40.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:40.500 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:40.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.500 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:20:40.500 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:40.500 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.28013203899999, 2024-09-17T21:20:40.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:20:40.500 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.500 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.501 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:40.501 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:40.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:40.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:20:40.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:20:40.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:20:40.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:20:40.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:20:40.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:20:40.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:20:40.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:20:40.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:20:40.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:20:40.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:20:40.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:20:40.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:40.504 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:20:40.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:20:40.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.504 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:20:40.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:20:40.504 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.504 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.504 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.505 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:20:40.505 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:40.505 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 137.62983137500001, 2024-09-17T21:20:40.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:20:40.505 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.505 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.505 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:40.505 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.505 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:40.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:40.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:20:40.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:20:40.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:20:40.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:20:40.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.507 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.507 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:20:40.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:20:40.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.507 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.507 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:20:40.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:20:40.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.507 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.507 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:20:40.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:20:40.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.508 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.508 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:20:40.508 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:20:40.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.508 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:40.508 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:20:40.508 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:20:40.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.508 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.508 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:20:40.508 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:20:40.508 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.509 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.509 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.509 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:20:40.509 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:40.509 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 137.62856629199999, 2024-09-17T21:20:40.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:20:40.509 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.509 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.509 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:40.509 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:40.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:40.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:20:40.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:20:40.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:20:40.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:20:40.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:20:40.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:20:40.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:20:40.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:20:40.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.512 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:20:40.512 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:20:40.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.512 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.512 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:20:40.512 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:20:40.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.512 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:40.512 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:20:40.512 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:20:40.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.513 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.513 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:20:40.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:20:40.513 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.513 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.516 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.516 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:20:40.516 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:40.516 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 137.61878688199999, 2024-09-17T21:20:40.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:20:40.516 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.516 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.516 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:40.517 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:40.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:40.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:20:40.518 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:20:40.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.518 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.518 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:20:40.518 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:20:40.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.518 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.518 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:20:40.518 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:20:40.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.518 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.518 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:20:40.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:20:40.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.519 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.519 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:20:40.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:20:40.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.519 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.519 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:20:40.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:20:40.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.519 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:40.519 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:20:40.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:20:40.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.520 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.520 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:20:40.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:20:40.520 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.520 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.520 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.520 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:20:40.520 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:40.520 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 137.61006447899999, 2024-09-17T21:20:40.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:20:40.520 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.521 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.521 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:40.521 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:40.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:40.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.522 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:20:40.522 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:20:40.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.522 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.522 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.522 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:20:40.522 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:20:40.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.522 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.522 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.522 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:20:40.522 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:20:40.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.522 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.523 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.523 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:20:40.523 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:20:40.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.523 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.523 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.523 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:20:40.523 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:20:40.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.523 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.523 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.523 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:20:40.523 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:20:40.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.523 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:40.524 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:20:40.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:20:40.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.524 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:20:40.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:20:40.524 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.524 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.524 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.524 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:20:40.525 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:40.525 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 124.22497391500001, 2024-09-17T21:20:40.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:20:40.525 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.525 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.525 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:40.525 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.525 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.525 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.525 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:40.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.525 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.525 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.525 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:40.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:20:40.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:20:40.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:20:40.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:20:40.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:20:40.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:20:40.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.527 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:20:40.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:20:40.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.527 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:20:40.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:20:40.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.527 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:20:40.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:20:40.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.528 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:40.528 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:20:40.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:20:40.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.528 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.528 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:20:40.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:20:40.528 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.528 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.528 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.529 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1780 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'195, mlcod=69'190)", 2024-09-17T21:20:40.529 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:40.529 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.247118310999994, 2024-09-17T21:20:40.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075761769999999999, 2024-09-17T21:20:40.529 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.529 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.529 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:40.529 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.529 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.529 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.529 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:40.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.530 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.530 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:40.530 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.530 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.530 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250375+0000", 2024-09-17T21:20:40.530 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:20:40.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.530 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.530 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250383+0000", 2024-09-17T21:20:40.530 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7870000000000002e-06 2024-09-17T21:20:40.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250402+0000", 2024-09-17T21:20:40.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9346000000000001e-05 2024-09-17T21:20:40.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250413+0000", 2024-09-17T21:20:40.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.039e-05 2024-09-17T21:20:40.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251353+0000", 2024-09-17T21:20:40.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000940091 2024-09-17T21:20:40.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251407+0000", 2024-09-17T21:20:40.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4701999999999999e-05 2024-09-17T21:20:40.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:40.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257829+0000", 2024-09-17T21:20:40.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064212779999999999 2024-09-17T21:20:40.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257950+0000", 2024-09-17T21:20:40.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121623 2024-09-17T21:20:40.533 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.533 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.533 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.533 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1781 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'196, mlcod=69'190)", 2024-09-17T21:20:40.533 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:40.533 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.246035359999993, 2024-09-17T21:20:40.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065632939999999999, 2024-09-17T21:20:40.533 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.533 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.534 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:40.534 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:40.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:40.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.535 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.535 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251459+0000", 2024-09-17T21:20:40.535 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.624e-06 2024-09-17T21:20:40.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.535 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.535 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251471+0000", 2024-09-17T21:20:40.535 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2636000000000001e-05 2024-09-17T21:20:40.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.536 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.536 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251498+0000", 2024-09-17T21:20:40.536 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:20:40.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.536 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.536 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251515+0000", 2024-09-17T21:20:40.536 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6939000000000002e-05 2024-09-17T21:20:40.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.536 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252938+0000", 2024-09-17T21:20:40.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001423082 2024-09-17T21:20:40.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252999+0000", 2024-09-17T21:20:40.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1138999999999999e-05 2024-09-17T21:20:40.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:40.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257955+0000", 2024-09-17T21:20:40.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049559640000000002 2024-09-17T21:20:40.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.258020+0000", 2024-09-17T21:20:40.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5504000000000002e-05 2024-09-17T21:20:40.538 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.538 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.538 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.538 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1788 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'198, mlcod=69'191)", 2024-09-17T21:20:40.538 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:40.538 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.244009871000003, 2024-09-17T21:20:40.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073690370000000002, 2024-09-17T21:20:40.538 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.538 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.539 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:40.539 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:40.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:40.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253484+0000", 2024-09-17T21:20:40.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2470000000000001e-06 2024-09-17T21:20:40.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253492+0000", 2024-09-17T21:20:40.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7159999999999992e-06 2024-09-17T21:20:40.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253509+0000", 2024-09-17T21:20:40.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.666e-05 2024-09-17T21:20:40.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.541 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.541 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253522+0000", 2024-09-17T21:20:40.541 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2643000000000001e-05 2024-09-17T21:20:40.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.541 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.541 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254853+0000", 2024-09-17T21:20:40.541 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331238 2024-09-17T21:20:40.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.541 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.541 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254885+0000", 2024-09-17T21:20:40.541 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1562e-05 2024-09-17T21:20:40.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.542 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:40.542 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260788+0000", 2024-09-17T21:20:40.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059030749999999998 2024-09-17T21:20:40.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.542 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.542 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260852+0000", 2024-09-17T21:20:40.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3895999999999999e-05 2024-09-17T21:20:40.542 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.542 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.542 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.543 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5746 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'477, mlcod=71'473)", 2024-09-17T21:20:40.543 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:40.543 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 84.238582312999995, 2024-09-17T21:20:40.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074182709999999997, 2024-09-17T21:20:40.543 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.543 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.543 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:40.543 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.543 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:40.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:40.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.544 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258912+0000", 2024-09-17T21:20:40.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:20:40.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.544 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258926+0000", 2024-09-17T21:20:40.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4249e-05 2024-09-17T21:20:40.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258946+0000", 2024-09-17T21:20:40.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9879999999999999e-05 2024-09-17T21:20:40.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258956+0000", 2024-09-17T21:20:40.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:20:40.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259380+0000", 2024-09-17T21:20:40.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042337599999999999 2024-09-17T21:20:40.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259412+0000", 2024-09-17T21:20:40.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:20:40.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:40.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266191+0000", 2024-09-17T21:20:40.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067793269999999999 2024-09-17T21:20:40.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266328+0000", 2024-09-17T21:20:40.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013690999999999999 2024-09-17T21:20:40.547 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.547 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.547 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.547 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5747 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'478, mlcod=71'473)", 2024-09-17T21:20:40.547 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:40.547 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 84.237875963999997, 2024-09-17T21:20:40.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068779440000000004, 2024-09-17T21:20:40.547 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.547 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.547 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:40.547 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.548 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:40.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.548 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:40.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.548 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259618+0000", 2024-09-17T21:20:40.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6199999999999999e-06 2024-09-17T21:20:40.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259630+0000", 2024-09-17T21:20:40.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1884e-05 2024-09-17T21:20:40.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259657+0000", 2024-09-17T21:20:40.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7339e-05 2024-09-17T21:20:40.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259677+0000", 2024-09-17T21:20:40.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0156000000000001e-05 2024-09-17T21:20:40.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260521+0000", 2024-09-17T21:20:40.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084312899999999997 2024-09-17T21:20:40.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260585+0000", 2024-09-17T21:20:40.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4558000000000001e-05 2024-09-17T21:20:40.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:40.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266336+0000", 2024-09-17T21:20:40.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057505020000000002 2024-09-17T21:20:40.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266494+0000", 2024-09-17T21:20:40.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158756 2024-09-17T21:20:40.551 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.551 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.551 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.551 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5750 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'479, mlcod=71'473)", 2024-09-17T21:20:40.551 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:40.551 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 84.237163770999999, 2024-09-17T21:20:40.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006302874, 2024-09-17T21:20:40.552 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.552 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.552 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:40.552 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:40.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:40.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260330+0000", 2024-09-17T21:20:40.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:20:40.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260343+0000", 2024-09-17T21:20:40.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2862e-05 2024-09-17T21:20:40.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260367+0000", 2024-09-17T21:20:40.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3723e-05 2024-09-17T21:20:40.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260386+0000", 2024-09-17T21:20:40.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8879999999999999e-05 2024-09-17T21:20:40.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261579+0000", 2024-09-17T21:20:40.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001193431 2024-09-17T21:20:40.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261608+0000", 2024-09-17T21:20:40.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8858999999999999e-05 2024-09-17T21:20:40.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:40.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266505+0000", 2024-09-17T21:20:40.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004896999 2024-09-17T21:20:40.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266631+0000", 2024-09-17T21:20:40.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012655300000000001 2024-09-17T21:20:40.555 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.556 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.556 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.556 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5751 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'480, mlcod=71'473)", 2024-09-17T21:20:40.556 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:40.556 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 84.236456559000004, 2024-09-17T21:20:40.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075756299999999999, 2024-09-17T21:20:40.556 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.556 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.556 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:40.556 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:40.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:40.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261037+0000", 2024-09-17T21:20:40.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:20:40.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261049+0000", 2024-09-17T21:20:40.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1063999999999999e-05 2024-09-17T21:20:40.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261072+0000", 2024-09-17T21:20:40.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3102e-05 2024-09-17T21:20:40.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261133+0000", 2024-09-17T21:20:40.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1276000000000001e-05 2024-09-17T21:20:40.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:20:40.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015334319999999999 2024-09-17T21:20:40.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262702+0000", 2024-09-17T21:20:40.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5281999999999999e-05 2024-09-17T21:20:40.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:40.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268527+0000", 2024-09-17T21:20:40.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058250389999999997 2024-09-17T21:20:40.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.560 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268611+0000", 2024-09-17T21:20:40.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4724999999999999e-05 2024-09-17T21:20:40.560 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.560 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.560 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.560 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5756 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'481, mlcod=71'473)", 2024-09-17T21:20:40.560 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:40.560 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 84.235595703000001, 2024-09-17T21:20:40.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006772386, 2024-09-17T21:20:40.561 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.561 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.561 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:40.561 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.561 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:40.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.561 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:40.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261899+0000", 2024-09-17T21:20:40.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.903e-06 2024-09-17T21:20:40.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261912+0000", 2024-09-17T21:20:40.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-05 2024-09-17T21:20:40.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261938+0000", 2024-09-17T21:20:40.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6098999999999999e-05 2024-09-17T21:20:40.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261955+0000", 2024-09-17T21:20:40.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.696e-05 2024-09-17T21:20:40.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263845+0000", 2024-09-17T21:20:40.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001890187 2024-09-17T21:20:40.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263907+0000", 2024-09-17T21:20:40.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1600999999999995e-05 2024-09-17T21:20:40.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:40.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268615+0000", 2024-09-17T21:20:40.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047086070000000001 2024-09-17T21:20:40.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268669+0000", 2024-09-17T21:20:40.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3588999999999998e-05 2024-09-17T21:20:40.564 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.564 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.564 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.565 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4978.0:598 3.9 e80/23 3:9701cc17:::rbd_data.1152724ff2a3.0000000000000019:head v 80'202, mlcod=80'198)", 2024-09-17T21:20:40.565 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:40.565 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.734021319999997, 2024-09-17T21:20:40.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066473340000000004, 2024-09-17T21:20:40.565 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.565 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.565 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:40.565 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.565 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.565 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:40.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:40.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763472+0000", 2024-09-17T21:20:40.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6899999999999997e-07 2024-09-17T21:20:40.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763478+0000", 2024-09-17T21:20:40.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:20:40.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.567 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763489+0000", 2024-09-17T21:20:40.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0628999999999999e-05 2024-09-17T21:20:40.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.567 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763495+0000", 2024-09-17T21:20:40.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6649999999999997e-06 2024-09-17T21:20:40.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.567 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763907+0000", 2024-09-17T21:20:40.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041163899999999999 2024-09-17T21:20:40.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.568 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.568 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763976+0000", 2024-09-17T21:20:40.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8534999999999993e-05 2024-09-17T21:20:40.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.568 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.568 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:40.568 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.769900+0000", 2024-09-17T21:20:40.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059243270000000001 2024-09-17T21:20:40.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.568 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.568 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.568 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.770118+0000", 2024-09-17T21:20:40.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021842500000000001 2024-09-17T21:20:40.568 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.569 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.569 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.569 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.569 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:468 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'94, mlcod=89'87)", 2024-09-17T21:20:40.569 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:40.569 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.811276020999998, 2024-09-17T21:20:40.569 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066528790000000004, 2024-09-17T21:20:40.569 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.569 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.569 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:40.569 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.569 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.569 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.570 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:40.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.570 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.570 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:40.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.570 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.570 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686218+0000", 2024-09-17T21:20:40.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3060000000000001e-06 2024-09-17T21:20:40.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.571 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.571 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686256+0000", 2024-09-17T21:20:40.571 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8333e-05 2024-09-17T21:20:40.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.571 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.571 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.571 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686272+0000", 2024-09-17T21:20:40.571 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:20:40.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.571 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.571 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.571 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686283+0000", 2024-09-17T21:20:40.571 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0903e-05 2024-09-17T21:20:40.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.572 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.572 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686682+0000", 2024-09-17T21:20:40.572 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039935099999999999 2024-09-17T21:20:40.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.572 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.572 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686785+0000", 2024-09-17T21:20:40.572 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010275199999999999 2024-09-17T21:20:40.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.572 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:40.572 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692816+0000", 2024-09-17T21:20:40.573 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060317579999999999 2024-09-17T21:20:40.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.573 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.573 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.573 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692869+0000", 2024-09-17T21:20:40.573 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2775999999999998e-05 2024-09-17T21:20:40.573 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.573 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.573 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.573 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.573 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:755 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'127, mlcod=89'122)", 2024-09-17T21:20:40.573 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:40.573 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.674820807000003, 2024-09-17T21:20:40.574 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084014080000000008, 2024-09-17T21:20:40.574 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.574 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.574 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:40.574 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.574 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.574 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:40.574 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.574 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.574 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:40.574 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.575 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.575 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:40.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:20:40.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.575 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.575 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.575 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822683+0000", 2024-09-17T21:20:40.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0674e-05 2024-09-17T21:20:40.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.575 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.575 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.575 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822698+0000", 2024-09-17T21:20:40.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5189e-05 2024-09-17T21:20:40.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.575 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.575 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.576 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822708+0000", 2024-09-17T21:20:40.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9119999999999993e-06 2024-09-17T21:20:40.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.576 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.576 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.576 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822748+0000", 2024-09-17T21:20:40.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9727000000000001e-05 2024-09-17T21:20:40.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.576 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.576 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.576 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822759+0000", 2024-09-17T21:20:40.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0865e-05 2024-09-17T21:20:40.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.576 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.577 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:40.577 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831034+0000", 2024-09-17T21:20:40.577 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082750710000000002 2024-09-17T21:20:40.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.577 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.577 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831073+0000", 2024-09-17T21:20:40.577 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9063999999999997e-05 2024-09-17T21:20:40.577 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.577 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.577 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.577 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:756 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'128, mlcod=89'124)", 2024-09-17T21:20:40.578 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:40.578 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.672898175, 2024-09-17T21:20:40.578 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0076300270000000002, 2024-09-17T21:20:40.578 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.578 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.578 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:40.578 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.578 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.578 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.578 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:40.578 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.578 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.578 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.579 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:40.579 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.579 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.579 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824595+0000", 2024-09-17T21:20:40.579 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-17T21:20:40.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.579 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.579 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824600+0000", 2024-09-17T21:20:40.579 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:20:40.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.580 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.580 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824612+0000", 2024-09-17T21:20:40.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.181e-05 2024-09-17T21:20:40.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.580 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.580 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.580 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824619+0000", 2024-09-17T21:20:40.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:20:40.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.580 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.580 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.580 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824671+0000", 2024-09-17T21:20:40.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2262999999999998e-05 2024-09-17T21:20:40.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.581 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.581 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824695+0000", 2024-09-17T21:20:40.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3883000000000002e-05 2024-09-17T21:20:40.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.581 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:40.581 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832185+0000", 2024-09-17T21:20:40.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007490048 2024-09-17T21:20:40.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.581 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.581 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832224+0000", 2024-09-17T21:20:40.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8957e-05 2024-09-17T21:20:40.582 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.582 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.582 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.582 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.582 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2617 3.d e90/23 3:b0143718:::rbd_data.1152a8ae2019.0000000000000016:head v 90'249, mlcod=90'246)", 2024-09-17T21:20:40.582 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:40.582 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 59.653047174999998, 2024-09-17T21:20:40.582 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063715559999999996, 2024-09-17T21:20:40.582 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.582 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.582 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:40.582 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.582 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.583 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.583 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:40.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.583 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.583 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:40.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.583 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.583 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844455+0000", 2024-09-17T21:20:40.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4259999999999992e-06 2024-09-17T21:20:40.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.584 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.584 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844470+0000", 2024-09-17T21:20:40.584 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5386000000000001e-05 2024-09-17T21:20:40.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.584 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.584 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.584 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844499+0000", 2024-09-17T21:20:40.584 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.883e-05 2024-09-17T21:20:40.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.584 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.584 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.584 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844518+0000", 2024-09-17T21:20:40.584 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8830000000000001e-05 2024-09-17T21:20:40.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.585 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.585 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.585 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844649+0000", 2024-09-17T21:20:40.585 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013143399999999999 2024-09-17T21:20:40.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.585 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.585 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.585 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844702+0000", 2024-09-17T21:20:40.585 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2876e-05 2024-09-17T21:20:40.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.585 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.585 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:40.585 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850696+0000", 2024-09-17T21:20:40.585 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059943729999999999 2024-09-17T21:20:40.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.586 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.586 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.586 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850817+0000", 2024-09-17T21:20:40.586 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120401 2024-09-17T21:20:40.586 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.586 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.586 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.586 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.586 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2635 3.d e90/23 3:b3d1e920:::rbd_data.1152a8ae2019.000000000000001f:head v 90'252, mlcod=90'246)", 2024-09-17T21:20:40.586 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:40.586 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 59.650339105, 2024-09-17T21:20:40.586 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0077265190000000003, 2024-09-17T21:20:40.586 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:40.587 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:40.587 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:40.587 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:40.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.587 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:40.587 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:40.587 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.587 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:40.587 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:40.587 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:40.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.587 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:40.588 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847155+0000", 2024-09-17T21:20:40.588 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.544e-06 2024-09-17T21:20:40.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.588 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:40.588 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847165+0000", 2024-09-17T21:20:40.588 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.043e-05 2024-09-17T21:20:40.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.588 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:40.588 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847189+0000", 2024-09-17T21:20:40.588 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3659000000000002e-05 2024-09-17T21:20:40.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.589 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:40.589 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847210+0000", 2024-09-17T21:20:40.589 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1163e-05 2024-09-17T21:20:40.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.589 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.589 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:40.589 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848856+0000", 2024-09-17T21:20:40.589 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001645521 2024-09-17T21:20:40.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.589 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.589 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:40.589 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848905+0000", 2024-09-17T21:20:40.589 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9048999999999997e-05 2024-09-17T21:20:40.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.589 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:40.590 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854770+0000", 2024-09-17T21:20:40.590 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058652879999999998 2024-09-17T21:20:40.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:40.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:40.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:40.590 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854880+0000", 2024-09-17T21:20:40.590 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010986500000000001 2024-09-17T21:20:40.590 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.590 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.590 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.590 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:40.590 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:40.590 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:40.592 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:20:40.592 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:40.584+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:40.692 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:40.688+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:40.726 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:20:40.726 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:40.740 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:20:40.793 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:40.786+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:40.868 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:40.869 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:40.883 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:20:40.894 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:40.888+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:40.950 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-17T21:20:40.995 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:40.986+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:41.056 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:20:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:20:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 164.03674288799999, 2024-09-17T21:20:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:20:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:20:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:20:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:20:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:20:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:20:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:20:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:20:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:20:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:20:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:20:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:20:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:20:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:20:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:20:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:20:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:20:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:20:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:20:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:20:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:20:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:20:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:20:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:20:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:20:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:20:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:20:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:20:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:20:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:20:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:20:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:20:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:20:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:20:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:20:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 144.456578608, 2024-09-17T21:20:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:20:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:20:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:20:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:20:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:20:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:20:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:20:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:20:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:20:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:20:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:20:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:20:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:20:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:20:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:20:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:20:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:20:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:20:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:20:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:20:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:20:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:20:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:20:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:20:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:20:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:20:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:20:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:20:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:20:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:20:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:20:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 144.45648129599999, 2024-09-17T21:20:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:20:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:20:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:20:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:20:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:20:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:20:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:20:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:20:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:20:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:20:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:20:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:20:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:20:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:20:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:20:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:20:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:20:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:20:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:20:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:20:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:20:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:20:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:20:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:20:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:20:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:20:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:20:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:20:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:20:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:20:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:20:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:20:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:20:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:20:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 144.454394495, 2024-09-17T21:20:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:20:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:20:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:20:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:20:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:20:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:20:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:20:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:20:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:20:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:20:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:20:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:20:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:20:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:20:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:20:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:20:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:20:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:20:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:20:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:20:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:20:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:20:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:20:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:20:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:20:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:20:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:20:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:20:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:20:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:20:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:20:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:20:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:20:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:20:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:20:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:20:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:20:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:20:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.77049571400001, 2024-09-17T21:20:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:20:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:20:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:20:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:20:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:20:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:20:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:20:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:20:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:20:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:20:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:20:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:20:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:20:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:20:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:20:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:20:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:20:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:20:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:20:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:20:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:20:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:20:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:20:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:20:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:20:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:20:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:20:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:20:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:20:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:20:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.761404003, 2024-09-17T21:20:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:20:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:20:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:20:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:20:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:20:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:20:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:20:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:20:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:20:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:20:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:20:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:20:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:20:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:20:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:20:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:20:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:20:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:20:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:20:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:20:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:20:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:20:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:20:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:20:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:20:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.761231952, 2024-09-17T21:20:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:20:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:20:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:20:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:20:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:20:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:20:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:20:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:20:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:20:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:20:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:20:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:20:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:20:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:20:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:20:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:20:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:20:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:20:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:20:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:20:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:20:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:20:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:20:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:20:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:20:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.586299205, 2024-09-17T21:20:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:20:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:20:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:20:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:20:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:20:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:20:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:20:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:20:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:20:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:20:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:20:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:20:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:20:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:20:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:20:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:20:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:20:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:20:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:20:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:20:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:20:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:20:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.586214648, 2024-09-17T21:20:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:20:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:20:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:20:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:20:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:20:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:20:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:20:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:20:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:20:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:20:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:20:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:20:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:20:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:20:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:20:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:20:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:20:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:20:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:20:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:20:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.586149011, 2024-09-17T21:20:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:20:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:20:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:20:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:20:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:20:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:20:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:20:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:20:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:20:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:20:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:20:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:20:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:20:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:20:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:20:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:20:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:20:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:20:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:20:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:20:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:20:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:20:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:20:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 60.212573913999996, 2024-09-17T21:20:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:20:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:20:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:20:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:20:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:20:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:20:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:20:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:20:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:20:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:20:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:20:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:20:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:20:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:20:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:20:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:20:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:20:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:20:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:20:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 60.212454401999999, 2024-09-17T21:20:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:20:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:20:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:20:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:20:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:20:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:20:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:20:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:20:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:20:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:20:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:20:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:20:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:20:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:20:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:20:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:20:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:20:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:20:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:994 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 571392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:20:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.619482428, 2024-09-17T21:20:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083898465000000005, 2024-09-17T21:20:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 994 2024-09-17T21:20:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:20:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:20:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:20:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:20:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431058+0000", 2024-09-17T21:20:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1667e-05 2024-09-17T21:20:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431067+0000", 2024-09-17T21:20:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0720000000000006e-06 2024-09-17T21:20:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431075+0000", 2024-09-17T21:20:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9200000000000004e-06 2024-09-17T21:20:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514671+0000", 2024-09-17T21:20:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083596229999999994 2024-09-17T21:20:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514741+0000", 2024-09-17T21:20:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9929000000000001e-05 2024-09-17T21:20:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514924+0000", 2024-09-17T21:20:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000183122 2024-09-17T21:20:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:995 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 575488~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:20:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.619423412, 2024-09-17T21:20:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084214699000000004, 2024-09-17T21:20:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 995 2024-09-17T21:20:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:20:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:20:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431086+0000", 2024-09-17T21:20:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7999999999999995e-07 2024-09-17T21:20:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431088+0000", 2024-09-17T21:20:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4729999999999999e-06 2024-09-17T21:20:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431126+0000", 2024-09-17T21:20:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7583999999999998e-05 2024-09-17T21:20:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431137+0000", 2024-09-17T21:20:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1768e-05 2024-09-17T21:20:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515003+0000", 2024-09-17T21:20:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083865097 2024-09-17T21:20:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515115+0000", 2024-09-17T21:20:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112561 2024-09-17T21:20:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515300+0000", 2024-09-17T21:20:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018453600000000001 2024-09-17T21:20:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:996 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:20:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.619341030999999, 2024-09-17T21:20:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084493736999999999, 2024-09-17T21:20:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 996 2024-09-17T21:20:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:20:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:20:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431168+0000", 2024-09-17T21:20:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4e-07 2024-09-17T21:20:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431172+0000", 2024-09-17T21:20:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.608e-06 2024-09-17T21:20:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431180+0000", 2024-09-17T21:20:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5650000000000002e-06 2024-09-17T21:20:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431188+0000", 2024-09-17T21:20:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:20:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515354+0000", 2024-09-17T21:20:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084165977000000003 2024-09-17T21:20:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515451+0000", 2024-09-17T21:20:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6669000000000006e-05 2024-09-17T21:20:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515661+0000", 2024-09-17T21:20:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021021599999999999 2024-09-17T21:20:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:997 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 583680~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:20:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.619309636000001, 2024-09-17T21:20:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084772419000000002, 2024-09-17T21:20:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 997 2024-09-17T21:20:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:20:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:20:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:20:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0400000000000005e-07 2024-09-17T21:20:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431202+0000", 2024-09-17T21:20:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:20:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431208+0000", 2024-09-17T21:20:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4740000000000002e-06 2024-09-17T21:20:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431214+0000", 2024-09-17T21:20:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6550000000000001e-06 2024-09-17T21:20:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515741+0000", 2024-09-17T21:20:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084526904 2024-09-17T21:20:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515826+0000", 2024-09-17T21:20:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4643999999999998e-05 2024-09-17T21:20:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515971+0000", 2024-09-17T21:20:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145444 2024-09-17T21:20:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:998 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 587776~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:20:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.619286664000001, 2024-09-17T21:20:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085108565999999997, 2024-09-17T21:20:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 998 2024-09-17T21:20:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:20:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:20:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:20:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:20:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431224+0000", 2024-09-17T21:20:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:20:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431231+0000", 2024-09-17T21:20:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000002e-06 2024-09-17T21:20:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431236+0000", 2024-09-17T21:20:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.648e-06 2024-09-17T21:20:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516024+0000", 2024-09-17T21:20:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084787584999999999 2024-09-17T21:20:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516122+0000", 2024-09-17T21:20:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8076000000000001e-05 2024-09-17T21:20:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516330+0000", 2024-09-17T21:20:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020808 2024-09-17T21:20:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1000 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 591872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:20:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.619264549, 2024-09-17T21:20:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085389065, 2024-09-17T21:20:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1000 2024-09-17T21:20:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:20:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:20:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:20:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:20:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431246+0000", 2024-09-17T21:20:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1749999999999999e-06 2024-09-17T21:20:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431253+0000", 2024-09-17T21:20:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2790000000000004e-06 2024-09-17T21:20:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431258+0000", 2024-09-17T21:20:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3299999999999998e-06 2024-09-17T21:20:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516384+0000", 2024-09-17T21:20:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085125549999999994 2024-09-17T21:20:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516460+0000", 2024-09-17T21:20:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6359999999999994e-05 2024-09-17T21:20:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516633+0000", 2024-09-17T21:20:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017285900000000001 2024-09-17T21:20:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1001 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 595968~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:20:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.619241143, 2024-09-17T21:20:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085614715999999993, 2024-09-17T21:20:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1001 2024-09-17T21:20:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:20:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:20:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438066+0000", 2024-09-17T21:20:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067989490000000003 2024-09-17T21:20:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438073+0000", 2024-09-17T21:20:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8419999999999999e-06 2024-09-17T21:20:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438090+0000", 2024-09-17T21:20:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6657e-05 2024-09-17T21:20:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438099+0000", 2024-09-17T21:20:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0529999999999996e-06 2024-09-17T21:20:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516678+0000", 2024-09-17T21:20:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078579422999999995 2024-09-17T21:20:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516739+0000", 2024-09-17T21:20:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1315000000000005e-05 2024-09-17T21:20:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516882+0000", 2024-09-17T21:20:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014247700000000001 2024-09-17T21:20:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:19998 3.5 3:a4885450:::rbd_object_map.1152d139f9f9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-17T21:20:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:20:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.647973898, 2024-09-17T21:20:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90715489299999996, 2024-09-17T21:20:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19998 2024-09-17T21:20:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:20:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:20:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402537+0000", 2024-09-17T21:20:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5600000000000001e-06 2024-09-17T21:20:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402549+0000", 2024-09-17T21:20:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1929999999999999e-05 2024-09-17T21:20:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402579+0000", 2024-09-17T21:20:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0093000000000001e-05 2024-09-17T21:20:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402595+0000", 2024-09-17T21:20:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5855999999999999e-05 2024-09-17T21:20:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402731+0000", 2024-09-17T21:20:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013610799999999999 2024-09-17T21:20:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402750+0000", 2024-09-17T21:20:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8737e-05 2024-09-17T21:20:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.305977+0000", 2024-09-17T21:20:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90322768399999998 2024-09-17T21:20:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306236+0000", 2024-09-17T21:20:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000258111 2024-09-17T21:20:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306557+0000", 2024-09-17T21:20:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032159799999999998 2024-09-17T21:20:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.308286+0000", 2024-09-17T21:20:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017292150000000001 2024-09-17T21:20:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309530+0000", 2024-09-17T21:20:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00124409 2024-09-17T21:20:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309630+0000", 2024-09-17T21:20:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9689e-05 2024-09-17T21:20:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309689+0000", 2024-09-17T21:20:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9222000000000003e-05 2024-09-17T21:20:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.158 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:41.158 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:41.158 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:41.159 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:20:41.160 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:41.154+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:41.172 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:20:41.257 INFO:teuthology.orchestra.run.smithi184.stdout:5175 trunc from 0xdfcf600 to 0x72e9a00 2024-09-17T21:20:41.260 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:41.252+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:41.309 INFO:teuthology.orchestra.run.smithi184.stdout:5176 punch from 0x62e9a00 to 0x62f2200, (0x8800 bytes) 2024-09-17T21:20:41.323 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:20:41.326 INFO:teuthology.orchestra.run.smithi184.stdout:5177 trunc from 0x72e9a00 to 0xd69a000 2024-09-17T21:20:41.334 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:41.335 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:41.342 INFO:teuthology.orchestra.run.smithi184.stdout:5181 read 0x7ad7a00 thru 0x7adbc8d (0x428e bytes) 2024-09-17T21:20:41.343 INFO:teuthology.orchestra.run.smithi184.stdout:5182 write 0x24c4e00 thru 0x24d3bff (0xee00 bytes) 2024-09-17T21:20:41.343 INFO:teuthology.orchestra.run.smithi184.stdout:5183 read 0x2494400 thru 0x249af7f (0x6b80 bytes) 2024-09-17T21:20:41.344 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:20:41.344 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:41.344 INFO:teuthology.orchestra.run.smithi184.stdout:5184 punch from 0x2fe5a00 to 0x2fe8400, (0x2a00 bytes) 2024-09-17T21:20:41.349 INFO:teuthology.orchestra.run.smithi184.stdout:5185 trunc from 0xd69a000 to 0xc83c600 2024-09-17T21:20:41.355 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:41.355 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:41.360 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:41.356+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:41.363 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:20:41.375 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:41.375 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:41.382 INFO:teuthology.orchestra.run.smithi184.stdout:5186 read 0xc125400 thru 0xc130568 (0xb169 bytes) 2024-09-17T21:20:41.384 INFO:teuthology.orchestra.run.smithi184.stdout:5189 write 0x9c400 thru 0xa7e83 (0xba84 bytes) 2024-09-17T21:20:41.385 INFO:teuthology.orchestra.run.smithi184.stdout:5190 punch from 0xb542200 to 0xb54d400, (0xb200 bytes) 2024-09-17T21:20:41.386 INFO:teuthology.orchestra.run.smithi184.stdout:5191 read 0x7a20a00 thru 0x7a298e9 (0x8eea bytes) 2024-09-17T21:20:41.387 INFO:teuthology.orchestra.run.smithi184.stdout:5192 punch from 0x598b400 to 0x5998200, (0xce00 bytes) 2024-09-17T21:20:41.396 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:41.397 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:41.416 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:41.416 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:41.419 INFO:teuthology.orchestra.run.smithi184.stdout:5193 write 0xcce5a00 thru 0xccf0c0a (0xb20b bytes) 2024-09-17T21:20:41.423 INFO:teuthology.orchestra.run.smithi184.stdout:5194 trunc from 0xccf0e00 to 0xb737c00 2024-09-17T21:20:41.437 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:41.437 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:41.461 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:41.454+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:41.467 INFO:teuthology.orchestra.run.smithi184.stdout:5195 write 0xe0a3e00 thru 0xe0a9702 (0x5903 bytes) 2024-09-17T21:20:41.468 INFO:teuthology.orchestra.run.smithi184.stdout:5197 read 0x54f1600 thru 0x54f27c4 (0x11c5 bytes) 2024-09-17T21:20:41.469 INFO:teuthology.orchestra.run.smithi184.stdout:5198 write 0x7269e00 thru 0x7273213 (0x9414 bytes) 2024-09-17T21:20:41.470 INFO:teuthology.orchestra.run.smithi184.stdout:5199 read 0xb68c00 thru 0xb73f0e (0xb30f bytes) 2024-09-17T21:20:41.473 INFO:teuthology.orchestra.run.smithi184.stdout:5200 trunc from 0xe0a9800 to 0xd361c00 2024-09-17T21:20:41.502 INFO:teuthology.orchestra.run.smithi184.stdout:5201 write 0x8773800 thru 0x8775d55 (0x2556 bytes) 2024-09-17T21:20:41.502 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:41.502 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:41.518 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:20:41.549 INFO:teuthology.orchestra.run.smithi184.stdout:5203 trunc from 0xd361c00 to 0x4d83a00 2024-09-17T21:20:41.562 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:41.556+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:41.616 INFO:teuthology.orchestra.run.smithi184.stdout:5204 trunc from 0x4d83a00 to 0x6b07600 2024-09-17T21:20:41.631 INFO:teuthology.orchestra.run.smithi184.stdout:5207 read 0x437d000 thru 0x4384f76 (0x7f77 bytes) 2024-09-17T21:20:41.635 INFO:teuthology.orchestra.run.smithi184.stdout:5208 punch from 0x1a4c600 to 0x1a55a00, (0x9400 bytes) 2024-09-17T21:20:41.636 INFO:teuthology.orchestra.run.smithi184.stdout:5210 read 0x43a7a00 thru 0x43b37a1 (0xbda2 bytes) 2024-09-17T21:20:41.645 INFO:teuthology.orchestra.run.smithi184.stdout:5211 trunc from 0x6b07600 to 0x4d88400 2024-09-17T21:20:41.663 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:41.656+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:41.669 INFO:teuthology.orchestra.run.smithi184.stdout:5212 read 0x4703a00 thru 0x4709fb4 (0x65b5 bytes) 2024-09-17T21:20:41.673 INFO:teuthology.orchestra.run.smithi184.stdout:5214 read 0x1205800 thru 0x1205eee (0x6ef bytes) 2024-09-17T21:20:41.699 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:41.699 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:41.704 INFO:teuthology.orchestra.run.smithi184.stdout:5215 write 0xa203000 thru 0xa20e8dd (0xb8de bytes) 2024-09-17T21:20:41.705 INFO:teuthology.orchestra.run.smithi184.stdout:5216 write 0x3df9600 thru 0x3e07f39 (0xe93a bytes) 2024-09-17T21:20:41.706 INFO:teuthology.orchestra.run.smithi184.stdout:5217 write 0x5d8f200 thru 0x5d9b8ce (0xc6cf bytes) 2024-09-17T21:20:41.714 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:20:41.718 INFO:teuthology.orchestra.run.smithi184.stdout:5218 trunc from 0xa20ea00 to 0x5ba4c00 2024-09-17T21:20:41.764 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:41.758+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:41.768 INFO:teuthology.orchestra.run.smithi184.stdout:5220 punch from 0x219a400 to 0x219ae00, (0xa00 bytes) 2024-09-17T21:20:41.770 INFO:teuthology.orchestra.run.smithi184.stdout:5222 punch from 0xa5d600 to 0xa67c00, (0xa600 bytes) 2024-09-17T21:20:41.785 INFO:teuthology.orchestra.run.smithi184.stdout:5223 trunc from 0x5ba4c00 to 0xd9e8a00 2024-09-17T21:20:41.838 INFO:teuthology.orchestra.run.smithi184.stdout:5224 write 0xea4ba00 thru 0xea5853b (0xcb3c bytes) 2024-09-17T21:20:41.839 INFO:teuthology.orchestra.run.smithi184.stdout:5226 read 0x830f800 thru 0x83186b6 (0x8eb7 bytes) 2024-09-17T21:20:41.839 INFO:teuthology.orchestra.run.smithi184.stdout:5230 write 0xc2c7000 thru 0xc2c9df4 (0x2df5 bytes) 2024-09-17T21:20:41.850 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:41.850 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:41.865 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:20:41.866 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:41.858+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:41.911 INFO:teuthology.orchestra.run.smithi184.stdout:5234 trunc from 0xea58600 to 0x4e1a00 2024-09-17T21:20:41.966 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:41.960+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:42.011 INFO:teuthology.orchestra.run.smithi184.stdout:5240 read 0x19da00 thru 0x1acad9 (0xf0da bytes) 2024-09-17T21:20:42.015 INFO:teuthology.orchestra.run.smithi184.stdout:5241 read 0x464200 thru 0x46ee81 (0xac82 bytes) 2024-09-17T21:20:42.035 INFO:teuthology.orchestra.run.smithi184.stdout:5243 trunc from 0x4e1a00 to 0x7c1d600 2024-09-17T21:20:42.042 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:42.042 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:42.057 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:20:42.066 INFO:teuthology.orchestra.run.smithi184.stdout:5245 trunc from 0x7c1d600 to 0x9e0d000 2024-09-17T21:20:42.092 INFO:teuthology.orchestra.run.smithi184.stdout:5246 write 0x9212000 thru 0x921c94c (0xa94d bytes) 2024-09-17T21:20:42.107 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:42.102+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:42.116 INFO:teuthology.orchestra.run.smithi184.stdout:5251 trunc from 0x9e0d000 to 0x1d84600 2024-09-17T21:20:42.155 INFO:teuthology.orchestra.run.smithi184.stdout:5252 clone 36 order 22 su 2097152 sc 13 2024-09-17T21:20:42.167 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:42.162+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:42.185 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:42.185 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:42.200 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:20:42.269 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:42.264+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:42.331 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:42.331 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:42.345 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:20:42.370 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:42.362+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:42.454 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-17T21:20:42.471 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:42.466+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:42.518 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:42.518 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:42.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:20:42.580 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:42.576+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:42.629 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone35 from 0x1d84600 (overlap 0x4e1a00) to 0x36aa00 2024-09-17T21:20:42.665 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:42.665 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:42.672 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:42.668+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:42.681 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:20:42.774 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:42.766+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:42.850 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:42.850 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:42.864 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:20:42.909 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:42.904+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:42.975 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:42.970+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:42.992 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:42.993 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:43.007 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:20:43.076 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:43.070+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:43.133 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 0, 5] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:20:43.133 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:20:43.133 INFO:tasks.thrashosds.thrasher:Setting full ratio to .001 2024-09-17T21:20:43.133 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .001 2024-09-17T21:20:43.172 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:43.172 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:43.177 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:43.170+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:43.187 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:20:43.250 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-17T21:20:43.277 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:43.272+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:43.357 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:43.357 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:43.371 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:20:43.374 INFO:teuthology.orchestra.run.smithi184.stdout:5254 punch from 0x799a00 to 0x79fc00, (0x6200 bytes) 2024-09-17T21:20:43.379 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:43.374+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:43.459 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:20:43.476 INFO:teuthology.orchestra.run.smithi184.stdout:5255 write 0xbdab600 thru 0xbdb1ed7 (0x68d8 bytes) 2024-09-17T21:20:43.480 INFO:teuthology.orchestra.run.smithi184.stdout:5256 punch from 0x7849a00 to 0x7856800, (0xce00 bytes) 2024-09-17T21:20:43.480 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:43.474+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:43.480 INFO:teuthology.orchestra.run.smithi184.stdout:5257 punch from 0xbd9f400 to 0xbdaba00, (0xc600 bytes) 2024-09-17T21:20:43.489 INFO:teuthology.orchestra.run.smithi184.stdout:5258 punch from 0x551a600 to 0x5529e00, (0xf800 bytes) 2024-09-17T21:20:43.489 INFO:teuthology.orchestra.run.smithi184.stdout:5259 write 0x24fbc00 thru 0x25067c2 (0xabc3 bytes) 2024-09-17T21:20:43.489 INFO:teuthology.orchestra.run.smithi184.stdout:5260 punch from 0x3061a00 to 0x3068800, (0x6e00 bytes) 2024-09-17T21:20:43.490 INFO:teuthology.orchestra.run.smithi184.stdout:5261 read 0x9edb400 thru 0x9ee3a53 (0x8654 bytes) 2024-09-17T21:20:43.490 INFO:teuthology.orchestra.run.smithi184.stdout:5262 punch from 0x205da00 to 0x2067400, (0x9a00 bytes) 2024-09-17T21:20:43.490 INFO:teuthology.orchestra.run.smithi184.stdout:5263 write 0x7d38600 thru 0x7d44221 (0xbc22 bytes) 2024-09-17T21:20:43.491 INFO:teuthology.orchestra.run.smithi184.stdout:5265 write 0x116a600 thru 0x11756fa (0xb0fb bytes) 2024-09-17T21:20:43.509 INFO:teuthology.orchestra.run.smithi184.stdout:5266 trunc from 0xbdb2000 to 0x155a600 2024-09-17T21:20:43.542 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:43.543 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:43.561 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:20:43.588 INFO:teuthology.orchestra.run.smithi184.stdout:5267 punch from 0xfcf400 to 0xfda600, (0xb200 bytes) 2024-09-17T21:20:43.604 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:43.600+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:43.605 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:43.605 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:43.622 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:43.622 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:43.626 INFO:teuthology.orchestra.run.smithi184.stdout:5270 write 0x8d27800 thru 0x8d2a74a (0x2f4b bytes) 2024-09-17T21:20:43.629 INFO:teuthology.orchestra.run.smithi184.stdout:5271 read 0x8cec200 thru 0x8cf54c9 (0x92ca bytes) 2024-09-17T21:20:43.629 INFO:teuthology.orchestra.run.smithi184.stdout:5274 punch from 0x664be00 to 0x6659e00, (0xe000 bytes) 2024-09-17T21:20:43.629 INFO:teuthology.orchestra.run.smithi184.stdout:5275 read 0x7e0ee00 thru 0x7e17631 (0x8832 bytes) 2024-09-17T21:20:43.630 INFO:teuthology.orchestra.run.smithi184.stdout:5276 write 0x2a8ec00 thru 0x2a91719 (0x2b1a bytes) 2024-09-17T21:20:43.639 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:43.639 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:43.659 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:43.659 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:43.670 INFO:teuthology.orchestra.run.smithi184.stdout:5278 write 0xd0a5200 thru 0xd0ab825 (0x6626 bytes) 2024-09-17T21:20:43.672 INFO:teuthology.orchestra.run.smithi184.stdout:5281 read 0x36dc00 thru 0x37496b (0x6d6c bytes) 2024-09-17T21:20:43.673 INFO:teuthology.orchestra.run.smithi184.stdout:5284 write 0x59d8a00 thru 0x59e4b4b (0xc14c bytes) 2024-09-17T21:20:43.673 INFO:teuthology.orchestra.run.smithi184.stdout:5285 read 0x9e40000 thru 0x9e4a3a6 (0xa3a7 bytes) 2024-09-17T21:20:43.674 INFO:teuthology.orchestra.run.smithi184.stdout:5286 read 0x8807e00 thru 0x880c5eb (0x47ec bytes) 2024-09-17T21:20:43.674 INFO:teuthology.orchestra.run.smithi184.stdout:5288 read 0x4cfac00 thru 0x4d035da (0x89db bytes) 2024-09-17T21:20:43.678 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:43.678 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:43.679 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:43.679 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:43.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:43.680 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:43.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.680 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:20:43.680 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:43.680 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 145.45995797500001, 2024-09-17T21:20:43.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:20:43.680 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:43.680 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:43.680 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:43.680 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:43.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:43.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:43.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:43.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:43.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:43.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:20:43.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:20:43.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:43.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:20:43.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:20:43.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:43.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:20:43.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:20:43.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:43.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:20:43.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:20:43.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:43.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:20:43.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:20:43.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:43.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:20:43.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:20:43.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:43.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:20:43.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:20:43.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:43.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:20:43.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:20:43.684 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.684 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:43.684 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.684 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:20:43.684 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:43.684 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 140.809657311, 2024-09-17T21:20:43.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:20:43.684 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:43.684 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:43.685 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:43.685 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:43.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:43.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:43.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:43.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:43.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:43.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:20:43.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:20:43.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:43.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:20:43.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:20:43.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:43.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:20:43.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:20:43.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:43.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:20:43.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:20:43.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:43.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:20:43.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:20:43.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:43.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:20:43.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:20:43.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:43.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:20:43.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:20:43.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:43.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:20:43.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:20:43.688 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.688 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:43.688 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.688 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:20:43.689 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:43.689 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 140.808392228, 2024-09-17T21:20:43.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:20:43.689 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:43.689 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:43.689 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:43.689 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:43.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:43.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:43.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:43.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:43.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:43.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:20:43.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:20:43.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:43.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:20:43.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:20:43.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:43.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:20:43.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:20:43.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:43.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:20:43.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:20:43.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:43.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:20:43.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:20:43.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:43.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:20:43.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:20:43.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:43.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:20:43.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:20:43.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:43.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:20:43.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:20:43.693 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.693 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:43.693 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.693 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:20:43.693 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:43.693 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 140.79861281800001, 2024-09-17T21:20:43.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:20:43.693 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:43.693 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:43.693 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:43.693 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:43.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:43.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:43.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:43.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:43.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:43.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:20:43.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:20:43.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:43.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:20:43.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:20:43.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:43.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:20:43.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:20:43.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:43.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:20:43.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:20:43.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:43.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:20:43.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:20:43.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:43.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:20:43.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:20:43.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:43.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:20:43.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:20:43.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:43.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:20:43.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:20:43.697 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.697 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:43.697 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.697 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:20:43.697 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:43.697 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 140.789890415, 2024-09-17T21:20:43.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:20:43.698 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:43.698 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:43.698 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:43.698 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:43.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:43.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:43.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:43.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:43.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:43.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:20:43.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:20:43.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:43.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:20:43.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:20:43.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:43.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:20:43.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:20:43.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:43.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:20:43.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:20:43.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:43.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:20:43.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:20:43.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:43.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:20:43.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:20:43.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:43.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:20:43.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:20:43.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:43.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:20:43.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:20:43.702 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.702 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:43.702 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.702 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:20:43.702 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:43.702 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 127.40479985100001, 2024-09-17T21:20:43.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:20:43.702 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:43.702 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:43.702 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:43.702 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:43.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:43.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:43.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:43.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:43.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:43.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:20:43.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:20:43.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:43.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:20:43.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:20:43.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:43.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:20:43.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:20:43.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:43.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:20:43.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:20:43.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:43.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:20:43.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:20:43.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:43.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:20:43.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:20:43.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:43.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:20:43.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:20:43.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:43.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:20:43.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:20:43.706 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.706 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:43.706 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.706 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1780 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'195, mlcod=69'190)", 2024-09-17T21:20:43.706 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:43.706 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 90.426944246999994, 2024-09-17T21:20:43.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075761769999999999, 2024-09-17T21:20:43.707 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:43.707 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:43.707 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:43.707 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:43.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:43.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:43.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:43.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:43.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:43.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250375+0000", 2024-09-17T21:20:43.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:20:43.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:43.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250383+0000", 2024-09-17T21:20:43.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7870000000000002e-06 2024-09-17T21:20:43.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:43.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250402+0000", 2024-09-17T21:20:43.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9346000000000001e-05 2024-09-17T21:20:43.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:43.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250413+0000", 2024-09-17T21:20:43.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.039e-05 2024-09-17T21:20:43.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:43.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251353+0000", 2024-09-17T21:20:43.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000940091 2024-09-17T21:20:43.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:43.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251407+0000", 2024-09-17T21:20:43.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4701999999999999e-05 2024-09-17T21:20:43.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:43.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257829+0000", 2024-09-17T21:20:43.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064212779999999999 2024-09-17T21:20:43.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:43.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257950+0000", 2024-09-17T21:20:43.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121623 2024-09-17T21:20:43.710 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.710 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:43.710 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.711 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1781 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'196, mlcod=69'190)", 2024-09-17T21:20:43.711 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:43.711 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 90.425861295999994, 2024-09-17T21:20:43.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065632939999999999, 2024-09-17T21:20:43.711 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:43.711 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:43.711 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:43.711 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:43.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:43.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:43.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:43.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:43.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:43.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251459+0000", 2024-09-17T21:20:43.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.624e-06 2024-09-17T21:20:43.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:43.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251471+0000", 2024-09-17T21:20:43.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2636000000000001e-05 2024-09-17T21:20:43.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:43.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251498+0000", 2024-09-17T21:20:43.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:20:43.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:43.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251515+0000", 2024-09-17T21:20:43.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6939000000000002e-05 2024-09-17T21:20:43.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:43.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252938+0000", 2024-09-17T21:20:43.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001423082 2024-09-17T21:20:43.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:43.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252999+0000", 2024-09-17T21:20:43.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1138999999999999e-05 2024-09-17T21:20:43.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:43.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257955+0000", 2024-09-17T21:20:43.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049559640000000002 2024-09-17T21:20:43.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:43.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.258020+0000", 2024-09-17T21:20:43.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5504000000000002e-05 2024-09-17T21:20:43.715 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.715 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:43.715 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.715 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1788 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'198, mlcod=69'191)", 2024-09-17T21:20:43.715 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:43.715 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 90.423835807000003, 2024-09-17T21:20:43.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073690370000000002, 2024-09-17T21:20:43.715 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:43.715 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:43.716 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:43.716 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:43.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:43.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:43.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:43.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:43.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:43.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253484+0000", 2024-09-17T21:20:43.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2470000000000001e-06 2024-09-17T21:20:43.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:43.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253492+0000", 2024-09-17T21:20:43.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7159999999999992e-06 2024-09-17T21:20:43.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:43.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253509+0000", 2024-09-17T21:20:43.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.666e-05 2024-09-17T21:20:43.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:43.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253522+0000", 2024-09-17T21:20:43.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2643000000000001e-05 2024-09-17T21:20:43.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:43.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254853+0000", 2024-09-17T21:20:43.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331238 2024-09-17T21:20:43.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:43.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254885+0000", 2024-09-17T21:20:43.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1562e-05 2024-09-17T21:20:43.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:43.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260788+0000", 2024-09-17T21:20:43.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059030749999999998 2024-09-17T21:20:43.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:43.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260852+0000", 2024-09-17T21:20:43.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3895999999999999e-05 2024-09-17T21:20:43.719 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.719 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:43.719 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.720 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5746 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'477, mlcod=71'473)", 2024-09-17T21:20:43.720 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:43.720 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.418408248999995, 2024-09-17T21:20:43.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074182709999999997, 2024-09-17T21:20:43.720 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:43.720 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:43.720 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:43.720 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:43.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.720 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:43.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:43.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:43.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:43.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:43.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258912+0000", 2024-09-17T21:20:43.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:20:43.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:43.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258926+0000", 2024-09-17T21:20:43.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4249e-05 2024-09-17T21:20:43.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:43.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258946+0000", 2024-09-17T21:20:43.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9879999999999999e-05 2024-09-17T21:20:43.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:43.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258956+0000", 2024-09-17T21:20:43.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:20:43.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:43.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259380+0000", 2024-09-17T21:20:43.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042337599999999999 2024-09-17T21:20:43.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:43.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259412+0000", 2024-09-17T21:20:43.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:20:43.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:43.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266191+0000", 2024-09-17T21:20:43.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067793269999999999 2024-09-17T21:20:43.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:43.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266328+0000", 2024-09-17T21:20:43.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013690999999999999 2024-09-17T21:20:43.724 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.724 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:43.724 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.724 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5747 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'478, mlcod=71'473)", 2024-09-17T21:20:43.724 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:43.724 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.417701899999997, 2024-09-17T21:20:43.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068779440000000004, 2024-09-17T21:20:43.724 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:43.724 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:43.725 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:43.725 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:43.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:43.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:43.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:43.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:43.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:43.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259618+0000", 2024-09-17T21:20:43.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6199999999999999e-06 2024-09-17T21:20:43.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:43.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259630+0000", 2024-09-17T21:20:43.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1884e-05 2024-09-17T21:20:43.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:43.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259657+0000", 2024-09-17T21:20:43.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7339e-05 2024-09-17T21:20:43.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:43.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259677+0000", 2024-09-17T21:20:43.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0156000000000001e-05 2024-09-17T21:20:43.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:43.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260521+0000", 2024-09-17T21:20:43.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084312899999999997 2024-09-17T21:20:43.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:43.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260585+0000", 2024-09-17T21:20:43.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4558000000000001e-05 2024-09-17T21:20:43.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:43.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266336+0000", 2024-09-17T21:20:43.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057505020000000002 2024-09-17T21:20:43.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:43.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266494+0000", 2024-09-17T21:20:43.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158756 2024-09-17T21:20:43.728 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.728 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:43.728 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.729 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5750 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'479, mlcod=71'473)", 2024-09-17T21:20:43.729 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:43.729 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.416989706999999, 2024-09-17T21:20:43.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006302874, 2024-09-17T21:20:43.729 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:43.729 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:43.729 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:43.729 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:43.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:43.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:43.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260330+0000", 2024-09-17T21:20:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:20:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260343+0000", 2024-09-17T21:20:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2862e-05 2024-09-17T21:20:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260367+0000", 2024-09-17T21:20:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3723e-05 2024-09-17T21:20:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260386+0000", 2024-09-17T21:20:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8879999999999999e-05 2024-09-17T21:20:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261579+0000", 2024-09-17T21:20:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001193431 2024-09-17T21:20:43.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:43.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261608+0000", 2024-09-17T21:20:43.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8858999999999999e-05 2024-09-17T21:20:43.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:43.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266505+0000", 2024-09-17T21:20:43.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004896999 2024-09-17T21:20:43.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:43.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266631+0000", 2024-09-17T21:20:43.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012655300000000001 2024-09-17T21:20:43.733 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.733 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:43.733 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.733 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5751 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'480, mlcod=71'473)", 2024-09-17T21:20:43.733 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:43.733 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.416282495000004, 2024-09-17T21:20:43.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075756299999999999, 2024-09-17T21:20:43.733 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:43.733 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:43.733 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:43.733 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:43.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:43.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:43.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:43.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:43.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:43.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261037+0000", 2024-09-17T21:20:43.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:20:43.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:43.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261049+0000", 2024-09-17T21:20:43.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1063999999999999e-05 2024-09-17T21:20:43.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:43.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261072+0000", 2024-09-17T21:20:43.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3102e-05 2024-09-17T21:20:43.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:43.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261133+0000", 2024-09-17T21:20:43.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1276000000000001e-05 2024-09-17T21:20:43.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:43.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:20:43.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015334319999999999 2024-09-17T21:20:43.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:43.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262702+0000", 2024-09-17T21:20:43.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5281999999999999e-05 2024-09-17T21:20:43.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:43.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268527+0000", 2024-09-17T21:20:43.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058250389999999997 2024-09-17T21:20:43.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:43.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268611+0000", 2024-09-17T21:20:43.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4724999999999999e-05 2024-09-17T21:20:43.737 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.737 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:43.737 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.737 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5756 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'481, mlcod=71'473)", 2024-09-17T21:20:43.737 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:43.738 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.415421639000002, 2024-09-17T21:20:43.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006772386, 2024-09-17T21:20:43.738 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:43.738 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:43.738 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:43.738 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:43.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:43.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:43.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:43.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:43.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:43.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261899+0000", 2024-09-17T21:20:43.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.903e-06 2024-09-17T21:20:43.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:43.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261912+0000", 2024-09-17T21:20:43.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-05 2024-09-17T21:20:43.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:43.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261938+0000", 2024-09-17T21:20:43.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6098999999999999e-05 2024-09-17T21:20:43.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:43.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261955+0000", 2024-09-17T21:20:43.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.696e-05 2024-09-17T21:20:43.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:43.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263845+0000", 2024-09-17T21:20:43.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001890187 2024-09-17T21:20:43.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:43.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263907+0000", 2024-09-17T21:20:43.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1600999999999995e-05 2024-09-17T21:20:43.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:43.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268615+0000", 2024-09-17T21:20:43.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047086070000000001 2024-09-17T21:20:43.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:43.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268669+0000", 2024-09-17T21:20:43.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3588999999999998e-05 2024-09-17T21:20:43.741 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.741 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:43.742 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.742 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4978.0:598 3.9 e80/23 3:9701cc17:::rbd_data.1152724ff2a3.0000000000000019:head v 80'202, mlcod=80'198)", 2024-09-17T21:20:43.742 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:43.742 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 76.913847255999997, 2024-09-17T21:20:43.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066473340000000004, 2024-09-17T21:20:43.742 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:43.742 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:43.742 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:43.742 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:43.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:43.742 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:43.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:43.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:43.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:43.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763472+0000", 2024-09-17T21:20:43.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6899999999999997e-07 2024-09-17T21:20:43.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:43.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763478+0000", 2024-09-17T21:20:43.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:20:43.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:43.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763489+0000", 2024-09-17T21:20:43.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0628999999999999e-05 2024-09-17T21:20:43.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:43.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763495+0000", 2024-09-17T21:20:43.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6649999999999997e-06 2024-09-17T21:20:43.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:43.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763907+0000", 2024-09-17T21:20:43.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041163899999999999 2024-09-17T21:20:43.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:43.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763976+0000", 2024-09-17T21:20:43.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8534999999999993e-05 2024-09-17T21:20:43.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:43.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.769900+0000", 2024-09-17T21:20:43.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059243270000000001 2024-09-17T21:20:43.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:43.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.770118+0000", 2024-09-17T21:20:43.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021842500000000001 2024-09-17T21:20:43.746 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.746 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:43.746 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.747 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:468 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'94, mlcod=89'87)", 2024-09-17T21:20:43.747 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:43.747 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.991101956999998, 2024-09-17T21:20:43.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066528790000000004, 2024-09-17T21:20:43.747 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:43.747 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:43.747 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:43.747 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:43.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.747 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:43.747 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:43.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:43.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:43.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:43.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686218+0000", 2024-09-17T21:20:43.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3060000000000001e-06 2024-09-17T21:20:43.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:43.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686256+0000", 2024-09-17T21:20:43.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8333e-05 2024-09-17T21:20:43.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:43.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686272+0000", 2024-09-17T21:20:43.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:20:43.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:43.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686283+0000", 2024-09-17T21:20:43.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0903e-05 2024-09-17T21:20:43.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:43.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686682+0000", 2024-09-17T21:20:43.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039935099999999999 2024-09-17T21:20:43.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:43.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686785+0000", 2024-09-17T21:20:43.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010275199999999999 2024-09-17T21:20:43.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:43.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692816+0000", 2024-09-17T21:20:43.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060317579999999999 2024-09-17T21:20:43.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:43.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692869+0000", 2024-09-17T21:20:43.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2775999999999998e-05 2024-09-17T21:20:43.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.751 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:43.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.751 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:755 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'127, mlcod=89'122)", 2024-09-17T21:20:43.751 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:43.751 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.854646743000004, 2024-09-17T21:20:43.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084014080000000008, 2024-09-17T21:20:43.752 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:43.752 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:43.752 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:43.752 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:43.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.752 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:43.752 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:43.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.752 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:43.752 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:43.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:43.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:43.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:20:43.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:43.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822683+0000", 2024-09-17T21:20:43.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0674e-05 2024-09-17T21:20:43.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:43.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822698+0000", 2024-09-17T21:20:43.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5189e-05 2024-09-17T21:20:43.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:43.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822708+0000", 2024-09-17T21:20:43.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9119999999999993e-06 2024-09-17T21:20:43.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:43.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822748+0000", 2024-09-17T21:20:43.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9727000000000001e-05 2024-09-17T21:20:43.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:43.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822759+0000", 2024-09-17T21:20:43.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0865e-05 2024-09-17T21:20:43.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:43.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831034+0000", 2024-09-17T21:20:43.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082750710000000002 2024-09-17T21:20:43.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:43.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831073+0000", 2024-09-17T21:20:43.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9063999999999997e-05 2024-09-17T21:20:43.756 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.756 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:43.756 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.756 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:756 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'128, mlcod=89'124)", 2024-09-17T21:20:43.756 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:43.756 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.852724111000001, 2024-09-17T21:20:43.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0076300270000000002, 2024-09-17T21:20:43.756 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:43.757 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:43.757 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:43.757 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:43.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:43.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:43.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:43.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:43.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:43.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824595+0000", 2024-09-17T21:20:43.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-17T21:20:43.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:43.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824600+0000", 2024-09-17T21:20:43.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:20:43.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:43.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824612+0000", 2024-09-17T21:20:43.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.181e-05 2024-09-17T21:20:43.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:43.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824619+0000", 2024-09-17T21:20:43.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:20:43.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:43.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824671+0000", 2024-09-17T21:20:43.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2262999999999998e-05 2024-09-17T21:20:43.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:43.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824695+0000", 2024-09-17T21:20:43.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3883000000000002e-05 2024-09-17T21:20:43.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:43.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832185+0000", 2024-09-17T21:20:43.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007490048 2024-09-17T21:20:43.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:43.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832224+0000", 2024-09-17T21:20:43.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8957e-05 2024-09-17T21:20:43.760 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.760 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:43.761 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.761 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2617 3.d e90/23 3:b0143718:::rbd_data.1152a8ae2019.0000000000000016:head v 90'249, mlcod=90'246)", 2024-09-17T21:20:43.761 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:43.761 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 62.832873110999998, 2024-09-17T21:20:43.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063715559999999996, 2024-09-17T21:20:43.761 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:43.761 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:43.761 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:43.761 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:43.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:43.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:43.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:43.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:43.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:43.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844455+0000", 2024-09-17T21:20:43.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4259999999999992e-06 2024-09-17T21:20:43.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:43.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844470+0000", 2024-09-17T21:20:43.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5386000000000001e-05 2024-09-17T21:20:43.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:43.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844499+0000", 2024-09-17T21:20:43.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.883e-05 2024-09-17T21:20:43.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:43.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844518+0000", 2024-09-17T21:20:43.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8830000000000001e-05 2024-09-17T21:20:43.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:43.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844649+0000", 2024-09-17T21:20:43.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013143399999999999 2024-09-17T21:20:43.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:43.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844702+0000", 2024-09-17T21:20:43.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2876e-05 2024-09-17T21:20:43.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:43.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850696+0000", 2024-09-17T21:20:43.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059943729999999999 2024-09-17T21:20:43.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:43.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850817+0000", 2024-09-17T21:20:43.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120401 2024-09-17T21:20:43.765 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.765 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:43.765 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.765 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2635 3.d e90/23 3:b3d1e920:::rbd_data.1152a8ae2019.000000000000001f:head v 90'252, mlcod=90'246)", 2024-09-17T21:20:43.765 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:43.765 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 62.830165041000001, 2024-09-17T21:20:43.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0077265190000000003, 2024-09-17T21:20:43.766 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:43.766 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:43.766 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:43.766 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:43.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:43.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:43.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:43.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:43.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:43.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:43.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847155+0000", 2024-09-17T21:20:43.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.544e-06 2024-09-17T21:20:43.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:43.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847165+0000", 2024-09-17T21:20:43.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.043e-05 2024-09-17T21:20:43.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:43.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847189+0000", 2024-09-17T21:20:43.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3659000000000002e-05 2024-09-17T21:20:43.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:43.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847210+0000", 2024-09-17T21:20:43.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1163e-05 2024-09-17T21:20:43.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:43.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848856+0000", 2024-09-17T21:20:43.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001645521 2024-09-17T21:20:43.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:43.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848905+0000", 2024-09-17T21:20:43.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9048999999999997e-05 2024-09-17T21:20:43.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:43.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854770+0000", 2024-09-17T21:20:43.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058652879999999998 2024-09-17T21:20:43.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:43.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:43.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:43.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854880+0000", 2024-09-17T21:20:43.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010986500000000001 2024-09-17T21:20:43.769 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.770 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:43.770 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.770 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:43.770 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:43.770 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:43.772 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:20:43.772 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:43.773 INFO:teuthology.orchestra.run.smithi184.stdout:5289 read 0x1003e00 thru 0x1010fdb (0xd1dc bytes) 2024-09-17T21:20:43.773 INFO:teuthology.orchestra.run.smithi184.stdout:5290 trunc from 0xd0aba00 to 0xad0ba00 2024-09-17T21:20:43.773 INFO:teuthology.orchestra.run.smithi184.stdout:5291 write 0x1c2a00 thru 0x1c63a9 (0x39aa bytes) 2024-09-17T21:20:43.773 INFO:teuthology.orchestra.run.smithi184.stdout:5293 read 0x239800 thru 0x244f02 (0xb703 bytes) 2024-09-17T21:20:43.773 INFO:teuthology.orchestra.run.smithi184.stdout:5296 write 0x7456c00 thru 0x7463840 (0xcc41 bytes) 2024-09-17T21:20:43.773 INFO:teuthology.orchestra.run.smithi184.stdout:5297 punch from 0x34e8600 to 0x34ed200, (0x4c00 bytes) 2024-09-17T21:20:43.773 INFO:teuthology.orchestra.run.smithi184.stdout:5298 punch from 0x37b7c00 to 0x37b8800, (0xc00 bytes) 2024-09-17T21:20:43.773 INFO:teuthology.orchestra.run.smithi184.stdout:5301 write 0x6b16400 thru 0x6b23719 (0xd31a bytes) 2024-09-17T21:20:43.774 INFO:teuthology.orchestra.run.smithi184.stdout:5303 read 0x28a5000 thru 0x28ad56d (0x856e bytes) 2024-09-17T21:20:43.774 INFO:teuthology.orchestra.run.smithi184.stdout:5304 clone 37 order 20 su 16384 sc 13 2024-09-17T21:20:43.774 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:43.775 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:43.766+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:43.873 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:43.866+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:43.901 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:43.901 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:43.901 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:20:43.901 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:43.915 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:20:44.003 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:43.998+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:44.074 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:44.066+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:44.082 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:44.082 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:44.082 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:44.082 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:44.083 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:44.097 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:20:44.191 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:44.186+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:44.273 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:44.273 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:44.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:44.273 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:44.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.273 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:20:44.273 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:20:44.273 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 167.25446410399999, 2024-09-17T21:20:44.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:20:44.273 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.273 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.274 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:44.274 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.274 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:20:44.274 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:20:44.274 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:20:44.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.274 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:20:44.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:20:44.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:20:44.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:20:44.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:20:44.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:20:44.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:20:44.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:20:44.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:20:44.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:20:44.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:20:44.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:20:44.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:44.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:20:44.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:20:44.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:20:44.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:20:44.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:20:44.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:20:44.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:20:44.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:20:44.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:20:44.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:44.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:20:44.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:20:44.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:44.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:20:44.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:20:44.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:44.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:20:44.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:20:44.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:20:44.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:20:44.279 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.279 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.279 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.280 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:20:44.280 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:20:44.280 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 147.674299824, 2024-09-17T21:20:44.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:20:44.280 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.280 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.280 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.280 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.280 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:20:44.280 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:20:44.280 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:20:44.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.280 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:20:44.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:20:44.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:20:44.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:20:44.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:20:44.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:20:44.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:20:44.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:20:44.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:20:44.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:20:44.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:20:44.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:20:44.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:20:44.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:20:44.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:20:44.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:20:44.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:20:44.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:44.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:20:44.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:20:44.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:20:44.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:20:44.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:20:44.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:20:44.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:20:44.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:20:44.285 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.285 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.285 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.286 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:20:44.286 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:20:44.286 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 147.67420251199999, 2024-09-17T21:20:44.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:20:44.286 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.286 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.286 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.286 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.286 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:44.286 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:44.286 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:20:44.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.286 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:20:44.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:20:44.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:20:44.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:20:44.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:20:44.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:20:44.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:20:44.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:20:44.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:20:44.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:20:44.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:20:44.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:20:44.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:20:44.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:20:44.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:20:44.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:20:44.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:20:44.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:44.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:20:44.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:20:44.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:20:44.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:20:44.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:20:44.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:20:44.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:20:44.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:20:44.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:20:44.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:20:44.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:20:44.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:20:44.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:20:44.291 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.291 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.292 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.292 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:20:44.292 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:20:44.292 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 147.672115711, 2024-09-17T21:20:44.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:20:44.292 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.292 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.292 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:44.292 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.292 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:20:44.292 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:20:44.292 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:20:44.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.293 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:20:44.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:20:44.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:20:44.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:20:44.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:20:44.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:20:44.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:20:44.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:20:44.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:20:44.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:20:44.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:20:44.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:20:44.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:20:44.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:20:44.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:20:44.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:20:44.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:20:44.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:44.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:20:44.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:20:44.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:20:44.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:20:44.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:20:44.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:20:44.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:44.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:20:44.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:20:44.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:44.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:20:44.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:20:44.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:44.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:20:44.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:20:44.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:44.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:20:44.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:20:44.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:20:44.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:20:44.298 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.298 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.298 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.299 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:44.299 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:20:44.299 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 114.98821692999999, 2024-09-17T21:20:44.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:20:44.299 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.299 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.299 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:20:44.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:20:44.299 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:20:44.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.299 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:20:44.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:20:44.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:20:44.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:20:44.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:20:44.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:20:44.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:20:44.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:20:44.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:20:44.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:20:44.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:20:44.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:20:44.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:20:44.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:20:44.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:20:44.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:20:44.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:20:44.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:44.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:20:44.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:20:44.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:20:44.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:20:44.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:20:44.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:20:44.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:20:44.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:20:44.304 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.304 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.304 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.304 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:44.304 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:20:44.304 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 114.979125219, 2024-09-17T21:20:44.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:20:44.304 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.305 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.305 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.305 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.305 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:20:44.305 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:20:44.305 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:20:44.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.305 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:20:44.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:20:44.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:20:44.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:20:44.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:20:44.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:20:44.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:20:44.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:20:44.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:20:44.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:20:44.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:20:44.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:20:44.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:20:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:20:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:20:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:20:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:20:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:20:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:20:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:20:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:20:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 114.978953168, 2024-09-17T21:20:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:20:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:20:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:20:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:20:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:20:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:20:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:20:44.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:20:44.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:20:44.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:20:44.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:20:44.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:20:44.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:20:44.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:20:44.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:20:44.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:20:44.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:44.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:20:44.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:20:44.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:20:44.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:20:44.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:20:44.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:20:44.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:20:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:20:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:20:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 114.804020421, 2024-09-17T21:20:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:20:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:20:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:20:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:20:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:20:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:20:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:20:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:20:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:20:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:20:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:20:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:20:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:20:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:20:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:20:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:20:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:20:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:20:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:20:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:20:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:20:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:20:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 114.803935864, 2024-09-17T21:20:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:20:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:20:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:20:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:20:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:20:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:20:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:20:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:20:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:20:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:20:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:20:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:20:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:20:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:20:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:20:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:20:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:20:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:20:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:20:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:20:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 114.803870227, 2024-09-17T21:20:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:20:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:20:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:20:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:20:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:20:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:20:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:20:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:20:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:20:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:20:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:20:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:20:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:20:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:20:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:20:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:20:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:20:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:20:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:20:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:20:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:20:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:20:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:20:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 63.430295129999998, 2024-09-17T21:20:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:20:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:20:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:20:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:20:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:20:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:20:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:20:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:20:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:20:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:20:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:20:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:20:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:20:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:20:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:20:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:20:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:20:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:20:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:20:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 63.430175618, 2024-09-17T21:20:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:20:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:20:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:20:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:20:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:20:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:20:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:20:44.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:20:44.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:20:44.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:20:44.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:20:44.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:20:44.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:20:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:20:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:20:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:20:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:20:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:20:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:994 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 571392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:20:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.837203643999999, 2024-09-17T21:20:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083898465000000005, 2024-09-17T21:20:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 994 2024-09-17T21:20:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:20:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:20:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:20:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:20:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431058+0000", 2024-09-17T21:20:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1667e-05 2024-09-17T21:20:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431067+0000", 2024-09-17T21:20:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0720000000000006e-06 2024-09-17T21:20:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431075+0000", 2024-09-17T21:20:44.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9200000000000004e-06 2024-09-17T21:20:44.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514671+0000", 2024-09-17T21:20:44.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083596229999999994 2024-09-17T21:20:44.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514741+0000", 2024-09-17T21:20:44.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9929000000000001e-05 2024-09-17T21:20:44.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514924+0000", 2024-09-17T21:20:44.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000183122 2024-09-17T21:20:44.343 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.343 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.343 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.343 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:995 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 575488~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:44.343 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:20:44.344 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.837144628000001, 2024-09-17T21:20:44.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084214699000000004, 2024-09-17T21:20:44.344 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.344 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.344 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.344 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.344 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:44.344 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:44.344 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 995 2024-09-17T21:20:44.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.344 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:20:44.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:20:44.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431086+0000", 2024-09-17T21:20:44.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7999999999999995e-07 2024-09-17T21:20:44.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431088+0000", 2024-09-17T21:20:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4729999999999999e-06 2024-09-17T21:20:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431126+0000", 2024-09-17T21:20:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7583999999999998e-05 2024-09-17T21:20:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431137+0000", 2024-09-17T21:20:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1768e-05 2024-09-17T21:20:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515003+0000", 2024-09-17T21:20:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083865097 2024-09-17T21:20:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515115+0000", 2024-09-17T21:20:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112561 2024-09-17T21:20:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515300+0000", 2024-09-17T21:20:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018453600000000001 2024-09-17T21:20:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:996 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:20:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.837062246999999, 2024-09-17T21:20:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084493736999999999, 2024-09-17T21:20:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 996 2024-09-17T21:20:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:20:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:20:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431168+0000", 2024-09-17T21:20:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4e-07 2024-09-17T21:20:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431172+0000", 2024-09-17T21:20:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.608e-06 2024-09-17T21:20:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431180+0000", 2024-09-17T21:20:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5650000000000002e-06 2024-09-17T21:20:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431188+0000", 2024-09-17T21:20:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:20:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515354+0000", 2024-09-17T21:20:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084165977000000003 2024-09-17T21:20:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515451+0000", 2024-09-17T21:20:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6669000000000006e-05 2024-09-17T21:20:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515661+0000", 2024-09-17T21:20:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021021599999999999 2024-09-17T21:20:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:997 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 583680~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:20:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.837030852000002, 2024-09-17T21:20:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084772419000000002, 2024-09-17T21:20:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 997 2024-09-17T21:20:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:20:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:20:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:20:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0400000000000005e-07 2024-09-17T21:20:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431202+0000", 2024-09-17T21:20:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:20:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431208+0000", 2024-09-17T21:20:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4740000000000002e-06 2024-09-17T21:20:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431214+0000", 2024-09-17T21:20:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6550000000000001e-06 2024-09-17T21:20:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515741+0000", 2024-09-17T21:20:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084526904 2024-09-17T21:20:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515826+0000", 2024-09-17T21:20:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4643999999999998e-05 2024-09-17T21:20:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515971+0000", 2024-09-17T21:20:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145444 2024-09-17T21:20:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:998 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 587776~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:20:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.837007880000002, 2024-09-17T21:20:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085108565999999997, 2024-09-17T21:20:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 998 2024-09-17T21:20:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:20:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:20:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:20:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:20:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431224+0000", 2024-09-17T21:20:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:20:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431231+0000", 2024-09-17T21:20:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000002e-06 2024-09-17T21:20:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431236+0000", 2024-09-17T21:20:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.648e-06 2024-09-17T21:20:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516024+0000", 2024-09-17T21:20:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084787584999999999 2024-09-17T21:20:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516122+0000", 2024-09-17T21:20:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8076000000000001e-05 2024-09-17T21:20:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516330+0000", 2024-09-17T21:20:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020808 2024-09-17T21:20:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1000 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 591872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:20:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.836985765000001, 2024-09-17T21:20:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085389065, 2024-09-17T21:20:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1000 2024-09-17T21:20:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:20:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:20:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:20:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:20:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431246+0000", 2024-09-17T21:20:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1749999999999999e-06 2024-09-17T21:20:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431253+0000", 2024-09-17T21:20:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2790000000000004e-06 2024-09-17T21:20:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431258+0000", 2024-09-17T21:20:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3299999999999998e-06 2024-09-17T21:20:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516384+0000", 2024-09-17T21:20:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085125549999999994 2024-09-17T21:20:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516460+0000", 2024-09-17T21:20:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6359999999999994e-05 2024-09-17T21:20:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516633+0000", 2024-09-17T21:20:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017285900000000001 2024-09-17T21:20:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1001 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 595968~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:44.367 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:20:44.367 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.836962359000001, 2024-09-17T21:20:44.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085614715999999993, 2024-09-17T21:20:44.367 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.367 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.367 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.367 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.367 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:44.367 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:44.367 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1001 2024-09-17T21:20:44.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.367 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:20:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:20:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438066+0000", 2024-09-17T21:20:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067989490000000003 2024-09-17T21:20:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438073+0000", 2024-09-17T21:20:44.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8419999999999999e-06 2024-09-17T21:20:44.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438090+0000", 2024-09-17T21:20:44.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6657e-05 2024-09-17T21:20:44.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438099+0000", 2024-09-17T21:20:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0529999999999996e-06 2024-09-17T21:20:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516678+0000", 2024-09-17T21:20:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078579422999999995 2024-09-17T21:20:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516739+0000", 2024-09-17T21:20:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1315000000000005e-05 2024-09-17T21:20:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516882+0000", 2024-09-17T21:20:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014247700000000001 2024-09-17T21:20:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:19998 3.5 3:a4885450:::rbd_object_map.1152d139f9f9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-17T21:20:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:20:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8656951140000002, 2024-09-17T21:20:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90715489299999996, 2024-09-17T21:20:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.372 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.372 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:44.372 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.372 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:44.372 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:44.372 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19998 2024-09-17T21:20:44.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.372 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:20:44.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:20:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402537+0000", 2024-09-17T21:20:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5600000000000001e-06 2024-09-17T21:20:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402549+0000", 2024-09-17T21:20:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1929999999999999e-05 2024-09-17T21:20:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402579+0000", 2024-09-17T21:20:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0093000000000001e-05 2024-09-17T21:20:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402595+0000", 2024-09-17T21:20:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5855999999999999e-05 2024-09-17T21:20:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402731+0000", 2024-09-17T21:20:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013610799999999999 2024-09-17T21:20:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402750+0000", 2024-09-17T21:20:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8737e-05 2024-09-17T21:20:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.305977+0000", 2024-09-17T21:20:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90322768399999998 2024-09-17T21:20:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306236+0000", 2024-09-17T21:20:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000258111 2024-09-17T21:20:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306557+0000", 2024-09-17T21:20:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032159799999999998 2024-09-17T21:20:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.308286+0000", 2024-09-17T21:20:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017292150000000001 2024-09-17T21:20:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:44.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309530+0000", 2024-09-17T21:20:44.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00124409 2024-09-17T21:20:44.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:44.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309630+0000", 2024-09-17T21:20:44.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9689e-05 2024-09-17T21:20:44.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309689+0000", 2024-09-17T21:20:44.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9222000000000003e-05 2024-09-17T21:20:44.377 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.377 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.378 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.378 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.378 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.378 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:44.379 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:20:44.380 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:44.376+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:44.459 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-09-17T21:20:44.456+0000 7fe379e94640 -1 log_channel(cluster) log [ERR] : Health check failed: full ratio(s) out of order (OSD_OUT_OF_ORDER_FULL) 2024-09-17T21:20:44.468 INFO:teuthology.orchestra.run.smithi083.stderr:osd set-full-ratio 0.001 2024-09-17T21:20:44.480 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:44.472+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:44.480 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:20:44.565 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:44.565 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:44.565 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:20:44.565 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:44.580 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:20:44.581 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:44.574+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:44.644 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone36 from 0xad0ba00 (overlap 0x155a600) to 0x0 2024-09-17T21:20:44.680 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:44.674+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:44.755 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:44.755 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:44.770 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:20:44.815 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:44.810+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:44.882 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:44.874+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:44.940 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:44.940 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:44.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:44.940 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:44.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.940 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:18003 3.1d 3:b9765580:::rbd_header.11527d2b15f6:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T21:20:44.941 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:20:44.941 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.578071801, 2024-09-17T21:20:44.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94165346900000002, 2024-09-17T21:20:44.941 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.941 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.941 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.941 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.941 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:44.941 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:44.941 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18003 2024-09-17T21:20:44.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.941 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:20:44.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:20:44.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357880+0000", 2024-09-17T21:20:44.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:20:44.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357891+0000", 2024-09-17T21:20:44.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0417999999999999e-05 2024-09-17T21:20:44.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357913+0000", 2024-09-17T21:20:44.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2215e-05 2024-09-17T21:20:44.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.944 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.944 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357922+0000", 2024-09-17T21:20:44.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4779999999999997e-06 2024-09-17T21:20:44.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.944 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.944 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358023+0000", 2024-09-17T21:20:44.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101025 2024-09-17T21:20:44.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.944 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:44.944 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358040+0000", 2024-09-17T21:20:44.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6291999999999999e-05 2024-09-17T21:20:44.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.945 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.945 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.945 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298203+0000", 2024-09-17T21:20:44.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94016297500000001 2024-09-17T21:20:44.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.945 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.945 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.945 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298626+0000", 2024-09-17T21:20:44.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000422853 2024-09-17T21:20:44.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.945 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.945 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.299532+0000", 2024-09-17T21:20:44.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00090657300000000004 2024-09-17T21:20:44.946 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.946 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.946 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.946 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15706 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 977920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:44.946 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:20:44.946 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.414970010000001, 2024-09-17T21:20:44.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091952328, 2024-09-17T21:20:44.946 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.946 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.947 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.947 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.947 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:44.947 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:44.947 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15706 2024-09-17T21:20:44.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.947 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:20:44.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:20:44.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520982+0000", 2024-09-17T21:20:44.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.277e-06 2024-09-17T21:20:44.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520988+0000", 2024-09-17T21:20:44.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4470000000000003e-06 2024-09-17T21:20:44.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521008+0000", 2024-09-17T21:20:44.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9309000000000001e-05 2024-09-17T21:20:44.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521023+0000", 2024-09-17T21:20:44.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5119999999999999e-05 2024-09-17T21:20:44.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612638+0000", 2024-09-17T21:20:44.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091615328999999995 2024-09-17T21:20:44.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.950 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.950 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.950 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612738+0000", 2024-09-17T21:20:44.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100466 2024-09-17T21:20:44.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.950 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.950 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.950 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612933+0000", 2024-09-17T21:20:44.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019437999999999999 2024-09-17T21:20:44.950 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.950 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.951 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.951 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15707 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 982016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:44.951 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:20:44.951 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.414906554000002, 2024-09-17T21:20:44.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092305228000000003, 2024-09-17T21:20:44.951 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.951 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.951 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.951 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.951 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:44.951 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:44.951 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15707 2024-09-17T21:20:44.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.952 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:20:44.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:20:44.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.953 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.953 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521045+0000", 2024-09-17T21:20:44.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4899999999999999e-06 2024-09-17T21:20:44.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.953 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.953 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521066+0000", 2024-09-17T21:20:44.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0236000000000001e-05 2024-09-17T21:20:44.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.953 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.953 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521085+0000", 2024-09-17T21:20:44.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8819999999999999e-05 2024-09-17T21:20:44.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.954 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.954 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521100+0000", 2024-09-17T21:20:44.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5146e-05 2024-09-17T21:20:44.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.954 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.954 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612978+0000", 2024-09-17T21:20:44.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091878847 2024-09-17T21:20:44.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.954 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.954 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613079+0000", 2024-09-17T21:20:44.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100503 2024-09-17T21:20:44.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.955 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.955 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613349+0000", 2024-09-17T21:20:44.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000270186 2024-09-17T21:20:44.955 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.955 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.955 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.955 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15708 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 986112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:44.955 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:20:44.955 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.414828378999999, 2024-09-17T21:20:44.956 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092547217000000001, 2024-09-17T21:20:44.956 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.956 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.956 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.956 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.956 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:44.956 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:44.956 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15708 2024-09-17T21:20:44.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.956 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.956 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.956 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:20:44.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:20:44.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521123+0000", 2024-09-17T21:20:44.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.353e-06 2024-09-17T21:20:44.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521129+0000", 2024-09-17T21:20:44.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:20:44.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521145+0000", 2024-09-17T21:20:44.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.611e-05 2024-09-17T21:20:44.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521154+0000", 2024-09-17T21:20:44.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4500000000000004e-06 2024-09-17T21:20:44.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.959 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613397+0000", 2024-09-17T21:20:44.959 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092243275 2024-09-17T21:20:44.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.959 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613503+0000", 2024-09-17T21:20:44.959 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106013 2024-09-17T21:20:44.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.959 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613669+0000", 2024-09-17T21:20:44.959 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016642299999999999 2024-09-17T21:20:44.960 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.960 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.960 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.960 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15709 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 990208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:44.960 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:20:44.960 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.414785363, 2024-09-17T21:20:44.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092728678999999994, 2024-09-17T21:20:44.960 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.960 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.960 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.960 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.960 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:44.961 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:44.961 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15709 2024-09-17T21:20:44.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.961 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:20:44.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:20:44.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.962 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.962 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521166+0000", 2024-09-17T21:20:44.962 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7000000000000004e-07 2024-09-17T21:20:44.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.962 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.962 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521169+0000", 2024-09-17T21:20:44.962 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2660000000000002e-06 2024-09-17T21:20:44.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.963 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521181+0000", 2024-09-17T21:20:44.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.208e-05 2024-09-17T21:20:44.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.963 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521194+0000", 2024-09-17T21:20:44.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2707e-05 2024-09-17T21:20:44.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613695+0000", 2024-09-17T21:20:44.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092500709 2024-09-17T21:20:44.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613750+0000", 2024-09-17T21:20:44.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.503e-05 2024-09-17T21:20:44.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613894+0000", 2024-09-17T21:20:44.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144117 2024-09-17T21:20:44.964 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.965 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.965 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.965 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15710 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 994304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:44.965 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:20:44.965 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.414740797, 2024-09-17T21:20:44.965 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092957729000000003, 2024-09-17T21:20:44.965 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.965 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.965 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.965 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.965 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:44.965 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:44.966 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15710 2024-09-17T21:20:44.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.966 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.966 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:20:44.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.966 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:20:44.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.967 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.967 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:20:44.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8000000000000005e-07 2024-09-17T21:20:44.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.967 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.967 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521216+0000", 2024-09-17T21:20:44.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3859999999999998e-06 2024-09-17T21:20:44.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.967 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.967 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521234+0000", 2024-09-17T21:20:44.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:20:44.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521248+0000", 2024-09-17T21:20:44.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:20:44.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613926+0000", 2024-09-17T21:20:44.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092678324000000006 2024-09-17T21:20:44.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613974+0000", 2024-09-17T21:20:44.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7754999999999999e-05 2024-09-17T21:20:44.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.969 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.969 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614167+0000", 2024-09-17T21:20:44.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193508 2024-09-17T21:20:44.969 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.969 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.969 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.969 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15711 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 998400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:44.970 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:20:44.970 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.414683414999999, 2024-09-17T21:20:44.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093268000000000004, 2024-09-17T21:20:44.970 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.970 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.970 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.970 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.970 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:44.970 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:44.970 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15711 2024-09-17T21:20:44.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.971 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:20:44.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:20:44.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521268+0000", 2024-09-17T21:20:44.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-17T21:20:44.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521273+0000", 2024-09-17T21:20:44.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:20:44.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521290+0000", 2024-09-17T21:20:44.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7269999999999999e-05 2024-09-17T21:20:44.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521304+0000", 2024-09-17T21:20:44.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3626e-05 2024-09-17T21:20:44.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614220+0000", 2024-09-17T21:20:44.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092916367 2024-09-17T21:20:44.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614322+0000", 2024-09-17T21:20:44.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102212 2024-09-17T21:20:44.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614535+0000", 2024-09-17T21:20:44.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021266199999999999 2024-09-17T21:20:44.974 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.974 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.974 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.974 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15712 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1002496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:44.974 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:20:44.974 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.414627920000001, 2024-09-17T21:20:44.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093568707000000001, 2024-09-17T21:20:44.975 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.975 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.975 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.975 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.975 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:44.975 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:44.975 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15712 2024-09-17T21:20:44.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.975 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.975 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.975 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:20:44.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.976 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:20:44.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.976 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521324+0000", 2024-09-17T21:20:44.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.17e-06 2024-09-17T21:20:44.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521334+0000", 2024-09-17T21:20:44.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9369999999999999e-06 2024-09-17T21:20:44.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521356+0000", 2024-09-17T21:20:44.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2045999999999999e-05 2024-09-17T21:20:44.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521370+0000", 2024-09-17T21:20:44.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4270000000000001e-05 2024-09-17T21:20:44.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614583+0000", 2024-09-17T21:20:44.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093213220999999999 2024-09-17T21:20:44.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614680+0000", 2024-09-17T21:20:44.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6900999999999999e-05 2024-09-17T21:20:44.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614891+0000", 2024-09-17T21:20:44.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021116200000000001 2024-09-17T21:20:44.979 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.979 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.979 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.979 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15713 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1006592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:44.979 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:20:44.979 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.414561217999999, 2024-09-17T21:20:44.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093834641999999996, 2024-09-17T21:20:44.980 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.980 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.980 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.980 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.980 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:44.980 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:44.980 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15713 2024-09-17T21:20:44.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.980 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:20:44.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:20:44.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521390+0000", 2024-09-17T21:20:44.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:20:44.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521396+0000", 2024-09-17T21:20:44.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:20:44.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521417+0000", 2024-09-17T21:20:44.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1236000000000001e-05 2024-09-17T21:20:44.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521431+0000", 2024-09-17T21:20:44.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3873000000000001e-05 2024-09-17T21:20:44.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614938+0000", 2024-09-17T21:20:44.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093506522999999994 2024-09-17T21:20:44.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615025+0000", 2024-09-17T21:20:44.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7635999999999999e-05 2024-09-17T21:20:44.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615224+0000", 2024-09-17T21:20:44.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019844700000000001 2024-09-17T21:20:44.984 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.984 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.984 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.984 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15714 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:44.984 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:20:44.984 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.414497961999999, 2024-09-17T21:20:44.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094071925000000001, 2024-09-17T21:20:44.984 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.984 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.985 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.985 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.985 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:44.985 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:44.985 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15714 2024-09-17T21:20:44.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.985 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:20:44.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.986 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:20:44.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.986 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521454+0000", 2024-09-17T21:20:44.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.097e-06 2024-09-17T21:20:44.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.986 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521459+0000", 2024-09-17T21:20:44.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2660000000000001e-06 2024-09-17T21:20:44.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521481+0000", 2024-09-17T21:20:44.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1878999999999998e-05 2024-09-17T21:20:44.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521498+0000", 2024-09-17T21:20:44.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6807e-05 2024-09-17T21:20:44.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615271+0000", 2024-09-17T21:20:44.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093773286999999997 2024-09-17T21:20:44.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615366+0000", 2024-09-17T21:20:44.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5322000000000007e-05 2024-09-17T21:20:44.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615524+0000", 2024-09-17T21:20:44.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015826699999999999 2024-09-17T21:20:44.988 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.988 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.988 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.989 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15715 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1014784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:44.989 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:20:44.989 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.414433627000001, 2024-09-17T21:20:44.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094243840999999995, 2024-09-17T21:20:44.989 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.989 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.989 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.989 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.989 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:44.989 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:44.989 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15715 2024-09-17T21:20:44.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.990 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:20:44.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:20:44.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521518+0000", 2024-09-17T21:20:44.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4300000000000001e-06 2024-09-17T21:20:44.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521524+0000", 2024-09-17T21:20:44.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5860000000000004e-06 2024-09-17T21:20:44.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521541+0000", 2024-09-17T21:20:44.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7574000000000001e-05 2024-09-17T21:20:44.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.992 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.992 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521555+0000", 2024-09-17T21:20:44.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3540000000000001e-05 2024-09-17T21:20:44.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.992 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.992 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615560+0000", 2024-09-17T21:20:44.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.09400472 2024-09-17T21:20:44.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.992 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615614+0000", 2024-09-17T21:20:44.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4693e-05 2024-09-17T21:20:44.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615761+0000", 2024-09-17T21:20:44.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000146298 2024-09-17T21:20:44.993 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.993 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.993 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.993 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15716 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1018880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:44.994 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:20:44.994 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.414377282, 2024-09-17T21:20:44.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094400676000000003, 2024-09-17T21:20:44.994 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.994 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.994 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.994 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.994 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:44.994 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:44.994 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15716 2024-09-17T21:20:44.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.994 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:20:44.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:44.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:20:44.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:44.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521574+0000", 2024-09-17T21:20:44.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1939999999999999e-06 2024-09-17T21:20:44.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:44.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521579+0000", 2024-09-17T21:20:44.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:20:44.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:44.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521597+0000", 2024-09-17T21:20:44.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8074e-05 2024-09-17T21:20:44.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:44.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521613+0000", 2024-09-17T21:20:44.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5316e-05 2024-09-17T21:20:44.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:44.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615780+0000", 2024-09-17T21:20:44.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094166874999999997 2024-09-17T21:20:44.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:44.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615836+0000", 2024-09-17T21:20:44.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6926999999999999e-05 2024-09-17T21:20:44.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:44.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615974+0000", 2024-09-17T21:20:44.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013741400000000001 2024-09-17T21:20:44.998 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.998 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:44.998 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:44.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.998 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15717 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1022976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:44.998 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:20:44.998 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.414318586, 2024-09-17T21:20:44.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094586723999999997, 2024-09-17T21:20:44.998 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:44.998 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:44.999 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:44.999 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:44.999 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:44.999 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:44.999 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15717 2024-09-17T21:20:44.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.999 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:44.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:44.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:20:44.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:44.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:44.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:44.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:20:45.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521633+0000", 2024-09-17T21:20:45.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:20:45.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521638+0000", 2024-09-17T21:20:45.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:20:45.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521659+0000", 2024-09-17T21:20:45.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-05 2024-09-17T21:20:45.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521675+0000", 2024-09-17T21:20:45.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5688999999999998e-05 2024-09-17T21:20:45.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616008+0000", 2024-09-17T21:20:45.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094333314000000001 2024-09-17T21:20:45.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616066+0000", 2024-09-17T21:20:45.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7797000000000002e-05 2024-09-17T21:20:45.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616219+0000", 2024-09-17T21:20:45.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015281100000000001 2024-09-17T21:20:45.002 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.002 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.002 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.003 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15718 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:45.003 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:20:45.003 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.414218511000001, 2024-09-17T21:20:45.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094703674000000002, 2024-09-17T21:20:45.003 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.003 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.003 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:45.003 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:45.003 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:45.003 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:45.003 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15718 2024-09-17T21:20:45.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.004 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.004 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:20:45.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.004 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:20:45.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.004 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521734+0000", 2024-09-17T21:20:45.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.61e-06 2024-09-17T21:20:45.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521740+0000", 2024-09-17T21:20:45.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:20:45.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521754+0000", 2024-09-17T21:20:45.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3793e-05 2024-09-17T21:20:45.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521763+0000", 2024-09-17T21:20:45.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5129999999999996e-06 2024-09-17T21:20:45.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616256+0000", 2024-09-17T21:20:45.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094492919999999994 2024-09-17T21:20:45.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616312+0000", 2024-09-17T21:20:45.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6737000000000002e-05 2024-09-17T21:20:45.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616436+0000", 2024-09-17T21:20:45.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012335800000000001 2024-09-17T21:20:45.007 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.007 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.007 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.007 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15719 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1031168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:45.007 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:20:45.007 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.414165105999999, 2024-09-17T21:20:45.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094888963000000007, 2024-09-17T21:20:45.008 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.008 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.008 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:45.008 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:45.008 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:45.008 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:45.008 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15719 2024-09-17T21:20:45.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.008 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:20:45.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:20:45.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521786+0000", 2024-09-17T21:20:45.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.074e-06 2024-09-17T21:20:45.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521791+0000", 2024-09-17T21:20:45.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3030000000000001e-06 2024-09-17T21:20:45.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521804+0000", 2024-09-17T21:20:45.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3027000000000001e-05 2024-09-17T21:20:45.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521811+0000", 2024-09-17T21:20:45.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6090000000000003e-06 2024-09-17T21:20:45.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616466+0000", 2024-09-17T21:20:45.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094654294 2024-09-17T21:20:45.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616523+0000", 2024-09-17T21:20:45.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6953999999999997e-05 2024-09-17T21:20:45.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616674+0000", 2024-09-17T21:20:45.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015170200000000001 2024-09-17T21:20:45.013 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.013 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.013 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.013 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15720 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1035264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:45.013 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:20:45.013 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.414128265999999, 2024-09-17T21:20:45.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095067061999999994, 2024-09-17T21:20:45.014 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.014 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.014 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:45.014 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:45.014 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:45.014 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:45.014 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15720 2024-09-17T21:20:45.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.014 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:20:45.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:20:45.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521823+0000", 2024-09-17T21:20:45.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:20:45.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521826+0000", 2024-09-17T21:20:45.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2100000000000002e-06 2024-09-17T21:20:45.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521835+0000", 2024-09-17T21:20:45.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1630000000000002e-06 2024-09-17T21:20:45.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521842+0000", 2024-09-17T21:20:45.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:20:45.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616711+0000", 2024-09-17T21:20:45.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094868859 2024-09-17T21:20:45.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616764+0000", 2024-09-17T21:20:45.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2778999999999997e-05 2024-09-17T21:20:45.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616889+0000", 2024-09-17T21:20:45.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012533100000000001 2024-09-17T21:20:45.018 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.018 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.018 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.018 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15721 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1039360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:45.018 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:20:45.018 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.414081156999998, 2024-09-17T21:20:45.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095277219999999996, 2024-09-17T21:20:45.018 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.018 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.019 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:45.019 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:45.019 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:45.019 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:45.019 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15721 2024-09-17T21:20:45.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.019 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:20:45.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:20:45.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521871+0000", 2024-09-17T21:20:45.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.443e-06 2024-09-17T21:20:45.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521876+0000", 2024-09-17T21:20:45.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0300000000000001e-06 2024-09-17T21:20:45.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521886+0000", 2024-09-17T21:20:45.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186e-05 2024-09-17T21:20:45.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521893+0000", 2024-09-17T21:20:45.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.08e-06 2024-09-17T21:20:45.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616918+0000", 2024-09-17T21:20:45.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095025223000000006 2024-09-17T21:20:45.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616977+0000", 2024-09-17T21:20:45.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8403999999999999e-05 2024-09-17T21:20:45.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617147+0000", 2024-09-17T21:20:45.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016985399999999999 2024-09-17T21:20:45.022 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.022 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.022 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.023 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15722 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:45.023 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:20:45.023 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.414046708000001, 2024-09-17T21:20:45.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095459783000000006, 2024-09-17T21:20:45.023 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.023 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.023 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:45.023 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:45.023 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:45.023 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:45.023 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15722 2024-09-17T21:20:45.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.023 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:20:45.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:20:45.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:20:45.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7700000000000004e-07 2024-09-17T21:20:45.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521907+0000", 2024-09-17T21:20:45.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.813e-06 2024-09-17T21:20:45.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521922+0000", 2024-09-17T21:20:45.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4533e-05 2024-09-17T21:20:45.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521934+0000", 2024-09-17T21:20:45.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2469999999999999e-05 2024-09-17T21:20:45.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617187+0000", 2024-09-17T21:20:45.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095252311000000006 2024-09-17T21:20:45.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617240+0000", 2024-09-17T21:20:45.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3740999999999999e-05 2024-09-17T21:20:45.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617364+0000", 2024-09-17T21:20:45.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-17T21:20:45.027 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.027 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.027 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.027 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15723 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:45.027 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:20:45.027 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.414002841999999, 2024-09-17T21:20:45.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095653563999999996, 2024-09-17T21:20:45.027 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.027 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.027 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:45.028 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:45.028 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:45.028 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:45.028 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15723 2024-09-17T21:20:45.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.028 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:20:45.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:20:45.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521949+0000", 2024-09-17T21:20:45.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:20:45.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521953+0000", 2024-09-17T21:20:45.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5870000000000003e-06 2024-09-17T21:20:45.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521970+0000", 2024-09-17T21:20:45.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7282000000000002e-05 2024-09-17T21:20:45.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521984+0000", 2024-09-17T21:20:45.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:20:45.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617383+0000", 2024-09-17T21:20:45.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095399063000000006 2024-09-17T21:20:45.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617437+0000", 2024-09-17T21:20:45.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3931999999999998e-05 2024-09-17T21:20:45.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617601+0000", 2024-09-17T21:20:45.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016407300000000001 2024-09-17T21:20:45.031 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.031 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.031 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.032 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5629.0:2384 3.1c 3:38c1263e:::rbd_data.1152d37183c3.000000000000000b:head [read 11780096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-17T21:20:45.032 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:20:45.032 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.5111720119999994, 2024-09-17T21:20:45.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0246185320000001, 2024-09-17T21:20:45.032 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.032 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.032 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:45.032 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:45.032 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5629", 2024-09-17T21:20:45.032 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3798711329", 2024-09-17T21:20:45.032 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2384 2024-09-17T21:20:45.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.032 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:20:45.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:20:45.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424780+0000", 2024-09-17T21:20:45.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.13e-06 2024-09-17T21:20:45.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424785+0000", 2024-09-17T21:20:45.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4739999999999996e-06 2024-09-17T21:20:45.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424798+0000", 2024-09-17T21:20:45.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2704000000000001e-05 2024-09-17T21:20:45.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424806+0000", 2024-09-17T21:20:45.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9899999999999997e-06 2024-09-17T21:20:45.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424868+0000", 2024-09-17T21:20:45.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999997e-05 2024-09-17T21:20:45.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:20:45.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424875+0000", 2024-09-17T21:20:45.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0870000000000001e-06 2024-09-17T21:20:45.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428897+0000", 2024-09-17T21:20:45.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040226080000000004 2024-09-17T21:20:45.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:45.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428913+0000", 2024-09-17T21:20:45.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:20:45.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.448963+0000", 2024-09-17T21:20:45.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0200498659999999 2024-09-17T21:20:45.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449158+0000", 2024-09-17T21:20:45.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019581000000000001 2024-09-17T21:20:45.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449397+0000", 2024-09-17T21:20:45.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023871 2024-09-17T21:20:45.037 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.037 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.037 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.037 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.037 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.037 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:45.039 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:20:45.039 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:45.032+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:45.140 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:45.134+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:45.218 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:45.218 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:45.218 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:20:45.218 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:45.234 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:20:45.241 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:45.236+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:45.274 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-17T21:20:45.342 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:45.334+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:45.403 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:45.403 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:45.403 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:45.403 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:45.403 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:45.420 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:20:45.442 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:45.436+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:45.497 INFO:tasks.thrashosds.thrasher:Setting full ratio back to .95 2024-09-17T21:20:45.498 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .95 2024-09-17T21:20:45.529 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-17T21:20:45.543 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:45.538+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:45.587 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:45.587 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:45.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:45.587 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:45.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.588 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:20:45.588 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:45.588 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 143.97918547200001, 2024-09-17T21:20:45.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:20:45.588 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.588 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.588 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:45.588 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:45.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:45.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:20:45.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:20:45.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:20:45.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:20:45.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:20:45.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:20:45.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:20:45.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:20:45.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:20:45.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:20:45.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:20:45.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:20:45.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:45.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:20:45.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:20:45.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:20:45.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:20:45.592 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.592 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.592 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.592 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:20:45.592 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:45.592 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 143.96923883599999, 2024-09-17T21:20:45.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:20:45.592 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.592 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.593 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:45.593 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:45.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:45.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:20:45.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:20:45.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:20:45.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:20:45.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:20:45.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:20:45.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:20:45.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:20:45.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:20:45.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:20:45.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:20:45.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:20:45.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:45.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:20:45.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:20:45.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:20:45.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:20:45.597 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.597 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.597 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.598 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:20:45.598 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:45.598 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 142.72608799599999, 2024-09-17T21:20:45.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:20:45.598 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.598 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.598 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:45.598 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:45.598 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:20:45.598 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:20:45.598 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:20:45.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.598 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:45.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:45.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:20:45.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:20:45.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:20:45.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:20:45.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:20:45.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:20:45.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:20:45.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:20:45.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:20:45.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:20:45.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:20:45.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:20:45.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:45.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:20:45.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:20:45.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:45.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:20:45.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:20:45.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:45.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:20:45.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:20:45.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:45.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:20:45.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:20:45.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:20:45.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:20:45.603 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.603 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.604 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.604 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:20:45.604 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:45.604 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 142.71644251000001, 2024-09-17T21:20:45.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:20:45.604 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.604 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.604 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:45.604 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:45.604 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:20:45.605 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:20:45.605 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:20:45.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.605 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:45.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:45.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:20:45.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:20:45.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:20:45.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:20:45.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:20:45.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:20:45.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:20:45.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:20:45.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:20:45.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:20:45.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:20:45.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:20:45.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:45.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:20:45.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:20:45.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:45.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:20:45.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:20:45.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:45.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:20:45.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:20:45.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:45.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:20:45.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:20:45.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:20:45.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:20:45.609 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.609 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.610 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.610 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:20:45.610 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:45.610 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 129.347506477, 2024-09-17T21:20:45.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:20:45.610 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.610 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.610 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:45.610 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:45.610 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:20:45.610 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:20:45.610 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:20:45.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.611 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:45.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:45.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:20:45.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:20:45.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:20:45.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:20:45.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:20:45.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:20:45.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:20:45.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:20:45.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:20:45.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:20:45.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:20:45.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:20:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:20:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:20:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:819 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5111808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.14593418699999, 2024-09-17T21:20:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024344811000000001, 2024-09-17T21:20:45.615 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.615 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.615 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:45.615 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:45.615 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:45.615 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:45.615 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 819 2024-09-17T21:20:45.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.615 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:45.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:45.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437443+0000", 2024-09-17T21:20:45.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1969999999999999e-06 2024-09-17T21:20:45.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437448+0000", 2024-09-17T21:20:45.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9490000000000004e-06 2024-09-17T21:20:45.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437467+0000", 2024-09-17T21:20:45.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8862999999999999e-05 2024-09-17T21:20:45.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437480+0000", 2024-09-17T21:20:45.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2833000000000001e-05 2024-09-17T21:20:45.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461552+0000", 2024-09-17T21:20:45.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024072695000000002 2024-09-17T21:20:45.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461639+0000", 2024-09-17T21:20:45.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6996000000000005e-05 2024-09-17T21:20:45.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461787+0000", 2024-09-17T21:20:45.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014727800000000001 2024-09-17T21:20:45.618 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.618 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.618 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.619 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:45.619 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:45.619 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.145821915, 2024-09-17T21:20:45.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:20:45.619 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.619 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.619 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:45.619 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:45.619 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:45.619 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:45.619 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:20:45.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.619 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:45.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:45.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:20:45.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:20:45.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:20:45.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:20:45.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:20:45.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:20:45.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:20:45.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:20:45.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:20:45.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:20:45.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:20:45.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:20:45.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:20:45.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:20:45.623 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.623 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.623 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.623 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:833 3.5 3:a772dfb1:::rbd_data.11523651b015.0000000000000001:d [sparse-read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:45.623 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:45.623 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.145387629, 2024-09-17T21:20:45.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024216024999999999, 2024-09-17T21:20:45.623 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.623 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.624 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:45.624 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:45.624 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:45.624 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:45.624 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 833 2024-09-17T21:20:45.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.624 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:45.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:45.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437990+0000", 2024-09-17T21:20:45.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2500000000000001e-06 2024-09-17T21:20:45.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437995+0000", 2024-09-17T21:20:45.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1240000000000004e-06 2024-09-17T21:20:45.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438012+0000", 2024-09-17T21:20:45.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7572e-05 2024-09-17T21:20:45.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438031+0000", 2024-09-17T21:20:45.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8612000000000001e-05 2024-09-17T21:20:45.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461830+0000", 2024-09-17T21:20:45.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023799135999999999 2024-09-17T21:20:45.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462005+0000", 2024-09-17T21:20:45.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000174747 2024-09-17T21:20:45.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:20:45.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019958399999999999 2024-09-17T21:20:45.627 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.627 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.627 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.628 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:45.628 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:45.628 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.145327338, 2024-09-17T21:20:45.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:20:45.628 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.628 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.628 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:45.628 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:45.628 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:45.628 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:45.628 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:20:45.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.628 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:45.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:45.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:20:45.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:20:45.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:20:45.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:20:45.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:20:45.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:20:45.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:20:45.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:20:45.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:20:45.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:20:45.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:20:45.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:20:45.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:20:45.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:20:45.632 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.632 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.632 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.632 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:45.632 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:45.632 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.14527195700001, 2024-09-17T21:20:45.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:20:45.632 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.632 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.632 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:45.632 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:45.633 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:45.633 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:45.633 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:20:45.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.633 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:45.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:45.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:20:45.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:20:45.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:20:45.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:20:45.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:20:45.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:20:45.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:20:45.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:20:45.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:20:45.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:20:45.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:20:45.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:20:45.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:20:45.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:20:45.636 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.636 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.636 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.636 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:45.636 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:45.637 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.145215679, 2024-09-17T21:20:45.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:20:45.637 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.637 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.637 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:45.637 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:45.637 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:45.637 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:45.637 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:20:45.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.637 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:45.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:45.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:20:45.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:20:45.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:20:45.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:20:45.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:20:45.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:20:45.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:20:45.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:20:45.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:20:45.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:20:45.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:20:45.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:20:45.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:20:45.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:20:45.640 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.640 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.641 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.641 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:45.641 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:45.641 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.14426604000001, 2024-09-17T21:20:45.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:20:45.641 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.641 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.641 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:45.641 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:45.641 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:45.641 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:45.641 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:20:45.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.642 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:45.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:45.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:20:45.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:20:45.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:20:45.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:20:45.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:20:45.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:20:45.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:20:45.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:20:45.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:20:45.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:20:45.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:20:45.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:20:45.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:20:45.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:20:45.645 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.645 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.645 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.645 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:45.645 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:45.645 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.14418741999999, 2024-09-17T21:20:45.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:20:45.645 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.645 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.645 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:45.645 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:45.645 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:45.645 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:45.646 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:20:45.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.646 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:45.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:45.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:20:45.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:20:45.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:20:45.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:20:45.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:20:45.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:20:45.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:20:45.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:20:45.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:20:45.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:20:45.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:20:45.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:20:45.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:20:45.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:20:45.649 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.649 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.649 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.649 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:45.649 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:45.649 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.144074481, 2024-09-17T21:20:45.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:20:45.650 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.650 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.650 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:45.650 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:45.650 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:45.650 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:45.650 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:20:45.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.650 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:45.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:45.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:20:45.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:20:45.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:20:45.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:20:45.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:20:45.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:20:45.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:20:45.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:20:45.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:20:45.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:20:45.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:20:45.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:20:45.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:20:45.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:20:45.654 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.654 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.654 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.654 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:849 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5963776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:45.654 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:45.654 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.14348044499999, 2024-09-17T21:20:45.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024537514, 2024-09-17T21:20:45.654 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.654 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.654 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:45.654 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:45.654 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:45.655 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:45.655 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 849 2024-09-17T21:20:45.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.655 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:45.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:45.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439897+0000", 2024-09-17T21:20:45.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9099999999999999e-06 2024-09-17T21:20:45.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439905+0000", 2024-09-17T21:20:45.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0870000000000007e-06 2024-09-17T21:20:45.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439924+0000", 2024-09-17T21:20:45.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9042000000000002e-05 2024-09-17T21:20:45.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439941+0000", 2024-09-17T21:20:45.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.624e-05 2024-09-17T21:20:45.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464132+0000", 2024-09-17T21:20:45.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024191579000000001 2024-09-17T21:20:45.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464291+0000", 2024-09-17T21:20:45.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015904800000000001 2024-09-17T21:20:45.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464433+0000", 2024-09-17T21:20:45.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014160799999999999 2024-09-17T21:20:45.658 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.659 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.659 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.659 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:45.659 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:45.659 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.143413427, 2024-09-17T21:20:45.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:20:45.659 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.659 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.659 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:45.659 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:45.659 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:45.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:45.660 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:20:45.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.660 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:45.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:45.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:20:45.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:20:45.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:20:45.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:20:45.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:20:45.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:20:45.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:20:45.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:20:45.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:20:45.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:20:45.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:20:45.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:20:45.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:20:45.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:20:45.663 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.663 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.663 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.663 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:45.663 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:45.663 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.143348616, 2024-09-17T21:20:45.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:20:45.664 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.664 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.664 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:45.664 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:45.664 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:45.664 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:45.664 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:20:45.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.664 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:45.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:45.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:20:45.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:20:45.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:20:45.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:20:45.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:20:45.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:20:45.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:20:45.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:20:45.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:20:45.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:20:45.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:20:45.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:20:45.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:20:45.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:20:45.667 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.667 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.668 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.668 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:45.668 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:45.668 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.14328763899999, 2024-09-17T21:20:45.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:20:45.668 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.668 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.668 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:45.668 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:45.668 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:45.668 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:45.669 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:20:45.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.669 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:45.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:45.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:20:45.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:20:45.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:20:45.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:20:45.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:20:45.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:20:45.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:20:45.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:20:45.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:20:45.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:20:45.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:20:45.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:20:45.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:20:45.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:20:45.672 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.672 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.672 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.672 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:20:45.672 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:45.672 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 56.245716453999997, 2024-09-17T21:20:45.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:20:45.673 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.673 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.673 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:45.673 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:45.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:45.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:20:45.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:20:45.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:20:45.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:20:45.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:20:45.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:20:45.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:20:45.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:20:45.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:20:45.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:20:45.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:20:45.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:20:45.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:45.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:20:45.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:20:45.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:20:45.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:20:45.676 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.676 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.677 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.677 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:20:45.677 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:45.677 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 56.03705806, 2024-09-17T21:20:45.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:20:45.677 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:45.677 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:45.677 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:45.677 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:45.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:45.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:45.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:45.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:45.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:45.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:45.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:20:45.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:20:45.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:20:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:20:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:20:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:20:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:20:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:20:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:20:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:20:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:20:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:20:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:20:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:20:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:45.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:20:45.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:20:45.681 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.681 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.681 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.681 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:45.681 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:45.681 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:45.682 INFO:teuthology.orchestra.run.smithi184.stdout:5305 read 0x5265600 thru 0x526e872 (0x9273 bytes) 2024-09-17T21:20:45.683 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:20:45.683 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:45.676+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:45.714 INFO:teuthology.orchestra.run.smithi184.stdout:5308 trunc from 0xad0ba00 to 0x536c00 2024-09-17T21:20:45.775 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:20:45.814 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:45.814 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:45.814 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:45.814 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:45.814 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:45.808+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:45.830 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:20:45.855 INFO:teuthology.orchestra.run.smithi184.stdout:5309 punch from 0x51e200 to 0x52cc00, (0xea00 bytes) 2024-09-17T21:20:45.857 INFO:teuthology.orchestra.run.smithi184.stdout:5310 read 0x4eb200 thru 0x4f5293 (0xa094 bytes) 2024-09-17T21:20:45.866 INFO:teuthology.orchestra.run.smithi184.stdout:5311 read 0x4c0400 thru 0x4c9f37 (0x9b38 bytes) 2024-09-17T21:20:45.869 INFO:teuthology.orchestra.run.smithi184.stdout:5312 trunc from 0x536c00 to 0x10bfa00 2024-09-17T21:20:45.924 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:45.920+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:45.929 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:45.929 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:45.935 INFO:teuthology.orchestra.run.smithi184.stdout:5314 write 0xd92c000 thru 0xd9314e1 (0x54e2 bytes) 2024-09-17T21:20:45.936 INFO:teuthology.orchestra.run.smithi184.stdout:5317 read 0x215c600 thru 0x2160fc2 (0x49c3 bytes) 2024-09-17T21:20:45.936 INFO:teuthology.orchestra.run.smithi184.stdout:5321 write 0x77ae400 thru 0x77b4193 (0x5d94 bytes) 2024-09-17T21:20:45.936 INFO:teuthology.orchestra.run.smithi184.stdout:5324 read 0x956600 thru 0x9647c0 (0xe1c1 bytes) 2024-09-17T21:20:45.939 INFO:teuthology.orchestra.run.smithi184.stdout:5325 write 0x67cfe00 thru 0x67daaa0 (0xaca1 bytes) 2024-09-17T21:20:45.939 INFO:teuthology.orchestra.run.smithi184.stdout:5328 punch from 0x2e3d400 to 0x2e41200, (0x3e00 bytes) 2024-09-17T21:20:45.940 INFO:teuthology.orchestra.run.smithi184.stdout:5329 punch from 0x6a76a00 to 0x6a7a200, (0x3800 bytes) 2024-09-17T21:20:45.940 INFO:teuthology.orchestra.run.smithi184.stdout:5330 read 0xa751000 thru 0xa75da51 (0xca52 bytes) 2024-09-17T21:20:45.941 INFO:teuthology.orchestra.run.smithi184.stdout:5333 write 0x589e800 thru 0x58a3b12 (0x5313 bytes) 2024-09-17T21:20:45.941 INFO:teuthology.orchestra.run.smithi184.stdout:5336 write 0x7e2d600 thru 0x7e34e0a (0x780b bytes) 2024-09-17T21:20:45.941 INFO:teuthology.orchestra.run.smithi184.stdout:5337 write 0x87a3a00 thru 0x87ae0c6 (0xa6c7 bytes) 2024-09-17T21:20:45.941 INFO:teuthology.orchestra.run.smithi184.stdout:5338 read 0xc7e0400 thru 0xc7f0359 (0xff5a bytes) 2024-09-17T21:20:45.942 INFO:teuthology.orchestra.run.smithi184.stdout:5339 write 0xc5f1200 thru 0xc5fb598 (0xa399 bytes) 2024-09-17T21:20:45.942 INFO:teuthology.orchestra.run.smithi184.stdout:5343 write 0x88d1800 thru 0x88d558c (0x3d8d bytes) 2024-09-17T21:20:45.942 INFO:teuthology.orchestra.run.smithi184.stdout:5347 punch from 0xc10ba00 to 0xc10fa00, (0x4000 bytes) 2024-09-17T21:20:45.943 INFO:teuthology.orchestra.run.smithi184.stdout:5348 write 0x1da0a00 thru 0x1da3b28 (0x3129 bytes) 2024-09-17T21:20:45.943 INFO:teuthology.orchestra.run.smithi184.stdout:5349 punch from 0xcea0e00 to 0xceabc00, (0xae00 bytes) 2024-09-17T21:20:45.944 INFO:teuthology.orchestra.run.smithi184.stdout:5351 read 0x5191200 thru 0x519ea62 (0xd863 bytes) 2024-09-17T21:20:45.944 INFO:teuthology.orchestra.run.smithi184.stdout:5352 write 0x7723000 thru 0x772baeb (0x8aec bytes) 2024-09-17T21:20:45.947 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:45.947 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:45.953 INFO:teuthology.orchestra.run.smithi184.stdout:5353 trunc from 0xd931600 to 0x8333e00 2024-09-17T21:20:45.969 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:45.969 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:45.985 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:45.978+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:45.993 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:45.993 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:46.011 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:46.011 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:46.014 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:46.014 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:46.026 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:20:46.033 INFO:teuthology.orchestra.run.smithi184.stdout:5354 read 0x62bd400 thru 0x62bde2b (0xa2c bytes) 2024-09-17T21:20:46.034 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:46.034 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:46.035 INFO:teuthology.orchestra.run.smithi184.stdout:5355 clone 38 order 23 su 262144 sc 13 2024-09-17T21:20:46.097 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:46.092+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:46.148 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:46.148 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:46.162 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:20:46.205 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:46.200+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:46.287 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:46.280+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:46.323 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:46.323 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:46.336 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:20:46.429 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:46.424+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:46.503 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:46.503 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:46.507 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:46.482+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:46.516 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:20:46.532 INFO:teuthology.orchestra.run.smithi083.stderr:osd set-full-ratio 0.95 2024-09-17T21:20:46.589 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:46.582+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:46.653 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:46.653 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:46.666 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:20:46.689 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:46.682+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:46.733 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone37 from 0x8333e00 (overlap 0x536c00) to 0x10ae00 2024-09-17T21:20:46.790 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:46.782+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:46.791 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:46.791 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:46.806 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:20:46.891 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:46.886+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:46.926 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:46.926 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:46.942 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:20:46.992 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:46.986+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:47.068 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:47.068 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:47.083 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:20:47.092 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:47.088+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:47.194 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:47.186+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:47.218 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:47.219 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:47.233 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:20:47.295 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:47.288+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:47.396 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:47.390+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:47.400 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-17T21:20:47.401 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:47.401 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:47.415 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:20:47.468 INFO:teuthology.orchestra.run.smithi184.stdout:5356 read 0x1259a00 thru 0x1266b2d (0xd12e bytes) 2024-09-17T21:20:47.469 INFO:teuthology.orchestra.run.smithi184.stdout:5361 read 0x6548e00 thru 0x654d5c5 (0x47c6 bytes) 2024-09-17T21:20:47.470 INFO:teuthology.orchestra.run.smithi184.stdout:5363 punch from 0x63d9600 to 0x63e6400, (0xce00 bytes) 2024-09-17T21:20:47.494 INFO:teuthology.orchestra.run.smithi184.stdout:5365 write 0x317200 thru 0x318d25 (0x1b26 bytes) 2024-09-17T21:20:47.495 INFO:teuthology.orchestra.run.smithi184.stdout:5366 punch from 0x263c400 to 0x2645600, (0x9200 bytes) 2024-09-17T21:20:47.503 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:47.498+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:47.531 INFO:teuthology.orchestra.run.smithi184.stdout:5368 read 0x214cc00 thru 0x2153c9f (0x70a0 bytes) 2024-09-17T21:20:47.531 INFO:teuthology.orchestra.run.smithi184.stdout:5369 read 0x6adee00 thru 0x6ae9387 (0xa588 bytes) 2024-09-17T21:20:47.532 INFO:teuthology.orchestra.run.smithi184.stdout:5370 read 0x3981200 thru 0x3983c67 (0x2a68 bytes) 2024-09-17T21:20:47.555 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:20:47.585 INFO:teuthology.orchestra.run.smithi184.stdout:5371 trunc from 0x8333e00 to 0xbf78c00 2024-09-17T21:20:47.590 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:47.590 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:47.597 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:47.590+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:47.603 INFO:teuthology.orchestra.run.smithi184.stdout:5372 punch from 0x39dd800 to 0x39eca00, (0xf200 bytes) 2024-09-17T21:20:47.605 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:20:47.606 INFO:teuthology.orchestra.run.smithi184.stdout:5373 write 0x779de00 thru 0x77a4bf2 (0x6df3 bytes) 2024-09-17T21:20:47.606 INFO:teuthology.orchestra.run.smithi184.stdout:5376 write 0x7b20200 thru 0x7b2636a (0x616b bytes) 2024-09-17T21:20:47.606 INFO:teuthology.orchestra.run.smithi184.stdout:5379 read 0x5ca1800 thru 0x5ca4f8f (0x3790 bytes) 2024-09-17T21:20:47.607 INFO:teuthology.orchestra.run.smithi184.stdout:5380 write 0x2fb4400 thru 0x2fba242 (0x5e43 bytes) 2024-09-17T21:20:47.608 INFO:teuthology.orchestra.run.smithi184.stdout:5381 write 0xa2b9c00 thru 0xa2beffc (0x53fd bytes) 2024-09-17T21:20:47.608 INFO:teuthology.orchestra.run.smithi184.stdout:5382 read 0xd9600 thru 0xe6e4f (0xd850 bytes) 2024-09-17T21:20:47.609 INFO:teuthology.orchestra.run.smithi184.stdout:5383 read 0x1b83e00 thru 0x1b911d5 (0xd3d6 bytes) 2024-09-17T21:20:47.609 INFO:teuthology.orchestra.run.smithi184.stdout:5384 read 0x7b0f000 thru 0x7b14487 (0x5488 bytes) 2024-09-17T21:20:47.627 INFO:teuthology.orchestra.run.smithi184.stdout:5385 trunc from 0xbf78c00 to 0xe185200 2024-09-17T21:20:47.679 INFO:teuthology.orchestra.run.smithi184.stdout:5387 punch from 0x1ca9600 to 0x1cb3000, (0x9a00 bytes) 2024-09-17T21:20:47.698 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:47.690+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:47.744 INFO:teuthology.orchestra.run.smithi184.stdout:5389 trunc from 0xe185200 to 0x3f3b200 2024-09-17T21:20:47.786 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:47.786 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:47.800 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:47.794+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:47.809 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:20:47.828 INFO:teuthology.orchestra.run.smithi184.stdout:5392 read 0x17b9600 thru 0x17c1356 (0x7d57 bytes) 2024-09-17T21:20:47.831 INFO:teuthology.orchestra.run.smithi184.stdout:5393 read 0x2400600 thru 0x2407cf3 (0x76f4 bytes) 2024-09-17T21:20:47.831 INFO:teuthology.orchestra.run.smithi184.stdout:5394 read 0x3171c00 thru 0x3179c68 (0x8069 bytes) 2024-09-17T21:20:47.885 INFO:teuthology.orchestra.run.smithi184.stdout:5397 write 0xe85ea00 thru 0xe86a74b (0xbd4c bytes) 2024-09-17T21:20:47.887 INFO:teuthology.orchestra.run.smithi184.stdout:5401 clone 39 order 18 su 1024 sc 15 2024-09-17T21:20:47.901 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:47.896+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:47.978 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:47.978 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:47.992 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:20:48.002 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:48.000+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:48.052 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:20:48.103 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:48.094+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:48.114 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:48.114 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:48.128 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:20:48.202 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:48.202 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:48.203 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:48.196+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:48.220 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:48.236 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:48.237 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:48.256 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:48.256 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:48.276 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:48.276 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:48.296 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:48.297 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:48.297 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:48.297 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:48.297 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:20:48.297 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:48.304 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:48.300+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:48.311 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:20:48.405 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:48.400+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:48.433 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:48.433 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:48.433 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:48.433 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:48.433 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:48.447 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:20:48.473 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-17T21:20:48.549 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:48.544+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:48.578 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:48.578 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:48.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:48.578 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:48.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.578 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:20:48.578 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:48.579 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 146.96958881800001, 2024-09-17T21:20:48.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:20:48.579 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:48.579 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:48.579 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:48.579 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:48.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:48.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:48.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:48.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:48.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:48.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:20:48.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:20:48.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:48.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:20:48.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:20:48.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:48.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:20:48.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:20:48.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:48.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:20:48.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:20:48.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:48.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:20:48.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:20:48.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:48.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:20:48.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:20:48.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:48.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:20:48.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:20:48.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:48.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:20:48.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:20:48.583 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.583 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:48.583 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.583 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:20:48.583 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:48.583 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 146.95964218200001, 2024-09-17T21:20:48.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:20:48.583 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:48.583 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:48.583 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:48.583 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:48.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:48.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:48.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:48.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:48.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:48.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:20:48.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:20:48.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:48.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:20:48.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:20:48.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:48.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:20:48.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:20:48.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:48.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:20:48.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:20:48.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.586 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:48.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:20:48.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:20:48.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.586 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:48.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:20:48.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:20:48.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:48.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:20:48.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:20:48.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:48.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:20:48.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:20:48.587 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.587 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:48.587 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.587 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:20:48.588 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:48.588 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 145.71649134200001, 2024-09-17T21:20:48.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:20:48.588 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:48.588 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:48.588 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:48.588 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:48.588 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:20:48.588 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:20:48.588 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:20:48.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.588 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:48.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:48.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:48.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:48.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:48.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:48.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:20:48.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:20:48.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:48.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:20:48.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:20:48.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:48.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:20:48.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:20:48.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:48.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:20:48.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:20:48.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:48.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:20:48.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:20:48.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:48.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:20:48.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:20:48.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:48.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:20:48.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:20:48.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:48.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:20:48.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:20:48.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:48.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:20:48.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:20:48.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:48.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:20:48.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:20:48.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:48.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:20:48.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:20:48.593 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.593 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:48.594 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.594 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:20:48.594 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:48.594 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 145.706845856, 2024-09-17T21:20:48.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:20:48.594 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:48.594 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:48.594 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:48.594 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:48.594 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:20:48.594 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:20:48.595 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:20:48.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.595 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:48.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:48.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:48.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:48.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:48.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:48.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:20:48.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:20:48.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:48.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:20:48.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:20:48.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:48.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:20:48.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:20:48.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:48.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:20:48.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:20:48.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:48.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:20:48.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:20:48.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:48.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:20:48.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:20:48.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:48.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:20:48.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:20:48.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:48.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:20:48.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:20:48.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:48.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:20:48.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:20:48.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:48.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:20:48.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:20:48.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:48.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:20:48.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:20:48.599 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.599 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:20:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 132.33790982299999, 2024-09-17T21:20:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:20:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:20:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:20:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:20:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:20:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:20:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:20:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:20:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:20:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:20:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:20:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:20:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:20:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:20:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:20:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:20:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:20:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:20:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:819 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5111808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 114.136337533, 2024-09-17T21:20:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024344811000000001, 2024-09-17T21:20:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 819 2024-09-17T21:20:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437443+0000", 2024-09-17T21:20:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1969999999999999e-06 2024-09-17T21:20:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437448+0000", 2024-09-17T21:20:48.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9490000000000004e-06 2024-09-17T21:20:48.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:48.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437467+0000", 2024-09-17T21:20:48.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8862999999999999e-05 2024-09-17T21:20:48.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:48.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437480+0000", 2024-09-17T21:20:48.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2833000000000001e-05 2024-09-17T21:20:48.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461552+0000", 2024-09-17T21:20:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024072695000000002 2024-09-17T21:20:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461639+0000", 2024-09-17T21:20:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6996000000000005e-05 2024-09-17T21:20:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461787+0000", 2024-09-17T21:20:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014727800000000001 2024-09-17T21:20:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 114.13622526100001, 2024-09-17T21:20:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:20:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:20:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:20:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:20:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:20:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:20:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:48.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:20:48.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:20:48.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:48.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:20:48.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:20:48.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:48.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:20:48.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:20:48.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:48.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:20:48.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:20:48.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:48.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:20:48.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:20:48.613 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.613 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:48.613 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.614 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:833 3.5 3:a772dfb1:::rbd_data.11523651b015.0000000000000001:d [sparse-read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:48.614 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:48.614 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 114.13579097500001, 2024-09-17T21:20:48.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024216024999999999, 2024-09-17T21:20:48.614 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:48.614 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:48.614 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:48.614 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:48.614 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:48.614 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:48.614 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 833 2024-09-17T21:20:48.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.614 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:48.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:48.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:48.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:48.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:48.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:48.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437990+0000", 2024-09-17T21:20:48.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2500000000000001e-06 2024-09-17T21:20:48.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:48.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437995+0000", 2024-09-17T21:20:48.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1240000000000004e-06 2024-09-17T21:20:48.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:48.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438012+0000", 2024-09-17T21:20:48.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7572e-05 2024-09-17T21:20:48.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:48.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438031+0000", 2024-09-17T21:20:48.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8612000000000001e-05 2024-09-17T21:20:48.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:48.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461830+0000", 2024-09-17T21:20:48.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023799135999999999 2024-09-17T21:20:48.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:48.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462005+0000", 2024-09-17T21:20:48.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000174747 2024-09-17T21:20:48.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:48.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:20:48.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019958399999999999 2024-09-17T21:20:48.618 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.618 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:48.618 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.618 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:48.618 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:48.618 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 114.13573068399999, 2024-09-17T21:20:48.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:20:48.618 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:48.618 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:48.619 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:48.619 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:48.619 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:48.619 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:48.619 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:20:48.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.619 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:48.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:48.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:48.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:48.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:48.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:48.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:20:48.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:20:48.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:48.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:20:48.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:20:48.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:48.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:20:48.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:20:48.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:48.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:20:48.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:20:48.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:48.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:20:48.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:20:48.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:48.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:20:48.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:20:48.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:48.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:20:48.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:20:48.622 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.622 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:48.622 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.623 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:48.623 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:48.623 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 114.135675303, 2024-09-17T21:20:48.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:20:48.623 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:48.623 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:48.623 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:48.623 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:48.623 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:48.623 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:48.623 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:20:48.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.623 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:48.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:48.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:48.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:48.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:48.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:48.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:20:48.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:20:48.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:48.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:20:48.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:20:48.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:48.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:20:48.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:20:48.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:48.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:20:48.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:20:48.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:48.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:20:48.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:20:48.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:48.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:20:48.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:20:48.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:48.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:20:48.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:20:48.627 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.627 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:48.627 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.627 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:48.627 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:48.627 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 114.135619025, 2024-09-17T21:20:48.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:20:48.627 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:48.628 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:48.628 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:48.628 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:48.628 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:48.628 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:48.628 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:20:48.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.628 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:48.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:48.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:48.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:48.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:48.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:48.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:20:48.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:20:48.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:48.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:20:48.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:20:48.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:48.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:20:48.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:20:48.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:48.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:20:48.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:20:48.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:48.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:20:48.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:20:48.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:48.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:20:48.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:20:48.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:48.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:20:48.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:20:48.631 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.632 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:48.632 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.632 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:48.632 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:48.632 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 114.134669386, 2024-09-17T21:20:48.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:20:48.632 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:48.632 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:48.632 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:48.632 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:48.632 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:48.633 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:48.633 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:20:48.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.633 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:48.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:48.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:48.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:48.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:48.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:48.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:20:48.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:20:48.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:48.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:20:48.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:20:48.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:48.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:20:48.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:20:48.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:48.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:20:48.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:20:48.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:48.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:20:48.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:20:48.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:48.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:20:48.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:20:48.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:48.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:20:48.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:20:48.636 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.636 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:48.636 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.637 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:48.637 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:48.637 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 114.134590766, 2024-09-17T21:20:48.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:20:48.637 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:48.637 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:48.637 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:48.637 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:48.637 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:48.637 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:48.637 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:20:48.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.637 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:48.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:48.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:48.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:48.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:48.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:48.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:20:48.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:20:48.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:48.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:20:48.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:20:48.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:48.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:20:48.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:20:48.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:48.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:20:48.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:20:48.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:48.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:20:48.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:20:48.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:48.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:20:48.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:20:48.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:48.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:20:48.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:20:48.641 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.641 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:48.641 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.641 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:48.641 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:48.641 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 114.134477827, 2024-09-17T21:20:48.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:20:48.641 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:48.642 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:48.642 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:48.642 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:48.642 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:48.642 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:48.642 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:20:48.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.642 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:48.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:48.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:48.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:48.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:48.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:48.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:20:48.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:20:48.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:48.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:20:48.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:20:48.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:48.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:20:48.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:20:48.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:48.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:20:48.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:20:48.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:48.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:20:48.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:20:48.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:48.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:20:48.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:20:48.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:48.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:20:48.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:20:48.645 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.645 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:48.645 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.646 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:849 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5963776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:48.646 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:48.646 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 114.133883791, 2024-09-17T21:20:48.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024537514, 2024-09-17T21:20:48.646 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:48.646 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:48.646 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:48.646 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:48.646 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:48.646 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:48.646 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 849 2024-09-17T21:20:48.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.647 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:48.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:48.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:48.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:48.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:48.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:48.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439897+0000", 2024-09-17T21:20:48.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9099999999999999e-06 2024-09-17T21:20:48.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:48.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439905+0000", 2024-09-17T21:20:48.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0870000000000007e-06 2024-09-17T21:20:48.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:48.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439924+0000", 2024-09-17T21:20:48.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9042000000000002e-05 2024-09-17T21:20:48.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:48.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439941+0000", 2024-09-17T21:20:48.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.624e-05 2024-09-17T21:20:48.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:48.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464132+0000", 2024-09-17T21:20:48.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024191579000000001 2024-09-17T21:20:48.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:48.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464291+0000", 2024-09-17T21:20:48.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015904800000000001 2024-09-17T21:20:48.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:48.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464433+0000", 2024-09-17T21:20:48.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014160799999999999 2024-09-17T21:20:48.650 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.650 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:48.650 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.650 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:48.650 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:48.650 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 114.13381677300001, 2024-09-17T21:20:48.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:20:48.651 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:48.651 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:48.651 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:48.651 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:48.651 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:48.651 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:48.651 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:20:48.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.651 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:48.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:48.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:48.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:48.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:48.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:48.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:20:48.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:20:48.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:48.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:20:48.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:20:48.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:48.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:20:48.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:20:48.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:48.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:20:48.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:20:48.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:48.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:20:48.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:20:48.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:48.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:20:48.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:20:48.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:48.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:20:48.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:20:48.654 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.654 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:48.655 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.655 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:48.655 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:48.655 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 114.13375196200001, 2024-09-17T21:20:48.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:20:48.655 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:48.655 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:48.655 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:48.655 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:48.655 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:48.655 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:48.656 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:20:48.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.656 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:48.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:48.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:48.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:48.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:48.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:48.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:20:48.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:20:48.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:48.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:20:48.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:20:48.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:20:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:20:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:20:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:20:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:20:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:20:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:20:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:20:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:20:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:20:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 114.133690985, 2024-09-17T21:20:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:20:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:20:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:20:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:20:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:20:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:20:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:20:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:20:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:20:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:20:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:20:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:20:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:20:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:20:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:20:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:20:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:20:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.236119799999997, 2024-09-17T21:20:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:20:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:20:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:20:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:20:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:20:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:20:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:20:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:20:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:20:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:20:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:20:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:48.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:20:48.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:20:48.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:48.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:20:48.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:20:48.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:20:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:20:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:20:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.027461406, 2024-09-17T21:20:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:20:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:20:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:20:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:20:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:20:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:20:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:20:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:20:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:20:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:20:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:20:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:20:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:20:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:20:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:20:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:20:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:20:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.674 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:48.674 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.674 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:48.674 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:48.674 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:48.675 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:20:48.676 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:48.668+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:48.688 INFO:teuthology.orchestra.run.smithi184.stdout:leaving image image_client.0-clone38 intact 2024-09-17T21:20:48.776 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:48.774+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:48.808 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:48.808 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:48.808 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:48.809 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:48.822 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:20:48.877 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:48.872+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:48.947 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:48.947 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:48.947 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:48.947 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:48.947 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:48.962 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:20:49.005 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:49.000+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:49.079 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:49.074+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:49.132 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:49.132 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:49.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:49.133 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:49.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.133 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:525 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3670016~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:49.133 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:49.133 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.928472868, 2024-09-17T21:20:49.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046645697, 2024-09-17T21:20:49.133 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.133 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.133 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.133 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.133 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:49.133 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:49.133 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 525 2024-09-17T21:20:49.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.134 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:49.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:49.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:20:49.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:20:49.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200234+0000", 2024-09-17T21:20:49.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:20:49.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200254+0000", 2024-09-17T21:20:49.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9959999999999999e-05 2024-09-17T21:20:49.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200293+0000", 2024-09-17T21:20:49.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9755999999999997e-05 2024-09-17T21:20:49.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245871+0000", 2024-09-17T21:20:49.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045577871999999998 2024-09-17T21:20:49.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246030+0000", 2024-09-17T21:20:49.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158975 2024-09-17T21:20:49.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246871+0000", 2024-09-17T21:20:49.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084111699999999995 2024-09-17T21:20:49.137 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.137 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.137 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.137 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:49.137 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:49.137 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.928227193, 2024-09-17T21:20:49.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:20:49.137 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.137 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.137 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.137 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.138 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:49.138 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:49.138 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:20:49.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.138 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.138 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.138 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.138 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:49.138 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.138 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.138 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.138 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:49.138 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:20:49.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:20:49.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:20:49.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:20:49.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:20:49.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:20:49.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.140 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:20:49.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:20:49.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.140 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:20:49.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:20:49.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:20:49.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:20:49.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:20:49.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:20:49.141 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.141 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.141 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.142 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:49.142 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:49.142 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.92804851099999, 2024-09-17T21:20:49.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:20:49.142 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.142 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.142 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.142 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.142 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:49.142 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:49.142 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:20:49.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.142 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.143 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.143 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:49.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.143 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.143 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:49.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.143 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.143 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:20:49.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:20:49.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:20:49.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:20:49.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:20:49.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:20:49.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:20:49.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:20:49.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:20:49.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:20:49.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:20:49.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:20:49.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:20:49.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:20:49.146 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.146 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.146 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.146 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:49.146 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:49.146 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.92798923800001, 2024-09-17T21:20:49.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:20:49.146 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.146 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.147 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.147 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.147 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:49.147 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:49.147 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:20:49.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.147 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.147 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:49.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.147 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:49.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:20:49.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:20:49.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:20:49.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:20:49.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:20:49.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:20:49.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:20:49.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:20:49.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:20:49.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:20:49.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.150 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.150 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:20:49.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:20:49.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.150 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.150 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:20:49.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:20:49.150 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.150 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.150 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.151 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:49.151 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:49.151 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.92784704300001, 2024-09-17T21:20:49.151 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:20:49.151 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.151 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.151 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.151 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.151 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:49.151 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:49.151 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:20:49.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.151 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:49.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:49.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:20:49.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:20:49.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:20:49.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:20:49.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:20:49.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:20:49.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:20:49.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:20:49.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:20:49.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:20:49.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:20:49.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:20:49.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:20:49.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:20:49.155 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.155 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.155 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.155 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:49.155 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:49.155 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.92779111999999, 2024-09-17T21:20:49.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:20:49.155 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.155 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.155 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.155 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.156 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:49.156 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:49.156 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:20:49.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.156 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.156 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.156 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:49.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.156 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.156 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:49.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.157 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.157 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.157 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:20:49.157 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:20:49.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.157 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.157 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.157 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:20:49.157 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:20:49.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.157 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.157 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.157 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:20:49.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:20:49.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:20:49.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:20:49.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:20:49.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:20:49.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:20:49.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:20:49.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:20:49.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:20:49.159 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.159 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.159 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.159 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:49.159 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:49.159 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.927737503, 2024-09-17T21:20:49.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:20:49.160 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.160 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.160 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.160 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.160 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:49.160 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:49.160 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:20:49.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.160 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.161 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:49.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.161 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:49.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.161 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:20:49.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:20:49.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:20:49.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:20:49.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:20:49.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:20:49.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:20:49.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:20:49.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.163 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.163 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:20:49.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:20:49.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.163 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.163 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:20:49.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:20:49.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.163 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:20:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:20:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:20:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 92.618509261, 2024-09-17T21:20:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:20:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:20:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:20:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:20:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:20:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:49.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:20:49.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:20:49.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:20:49.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:20:49.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:20:49.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:20:49.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:20:49.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:20:49.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:20:49.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:20:49.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:49.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:20:49.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:20:49.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:20:49.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:20:49.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:20:49.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:20:49.168 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.169 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.169 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.169 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2597 3.1f 3:fb0e9615:::rbd_data.1152a8ae2019.000000000000000c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:49.169 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:49.169 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 68.290072832999996, 2024-09-17T21:20:49.169 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097064994000000002, 2024-09-17T21:20:49.169 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.169 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.169 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.169 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.169 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:49.169 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2597 2024-09-17T21:20:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838628+0000", 2024-09-17T21:20:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.187e-06 2024-09-17T21:20:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838654+0000", 2024-09-17T21:20:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:20:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838696+0000", 2024-09-17T21:20:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1226000000000001e-05 2024-09-17T21:20:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838715+0000", 2024-09-17T21:20:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9646000000000001e-05 2024-09-17T21:20:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838937+0000", 2024-09-17T21:20:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022163 2024-09-17T21:20:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.839284+0000", 2024-09-17T21:20:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034753799999999999 2024-09-17T21:20:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935691+0000", 2024-09-17T21:20:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.096406360999999996 2024-09-17T21:20:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2641 3.12 3:488a729f:::rbd_data.1152a8ae2019.0000000000000022:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 68.28661606, 2024-09-17T21:20:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098280953000000004, 2024-09-17T21:20:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2641 2024-09-17T21:20:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842083+0000", 2024-09-17T21:20:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5300000000000002e-07 2024-09-17T21:20:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842088+0000", 2024-09-17T21:20:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:20:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842105+0000", 2024-09-17T21:20:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6773e-05 2024-09-17T21:20:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842119+0000", 2024-09-17T21:20:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:20:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842320+0000", 2024-09-17T21:20:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020052700000000001 2024-09-17T21:20:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842845+0000", 2024-09-17T21:20:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052508600000000002 2024-09-17T21:20:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940363+0000", 2024-09-17T21:20:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097518795000000005 2024-09-17T21:20:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2674 3.12 3:4c87e3ac:::rbd_data.1152a8ae2019.0000000000000033:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 68.286253423000005, 2024-09-17T21:20:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098025125000000005, 2024-09-17T21:20:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2674 2024-09-17T21:20:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842447+0000", 2024-09-17T21:20:49.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3999999999999999e-06 2024-09-17T21:20:49.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842453+0000", 2024-09-17T21:20:49.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:20:49.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842472+0000", 2024-09-17T21:20:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9006000000000001e-05 2024-09-17T21:20:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842487+0000", 2024-09-17T21:20:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4866e-05 2024-09-17T21:20:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843594+0000", 2024-09-17T21:20:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011073249999999999 2024-09-17T21:20:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843668+0000", 2024-09-17T21:20:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3188000000000005e-05 2024-09-17T21:20:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843893+0000", 2024-09-17T21:20:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022503399999999999 2024-09-17T21:20:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850439+0000", 2024-09-17T21:20:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065467880000000004 2024-09-17T21:20:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940409+0000", 2024-09-17T21:20:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089969511000000002 2024-09-17T21:20:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940444+0000", 2024-09-17T21:20:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5345999999999997e-05 2024-09-17T21:20:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940470+0000", 2024-09-17T21:20:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5964000000000001e-05 2024-09-17T21:20:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2707 3.1f 3:f805e65e:::rbd_data.1152a8ae2019.0000000000000049:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 68.281206761999997, 2024-09-17T21:20:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088327300999999997, 2024-09-17T21:20:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2707 2024-09-17T21:20:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847494+0000", 2024-09-17T21:20:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:20:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847504+0000", 2024-09-17T21:20:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:20:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847526+0000", 2024-09-17T21:20:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1976000000000001e-05 2024-09-17T21:20:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847541+0000", 2024-09-17T21:20:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5277e-05 2024-09-17T21:20:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856142+0000", 2024-09-17T21:20:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086006710000000007 2024-09-17T21:20:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856435+0000", 2024-09-17T21:20:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029301499999999999 2024-09-17T21:20:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935819+0000", 2024-09-17T21:20:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079384448999999996 2024-09-17T21:20:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2730 3.12 3:48b3422a:::rbd_data.1152a8ae2019.0000000000000057:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 68.272013862999998, 2024-09-17T21:20:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083879975999999995, 2024-09-17T21:20:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2730 2024-09-17T21:20:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856687+0000", 2024-09-17T21:20:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:20:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856697+0000", 2024-09-17T21:20:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0207000000000001e-05 2024-09-17T21:20:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856808+0000", 2024-09-17T21:20:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011071200000000001 2024-09-17T21:20:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856828+0000", 2024-09-17T21:20:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0219000000000001e-05 2024-09-17T21:20:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856907+0000", 2024-09-17T21:20:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9661999999999997e-05 2024-09-17T21:20:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.857019+0000", 2024-09-17T21:20:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011148499999999999 2024-09-17T21:20:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940565+0000", 2024-09-17T21:20:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083545791999999994 2024-09-17T21:20:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2743 3.12 3:4d036e62:::rbd_data.1152a8ae2019.000000000000005f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 68.268068361999994, 2024-09-17T21:20:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080059962999999998, 2024-09-17T21:20:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2743 2024-09-17T21:20:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860631+0000", 2024-09-17T21:20:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:20:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860636+0000", 2024-09-17T21:20:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5279999999999997e-06 2024-09-17T21:20:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860649+0000", 2024-09-17T21:20:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-17T21:20:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860655+0000", 2024-09-17T21:20:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:20:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860731+0000", 2024-09-17T21:20:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5920000000000005e-05 2024-09-17T21:20:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.861070+0000", 2024-09-17T21:20:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338535 2024-09-17T21:20:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940690+0000", 2024-09-17T21:20:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079620503999999995 2024-09-17T21:20:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3378 3.12 3:4832a51b:::rbd_data.1152a8ae2019.000000000000006c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 68.239704512000003, 2024-09-17T21:20:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051812516000000003, 2024-09-17T21:20:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3378 2024-09-17T21:20:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888995+0000", 2024-09-17T21:20:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:20:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889006+0000", 2024-09-17T21:20:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1012999999999999e-05 2024-09-17T21:20:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889032+0000", 2024-09-17T21:20:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6441e-05 2024-09-17T21:20:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889040+0000", 2024-09-17T21:20:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0639999999999994e-06 2024-09-17T21:20:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889278+0000", 2024-09-17T21:20:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023785899999999999 2024-09-17T21:20:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889408+0000", 2024-09-17T21:20:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129219 2024-09-17T21:20:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889723+0000", 2024-09-17T21:20:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000315264 2024-09-17T21:20:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.891972+0000", 2024-09-17T21:20:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002249633 2024-09-17T21:20:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940733+0000", 2024-09-17T21:20:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048760626000000001 2024-09-17T21:20:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940785+0000", 2024-09-17T21:20:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1906000000000001e-05 2024-09-17T21:20:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940807+0000", 2024-09-17T21:20:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1554999999999999e-05 2024-09-17T21:20:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3379 3.12 3:4e83fd71:::rbd_data.1152a8ae2019.000000000000006d:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 68.239620305000003, 2024-09-17T21:20:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051804144000000003, 2024-09-17T21:20:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3379 2024-09-17T21:20:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:49.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889079+0000", 2024-09-17T21:20:49.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1800000000000004e-07 2024-09-17T21:20:49.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889083+0000", 2024-09-17T21:20:49.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:20:49.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889094+0000", 2024-09-17T21:20:49.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1562999999999999e-05 2024-09-17T21:20:49.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889102+0000", 2024-09-17T21:20:49.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.401e-06 2024-09-17T21:20:49.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:20:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014279259999999999 2024-09-17T21:20:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890609+0000", 2024-09-17T21:20:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9566000000000004e-05 2024-09-17T21:20:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890789+0000", 2024-09-17T21:20:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017949700000000001 2024-09-17T21:20:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:49.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.893929+0000", 2024-09-17T21:20:49.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003140538 2024-09-17T21:20:49.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:49.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940834+0000", 2024-09-17T21:20:49.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046904811999999997 2024-09-17T21:20:49.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:49.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940860+0000", 2024-09-17T21:20:49.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5341999999999999e-05 2024-09-17T21:20:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940882+0000", 2024-09-17T21:20:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2878999999999999e-05 2024-09-17T21:20:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:2644 3.1e 3:79615585:::rbd_data.115266d0b9f2.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2475520~14848,write 3106304~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:20:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 59.616991902000002, 2024-09-17T21:20:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050733484000000002, 2024-09-17T21:20:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:20:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:20:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2644 2024-09-17T21:20:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511708+0000", 2024-09-17T21:20:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-17T21:20:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522196+0000", 2024-09-17T21:20:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010487712999999999 2024-09-17T21:20:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522223+0000", 2024-09-17T21:20:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7064999999999999e-05 2024-09-17T21:20:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522236+0000", 2024-09-17T21:20:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2931e-05 2024-09-17T21:20:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522348+0000", 2024-09-17T21:20:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111906 2024-09-17T21:20:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522554+0000", 2024-09-17T21:20:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020632699999999999 2024-09-17T21:20:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:20:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.523185+0000", 2024-09-17T21:20:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063081400000000005 2024-09-17T21:20:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.547011+0000", 2024-09-17T21:20:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023825570000000001 2024-09-17T21:20:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562268+0000", 2024-09-17T21:20:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015257514 2024-09-17T21:20:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562367+0000", 2024-09-17T21:20:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8678000000000006e-05 2024-09-17T21:20:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562440+0000", 2024-09-17T21:20:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3337000000000006e-05 2024-09-17T21:20:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:231 3.1 3:856fed78:::rbd_data.115220dd393e.000000000000000b:head [read 0~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:20:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:20:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.174236633, 2024-09-17T21:20:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34254251699999999, 2024-09-17T21:20:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:20:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 231 2024-09-17T21:20:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:20:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:20:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954464+0000", 2024-09-17T21:20:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:20:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954475+0000", 2024-09-17T21:20:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1146e-05 2024-09-17T21:20:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954506+0000", 2024-09-17T21:20:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0292999999999999e-05 2024-09-17T21:20:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954524+0000", 2024-09-17T21:20:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8430000000000001e-05 2024-09-17T21:20:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954661+0000", 2024-09-17T21:20:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136403 2024-09-17T21:20:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954680+0000", 2024-09-17T21:20:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9743000000000001e-05 2024-09-17T21:20:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.296896+0000", 2024-09-17T21:20:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34221550499999998 2024-09-17T21:20:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297004+0000", 2024-09-17T21:20:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010865399999999999 2024-09-17T21:20:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:56 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:20:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:20:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.018602936000001, 2024-09-17T21:20:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18718773899999999, 2024-09-17T21:20:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:20:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:20:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 56 2024-09-17T21:20:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:20:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:20:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110098+0000", 2024-09-17T21:20:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:20:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110111+0000", 2024-09-17T21:20:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2289999999999999e-05 2024-09-17T21:20:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110143+0000", 2024-09-17T21:20:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1979e-05 2024-09-17T21:20:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110161+0000", 2024-09-17T21:20:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7839e-05 2024-09-17T21:20:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110319+0000", 2024-09-17T21:20:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015835400000000001 2024-09-17T21:20:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110353+0000", 2024-09-17T21:20:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3849000000000001e-05 2024-09-17T21:20:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297041+0000", 2024-09-17T21:20:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18668842299999999 2024-09-17T21:20:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297169+0000", 2024-09-17T21:20:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012786199999999999 2024-09-17T21:20:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297283+0000", 2024-09-17T21:20:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011430300000000001 2024-09-17T21:20:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:68 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:20:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:20:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.010062005000002, 2024-09-17T21:20:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17885582999999999, 2024-09-17T21:20:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:20:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:20:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 68 2024-09-17T21:20:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:20:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:20:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118639+0000", 2024-09-17T21:20:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.306e-06 2024-09-17T21:20:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118651+0000", 2024-09-17T21:20:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2567e-05 2024-09-17T21:20:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118682+0000", 2024-09-17T21:20:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0502e-05 2024-09-17T21:20:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118707+0000", 2024-09-17T21:20:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5463e-05 2024-09-17T21:20:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118889+0000", 2024-09-17T21:20:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018150999999999999 2024-09-17T21:20:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118910+0000", 2024-09-17T21:20:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:20:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297313+0000", 2024-09-17T21:20:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17840366699999999 2024-09-17T21:20:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297389+0000", 2024-09-17T21:20:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-17T21:20:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297492+0000", 2024-09-17T21:20:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103019 2024-09-17T21:20:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.231 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:49.232 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:20:49.233 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:49.226+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:49.333 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:49.330+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:49.369 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:49.369 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:49.369 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:49.369 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:49.385 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:20:49.436 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:49.432+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:49.511 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:49.511 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:49.512 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:49.512 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:49.512 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:49.525 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:20:49.535 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:49.530+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:49.636 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:49.628+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:49.714 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:49.714 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:49.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:49.715 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:49.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.715 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:20:49.715 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:49.715 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 129.90222376299999, 2024-09-17T21:20:49.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:20:49.715 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.715 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.715 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.715 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.715 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:20:49.715 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:20:49.715 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:20:49.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.716 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:49.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:49.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:20:49.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:20:49.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:20:49.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:20:49.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:20:49.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:20:49.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:20:49.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:20:49.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:20:49.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:20:49.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:49.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:20:49.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:20:49.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:20:49.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:20:49.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:20:49.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:20:49.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:20:49.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:20:49.720 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.720 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.720 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.720 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:20:49.720 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:49.720 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 116.347121977, 2024-09-17T21:20:49.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:20:49.720 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.720 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.720 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:49.720 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.721 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:20:49.721 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:49.721 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:20:49.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.721 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:49.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:49.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:20:49.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:20:49.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:20:49.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:20:49.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:20:49.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:20:49.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:20:49.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:20:49.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:20:49.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:20:49.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:49.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:20:49.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:20:49.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:20:49.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:20:49.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:20:49.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:20:49.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:49.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:20:49.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:20:49.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:49.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:20:49.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:20:49.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:49.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:20:49.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:20:49.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:49.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:20:49.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:20:49.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:20:49.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:20:49.726 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.727 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.727 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.727 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:793 3.9 3:95f63599:::rbd_data.1152eb799094.0000000000000007:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:20:49.727 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:20:49.727 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 33.280065821999997, 2024-09-17T21:20:49.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88245766400000003, 2024-09-17T21:20:49.727 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.727 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.727 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:49.727 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.727 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:20:49.727 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:20:49.728 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 793 2024-09-17T21:20:49.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.728 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:20:49.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:20:49.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431941+0000", 2024-09-17T21:20:49.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.864e-06 2024-09-17T21:20:49.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431953+0000", 2024-09-17T21:20:49.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1486e-05 2024-09-17T21:20:49.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431984+0000", 2024-09-17T21:20:49.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1075999999999999e-05 2024-09-17T21:20:49.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431999+0000", 2024-09-17T21:20:49.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.503e-05 2024-09-17T21:20:49.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432095+0000", 2024-09-17T21:20:49.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5698000000000004e-05 2024-09-17T21:20:49.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:49.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432109+0000", 2024-09-17T21:20:49.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4188e-05 2024-09-17T21:20:49.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.309642+0000", 2024-09-17T21:20:49.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87753353000000001 2024-09-17T21:20:49.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310104+0000", 2024-09-17T21:20:49.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046103600000000001 2024-09-17T21:20:49.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:49.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310630+0000", 2024-09-17T21:20:49.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052682800000000002 2024-09-17T21:20:49.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:49.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313100+0000", 2024-09-17T21:20:49.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024693369999999998 2024-09-17T21:20:49.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:49.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313935+0000", 2024-09-17T21:20:49.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083540100000000005 2024-09-17T21:20:49.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:49.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314086+0000", 2024-09-17T21:20:49.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015053399999999999 2024-09-17T21:20:49.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314396+0000", 2024-09-17T21:20:49.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031065599999999997 2024-09-17T21:20:49.733 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.733 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.733 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.734 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:800 3.19 3:9dc17f12:::rbd_data.1152eb799094.000000000000000e:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:20:49.734 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:20:49.734 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 33.279818577, 2024-09-17T21:20:49.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88154663799999999, 2024-09-17T21:20:49.734 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.734 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.734 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:49.734 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.734 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:20:49.734 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:20:49.734 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 800 2024-09-17T21:20:49.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.734 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:20:49.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:20:49.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432187+0000", 2024-09-17T21:20:49.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4100000000000001e-06 2024-09-17T21:20:49.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432194+0000", 2024-09-17T21:20:49.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0389999999999999e-06 2024-09-17T21:20:49.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432216+0000", 2024-09-17T21:20:49.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1614e-05 2024-09-17T21:20:49.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432232+0000", 2024-09-17T21:20:49.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5739e-05 2024-09-17T21:20:49.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432405+0000", 2024-09-17T21:20:49.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017355300000000001 2024-09-17T21:20:49.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:49.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432432+0000", 2024-09-17T21:20:49.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6979e-05 2024-09-17T21:20:49.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.307767+0000", 2024-09-17T21:20:49.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87533472899999998 2024-09-17T21:20:49.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308192+0000", 2024-09-17T21:20:49.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042485999999999999 2024-09-17T21:20:49.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:49.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308727+0000", 2024-09-17T21:20:49.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00053550400000000001 2024-09-17T21:20:49.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:49.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310347+0000", 2024-09-17T21:20:49.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016198670000000001 2024-09-17T21:20:49.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:49.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313424+0000", 2024-09-17T21:20:49.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030764640000000001 2024-09-17T21:20:49.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:49.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313510+0000", 2024-09-17T21:20:49.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5938000000000003e-05 2024-09-17T21:20:49.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313733+0000", 2024-09-17T21:20:49.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022294200000000001 2024-09-17T21:20:49.740 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.740 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.740 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.740 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:805 3.19 3:9cab2ed6:::rbd_data.1152eb799094.0000000000000013:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:20:49.740 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:20:49.740 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 33.279614557999999, 2024-09-17T21:20:49.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88274355699999996, 2024-09-17T21:20:49.740 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.741 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.741 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:49.741 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.741 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:20:49.741 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:20:49.741 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 805 2024-09-17T21:20:49.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.741 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:20:49.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.742 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:20:49.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.742 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432391+0000", 2024-09-17T21:20:49.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3e-06 2024-09-17T21:20:49.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.742 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432398+0000", 2024-09-17T21:20:49.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4489999999999996e-06 2024-09-17T21:20:49.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432415+0000", 2024-09-17T21:20:49.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7750000000000001e-05 2024-09-17T21:20:49.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432430+0000", 2024-09-17T21:20:49.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.449e-05 2024-09-17T21:20:49.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432635+0000", 2024-09-17T21:20:49.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205031 2024-09-17T21:20:49.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:49.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432657+0000", 2024-09-17T21:20:49.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2064000000000001e-05 2024-09-17T21:20:49.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310444+0000", 2024-09-17T21:20:49.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87778699199999999 2024-09-17T21:20:49.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310849+0000", 2024-09-17T21:20:49.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00040497800000000001 2024-09-17T21:20:49.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:49.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.311396+0000", 2024-09-17T21:20:49.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00054729099999999999 2024-09-17T21:20:49.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:49.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313847+0000", 2024-09-17T21:20:49.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024504940000000001 2024-09-17T21:20:49.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:49.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314879+0000", 2024-09-17T21:20:49.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001031817 2024-09-17T21:20:49.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:49.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314964+0000", 2024-09-17T21:20:49.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5685000000000004e-05 2024-09-17T21:20:49.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.315133+0000", 2024-09-17T21:20:49.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000169216 2024-09-17T21:20:49.746 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.746 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.747 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.747 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12087 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 52224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:49.747 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:20:49.747 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.942525880000002, 2024-09-17T21:20:49.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087684460000000006, 2024-09-17T21:20:49.747 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.747 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.747 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.747 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.747 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:49.747 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:49.747 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12087 2024-09-17T21:20:49.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.748 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:20:49.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:20:49.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769480+0000", 2024-09-17T21:20:49.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:20:49.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769485+0000", 2024-09-17T21:20:49.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8880000000000002e-06 2024-09-17T21:20:49.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769509+0000", 2024-09-17T21:20:49.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3343e-05 2024-09-17T21:20:49.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769521+0000", 2024-09-17T21:20:49.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2497000000000001e-05 2024-09-17T21:20:49.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.856883+0000", 2024-09-17T21:20:49.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087362191000000006 2024-09-17T21:20:49.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857014+0000", 2024-09-17T21:20:49.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013090399999999999 2024-09-17T21:20:49.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857163+0000", 2024-09-17T21:20:49.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014864000000000001 2024-09-17T21:20:49.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.751 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.751 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12088 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 56320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:49.751 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:20:49.751 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.942442225000001, 2024-09-17T21:20:49.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087917772000000005, 2024-09-17T21:20:49.751 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.751 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.752 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.752 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.752 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:49.752 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:49.752 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12088 2024-09-17T21:20:49.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.752 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.752 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.752 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:20:49.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.752 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:20:49.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769569+0000", 2024-09-17T21:20:49.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:20:49.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769574+0000", 2024-09-17T21:20:49.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4659999999999998e-06 2024-09-17T21:20:49.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769586+0000", 2024-09-17T21:20:49.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2311e-05 2024-09-17T21:20:49.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769597+0000", 2024-09-17T21:20:49.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0548e-05 2024-09-17T21:20:49.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857217+0000", 2024-09-17T21:20:49.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087619896000000003 2024-09-17T21:20:49.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857335+0000", 2024-09-17T21:20:49.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118538 2024-09-17T21:20:49.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857480+0000", 2024-09-17T21:20:49.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014470999999999999 2024-09-17T21:20:49.755 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.755 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.755 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.755 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12089 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 60416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:49.756 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:20:49.756 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.942389525999999, 2024-09-17T21:20:49.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088293233999999998, 2024-09-17T21:20:49.756 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.756 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.756 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.756 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.756 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:49.756 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:49.756 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12089 2024-09-17T21:20:49.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.756 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:20:49.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:20:49.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769616+0000", 2024-09-17T21:20:49.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:20:49.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769620+0000", 2024-09-17T21:20:49.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5140000000000002e-06 2024-09-17T21:20:49.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769633+0000", 2024-09-17T21:20:49.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2849e-05 2024-09-17T21:20:49.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769644+0000", 2024-09-17T21:20:49.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1471000000000001e-05 2024-09-17T21:20:49.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857564+0000", 2024-09-17T21:20:49.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087919445999999998 2024-09-17T21:20:49.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857716+0000", 2024-09-17T21:20:49.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000152065 2024-09-17T21:20:49.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857908+0000", 2024-09-17T21:20:49.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019252599999999999 2024-09-17T21:20:49.760 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.760 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.760 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.760 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12090 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 64512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:49.760 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:20:49.760 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.942343140999998, 2024-09-17T21:20:49.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088582672000000001, 2024-09-17T21:20:49.760 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.760 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.761 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.761 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.761 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:49.761 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:49.761 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12090 2024-09-17T21:20:49.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.761 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:20:49.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:20:49.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769662+0000", 2024-09-17T21:20:49.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:20:49.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769683+0000", 2024-09-17T21:20:49.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0979e-05 2024-09-17T21:20:49.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769699+0000", 2024-09-17T21:20:49.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5446000000000001e-05 2024-09-17T21:20:49.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769708+0000", 2024-09-17T21:20:49.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1679999999999993e-06 2024-09-17T21:20:49.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857976+0000", 2024-09-17T21:20:49.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088267795999999996 2024-09-17T21:20:49.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858095+0000", 2024-09-17T21:20:49.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119147 2024-09-17T21:20:49.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858244+0000", 2024-09-17T21:20:49.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014917000000000001 2024-09-17T21:20:49.764 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.764 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.764 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.765 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12091 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 68608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:49.765 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:20:49.765 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.942271757, 2024-09-17T21:20:49.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088832752000000001, 2024-09-17T21:20:49.765 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.765 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.765 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.765 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.765 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:49.765 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:49.765 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12091 2024-09-17T21:20:49.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.765 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:20:49.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:20:49.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769734+0000", 2024-09-17T21:20:49.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:20:49.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769739+0000", 2024-09-17T21:20:49.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8080000000000004e-06 2024-09-17T21:20:49.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769764+0000", 2024-09-17T21:20:49.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4839999999999999e-05 2024-09-17T21:20:49.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769777+0000", 2024-09-17T21:20:49.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3037000000000001e-05 2024-09-17T21:20:49.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858295+0000", 2024-09-17T21:20:49.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088518475999999999 2024-09-17T21:20:49.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858419+0000", 2024-09-17T21:20:49.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012378700000000001 2024-09-17T21:20:49.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858565+0000", 2024-09-17T21:20:49.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014630099999999999 2024-09-17T21:20:49.769 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.769 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.769 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.769 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12092 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:49.769 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:20:49.769 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.942189432999999, 2024-09-17T21:20:49.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089191555000000006, 2024-09-17T21:20:49.769 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.770 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.770 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.770 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.770 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:49.770 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:49.770 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12092 2024-09-17T21:20:49.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.770 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:20:49.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:20:49.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769816+0000", 2024-09-17T21:20:49.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7199999999999997e-07 2024-09-17T21:20:49.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769822+0000", 2024-09-17T21:20:49.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:20:49.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769838+0000", 2024-09-17T21:20:49.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6674e-05 2024-09-17T21:20:49.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769851+0000", 2024-09-17T21:20:49.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2527999999999999e-05 2024-09-17T21:20:49.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858660+0000", 2024-09-17T21:20:49.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088808556999999996 2024-09-17T21:20:49.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858817+0000", 2024-09-17T21:20:49.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015730400000000001 2024-09-17T21:20:49.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859007+0000", 2024-09-17T21:20:49.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018974899999999999 2024-09-17T21:20:49.774 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.774 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.774 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.774 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12093 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:49.774 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:20:49.774 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.942081329000001, 2024-09-17T21:20:49.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089429710999999995, 2024-09-17T21:20:49.774 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.774 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.775 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.775 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.775 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:49.775 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:49.775 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12093 2024-09-17T21:20:49.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.775 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:20:49.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:20:49.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769924+0000", 2024-09-17T21:20:49.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.342e-06 2024-09-17T21:20:49.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769929+0000", 2024-09-17T21:20:49.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0350000000000001e-06 2024-09-17T21:20:49.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769946+0000", 2024-09-17T21:20:49.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6793999999999999e-05 2024-09-17T21:20:49.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769959+0000", 2024-09-17T21:20:49.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3062e-05 2024-09-17T21:20:49.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859075+0000", 2024-09-17T21:20:49.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089115696999999994 2024-09-17T21:20:49.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859202+0000", 2024-09-17T21:20:49.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012657 2024-09-17T21:20:49.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859353+0000", 2024-09-17T21:20:49.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151211 2024-09-17T21:20:49.778 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.778 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.778 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.779 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12094 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:49.779 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:20:49.779 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.942027153000002, 2024-09-17T21:20:49.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089695001999999996, 2024-09-17T21:20:49.779 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.779 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.779 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.779 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.779 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:49.779 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:49.779 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12094 2024-09-17T21:20:49.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.780 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:20:49.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:20:49.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769978+0000", 2024-09-17T21:20:49.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.142e-06 2024-09-17T21:20:49.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769983+0000", 2024-09-17T21:20:49.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.386e-06 2024-09-17T21:20:49.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769997+0000", 2024-09-17T21:20:49.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.416e-05 2024-09-17T21:20:49.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770012+0000", 2024-09-17T21:20:49.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5384999999999999e-05 2024-09-17T21:20:49.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859405+0000", 2024-09-17T21:20:49.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089392376999999995 2024-09-17T21:20:49.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859525+0000", 2024-09-17T21:20:49.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120555 2024-09-17T21:20:49.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859672+0000", 2024-09-17T21:20:49.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000146997 2024-09-17T21:20:49.783 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.783 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.783 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.784 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12095 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:49.784 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:20:49.784 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.941974666, 2024-09-17T21:20:49.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090073476, 2024-09-17T21:20:49.784 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.784 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.784 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.784 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.784 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:49.784 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:49.784 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12095 2024-09-17T21:20:49.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.785 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:20:49.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:20:49.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770031+0000", 2024-09-17T21:20:49.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1699999999999997e-07 2024-09-17T21:20:49.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770036+0000", 2024-09-17T21:20:49.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.023e-06 2024-09-17T21:20:49.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770051+0000", 2024-09-17T21:20:49.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4953999999999999e-05 2024-09-17T21:20:49.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770063+0000", 2024-09-17T21:20:49.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:20:49.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859756+0000", 2024-09-17T21:20:49.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089692980000000005 2024-09-17T21:20:49.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859891+0000", 2024-09-17T21:20:49.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013459499999999999 2024-09-17T21:20:49.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860103+0000", 2024-09-17T21:20:49.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021274400000000001 2024-09-17T21:20:49.788 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.788 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.788 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.788 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12096 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:49.788 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:20:49.788 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.941924981, 2024-09-17T21:20:49.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090358274000000002, 2024-09-17T21:20:49.789 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.789 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.789 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.789 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.789 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:49.789 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:49.789 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12096 2024-09-17T21:20:49.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.789 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:20:49.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:20:49.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770080+0000", 2024-09-17T21:20:49.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9400000000000004e-07 2024-09-17T21:20:49.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770085+0000", 2024-09-17T21:20:49.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2169999999999996e-06 2024-09-17T21:20:49.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770100+0000", 2024-09-17T21:20:49.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5077e-05 2024-09-17T21:20:49.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770112+0000", 2024-09-17T21:20:49.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1951e-05 2024-09-17T21:20:49.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860160+0000", 2024-09-17T21:20:49.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090048413999999993 2024-09-17T21:20:49.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860285+0000", 2024-09-17T21:20:49.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012466699999999999 2024-09-17T21:20:49.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860438+0000", 2024-09-17T21:20:49.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015305399999999999 2024-09-17T21:20:49.793 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.793 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.793 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.793 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12097 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:49.793 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:20:49.793 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.941877234, 2024-09-17T21:20:49.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090702375000000002, 2024-09-17T21:20:49.793 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.793 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.793 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.793 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.794 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:49.794 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:49.794 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12097 2024-09-17T21:20:49.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.794 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:20:49.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:20:49.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770128+0000", 2024-09-17T21:20:49.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.04e-06 2024-09-17T21:20:49.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770133+0000", 2024-09-17T21:20:49.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8999999999999997e-06 2024-09-17T21:20:49.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770152+0000", 2024-09-17T21:20:49.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8890999999999999e-05 2024-09-17T21:20:49.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770164+0000", 2024-09-17T21:20:49.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1769e-05 2024-09-17T21:20:49.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860511+0000", 2024-09-17T21:20:49.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090347527999999996 2024-09-17T21:20:49.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860652+0000", 2024-09-17T21:20:49.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140894 2024-09-17T21:20:49.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860830+0000", 2024-09-17T21:20:49.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017735299999999999 2024-09-17T21:20:49.797 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.797 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.797 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.797 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12098 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:49.797 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:20:49.798 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.941823765999999, 2024-09-17T21:20:49.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090997262999999995, 2024-09-17T21:20:49.798 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.798 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.798 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.798 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.798 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:49.798 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:49.798 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12098 2024-09-17T21:20:49.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.798 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:20:49.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:20:49.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775465+0000", 2024-09-17T21:20:49.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0052843350000000002 2024-09-17T21:20:49.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775476+0000", 2024-09-17T21:20:49.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0797e-05 2024-09-17T21:20:49.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775506+0000", 2024-09-17T21:20:49.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9709999999999998e-05 2024-09-17T21:20:49.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775530+0000", 2024-09-17T21:20:49.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4528999999999998e-05 2024-09-17T21:20:49.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860894+0000", 2024-09-17T21:20:49.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.085364195000000004 2024-09-17T21:20:49.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861019+0000", 2024-09-17T21:20:49.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012467399999999999 2024-09-17T21:20:49.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861178+0000", 2024-09-17T21:20:49.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159023 2024-09-17T21:20:49.802 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.802 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.802 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.802 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12104 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:49.802 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:20:49.802 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.936122656999999, 2024-09-17T21:20:49.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087969628999999994, 2024-09-17T21:20:49.802 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.802 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.802 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.803 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.803 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:49.803 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:49.803 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12104 2024-09-17T21:20:49.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.803 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:20:49.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:20:49.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775883+0000", 2024-09-17T21:20:49.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3399999999999998e-07 2024-09-17T21:20:49.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775886+0000", 2024-09-17T21:20:49.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2119999999999999e-06 2024-09-17T21:20:49.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775895+0000", 2024-09-17T21:20:49.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8710000000000003e-06 2024-09-17T21:20:49.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775901+0000", 2024-09-17T21:20:49.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:20:49.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863544+0000", 2024-09-17T21:20:49.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087642642000000007 2024-09-17T21:20:49.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863694+0000", 2024-09-17T21:20:49.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014952300000000001 2024-09-17T21:20:49.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863851+0000", 2024-09-17T21:20:49.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000157804 2024-09-17T21:20:49.806 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.806 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.806 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.807 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12105 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:49.807 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:49.807 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.936093762999999, 2024-09-17T21:20:49.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088330400000000003, 2024-09-17T21:20:49.807 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.807 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.807 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.807 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.807 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:49.807 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:49.807 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12105 2024-09-17T21:20:49.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.808 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:49.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:49.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:49.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9999999999999997e-07 2024-09-17T21:20:49.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775914+0000", 2024-09-17T21:20:49.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4710000000000001e-06 2024-09-17T21:20:49.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775921+0000", 2024-09-17T21:20:49.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:20:49.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775928+0000", 2024-09-17T21:20:49.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:20:49.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863923+0000", 2024-09-17T21:20:49.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087995126000000007 2024-09-17T21:20:49.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864059+0000", 2024-09-17T21:20:49.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135797 2024-09-17T21:20:49.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864241+0000", 2024-09-17T21:20:49.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018252599999999999 2024-09-17T21:20:49.811 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.811 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.811 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.811 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12106 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:49.811 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:20:49.811 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.936004584999999, 2024-09-17T21:20:49.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088586394999999998, 2024-09-17T21:20:49.811 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:49.812 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:49.812 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:49.812 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:49.812 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:49.812 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:49.812 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12106 2024-09-17T21:20:49.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.812 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:49.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:49.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:20:49.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:49.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:20:49.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:49.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:49.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776001+0000", 2024-09-17T21:20:49.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4300000000000001e-07 2024-09-17T21:20:49.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:49.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776004+0000", 2024-09-17T21:20:49.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8049999999999998e-06 2024-09-17T21:20:49.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:49.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776018+0000", 2024-09-17T21:20:49.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4613999999999999e-05 2024-09-17T21:20:49.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:49.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776025+0000", 2024-09-17T21:20:49.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:20:49.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:49.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864279+0000", 2024-09-17T21:20:49.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088253570000000003 2024-09-17T21:20:49.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:49.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864433+0000", 2024-09-17T21:20:49.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015469400000000001 2024-09-17T21:20:49.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:49.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:49.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:49.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864586+0000", 2024-09-17T21:20:49.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000153003 2024-09-17T21:20:49.815 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.815 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.816 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.816 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:49.816 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:49.816 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:49.817 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-17T21:20:49.818 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:20:49.818 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:49.812+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:49.894 INFO:teuthology.orchestra.run.smithi184.stdout:5402 write 0x4a00600 thru 0x4a0fe91 (0xf892 bytes) 2024-09-17T21:20:49.897 INFO:teuthology.orchestra.run.smithi184.stdout:5403 read 0x780e00 thru 0x78af81 (0xa182 bytes) 2024-09-17T21:20:49.907 INFO:teuthology.orchestra.run.smithi184.stdout:5404 punch from 0xd882400 to 0xd888800, (0x6400 bytes) 2024-09-17T21:20:49.921 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:49.912+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:50.020 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:50.012+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:50.049 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:50.049 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:50.049 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:50.049 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:50.070 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:20:50.161 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:50.156+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:50.167 INFO:teuthology.orchestra.run.smithi184.stdout:5405 read 0xe360600 thru 0xe36e160 (0xdb61 bytes) 2024-09-17T21:20:50.168 INFO:teuthology.orchestra.run.smithi184.stdout:5406 read 0xdd80800 thru 0xdd8a402 (0x9c03 bytes) 2024-09-17T21:20:50.215 INFO:teuthology.orchestra.run.smithi184.stdout:5410 trunc from 0xe86a800 to 0xb12e400 2024-09-17T21:20:50.220 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:50.214+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:50.243 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:50.243 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:50.243 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:50.243 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:50.243 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:50.260 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:20:50.289 INFO:teuthology.orchestra.run.smithi184.stdout:5411 read 0x440fe00 thru 0x4415b0b (0x5d0c bytes) 2024-09-17T21:20:50.301 INFO:teuthology.orchestra.run.smithi184.stdout:5412 write 0x85f5600 thru 0x85f7d41 (0x2742 bytes) 2024-09-17T21:20:50.302 INFO:teuthology.orchestra.run.smithi184.stdout:5413 read 0x8395c00 thru 0x839a7c9 (0x4bca bytes) 2024-09-17T21:20:50.302 INFO:teuthology.orchestra.run.smithi184.stdout:5415 read 0x4870800 thru 0x4871a60 (0x1261 bytes) 2024-09-17T21:20:50.303 INFO:teuthology.orchestra.run.smithi184.stdout:5418 read 0x8911a00 thru 0x8918045 (0x6646 bytes) 2024-09-17T21:20:50.303 INFO:teuthology.orchestra.run.smithi184.stdout:5420 read 0x26b0a00 thru 0x26b49c6 (0x3fc7 bytes) 2024-09-17T21:20:50.309 INFO:teuthology.orchestra.run.smithi184.stdout:5422 punch from 0x1142800 to 0x1149800, (0x7000 bytes) 2024-09-17T21:20:50.314 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:20:50.348 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:50.340+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:50.422 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:50.414+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:50.475 INFO:teuthology.orchestra.run.smithi184.stdout:5423 trunc from 0xb12e400 to 0xe849400 2024-09-17T21:20:50.495 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:50.495 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:50.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:50.495 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:50.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.495 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:20:50.495 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:50.496 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 152.275955697, 2024-09-17T21:20:50.496 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:20:50.496 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:50.496 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:50.496 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:50.496 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:50.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.496 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:50.496 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:50.496 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.496 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:50.496 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:50.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.497 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:50.497 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:20:50.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:20:50.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.497 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:50.497 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:20:50.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:20:50.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.497 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:50.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:20:50.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:20:50.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:50.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:20:50.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:20:50.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:50.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:20:50.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:20:50.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:50.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:20:50.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:20:50.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:50.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:20:50.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:20:50.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:50.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:20:50.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:20:50.499 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.499 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:50.499 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.500 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:20:50.500 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:50.500 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 147.62565503299999, 2024-09-17T21:20:50.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:20:50.500 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:50.500 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:50.500 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:50.500 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:50.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:50.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:50.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:50.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:50.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:50.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:20:50.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:20:50.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:50.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:20:50.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:20:50.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:50.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:20:50.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:20:50.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:50.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:20:50.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:20:50.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:50.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:20:50.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:20:50.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:50.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:20:50.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:20:50.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:50.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:20:50.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:20:50.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:50.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:20:50.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:20:50.503 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.504 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:50.504 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.504 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:20:50.504 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:50.504 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 147.62438994999999, 2024-09-17T21:20:50.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:20:50.504 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:50.504 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:50.504 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:50.504 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:50.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:50.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:50.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.505 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:50.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:50.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.505 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:50.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:20:50.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:20:50.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:50.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:20:50.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:20:50.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:50.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:20:50.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:20:50.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:50.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:20:50.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:20:50.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.507 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:50.507 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:20:50.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:20:50.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.507 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:50.507 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:20:50.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:20:50.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.507 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:50.507 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:20:50.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:20:50.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.507 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:50.508 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:20:50.508 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:20:50.508 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.508 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:50.508 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.508 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:20:50.508 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:50.508 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 147.61461054, 2024-09-17T21:20:50.508 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:20:50.508 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:50.508 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:50.508 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:50.509 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:50.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:50.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:50.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:50.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:50.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:50.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:20:50.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:20:50.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:50.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:20:50.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:20:50.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:50.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:20:50.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:20:50.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:50.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:20:50.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:20:50.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:50.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:20:50.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:20:50.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:50.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:20:50.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:20:50.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:50.512 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:20:50.512 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:20:50.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.512 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:50.512 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:20:50.512 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:20:50.512 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.512 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:50.512 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.513 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:20:50.513 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:50.513 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 147.60588813699999, 2024-09-17T21:20:50.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:20:50.513 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:50.513 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:50.513 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:50.513 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:50.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.513 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:50.513 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:50.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.514 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:50.514 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:50.514 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.514 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:50.514 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:20:50.514 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:20:50.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.514 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:50.514 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:20:50.514 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:20:50.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.515 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:50.515 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:20:50.515 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:20:50.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.515 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:50.515 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:20:50.515 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:20:50.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.515 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:50.515 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:20:50.515 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:20:50.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.516 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:50.516 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:20:50.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:20:50.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.516 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:50.516 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:20:50.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:20:50.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.516 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:50.516 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:20:50.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:20:50.516 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.517 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:50.517 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.517 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:20:50.517 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:50.517 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 134.220797573, 2024-09-17T21:20:50.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:20:50.517 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:50.517 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:50.517 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:50.517 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:50.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:50.518 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:50.518 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.518 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:50.518 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:50.518 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.518 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:50.518 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:20:50.518 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:20:50.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.518 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:50.519 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:20:50.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:20:50.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.519 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:50.519 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:20:50.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:20:50.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.519 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:50.519 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:20:50.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:20:50.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.519 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:50.520 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:20:50.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:20:50.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.520 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:50.520 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:20:50.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:20:50.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.520 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:50.520 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:20:50.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:20:50.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.520 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:50.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:20:50.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:20:50.521 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.521 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:50.521 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.521 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1780 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'195, mlcod=69'190)", 2024-09-17T21:20:50.521 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:50.521 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.242941969, 2024-09-17T21:20:50.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075761769999999999, 2024-09-17T21:20:50.521 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:50.521 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:50.521 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:50.521 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:50.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.522 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:50.522 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:50.522 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.522 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.522 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:50.522 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:50.522 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.522 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.522 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:50.522 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250375+0000", 2024-09-17T21:20:50.522 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:20:50.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.523 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.523 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:50.523 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250383+0000", 2024-09-17T21:20:50.523 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7870000000000002e-06 2024-09-17T21:20:50.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.523 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.523 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:50.523 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250402+0000", 2024-09-17T21:20:50.523 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9346000000000001e-05 2024-09-17T21:20:50.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.523 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.523 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:50.523 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250413+0000", 2024-09-17T21:20:50.523 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.039e-05 2024-09-17T21:20:50.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:50.524 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251353+0000", 2024-09-17T21:20:50.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000940091 2024-09-17T21:20:50.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:50.525 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251407+0000", 2024-09-17T21:20:50.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4701999999999999e-05 2024-09-17T21:20:50.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.528 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:50.528 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257829+0000", 2024-09-17T21:20:50.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064212779999999999 2024-09-17T21:20:50.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.528 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:50.528 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257950+0000", 2024-09-17T21:20:50.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121623 2024-09-17T21:20:50.529 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.529 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:50.529 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.529 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.529 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1781 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'196, mlcod=69'190)", 2024-09-17T21:20:50.529 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:50.529 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.241859018, 2024-09-17T21:20:50.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065632939999999999, 2024-09-17T21:20:50.529 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:50.529 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:50.529 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:50.529 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:50.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.530 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:50.530 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:50.530 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.530 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:50.530 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:50.530 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.530 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:50.530 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251459+0000", 2024-09-17T21:20:50.530 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.624e-06 2024-09-17T21:20:50.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:50.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251471+0000", 2024-09-17T21:20:50.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2636000000000001e-05 2024-09-17T21:20:50.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:50.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251498+0000", 2024-09-17T21:20:50.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:20:50.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:50.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251515+0000", 2024-09-17T21:20:50.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6939000000000002e-05 2024-09-17T21:20:50.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:50.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252938+0000", 2024-09-17T21:20:50.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001423082 2024-09-17T21:20:50.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:50.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252999+0000", 2024-09-17T21:20:50.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1138999999999999e-05 2024-09-17T21:20:50.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:50.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257955+0000", 2024-09-17T21:20:50.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049559640000000002 2024-09-17T21:20:50.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:50.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.258020+0000", 2024-09-17T21:20:50.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5504000000000002e-05 2024-09-17T21:20:50.533 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.534 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:50.534 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.534 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1788 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'198, mlcod=69'191)", 2024-09-17T21:20:50.534 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:50.534 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.239833528999995, 2024-09-17T21:20:50.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073690370000000002, 2024-09-17T21:20:50.534 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:50.534 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:50.534 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:50.534 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:50.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:50.535 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:50.535 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.535 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:50.535 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:50.535 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.535 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:50.535 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253484+0000", 2024-09-17T21:20:50.535 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2470000000000001e-06 2024-09-17T21:20:50.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.536 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:50.536 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253492+0000", 2024-09-17T21:20:50.536 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7159999999999992e-06 2024-09-17T21:20:50.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.536 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:50.536 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253509+0000", 2024-09-17T21:20:50.536 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.666e-05 2024-09-17T21:20:50.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.536 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:50.536 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253522+0000", 2024-09-17T21:20:50.536 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2643000000000001e-05 2024-09-17T21:20:50.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:50.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254853+0000", 2024-09-17T21:20:50.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331238 2024-09-17T21:20:50.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:50.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254885+0000", 2024-09-17T21:20:50.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1562e-05 2024-09-17T21:20:50.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:50.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260788+0000", 2024-09-17T21:20:50.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059030749999999998 2024-09-17T21:20:50.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:50.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260852+0000", 2024-09-17T21:20:50.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3895999999999999e-05 2024-09-17T21:20:50.538 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.538 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:50.538 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.538 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5746 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'477, mlcod=71'473)", 2024-09-17T21:20:50.538 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:50.538 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 94.234405971000001, 2024-09-17T21:20:50.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074182709999999997, 2024-09-17T21:20:50.539 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:50.539 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:50.539 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:50.539 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:50.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:50.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:50.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:50.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:50.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:50.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258912+0000", 2024-09-17T21:20:50.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:20:50.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:50.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258926+0000", 2024-09-17T21:20:50.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4249e-05 2024-09-17T21:20:50.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.541 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:50.541 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258946+0000", 2024-09-17T21:20:50.541 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9879999999999999e-05 2024-09-17T21:20:50.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.541 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:50.541 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258956+0000", 2024-09-17T21:20:50.541 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:20:50.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.541 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:50.541 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259380+0000", 2024-09-17T21:20:50.541 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042337599999999999 2024-09-17T21:20:50.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.542 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:50.542 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259412+0000", 2024-09-17T21:20:50.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:20:50.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.542 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:50.542 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266191+0000", 2024-09-17T21:20:50.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067793269999999999 2024-09-17T21:20:50.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.542 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:50.542 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266328+0000", 2024-09-17T21:20:50.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013690999999999999 2024-09-17T21:20:50.543 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.543 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:50.543 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.543 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5747 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'478, mlcod=71'473)", 2024-09-17T21:20:50.543 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:50.543 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 94.233699622000003, 2024-09-17T21:20:50.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068779440000000004, 2024-09-17T21:20:50.543 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:50.543 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:50.543 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:50.543 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:50.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.544 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:50.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:50.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.544 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:50.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:50.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.544 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:50.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259618+0000", 2024-09-17T21:20:50.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6199999999999999e-06 2024-09-17T21:20:50.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:50.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259630+0000", 2024-09-17T21:20:50.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1884e-05 2024-09-17T21:20:50.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:50.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259657+0000", 2024-09-17T21:20:50.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7339e-05 2024-09-17T21:20:50.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:50.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259677+0000", 2024-09-17T21:20:50.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0156000000000001e-05 2024-09-17T21:20:50.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:50.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260521+0000", 2024-09-17T21:20:50.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084312899999999997 2024-09-17T21:20:50.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:50.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260585+0000", 2024-09-17T21:20:50.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4558000000000001e-05 2024-09-17T21:20:50.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:50.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266336+0000", 2024-09-17T21:20:50.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057505020000000002 2024-09-17T21:20:50.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:50.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266494+0000", 2024-09-17T21:20:50.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158756 2024-09-17T21:20:50.547 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.547 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:50.547 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.547 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5750 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'479, mlcod=71'473)", 2024-09-17T21:20:50.547 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:50.548 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 94.232987429000005, 2024-09-17T21:20:50.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006302874, 2024-09-17T21:20:50.548 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:50.548 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:50.548 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:50.548 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:50.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:50.548 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:50.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:50.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:50.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:50.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260330+0000", 2024-09-17T21:20:50.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:20:50.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:50.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260343+0000", 2024-09-17T21:20:50.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2862e-05 2024-09-17T21:20:50.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:50.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260367+0000", 2024-09-17T21:20:50.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3723e-05 2024-09-17T21:20:50.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:50.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260386+0000", 2024-09-17T21:20:50.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8879999999999999e-05 2024-09-17T21:20:50.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:50.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261579+0000", 2024-09-17T21:20:50.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001193431 2024-09-17T21:20:50.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:50.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261608+0000", 2024-09-17T21:20:50.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8858999999999999e-05 2024-09-17T21:20:50.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:50.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266505+0000", 2024-09-17T21:20:50.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004896999 2024-09-17T21:20:50.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:50.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266631+0000", 2024-09-17T21:20:50.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012655300000000001 2024-09-17T21:20:50.552 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.552 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:50.552 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.552 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5751 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'480, mlcod=71'473)", 2024-09-17T21:20:50.552 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:50.552 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 94.232280216999996, 2024-09-17T21:20:50.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075756299999999999, 2024-09-17T21:20:50.552 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:50.552 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:50.552 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:50.552 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:50.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:50.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:50.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:50.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:50.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:50.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261037+0000", 2024-09-17T21:20:50.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:20:50.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:50.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261049+0000", 2024-09-17T21:20:50.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1063999999999999e-05 2024-09-17T21:20:50.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:50.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261072+0000", 2024-09-17T21:20:50.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3102e-05 2024-09-17T21:20:50.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:50.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261133+0000", 2024-09-17T21:20:50.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1276000000000001e-05 2024-09-17T21:20:50.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:50.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:20:50.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015334319999999999 2024-09-17T21:20:50.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:50.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262702+0000", 2024-09-17T21:20:50.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5281999999999999e-05 2024-09-17T21:20:50.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:50.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268527+0000", 2024-09-17T21:20:50.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058250389999999997 2024-09-17T21:20:50.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:50.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268611+0000", 2024-09-17T21:20:50.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4724999999999999e-05 2024-09-17T21:20:50.556 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.556 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:50.556 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.556 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5756 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'481, mlcod=71'473)", 2024-09-17T21:20:50.557 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:50.557 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 94.231419360999993, 2024-09-17T21:20:50.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006772386, 2024-09-17T21:20:50.557 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:50.557 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:50.557 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:50.557 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:50.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:50.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:50.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:50.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:50.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:50.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261899+0000", 2024-09-17T21:20:50.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.903e-06 2024-09-17T21:20:50.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:50.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261912+0000", 2024-09-17T21:20:50.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-05 2024-09-17T21:20:50.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:50.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261938+0000", 2024-09-17T21:20:50.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6098999999999999e-05 2024-09-17T21:20:50.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:50.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261955+0000", 2024-09-17T21:20:50.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.696e-05 2024-09-17T21:20:50.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:50.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263845+0000", 2024-09-17T21:20:50.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001890187 2024-09-17T21:20:50.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.560 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:50.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263907+0000", 2024-09-17T21:20:50.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1600999999999995e-05 2024-09-17T21:20:50.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.560 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:50.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268615+0000", 2024-09-17T21:20:50.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047086070000000001 2024-09-17T21:20:50.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.560 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:50.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268669+0000", 2024-09-17T21:20:50.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3588999999999998e-05 2024-09-17T21:20:50.561 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.561 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:50.561 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.561 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4978.0:598 3.9 e80/23 3:9701cc17:::rbd_data.1152724ff2a3.0000000000000019:head v 80'202, mlcod=80'198)", 2024-09-17T21:20:50.561 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:50.561 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.729844978000003, 2024-09-17T21:20:50.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066473340000000004, 2024-09-17T21:20:50.561 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:50.561 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:50.561 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:50.561 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:50.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:50.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:50.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:50.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:50.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:50.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763472+0000", 2024-09-17T21:20:50.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6899999999999997e-07 2024-09-17T21:20:50.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:50.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763478+0000", 2024-09-17T21:20:50.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:20:50.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:50.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763489+0000", 2024-09-17T21:20:50.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0628999999999999e-05 2024-09-17T21:20:50.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:50.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763495+0000", 2024-09-17T21:20:50.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6649999999999997e-06 2024-09-17T21:20:50.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:50.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763907+0000", 2024-09-17T21:20:50.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041163899999999999 2024-09-17T21:20:50.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:50.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763976+0000", 2024-09-17T21:20:50.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8534999999999993e-05 2024-09-17T21:20:50.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.565 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:50.565 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.769900+0000", 2024-09-17T21:20:50.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059243270000000001 2024-09-17T21:20:50.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.565 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:50.565 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.770118+0000", 2024-09-17T21:20:50.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021842500000000001 2024-09-17T21:20:50.565 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.565 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:50.565 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.565 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:468 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'94, mlcod=89'87)", 2024-09-17T21:20:50.566 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:50.566 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.807099679000004, 2024-09-17T21:20:50.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066528790000000004, 2024-09-17T21:20:50.566 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:50.566 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:50.566 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:50.566 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:50.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:50.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:50.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:50.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:50.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.567 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:50.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686218+0000", 2024-09-17T21:20:50.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3060000000000001e-06 2024-09-17T21:20:50.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.567 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:50.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686256+0000", 2024-09-17T21:20:50.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8333e-05 2024-09-17T21:20:50.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.568 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:50.568 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686272+0000", 2024-09-17T21:20:50.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:20:50.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.568 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.568 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:50.568 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686283+0000", 2024-09-17T21:20:50.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0903e-05 2024-09-17T21:20:50.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.568 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.568 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:50.568 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686682+0000", 2024-09-17T21:20:50.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039935099999999999 2024-09-17T21:20:50.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.569 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.569 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:50.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686785+0000", 2024-09-17T21:20:50.569 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010275199999999999 2024-09-17T21:20:50.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.569 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.569 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:50.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692816+0000", 2024-09-17T21:20:50.569 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060317579999999999 2024-09-17T21:20:50.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.569 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.569 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:50.570 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692869+0000", 2024-09-17T21:20:50.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2775999999999998e-05 2024-09-17T21:20:50.570 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.570 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:50.570 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.570 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:755 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'127, mlcod=89'122)", 2024-09-17T21:20:50.570 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:50.570 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.670644464999995, 2024-09-17T21:20:50.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084014080000000008, 2024-09-17T21:20:50.570 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:50.570 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:50.570 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:50.571 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:50.571 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.571 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:50.571 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:50.571 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.571 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.571 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:50.571 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:50.571 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.571 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.571 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:50.572 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:50.572 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:20:50.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.572 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:50.572 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822683+0000", 2024-09-17T21:20:50.572 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0674e-05 2024-09-17T21:20:50.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.572 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:50.572 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822698+0000", 2024-09-17T21:20:50.572 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5189e-05 2024-09-17T21:20:50.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.573 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.573 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:50.573 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822708+0000", 2024-09-17T21:20:50.573 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9119999999999993e-06 2024-09-17T21:20:50.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.573 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.573 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:50.573 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822748+0000", 2024-09-17T21:20:50.573 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9727000000000001e-05 2024-09-17T21:20:50.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.573 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.573 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:50.573 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822759+0000", 2024-09-17T21:20:50.573 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0865e-05 2024-09-17T21:20:50.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.574 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:50.574 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831034+0000", 2024-09-17T21:20:50.574 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082750710000000002 2024-09-17T21:20:50.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.574 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:50.574 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831073+0000", 2024-09-17T21:20:50.574 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9063999999999997e-05 2024-09-17T21:20:50.574 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.574 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:50.574 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.575 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:756 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'128, mlcod=89'124)", 2024-09-17T21:20:50.575 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:50.575 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 70.668721833000006, 2024-09-17T21:20:50.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0076300270000000002, 2024-09-17T21:20:50.575 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:50.575 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:50.575 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:50.575 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:50.575 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.575 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:50.575 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:50.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.576 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.576 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:50.576 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:50.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.576 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.576 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:50.576 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824595+0000", 2024-09-17T21:20:50.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-17T21:20:50.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.576 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.576 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:50.576 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824600+0000", 2024-09-17T21:20:50.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:20:50.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.577 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:50.577 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824612+0000", 2024-09-17T21:20:50.577 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.181e-05 2024-09-17T21:20:50.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.577 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:50.577 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824619+0000", 2024-09-17T21:20:50.577 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:20:50.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.578 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.578 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:50.578 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824671+0000", 2024-09-17T21:20:50.578 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2262999999999998e-05 2024-09-17T21:20:50.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.578 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.578 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:50.578 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824695+0000", 2024-09-17T21:20:50.578 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3883000000000002e-05 2024-09-17T21:20:50.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.579 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:50.579 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832185+0000", 2024-09-17T21:20:50.579 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007490048 2024-09-17T21:20:50.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.579 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:50.579 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832224+0000", 2024-09-17T21:20:50.579 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8957e-05 2024-09-17T21:20:50.579 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.579 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:50.579 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.580 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2617 3.d e90/23 3:b0143718:::rbd_data.1152a8ae2019.0000000000000016:head v 90'249, mlcod=90'246)", 2024-09-17T21:20:50.580 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:50.580 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 69.648870833000004, 2024-09-17T21:20:50.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063715559999999996, 2024-09-17T21:20:50.580 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:50.580 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:50.580 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:50.580 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:50.580 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.580 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:50.580 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:50.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.581 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:50.581 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:50.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.581 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:50.581 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844455+0000", 2024-09-17T21:20:50.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4259999999999992e-06 2024-09-17T21:20:50.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.581 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:50.581 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844470+0000", 2024-09-17T21:20:50.582 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5386000000000001e-05 2024-09-17T21:20:50.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.582 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.582 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:50.582 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844499+0000", 2024-09-17T21:20:50.582 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.883e-05 2024-09-17T21:20:50.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.582 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.582 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:50.582 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844518+0000", 2024-09-17T21:20:50.582 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8830000000000001e-05 2024-09-17T21:20:50.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.582 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.582 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:50.583 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844649+0000", 2024-09-17T21:20:50.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013143399999999999 2024-09-17T21:20:50.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.583 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:50.583 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844702+0000", 2024-09-17T21:20:50.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2876e-05 2024-09-17T21:20:50.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.583 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:50.583 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850696+0000", 2024-09-17T21:20:50.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059943729999999999 2024-09-17T21:20:50.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.584 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:50.584 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850817+0000", 2024-09-17T21:20:50.584 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120401 2024-09-17T21:20:50.584 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.584 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:50.584 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.584 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.584 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2635 3.d e90/23 3:b3d1e920:::rbd_data.1152a8ae2019.000000000000001f:head v 90'252, mlcod=90'246)", 2024-09-17T21:20:50.584 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:50.584 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 69.646162763000007, 2024-09-17T21:20:50.584 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0077265190000000003, 2024-09-17T21:20:50.584 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:50.584 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:50.585 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:50.585 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:50.585 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.585 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:50.585 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:50.585 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.585 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.585 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:50.586 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:50.586 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:50.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.586 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.586 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:50.586 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847155+0000", 2024-09-17T21:20:50.586 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.544e-06 2024-09-17T21:20:50.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.586 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.586 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:50.586 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847165+0000", 2024-09-17T21:20:50.586 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.043e-05 2024-09-17T21:20:50.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.587 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:50.587 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847189+0000", 2024-09-17T21:20:50.587 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3659000000000002e-05 2024-09-17T21:20:50.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.587 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:50.587 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847210+0000", 2024-09-17T21:20:50.587 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1163e-05 2024-09-17T21:20:50.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.587 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:50.587 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848856+0000", 2024-09-17T21:20:50.587 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001645521 2024-09-17T21:20:50.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.588 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:50.588 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848905+0000", 2024-09-17T21:20:50.588 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9048999999999997e-05 2024-09-17T21:20:50.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.588 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:50.588 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854770+0000", 2024-09-17T21:20:50.588 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058652879999999998 2024-09-17T21:20:50.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:50.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:50.588 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:50.588 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854880+0000", 2024-09-17T21:20:50.589 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010986500000000001 2024-09-17T21:20:50.589 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.589 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:50.589 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.589 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:50.589 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:50.589 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:50.589 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:20:50.590 INFO:teuthology.orchestra.run.smithi184.stdout:5424 write 0xb53f800 thru 0xb546d87 (0x7588 bytes) 2024-09-17T21:20:50.591 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:20:50.592 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:50.592 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:50.592 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:50.593 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:50.593 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:50.584+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:50.618 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:50.637 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:50.637 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:50.655 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:50.655 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:50.672 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:50.672 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:50.674 INFO:teuthology.orchestra.run.smithi184.stdout:5425 trunc from 0xe849400 to 0x1525400 2024-09-17T21:20:50.691 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:50.686+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:50.734 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:50.735 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:50.752 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:20:50.792 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:50.784+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:50.893 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:50.888+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:50.922 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:50.923 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:50.937 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:20:50.993 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:50.988+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:51.021 INFO:teuthology.orchestra.run.smithi184.stdout:5426 trunc from 0x1525400 to 0x7fcb600 2024-09-17T21:20:51.052 INFO:teuthology.orchestra.run.smithi184.stdout:5430 write 0x894b200 thru 0x895388f (0x8690 bytes) 2024-09-17T21:20:51.054 INFO:teuthology.orchestra.run.smithi184.stdout:5431 punch from 0x6230e00 to 0x6239c00, (0x8e00 bytes) 2024-09-17T21:20:51.056 INFO:teuthology.orchestra.run.smithi184.stdout:5432 read 0x43af600 thru 0x43b64bc (0x6ebd bytes) 2024-09-17T21:20:51.056 INFO:teuthology.orchestra.run.smithi184.stdout:5434 punch from 0x236e400 to 0x2376e00, (0x8a00 bytes) 2024-09-17T21:20:51.065 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:51.065 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:51.085 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:20:51.094 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:51.088+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:51.124 INFO:teuthology.orchestra.run.smithi184.stdout:5435 write 0xa10aa00 thru 0xa113343 (0x8944 bytes) 2024-09-17T21:20:51.125 INFO:teuthology.orchestra.run.smithi184.stdout:5438 write 0x3e88000 thru 0x3e96a40 (0xea41 bytes) 2024-09-17T21:20:51.125 INFO:teuthology.orchestra.run.smithi184.stdout:5439 punch from 0x4822400 to 0x482c800, (0xa400 bytes) 2024-09-17T21:20:51.129 INFO:teuthology.orchestra.run.smithi184.stdout:5441 trunc from 0xa113400 to 0xa7b2600 2024-09-17T21:20:51.195 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:51.190+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:51.224 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:51.224 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:51.238 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:20:51.290 INFO:teuthology.orchestra.run.smithi184.stdout:5443 write 0xb111e00 thru 0xb11557e (0x377f bytes) 2024-09-17T21:20:51.295 INFO:teuthology.orchestra.run.smithi184.stdout:5444 punch from 0x156ce00 to 0x157be00, (0xf000 bytes) 2024-09-17T21:20:51.295 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:51.288+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:51.297 INFO:teuthology.orchestra.run.smithi184.stdout:5445 write 0x60f000 thru 0x61a605 (0xb606 bytes) 2024-09-17T21:20:51.299 INFO:teuthology.orchestra.run.smithi184.stdout:5446 read 0x4b1f400 thru 0x4b29463 (0xa064 bytes) 2024-09-17T21:20:51.299 INFO:teuthology.orchestra.run.smithi184.stdout:5447 write 0x85cc400 thru 0x85d31dd (0x6dde bytes) 2024-09-17T21:20:51.299 INFO:teuthology.orchestra.run.smithi184.stdout:5450 read 0x8467c00 thru 0x846fe7d (0x827e bytes) 2024-09-17T21:20:51.300 INFO:teuthology.orchestra.run.smithi184.stdout:5453 read 0x5553c00 thru 0x555859f (0x49a0 bytes) 2024-09-17T21:20:51.396 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:51.388+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:51.445 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:51.445 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:51.461 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:20:51.486 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-17T21:20:51.511 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:51.506+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:51.555 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 0, 5] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:20:51.555 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:20:51.555 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T21:20:51.556 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 0.252500 2024-09-17T21:20:51.556 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 0.25250006657570023 2024-09-17T21:20:51.581 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:51.581 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:51.596 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:20:51.600 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:51.596+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:51.630 INFO:teuthology.orchestra.run.smithi184.stdout:5454 write 0xcc08600 thru 0xcc0d715 (0x5116 bytes) 2024-09-17T21:20:51.632 INFO:teuthology.orchestra.run.smithi184.stdout:5455 punch from 0x7862c00 to 0x7863e00, (0x1200 bytes) 2024-09-17T21:20:51.633 INFO:teuthology.orchestra.run.smithi184.stdout:5456 write 0x659e600 thru 0x65a0a7c (0x247d bytes) 2024-09-17T21:20:51.661 INFO:teuthology.orchestra.run.smithi184.stdout:5457 trunc from 0xcc0d800 to 0x2aba400 2024-09-17T21:20:51.698 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:51.690+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:51.783 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:51.783 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:51.796 INFO:teuthology.orchestra.run.smithi184.stdout:5458 punch from 0xfbda00 to 0xfbf800, (0x1e00 bytes) 2024-09-17T21:20:51.797 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:20:51.798 INFO:teuthology.orchestra.run.smithi184.stdout:5459 clone 40 order 20 su 32768 sc 9 2024-09-17T21:20:51.839 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:51.834+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:51.900 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:51.894+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:51.961 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:51.962 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:51.977 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:20:52.019 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:52.014+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:52.102 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:52.096+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:52.143 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:52.143 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:52.157 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:20:52.203 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:52.196+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:52.280 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:52.280 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:52.293 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:20:52.304 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:52.298+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:52.405 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:52.398+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:52.458 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:52.458 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:52.472 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:20:52.501 INFO:teuthology.orchestra.run.smithi083.stderr:set osd.0 primary-affinity to 0.2525 (8165472) 2024-09-17T21:20:52.507 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:52.500+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:52.594 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:52.594 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:52.606 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:52.598+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:52.607 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:20:52.696 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:20:52.707 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:52.702+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:52.725 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:52.726 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:52.740 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:20:52.807 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:52.802+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:52.845 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:52.846 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:52.862 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:52.862 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:52.878 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:52.879 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:52.899 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:52.899 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:52.901 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:52.902 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:52.902 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:52.902 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:52.902 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:52.908 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:52.900+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:52.916 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:20:52.920 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:52.920 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:52.940 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:52.940 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:53.008 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:53.004+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:53.038 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:53.038 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:53.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:53.038 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:53.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.038 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:20:53.038 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:53.038 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 133.22466262500001, 2024-09-17T21:20:53.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:20:53.038 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.039 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.039 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:53.039 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:53.039 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:20:53.039 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:20:53.039 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:20:53.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.039 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.039 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:53.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:53.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:20:53.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:20:53.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:20:53.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:20:53.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:20:53.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:20:53.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:20:53.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:20:53.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:20:53.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:20:53.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:53.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:20:53.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:20:53.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:20:53.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:20:53.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:20:53.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:20:53.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:20:53.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:20:53.043 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.043 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.043 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.043 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:20:53.043 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:53.043 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 119.669560839, 2024-09-17T21:20:53.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:20:53.044 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.044 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.044 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.044 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:53.044 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:20:53.044 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:53.044 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:20:53.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.044 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:53.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:53.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:20:53.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:20:53.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:20:53.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:20:53.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:20:53.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:20:53.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:20:53.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:20:53.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:20:53.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:20:53.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:53.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:20:53.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:20:53.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:20:53.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:20:53.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:20:53.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:20:53.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:53.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:20:53.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:20:53.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:53.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:20:53.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:20:53.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.049 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:53.049 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:20:53.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:20:53.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.049 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.049 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:20:53.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:20:53.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.049 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.049 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:20:53.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:20:53.050 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.050 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.050 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.050 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:793 3.9 3:95f63599:::rbd_data.1152eb799094.0000000000000007:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:20:53.050 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:20:53.050 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 36.602504684000003, 2024-09-17T21:20:53.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88245766400000003, 2024-09-17T21:20:53.050 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.050 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.050 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.050 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:53.051 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:20:53.051 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:20:53.051 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 793 2024-09-17T21:20:53.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.051 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:20:53.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:20:53.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431941+0000", 2024-09-17T21:20:53.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.864e-06 2024-09-17T21:20:53.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431953+0000", 2024-09-17T21:20:53.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1486e-05 2024-09-17T21:20:53.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431984+0000", 2024-09-17T21:20:53.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1075999999999999e-05 2024-09-17T21:20:53.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431999+0000", 2024-09-17T21:20:53.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.503e-05 2024-09-17T21:20:53.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432095+0000", 2024-09-17T21:20:53.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5698000000000004e-05 2024-09-17T21:20:53.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:53.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432109+0000", 2024-09-17T21:20:53.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4188e-05 2024-09-17T21:20:53.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.309642+0000", 2024-09-17T21:20:53.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87753353000000001 2024-09-17T21:20:53.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310104+0000", 2024-09-17T21:20:53.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046103600000000001 2024-09-17T21:20:53.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.055 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:53.055 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310630+0000", 2024-09-17T21:20:53.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052682800000000002 2024-09-17T21:20:53.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.055 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:53.055 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313100+0000", 2024-09-17T21:20:53.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024693369999999998 2024-09-17T21:20:53.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.055 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:53.055 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313935+0000", 2024-09-17T21:20:53.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083540100000000005 2024-09-17T21:20:53.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314086+0000", 2024-09-17T21:20:53.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015053399999999999 2024-09-17T21:20:53.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314396+0000", 2024-09-17T21:20:53.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031065599999999997 2024-09-17T21:20:53.056 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.056 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.056 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.057 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:800 3.19 3:9dc17f12:::rbd_data.1152eb799094.000000000000000e:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:20:53.057 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:20:53.057 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 36.602257438999999, 2024-09-17T21:20:53.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88154663799999999, 2024-09-17T21:20:53.057 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.057 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.057 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.057 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:53.057 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:20:53.057 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:20:53.057 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 800 2024-09-17T21:20:53.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.057 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:20:53.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:20:53.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432187+0000", 2024-09-17T21:20:53.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4100000000000001e-06 2024-09-17T21:20:53.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432194+0000", 2024-09-17T21:20:53.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0389999999999999e-06 2024-09-17T21:20:53.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432216+0000", 2024-09-17T21:20:53.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1614e-05 2024-09-17T21:20:53.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.060 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432232+0000", 2024-09-17T21:20:53.060 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5739e-05 2024-09-17T21:20:53.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.060 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.060 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.060 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432405+0000", 2024-09-17T21:20:53.060 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017355300000000001 2024-09-17T21:20:53.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.060 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.060 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:53.060 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432432+0000", 2024-09-17T21:20:53.060 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6979e-05 2024-09-17T21:20:53.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.307767+0000", 2024-09-17T21:20:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87533472899999998 2024-09-17T21:20:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308192+0000", 2024-09-17T21:20:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042485999999999999 2024-09-17T21:20:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308727+0000", 2024-09-17T21:20:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00053550400000000001 2024-09-17T21:20:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310347+0000", 2024-09-17T21:20:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016198670000000001 2024-09-17T21:20:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313424+0000", 2024-09-17T21:20:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030764640000000001 2024-09-17T21:20:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313510+0000", 2024-09-17T21:20:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5938000000000003e-05 2024-09-17T21:20:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313733+0000", 2024-09-17T21:20:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022294200000000001 2024-09-17T21:20:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:805 3.19 3:9cab2ed6:::rbd_data.1152eb799094.0000000000000013:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:20:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:20:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 36.602053419999997, 2024-09-17T21:20:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88274355699999996, 2024-09-17T21:20:53.064 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.064 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.064 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.064 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:53.064 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:20:53.064 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:20:53.064 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 805 2024-09-17T21:20:53.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.064 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.064 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.064 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:20:53.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.065 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.065 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.065 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:20:53.065 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.065 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.065 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.065 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432391+0000", 2024-09-17T21:20:53.065 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3e-06 2024-09-17T21:20:53.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.065 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.065 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.065 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432398+0000", 2024-09-17T21:20:53.065 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4489999999999996e-06 2024-09-17T21:20:53.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.066 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.066 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432415+0000", 2024-09-17T21:20:53.066 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7750000000000001e-05 2024-09-17T21:20:53.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.066 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.066 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432430+0000", 2024-09-17T21:20:53.066 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.449e-05 2024-09-17T21:20:53.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.066 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.066 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432635+0000", 2024-09-17T21:20:53.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205031 2024-09-17T21:20:53.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:53.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432657+0000", 2024-09-17T21:20:53.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2064000000000001e-05 2024-09-17T21:20:53.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310444+0000", 2024-09-17T21:20:53.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87778699199999999 2024-09-17T21:20:53.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.068 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310849+0000", 2024-09-17T21:20:53.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00040497800000000001 2024-09-17T21:20:53.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.068 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:53.068 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.311396+0000", 2024-09-17T21:20:53.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00054729099999999999 2024-09-17T21:20:53.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.068 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:53.068 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313847+0000", 2024-09-17T21:20:53.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024504940000000001 2024-09-17T21:20:53.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.068 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:53.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314879+0000", 2024-09-17T21:20:53.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001031817 2024-09-17T21:20:53.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.069 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.069 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314964+0000", 2024-09-17T21:20:53.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5685000000000004e-05 2024-09-17T21:20:53.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.069 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.069 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.315133+0000", 2024-09-17T21:20:53.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000169216 2024-09-17T21:20:53.069 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.069 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.070 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.070 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.070 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12087 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 52224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:53.070 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:20:53.070 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.264964742, 2024-09-17T21:20:53.070 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087684460000000006, 2024-09-17T21:20:53.070 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.070 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.070 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:53.070 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:53.070 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:53.070 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:53.071 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12087 2024-09-17T21:20:53.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.071 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.071 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:20:53.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.071 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:20:53.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769480+0000", 2024-09-17T21:20:53.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:20:53.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769485+0000", 2024-09-17T21:20:53.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8880000000000002e-06 2024-09-17T21:20:53.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769509+0000", 2024-09-17T21:20:53.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3343e-05 2024-09-17T21:20:53.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769521+0000", 2024-09-17T21:20:53.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2497000000000001e-05 2024-09-17T21:20:53.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.856883+0000", 2024-09-17T21:20:53.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087362191000000006 2024-09-17T21:20:53.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857014+0000", 2024-09-17T21:20:53.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013090399999999999 2024-09-17T21:20:53.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857163+0000", 2024-09-17T21:20:53.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014864000000000001 2024-09-17T21:20:53.074 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.074 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.074 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.074 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12088 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 56320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:53.074 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:20:53.074 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.264881086999999, 2024-09-17T21:20:53.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087917772000000005, 2024-09-17T21:20:53.075 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.075 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.075 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:53.075 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:53.075 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:53.075 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:53.075 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12088 2024-09-17T21:20:53.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.075 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.075 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.075 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:20:53.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.076 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.076 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.076 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:20:53.076 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.076 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.076 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.076 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769569+0000", 2024-09-17T21:20:53.076 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:20:53.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.076 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.076 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.076 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769574+0000", 2024-09-17T21:20:53.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4659999999999998e-06 2024-09-17T21:20:53.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769586+0000", 2024-09-17T21:20:53.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2311e-05 2024-09-17T21:20:53.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769597+0000", 2024-09-17T21:20:53.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0548e-05 2024-09-17T21:20:53.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857217+0000", 2024-09-17T21:20:53.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087619896000000003 2024-09-17T21:20:53.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857335+0000", 2024-09-17T21:20:53.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118538 2024-09-17T21:20:53.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857480+0000", 2024-09-17T21:20:53.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014470999999999999 2024-09-17T21:20:53.078 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.078 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.078 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.079 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12089 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 60416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:53.079 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:20:53.079 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.264828388000002, 2024-09-17T21:20:53.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088293233999999998, 2024-09-17T21:20:53.079 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.079 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.079 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:53.079 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:53.079 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:53.079 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:53.079 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12089 2024-09-17T21:20:53.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.080 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:20:53.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:20:53.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769616+0000", 2024-09-17T21:20:53.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:20:53.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.081 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.081 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769620+0000", 2024-09-17T21:20:53.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5140000000000002e-06 2024-09-17T21:20:53.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.081 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.081 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769633+0000", 2024-09-17T21:20:53.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2849e-05 2024-09-17T21:20:53.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.081 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769644+0000", 2024-09-17T21:20:53.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1471000000000001e-05 2024-09-17T21:20:53.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.082 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857564+0000", 2024-09-17T21:20:53.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087919445999999998 2024-09-17T21:20:53.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.082 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857716+0000", 2024-09-17T21:20:53.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000152065 2024-09-17T21:20:53.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.082 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.083 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857908+0000", 2024-09-17T21:20:53.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019252599999999999 2024-09-17T21:20:53.083 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.083 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.083 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.083 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12090 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 64512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:53.083 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:20:53.083 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.264782003000001, 2024-09-17T21:20:53.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088582672000000001, 2024-09-17T21:20:53.083 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.083 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.083 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:53.084 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:53.084 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:53.084 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:53.084 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12090 2024-09-17T21:20:53.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.084 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.084 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:20:53.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:20:53.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769662+0000", 2024-09-17T21:20:53.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:20:53.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769683+0000", 2024-09-17T21:20:53.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0979e-05 2024-09-17T21:20:53.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769699+0000", 2024-09-17T21:20:53.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5446000000000001e-05 2024-09-17T21:20:53.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769708+0000", 2024-09-17T21:20:53.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1679999999999993e-06 2024-09-17T21:20:53.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857976+0000", 2024-09-17T21:20:53.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088267795999999996 2024-09-17T21:20:53.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.087 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.087 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858095+0000", 2024-09-17T21:20:53.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119147 2024-09-17T21:20:53.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.088 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.088 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858244+0000", 2024-09-17T21:20:53.088 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014917000000000001 2024-09-17T21:20:53.088 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.088 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.088 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.088 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12091 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 68608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:53.088 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:20:53.088 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.264710618999999, 2024-09-17T21:20:53.088 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088832752000000001, 2024-09-17T21:20:53.088 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.089 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.089 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:53.089 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:53.089 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:53.089 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:53.089 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12091 2024-09-17T21:20:53.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.089 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:20:53.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:20:53.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769734+0000", 2024-09-17T21:20:53.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:20:53.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769739+0000", 2024-09-17T21:20:53.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8080000000000004e-06 2024-09-17T21:20:53.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.091 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.091 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769764+0000", 2024-09-17T21:20:53.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4839999999999999e-05 2024-09-17T21:20:53.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.091 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.091 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769777+0000", 2024-09-17T21:20:53.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3037000000000001e-05 2024-09-17T21:20:53.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.091 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.091 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858295+0000", 2024-09-17T21:20:53.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088518475999999999 2024-09-17T21:20:53.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.092 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.092 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.092 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858419+0000", 2024-09-17T21:20:53.092 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012378700000000001 2024-09-17T21:20:53.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.092 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.092 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.092 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858565+0000", 2024-09-17T21:20:53.092 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014630099999999999 2024-09-17T21:20:53.092 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.092 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.092 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.093 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12092 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:53.093 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:20:53.093 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.264628295000001, 2024-09-17T21:20:53.093 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089191555000000006, 2024-09-17T21:20:53.093 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.093 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.093 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:53.093 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:53.093 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:53.093 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:53.093 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12092 2024-09-17T21:20:53.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.094 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:20:53.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:20:53.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769816+0000", 2024-09-17T21:20:53.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7199999999999997e-07 2024-09-17T21:20:53.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769822+0000", 2024-09-17T21:20:53.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:20:53.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.096 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769838+0000", 2024-09-17T21:20:53.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6674e-05 2024-09-17T21:20:53.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.096 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.096 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769851+0000", 2024-09-17T21:20:53.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2527999999999999e-05 2024-09-17T21:20:53.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.096 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.096 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858660+0000", 2024-09-17T21:20:53.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088808556999999996 2024-09-17T21:20:53.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.097 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858817+0000", 2024-09-17T21:20:53.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015730400000000001 2024-09-17T21:20:53.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.097 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859007+0000", 2024-09-17T21:20:53.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018974899999999999 2024-09-17T21:20:53.097 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.097 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.097 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.098 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12093 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:53.098 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:20:53.098 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.264520190999999, 2024-09-17T21:20:53.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089429710999999995, 2024-09-17T21:20:53.098 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.098 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.098 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:53.098 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:53.098 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:53.098 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:53.098 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12093 2024-09-17T21:20:53.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.098 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:20:53.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:20:53.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769924+0000", 2024-09-17T21:20:53.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.342e-06 2024-09-17T21:20:53.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769929+0000", 2024-09-17T21:20:53.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0350000000000001e-06 2024-09-17T21:20:53.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769946+0000", 2024-09-17T21:20:53.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6793999999999999e-05 2024-09-17T21:20:53.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769959+0000", 2024-09-17T21:20:53.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3062e-05 2024-09-17T21:20:53.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.101 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.101 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859075+0000", 2024-09-17T21:20:53.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089115696999999994 2024-09-17T21:20:53.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.101 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.101 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859202+0000", 2024-09-17T21:20:53.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012657 2024-09-17T21:20:53.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.102 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.102 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859353+0000", 2024-09-17T21:20:53.102 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151211 2024-09-17T21:20:53.102 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.102 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.102 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.102 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12094 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:53.102 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:20:53.102 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.264466015, 2024-09-17T21:20:53.102 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089695001999999996, 2024-09-17T21:20:53.102 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.102 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.103 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:53.103 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:53.103 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:53.103 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:53.103 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12094 2024-09-17T21:20:53.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.103 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:20:53.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:20:53.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769978+0000", 2024-09-17T21:20:53.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.142e-06 2024-09-17T21:20:53.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769983+0000", 2024-09-17T21:20:53.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.386e-06 2024-09-17T21:20:53.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769997+0000", 2024-09-17T21:20:53.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.416e-05 2024-09-17T21:20:53.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770012+0000", 2024-09-17T21:20:53.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5384999999999999e-05 2024-09-17T21:20:53.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859405+0000", 2024-09-17T21:20:53.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089392376999999995 2024-09-17T21:20:53.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.106 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.106 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859525+0000", 2024-09-17T21:20:53.106 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120555 2024-09-17T21:20:53.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.106 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.106 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859672+0000", 2024-09-17T21:20:53.106 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000146997 2024-09-17T21:20:53.106 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.106 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.106 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.107 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12095 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:53.107 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:20:53.107 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.264413527999999, 2024-09-17T21:20:53.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090073476, 2024-09-17T21:20:53.107 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.107 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.107 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:53.107 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:53.107 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:53.107 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:53.107 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12095 2024-09-17T21:20:53.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.108 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:20:53.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:20:53.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770031+0000", 2024-09-17T21:20:53.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1699999999999997e-07 2024-09-17T21:20:53.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770036+0000", 2024-09-17T21:20:53.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.023e-06 2024-09-17T21:20:53.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770051+0000", 2024-09-17T21:20:53.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4953999999999999e-05 2024-09-17T21:20:53.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.110 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770063+0000", 2024-09-17T21:20:53.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:20:53.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.110 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.110 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859756+0000", 2024-09-17T21:20:53.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089692980000000005 2024-09-17T21:20:53.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.110 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.110 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859891+0000", 2024-09-17T21:20:53.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013459499999999999 2024-09-17T21:20:53.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860103+0000", 2024-09-17T21:20:53.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021274400000000001 2024-09-17T21:20:53.111 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.111 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.111 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.111 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12096 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:53.111 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:20:53.111 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.264363843000002, 2024-09-17T21:20:53.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090358274000000002, 2024-09-17T21:20:53.111 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.111 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.112 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:53.112 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:53.112 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:53.112 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:53.112 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12096 2024-09-17T21:20:53.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.112 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:20:53.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:20:53.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770080+0000", 2024-09-17T21:20:53.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9400000000000004e-07 2024-09-17T21:20:53.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770085+0000", 2024-09-17T21:20:53.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2169999999999996e-06 2024-09-17T21:20:53.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770100+0000", 2024-09-17T21:20:53.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5077e-05 2024-09-17T21:20:53.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770112+0000", 2024-09-17T21:20:53.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1951e-05 2024-09-17T21:20:53.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860160+0000", 2024-09-17T21:20:53.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090048413999999993 2024-09-17T21:20:53.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860285+0000", 2024-09-17T21:20:53.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012466699999999999 2024-09-17T21:20:53.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860438+0000", 2024-09-17T21:20:53.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015305399999999999 2024-09-17T21:20:53.115 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.115 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.115 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.116 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12097 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:53.116 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:20:53.116 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.264316096000002, 2024-09-17T21:20:53.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090702375000000002, 2024-09-17T21:20:53.116 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.116 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.116 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:53.116 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:53.116 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:53.116 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:53.116 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12097 2024-09-17T21:20:53.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.116 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:20:53.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:20:53.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770128+0000", 2024-09-17T21:20:53.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.04e-06 2024-09-17T21:20:53.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770133+0000", 2024-09-17T21:20:53.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8999999999999997e-06 2024-09-17T21:20:53.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770152+0000", 2024-09-17T21:20:53.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8890999999999999e-05 2024-09-17T21:20:53.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.119 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770164+0000", 2024-09-17T21:20:53.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1769e-05 2024-09-17T21:20:53.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.119 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.119 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860511+0000", 2024-09-17T21:20:53.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090347527999999996 2024-09-17T21:20:53.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.119 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.119 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860652+0000", 2024-09-17T21:20:53.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140894 2024-09-17T21:20:53.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860830+0000", 2024-09-17T21:20:53.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017735299999999999 2024-09-17T21:20:53.120 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.120 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.120 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.120 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12098 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:53.120 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:20:53.120 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.264262628000001, 2024-09-17T21:20:53.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090997262999999995, 2024-09-17T21:20:53.120 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.120 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.120 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:53.121 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:53.121 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:53.121 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:53.121 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12098 2024-09-17T21:20:53.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.121 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:20:53.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:20:53.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775465+0000", 2024-09-17T21:20:53.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0052843350000000002 2024-09-17T21:20:53.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775476+0000", 2024-09-17T21:20:53.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0797e-05 2024-09-17T21:20:53.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.123 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775506+0000", 2024-09-17T21:20:53.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9709999999999998e-05 2024-09-17T21:20:53.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.123 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.123 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775530+0000", 2024-09-17T21:20:53.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4528999999999998e-05 2024-09-17T21:20:53.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.123 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.123 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860894+0000", 2024-09-17T21:20:53.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.085364195000000004 2024-09-17T21:20:53.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.124 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.124 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861019+0000", 2024-09-17T21:20:53.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012467399999999999 2024-09-17T21:20:53.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.124 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.124 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861178+0000", 2024-09-17T21:20:53.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159023 2024-09-17T21:20:53.124 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.124 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.124 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.125 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12104 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:53.125 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:20:53.125 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.258561519000001, 2024-09-17T21:20:53.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087969628999999994, 2024-09-17T21:20:53.125 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.125 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.125 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:53.125 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:53.125 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:53.125 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:53.125 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12104 2024-09-17T21:20:53.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.125 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:20:53.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:20:53.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775883+0000", 2024-09-17T21:20:53.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3399999999999998e-07 2024-09-17T21:20:53.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775886+0000", 2024-09-17T21:20:53.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2119999999999999e-06 2024-09-17T21:20:53.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775895+0000", 2024-09-17T21:20:53.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8710000000000003e-06 2024-09-17T21:20:53.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775901+0000", 2024-09-17T21:20:53.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:20:53.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863544+0000", 2024-09-17T21:20:53.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087642642000000007 2024-09-17T21:20:53.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863694+0000", 2024-09-17T21:20:53.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014952300000000001 2024-09-17T21:20:53.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863851+0000", 2024-09-17T21:20:53.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000157804 2024-09-17T21:20:53.129 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.129 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.129 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.129 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12105 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:53.129 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:53.129 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.258532625000001, 2024-09-17T21:20:53.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088330400000000003, 2024-09-17T21:20:53.129 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.129 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.129 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:53.129 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:53.129 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:53.129 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:53.130 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12105 2024-09-17T21:20:53.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.130 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:53.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:53.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:53.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9999999999999997e-07 2024-09-17T21:20:53.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775914+0000", 2024-09-17T21:20:53.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4710000000000001e-06 2024-09-17T21:20:53.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775921+0000", 2024-09-17T21:20:53.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:20:53.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775928+0000", 2024-09-17T21:20:53.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:20:53.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863923+0000", 2024-09-17T21:20:53.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087995126000000007 2024-09-17T21:20:53.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864059+0000", 2024-09-17T21:20:53.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135797 2024-09-17T21:20:53.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.133 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.133 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864241+0000", 2024-09-17T21:20:53.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018252599999999999 2024-09-17T21:20:53.133 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.133 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.133 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.133 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12106 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:53.133 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:20:53.133 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.258443447000001, 2024-09-17T21:20:53.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088586394999999998, 2024-09-17T21:20:53.134 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.134 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.134 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:53.134 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:53.134 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:53.134 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:53.134 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12106 2024-09-17T21:20:53.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.134 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:20:53.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:20:53.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776001+0000", 2024-09-17T21:20:53.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4300000000000001e-07 2024-09-17T21:20:53.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776004+0000", 2024-09-17T21:20:53.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8049999999999998e-06 2024-09-17T21:20:53.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776018+0000", 2024-09-17T21:20:53.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4613999999999999e-05 2024-09-17T21:20:53.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776025+0000", 2024-09-17T21:20:53.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:20:53.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864279+0000", 2024-09-17T21:20:53.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088253570000000003 2024-09-17T21:20:53.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.137 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.137 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864433+0000", 2024-09-17T21:20:53.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015469400000000001 2024-09-17T21:20:53.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.137 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.137 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864586+0000", 2024-09-17T21:20:53.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000153003 2024-09-17T21:20:53.137 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.137 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.137 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.137 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.138 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.138 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:53.139 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:20:53.140 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:53.134+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:53.240 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:53.234+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:53.310 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:53.310 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:53.310 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:53.310 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:53.324 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:20:53.341 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:53.334+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:53.441 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:53.434+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:53.445 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:53.445 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:53.446 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:53.446 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:53.446 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:53.461 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:20:53.510 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:20:53.542 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:53.534+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:53.592 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:53.593 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:53.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:53.593 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:53.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.593 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:20:53.593 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:53.593 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 155.37230645899999, 2024-09-17T21:20:53.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:20:53.593 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.593 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.593 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.593 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.594 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.594 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:53.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.594 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.594 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:20:53.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.594 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.594 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:20:53.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:20:53.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.595 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.595 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:20:53.595 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:20:53.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.595 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.595 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:20:53.595 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:20:53.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.595 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.596 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:20:53.596 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:20:53.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.596 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.596 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:20:53.596 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:20:53.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.596 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:20:53.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:20:53.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:20:53.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:20:53.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:20:53.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:20:53.598 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.598 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.598 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.598 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:20:53.598 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:53.598 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 150.722005795, 2024-09-17T21:20:53.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:20:53.598 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.600 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.600 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.600 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.600 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.600 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:53.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.601 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:20:53.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:20:53.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:20:53.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.602 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:20:53.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:20:53.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.602 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:20:53.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:20:53.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:20:53.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:20:53.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:20:53.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:20:53.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:20:53.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:20:53.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:20:53.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:20:53.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:20:53.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:20:53.604 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.604 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.604 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.605 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:20:53.605 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:53.605 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 150.72074071200001, 2024-09-17T21:20:53.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:20:53.605 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.605 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.605 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.605 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.605 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.605 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:53.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:20:53.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:20:53.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:20:53.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:20:53.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:20:53.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:20:53.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:20:53.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:20:53.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:20:53.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:20:53.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:20:53.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:20:53.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:20:53.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:20:53.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:20:53.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.609 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.609 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:20:53.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:20:53.609 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.609 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.609 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.609 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:20:53.609 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:53.609 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 150.71096130199999, 2024-09-17T21:20:53.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:20:53.610 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.610 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.610 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.610 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:53.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:20:53.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:20:53.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:20:53.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:20:53.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:20:53.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:20:53.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:20:53.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:20:53.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:20:53.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:20:53.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:20:53.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.613 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:20:53.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:20:53.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.613 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:20:53.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:20:53.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.613 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:20:53.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:20:53.614 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.614 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.614 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.614 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:20:53.614 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:53.614 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 150.70223889900001, 2024-09-17T21:20:53.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:20:53.614 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.614 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.614 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.614 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:53.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:20:53.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:20:53.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:20:53.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:20:53.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:20:53.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:20:53.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:20:53.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:20:53.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:20:53.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:20:53.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:20:53.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:20:53.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:20:53.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:20:53.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:20:53.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:20:53.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:20:53.618 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.618 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.618 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.619 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:20:53.619 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:53.619 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 137.31714833500001, 2024-09-17T21:20:53.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:20:53.619 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.619 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.619 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.619 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:53.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:20:53.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:20:53.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:20:53.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:20:53.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:20:53.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:20:53.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:20:53.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:20:53.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:20:53.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:20:53.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:20:53.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:20:53.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:20:53.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:20:53.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:20:53.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:20:53.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:20:53.623 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.623 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.623 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.623 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1780 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'195, mlcod=69'190)", 2024-09-17T21:20:53.623 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:53.623 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 100.339292731, 2024-09-17T21:20:53.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075761769999999999, 2024-09-17T21:20:53.623 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.623 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.623 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.623 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:53.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:20:53.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250375+0000", 2024-09-17T21:20:53.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:20:53.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250383+0000", 2024-09-17T21:20:53.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7870000000000002e-06 2024-09-17T21:20:53.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250402+0000", 2024-09-17T21:20:53.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9346000000000001e-05 2024-09-17T21:20:53.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250413+0000", 2024-09-17T21:20:53.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.039e-05 2024-09-17T21:20:53.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251353+0000", 2024-09-17T21:20:53.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000940091 2024-09-17T21:20:53.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251407+0000", 2024-09-17T21:20:53.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4701999999999999e-05 2024-09-17T21:20:53.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257829+0000", 2024-09-17T21:20:53.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064212779999999999 2024-09-17T21:20:53.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257950+0000", 2024-09-17T21:20:53.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121623 2024-09-17T21:20:53.627 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.627 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.627 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.627 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1781 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'196, mlcod=69'190)", 2024-09-17T21:20:53.627 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:53.627 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 100.33820978, 2024-09-17T21:20:53.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065632939999999999, 2024-09-17T21:20:53.627 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.628 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.628 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.628 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:53.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:20:53.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251459+0000", 2024-09-17T21:20:53.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.624e-06 2024-09-17T21:20:53.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251471+0000", 2024-09-17T21:20:53.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2636000000000001e-05 2024-09-17T21:20:53.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251498+0000", 2024-09-17T21:20:53.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:20:53.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251515+0000", 2024-09-17T21:20:53.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6939000000000002e-05 2024-09-17T21:20:53.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252938+0000", 2024-09-17T21:20:53.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001423082 2024-09-17T21:20:53.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252999+0000", 2024-09-17T21:20:53.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1138999999999999e-05 2024-09-17T21:20:53.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257955+0000", 2024-09-17T21:20:53.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049559640000000002 2024-09-17T21:20:53.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.258020+0000", 2024-09-17T21:20:53.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5504000000000002e-05 2024-09-17T21:20:53.631 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.631 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.631 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.632 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1788 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'198, mlcod=69'191)", 2024-09-17T21:20:53.632 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:53.632 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 100.336184291, 2024-09-17T21:20:53.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073690370000000002, 2024-09-17T21:20:53.632 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.632 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.632 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.632 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:53.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:20:53.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253484+0000", 2024-09-17T21:20:53.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2470000000000001e-06 2024-09-17T21:20:53.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253492+0000", 2024-09-17T21:20:53.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7159999999999992e-06 2024-09-17T21:20:53.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253509+0000", 2024-09-17T21:20:53.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.666e-05 2024-09-17T21:20:53.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253522+0000", 2024-09-17T21:20:53.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2643000000000001e-05 2024-09-17T21:20:53.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254853+0000", 2024-09-17T21:20:53.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331238 2024-09-17T21:20:53.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254885+0000", 2024-09-17T21:20:53.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1562e-05 2024-09-17T21:20:53.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260788+0000", 2024-09-17T21:20:53.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059030749999999998 2024-09-17T21:20:53.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260852+0000", 2024-09-17T21:20:53.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3895999999999999e-05 2024-09-17T21:20:53.636 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.637 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.637 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.637 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5746 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'477, mlcod=71'473)", 2024-09-17T21:20:53.637 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:53.637 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.330756733000001, 2024-09-17T21:20:53.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074182709999999997, 2024-09-17T21:20:53.637 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.637 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.637 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.637 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:53.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:20:53.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258912+0000", 2024-09-17T21:20:53.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:20:53.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258926+0000", 2024-09-17T21:20:53.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4249e-05 2024-09-17T21:20:53.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258946+0000", 2024-09-17T21:20:53.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9879999999999999e-05 2024-09-17T21:20:53.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258956+0000", 2024-09-17T21:20:53.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:20:53.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259380+0000", 2024-09-17T21:20:53.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042337599999999999 2024-09-17T21:20:53.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259412+0000", 2024-09-17T21:20:53.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:20:53.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266191+0000", 2024-09-17T21:20:53.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067793269999999999 2024-09-17T21:20:53.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266328+0000", 2024-09-17T21:20:53.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013690999999999999 2024-09-17T21:20:53.641 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.641 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.641 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.641 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5747 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'478, mlcod=71'473)", 2024-09-17T21:20:53.642 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:53.642 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.330050384000003, 2024-09-17T21:20:53.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068779440000000004, 2024-09-17T21:20:53.642 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.642 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.642 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.642 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:53.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:20:53.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259618+0000", 2024-09-17T21:20:53.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6199999999999999e-06 2024-09-17T21:20:53.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259630+0000", 2024-09-17T21:20:53.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1884e-05 2024-09-17T21:20:53.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259657+0000", 2024-09-17T21:20:53.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7339e-05 2024-09-17T21:20:53.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259677+0000", 2024-09-17T21:20:53.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0156000000000001e-05 2024-09-17T21:20:53.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260521+0000", 2024-09-17T21:20:53.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084312899999999997 2024-09-17T21:20:53.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260585+0000", 2024-09-17T21:20:53.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4558000000000001e-05 2024-09-17T21:20:53.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266336+0000", 2024-09-17T21:20:53.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057505020000000002 2024-09-17T21:20:53.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266494+0000", 2024-09-17T21:20:53.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158756 2024-09-17T21:20:53.646 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.646 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.646 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.646 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5750 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'479, mlcod=71'473)", 2024-09-17T21:20:53.646 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:53.646 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.329338191000005, 2024-09-17T21:20:53.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006302874, 2024-09-17T21:20:53.646 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.646 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.646 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.647 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:53.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:20:53.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260330+0000", 2024-09-17T21:20:53.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:20:53.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.648 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260343+0000", 2024-09-17T21:20:53.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2862e-05 2024-09-17T21:20:53.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.648 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260367+0000", 2024-09-17T21:20:53.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3723e-05 2024-09-17T21:20:53.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.649 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.649 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260386+0000", 2024-09-17T21:20:53.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8879999999999999e-05 2024-09-17T21:20:53.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.649 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.649 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261579+0000", 2024-09-17T21:20:53.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001193431 2024-09-17T21:20:53.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.649 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.649 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261608+0000", 2024-09-17T21:20:53.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8858999999999999e-05 2024-09-17T21:20:53.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266505+0000", 2024-09-17T21:20:53.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004896999 2024-09-17T21:20:53.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266631+0000", 2024-09-17T21:20:53.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012655300000000001 2024-09-17T21:20:53.650 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.650 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.650 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.650 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5751 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'480, mlcod=71'473)", 2024-09-17T21:20:53.651 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:53.651 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.328630978999996, 2024-09-17T21:20:53.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075756299999999999, 2024-09-17T21:20:53.651 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.651 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.651 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.651 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:53.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:20:53.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261037+0000", 2024-09-17T21:20:53.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:20:53.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261049+0000", 2024-09-17T21:20:53.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1063999999999999e-05 2024-09-17T21:20:53.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.653 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261072+0000", 2024-09-17T21:20:53.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3102e-05 2024-09-17T21:20:53.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.653 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261133+0000", 2024-09-17T21:20:53.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1276000000000001e-05 2024-09-17T21:20:53.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.653 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:20:53.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015334319999999999 2024-09-17T21:20:53.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262702+0000", 2024-09-17T21:20:53.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5281999999999999e-05 2024-09-17T21:20:53.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268527+0000", 2024-09-17T21:20:53.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058250389999999997 2024-09-17T21:20:53.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268611+0000", 2024-09-17T21:20:53.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4724999999999999e-05 2024-09-17T21:20:53.655 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.655 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.655 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.655 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5756 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'481, mlcod=71'473)", 2024-09-17T21:20:53.655 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:53.655 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.327770122999993, 2024-09-17T21:20:53.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006772386, 2024-09-17T21:20:53.655 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.655 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.655 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.655 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:53.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:20:53.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261899+0000", 2024-09-17T21:20:53.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.903e-06 2024-09-17T21:20:53.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261912+0000", 2024-09-17T21:20:53.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-05 2024-09-17T21:20:53.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261938+0000", 2024-09-17T21:20:53.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6098999999999999e-05 2024-09-17T21:20:53.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261955+0000", 2024-09-17T21:20:53.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.696e-05 2024-09-17T21:20:53.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263845+0000", 2024-09-17T21:20:53.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001890187 2024-09-17T21:20:53.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263907+0000", 2024-09-17T21:20:53.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1600999999999995e-05 2024-09-17T21:20:53.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268615+0000", 2024-09-17T21:20:53.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047086070000000001 2024-09-17T21:20:53.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268669+0000", 2024-09-17T21:20:53.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3588999999999998e-05 2024-09-17T21:20:53.659 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.659 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.659 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.659 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4978.0:598 3.9 e80/23 3:9701cc17:::rbd_data.1152724ff2a3.0000000000000019:head v 80'202, mlcod=80'198)", 2024-09-17T21:20:53.659 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:53.659 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 86.826195740000003, 2024-09-17T21:20:53.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066473340000000004, 2024-09-17T21:20:53.660 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.660 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.660 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.660 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:53.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:20:53.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763472+0000", 2024-09-17T21:20:53.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6899999999999997e-07 2024-09-17T21:20:53.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763478+0000", 2024-09-17T21:20:53.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:20:53.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763489+0000", 2024-09-17T21:20:53.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0628999999999999e-05 2024-09-17T21:20:53.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763495+0000", 2024-09-17T21:20:53.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6649999999999997e-06 2024-09-17T21:20:53.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763907+0000", 2024-09-17T21:20:53.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041163899999999999 2024-09-17T21:20:53.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763976+0000", 2024-09-17T21:20:53.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8534999999999993e-05 2024-09-17T21:20:53.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.769900+0000", 2024-09-17T21:20:53.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059243270000000001 2024-09-17T21:20:53.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.770118+0000", 2024-09-17T21:20:53.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021842500000000001 2024-09-17T21:20:53.663 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.663 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.664 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.664 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:468 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'94, mlcod=89'87)", 2024-09-17T21:20:53.664 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:53.664 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.903450441000004, 2024-09-17T21:20:53.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066528790000000004, 2024-09-17T21:20:53.664 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.664 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.664 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.664 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:53.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:20:53.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686218+0000", 2024-09-17T21:20:53.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3060000000000001e-06 2024-09-17T21:20:53.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686256+0000", 2024-09-17T21:20:53.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8333e-05 2024-09-17T21:20:53.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686272+0000", 2024-09-17T21:20:53.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:20:53.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686283+0000", 2024-09-17T21:20:53.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0903e-05 2024-09-17T21:20:53.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686682+0000", 2024-09-17T21:20:53.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039935099999999999 2024-09-17T21:20:53.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686785+0000", 2024-09-17T21:20:53.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010275199999999999 2024-09-17T21:20:53.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692816+0000", 2024-09-17T21:20:53.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060317579999999999 2024-09-17T21:20:53.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692869+0000", 2024-09-17T21:20:53.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2775999999999998e-05 2024-09-17T21:20:53.667 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.668 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.668 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.668 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:755 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'127, mlcod=89'122)", 2024-09-17T21:20:53.668 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:53.668 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.766995226999995, 2024-09-17T21:20:53.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084014080000000008, 2024-09-17T21:20:53.668 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.668 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.668 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.668 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:53.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:53.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:20:53.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:20:53.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822683+0000", 2024-09-17T21:20:53.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0674e-05 2024-09-17T21:20:53.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822698+0000", 2024-09-17T21:20:53.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5189e-05 2024-09-17T21:20:53.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822708+0000", 2024-09-17T21:20:53.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9119999999999993e-06 2024-09-17T21:20:53.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822748+0000", 2024-09-17T21:20:53.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9727000000000001e-05 2024-09-17T21:20:53.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822759+0000", 2024-09-17T21:20:53.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0865e-05 2024-09-17T21:20:53.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831034+0000", 2024-09-17T21:20:53.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082750710000000002 2024-09-17T21:20:53.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.672 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831073+0000", 2024-09-17T21:20:53.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9063999999999997e-05 2024-09-17T21:20:53.672 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.672 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.672 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.672 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:756 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'128, mlcod=89'124)", 2024-09-17T21:20:53.672 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:53.672 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.765072595000007, 2024-09-17T21:20:53.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0076300270000000002, 2024-09-17T21:20:53.672 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.673 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.673 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.673 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:53.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:20:53.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824595+0000", 2024-09-17T21:20:53.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-17T21:20:53.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824600+0000", 2024-09-17T21:20:53.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:20:53.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824612+0000", 2024-09-17T21:20:53.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.181e-05 2024-09-17T21:20:53.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824619+0000", 2024-09-17T21:20:53.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:20:53.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824671+0000", 2024-09-17T21:20:53.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2262999999999998e-05 2024-09-17T21:20:53.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824695+0000", 2024-09-17T21:20:53.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3883000000000002e-05 2024-09-17T21:20:53.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832185+0000", 2024-09-17T21:20:53.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007490048 2024-09-17T21:20:53.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832224+0000", 2024-09-17T21:20:53.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8957e-05 2024-09-17T21:20:53.676 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.676 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.676 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.677 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2617 3.d e90/23 3:b0143718:::rbd_data.1152a8ae2019.0000000000000016:head v 90'249, mlcod=90'246)", 2024-09-17T21:20:53.677 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:53.677 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 72.745221595000004, 2024-09-17T21:20:53.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063715559999999996, 2024-09-17T21:20:53.677 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.677 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.677 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.677 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:53.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:20:53.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844455+0000", 2024-09-17T21:20:53.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4259999999999992e-06 2024-09-17T21:20:53.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844470+0000", 2024-09-17T21:20:53.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5386000000000001e-05 2024-09-17T21:20:53.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844499+0000", 2024-09-17T21:20:53.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.883e-05 2024-09-17T21:20:53.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844518+0000", 2024-09-17T21:20:53.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8830000000000001e-05 2024-09-17T21:20:53.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844649+0000", 2024-09-17T21:20:53.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013143399999999999 2024-09-17T21:20:53.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844702+0000", 2024-09-17T21:20:53.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2876e-05 2024-09-17T21:20:53.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850696+0000", 2024-09-17T21:20:53.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059943729999999999 2024-09-17T21:20:53.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850817+0000", 2024-09-17T21:20:53.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120401 2024-09-17T21:20:53.681 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.681 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.681 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.681 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2635 3.d e90/23 3:b3d1e920:::rbd_data.1152a8ae2019.000000000000001f:head v 90'252, mlcod=90'246)", 2024-09-17T21:20:53.681 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:53.681 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 72.742513525000007, 2024-09-17T21:20:53.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0077265190000000003, 2024-09-17T21:20:53.681 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:53.681 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:53.682 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:53.682 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:53.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:53.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:53.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:53.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:20:53.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:53.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:53.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847155+0000", 2024-09-17T21:20:53.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.544e-06 2024-09-17T21:20:53.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:53.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847165+0000", 2024-09-17T21:20:53.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.043e-05 2024-09-17T21:20:53.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:53.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847189+0000", 2024-09-17T21:20:53.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3659000000000002e-05 2024-09-17T21:20:53.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847210+0000", 2024-09-17T21:20:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1163e-05 2024-09-17T21:20:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848856+0000", 2024-09-17T21:20:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001645521 2024-09-17T21:20:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848905+0000", 2024-09-17T21:20:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9048999999999997e-05 2024-09-17T21:20:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854770+0000", 2024-09-17T21:20:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058652879999999998 2024-09-17T21:20:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854880+0000", 2024-09-17T21:20:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010986500000000001 2024-09-17T21:20:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:53.685 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:53.687 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:20:53.687 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:53.678+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:53.755 INFO:teuthology.orchestra.run.smithi184.stdout:leaving image image_client.0-clone39 intact 2024-09-17T21:20:53.787 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:53.780+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:53.822 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:53.822 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:53.822 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:20:53.822 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:53.837 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:20:53.889 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:53.884+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:53.957 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:53.957 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:53.957 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:53.957 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:53.957 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:53.971 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:20:53.990 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:53.984+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:54.091 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:54.082+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:54.102 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:54.102 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:54.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:54.102 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:54.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.103 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:20:54.103 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:20:54.103 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 177.08319757999999, 2024-09-17T21:20:54.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:20:54.103 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.103 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.103 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:54.103 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:20:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:20:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:20:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:20:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:20:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:20:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:20:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:20:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:20:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:20:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:20:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:20:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:20:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:20:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:20:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:20:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:20:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:20:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:20:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:20:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:20:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:20:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:20:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:20:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:20:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:20:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:20:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:20:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:20:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:20:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:20:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:20:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:20:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:20:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 157.5030333, 2024-09-17T21:20:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:20:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:20:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:20:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:20:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:20:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:20:54.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:20:54.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:20:54.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:20:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:20:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:20:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:20:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:20:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:20:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:20:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:20:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:20:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:20:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:20:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:20:54.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:20:54.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:54.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:20:54.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:20:54.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:20:54.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:20:54.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:20:54.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:20:54.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:20:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:20:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:20:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:20:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 157.50293598799999, 2024-09-17T21:20:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:20:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:20:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:20:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:20:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:20:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:20:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:20:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:20:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:20:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:20:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:20:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:20:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:20:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:20:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:20:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:20:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:20:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:20:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:20:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:54.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:20:54.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:20:54.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:20:54.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:20:54.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:20:54.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:20:54.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:20:54.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:20:54.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:20:54.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:20:54.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:20:54.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:20:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:20:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:20:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:20:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 157.500849187, 2024-09-17T21:20:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:20:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:20:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:20:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:20:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:20:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:20:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:20:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:20:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:20:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:20:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:20:54.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:20:54.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:20:54.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:20:54.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:20:54.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:20:54.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:20:54.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:20:54.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:20:54.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:20:54.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:20:54.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:54.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:20:54.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:20:54.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:20:54.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:20:54.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:20:54.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:20:54.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:20:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:20:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:20:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:20:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:20:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:20:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:20:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:20:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:20:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:20:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:20:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 124.816950406, 2024-09-17T21:20:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:20:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:20:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:20:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:20:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:20:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:20:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:20:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:20:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:20:54.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:20:54.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:20:54.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:20:54.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:20:54.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:20:54.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:20:54.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:20:54.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:20:54.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:20:54.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:20:54.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:20:54.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:20:54.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:54.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:20:54.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:20:54.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:20:54.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:20:54.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:20:54.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:20:54.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:20:54.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:20:54.141 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.141 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.141 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.141 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:54.141 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:20:54.141 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 124.80785869499999, 2024-09-17T21:20:54.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:20:54.141 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.141 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.141 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.141 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.142 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:20:54.142 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:20:54.142 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:20:54.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.142 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:20:54.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:20:54.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:20:54.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:20:54.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:20:54.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:20:54.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:20:54.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:20:54.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:20:54.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:20:54.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:20:54.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:20:54.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:54.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:20:54.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:20:54.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:20:54.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:20:54.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:20:54.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:20:54.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:20:54.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:20:54.146 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.146 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.146 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.146 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:54.146 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:20:54.146 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 124.807686644, 2024-09-17T21:20:54.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:20:54.146 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.147 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.147 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.147 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.147 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:20:54.147 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:20:54.147 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:20:54.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.147 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:20:54.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:20:54.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:20:54.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:20:54.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:20:54.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:20:54.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:20:54.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:20:54.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:20:54.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:20:54.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:20:54.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:20:54.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:54.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:20:54.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:20:54.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:20:54.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:20:54.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:20:54.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:20:54.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:20:54.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:20:54.151 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.151 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.152 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.152 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:54.152 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:20:54.152 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 124.632753897, 2024-09-17T21:20:54.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:20:54.152 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.152 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.152 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.152 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.152 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:54.152 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:54.152 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:20:54.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.153 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:20:54.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:20:54.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:20:54.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:20:54.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:20:54.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:20:54.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:20:54.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:20:54.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:20:54.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:20:54.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:20:54.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:20:54.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:54.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:20:54.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:20:54.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:20:54.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:20:54.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:20:54.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:20:54.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:20:54.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:20:54.157 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.157 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.157 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.158 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:54.158 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:54.158 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 124.63266934000001, 2024-09-17T21:20:54.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:20:54.158 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.158 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.158 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.158 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.158 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:54.158 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:54.159 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:20:54.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.159 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:54.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:54.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:20:54.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:20:54.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:20:54.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:20:54.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:20:54.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:20:54.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:20:54.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:20:54.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:20:54.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:20:54.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:54.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:20:54.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:20:54.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:20:54.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:20:54.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:20:54.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:20:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:20:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:20:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:20:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:20:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 124.632603703, 2024-09-17T21:20:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:20:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:20:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:20:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:20:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:20:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:20:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:20:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:20:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:20:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:20:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:20:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:20:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:20:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:20:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:20:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:20:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:20:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:20:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:20:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:20:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:20:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:20:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:20:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 73.259028606000001, 2024-09-17T21:20:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:20:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:20:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:20:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:20:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:20:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:20:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:20:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:20:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:20:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:20:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:20:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:20:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:20:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:20:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:20:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:20:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:20:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:20:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:20:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 73.258909094000003, 2024-09-17T21:20:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:20:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:20:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:20:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:20:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:20:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:20:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:20:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:20:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:20:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:20:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:20:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:20:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:20:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:20:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:20:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:20:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:20:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:20:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:20:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:994 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 571392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:20:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.665937119999999, 2024-09-17T21:20:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083898465000000005, 2024-09-17T21:20:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 994 2024-09-17T21:20:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:20:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:20:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:20:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:20:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431058+0000", 2024-09-17T21:20:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1667e-05 2024-09-17T21:20:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431067+0000", 2024-09-17T21:20:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0720000000000006e-06 2024-09-17T21:20:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431075+0000", 2024-09-17T21:20:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9200000000000004e-06 2024-09-17T21:20:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514671+0000", 2024-09-17T21:20:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083596229999999994 2024-09-17T21:20:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514741+0000", 2024-09-17T21:20:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9929000000000001e-05 2024-09-17T21:20:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514924+0000", 2024-09-17T21:20:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000183122 2024-09-17T21:20:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:995 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 575488~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:20:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.665878104000001, 2024-09-17T21:20:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084214699000000004, 2024-09-17T21:20:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 995 2024-09-17T21:20:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:20:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:20:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431086+0000", 2024-09-17T21:20:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7999999999999995e-07 2024-09-17T21:20:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431088+0000", 2024-09-17T21:20:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4729999999999999e-06 2024-09-17T21:20:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431126+0000", 2024-09-17T21:20:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7583999999999998e-05 2024-09-17T21:20:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431137+0000", 2024-09-17T21:20:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1768e-05 2024-09-17T21:20:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515003+0000", 2024-09-17T21:20:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083865097 2024-09-17T21:20:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515115+0000", 2024-09-17T21:20:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112561 2024-09-17T21:20:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515300+0000", 2024-09-17T21:20:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018453600000000001 2024-09-17T21:20:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:996 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:20:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.665795722999999, 2024-09-17T21:20:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084493736999999999, 2024-09-17T21:20:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 996 2024-09-17T21:20:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:20:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:20:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431168+0000", 2024-09-17T21:20:54.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4e-07 2024-09-17T21:20:54.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431172+0000", 2024-09-17T21:20:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.608e-06 2024-09-17T21:20:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431180+0000", 2024-09-17T21:20:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5650000000000002e-06 2024-09-17T21:20:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431188+0000", 2024-09-17T21:20:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:20:54.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515354+0000", 2024-09-17T21:20:54.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084165977000000003 2024-09-17T21:20:54.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515451+0000", 2024-09-17T21:20:54.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6669000000000006e-05 2024-09-17T21:20:54.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515661+0000", 2024-09-17T21:20:54.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021021599999999999 2024-09-17T21:20:54.194 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.194 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.194 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:997 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 583680~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:20:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.665764328000002, 2024-09-17T21:20:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084772419000000002, 2024-09-17T21:20:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 997 2024-09-17T21:20:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:20:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:20:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:20:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0400000000000005e-07 2024-09-17T21:20:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431202+0000", 2024-09-17T21:20:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:20:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431208+0000", 2024-09-17T21:20:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4740000000000002e-06 2024-09-17T21:20:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431214+0000", 2024-09-17T21:20:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6550000000000001e-06 2024-09-17T21:20:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515741+0000", 2024-09-17T21:20:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084526904 2024-09-17T21:20:54.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515826+0000", 2024-09-17T21:20:54.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4643999999999998e-05 2024-09-17T21:20:54.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515971+0000", 2024-09-17T21:20:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145444 2024-09-17T21:20:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:998 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 587776~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:20:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.665741356000002, 2024-09-17T21:20:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085108565999999997, 2024-09-17T21:20:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 998 2024-09-17T21:20:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:20:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:20:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:20:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:20:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431224+0000", 2024-09-17T21:20:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:20:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431231+0000", 2024-09-17T21:20:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000002e-06 2024-09-17T21:20:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431236+0000", 2024-09-17T21:20:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.648e-06 2024-09-17T21:20:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516024+0000", 2024-09-17T21:20:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084787584999999999 2024-09-17T21:20:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516122+0000", 2024-09-17T21:20:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8076000000000001e-05 2024-09-17T21:20:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516330+0000", 2024-09-17T21:20:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020808 2024-09-17T21:20:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1000 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 591872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:20:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.665719241000001, 2024-09-17T21:20:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085389065, 2024-09-17T21:20:54.206 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.206 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.206 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.206 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.206 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:54.208 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:54.208 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1000 2024-09-17T21:20:54.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.208 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:20:54.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:20:54.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:20:54.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:20:54.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431246+0000", 2024-09-17T21:20:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1749999999999999e-06 2024-09-17T21:20:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431253+0000", 2024-09-17T21:20:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2790000000000004e-06 2024-09-17T21:20:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431258+0000", 2024-09-17T21:20:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3299999999999998e-06 2024-09-17T21:20:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516384+0000", 2024-09-17T21:20:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085125549999999994 2024-09-17T21:20:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516460+0000", 2024-09-17T21:20:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6359999999999994e-05 2024-09-17T21:20:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516633+0000", 2024-09-17T21:20:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017285900000000001 2024-09-17T21:20:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1001 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 595968~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:20:54.213 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:20:54.213 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.665695835000001, 2024-09-17T21:20:54.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085614715999999993, 2024-09-17T21:20:54.213 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.213 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.213 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:20:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:20:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1001 2024-09-17T21:20:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:20:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:20:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438066+0000", 2024-09-17T21:20:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067989490000000003 2024-09-17T21:20:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438073+0000", 2024-09-17T21:20:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8419999999999999e-06 2024-09-17T21:20:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438090+0000", 2024-09-17T21:20:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6657e-05 2024-09-17T21:20:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438099+0000", 2024-09-17T21:20:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0529999999999996e-06 2024-09-17T21:20:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516678+0000", 2024-09-17T21:20:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078579422999999995 2024-09-17T21:20:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516739+0000", 2024-09-17T21:20:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1315000000000005e-05 2024-09-17T21:20:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516882+0000", 2024-09-17T21:20:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014247700000000001 2024-09-17T21:20:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:19998 3.5 3:a4885450:::rbd_object_map.1152d139f9f9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-17T21:20:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:20:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.694428589999999, 2024-09-17T21:20:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90715489299999996, 2024-09-17T21:20:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19998 2024-09-17T21:20:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:20:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:20:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402537+0000", 2024-09-17T21:20:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5600000000000001e-06 2024-09-17T21:20:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402549+0000", 2024-09-17T21:20:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1929999999999999e-05 2024-09-17T21:20:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402579+0000", 2024-09-17T21:20:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0093000000000001e-05 2024-09-17T21:20:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402595+0000", 2024-09-17T21:20:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5855999999999999e-05 2024-09-17T21:20:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402731+0000", 2024-09-17T21:20:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013610799999999999 2024-09-17T21:20:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402750+0000", 2024-09-17T21:20:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8737e-05 2024-09-17T21:20:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.305977+0000", 2024-09-17T21:20:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90322768399999998 2024-09-17T21:20:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306236+0000", 2024-09-17T21:20:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000258111 2024-09-17T21:20:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306557+0000", 2024-09-17T21:20:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032159799999999998 2024-09-17T21:20:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.308286+0000", 2024-09-17T21:20:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017292150000000001 2024-09-17T21:20:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309530+0000", 2024-09-17T21:20:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00124409 2024-09-17T21:20:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309630+0000", 2024-09-17T21:20:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9689e-05 2024-09-17T21:20:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309689+0000", 2024-09-17T21:20:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9222000000000003e-05 2024-09-17T21:20:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.225 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.225 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.225 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:54.226 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:20:54.227 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:54.220+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:54.327 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:54.324+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:54.408 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:54.408 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:54.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.408 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:22227 3.1a 3.c62dc91a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e144)", 2024-09-17T21:20:54.408 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:54.405819+0000", 2024-09-17T21:20:54.409 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.4063999999999999e-05, 2024-09-17T21:20:54.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013819700000000001, 2024-09-17T21:20:54.409 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.409 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.409 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "queued for pg", 2024-09-17T21:20:54.410 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.410 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:54.410 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:54.410 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22227 2024-09-17T21:20:54.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.410 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:54.405819+0000", 2024-09-17T21:20:54.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.412 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:54.405819+0000", 2024-09-17T21:20:54.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:54.405821+0000", 2024-09-17T21:20:54.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9259999999999999e-06 2024-09-17T21:20:54.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:54.405831+0000", 2024-09-17T21:20:54.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.011e-05 2024-09-17T21:20:54.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:54.405873+0000", 2024-09-17T21:20:54.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2472999999999998e-05 2024-09-17T21:20:54.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:54.405894+0000", 2024-09-17T21:20:54.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0602e-05 2024-09-17T21:20:54.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:54.405972+0000", 2024-09-17T21:20:54.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8516000000000005e-05 2024-09-17T21:20:54.417 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.417 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.417 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.420 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.420 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-09-17T21:20:54.420 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 1 2024-09-17T21:20:54.421 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:54.426 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:20:54.428 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:54.422+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:54.527 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-17T21:20:54.528 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:54.520+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:54.553 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:54.554 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:54.554 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:54.554 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:54.554 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:54.568 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:20:54.627 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-17T21:20:54.629 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:54.624+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:54.695 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:54.695 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:54.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:54.695 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:54.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.695 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:18003 3.1d 3:b9765580:::rbd_header.11527d2b15f6:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T21:20:54.695 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:20:54.695 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 42.333118231999997, 2024-09-17T21:20:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94165346900000002, 2024-09-17T21:20:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18003 2024-09-17T21:20:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:20:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:20:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357880+0000", 2024-09-17T21:20:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:20:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357891+0000", 2024-09-17T21:20:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0417999999999999e-05 2024-09-17T21:20:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357913+0000", 2024-09-17T21:20:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2215e-05 2024-09-17T21:20:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357922+0000", 2024-09-17T21:20:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4779999999999997e-06 2024-09-17T21:20:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358023+0000", 2024-09-17T21:20:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101025 2024-09-17T21:20:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358040+0000", 2024-09-17T21:20:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6291999999999999e-05 2024-09-17T21:20:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298203+0000", 2024-09-17T21:20:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94016297500000001 2024-09-17T21:20:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298626+0000", 2024-09-17T21:20:54.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000422853 2024-09-17T21:20:54.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.299532+0000", 2024-09-17T21:20:54.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00090657300000000004 2024-09-17T21:20:54.701 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.701 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.701 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15706 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 977920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:20:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.170016441000001, 2024-09-17T21:20:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091952328, 2024-09-17T21:20:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15706 2024-09-17T21:20:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:20:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:20:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520982+0000", 2024-09-17T21:20:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.277e-06 2024-09-17T21:20:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520988+0000", 2024-09-17T21:20:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4470000000000003e-06 2024-09-17T21:20:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521008+0000", 2024-09-17T21:20:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9309000000000001e-05 2024-09-17T21:20:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521023+0000", 2024-09-17T21:20:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5119999999999999e-05 2024-09-17T21:20:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612638+0000", 2024-09-17T21:20:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091615328999999995 2024-09-17T21:20:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612738+0000", 2024-09-17T21:20:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100466 2024-09-17T21:20:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612933+0000", 2024-09-17T21:20:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019437999999999999 2024-09-17T21:20:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15707 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 982016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:20:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.169952985000002, 2024-09-17T21:20:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092305228000000003, 2024-09-17T21:20:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15707 2024-09-17T21:20:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:20:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:20:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521045+0000", 2024-09-17T21:20:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4899999999999999e-06 2024-09-17T21:20:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521066+0000", 2024-09-17T21:20:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0236000000000001e-05 2024-09-17T21:20:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521085+0000", 2024-09-17T21:20:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8819999999999999e-05 2024-09-17T21:20:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521100+0000", 2024-09-17T21:20:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5146e-05 2024-09-17T21:20:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612978+0000", 2024-09-17T21:20:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091878847 2024-09-17T21:20:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613079+0000", 2024-09-17T21:20:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100503 2024-09-17T21:20:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613349+0000", 2024-09-17T21:20:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000270186 2024-09-17T21:20:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15708 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 986112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:20:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.169874810000003, 2024-09-17T21:20:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092547217000000001, 2024-09-17T21:20:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15708 2024-09-17T21:20:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:20:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:20:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521123+0000", 2024-09-17T21:20:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.353e-06 2024-09-17T21:20:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521129+0000", 2024-09-17T21:20:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:20:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521145+0000", 2024-09-17T21:20:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.611e-05 2024-09-17T21:20:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521154+0000", 2024-09-17T21:20:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4500000000000004e-06 2024-09-17T21:20:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613397+0000", 2024-09-17T21:20:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092243275 2024-09-17T21:20:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613503+0000", 2024-09-17T21:20:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106013 2024-09-17T21:20:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613669+0000", 2024-09-17T21:20:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016642299999999999 2024-09-17T21:20:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15709 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 990208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:20:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.169831793999997, 2024-09-17T21:20:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092728678999999994, 2024-09-17T21:20:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15709 2024-09-17T21:20:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:20:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:20:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521166+0000", 2024-09-17T21:20:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7000000000000004e-07 2024-09-17T21:20:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521169+0000", 2024-09-17T21:20:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2660000000000002e-06 2024-09-17T21:20:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521181+0000", 2024-09-17T21:20:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.208e-05 2024-09-17T21:20:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521194+0000", 2024-09-17T21:20:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2707e-05 2024-09-17T21:20:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613695+0000", 2024-09-17T21:20:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092500709 2024-09-17T21:20:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613750+0000", 2024-09-17T21:20:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.503e-05 2024-09-17T21:20:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613894+0000", 2024-09-17T21:20:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144117 2024-09-17T21:20:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15710 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 994304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:20:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.169787227999997, 2024-09-17T21:20:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092957729000000003, 2024-09-17T21:20:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15710 2024-09-17T21:20:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:20:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:20:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:20:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8000000000000005e-07 2024-09-17T21:20:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521216+0000", 2024-09-17T21:20:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3859999999999998e-06 2024-09-17T21:20:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521234+0000", 2024-09-17T21:20:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:20:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521248+0000", 2024-09-17T21:20:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:20:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613926+0000", 2024-09-17T21:20:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092678324000000006 2024-09-17T21:20:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613974+0000", 2024-09-17T21:20:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7754999999999999e-05 2024-09-17T21:20:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614167+0000", 2024-09-17T21:20:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193508 2024-09-17T21:20:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15711 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 998400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:20:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.169729846000003, 2024-09-17T21:20:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093268000000000004, 2024-09-17T21:20:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15711 2024-09-17T21:20:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:20:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:20:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521268+0000", 2024-09-17T21:20:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-17T21:20:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521273+0000", 2024-09-17T21:20:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:20:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521290+0000", 2024-09-17T21:20:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7269999999999999e-05 2024-09-17T21:20:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521304+0000", 2024-09-17T21:20:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3626e-05 2024-09-17T21:20:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614220+0000", 2024-09-17T21:20:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092916367 2024-09-17T21:20:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614322+0000", 2024-09-17T21:20:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102212 2024-09-17T21:20:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614535+0000", 2024-09-17T21:20:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021266199999999999 2024-09-17T21:20:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15712 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1002496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:20:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.169674350999998, 2024-09-17T21:20:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093568707000000001, 2024-09-17T21:20:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15712 2024-09-17T21:20:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:20:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:20:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521324+0000", 2024-09-17T21:20:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.17e-06 2024-09-17T21:20:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521334+0000", 2024-09-17T21:20:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9369999999999999e-06 2024-09-17T21:20:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521356+0000", 2024-09-17T21:20:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2045999999999999e-05 2024-09-17T21:20:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521370+0000", 2024-09-17T21:20:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4270000000000001e-05 2024-09-17T21:20:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614583+0000", 2024-09-17T21:20:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093213220999999999 2024-09-17T21:20:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614680+0000", 2024-09-17T21:20:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6900999999999999e-05 2024-09-17T21:20:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614891+0000", 2024-09-17T21:20:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021116200000000001 2024-09-17T21:20:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15713 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1006592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:20:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.169607649, 2024-09-17T21:20:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093834641999999996, 2024-09-17T21:20:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15713 2024-09-17T21:20:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:20:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:20:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521390+0000", 2024-09-17T21:20:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:20:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521396+0000", 2024-09-17T21:20:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:20:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521417+0000", 2024-09-17T21:20:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1236000000000001e-05 2024-09-17T21:20:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521431+0000", 2024-09-17T21:20:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3873000000000001e-05 2024-09-17T21:20:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614938+0000", 2024-09-17T21:20:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093506522999999994 2024-09-17T21:20:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615025+0000", 2024-09-17T21:20:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7635999999999999e-05 2024-09-17T21:20:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615224+0000", 2024-09-17T21:20:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019844700000000001 2024-09-17T21:20:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15714 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:20:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.169544393000002, 2024-09-17T21:20:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094071925000000001, 2024-09-17T21:20:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15714 2024-09-17T21:20:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:20:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:20:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521454+0000", 2024-09-17T21:20:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.097e-06 2024-09-17T21:20:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521459+0000", 2024-09-17T21:20:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2660000000000001e-06 2024-09-17T21:20:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521481+0000", 2024-09-17T21:20:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1878999999999998e-05 2024-09-17T21:20:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521498+0000", 2024-09-17T21:20:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6807e-05 2024-09-17T21:20:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615271+0000", 2024-09-17T21:20:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093773286999999997 2024-09-17T21:20:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615366+0000", 2024-09-17T21:20:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5322000000000007e-05 2024-09-17T21:20:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615524+0000", 2024-09-17T21:20:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015826699999999999 2024-09-17T21:20:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15715 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1014784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:20:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.169480057999998, 2024-09-17T21:20:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094243840999999995, 2024-09-17T21:20:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15715 2024-09-17T21:20:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:20:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:20:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521518+0000", 2024-09-17T21:20:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4300000000000001e-06 2024-09-17T21:20:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521524+0000", 2024-09-17T21:20:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5860000000000004e-06 2024-09-17T21:20:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521541+0000", 2024-09-17T21:20:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7574000000000001e-05 2024-09-17T21:20:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521555+0000", 2024-09-17T21:20:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3540000000000001e-05 2024-09-17T21:20:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615560+0000", 2024-09-17T21:20:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.09400472 2024-09-17T21:20:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615614+0000", 2024-09-17T21:20:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4693e-05 2024-09-17T21:20:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615761+0000", 2024-09-17T21:20:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000146298 2024-09-17T21:20:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15716 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1018880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:20:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.169423713, 2024-09-17T21:20:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094400676000000003, 2024-09-17T21:20:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15716 2024-09-17T21:20:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:20:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:20:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521574+0000", 2024-09-17T21:20:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1939999999999999e-06 2024-09-17T21:20:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521579+0000", 2024-09-17T21:20:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:20:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521597+0000", 2024-09-17T21:20:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8074e-05 2024-09-17T21:20:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521613+0000", 2024-09-17T21:20:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5316e-05 2024-09-17T21:20:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615780+0000", 2024-09-17T21:20:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094166874999999997 2024-09-17T21:20:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615836+0000", 2024-09-17T21:20:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6926999999999999e-05 2024-09-17T21:20:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615974+0000", 2024-09-17T21:20:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013741400000000001 2024-09-17T21:20:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15717 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1022976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:20:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.169365016999997, 2024-09-17T21:20:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094586723999999997, 2024-09-17T21:20:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15717 2024-09-17T21:20:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:20:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:20:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521633+0000", 2024-09-17T21:20:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:20:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521638+0000", 2024-09-17T21:20:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:20:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521659+0000", 2024-09-17T21:20:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-05 2024-09-17T21:20:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521675+0000", 2024-09-17T21:20:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5688999999999998e-05 2024-09-17T21:20:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616008+0000", 2024-09-17T21:20:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094333314000000001 2024-09-17T21:20:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616066+0000", 2024-09-17T21:20:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7797000000000002e-05 2024-09-17T21:20:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616219+0000", 2024-09-17T21:20:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015281100000000001 2024-09-17T21:20:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15718 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:20:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.169264941999998, 2024-09-17T21:20:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094703674000000002, 2024-09-17T21:20:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15718 2024-09-17T21:20:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:20:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:20:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521734+0000", 2024-09-17T21:20:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.61e-06 2024-09-17T21:20:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521740+0000", 2024-09-17T21:20:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:20:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521754+0000", 2024-09-17T21:20:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3793e-05 2024-09-17T21:20:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521763+0000", 2024-09-17T21:20:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5129999999999996e-06 2024-09-17T21:20:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616256+0000", 2024-09-17T21:20:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094492919999999994 2024-09-17T21:20:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616312+0000", 2024-09-17T21:20:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6737000000000002e-05 2024-09-17T21:20:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616436+0000", 2024-09-17T21:20:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012335800000000001 2024-09-17T21:20:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15719 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1031168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:20:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.169211537000002, 2024-09-17T21:20:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094888963000000007, 2024-09-17T21:20:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15719 2024-09-17T21:20:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:20:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:20:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521786+0000", 2024-09-17T21:20:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.074e-06 2024-09-17T21:20:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521791+0000", 2024-09-17T21:20:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3030000000000001e-06 2024-09-17T21:20:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521804+0000", 2024-09-17T21:20:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3027000000000001e-05 2024-09-17T21:20:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521811+0000", 2024-09-17T21:20:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6090000000000003e-06 2024-09-17T21:20:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616466+0000", 2024-09-17T21:20:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094654294 2024-09-17T21:20:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616523+0000", 2024-09-17T21:20:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6953999999999997e-05 2024-09-17T21:20:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616674+0000", 2024-09-17T21:20:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015170200000000001 2024-09-17T21:20:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15720 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1035264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:20:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.169174697000003, 2024-09-17T21:20:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095067061999999994, 2024-09-17T21:20:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15720 2024-09-17T21:20:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:20:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:20:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521823+0000", 2024-09-17T21:20:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:20:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521826+0000", 2024-09-17T21:20:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2100000000000002e-06 2024-09-17T21:20:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521835+0000", 2024-09-17T21:20:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1630000000000002e-06 2024-09-17T21:20:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521842+0000", 2024-09-17T21:20:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:20:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616711+0000", 2024-09-17T21:20:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094868859 2024-09-17T21:20:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616764+0000", 2024-09-17T21:20:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2778999999999997e-05 2024-09-17T21:20:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616889+0000", 2024-09-17T21:20:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012533100000000001 2024-09-17T21:20:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15721 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1039360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:20:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.169127588000002, 2024-09-17T21:20:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095277219999999996, 2024-09-17T21:20:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15721 2024-09-17T21:20:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:20:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:20:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521871+0000", 2024-09-17T21:20:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.443e-06 2024-09-17T21:20:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521876+0000", 2024-09-17T21:20:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0300000000000001e-06 2024-09-17T21:20:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521886+0000", 2024-09-17T21:20:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186e-05 2024-09-17T21:20:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521893+0000", 2024-09-17T21:20:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.08e-06 2024-09-17T21:20:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616918+0000", 2024-09-17T21:20:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095025223000000006 2024-09-17T21:20:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616977+0000", 2024-09-17T21:20:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8403999999999999e-05 2024-09-17T21:20:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617147+0000", 2024-09-17T21:20:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016985399999999999 2024-09-17T21:20:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15722 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:20:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.169093138999997, 2024-09-17T21:20:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095459783000000006, 2024-09-17T21:20:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15722 2024-09-17T21:20:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:20:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:20:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:20:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7700000000000004e-07 2024-09-17T21:20:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521907+0000", 2024-09-17T21:20:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.813e-06 2024-09-17T21:20:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521922+0000", 2024-09-17T21:20:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4533e-05 2024-09-17T21:20:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521934+0000", 2024-09-17T21:20:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2469999999999999e-05 2024-09-17T21:20:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617187+0000", 2024-09-17T21:20:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095252311000000006 2024-09-17T21:20:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617240+0000", 2024-09-17T21:20:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3740999999999999e-05 2024-09-17T21:20:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617364+0000", 2024-09-17T21:20:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-17T21:20:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15723 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:20:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.169049272999999, 2024-09-17T21:20:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095653563999999996, 2024-09-17T21:20:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15723 2024-09-17T21:20:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:20:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:20:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521949+0000", 2024-09-17T21:20:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:20:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521953+0000", 2024-09-17T21:20:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5870000000000003e-06 2024-09-17T21:20:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521970+0000", 2024-09-17T21:20:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7282000000000002e-05 2024-09-17T21:20:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521984+0000", 2024-09-17T21:20:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:20:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617383+0000", 2024-09-17T21:20:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095399063000000006 2024-09-17T21:20:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617437+0000", 2024-09-17T21:20:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3931999999999998e-05 2024-09-17T21:20:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617601+0000", 2024-09-17T21:20:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016407300000000001 2024-09-17T21:20:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5629.0:2384 3.1c 3:38c1263e:::rbd_data.1152d37183c3.000000000000000b:head [read 11780096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-17T21:20:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:20:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.266218443, 2024-09-17T21:20:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0246185320000001, 2024-09-17T21:20:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5629", 2024-09-17T21:20:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3798711329", 2024-09-17T21:20:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2384 2024-09-17T21:20:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:20:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:20:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424780+0000", 2024-09-17T21:20:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.13e-06 2024-09-17T21:20:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424785+0000", 2024-09-17T21:20:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4739999999999996e-06 2024-09-17T21:20:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424798+0000", 2024-09-17T21:20:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2704000000000001e-05 2024-09-17T21:20:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424806+0000", 2024-09-17T21:20:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9899999999999997e-06 2024-09-17T21:20:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424868+0000", 2024-09-17T21:20:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999997e-05 2024-09-17T21:20:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:20:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424875+0000", 2024-09-17T21:20:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0870000000000001e-06 2024-09-17T21:20:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428897+0000", 2024-09-17T21:20:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040226080000000004 2024-09-17T21:20:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428913+0000", 2024-09-17T21:20:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:20:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.448963+0000", 2024-09-17T21:20:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0200498659999999 2024-09-17T21:20:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449158+0000", 2024-09-17T21:20:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019581000000000001 2024-09-17T21:20:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449397+0000", 2024-09-17T21:20:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023871 2024-09-17T21:20:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.790 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.790 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:54.790 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:54.790 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:54.791 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:20:54.792 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:54.784+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:54.892 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:54.886+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:54.916 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:54.916 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:54.917 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:20:54.917 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:54.931 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:20:54.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:20:55.019 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:55.014+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:55.094 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:55.086+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:55.104 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:55.104 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:55.104 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:55.104 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:55.105 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:55.119 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:20:55.123 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:55.123 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:55.140 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:55.141 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:55.158 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:55.158 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:55.177 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:55.178 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:55.195 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:55.188+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:55.196 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:55.196 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:55.216 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:55.216 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:55.288 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:55.288 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:55.296 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:55.290+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:55.303 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:20:55.398 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:55.392+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:55.433 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:55.433 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:55.448 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:20:55.499 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:55.494+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:55.600 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:55.594+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:55.624 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:55.624 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:55.639 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:20:55.701 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:55.694+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:55.757 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:55.757 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:55.772 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:20:55.801 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:55.794+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:55.902 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:55.896+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:55.903 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:55.903 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:55.918 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:20:56.009 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:56.004+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:56.087 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:56.087 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:56.101 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:20:56.105 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:56.102+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:56.199 INFO:teuthology.orchestra.run.smithi184.stdout:5460 read 0x1ed9800 thru 0x1ee4ec5 (0xb6c6 bytes) 2024-09-17T21:20:56.200 INFO:teuthology.orchestra.run.smithi184.stdout:5461 read 0xda1600 thru 0xda3688 (0x2089 bytes) 2024-09-17T21:20:56.206 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:56.200+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:56.239 INFO:teuthology.orchestra.run.smithi184.stdout:5462 trunc from 0x2aba400 to 0x9073600 2024-09-17T21:20:56.270 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:56.270 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:56.284 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:20:56.293 INFO:teuthology.orchestra.run.smithi184.stdout:5463 trunc from 0x9073600 to 0xc706800 2024-09-17T21:20:56.306 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:56.298+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:56.318 INFO:teuthology.orchestra.run.smithi184.stdout:5465 punch from 0x44ac200 to 0x44ba600, (0xe400 bytes) 2024-09-17T21:20:56.337 INFO:teuthology.orchestra.run.smithi184.stdout:5469 trunc from 0xc706800 to 0x2caa200 2024-09-17T21:20:56.408 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:56.400+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:56.475 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:56.475 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:56.491 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:20:56.508 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:56.502+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:56.529 INFO:teuthology.orchestra.run.smithi184.stdout:5470 write 0xd399600 thru 0xd3a4da4 (0xb7a5 bytes) 2024-09-17T21:20:56.532 INFO:teuthology.orchestra.run.smithi184.stdout:5471 read 0x1256c00 thru 0x125fd2d (0x912e bytes) 2024-09-17T21:20:56.546 INFO:teuthology.orchestra.run.smithi184.stdout:5472 trunc from 0xd3a4e00 to 0x94d7000 2024-09-17T21:20:56.550 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:20:56.621 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:56.616+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:56.643 INFO:teuthology.orchestra.run.smithi184.stdout:5473 read 0x321ca00 thru 0x32214df (0x4ae0 bytes) 2024-09-17T21:20:56.644 INFO:teuthology.orchestra.run.smithi184.stdout:5474 punch from 0x8d3a600 to 0x8d3e000, (0x3a00 bytes) 2024-09-17T21:20:56.650 INFO:teuthology.orchestra.run.smithi184.stdout:5476 trunc from 0x94d7000 to 0xb2c0e00 2024-09-17T21:20:56.659 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:56.659 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:56.676 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:20:56.709 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:56.702+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:56.714 INFO:teuthology.orchestra.run.smithi184.stdout:5477 punch from 0x1740200 to 0x1740600, (0x400 bytes) 2024-09-17T21:20:56.723 INFO:teuthology.orchestra.run.smithi184.stdout:5479 read 0x7524600 thru 0x7530588 (0xbf89 bytes) 2024-09-17T21:20:56.723 INFO:teuthology.orchestra.run.smithi184.stdout:5480 punch from 0x3179200 to 0x317d200, (0x4000 bytes) 2024-09-17T21:20:56.810 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:56.804+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:56.836 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:56.836 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:56.851 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:20:56.911 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:56.904+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:56.980 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:56.980 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:56.993 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:20:57.038 INFO:teuthology.orchestra.run.smithi184.stdout:5481 write 0xe0a6800 thru 0xe0ab7e7 (0x4fe8 bytes) 2024-09-17T21:20:57.039 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:57.034+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:57.040 INFO:teuthology.orchestra.run.smithi184.stdout:5482 write 0x433f400 thru 0x43408d1 (0x14d2 bytes) 2024-09-17T21:20:57.061 INFO:teuthology.orchestra.run.smithi184.stdout:5483 trunc from 0xe0ab800 to 0x5f20400 2024-09-17T21:20:57.113 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:57.106+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:57.117 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:57.117 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:57.131 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:20:57.188 INFO:teuthology.orchestra.run.smithi184.stdout:5484 punch from 0x4103e00 to 0x410c000, (0x8200 bytes) 2024-09-17T21:20:57.189 INFO:teuthology.orchestra.run.smithi184.stdout:5485 read 0x4d85400 thru 0x4d8a707 (0x5308 bytes) 2024-09-17T21:20:57.189 INFO:teuthology.orchestra.run.smithi184.stdout:5488 read 0x2aeee00 thru 0x2aeefe5 (0x1e6 bytes) 2024-09-17T21:20:57.190 INFO:teuthology.orchestra.run.smithi184.stdout:5490 clone 41 order 24 su 524288 sc 4 2024-09-17T21:20:57.214 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:57.208+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:57.234 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:20:57.308 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:57.308 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:57.315 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:57.308+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:57.322 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:20:57.392 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:57.392 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:57.408 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:57.408 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:57.416 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:57.410+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:57.424 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:57.424 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:57.442 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:57.442 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:57.444 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:57.445 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:57.458 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:20:57.465 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:57.465 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:57.484 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:57.485 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:57.526 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 0, 5] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:20:57.526 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:20:57.526 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-17T21:20:57.527 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T21:20:57.527 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T21:20:57.527 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-17T21:20:57.532 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-17T21:20:57.532 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:57.524+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:57.618 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:57.612+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:57.631 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:57.631 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:57.631 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:57.632 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:57.632 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:57.648 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:20:57.667 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:57.668 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "" 2024-09-17T21:20:57.668 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:57.731 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:57.726+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:57.773 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:57.773 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:57.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:57.773 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:57.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.773 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:18003 3.1d 3:b9765580:::rbd_header.11527d2b15f6:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T21:20:57.773 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:20:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.412481210000003, 2024-09-17T21:20:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94165346900000002, 2024-09-17T21:20:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:20:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:20:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18003 2024-09-17T21:20:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:20:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:20:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357880+0000", 2024-09-17T21:20:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:20:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357891+0000", 2024-09-17T21:20:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0417999999999999e-05 2024-09-17T21:20:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357913+0000", 2024-09-17T21:20:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2215e-05 2024-09-17T21:20:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357922+0000", 2024-09-17T21:20:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4779999999999997e-06 2024-09-17T21:20:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358023+0000", 2024-09-17T21:20:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101025 2024-09-17T21:20:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358040+0000", 2024-09-17T21:20:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6291999999999999e-05 2024-09-17T21:20:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298203+0000", 2024-09-17T21:20:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94016297500000001 2024-09-17T21:20:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298626+0000", 2024-09-17T21:20:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000422853 2024-09-17T21:20:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.299532+0000", 2024-09-17T21:20:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00090657300000000004 2024-09-17T21:20:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15706 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 977920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:20:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.249379419, 2024-09-17T21:20:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091952328, 2024-09-17T21:20:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15706 2024-09-17T21:20:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:20:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:20:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520982+0000", 2024-09-17T21:20:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.277e-06 2024-09-17T21:20:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520988+0000", 2024-09-17T21:20:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4470000000000003e-06 2024-09-17T21:20:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:57.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521008+0000", 2024-09-17T21:20:57.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9309000000000001e-05 2024-09-17T21:20:57.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:57.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521023+0000", 2024-09-17T21:20:57.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5119999999999999e-05 2024-09-17T21:20:57.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:57.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612638+0000", 2024-09-17T21:20:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091615328999999995 2024-09-17T21:20:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612738+0000", 2024-09-17T21:20:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100466 2024-09-17T21:20:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612933+0000", 2024-09-17T21:20:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019437999999999999 2024-09-17T21:20:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15707 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 982016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:20:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.249315963000001, 2024-09-17T21:20:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092305228000000003, 2024-09-17T21:20:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:57.786 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15707 2024-09-17T21:20:57.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.786 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:57.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:57.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:20:57.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:57.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:20:57.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:57.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521045+0000", 2024-09-17T21:20:57.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4899999999999999e-06 2024-09-17T21:20:57.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:57.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521066+0000", 2024-09-17T21:20:57.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0236000000000001e-05 2024-09-17T21:20:57.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:57.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521085+0000", 2024-09-17T21:20:57.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8819999999999999e-05 2024-09-17T21:20:57.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:57.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521100+0000", 2024-09-17T21:20:57.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5146e-05 2024-09-17T21:20:57.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:57.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612978+0000", 2024-09-17T21:20:57.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091878847 2024-09-17T21:20:57.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:57.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613079+0000", 2024-09-17T21:20:57.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100503 2024-09-17T21:20:57.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:57.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613349+0000", 2024-09-17T21:20:57.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000270186 2024-09-17T21:20:57.792 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.792 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:57.792 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.793 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15708 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 986112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:57.793 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:20:57.793 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.249237788000002, 2024-09-17T21:20:57.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092547217000000001, 2024-09-17T21:20:57.793 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:57.793 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:57.794 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:57.794 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:57.794 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:57.794 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:57.794 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15708 2024-09-17T21:20:57.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.794 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:57.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:57.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:20:57.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:57.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:20:57.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:57.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521123+0000", 2024-09-17T21:20:57.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.353e-06 2024-09-17T21:20:57.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:57.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521129+0000", 2024-09-17T21:20:57.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:20:57.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:57.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521145+0000", 2024-09-17T21:20:57.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.611e-05 2024-09-17T21:20:57.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:57.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521154+0000", 2024-09-17T21:20:57.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4500000000000004e-06 2024-09-17T21:20:57.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:57.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613397+0000", 2024-09-17T21:20:57.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092243275 2024-09-17T21:20:57.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:57.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613503+0000", 2024-09-17T21:20:57.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106013 2024-09-17T21:20:57.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:57.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613669+0000", 2024-09-17T21:20:57.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016642299999999999 2024-09-17T21:20:57.800 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.800 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:57.800 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.801 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15709 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 990208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:57.801 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:20:57.801 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.249194772000003, 2024-09-17T21:20:57.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092728678999999994, 2024-09-17T21:20:57.801 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:57.802 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:57.802 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:57.802 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:57.802 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:57.802 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:57.802 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15709 2024-09-17T21:20:57.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.802 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:57.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:57.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:20:57.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:57.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:20:57.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:57.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521166+0000", 2024-09-17T21:20:57.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7000000000000004e-07 2024-09-17T21:20:57.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:57.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521169+0000", 2024-09-17T21:20:57.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2660000000000002e-06 2024-09-17T21:20:57.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:57.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521181+0000", 2024-09-17T21:20:57.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.208e-05 2024-09-17T21:20:57.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:57.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521194+0000", 2024-09-17T21:20:57.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2707e-05 2024-09-17T21:20:57.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:57.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613695+0000", 2024-09-17T21:20:57.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092500709 2024-09-17T21:20:57.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:57.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613750+0000", 2024-09-17T21:20:57.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.503e-05 2024-09-17T21:20:57.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:57.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613894+0000", 2024-09-17T21:20:57.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144117 2024-09-17T21:20:57.809 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.809 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:57.809 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.810 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15710 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 994304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:57.810 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:20:57.810 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.249150206000003, 2024-09-17T21:20:57.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092957729000000003, 2024-09-17T21:20:57.810 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:57.810 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:57.811 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:57.811 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:57.811 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:57.811 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:57.811 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15710 2024-09-17T21:20:57.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.811 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:57.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:57.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:20:57.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:57.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:20:57.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:57.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:20:57.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8000000000000005e-07 2024-09-17T21:20:57.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:57.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521216+0000", 2024-09-17T21:20:57.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3859999999999998e-06 2024-09-17T21:20:57.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:57.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521234+0000", 2024-09-17T21:20:57.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:20:57.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:57.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521248+0000", 2024-09-17T21:20:57.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:20:57.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:57.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613926+0000", 2024-09-17T21:20:57.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092678324000000006 2024-09-17T21:20:57.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:57.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613974+0000", 2024-09-17T21:20:57.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7754999999999999e-05 2024-09-17T21:20:57.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:57.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614167+0000", 2024-09-17T21:20:57.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193508 2024-09-17T21:20:57.817 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.817 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:57.817 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.817 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15711 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 998400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:57.817 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:20:57.818 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.249092824000002, 2024-09-17T21:20:57.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093268000000000004, 2024-09-17T21:20:57.818 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:57.818 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:57.818 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:57.819 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:57.819 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:57.819 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:57.819 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15711 2024-09-17T21:20:57.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.819 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:57.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:57.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:20:57.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:57.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:20:57.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:57.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521268+0000", 2024-09-17T21:20:57.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-17T21:20:57.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:57.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521273+0000", 2024-09-17T21:20:57.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:20:57.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:57.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521290+0000", 2024-09-17T21:20:57.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7269999999999999e-05 2024-09-17T21:20:57.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:57.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521304+0000", 2024-09-17T21:20:57.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3626e-05 2024-09-17T21:20:57.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:57.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614220+0000", 2024-09-17T21:20:57.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092916367 2024-09-17T21:20:57.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:57.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614322+0000", 2024-09-17T21:20:57.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102212 2024-09-17T21:20:57.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:57.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614535+0000", 2024-09-17T21:20:57.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021266199999999999 2024-09-17T21:20:57.822 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.822 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:57.822 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.822 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15712 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1002496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:57.823 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:20:57.823 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.249037328999997, 2024-09-17T21:20:57.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093568707000000001, 2024-09-17T21:20:57.823 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:57.823 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:57.823 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:57.823 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:57.823 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:57.823 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:57.823 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15712 2024-09-17T21:20:57.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.823 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:57.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:57.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:20:57.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:57.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:20:57.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:57.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521324+0000", 2024-09-17T21:20:57.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.17e-06 2024-09-17T21:20:57.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:57.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521334+0000", 2024-09-17T21:20:57.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9369999999999999e-06 2024-09-17T21:20:57.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:57.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521356+0000", 2024-09-17T21:20:57.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2045999999999999e-05 2024-09-17T21:20:57.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:57.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521370+0000", 2024-09-17T21:20:57.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4270000000000001e-05 2024-09-17T21:20:57.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:57.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614583+0000", 2024-09-17T21:20:57.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093213220999999999 2024-09-17T21:20:57.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:57.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614680+0000", 2024-09-17T21:20:57.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6900999999999999e-05 2024-09-17T21:20:57.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:57.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614891+0000", 2024-09-17T21:20:57.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021116200000000001 2024-09-17T21:20:57.827 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.827 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:57.827 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.827 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15713 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1006592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:57.827 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:20:57.827 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.248970626999999, 2024-09-17T21:20:57.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093834641999999996, 2024-09-17T21:20:57.827 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:57.827 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:57.827 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:57.827 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:57.827 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:57.828 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:57.828 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15713 2024-09-17T21:20:57.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.828 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:57.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:57.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:20:57.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:57.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:20:57.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:57.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521390+0000", 2024-09-17T21:20:57.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:20:57.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:57.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521396+0000", 2024-09-17T21:20:57.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:20:57.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:57.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521417+0000", 2024-09-17T21:20:57.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1236000000000001e-05 2024-09-17T21:20:57.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:57.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521431+0000", 2024-09-17T21:20:57.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3873000000000001e-05 2024-09-17T21:20:57.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:57.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614938+0000", 2024-09-17T21:20:57.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093506522999999994 2024-09-17T21:20:57.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:57.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615025+0000", 2024-09-17T21:20:57.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7635999999999999e-05 2024-09-17T21:20:57.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:57.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615224+0000", 2024-09-17T21:20:57.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019844700000000001 2024-09-17T21:20:57.831 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.831 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:57.831 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.832 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15714 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:57.832 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:20:57.832 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.248907371000001, 2024-09-17T21:20:57.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094071925000000001, 2024-09-17T21:20:57.832 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:57.832 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:57.832 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:57.832 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:57.832 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:57.832 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:57.832 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15714 2024-09-17T21:20:57.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.832 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:57.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.833 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:57.833 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:20:57.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.833 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:57.833 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:20:57.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.833 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:57.833 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521454+0000", 2024-09-17T21:20:57.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.097e-06 2024-09-17T21:20:57.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:57.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521459+0000", 2024-09-17T21:20:57.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2660000000000001e-06 2024-09-17T21:20:57.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:57.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521481+0000", 2024-09-17T21:20:57.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1878999999999998e-05 2024-09-17T21:20:57.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:57.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521498+0000", 2024-09-17T21:20:57.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6807e-05 2024-09-17T21:20:57.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:57.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615271+0000", 2024-09-17T21:20:57.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093773286999999997 2024-09-17T21:20:57.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:57.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615366+0000", 2024-09-17T21:20:57.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5322000000000007e-05 2024-09-17T21:20:57.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:57.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615524+0000", 2024-09-17T21:20:57.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015826699999999999 2024-09-17T21:20:57.836 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.836 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:57.836 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.836 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15715 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1014784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:57.836 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:20:57.836 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.248843035999997, 2024-09-17T21:20:57.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094243840999999995, 2024-09-17T21:20:57.837 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:57.837 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:57.837 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:57.837 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:57.837 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:57.837 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:57.837 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15715 2024-09-17T21:20:57.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.837 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:57.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:57.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:20:57.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:57.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:20:57.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:57.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521518+0000", 2024-09-17T21:20:57.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4300000000000001e-06 2024-09-17T21:20:57.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:57.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521524+0000", 2024-09-17T21:20:57.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5860000000000004e-06 2024-09-17T21:20:57.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:57.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521541+0000", 2024-09-17T21:20:57.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7574000000000001e-05 2024-09-17T21:20:57.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:57.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521555+0000", 2024-09-17T21:20:57.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3540000000000001e-05 2024-09-17T21:20:57.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:57.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615560+0000", 2024-09-17T21:20:57.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.09400472 2024-09-17T21:20:57.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:57.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615614+0000", 2024-09-17T21:20:57.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4693e-05 2024-09-17T21:20:57.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:57.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615761+0000", 2024-09-17T21:20:57.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000146298 2024-09-17T21:20:57.840 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.840 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:57.840 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.841 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15716 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1018880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:57.841 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:20:57.841 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.248786690999999, 2024-09-17T21:20:57.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094400676000000003, 2024-09-17T21:20:57.841 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:57.841 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:57.841 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:57.841 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:57.841 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:57.841 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:57.841 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15716 2024-09-17T21:20:57.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.842 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:57.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:57.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:20:57.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:57.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:20:57.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:57.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521574+0000", 2024-09-17T21:20:57.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1939999999999999e-06 2024-09-17T21:20:57.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:57.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521579+0000", 2024-09-17T21:20:57.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:20:57.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:57.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521597+0000", 2024-09-17T21:20:57.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8074e-05 2024-09-17T21:20:57.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:57.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521613+0000", 2024-09-17T21:20:57.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5316e-05 2024-09-17T21:20:57.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:57.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615780+0000", 2024-09-17T21:20:57.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094166874999999997 2024-09-17T21:20:57.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:57.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615836+0000", 2024-09-17T21:20:57.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6926999999999999e-05 2024-09-17T21:20:57.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:57.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615974+0000", 2024-09-17T21:20:57.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013741400000000001 2024-09-17T21:20:57.845 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.845 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:57.845 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.845 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15717 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1022976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:57.845 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:20:57.845 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.248727995000003, 2024-09-17T21:20:57.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094586723999999997, 2024-09-17T21:20:57.846 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:57.846 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:57.846 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:57.846 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:57.846 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:57.846 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:57.846 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15717 2024-09-17T21:20:57.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.846 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:57.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:57.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:20:57.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:57.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:20:57.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:57.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521633+0000", 2024-09-17T21:20:57.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:20:57.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:57.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521638+0000", 2024-09-17T21:20:57.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:20:57.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:57.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521659+0000", 2024-09-17T21:20:57.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-05 2024-09-17T21:20:57.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:57.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521675+0000", 2024-09-17T21:20:57.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5688999999999998e-05 2024-09-17T21:20:57.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:57.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616008+0000", 2024-09-17T21:20:57.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094333314000000001 2024-09-17T21:20:57.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:57.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616066+0000", 2024-09-17T21:20:57.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7797000000000002e-05 2024-09-17T21:20:57.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:57.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616219+0000", 2024-09-17T21:20:57.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015281100000000001 2024-09-17T21:20:57.849 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.850 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:57.850 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.850 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15718 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:57.850 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:20:57.850 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.248627919999997, 2024-09-17T21:20:57.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094703674000000002, 2024-09-17T21:20:57.850 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:57.850 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:57.850 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:57.850 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:57.850 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:57.850 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:57.851 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15718 2024-09-17T21:20:57.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.851 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:57.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:57.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:20:57.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:57.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:20:57.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:57.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521734+0000", 2024-09-17T21:20:57.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.61e-06 2024-09-17T21:20:57.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:57.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521740+0000", 2024-09-17T21:20:57.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:20:57.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:57.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521754+0000", 2024-09-17T21:20:57.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3793e-05 2024-09-17T21:20:57.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:57.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521763+0000", 2024-09-17T21:20:57.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5129999999999996e-06 2024-09-17T21:20:57.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:57.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616256+0000", 2024-09-17T21:20:57.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094492919999999994 2024-09-17T21:20:57.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:57.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616312+0000", 2024-09-17T21:20:57.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6737000000000002e-05 2024-09-17T21:20:57.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:57.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616436+0000", 2024-09-17T21:20:57.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012335800000000001 2024-09-17T21:20:57.854 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.854 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:57.854 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.854 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15719 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1031168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:57.854 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:20:57.855 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.248574515000001, 2024-09-17T21:20:57.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094888963000000007, 2024-09-17T21:20:57.855 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:57.855 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:57.855 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:57.855 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:57.855 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:57.855 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:57.855 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15719 2024-09-17T21:20:57.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.855 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:57.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:57.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:20:57.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:57.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:20:57.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:57.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521786+0000", 2024-09-17T21:20:57.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.074e-06 2024-09-17T21:20:57.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:57.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521791+0000", 2024-09-17T21:20:57.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3030000000000001e-06 2024-09-17T21:20:57.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:57.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521804+0000", 2024-09-17T21:20:57.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3027000000000001e-05 2024-09-17T21:20:57.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:57.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521811+0000", 2024-09-17T21:20:57.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6090000000000003e-06 2024-09-17T21:20:57.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:57.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616466+0000", 2024-09-17T21:20:57.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094654294 2024-09-17T21:20:57.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:57.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616523+0000", 2024-09-17T21:20:57.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6953999999999997e-05 2024-09-17T21:20:57.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:57.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616674+0000", 2024-09-17T21:20:57.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015170200000000001 2024-09-17T21:20:57.858 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.859 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:57.859 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.859 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15720 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1035264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:57.859 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:20:57.859 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.248537675000001, 2024-09-17T21:20:57.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095067061999999994, 2024-09-17T21:20:57.859 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:57.859 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:57.859 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:57.859 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:57.859 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:57.859 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:57.860 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15720 2024-09-17T21:20:57.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.860 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:57.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:57.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:20:57.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:57.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:20:57.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:57.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521823+0000", 2024-09-17T21:20:57.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:20:57.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:57.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521826+0000", 2024-09-17T21:20:57.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2100000000000002e-06 2024-09-17T21:20:57.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:57.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521835+0000", 2024-09-17T21:20:57.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1630000000000002e-06 2024-09-17T21:20:57.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:57.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521842+0000", 2024-09-17T21:20:57.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:20:57.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:57.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616711+0000", 2024-09-17T21:20:57.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094868859 2024-09-17T21:20:57.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:57.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616764+0000", 2024-09-17T21:20:57.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2778999999999997e-05 2024-09-17T21:20:57.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:57.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616889+0000", 2024-09-17T21:20:57.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012533100000000001 2024-09-17T21:20:57.863 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.863 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:57.863 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.863 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15721 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1039360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:57.863 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:20:57.863 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.248490566000001, 2024-09-17T21:20:57.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095277219999999996, 2024-09-17T21:20:57.864 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:57.864 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:57.864 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:57.864 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:57.864 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:57.864 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:57.864 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15721 2024-09-17T21:20:57.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.864 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:57.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:57.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:20:57.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:57.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:20:57.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:57.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521871+0000", 2024-09-17T21:20:57.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.443e-06 2024-09-17T21:20:57.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:57.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521876+0000", 2024-09-17T21:20:57.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0300000000000001e-06 2024-09-17T21:20:57.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:57.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521886+0000", 2024-09-17T21:20:57.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186e-05 2024-09-17T21:20:57.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:57.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521893+0000", 2024-09-17T21:20:57.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.08e-06 2024-09-17T21:20:57.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:57.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616918+0000", 2024-09-17T21:20:57.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095025223000000006 2024-09-17T21:20:57.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:57.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616977+0000", 2024-09-17T21:20:57.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8403999999999999e-05 2024-09-17T21:20:57.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:57.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617147+0000", 2024-09-17T21:20:57.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016985399999999999 2024-09-17T21:20:57.867 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.867 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:57.868 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.868 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15722 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:57.868 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:20:57.868 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.248456117000003, 2024-09-17T21:20:57.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095459783000000006, 2024-09-17T21:20:57.868 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:57.868 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:57.868 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:57.868 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:57.868 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:57.868 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:57.869 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15722 2024-09-17T21:20:57.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.869 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:57.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.869 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:57.869 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:20:57.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.869 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:57.869 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:20:57.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:57.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:20:57.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7700000000000004e-07 2024-09-17T21:20:57.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:57.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521907+0000", 2024-09-17T21:20:57.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.813e-06 2024-09-17T21:20:57.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:57.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521922+0000", 2024-09-17T21:20:57.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4533e-05 2024-09-17T21:20:57.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:57.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521934+0000", 2024-09-17T21:20:57.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2469999999999999e-05 2024-09-17T21:20:57.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:57.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617187+0000", 2024-09-17T21:20:57.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095252311000000006 2024-09-17T21:20:57.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:57.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617240+0000", 2024-09-17T21:20:57.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3740999999999999e-05 2024-09-17T21:20:57.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:57.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617364+0000", 2024-09-17T21:20:57.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-17T21:20:57.872 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.872 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:57.872 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.872 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15723 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:57.872 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:20:57.872 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.248412250999998, 2024-09-17T21:20:57.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095653563999999996, 2024-09-17T21:20:57.873 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:57.873 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:57.873 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:57.873 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:57.873 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:20:57.873 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:57.873 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15723 2024-09-17T21:20:57.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.873 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:57.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:57.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:20:57.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.874 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:57.874 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:20:57.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.874 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:57.874 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521949+0000", 2024-09-17T21:20:57.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:20:57.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.874 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:57.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521953+0000", 2024-09-17T21:20:57.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5870000000000003e-06 2024-09-17T21:20:57.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:57.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521970+0000", 2024-09-17T21:20:57.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7282000000000002e-05 2024-09-17T21:20:57.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:57.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521984+0000", 2024-09-17T21:20:57.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:20:57.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:57.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617383+0000", 2024-09-17T21:20:57.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095399063000000006 2024-09-17T21:20:57.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:57.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617437+0000", 2024-09-17T21:20:57.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3931999999999998e-05 2024-09-17T21:20:57.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:57.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617601+0000", 2024-09-17T21:20:57.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016407300000000001 2024-09-17T21:20:57.877 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.877 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:57.877 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.877 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5629.0:2384 3.1c 3:38c1263e:::rbd_data.1152d37183c3.000000000000000b:head [read 11780096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-17T21:20:57.877 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:20:57.877 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.345581420999999, 2024-09-17T21:20:57.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0246185320000001, 2024-09-17T21:20:57.877 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:57.877 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:57.877 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:57.877 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:57.878 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5629", 2024-09-17T21:20:57.878 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3798711329", 2024-09-17T21:20:57.878 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2384 2024-09-17T21:20:57.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.878 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:57.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:57.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:20:57.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:57.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:20:57.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:57.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:57.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424780+0000", 2024-09-17T21:20:57.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.13e-06 2024-09-17T21:20:57.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:57.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424785+0000", 2024-09-17T21:20:57.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4739999999999996e-06 2024-09-17T21:20:57.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:57.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424798+0000", 2024-09-17T21:20:57.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2704000000000001e-05 2024-09-17T21:20:57.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:57.880 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424806+0000", 2024-09-17T21:20:57.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9899999999999997e-06 2024-09-17T21:20:57.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:57.880 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424868+0000", 2024-09-17T21:20:57.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999997e-05 2024-09-17T21:20:57.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:20:57.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424875+0000", 2024-09-17T21:20:57.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0870000000000001e-06 2024-09-17T21:20:57.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:57.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428897+0000", 2024-09-17T21:20:57.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040226080000000004 2024-09-17T21:20:57.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:20:57.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428913+0000", 2024-09-17T21:20:57.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:20:57.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:57.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.448963+0000", 2024-09-17T21:20:57.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0200498659999999 2024-09-17T21:20:57.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:57.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449158+0000", 2024-09-17T21:20:57.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019581000000000001 2024-09-17T21:20:57.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:57.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:57.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:57.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449397+0000", 2024-09-17T21:20:57.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023871 2024-09-17T21:20:57.882 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.883 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:57.883 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.883 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:57.883 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:57.883 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:57.884 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone40 from 0x5f20400 (overlap 0x2aba400) to 0x155d200 2024-09-17T21:20:57.885 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:20:57.886 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:57.876+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:57.985 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:57.978+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:58.046 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:58.046 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:58.046 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:20:58.046 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:58.059 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:20:58.086 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:58.080+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:58.187 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:58.180+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:58.229 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:58.229 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:20:58.229 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:20:58.229 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:20:58.229 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:58.245 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:20:58.291 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:58.286+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:58.374 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:20:58.374 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:20:58.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:20:58.374 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:20:58.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.374 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:20:58.374 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:58.374 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 156.76525013, 2024-09-17T21:20:58.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:20:58.374 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:58.375 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:58.375 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:58.375 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:58.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:58.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:58.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:58.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:20:58.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:58.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:20:58.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:20:58.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:58.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:20:58.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:20:58.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:58.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:20:58.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:20:58.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:20:58.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:20:58.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:58.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:20:58.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:20:58.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:58.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:20:58.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:20:58.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:58.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:20:58.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:20:58.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:58.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:20:58.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:20:58.378 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.378 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:58.378 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.379 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:20:58.379 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:58.379 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 156.755303494, 2024-09-17T21:20:58.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:20:58.379 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:58.379 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:58.379 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:58.379 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:58.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:58.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:58.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:58.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:20:58.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:58.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:20:58.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:20:58.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:58.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:20:58.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:20:58.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:58.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:20:58.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:20:58.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:20:58.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:20:58.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:58.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:20:58.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:20:58.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:58.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:20:58.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:20:58.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:58.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:20:58.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:20:58.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:58.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:20:58.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:20:58.383 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.383 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:58.383 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.383 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:20:58.383 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:58.383 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 155.512152654, 2024-09-17T21:20:58.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:20:58.383 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:58.383 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:58.383 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:58.384 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:58.384 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:20:58.384 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:20:58.384 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:20:58.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.384 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:58.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:58.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:58.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:58.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:20:58.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:58.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:20:58.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:20:58.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:58.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:20:58.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:20:58.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:58.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:20:58.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:20:58.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:20:58.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:20:58.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:58.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:20:58.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:20:58.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.387 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:58.387 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:20:58.387 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:20:58.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.387 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.387 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:58.387 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:20:58.387 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:20:58.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.387 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.387 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:58.387 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:20:58.387 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:20:58.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.387 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.388 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:58.388 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:20:58.388 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:20:58.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.388 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.388 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:58.388 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:20:58.388 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:20:58.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.388 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.388 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:58.388 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:20:58.388 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:20:58.388 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.389 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:58.389 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.389 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:20:58.389 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:58.389 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 155.50250716799999, 2024-09-17T21:20:58.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:20:58.389 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:58.389 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:58.389 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:58.389 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:58.389 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:20:58.389 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:20:58.390 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:20:58.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.390 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:58.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.390 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:58.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:58.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.390 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:58.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:20:58.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:58.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:20:58.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:20:58.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:58.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:20:58.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:20:58.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:58.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:20:58.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:20:58.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.392 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:20:58.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:20:58.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.392 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:58.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:20:58.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:20:58.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:58.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:20:58.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:20:58.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:20:58.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:20:58.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:20:58.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:20:58.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:20:58.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:20:58.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:58.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:20:58.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:20:58.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:58.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:20:58.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:20:58.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:58.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:20:58.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:20:58.395 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.395 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:58.395 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.395 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:20:58.395 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:58.395 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 142.13357113500001, 2024-09-17T21:20:58.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:20:58.395 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:58.395 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:58.396 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:58.396 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:58.396 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:20:58.396 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:20:58.396 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:20:58.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.396 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:58.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:58.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:58.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:58.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:20:58.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:58.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:20:58.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:20:58.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:58.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:20:58.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:20:58.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:20:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:20:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:20:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:20:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:20:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:20:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:58.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:20:58.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:20:58.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:58.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:20:58.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:20:58.399 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.399 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:58.399 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.399 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:819 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5111808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:58.400 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:58.400 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.931998845, 2024-09-17T21:20:58.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024344811000000001, 2024-09-17T21:20:58.400 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:58.400 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:58.400 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:58.400 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:58.400 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:58.400 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:58.400 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 819 2024-09-17T21:20:58.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.400 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:58.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:58.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:58.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:58.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437442+0000", 2024-09-17T21:20:58.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:58.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437443+0000", 2024-09-17T21:20:58.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1969999999999999e-06 2024-09-17T21:20:58.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:58.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437448+0000", 2024-09-17T21:20:58.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9490000000000004e-06 2024-09-17T21:20:58.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:58.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437467+0000", 2024-09-17T21:20:58.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8862999999999999e-05 2024-09-17T21:20:58.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437480+0000", 2024-09-17T21:20:58.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2833000000000001e-05 2024-09-17T21:20:58.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:58.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461552+0000", 2024-09-17T21:20:58.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024072695000000002 2024-09-17T21:20:58.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:58.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461639+0000", 2024-09-17T21:20:58.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6996000000000005e-05 2024-09-17T21:20:58.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.404 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:58.404 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461787+0000", 2024-09-17T21:20:58.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014727800000000001 2024-09-17T21:20:58.404 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.404 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:58.404 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.404 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:58.404 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:58.404 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.931886573, 2024-09-17T21:20:58.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:20:58.405 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:58.405 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:58.405 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:58.405 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:58.405 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:58.405 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:58.405 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:20:58.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.405 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:58.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:58.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:58.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:58.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:20:58.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:58.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:20:58.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:20:58.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:58.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:20:58.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:20:58.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:58.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:20:58.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:20:58.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:20:58.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:20:58.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.408 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:58.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:20:58.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:20:58.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.408 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:58.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:20:58.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:20:58.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.408 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:58.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:20:58.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:20:58.408 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.409 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:58.409 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.409 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:833 3.5 3:a772dfb1:::rbd_data.11523651b015.0000000000000001:d [sparse-read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:58.409 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:58.409 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.931452287, 2024-09-17T21:20:58.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024216024999999999, 2024-09-17T21:20:58.409 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:58.409 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:58.409 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:58.409 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:58.409 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:58.409 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:58.409 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 833 2024-09-17T21:20:58.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.410 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:58.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:58.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:58.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:58.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437988+0000", 2024-09-17T21:20:58.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:58.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437990+0000", 2024-09-17T21:20:58.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2500000000000001e-06 2024-09-17T21:20:58.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:58.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437995+0000", 2024-09-17T21:20:58.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1240000000000004e-06 2024-09-17T21:20:58.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:58.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438012+0000", 2024-09-17T21:20:58.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7572e-05 2024-09-17T21:20:58.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438031+0000", 2024-09-17T21:20:58.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8612000000000001e-05 2024-09-17T21:20:58.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.412 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:58.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461830+0000", 2024-09-17T21:20:58.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023799135999999999 2024-09-17T21:20:58.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.412 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:58.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462005+0000", 2024-09-17T21:20:58.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000174747 2024-09-17T21:20:58.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:58.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:20:58.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019958399999999999 2024-09-17T21:20:58.413 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.413 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:58.413 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.413 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:58.413 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:58.413 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.931391996, 2024-09-17T21:20:58.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:20:58.413 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:58.414 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:58.414 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:58.414 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:58.414 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:58.414 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:58.414 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:20:58.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.414 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:58.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:58.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:58.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:58.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:20:58.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:58.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:20:58.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:20:58.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:58.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:20:58.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:20:58.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:58.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:20:58.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:20:58.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:20:58.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:20:58.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:58.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:20:58.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:20:58.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:58.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:20:58.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:20:58.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:58.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:20:58.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:20:58.417 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.417 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:58.417 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.418 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:58.418 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:58.418 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.93133661500001, 2024-09-17T21:20:58.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:20:58.418 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:58.418 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:58.418 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:58.418 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:58.418 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:58.418 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:58.418 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:20:58.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.419 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:58.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:58.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:58.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:58.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:20:58.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:58.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:20:58.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:20:58.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:58.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:20:58.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:20:58.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:58.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:20:58.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:20:58.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:20:58.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:20:58.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:58.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:20:58.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:20:58.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:58.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:20:58.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:20:58.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:58.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:20:58.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:20:58.422 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.422 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:58.422 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.422 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:58.422 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:58.422 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.931280337, 2024-09-17T21:20:58.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:20:58.422 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:58.422 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:58.422 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:58.423 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:58.423 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:58.423 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:58.423 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:20:58.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.423 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:58.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:58.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:58.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:58.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:20:58.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:58.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:20:58.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:20:58.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:58.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:20:58.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:20:58.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:58.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:20:58.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:20:58.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:20:58.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:20:58.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:58.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:20:58.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:20:58.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:58.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:20:58.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:20:58.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.426 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:58.426 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:20:58.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:20:58.426 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.426 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:58.426 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.426 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:58.426 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:58.426 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.93033069800001, 2024-09-17T21:20:58.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:20:58.427 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:58.427 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:58.427 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:58.427 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:58.427 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:58.427 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:58.427 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:20:58.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.427 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:58.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:58.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:58.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:58.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:20:58.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:58.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:20:58.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:20:58.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:58.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:20:58.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:20:58.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:58.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:20:58.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:20:58.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:20:58.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:20:58.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:58.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:20:58.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:20:58.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:58.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:20:58.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:20:58.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:58.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:20:58.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:20:58.430 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.430 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:58.430 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.431 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:58.431 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:58.431 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.930252078, 2024-09-17T21:20:58.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:20:58.431 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:58.431 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:58.431 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:58.431 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:58.431 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:58.431 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:58.431 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:20:58.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.431 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:58.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:58.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:58.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:58.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:20:58.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:58.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:20:58.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:20:58.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:58.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:20:58.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:20:58.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:58.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:20:58.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:20:58.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:20:58.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:20:58.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:58.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:20:58.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:20:58.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:58.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:20:58.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:20:58.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:58.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:20:58.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:20:58.434 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.435 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:58.435 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.435 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:58.435 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:58.435 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.930139139, 2024-09-17T21:20:58.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:20:58.435 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:58.435 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:58.435 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:58.435 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:58.435 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:58.435 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:58.435 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:20:58.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.436 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:58.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:58.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:58.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:58.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:20:58.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:58.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:20:58.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:20:58.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:58.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:20:58.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:20:58.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:58.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:20:58.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:20:58.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:20:58.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:20:58.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:58.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:20:58.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:20:58.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:58.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:20:58.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:20:58.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:58.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:20:58.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:20:58.439 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.439 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:58.439 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.439 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:849 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5963776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:58.439 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:58.439 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.929545103, 2024-09-17T21:20:58.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024537514, 2024-09-17T21:20:58.439 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:58.439 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:58.439 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:58.439 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:58.440 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:58.440 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:58.440 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 849 2024-09-17T21:20:58.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.440 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:58.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:58.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:58.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:58.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439895+0000", 2024-09-17T21:20:58.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:58.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439897+0000", 2024-09-17T21:20:58.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9099999999999999e-06 2024-09-17T21:20:58.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:58.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439905+0000", 2024-09-17T21:20:58.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0870000000000007e-06 2024-09-17T21:20:58.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:58.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439924+0000", 2024-09-17T21:20:58.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9042000000000002e-05 2024-09-17T21:20:58.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439941+0000", 2024-09-17T21:20:58.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.624e-05 2024-09-17T21:20:58.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:58.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464132+0000", 2024-09-17T21:20:58.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024191579000000001 2024-09-17T21:20:58.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:58.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464291+0000", 2024-09-17T21:20:58.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015904800000000001 2024-09-17T21:20:58.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:58.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464433+0000", 2024-09-17T21:20:58.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014160799999999999 2024-09-17T21:20:58.443 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.443 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:58.444 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.444 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:58.444 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:58.444 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.929478085, 2024-09-17T21:20:58.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:20:58.444 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:58.444 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:58.444 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:58.444 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:58.444 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:58.444 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:58.444 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:20:58.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.445 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:58.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:58.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:58.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:58.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:20:58.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:58.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:20:58.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:20:58.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:58.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:20:58.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:20:58.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:58.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:20:58.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:20:58.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:20:58.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:20:58.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:58.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:20:58.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:20:58.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:58.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:20:58.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:20:58.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:58.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:20:58.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:20:58.448 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.448 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:58.448 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.448 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:58.448 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:58.448 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.929413274, 2024-09-17T21:20:58.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:20:58.448 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:58.448 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:58.448 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:58.448 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:58.449 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:58.449 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:58.449 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:20:58.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.449 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:58.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:58.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:58.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:58.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:20:58.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:58.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:20:58.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:20:58.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:58.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:20:58.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:20:58.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:58.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:20:58.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:20:58.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:20:58.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:20:58.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:58.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:20:58.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:20:58.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:58.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:20:58.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:20:58.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:58.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:20:58.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:20:58.452 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.452 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:58.452 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.452 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:20:58.452 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:58.452 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.92935229699999, 2024-09-17T21:20:58.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:20:58.452 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:58.452 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:58.453 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:20:58.453 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:20:58.453 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:20:58.453 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:58.453 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:20:58.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.453 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:58.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:58.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:58.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:20:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:20:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:20:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:20:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:20:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:20:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:20:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:20:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:20:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:20:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:20:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:20:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:20:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:20:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:20:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:20:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 69.031781112000004, 2024-09-17T21:20:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:20:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:20:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:20:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:20:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:20:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:20:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:20:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:20:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:20:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:20:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:20:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:20:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:20:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:20:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:20:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:20:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:20:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:20:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:20:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.823122717999993, 2024-09-17T21:20:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:20:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:20:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:20:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:20:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:20:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:20:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:20:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:20:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:20:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:20:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:20:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:20:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:20:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:20:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:20:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:20:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:20:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:20:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:20:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:20:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:20:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:20:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:20:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:20:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:20:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:20:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:20:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:20:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:20:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:20:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:20:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:20:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:20:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.465 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:20:58.465 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:20:58.465 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:20:58.466 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:20:58.467 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:58.458+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:58.567 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:58.562+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:20:58.605 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:58.605 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:58.605 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:58.606 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:58.619 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:20:58.668 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:20:58.660+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:20:58.760 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:58.760 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:58.760 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:58.760 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:58.761 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:58.769 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:58.762+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:58.775 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:20:58.869 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:58.862+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:58.941 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:525 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3670016~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 152.73707024699999, 2024-09-17T21:20:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046645697, 2024-09-17T21:20:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 525 2024-09-17T21:20:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:20:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:20:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:20:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200234+0000", 2024-09-17T21:20:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:20:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200254+0000", 2024-09-17T21:20:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9959999999999999e-05 2024-09-17T21:20:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200293+0000", 2024-09-17T21:20:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9755999999999997e-05 2024-09-17T21:20:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245871+0000", 2024-09-17T21:20:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045577871999999998 2024-09-17T21:20:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246030+0000", 2024-09-17T21:20:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158975 2024-09-17T21:20:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246871+0000", 2024-09-17T21:20:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084111699999999995 2024-09-17T21:20:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 152.73682457199999, 2024-09-17T21:20:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:20:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:20:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:20:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:20:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:20:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:20:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:20:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:20:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:20:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:20:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:20:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:20:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:20:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:20:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:20:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:20:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:20:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 152.73664589000001, 2024-09-17T21:20:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:20:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:20:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:20:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:20:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:20:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:20:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:20:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:20:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:20:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:20:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:20:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:20:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:20:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:20:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:20:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:20:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:20:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 152.736586617, 2024-09-17T21:20:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:20:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:20:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:20:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:20:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:20:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:20:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:20:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:20:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:20:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:20:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:20:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:20:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:20:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:20:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:20:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:20:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:20:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 152.73644442200001, 2024-09-17T21:20:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:20:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:20:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:20:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:20:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:20:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:20:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:20:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:20:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:20:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:20:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:20:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:20:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:20:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:20:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:20:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:20:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:20:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 152.73638849899999, 2024-09-17T21:20:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:20:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:20:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:20:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:20:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:20:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:20:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:20:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:20:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:20:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:20:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:20:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:20:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:20:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:20:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:20:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:20:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:20:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:20:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 152.73633488199999, 2024-09-17T21:20:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:20:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:20:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:20:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:20:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:20:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:20:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:20:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:20:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:20:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:20:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:20:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:20:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:20:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:20:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:20:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:20:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:20:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:20:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:20:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:20:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 102.42710664000001, 2024-09-17T21:20:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:20:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:20:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:20:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:20:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:20:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:20:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:20:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:20:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:20:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:20:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:20:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:20:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:20:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:20:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:20:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:20:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:20:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:20:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:20:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:20:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:20:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:20:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2597 3.1f 3:fb0e9615:::rbd_data.1152a8ae2019.000000000000000c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 78.098670212000002, 2024-09-17T21:20:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097064994000000002, 2024-09-17T21:20:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2597 2024-09-17T21:20:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:20:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838628+0000", 2024-09-17T21:20:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.187e-06 2024-09-17T21:20:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838654+0000", 2024-09-17T21:20:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:20:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838696+0000", 2024-09-17T21:20:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1226000000000001e-05 2024-09-17T21:20:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838715+0000", 2024-09-17T21:20:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9646000000000001e-05 2024-09-17T21:20:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838937+0000", 2024-09-17T21:20:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022163 2024-09-17T21:20:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.839284+0000", 2024-09-17T21:20:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034753799999999999 2024-09-17T21:20:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935691+0000", 2024-09-17T21:20:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.096406360999999996 2024-09-17T21:20:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2641 3.12 3:488a729f:::rbd_data.1152a8ae2019.0000000000000022:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 78.095213439000005, 2024-09-17T21:20:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098280953000000004, 2024-09-17T21:20:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2641 2024-09-17T21:20:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:20:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842083+0000", 2024-09-17T21:20:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5300000000000002e-07 2024-09-17T21:20:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842088+0000", 2024-09-17T21:20:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:20:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842105+0000", 2024-09-17T21:20:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6773e-05 2024-09-17T21:20:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842119+0000", 2024-09-17T21:20:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:20:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842320+0000", 2024-09-17T21:20:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020052700000000001 2024-09-17T21:20:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842845+0000", 2024-09-17T21:20:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052508600000000002 2024-09-17T21:20:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940363+0000", 2024-09-17T21:20:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097518795000000005 2024-09-17T21:20:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2674 3.12 3:4c87e3ac:::rbd_data.1152a8ae2019.0000000000000033:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 78.094850801999996, 2024-09-17T21:20:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098025125000000005, 2024-09-17T21:20:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2674 2024-09-17T21:20:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:20:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842447+0000", 2024-09-17T21:20:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3999999999999999e-06 2024-09-17T21:20:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842453+0000", 2024-09-17T21:20:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:20:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842472+0000", 2024-09-17T21:20:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9006000000000001e-05 2024-09-17T21:20:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842487+0000", 2024-09-17T21:20:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4866e-05 2024-09-17T21:20:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843594+0000", 2024-09-17T21:20:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011073249999999999 2024-09-17T21:20:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843668+0000", 2024-09-17T21:20:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3188000000000005e-05 2024-09-17T21:20:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843893+0000", 2024-09-17T21:20:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022503399999999999 2024-09-17T21:20:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850439+0000", 2024-09-17T21:20:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065467880000000004 2024-09-17T21:20:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940409+0000", 2024-09-17T21:20:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089969511000000002 2024-09-17T21:20:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940444+0000", 2024-09-17T21:20:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5345999999999997e-05 2024-09-17T21:20:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940470+0000", 2024-09-17T21:20:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5964000000000001e-05 2024-09-17T21:20:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2707 3.1f 3:f805e65e:::rbd_data.1152a8ae2019.0000000000000049:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 78.089804141000002, 2024-09-17T21:20:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088327300999999997, 2024-09-17T21:20:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2707 2024-09-17T21:20:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:58.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:20:58.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:58.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847494+0000", 2024-09-17T21:20:58.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:20:58.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:58.994 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847504+0000", 2024-09-17T21:20:58.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:20:58.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.994 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:58.994 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847526+0000", 2024-09-17T21:20:58.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1976000000000001e-05 2024-09-17T21:20:58.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.994 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.994 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847541+0000", 2024-09-17T21:20:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5277e-05 2024-09-17T21:20:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856142+0000", 2024-09-17T21:20:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086006710000000007 2024-09-17T21:20:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856435+0000", 2024-09-17T21:20:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029301499999999999 2024-09-17T21:20:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:58.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935819+0000", 2024-09-17T21:20:58.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079384448999999996 2024-09-17T21:20:58.996 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.996 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:58.996 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:58.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.996 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2730 3.12 3:48b3422a:::rbd_data.1152a8ae2019.0000000000000057:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:58.996 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:58.996 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 78.080611242000003, 2024-09-17T21:20:58.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083879975999999995, 2024-09-17T21:20:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2730 2024-09-17T21:20:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:20:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856687+0000", 2024-09-17T21:20:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:20:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856697+0000", 2024-09-17T21:20:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0207000000000001e-05 2024-09-17T21:20:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856808+0000", 2024-09-17T21:20:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011071200000000001 2024-09-17T21:20:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856828+0000", 2024-09-17T21:20:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0219000000000001e-05 2024-09-17T21:20:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856907+0000", 2024-09-17T21:20:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9661999999999997e-05 2024-09-17T21:20:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.857019+0000", 2024-09-17T21:20:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011148499999999999 2024-09-17T21:20:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940565+0000", 2024-09-17T21:20:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083545791999999994 2024-09-17T21:20:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2743 3.12 3:4d036e62:::rbd_data.1152a8ae2019.000000000000005f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 78.076665740999999, 2024-09-17T21:20:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080059962999999998, 2024-09-17T21:20:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2743 2024-09-17T21:20:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:20:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860631+0000", 2024-09-17T21:20:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:20:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860636+0000", 2024-09-17T21:20:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5279999999999997e-06 2024-09-17T21:20:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860649+0000", 2024-09-17T21:20:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-17T21:20:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860655+0000", 2024-09-17T21:20:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:20:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860731+0000", 2024-09-17T21:20:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5920000000000005e-05 2024-09-17T21:20:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.861070+0000", 2024-09-17T21:20:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338535 2024-09-17T21:20:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940690+0000", 2024-09-17T21:20:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079620503999999995 2024-09-17T21:20:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3378 3.12 3:4832a51b:::rbd_data.1152a8ae2019.000000000000006c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 78.048301890999994, 2024-09-17T21:20:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051812516000000003, 2024-09-17T21:20:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3378 2024-09-17T21:20:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:20:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888995+0000", 2024-09-17T21:20:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:20:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889006+0000", 2024-09-17T21:20:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1012999999999999e-05 2024-09-17T21:20:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889032+0000", 2024-09-17T21:20:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6441e-05 2024-09-17T21:20:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889040+0000", 2024-09-17T21:20:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0639999999999994e-06 2024-09-17T21:20:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889278+0000", 2024-09-17T21:20:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023785899999999999 2024-09-17T21:20:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889408+0000", 2024-09-17T21:20:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129219 2024-09-17T21:20:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889723+0000", 2024-09-17T21:20:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000315264 2024-09-17T21:20:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.891972+0000", 2024-09-17T21:20:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002249633 2024-09-17T21:20:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940733+0000", 2024-09-17T21:20:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048760626000000001 2024-09-17T21:20:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940785+0000", 2024-09-17T21:20:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1906000000000001e-05 2024-09-17T21:20:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940807+0000", 2024-09-17T21:20:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1554999999999999e-05 2024-09-17T21:20:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3379 3.12 3:4e83fd71:::rbd_data.1152a8ae2019.000000000000006d:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:20:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 78.048217683999994, 2024-09-17T21:20:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051804144000000003, 2024-09-17T21:20:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:20:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:20:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3379 2024-09-17T21:20:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:20:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889079+0000", 2024-09-17T21:20:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1800000000000004e-07 2024-09-17T21:20:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889083+0000", 2024-09-17T21:20:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:20:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889094+0000", 2024-09-17T21:20:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1562999999999999e-05 2024-09-17T21:20:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889102+0000", 2024-09-17T21:20:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.401e-06 2024-09-17T21:20:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:20:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014279259999999999 2024-09-17T21:20:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890609+0000", 2024-09-17T21:20:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9566000000000004e-05 2024-09-17T21:20:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890789+0000", 2024-09-17T21:20:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017949700000000001 2024-09-17T21:20:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.893929+0000", 2024-09-17T21:20:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003140538 2024-09-17T21:20:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940834+0000", 2024-09-17T21:20:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046904811999999997 2024-09-17T21:20:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940860+0000", 2024-09-17T21:20:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5341999999999999e-05 2024-09-17T21:20:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940882+0000", 2024-09-17T21:20:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2878999999999999e-05 2024-09-17T21:20:59.017 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.017 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.017 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.017 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:2644 3.1e 3:79615585:::rbd_data.115266d0b9f2.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2475520~14848,write 3106304~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:20:59.017 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:59.017 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 69.425589281000001, 2024-09-17T21:20:59.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050733484000000002, 2024-09-17T21:20:59.017 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:20:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:20:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2644 2024-09-17T21:20:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:20:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511708+0000", 2024-09-17T21:20:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-17T21:20:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522196+0000", 2024-09-17T21:20:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010487712999999999 2024-09-17T21:20:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522223+0000", 2024-09-17T21:20:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7064999999999999e-05 2024-09-17T21:20:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522236+0000", 2024-09-17T21:20:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2931e-05 2024-09-17T21:20:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522348+0000", 2024-09-17T21:20:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111906 2024-09-17T21:20:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522554+0000", 2024-09-17T21:20:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020632699999999999 2024-09-17T21:20:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:20:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.523185+0000", 2024-09-17T21:20:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063081400000000005 2024-09-17T21:20:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.547011+0000", 2024-09-17T21:20:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023825570000000001 2024-09-17T21:20:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562268+0000", 2024-09-17T21:20:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015257514 2024-09-17T21:20:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562367+0000", 2024-09-17T21:20:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8678000000000006e-05 2024-09-17T21:20:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562440+0000", 2024-09-17T21:20:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3337000000000006e-05 2024-09-17T21:20:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:231 3.1 3:856fed78:::rbd_data.115220dd393e.000000000000000b:head [read 0~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:20:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:20:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.982834011999998, 2024-09-17T21:20:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34254251699999999, 2024-09-17T21:20:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:20:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 231 2024-09-17T21:20:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:20:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:20:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954464+0000", 2024-09-17T21:20:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:20:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954475+0000", 2024-09-17T21:20:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1146e-05 2024-09-17T21:20:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954506+0000", 2024-09-17T21:20:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0292999999999999e-05 2024-09-17T21:20:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954524+0000", 2024-09-17T21:20:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8430000000000001e-05 2024-09-17T21:20:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954661+0000", 2024-09-17T21:20:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136403 2024-09-17T21:20:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954680+0000", 2024-09-17T21:20:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9743000000000001e-05 2024-09-17T21:20:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.296896+0000", 2024-09-17T21:20:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34221550499999998 2024-09-17T21:20:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297004+0000", 2024-09-17T21:20:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010865399999999999 2024-09-17T21:20:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:56 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:20:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:20:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.827200314999999, 2024-09-17T21:20:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18718773899999999, 2024-09-17T21:20:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:20:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:20:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 56 2024-09-17T21:20:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:20:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:20:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110098+0000", 2024-09-17T21:20:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:20:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110111+0000", 2024-09-17T21:20:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2289999999999999e-05 2024-09-17T21:20:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110143+0000", 2024-09-17T21:20:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1979e-05 2024-09-17T21:20:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110161+0000", 2024-09-17T21:20:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7839e-05 2024-09-17T21:20:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110319+0000", 2024-09-17T21:20:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015835400000000001 2024-09-17T21:20:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110353+0000", 2024-09-17T21:20:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3849000000000001e-05 2024-09-17T21:20:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297041+0000", 2024-09-17T21:20:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18668842299999999 2024-09-17T21:20:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297169+0000", 2024-09-17T21:20:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012786199999999999 2024-09-17T21:20:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297283+0000", 2024-09-17T21:20:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011430300000000001 2024-09-17T21:20:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:68 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:20:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:20:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.818659384, 2024-09-17T21:20:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17885582999999999, 2024-09-17T21:20:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:20:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:20:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 68 2024-09-17T21:20:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:20:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:20:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118639+0000", 2024-09-17T21:20:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.306e-06 2024-09-17T21:20:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118651+0000", 2024-09-17T21:20:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2567e-05 2024-09-17T21:20:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118682+0000", 2024-09-17T21:20:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0502e-05 2024-09-17T21:20:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118707+0000", 2024-09-17T21:20:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5463e-05 2024-09-17T21:20:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118889+0000", 2024-09-17T21:20:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018150999999999999 2024-09-17T21:20:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118910+0000", 2024-09-17T21:20:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:20:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297313+0000", 2024-09-17T21:20:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17840366699999999 2024-09-17T21:20:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297389+0000", 2024-09-17T21:20:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-17T21:20:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.038 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297492+0000", 2024-09-17T21:20:59.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103019 2024-09-17T21:20:59.038 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.038 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.038 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.038 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.038 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.038 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:59.039 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:20:59.040 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:59.034+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:59.120 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-17T21:20:59.140 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:59.132+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:59.171 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:59.171 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:59.171 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:20:59.171 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:59.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:20:59.240 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:59.234+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:59.342 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:59.336+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:59.389 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:59.389 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:20:59.390 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:20:59.390 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:20:59.390 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:59.404 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:20:59.445 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:20:59.440+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:20:59.504 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:20:59.542 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:20:59.534+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:20:59.555 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:20:59.588 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:20:59.588 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:20:59.588 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:20:59.588 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:20:59.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.588 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:20:59.588 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:59.588 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 139.77607226500001, 2024-09-17T21:20:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:20:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:20:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:20:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:20:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:20:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:20:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:20:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:20:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:20:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:20:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:20:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:20:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:20:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:20:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:20:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:20:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:20:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:20:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:20:59.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.593 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.593 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:20:59.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:20:59.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.593 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.593 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:20:59.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:20:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:20:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 126.220970479, 2024-09-17T21:20:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:20:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:20:59.595 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:20:59.595 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:20:59.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.595 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.595 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.595 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:59.595 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:20:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:20:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:20:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:20:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:20:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:20:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:20:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:20:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:20:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:20:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:20:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:20:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:20:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:20:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:20:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:20:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:20:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:20:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:20:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:20:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:20:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:20:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:20:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:20:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:20:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:20:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:20:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:20:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:793 3.9 3:95f63599:::rbd_data.1152eb799094.0000000000000007:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:20:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:20:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 43.153914323999999, 2024-09-17T21:20:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88245766400000003, 2024-09-17T21:20:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:20:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:20:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 793 2024-09-17T21:20:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:20:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:20:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431941+0000", 2024-09-17T21:20:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.864e-06 2024-09-17T21:20:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431953+0000", 2024-09-17T21:20:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1486e-05 2024-09-17T21:20:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431984+0000", 2024-09-17T21:20:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1075999999999999e-05 2024-09-17T21:20:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431999+0000", 2024-09-17T21:20:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.503e-05 2024-09-17T21:20:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.605 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.605 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432095+0000", 2024-09-17T21:20:59.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5698000000000004e-05 2024-09-17T21:20:59.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.605 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:59.605 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432109+0000", 2024-09-17T21:20:59.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4188e-05 2024-09-17T21:20:59.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.309642+0000", 2024-09-17T21:20:59.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87753353000000001 2024-09-17T21:20:59.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310104+0000", 2024-09-17T21:20:59.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046103600000000001 2024-09-17T21:20:59.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310630+0000", 2024-09-17T21:20:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052682800000000002 2024-09-17T21:20:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313100+0000", 2024-09-17T21:20:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024693369999999998 2024-09-17T21:20:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313935+0000", 2024-09-17T21:20:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083540100000000005 2024-09-17T21:20:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314086+0000", 2024-09-17T21:20:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015053399999999999 2024-09-17T21:20:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314396+0000", 2024-09-17T21:20:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031065599999999997 2024-09-17T21:20:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:800 3.19 3:9dc17f12:::rbd_data.1152eb799094.000000000000000e:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:20:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:20:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 43.153667079000002, 2024-09-17T21:20:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88154663799999999, 2024-09-17T21:20:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:20:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:20:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 800 2024-09-17T21:20:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.610 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:20:59.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:20:59.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432187+0000", 2024-09-17T21:20:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4100000000000001e-06 2024-09-17T21:20:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432194+0000", 2024-09-17T21:20:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0389999999999999e-06 2024-09-17T21:20:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432216+0000", 2024-09-17T21:20:59.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1614e-05 2024-09-17T21:20:59.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432232+0000", 2024-09-17T21:20:59.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5739e-05 2024-09-17T21:20:59.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432405+0000", 2024-09-17T21:20:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017355300000000001 2024-09-17T21:20:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432432+0000", 2024-09-17T21:20:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6979e-05 2024-09-17T21:20:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.307767+0000", 2024-09-17T21:20:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87533472899999998 2024-09-17T21:20:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308192+0000", 2024-09-17T21:20:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042485999999999999 2024-09-17T21:20:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308727+0000", 2024-09-17T21:20:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00053550400000000001 2024-09-17T21:20:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310347+0000", 2024-09-17T21:20:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016198670000000001 2024-09-17T21:20:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313424+0000", 2024-09-17T21:20:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030764640000000001 2024-09-17T21:20:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313510+0000", 2024-09-17T21:20:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5938000000000003e-05 2024-09-17T21:20:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313733+0000", 2024-09-17T21:20:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022294200000000001 2024-09-17T21:20:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:805 3.19 3:9cab2ed6:::rbd_data.1152eb799094.0000000000000013:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:20:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:20:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 43.15346306, 2024-09-17T21:20:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88274355699999996, 2024-09-17T21:20:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:20:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:20:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:20:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 805 2024-09-17T21:20:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:20:59.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:20:59.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432391+0000", 2024-09-17T21:20:59.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3e-06 2024-09-17T21:20:59.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432398+0000", 2024-09-17T21:20:59.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4489999999999996e-06 2024-09-17T21:20:59.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432415+0000", 2024-09-17T21:20:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7750000000000001e-05 2024-09-17T21:20:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432430+0000", 2024-09-17T21:20:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.449e-05 2024-09-17T21:20:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432635+0000", 2024-09-17T21:20:59.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205031 2024-09-17T21:20:59.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:20:59.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432657+0000", 2024-09-17T21:20:59.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2064000000000001e-05 2024-09-17T21:20:59.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310444+0000", 2024-09-17T21:20:59.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87778699199999999 2024-09-17T21:20:59.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310849+0000", 2024-09-17T21:20:59.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00040497800000000001 2024-09-17T21:20:59.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:20:59.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.311396+0000", 2024-09-17T21:20:59.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00054729099999999999 2024-09-17T21:20:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:20:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313847+0000", 2024-09-17T21:20:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024504940000000001 2024-09-17T21:20:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:20:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314879+0000", 2024-09-17T21:20:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001031817 2024-09-17T21:20:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:20:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314964+0000", 2024-09-17T21:20:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5685000000000004e-05 2024-09-17T21:20:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.315133+0000", 2024-09-17T21:20:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000169216 2024-09-17T21:20:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12087 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 52224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:20:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.816374381999999, 2024-09-17T21:20:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087684460000000006, 2024-09-17T21:20:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12087 2024-09-17T21:20:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:20:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:20:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769480+0000", 2024-09-17T21:20:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:20:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769485+0000", 2024-09-17T21:20:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8880000000000002e-06 2024-09-17T21:20:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769509+0000", 2024-09-17T21:20:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3343e-05 2024-09-17T21:20:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769521+0000", 2024-09-17T21:20:59.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2497000000000001e-05 2024-09-17T21:20:59.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.856883+0000", 2024-09-17T21:20:59.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087362191000000006 2024-09-17T21:20:59.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857014+0000", 2024-09-17T21:20:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013090399999999999 2024-09-17T21:20:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857163+0000", 2024-09-17T21:20:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014864000000000001 2024-09-17T21:20:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12088 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 56320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:20:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.816290727000002, 2024-09-17T21:20:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087917772000000005, 2024-09-17T21:20:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12088 2024-09-17T21:20:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:20:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:20:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769569+0000", 2024-09-17T21:20:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:20:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769574+0000", 2024-09-17T21:20:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4659999999999998e-06 2024-09-17T21:20:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769586+0000", 2024-09-17T21:20:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2311e-05 2024-09-17T21:20:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769597+0000", 2024-09-17T21:20:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0548e-05 2024-09-17T21:20:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857217+0000", 2024-09-17T21:20:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087619896000000003 2024-09-17T21:20:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857335+0000", 2024-09-17T21:20:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118538 2024-09-17T21:20:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857480+0000", 2024-09-17T21:20:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014470999999999999 2024-09-17T21:20:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12089 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 60416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:20:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.816238028000001, 2024-09-17T21:20:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088293233999999998, 2024-09-17T21:20:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12089 2024-09-17T21:20:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:20:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:20:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769616+0000", 2024-09-17T21:20:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:20:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769620+0000", 2024-09-17T21:20:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5140000000000002e-06 2024-09-17T21:20:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769633+0000", 2024-09-17T21:20:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2849e-05 2024-09-17T21:20:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769644+0000", 2024-09-17T21:20:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1471000000000001e-05 2024-09-17T21:20:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857564+0000", 2024-09-17T21:20:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087919445999999998 2024-09-17T21:20:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857716+0000", 2024-09-17T21:20:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000152065 2024-09-17T21:20:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857908+0000", 2024-09-17T21:20:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019252599999999999 2024-09-17T21:20:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12090 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 64512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:20:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.816191643000003, 2024-09-17T21:20:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088582672000000001, 2024-09-17T21:20:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12090 2024-09-17T21:20:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:20:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:20:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769662+0000", 2024-09-17T21:20:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:20:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769683+0000", 2024-09-17T21:20:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0979e-05 2024-09-17T21:20:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769699+0000", 2024-09-17T21:20:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5446000000000001e-05 2024-09-17T21:20:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769708+0000", 2024-09-17T21:20:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1679999999999993e-06 2024-09-17T21:20:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857976+0000", 2024-09-17T21:20:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088267795999999996 2024-09-17T21:20:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858095+0000", 2024-09-17T21:20:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119147 2024-09-17T21:20:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858244+0000", 2024-09-17T21:20:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014917000000000001 2024-09-17T21:20:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12091 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 68608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:20:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.816120259000002, 2024-09-17T21:20:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088832752000000001, 2024-09-17T21:20:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12091 2024-09-17T21:20:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:20:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:20:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769734+0000", 2024-09-17T21:20:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:20:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769739+0000", 2024-09-17T21:20:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8080000000000004e-06 2024-09-17T21:20:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769764+0000", 2024-09-17T21:20:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4839999999999999e-05 2024-09-17T21:20:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769777+0000", 2024-09-17T21:20:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3037000000000001e-05 2024-09-17T21:20:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858295+0000", 2024-09-17T21:20:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088518475999999999 2024-09-17T21:20:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858419+0000", 2024-09-17T21:20:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012378700000000001 2024-09-17T21:20:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858565+0000", 2024-09-17T21:20:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014630099999999999 2024-09-17T21:20:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12092 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:20:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.816037934999997, 2024-09-17T21:20:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089191555000000006, 2024-09-17T21:20:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12092 2024-09-17T21:20:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:20:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:20:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769816+0000", 2024-09-17T21:20:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7199999999999997e-07 2024-09-17T21:20:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769822+0000", 2024-09-17T21:20:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:20:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769838+0000", 2024-09-17T21:20:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6674e-05 2024-09-17T21:20:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769851+0000", 2024-09-17T21:20:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2527999999999999e-05 2024-09-17T21:20:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858660+0000", 2024-09-17T21:20:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088808556999999996 2024-09-17T21:20:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858817+0000", 2024-09-17T21:20:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015730400000000001 2024-09-17T21:20:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859007+0000", 2024-09-17T21:20:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018974899999999999 2024-09-17T21:20:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12093 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:20:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.815929830999998, 2024-09-17T21:20:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089429710999999995, 2024-09-17T21:20:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12093 2024-09-17T21:20:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:20:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:20:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769924+0000", 2024-09-17T21:20:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.342e-06 2024-09-17T21:20:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769929+0000", 2024-09-17T21:20:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0350000000000001e-06 2024-09-17T21:20:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769946+0000", 2024-09-17T21:20:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6793999999999999e-05 2024-09-17T21:20:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769959+0000", 2024-09-17T21:20:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3062e-05 2024-09-17T21:20:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859075+0000", 2024-09-17T21:20:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089115696999999994 2024-09-17T21:20:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859202+0000", 2024-09-17T21:20:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012657 2024-09-17T21:20:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859353+0000", 2024-09-17T21:20:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151211 2024-09-17T21:20:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12094 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:20:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.815875654999999, 2024-09-17T21:20:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089695001999999996, 2024-09-17T21:20:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.659 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:59.659 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:59.659 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12094 2024-09-17T21:20:59.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.659 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:20:59.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:20:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769978+0000", 2024-09-17T21:20:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.142e-06 2024-09-17T21:20:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769983+0000", 2024-09-17T21:20:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.386e-06 2024-09-17T21:20:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769997+0000", 2024-09-17T21:20:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.416e-05 2024-09-17T21:20:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770012+0000", 2024-09-17T21:20:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5384999999999999e-05 2024-09-17T21:20:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859405+0000", 2024-09-17T21:20:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089392376999999995 2024-09-17T21:20:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859525+0000", 2024-09-17T21:20:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120555 2024-09-17T21:20:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859672+0000", 2024-09-17T21:20:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000146997 2024-09-17T21:20:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12095 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:20:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.815823168000001, 2024-09-17T21:20:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090073476, 2024-09-17T21:20:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12095 2024-09-17T21:20:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:20:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:20:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770031+0000", 2024-09-17T21:20:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1699999999999997e-07 2024-09-17T21:20:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770036+0000", 2024-09-17T21:20:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.023e-06 2024-09-17T21:20:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770051+0000", 2024-09-17T21:20:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4953999999999999e-05 2024-09-17T21:20:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770063+0000", 2024-09-17T21:20:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:20:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859756+0000", 2024-09-17T21:20:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089692980000000005 2024-09-17T21:20:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859891+0000", 2024-09-17T21:20:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013459499999999999 2024-09-17T21:20:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860103+0000", 2024-09-17T21:20:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021274400000000001 2024-09-17T21:20:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12096 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:20:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.815773483000001, 2024-09-17T21:20:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090358274000000002, 2024-09-17T21:20:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12096 2024-09-17T21:20:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:20:59.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:20:59.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770080+0000", 2024-09-17T21:20:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9400000000000004e-07 2024-09-17T21:20:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770085+0000", 2024-09-17T21:20:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2169999999999996e-06 2024-09-17T21:20:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770100+0000", 2024-09-17T21:20:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5077e-05 2024-09-17T21:20:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770112+0000", 2024-09-17T21:20:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1951e-05 2024-09-17T21:20:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860160+0000", 2024-09-17T21:20:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090048413999999993 2024-09-17T21:20:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860285+0000", 2024-09-17T21:20:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012466699999999999 2024-09-17T21:20:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860438+0000", 2024-09-17T21:20:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015305399999999999 2024-09-17T21:20:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12097 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:20:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.815725735999997, 2024-09-17T21:20:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090702375000000002, 2024-09-17T21:20:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12097 2024-09-17T21:20:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:20:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:20:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770128+0000", 2024-09-17T21:20:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.04e-06 2024-09-17T21:20:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770133+0000", 2024-09-17T21:20:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8999999999999997e-06 2024-09-17T21:20:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770152+0000", 2024-09-17T21:20:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8890999999999999e-05 2024-09-17T21:20:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770164+0000", 2024-09-17T21:20:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1769e-05 2024-09-17T21:20:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860511+0000", 2024-09-17T21:20:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090347527999999996 2024-09-17T21:20:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860652+0000", 2024-09-17T21:20:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140894 2024-09-17T21:20:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860830+0000", 2024-09-17T21:20:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017735299999999999 2024-09-17T21:20:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12098 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:20:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.815672268, 2024-09-17T21:20:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090997262999999995, 2024-09-17T21:20:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12098 2024-09-17T21:20:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:20:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:20:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775465+0000", 2024-09-17T21:20:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0052843350000000002 2024-09-17T21:20:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775476+0000", 2024-09-17T21:20:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0797e-05 2024-09-17T21:20:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775506+0000", 2024-09-17T21:20:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9709999999999998e-05 2024-09-17T21:20:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775530+0000", 2024-09-17T21:20:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4528999999999998e-05 2024-09-17T21:20:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860894+0000", 2024-09-17T21:20:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.085364195000000004 2024-09-17T21:20:59.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861019+0000", 2024-09-17T21:20:59.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012467399999999999 2024-09-17T21:20:59.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861178+0000", 2024-09-17T21:20:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159023 2024-09-17T21:20:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12104 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:20:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.809971159, 2024-09-17T21:20:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087969628999999994, 2024-09-17T21:20:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12104 2024-09-17T21:20:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:20:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:20:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775883+0000", 2024-09-17T21:20:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3399999999999998e-07 2024-09-17T21:20:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775886+0000", 2024-09-17T21:20:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2119999999999999e-06 2024-09-17T21:20:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775895+0000", 2024-09-17T21:20:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8710000000000003e-06 2024-09-17T21:20:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775901+0000", 2024-09-17T21:20:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:20:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863544+0000", 2024-09-17T21:20:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087642642000000007 2024-09-17T21:20:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863694+0000", 2024-09-17T21:20:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014952300000000001 2024-09-17T21:20:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863851+0000", 2024-09-17T21:20:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000157804 2024-09-17T21:20:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12105 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.809942264999997, 2024-09-17T21:20:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088330400000000003, 2024-09-17T21:20:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12105 2024-09-17T21:20:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:20:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9999999999999997e-07 2024-09-17T21:20:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775914+0000", 2024-09-17T21:20:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4710000000000001e-06 2024-09-17T21:20:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775921+0000", 2024-09-17T21:20:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:20:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775928+0000", 2024-09-17T21:20:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:20:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863923+0000", 2024-09-17T21:20:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087995126000000007 2024-09-17T21:20:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864059+0000", 2024-09-17T21:20:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135797 2024-09-17T21:20:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864241+0000", 2024-09-17T21:20:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018252599999999999 2024-09-17T21:20:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12106 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:20:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:20:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.809853087, 2024-09-17T21:20:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088586394999999998, 2024-09-17T21:20:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:20:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:20:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:20:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:20:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:20:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:20:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12106 2024-09-17T21:20:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:20:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:20:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:20:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:20:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:20:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:20:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:20:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776001+0000", 2024-09-17T21:20:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4300000000000001e-07 2024-09-17T21:20:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:20:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776004+0000", 2024-09-17T21:20:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8049999999999998e-06 2024-09-17T21:20:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:20:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776018+0000", 2024-09-17T21:20:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4613999999999999e-05 2024-09-17T21:20:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:20:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776025+0000", 2024-09-17T21:20:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:20:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:20:59.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864279+0000", 2024-09-17T21:20:59.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088253570000000003 2024-09-17T21:20:59.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:20:59.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864433+0000", 2024-09-17T21:20:59.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015469400000000001 2024-09-17T21:20:59.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:20:59.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:20:59.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:20:59.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864586+0000", 2024-09-17T21:20:59.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000153003 2024-09-17T21:20:59.697 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.697 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.697 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.697 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:20:59.697 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:20:59.697 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:20:59.699 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:20:59.699 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:20:59.699 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:20:59.700 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:20:59.700 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:20:59.700 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:20:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:20:59.701 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:20:59.690+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:20:59.731 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:20:59.731 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:20:59.767 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:20:59.767 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:20:59.789 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:20:59.790 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:20:59.799 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:59.792+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:20:59.853 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:20:59.853 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:20:59.870 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:20:59.900 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:20:59.894+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:00.001 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:20:59.996+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:00.058 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:00.058 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:00.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:21:00.121 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:00.116+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:00.203 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:00.196+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:00.236 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:00.236 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:00.249 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:21:00.304 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:00.298+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:00.387 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:00.388 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:00.402 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:21:00.443 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:00.438+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:00.507 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:00.498+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:00.561 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:00.561 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:00.575 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:21:00.619 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:00.614+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:00.642 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-17T21:21:00.708 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:00.700+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:00.757 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:00.757 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:00.771 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:21:00.809 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:00.804+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:00.904 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:00.904 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:00.909 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:00.904+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:00.918 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:21:01.010 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:01.004+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:01.047 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:01.047 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:01.061 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:21:01.111 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:01.104+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:01.182 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:01.182 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:01.196 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:21:01.212 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:01.204+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:01.312 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:01.308+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:01.325 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:01.326 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:01.344 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:21:01.413 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:01.406+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:01.493 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:01.493 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:01.510 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:21:01.514 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:01.508+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:01.615 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:01.610+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:01.668 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:01.668 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:01.682 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:21:01.723 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:01.718+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:01.733 INFO:teuthology.orchestra.run.smithi184.stdout:5492 read 0xaefc00 thru 0xafb948 (0xbd49 bytes) 2024-09-17T21:21:01.744 INFO:teuthology.orchestra.run.smithi184.stdout:5494 punch from 0x1966400 to 0x1969000, (0x2c00 bytes) 2024-09-17T21:21:01.814 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:21:01.817 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:01.812+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:01.854 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:01.854 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:01.877 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:21:01.918 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:01.910+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:02.000 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:02.000 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:02.016 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:02.016 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:02.018 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:02.012+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:02.034 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:21:02.034 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:21:02.053 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:02.053 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:02.053 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:21:02.053 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:21:02.053 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:02.056 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:02.056 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:02.068 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:21:02.078 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:02.078 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:02.107 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:02.107 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:02.116 INFO:teuthology.orchestra.run.smithi184.stdout:5496 punch from 0x29a800 to 0x29f600, (0x4e00 bytes) 2024-09-17T21:21:02.120 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:02.114+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:02.222 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:02.212+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:02.232 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:02.232 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:02.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:02.233 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:02.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.233 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:525 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3670016~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:21:02.233 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:21:02.233 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 156.02921252199999, 2024-09-17T21:21:02.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046645697, 2024-09-17T21:21:02.233 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.233 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.233 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.233 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.233 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:21:02.234 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:21:02.234 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 525 2024-09-17T21:21:02.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.234 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:21:02.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200226+0000", 2024-09-17T21:21:02.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200227+0000", 2024-09-17T21:21:02.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:21:02.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200234+0000", 2024-09-17T21:21:02.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:21:02.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200254+0000", 2024-09-17T21:21:02.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9959999999999999e-05 2024-09-17T21:21:02.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200293+0000", 2024-09-17T21:21:02.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9755999999999997e-05 2024-09-17T21:21:02.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.245871+0000", 2024-09-17T21:21:02.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045577871999999998 2024-09-17T21:21:02.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246030+0000", 2024-09-17T21:21:02.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158975 2024-09-17T21:21:02.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246871+0000", 2024-09-17T21:21:02.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084111699999999995 2024-09-17T21:21:02.237 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.237 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.237 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.238 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:21:02.238 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:21:02.238 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 156.02896684699999, 2024-09-17T21:21:02.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:21:02.238 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.238 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.238 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.238 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.238 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:21:02.238 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:21:02.239 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:21:02.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.239 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:21:02.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:21:02.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:21:02.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:21:02.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:21:02.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:21:02.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:21:02.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:21:02.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:21:02.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:21:02.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:21:02.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:21:02.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:21:02.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:21:02.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:21:02.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:21:02.242 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.242 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.242 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.242 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:21:02.242 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:21:02.243 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 156.02878816500001, 2024-09-17T21:21:02.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:21:02.243 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.243 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.243 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.243 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.243 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:21:02.243 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:21:02.243 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:21:02.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.243 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:21:02.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.244 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.244 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:21:02.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.244 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.244 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:21:02.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:21:02.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.244 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.244 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:21:02.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:21:02.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:21:02.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:21:02.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:21:02.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:21:02.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:21:02.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:21:02.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:21:02.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:21:02.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:21:02.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:21:02.246 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.247 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.247 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.247 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:21:02.247 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:21:02.247 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 156.028728892, 2024-09-17T21:21:02.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:21:02.247 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.247 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.247 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.248 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.248 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:21:02.248 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:21:02.248 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:21:02.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.248 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:21:02.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:21:02.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:21:02.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:21:02.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:21:02.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:21:02.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:21:02.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:21:02.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:21:02.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:21:02.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:21:02.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:21:02.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:21:02.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:21:02.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:21:02.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:21:02.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.252 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.252 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:21:02.252 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:21:02.252 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 156.02858669700001, 2024-09-17T21:21:02.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:21:02.252 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.252 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.252 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.252 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.253 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:21:02.253 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:21:02.253 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:21:02.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.253 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:21:02.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:21:02.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:21:02.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:21:02.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:21:02.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:21:02.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:21:02.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:21:02.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:21:02.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:21:02.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:21:02.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:21:02.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:21:02.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:21:02.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:21:02.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:21:02.256 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.257 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.257 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.257 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:21:02.257 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:21:02.257 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 156.02853077399999, 2024-09-17T21:21:02.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:21:02.257 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.257 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.257 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.257 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.257 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:21:02.257 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:21:02.258 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:21:02.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.258 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:21:02.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:21:02.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:21:02.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:21:02.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:21:02.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:21:02.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:21:02.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:21:02.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:21:02.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:21:02.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:21:02.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:21:02.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:21:02.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:21:02.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:21:02.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:21:02.261 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.261 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.261 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.262 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:21:02.262 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:21:02.262 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 156.028477157, 2024-09-17T21:21:02.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:21:02.262 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.262 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.262 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.262 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.262 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:21:02.262 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:21:02.262 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:21:02.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.262 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:21:02.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:21:02.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:21:02.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:21:02.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:21:02.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:21:02.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:21:02.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:21:02.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:21:02.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:21:02.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:21:02.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:21:02.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:21:02.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:21:02.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:21:02.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:21:02.266 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.266 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.266 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.266 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:21:02.266 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:02.266 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 105.71924891499999, 2024-09-17T21:21:02.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:21:02.266 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.267 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.267 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:02.267 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.267 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:21:02.267 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:21:02.267 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:21:02.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.267 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:02.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:02.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:21:02.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:21:02.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:21:02.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:21:02.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:21:02.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:21:02.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:21:02.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:21:02.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:21:02.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:21:02.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:02.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:21:02.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:21:02.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:21:02.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:21:02.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:21:02.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:21:02.272 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.272 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.272 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.273 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2597 3.1f 3:fb0e9615:::rbd_data.1152a8ae2019.000000000000000c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:02.273 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:02.273 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 81.390812487000005, 2024-09-17T21:21:02.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097064994000000002, 2024-09-17T21:21:02.273 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.273 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.273 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.273 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.273 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:02.273 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:02.273 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2597 2024-09-17T21:21:02.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.273 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:02.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:02.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838628+0000", 2024-09-17T21:21:02.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.187e-06 2024-09-17T21:21:02.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838654+0000", 2024-09-17T21:21:02.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:21:02.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838696+0000", 2024-09-17T21:21:02.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1226000000000001e-05 2024-09-17T21:21:02.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838715+0000", 2024-09-17T21:21:02.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9646000000000001e-05 2024-09-17T21:21:02.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838937+0000", 2024-09-17T21:21:02.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022163 2024-09-17T21:21:02.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.839284+0000", 2024-09-17T21:21:02.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034753799999999999 2024-09-17T21:21:02.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935691+0000", 2024-09-17T21:21:02.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.096406360999999996 2024-09-17T21:21:02.277 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.277 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.277 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.277 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2641 3.12 3:488a729f:::rbd_data.1152a8ae2019.0000000000000022:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:02.277 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:02.277 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 81.387355713999995, 2024-09-17T21:21:02.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098280953000000004, 2024-09-17T21:21:02.277 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.277 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.278 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.278 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.278 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:02.278 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:02.278 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2641 2024-09-17T21:21:02.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.278 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:02.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:02.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842083+0000", 2024-09-17T21:21:02.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5300000000000002e-07 2024-09-17T21:21:02.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842088+0000", 2024-09-17T21:21:02.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:21:02.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842105+0000", 2024-09-17T21:21:02.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6773e-05 2024-09-17T21:21:02.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842119+0000", 2024-09-17T21:21:02.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:21:02.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842320+0000", 2024-09-17T21:21:02.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020052700000000001 2024-09-17T21:21:02.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842845+0000", 2024-09-17T21:21:02.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052508600000000002 2024-09-17T21:21:02.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940363+0000", 2024-09-17T21:21:02.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097518795000000005 2024-09-17T21:21:02.281 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.281 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.281 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.282 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2674 3.12 3:4c87e3ac:::rbd_data.1152a8ae2019.0000000000000033:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:02.282 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:02.282 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 81.386993077, 2024-09-17T21:21:02.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098025125000000005, 2024-09-17T21:21:02.282 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.282 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.282 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:02.282 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.282 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:02.282 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:02.282 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2674 2024-09-17T21:21:02.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.282 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:02.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:02.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842447+0000", 2024-09-17T21:21:02.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3999999999999999e-06 2024-09-17T21:21:02.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842453+0000", 2024-09-17T21:21:02.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:21:02.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842472+0000", 2024-09-17T21:21:02.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9006000000000001e-05 2024-09-17T21:21:02.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842487+0000", 2024-09-17T21:21:02.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4866e-05 2024-09-17T21:21:02.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843594+0000", 2024-09-17T21:21:02.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011073249999999999 2024-09-17T21:21:02.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843668+0000", 2024-09-17T21:21:02.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3188000000000005e-05 2024-09-17T21:21:02.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:02.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843893+0000", 2024-09-17T21:21:02.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022503399999999999 2024-09-17T21:21:02.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:02.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850439+0000", 2024-09-17T21:21:02.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065467880000000004 2024-09-17T21:21:02.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:02.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940409+0000", 2024-09-17T21:21:02.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089969511000000002 2024-09-17T21:21:02.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:02.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940444+0000", 2024-09-17T21:21:02.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5345999999999997e-05 2024-09-17T21:21:02.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940470+0000", 2024-09-17T21:21:02.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5964000000000001e-05 2024-09-17T21:21:02.287 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.287 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.287 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.288 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2707 3.1f 3:f805e65e:::rbd_data.1152a8ae2019.0000000000000049:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:02.288 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:02.288 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 81.381946416000005, 2024-09-17T21:21:02.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088327300999999997, 2024-09-17T21:21:02.288 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.288 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.288 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.288 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.288 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:02.288 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:02.288 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2707 2024-09-17T21:21:02.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.288 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:02.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:02.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847494+0000", 2024-09-17T21:21:02.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:21:02.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847504+0000", 2024-09-17T21:21:02.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:21:02.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847526+0000", 2024-09-17T21:21:02.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1976000000000001e-05 2024-09-17T21:21:02.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847541+0000", 2024-09-17T21:21:02.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5277e-05 2024-09-17T21:21:02.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856142+0000", 2024-09-17T21:21:02.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086006710000000007 2024-09-17T21:21:02.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856435+0000", 2024-09-17T21:21:02.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029301499999999999 2024-09-17T21:21:02.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935819+0000", 2024-09-17T21:21:02.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079384448999999996 2024-09-17T21:21:02.292 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.292 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.292 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.292 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2730 3.12 3:48b3422a:::rbd_data.1152a8ae2019.0000000000000057:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:02.292 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:02.292 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 81.372753517000007, 2024-09-17T21:21:02.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083879975999999995, 2024-09-17T21:21:02.292 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.293 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.293 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.293 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.293 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:02.293 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:02.293 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2730 2024-09-17T21:21:02.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.293 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:02.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:02.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856687+0000", 2024-09-17T21:21:02.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:21:02.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856697+0000", 2024-09-17T21:21:02.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0207000000000001e-05 2024-09-17T21:21:02.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856808+0000", 2024-09-17T21:21:02.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011071200000000001 2024-09-17T21:21:02.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856828+0000", 2024-09-17T21:21:02.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0219000000000001e-05 2024-09-17T21:21:02.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856907+0000", 2024-09-17T21:21:02.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9661999999999997e-05 2024-09-17T21:21:02.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.857019+0000", 2024-09-17T21:21:02.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011148499999999999 2024-09-17T21:21:02.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940565+0000", 2024-09-17T21:21:02.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083545791999999994 2024-09-17T21:21:02.296 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.296 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.297 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.297 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2743 3.12 3:4d036e62:::rbd_data.1152a8ae2019.000000000000005f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:02.297 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:02.297 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 81.368808016000003, 2024-09-17T21:21:02.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080059962999999998, 2024-09-17T21:21:02.297 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.297 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.297 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.297 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.297 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:02.297 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:02.298 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2743 2024-09-17T21:21:02.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.298 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:02.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:02.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860631+0000", 2024-09-17T21:21:02.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:21:02.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860636+0000", 2024-09-17T21:21:02.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5279999999999997e-06 2024-09-17T21:21:02.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860649+0000", 2024-09-17T21:21:02.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-17T21:21:02.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860655+0000", 2024-09-17T21:21:02.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:21:02.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860731+0000", 2024-09-17T21:21:02.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5920000000000005e-05 2024-09-17T21:21:02.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.861070+0000", 2024-09-17T21:21:02.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338535 2024-09-17T21:21:02.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940690+0000", 2024-09-17T21:21:02.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079620503999999995 2024-09-17T21:21:02.301 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.301 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.301 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.302 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3378 3.12 3:4832a51b:::rbd_data.1152a8ae2019.000000000000006c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:02.302 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:02.302 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 81.340444165999997, 2024-09-17T21:21:02.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051812516000000003, 2024-09-17T21:21:02.302 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.302 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.302 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:02.302 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.302 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:02.302 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:02.302 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3378 2024-09-17T21:21:02.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.302 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:02.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:02.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888995+0000", 2024-09-17T21:21:02.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:21:02.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889006+0000", 2024-09-17T21:21:02.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1012999999999999e-05 2024-09-17T21:21:02.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889032+0000", 2024-09-17T21:21:02.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6441e-05 2024-09-17T21:21:02.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889040+0000", 2024-09-17T21:21:02.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0639999999999994e-06 2024-09-17T21:21:02.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889278+0000", 2024-09-17T21:21:02.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023785899999999999 2024-09-17T21:21:02.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889408+0000", 2024-09-17T21:21:02.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129219 2024-09-17T21:21:02.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:02.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889723+0000", 2024-09-17T21:21:02.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000315264 2024-09-17T21:21:02.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:02.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.891972+0000", 2024-09-17T21:21:02.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002249633 2024-09-17T21:21:02.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:02.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940733+0000", 2024-09-17T21:21:02.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048760626000000001 2024-09-17T21:21:02.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:02.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940785+0000", 2024-09-17T21:21:02.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1906000000000001e-05 2024-09-17T21:21:02.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940807+0000", 2024-09-17T21:21:02.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1554999999999999e-05 2024-09-17T21:21:02.307 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.307 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.308 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.308 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3379 3.12 3:4e83fd71:::rbd_data.1152a8ae2019.000000000000006d:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:02.308 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:02.308 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 81.340359958999997, 2024-09-17T21:21:02.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051804144000000003, 2024-09-17T21:21:02.308 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.308 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.308 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:02.308 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.308 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:02.308 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:02.309 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3379 2024-09-17T21:21:02.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.309 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.309 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.309 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:02.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.309 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.309 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:02.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889079+0000", 2024-09-17T21:21:02.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1800000000000004e-07 2024-09-17T21:21:02.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889083+0000", 2024-09-17T21:21:02.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:21:02.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889094+0000", 2024-09-17T21:21:02.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1562999999999999e-05 2024-09-17T21:21:02.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889102+0000", 2024-09-17T21:21:02.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.401e-06 2024-09-17T21:21:02.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:21:02.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014279259999999999 2024-09-17T21:21:02.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890609+0000", 2024-09-17T21:21:02.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9566000000000004e-05 2024-09-17T21:21:02.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:02.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890789+0000", 2024-09-17T21:21:02.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017949700000000001 2024-09-17T21:21:02.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:02.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.893929+0000", 2024-09-17T21:21:02.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003140538 2024-09-17T21:21:02.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:02.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940834+0000", 2024-09-17T21:21:02.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046904811999999997 2024-09-17T21:21:02.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:02.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940860+0000", 2024-09-17T21:21:02.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5341999999999999e-05 2024-09-17T21:21:02.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940882+0000", 2024-09-17T21:21:02.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2878999999999999e-05 2024-09-17T21:21:02.314 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.314 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.314 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.314 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:2644 3.1e 3:79615585:::rbd_data.115266d0b9f2.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2475520~14848,write 3106304~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:21:02.314 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:02.315 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 72.717731556000004, 2024-09-17T21:21:02.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050733484000000002, 2024-09-17T21:21:02.315 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.315 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.315 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:02.315 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.315 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:21:02.315 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:21:02.315 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2644 2024-09-17T21:21:02.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.315 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:02.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:02.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511708+0000", 2024-09-17T21:21:02.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-17T21:21:02.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522196+0000", 2024-09-17T21:21:02.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010487712999999999 2024-09-17T21:21:02.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522223+0000", 2024-09-17T21:21:02.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7064999999999999e-05 2024-09-17T21:21:02.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522236+0000", 2024-09-17T21:21:02.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2931e-05 2024-09-17T21:21:02.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522348+0000", 2024-09-17T21:21:02.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111906 2024-09-17T21:21:02.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522554+0000", 2024-09-17T21:21:02.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020632699999999999 2024-09-17T21:21:02.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:21:02.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.523185+0000", 2024-09-17T21:21:02.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063081400000000005 2024-09-17T21:21:02.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:02.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.547011+0000", 2024-09-17T21:21:02.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023825570000000001 2024-09-17T21:21:02.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:02.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562268+0000", 2024-09-17T21:21:02.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015257514 2024-09-17T21:21:02.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:02.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562367+0000", 2024-09-17T21:21:02.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8678000000000006e-05 2024-09-17T21:21:02.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562440+0000", 2024-09-17T21:21:02.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3337000000000006e-05 2024-09-17T21:21:02.320 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.320 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.320 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.321 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:231 3.1 3:856fed78:::rbd_data.115220dd393e.000000000000000b:head [read 0~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:02.321 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:02.321 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 44.274976287000001, 2024-09-17T21:21:02.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34254251699999999, 2024-09-17T21:21:02.321 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.321 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.321 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:02.321 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.321 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:02.321 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:02.321 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 231 2024-09-17T21:21:02.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.321 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:02.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:02.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954464+0000", 2024-09-17T21:21:02.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:21:02.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954475+0000", 2024-09-17T21:21:02.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1146e-05 2024-09-17T21:21:02.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954506+0000", 2024-09-17T21:21:02.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0292999999999999e-05 2024-09-17T21:21:02.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954524+0000", 2024-09-17T21:21:02.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8430000000000001e-05 2024-09-17T21:21:02.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954661+0000", 2024-09-17T21:21:02.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136403 2024-09-17T21:21:02.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:02.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954680+0000", 2024-09-17T21:21:02.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9743000000000001e-05 2024-09-17T21:21:02.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.296896+0000", 2024-09-17T21:21:02.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34221550499999998 2024-09-17T21:21:02.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297004+0000", 2024-09-17T21:21:02.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010865399999999999 2024-09-17T21:21:02.325 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.325 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.325 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.326 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:56 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:02.326 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:02.326 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 44.119342590000002, 2024-09-17T21:21:02.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18718773899999999, 2024-09-17T21:21:02.326 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.326 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.326 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.326 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.326 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:21:02.326 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:21:02.326 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 56 2024-09-17T21:21:02.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.326 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:02.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:02.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110098+0000", 2024-09-17T21:21:02.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:21:02.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110111+0000", 2024-09-17T21:21:02.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2289999999999999e-05 2024-09-17T21:21:02.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110143+0000", 2024-09-17T21:21:02.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1979e-05 2024-09-17T21:21:02.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110161+0000", 2024-09-17T21:21:02.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7839e-05 2024-09-17T21:21:02.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110319+0000", 2024-09-17T21:21:02.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015835400000000001 2024-09-17T21:21:02.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:02.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110353+0000", 2024-09-17T21:21:02.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3849000000000001e-05 2024-09-17T21:21:02.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297041+0000", 2024-09-17T21:21:02.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18668842299999999 2024-09-17T21:21:02.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297169+0000", 2024-09-17T21:21:02.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012786199999999999 2024-09-17T21:21:02.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297283+0000", 2024-09-17T21:21:02.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011430300000000001 2024-09-17T21:21:02.331 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.331 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.331 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.332 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:68 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:02.332 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:02.332 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 44.110801659000003, 2024-09-17T21:21:02.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17885582999999999, 2024-09-17T21:21:02.332 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.332 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.332 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.332 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.332 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:21:02.332 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:21:02.332 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 68 2024-09-17T21:21:02.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.333 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:02.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:02.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118639+0000", 2024-09-17T21:21:02.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.306e-06 2024-09-17T21:21:02.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118651+0000", 2024-09-17T21:21:02.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2567e-05 2024-09-17T21:21:02.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118682+0000", 2024-09-17T21:21:02.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0502e-05 2024-09-17T21:21:02.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118707+0000", 2024-09-17T21:21:02.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5463e-05 2024-09-17T21:21:02.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118889+0000", 2024-09-17T21:21:02.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018150999999999999 2024-09-17T21:21:02.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:02.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118910+0000", 2024-09-17T21:21:02.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:21:02.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297313+0000", 2024-09-17T21:21:02.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17840366699999999 2024-09-17T21:21:02.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297389+0000", 2024-09-17T21:21:02.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-17T21:21:02.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297492+0000", 2024-09-17T21:21:02.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103019 2024-09-17T21:21:02.336 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.336 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.336 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.337 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.337 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.337 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:02.338 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:21:02.339 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:02.332+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:02.439 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:02.432+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:02.496 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:02.496 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:02.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.497 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5890.0:9547 3.2 3:42b518d9:::rbd_data.1152f056b73f.0000000000000001:head [call rbd.sparse_copyup in=9482968b,set-alloc-hint object_size 16777216 write_size 16777216,write 632832~19968 in=19968b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-17T21:21:02.497 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:02.449484+0000", 2024-09-17T21:21:02.497 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.046165150000000002, 2024-09-17T21:21:02.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046188899999999998, 2024-09-17T21:21:02.497 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.497 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.497 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:21:02.497 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.497 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5890", 2024-09-17T21:21:02.497 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3064344947", 2024-09-17T21:21:02.497 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 9547 2024-09-17T21:21:02.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.498 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:02.449484+0000", 2024-09-17T21:21:02.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:02.449484+0000", 2024-09-17T21:21:02.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:02.449486+0000", 2024-09-17T21:21:02.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:21:02.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:02.470300+0000", 2024-09-17T21:21:02.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.020813793000000001 2024-09-17T21:21:02.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:02.470327+0000", 2024-09-17T21:21:02.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6883e-05 2024-09-17T21:21:02.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:02.470338+0000", 2024-09-17T21:21:02.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0879999999999999e-05 2024-09-17T21:21:02.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:02.470416+0000", 2024-09-17T21:21:02.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8242000000000001e-05 2024-09-17T21:21:02.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:02.470541+0000", 2024-09-17T21:21:02.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000124564 2024-09-17T21:21:02.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:02.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:02.470852+0000", 2024-09-17T21:21:02.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031151300000000002 2024-09-17T21:21:02.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:02.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:02.485832+0000", 2024-09-17T21:21:02.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014979658 2024-09-17T21:21:02.501 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.501 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.501 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.502 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.502 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:21:02.502 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 1 2024-09-17T21:21:02.502 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:02.510 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:21:02.540 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:02.534+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:02.560 INFO:teuthology.orchestra.run.smithi184.stdout:5497 trunc from 0x5f20400 to 0xc148800 2024-09-17T21:21:02.579 INFO:teuthology.orchestra.run.smithi184.stdout:5499 read 0x73ca800 thru 0x73d3c19 (0x941a bytes) 2024-09-17T21:21:02.581 INFO:teuthology.orchestra.run.smithi184.stdout:5501 write 0x8ec6c00 thru 0x8ed15e5 (0xa9e6 bytes) 2024-09-17T21:21:02.581 INFO:teuthology.orchestra.run.smithi184.stdout:5503 write 0xa1fa800 thru 0xa1feab0 (0x42b1 bytes) 2024-09-17T21:21:02.581 INFO:teuthology.orchestra.run.smithi184.stdout:5504 write 0xa4a4e00 thru 0xa4a56ae (0x8af bytes) 2024-09-17T21:21:02.581 INFO:teuthology.orchestra.run.smithi184.stdout:5505 read 0x2540a00 thru 0x254bf8f (0xb590 bytes) 2024-09-17T21:21:02.582 INFO:teuthology.orchestra.run.smithi184.stdout:5510 read 0x11ab000 thru 0x11af8fe (0x48ff bytes) 2024-09-17T21:21:02.584 INFO:teuthology.orchestra.run.smithi184.stdout:5511 write 0x8457a00 thru 0x8463c24 (0xc225 bytes) 2024-09-17T21:21:02.585 INFO:teuthology.orchestra.run.smithi184.stdout:5513 write 0xc056e00 thru 0xc05d630 (0x6831 bytes) 2024-09-17T21:21:02.640 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:02.632+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:02.648 INFO:teuthology.orchestra.run.smithi184.stdout:5515 write 0xcef6200 thru 0xcf005b1 (0xa3b2 bytes) 2024-09-17T21:21:02.649 INFO:teuthology.orchestra.run.smithi184.stdout:5518 read 0x9f42000 thru 0x9f48941 (0x6942 bytes) 2024-09-17T21:21:02.650 INFO:teuthology.orchestra.run.smithi184.stdout:5519 write 0x57df800 thru 0x57ef03c (0xf83d bytes) 2024-09-17T21:21:02.650 INFO:teuthology.orchestra.run.smithi184.stdout:5520 write 0xce7f600 thru 0xce84f90 (0x5991 bytes) 2024-09-17T21:21:02.651 INFO:teuthology.orchestra.run.smithi184.stdout:5521 read 0x83c2200 thru 0x83c8003 (0x5e04 bytes) 2024-09-17T21:21:02.655 INFO:teuthology.orchestra.run.smithi184.stdout:5522 trunc from 0xcf00600 to 0xb677600 2024-09-17T21:21:02.674 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:21:02.681 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:02.681 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:02.681 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:21:02.681 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:21:02.681 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:02.683 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 0, 5] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:21:02.683 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:21:02.684 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-17T21:21:02.684 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-17T21:21:02.701 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:21:02.745 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:02.740+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:02.750 INFO:teuthology.orchestra.run.smithi184.stdout:5523 write 0x200f600 thru 0x20165f0 (0x6ff1 bytes) 2024-09-17T21:21:02.753 INFO:teuthology.orchestra.run.smithi184.stdout:5524 write 0x72dce00 thru 0x72e4217 (0x7418 bytes) 2024-09-17T21:21:02.754 INFO:teuthology.orchestra.run.smithi184.stdout:5526 punch from 0x5d3e800 to 0x5d3f000, (0x800 bytes) 2024-09-17T21:21:02.841 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:02.836+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:02.915 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:02.915 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:02.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:02.916 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:02.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.916 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:21:02.916 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:02.916 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 143.102616668, 2024-09-17T21:21:02.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:21:02.916 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.916 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.916 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.916 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.916 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:21:02.916 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:21:02.917 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:21:02.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.917 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:02.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:02.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:21:02.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:21:02.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:21:02.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:21:02.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:21:02.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:21:02.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:21:02.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:21:02.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:21:02.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:21:02.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:02.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:21:02.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:21:02.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:21:02.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:21:02.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:21:02.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:21:02.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:21:02.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:21:02.921 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.921 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.921 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.922 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:21:02.922 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:02.922 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 129.547514882, 2024-09-17T21:21:02.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:21:02.922 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.922 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.922 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:02.922 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.922 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:21:02.922 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:02.922 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:21:02.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.922 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:02.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:02.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:21:02.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:21:02.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:21:02.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:21:02.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:21:02.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:21:02.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:21:02.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:21:02.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:21:02.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:21:02.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:02.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:21:02.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:21:02.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:21:02.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:21:02.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:21:02.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:21:02.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:02.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:21:02.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:21:02.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:02.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:21:02.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:21:02.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:02.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:21:02.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:21:02.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:02.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:21:02.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:21:02.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:21:02.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:21:02.928 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.928 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.928 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.928 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:793 3.9 3:95f63599:::rbd_data.1152eb799094.0000000000000007:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:02.928 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:02.928 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 46.480458726999998, 2024-09-17T21:21:02.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88245766400000003, 2024-09-17T21:21:02.928 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.929 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.929 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:02.929 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.929 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:02.929 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:02.929 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 793 2024-09-17T21:21:02.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.929 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:02.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:02.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431941+0000", 2024-09-17T21:21:02.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.864e-06 2024-09-17T21:21:02.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431953+0000", 2024-09-17T21:21:02.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1486e-05 2024-09-17T21:21:02.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431984+0000", 2024-09-17T21:21:02.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1075999999999999e-05 2024-09-17T21:21:02.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431999+0000", 2024-09-17T21:21:02.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.503e-05 2024-09-17T21:21:02.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432095+0000", 2024-09-17T21:21:02.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5698000000000004e-05 2024-09-17T21:21:02.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:02.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432109+0000", 2024-09-17T21:21:02.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4188e-05 2024-09-17T21:21:02.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.309642+0000", 2024-09-17T21:21:02.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87753353000000001 2024-09-17T21:21:02.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310104+0000", 2024-09-17T21:21:02.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046103600000000001 2024-09-17T21:21:02.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:02.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310630+0000", 2024-09-17T21:21:02.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052682800000000002 2024-09-17T21:21:02.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:02.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313100+0000", 2024-09-17T21:21:02.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024693369999999998 2024-09-17T21:21:02.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:02.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313935+0000", 2024-09-17T21:21:02.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083540100000000005 2024-09-17T21:21:02.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:02.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314086+0000", 2024-09-17T21:21:02.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015053399999999999 2024-09-17T21:21:02.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314396+0000", 2024-09-17T21:21:02.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031065599999999997 2024-09-17T21:21:02.935 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.935 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.936 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.936 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:800 3.19 3:9dc17f12:::rbd_data.1152eb799094.000000000000000e:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:02.936 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:02.936 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 46.480211482000001, 2024-09-17T21:21:02.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88154663799999999, 2024-09-17T21:21:02.936 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.936 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.936 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:02.936 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.937 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:02.937 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:02.937 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 800 2024-09-17T21:21:02.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.937 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:02.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:02.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432187+0000", 2024-09-17T21:21:02.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4100000000000001e-06 2024-09-17T21:21:02.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432194+0000", 2024-09-17T21:21:02.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0389999999999999e-06 2024-09-17T21:21:02.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432216+0000", 2024-09-17T21:21:02.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1614e-05 2024-09-17T21:21:02.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432232+0000", 2024-09-17T21:21:02.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5739e-05 2024-09-17T21:21:02.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432405+0000", 2024-09-17T21:21:02.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017355300000000001 2024-09-17T21:21:02.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:02.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432432+0000", 2024-09-17T21:21:02.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6979e-05 2024-09-17T21:21:02.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.307767+0000", 2024-09-17T21:21:02.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87533472899999998 2024-09-17T21:21:02.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308192+0000", 2024-09-17T21:21:02.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042485999999999999 2024-09-17T21:21:02.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:02.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308727+0000", 2024-09-17T21:21:02.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00053550400000000001 2024-09-17T21:21:02.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:02.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310347+0000", 2024-09-17T21:21:02.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016198670000000001 2024-09-17T21:21:02.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:02.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313424+0000", 2024-09-17T21:21:02.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030764640000000001 2024-09-17T21:21:02.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:02.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313510+0000", 2024-09-17T21:21:02.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5938000000000003e-05 2024-09-17T21:21:02.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313733+0000", 2024-09-17T21:21:02.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022294200000000001 2024-09-17T21:21:02.943 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.943 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.943 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.943 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:805 3.19 3:9cab2ed6:::rbd_data.1152eb799094.0000000000000013:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:02.943 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:02.943 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 46.480007463, 2024-09-17T21:21:02.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88274355699999996, 2024-09-17T21:21:02.944 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.944 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.944 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:02.944 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.944 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:02.944 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:02.944 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 805 2024-09-17T21:21:02.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.944 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:02.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:02.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432391+0000", 2024-09-17T21:21:02.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3e-06 2024-09-17T21:21:02.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432398+0000", 2024-09-17T21:21:02.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4489999999999996e-06 2024-09-17T21:21:02.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432415+0000", 2024-09-17T21:21:02.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7750000000000001e-05 2024-09-17T21:21:02.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432430+0000", 2024-09-17T21:21:02.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.449e-05 2024-09-17T21:21:02.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432635+0000", 2024-09-17T21:21:02.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205031 2024-09-17T21:21:02.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:02.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432657+0000", 2024-09-17T21:21:02.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2064000000000001e-05 2024-09-17T21:21:02.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310444+0000", 2024-09-17T21:21:02.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87778699199999999 2024-09-17T21:21:02.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310849+0000", 2024-09-17T21:21:02.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00040497800000000001 2024-09-17T21:21:02.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:02.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.311396+0000", 2024-09-17T21:21:02.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00054729099999999999 2024-09-17T21:21:02.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:02.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313847+0000", 2024-09-17T21:21:02.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024504940000000001 2024-09-17T21:21:02.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:02.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314879+0000", 2024-09-17T21:21:02.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001031817 2024-09-17T21:21:02.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:02.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314964+0000", 2024-09-17T21:21:02.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5685000000000004e-05 2024-09-17T21:21:02.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.315133+0000", 2024-09-17T21:21:02.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000169216 2024-09-17T21:21:02.950 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.950 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.950 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.950 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12087 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 52224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:02.950 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:21:02.951 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.142918784999999, 2024-09-17T21:21:02.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087684460000000006, 2024-09-17T21:21:02.951 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.951 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.951 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.951 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.951 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:02.951 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:02.951 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12087 2024-09-17T21:21:02.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.951 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:21:02.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:21:02.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769480+0000", 2024-09-17T21:21:02.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:21:02.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769485+0000", 2024-09-17T21:21:02.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8880000000000002e-06 2024-09-17T21:21:02.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769509+0000", 2024-09-17T21:21:02.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3343e-05 2024-09-17T21:21:02.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769521+0000", 2024-09-17T21:21:02.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2497000000000001e-05 2024-09-17T21:21:02.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.856883+0000", 2024-09-17T21:21:02.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087362191000000006 2024-09-17T21:21:02.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857014+0000", 2024-09-17T21:21:02.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013090399999999999 2024-09-17T21:21:02.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857163+0000", 2024-09-17T21:21:02.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014864000000000001 2024-09-17T21:21:02.955 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.955 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.955 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.955 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12088 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 56320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:02.955 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:21:02.955 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.142835130000002, 2024-09-17T21:21:02.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087917772000000005, 2024-09-17T21:21:02.955 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.955 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.956 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.956 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.956 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:02.956 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:02.956 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12088 2024-09-17T21:21:02.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.956 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:21:02.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:21:02.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769569+0000", 2024-09-17T21:21:02.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:21:02.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769574+0000", 2024-09-17T21:21:02.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4659999999999998e-06 2024-09-17T21:21:02.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769586+0000", 2024-09-17T21:21:02.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2311e-05 2024-09-17T21:21:02.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769597+0000", 2024-09-17T21:21:02.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0548e-05 2024-09-17T21:21:02.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857217+0000", 2024-09-17T21:21:02.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087619896000000003 2024-09-17T21:21:02.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857335+0000", 2024-09-17T21:21:02.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118538 2024-09-17T21:21:02.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857480+0000", 2024-09-17T21:21:02.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014470999999999999 2024-09-17T21:21:02.960 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.960 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.960 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.960 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12089 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 60416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:02.960 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:02.960 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.142782431000001, 2024-09-17T21:21:02.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088293233999999998, 2024-09-17T21:21:02.960 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.961 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.961 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.961 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.961 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:02.961 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:02.961 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12089 2024-09-17T21:21:02.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.961 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:02.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:02.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769616+0000", 2024-09-17T21:21:02.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:21:02.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769620+0000", 2024-09-17T21:21:02.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5140000000000002e-06 2024-09-17T21:21:02.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769633+0000", 2024-09-17T21:21:02.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2849e-05 2024-09-17T21:21:02.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769644+0000", 2024-09-17T21:21:02.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1471000000000001e-05 2024-09-17T21:21:02.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857564+0000", 2024-09-17T21:21:02.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087919445999999998 2024-09-17T21:21:02.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857716+0000", 2024-09-17T21:21:02.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000152065 2024-09-17T21:21:02.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857908+0000", 2024-09-17T21:21:02.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019252599999999999 2024-09-17T21:21:02.964 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.964 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.965 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.965 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12090 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 64512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:02.965 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:02.965 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.142736046000003, 2024-09-17T21:21:02.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088582672000000001, 2024-09-17T21:21:02.965 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.965 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.965 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.965 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.965 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:02.965 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:02.965 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12090 2024-09-17T21:21:02.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.966 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:02.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:02.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769662+0000", 2024-09-17T21:21:02.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:21:02.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769683+0000", 2024-09-17T21:21:02.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0979e-05 2024-09-17T21:21:02.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769699+0000", 2024-09-17T21:21:02.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5446000000000001e-05 2024-09-17T21:21:02.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769708+0000", 2024-09-17T21:21:02.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1679999999999993e-06 2024-09-17T21:21:02.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857976+0000", 2024-09-17T21:21:02.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088267795999999996 2024-09-17T21:21:02.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858095+0000", 2024-09-17T21:21:02.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119147 2024-09-17T21:21:02.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858244+0000", 2024-09-17T21:21:02.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014917000000000001 2024-09-17T21:21:02.969 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.969 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.969 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.969 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12091 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 68608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:02.969 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:02.969 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.142664662000001, 2024-09-17T21:21:02.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088832752000000001, 2024-09-17T21:21:02.970 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.970 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.970 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.970 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.970 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:02.970 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:02.970 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12091 2024-09-17T21:21:02.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.970 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:02.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:02.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769734+0000", 2024-09-17T21:21:02.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:21:02.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769739+0000", 2024-09-17T21:21:02.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8080000000000004e-06 2024-09-17T21:21:02.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769764+0000", 2024-09-17T21:21:02.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4839999999999999e-05 2024-09-17T21:21:02.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769777+0000", 2024-09-17T21:21:02.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3037000000000001e-05 2024-09-17T21:21:02.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858295+0000", 2024-09-17T21:21:02.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088518475999999999 2024-09-17T21:21:02.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858419+0000", 2024-09-17T21:21:02.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012378700000000001 2024-09-17T21:21:02.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858565+0000", 2024-09-17T21:21:02.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014630099999999999 2024-09-17T21:21:02.974 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.974 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.974 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.974 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12092 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:02.974 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:02.974 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.142582337999997, 2024-09-17T21:21:02.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089191555000000006, 2024-09-17T21:21:02.974 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.974 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.974 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.974 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.974 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:02.975 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:02.975 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12092 2024-09-17T21:21:02.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.975 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:02.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:02.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769816+0000", 2024-09-17T21:21:02.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7199999999999997e-07 2024-09-17T21:21:02.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769822+0000", 2024-09-17T21:21:02.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:21:02.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769838+0000", 2024-09-17T21:21:02.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6674e-05 2024-09-17T21:21:02.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769851+0000", 2024-09-17T21:21:02.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2527999999999999e-05 2024-09-17T21:21:02.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858660+0000", 2024-09-17T21:21:02.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088808556999999996 2024-09-17T21:21:02.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858817+0000", 2024-09-17T21:21:02.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015730400000000001 2024-09-17T21:21:02.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859007+0000", 2024-09-17T21:21:02.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018974899999999999 2024-09-17T21:21:02.978 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.978 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.978 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.979 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12093 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:02.979 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:02.979 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.142474233999998, 2024-09-17T21:21:02.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089429710999999995, 2024-09-17T21:21:02.979 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.979 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.979 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.979 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.979 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:02.979 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:02.979 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12093 2024-09-17T21:21:02.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.980 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:02.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:02.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769924+0000", 2024-09-17T21:21:02.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.342e-06 2024-09-17T21:21:02.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769929+0000", 2024-09-17T21:21:02.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0350000000000001e-06 2024-09-17T21:21:02.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769946+0000", 2024-09-17T21:21:02.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6793999999999999e-05 2024-09-17T21:21:02.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769959+0000", 2024-09-17T21:21:02.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3062e-05 2024-09-17T21:21:02.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859075+0000", 2024-09-17T21:21:02.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089115696999999994 2024-09-17T21:21:02.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859202+0000", 2024-09-17T21:21:02.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012657 2024-09-17T21:21:02.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859353+0000", 2024-09-17T21:21:02.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151211 2024-09-17T21:21:02.983 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.983 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.983 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.983 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12094 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:02.983 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:02.983 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.142420057999999, 2024-09-17T21:21:02.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089695001999999996, 2024-09-17T21:21:02.983 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.984 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.984 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.984 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.984 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:02.984 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:02.984 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12094 2024-09-17T21:21:02.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.984 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:02.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:02.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769978+0000", 2024-09-17T21:21:02.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.142e-06 2024-09-17T21:21:02.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769983+0000", 2024-09-17T21:21:02.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.386e-06 2024-09-17T21:21:02.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769997+0000", 2024-09-17T21:21:02.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.416e-05 2024-09-17T21:21:02.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770012+0000", 2024-09-17T21:21:02.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5384999999999999e-05 2024-09-17T21:21:02.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859405+0000", 2024-09-17T21:21:02.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089392376999999995 2024-09-17T21:21:02.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859525+0000", 2024-09-17T21:21:02.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120555 2024-09-17T21:21:02.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859672+0000", 2024-09-17T21:21:02.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000146997 2024-09-17T21:21:02.988 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.988 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.988 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.988 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12095 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:02.988 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:02.988 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.142367571000001, 2024-09-17T21:21:02.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090073476, 2024-09-17T21:21:02.988 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.988 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.988 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.989 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.989 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:02.989 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:02.989 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12095 2024-09-17T21:21:02.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.989 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:02.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:02.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770031+0000", 2024-09-17T21:21:02.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1699999999999997e-07 2024-09-17T21:21:02.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770036+0000", 2024-09-17T21:21:02.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.023e-06 2024-09-17T21:21:02.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770051+0000", 2024-09-17T21:21:02.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4953999999999999e-05 2024-09-17T21:21:02.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770063+0000", 2024-09-17T21:21:02.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:21:02.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859756+0000", 2024-09-17T21:21:02.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089692980000000005 2024-09-17T21:21:02.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859891+0000", 2024-09-17T21:21:02.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013459499999999999 2024-09-17T21:21:02.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860103+0000", 2024-09-17T21:21:02.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021274400000000001 2024-09-17T21:21:02.993 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.993 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.993 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.994 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12096 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:02.994 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:02.994 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.142317886000001, 2024-09-17T21:21:02.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090358274000000002, 2024-09-17T21:21:02.994 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.994 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.994 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.994 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.994 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:02.994 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:02.994 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12096 2024-09-17T21:21:02.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.994 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:02.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:02.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:02.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:02.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770080+0000", 2024-09-17T21:21:02.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9400000000000004e-07 2024-09-17T21:21:02.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:02.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770085+0000", 2024-09-17T21:21:02.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2169999999999996e-06 2024-09-17T21:21:02.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:02.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770100+0000", 2024-09-17T21:21:02.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5077e-05 2024-09-17T21:21:02.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:02.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770112+0000", 2024-09-17T21:21:02.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1951e-05 2024-09-17T21:21:02.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:02.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860160+0000", 2024-09-17T21:21:02.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090048413999999993 2024-09-17T21:21:02.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:02.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860285+0000", 2024-09-17T21:21:02.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012466699999999999 2024-09-17T21:21:02.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:02.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860438+0000", 2024-09-17T21:21:02.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015305399999999999 2024-09-17T21:21:02.998 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.998 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:02.998 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:02.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.998 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12097 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:02.998 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:02.998 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.142270138999997, 2024-09-17T21:21:02.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090702375000000002, 2024-09-17T21:21:02.998 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:02.999 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:02.999 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:02.999 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:02.999 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:02.999 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:02.999 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12097 2024-09-17T21:21:02.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.999 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:02.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:02.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:02.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:02.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:02.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:02.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:03.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770128+0000", 2024-09-17T21:21:03.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.04e-06 2024-09-17T21:21:03.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770133+0000", 2024-09-17T21:21:03.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8999999999999997e-06 2024-09-17T21:21:03.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770152+0000", 2024-09-17T21:21:03.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8890999999999999e-05 2024-09-17T21:21:03.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770164+0000", 2024-09-17T21:21:03.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1769e-05 2024-09-17T21:21:03.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860511+0000", 2024-09-17T21:21:03.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090347527999999996 2024-09-17T21:21:03.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860652+0000", 2024-09-17T21:21:03.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140894 2024-09-17T21:21:03.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860830+0000", 2024-09-17T21:21:03.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017735299999999999 2024-09-17T21:21:03.002 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.002 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.002 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.003 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12098 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:03.003 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:03.003 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.142216671, 2024-09-17T21:21:03.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090997262999999995, 2024-09-17T21:21:03.003 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.003 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.003 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:03.003 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:03.003 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:03.003 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:03.003 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12098 2024-09-17T21:21:03.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.004 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:03.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:03.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775465+0000", 2024-09-17T21:21:03.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0052843350000000002 2024-09-17T21:21:03.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775476+0000", 2024-09-17T21:21:03.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0797e-05 2024-09-17T21:21:03.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775506+0000", 2024-09-17T21:21:03.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9709999999999998e-05 2024-09-17T21:21:03.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775530+0000", 2024-09-17T21:21:03.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4528999999999998e-05 2024-09-17T21:21:03.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860894+0000", 2024-09-17T21:21:03.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.085364195000000004 2024-09-17T21:21:03.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861019+0000", 2024-09-17T21:21:03.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012467399999999999 2024-09-17T21:21:03.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861178+0000", 2024-09-17T21:21:03.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159023 2024-09-17T21:21:03.007 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.007 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.007 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.007 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12104 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:03.007 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:03.007 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.136515562, 2024-09-17T21:21:03.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087969628999999994, 2024-09-17T21:21:03.008 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.008 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.008 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:03.008 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:03.008 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:03.008 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:03.008 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12104 2024-09-17T21:21:03.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.008 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:03.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:03.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775883+0000", 2024-09-17T21:21:03.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3399999999999998e-07 2024-09-17T21:21:03.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775886+0000", 2024-09-17T21:21:03.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2119999999999999e-06 2024-09-17T21:21:03.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775895+0000", 2024-09-17T21:21:03.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8710000000000003e-06 2024-09-17T21:21:03.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.011 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.011 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775901+0000", 2024-09-17T21:21:03.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:21:03.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.011 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.011 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863544+0000", 2024-09-17T21:21:03.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087642642000000007 2024-09-17T21:21:03.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.011 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.011 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863694+0000", 2024-09-17T21:21:03.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014952300000000001 2024-09-17T21:21:03.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.012 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.012 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863851+0000", 2024-09-17T21:21:03.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000157804 2024-09-17T21:21:03.012 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.012 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.012 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.012 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12105 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:03.012 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:03.013 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.136486668000003, 2024-09-17T21:21:03.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088330400000000003, 2024-09-17T21:21:03.013 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.013 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.013 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:03.013 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:03.013 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:03.013 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:03.013 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12105 2024-09-17T21:21:03.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.013 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:03.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:03.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:03.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9999999999999997e-07 2024-09-17T21:21:03.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775914+0000", 2024-09-17T21:21:03.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4710000000000001e-06 2024-09-17T21:21:03.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775921+0000", 2024-09-17T21:21:03.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:21:03.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775928+0000", 2024-09-17T21:21:03.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:21:03.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.016 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.016 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863923+0000", 2024-09-17T21:21:03.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087995126000000007 2024-09-17T21:21:03.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.016 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.016 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864059+0000", 2024-09-17T21:21:03.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135797 2024-09-17T21:21:03.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.016 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.017 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864241+0000", 2024-09-17T21:21:03.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018252599999999999 2024-09-17T21:21:03.017 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.017 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.017 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.017 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12106 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:03.017 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:03.017 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.13639749, 2024-09-17T21:21:03.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088586394999999998, 2024-09-17T21:21:03.017 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.017 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.018 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:03.018 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:03.018 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:03.018 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:03.018 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12106 2024-09-17T21:21:03.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.018 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:03.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:03.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776001+0000", 2024-09-17T21:21:03.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4300000000000001e-07 2024-09-17T21:21:03.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776004+0000", 2024-09-17T21:21:03.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8049999999999998e-06 2024-09-17T21:21:03.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776018+0000", 2024-09-17T21:21:03.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4613999999999999e-05 2024-09-17T21:21:03.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776025+0000", 2024-09-17T21:21:03.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:21:03.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.021 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.021 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864279+0000", 2024-09-17T21:21:03.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088253570000000003 2024-09-17T21:21:03.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.021 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.021 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864433+0000", 2024-09-17T21:21:03.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015469400000000001 2024-09-17T21:21:03.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.022 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.022 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864586+0000", 2024-09-17T21:21:03.022 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000153003 2024-09-17T21:21:03.022 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.022 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.022 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.022 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.022 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.022 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:03.023 INFO:teuthology.orchestra.run.smithi184.stdout:5528 write 0x8048600 thru 0x804bc41 (0x3642 bytes) 2024-09-17T21:21:03.023 INFO:teuthology.orchestra.run.smithi184.stdout:5529 punch from 0xae27c00 to 0xae2b000, (0x3400 bytes) 2024-09-17T21:21:03.024 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:21:03.024 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:03.016+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:03.124 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:03.118+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:03.149 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-09-17T21:21:03.144+0000 7fe37768f640 -1 mon.a@0(leader).osd e147 definitely_dead 0 2024-09-17T21:21:03.169 INFO:teuthology.orchestra.run.smithi184.stdout:5530 write 0xe088400 thru 0xe08cdf8 (0x49f9 bytes) 2024-09-17T21:21:03.170 INFO:teuthology.orchestra.run.smithi184.stdout:5531 write 0x2fe1c00 thru 0x2fea2eb (0x86ec bytes) 2024-09-17T21:21:03.171 INFO:teuthology.orchestra.run.smithi184.stdout:5536 read 0x7e39400 thru 0x7e437ca (0xa3cb bytes) 2024-09-17T21:21:03.171 INFO:teuthology.orchestra.run.smithi184.stdout:5538 punch from 0xd7ea000 to 0xd7eae00, (0xe00 bytes) 2024-09-17T21:21:03.203 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:03.203 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:03.203 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:21:03.203 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:03.221 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:21:03.267 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:03.260+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:03.328 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:03.320+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:03.333 INFO:teuthology.orchestra.run.smithi184.stdout:5539 write 0x37d5a00 thru 0x37d9cc8 (0x42c9 bytes) 2024-09-17T21:21:03.334 INFO:teuthology.orchestra.run.smithi184.stdout:5542 write 0x3c76c00 thru 0x3c82f08 (0xc309 bytes) 2024-09-17T21:21:03.375 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:03.375 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:03.375 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:21:03.375 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:21:03.375 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:03.391 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:21:03.426 INFO:teuthology.orchestra.run.smithi184.stdout:5543 read 0x6d13800 thru 0x6d234b3 (0xfcb4 bytes) 2024-09-17T21:21:03.427 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:03.422+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:03.427 INFO:teuthology.orchestra.run.smithi184.stdout:5544 write 0xc1d9800 thru 0xc1d98c6 (0xc7 bytes) 2024-09-17T21:21:03.428 INFO:teuthology.orchestra.run.smithi184.stdout:5545 read 0xcacbc00 thru 0xcada79f (0xeba0 bytes) 2024-09-17T21:21:03.430 INFO:teuthology.orchestra.run.smithi184.stdout:5548 write 0x7083800 thru 0x709061e (0xce1f bytes) 2024-09-17T21:21:03.430 INFO:teuthology.orchestra.run.smithi184.stdout:5549 write 0xcffe400 thru 0xd00dfcd (0xfbce bytes) 2024-09-17T21:21:03.431 INFO:teuthology.orchestra.run.smithi184.stdout:5550 read 0x57f5800 thru 0x57ff606 (0x9e07 bytes) 2024-09-17T21:21:03.431 INFO:teuthology.orchestra.run.smithi184.stdout:5551 write 0xb793a00 thru 0xb7a00cd (0xc6ce bytes) 2024-09-17T21:21:03.433 INFO:teuthology.orchestra.run.smithi184.stdout:5552 read 0x2ee8200 thru 0x2ef592e (0xd72f bytes) 2024-09-17T21:21:03.434 INFO:teuthology.orchestra.run.smithi184.stdout:5556 write 0xa808c00 thru 0xa80d2fa (0x46fb bytes) 2024-09-17T21:21:03.434 INFO:teuthology.orchestra.run.smithi184.stdout:5558 write 0x203dc00 thru 0x20495e8 (0xb9e9 bytes) 2024-09-17T21:21:03.435 INFO:teuthology.orchestra.run.smithi184.stdout:5559 write 0xab38800 thru 0xab39e93 (0x1694 bytes) 2024-09-17T21:21:03.440 INFO:teuthology.orchestra.run.smithi184.stdout:5564 trunc from 0xe08ce00 to 0xbf01a00 2024-09-17T21:21:03.527 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:03.522+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:03.549 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:03.549 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:03.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:03.550 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:03.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.550 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:21:03.550 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:21:03.550 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 165.32930732899999, 2024-09-17T21:21:03.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:21:03.550 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.550 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.550 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:03.551 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:21:03.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:21:03.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:21:03.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:21:03.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:21:03.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:21:03.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:21:03.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:21:03.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:21:03.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:21:03.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:21:03.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:21:03.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:21:03.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:21:03.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:03.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:21:03.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:21:03.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:21:03.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:21:03.554 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.555 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.555 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.555 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:21:03.555 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:03.555 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 160.679006665, 2024-09-17T21:21:03.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:21:03.555 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.556 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.556 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:03.556 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:03.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:03.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:21:03.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:21:03.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:21:03.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:21:03.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:21:03.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:21:03.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:21:03.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:21:03.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:21:03.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:21:03.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:21:03.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:21:03.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:03.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:21:03.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:21:03.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:21:03.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:21:03.559 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.559 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.559 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.560 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:21:03.560 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:03.560 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 160.67774158200001, 2024-09-17T21:21:03.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:21:03.560 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.560 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.560 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:03.560 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.561 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:03.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.561 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:03.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.561 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:21:03.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:21:03.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:21:03.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:21:03.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:21:03.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:21:03.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:21:03.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:21:03.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:21:03.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:21:03.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:21:03.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:21:03.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:03.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:21:03.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:21:03.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:21:03.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:21:03.564 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.564 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.564 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.564 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:21:03.564 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:03.564 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 160.66796217199999, 2024-09-17T21:21:03.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:21:03.565 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.565 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.565 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:03.565 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.565 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.565 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:03.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.565 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.565 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:03.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:21:03.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:21:03.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:21:03.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:21:03.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:21:03.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:21:03.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.567 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:21:03.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:21:03.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.567 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:21:03.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:21:03.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.567 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.568 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:21:03.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:21:03.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.568 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.568 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:03.568 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:21:03.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:21:03.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.569 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.569 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:21:03.569 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:21:03.569 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.569 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.569 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.570 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:21:03.570 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:03.570 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 160.65923976900001, 2024-09-17T21:21:03.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:21:03.570 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.570 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.570 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:03.570 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.570 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.570 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:03.571 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.571 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.571 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.571 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:03.571 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.571 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.572 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.572 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:21:03.572 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:21:03.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.572 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.572 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:21:03.572 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:21:03.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.572 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.572 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:21:03.572 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:21:03.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.573 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.573 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.573 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:21:03.573 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:21:03.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.573 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.573 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.573 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:21:03.573 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:21:03.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.573 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.573 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.574 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:21:03.574 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:21:03.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.574 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:03.574 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:21:03.574 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:21:03.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.574 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.574 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:21:03.574 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:21:03.574 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.575 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.575 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.575 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.575 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:21:03.575 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:03.575 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 147.27414920499999, 2024-09-17T21:21:03.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:21:03.575 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.575 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.575 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:03.575 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.575 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.576 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.576 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:03.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.576 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.576 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.576 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:03.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.576 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.576 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.576 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:21:03.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:21:03.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.577 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.577 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:21:03.577 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:21:03.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.577 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.577 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:21:03.577 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:21:03.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.577 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.577 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:21:03.577 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:21:03.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.578 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.578 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.578 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:21:03.578 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:21:03.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.578 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.578 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.578 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:21:03.578 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:21:03.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.578 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.578 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:03.578 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:21:03.579 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:21:03.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.579 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.579 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:21:03.579 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:21:03.579 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.579 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.579 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.579 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1780 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'195, mlcod=69'190)", 2024-09-17T21:21:03.579 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:03.579 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.296293601, 2024-09-17T21:21:03.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075761769999999999, 2024-09-17T21:21:03.580 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.580 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.580 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:03.580 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.580 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.580 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.580 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:03.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.580 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.581 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.581 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:03.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.581 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.581 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250375+0000", 2024-09-17T21:21:03.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:21:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250383+0000", 2024-09-17T21:21:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7870000000000002e-06 2024-09-17T21:21:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250402+0000", 2024-09-17T21:21:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9346000000000001e-05 2024-09-17T21:21:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250413+0000", 2024-09-17T21:21:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.039e-05 2024-09-17T21:21:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251353+0000", 2024-09-17T21:21:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000940091 2024-09-17T21:21:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251407+0000", 2024-09-17T21:21:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4701999999999999e-05 2024-09-17T21:21:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257829+0000", 2024-09-17T21:21:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064212779999999999 2024-09-17T21:21:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257950+0000", 2024-09-17T21:21:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121623 2024-09-17T21:21:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1781 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'196, mlcod=69'190)", 2024-09-17T21:21:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:21:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.29521065, 2024-09-17T21:21:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065632939999999999, 2024-09-17T21:21:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:21:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:21:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251459+0000", 2024-09-17T21:21:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.624e-06 2024-09-17T21:21:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251471+0000", 2024-09-17T21:21:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2636000000000001e-05 2024-09-17T21:21:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251498+0000", 2024-09-17T21:21:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:21:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251515+0000", 2024-09-17T21:21:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6939000000000002e-05 2024-09-17T21:21:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252938+0000", 2024-09-17T21:21:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001423082 2024-09-17T21:21:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252999+0000", 2024-09-17T21:21:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1138999999999999e-05 2024-09-17T21:21:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257955+0000", 2024-09-17T21:21:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049559640000000002 2024-09-17T21:21:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.258020+0000", 2024-09-17T21:21:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5504000000000002e-05 2024-09-17T21:21:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1788 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'198, mlcod=69'191)", 2024-09-17T21:21:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.293185161, 2024-09-17T21:21:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073690370000000002, 2024-09-17T21:21:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253484+0000", 2024-09-17T21:21:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2470000000000001e-06 2024-09-17T21:21:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253492+0000", 2024-09-17T21:21:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7159999999999992e-06 2024-09-17T21:21:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253509+0000", 2024-09-17T21:21:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.666e-05 2024-09-17T21:21:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253522+0000", 2024-09-17T21:21:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2643000000000001e-05 2024-09-17T21:21:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254853+0000", 2024-09-17T21:21:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331238 2024-09-17T21:21:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254885+0000", 2024-09-17T21:21:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1562e-05 2024-09-17T21:21:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260788+0000", 2024-09-17T21:21:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059030749999999998 2024-09-17T21:21:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260852+0000", 2024-09-17T21:21:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3895999999999999e-05 2024-09-17T21:21:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5746 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'477, mlcod=71'473)", 2024-09-17T21:21:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 107.287757603, 2024-09-17T21:21:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074182709999999997, 2024-09-17T21:21:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258912+0000", 2024-09-17T21:21:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:21:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258926+0000", 2024-09-17T21:21:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4249e-05 2024-09-17T21:21:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258946+0000", 2024-09-17T21:21:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9879999999999999e-05 2024-09-17T21:21:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258956+0000", 2024-09-17T21:21:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:21:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259380+0000", 2024-09-17T21:21:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042337599999999999 2024-09-17T21:21:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259412+0000", 2024-09-17T21:21:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:21:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266191+0000", 2024-09-17T21:21:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067793269999999999 2024-09-17T21:21:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266328+0000", 2024-09-17T21:21:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013690999999999999 2024-09-17T21:21:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5747 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'478, mlcod=71'473)", 2024-09-17T21:21:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 107.287051254, 2024-09-17T21:21:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068779440000000004, 2024-09-17T21:21:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259618+0000", 2024-09-17T21:21:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6199999999999999e-06 2024-09-17T21:21:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259630+0000", 2024-09-17T21:21:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1884e-05 2024-09-17T21:21:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259657+0000", 2024-09-17T21:21:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7339e-05 2024-09-17T21:21:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259677+0000", 2024-09-17T21:21:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0156000000000001e-05 2024-09-17T21:21:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260521+0000", 2024-09-17T21:21:03.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084312899999999997 2024-09-17T21:21:03.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.602 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260585+0000", 2024-09-17T21:21:03.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4558000000000001e-05 2024-09-17T21:21:03.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266336+0000", 2024-09-17T21:21:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057505020000000002 2024-09-17T21:21:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266494+0000", 2024-09-17T21:21:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158756 2024-09-17T21:21:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5750 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'479, mlcod=71'473)", 2024-09-17T21:21:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:21:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 107.28633906100001, 2024-09-17T21:21:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006302874, 2024-09-17T21:21:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:21:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:21:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260330+0000", 2024-09-17T21:21:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:21:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260343+0000", 2024-09-17T21:21:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2862e-05 2024-09-17T21:21:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260367+0000", 2024-09-17T21:21:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3723e-05 2024-09-17T21:21:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260386+0000", 2024-09-17T21:21:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8879999999999999e-05 2024-09-17T21:21:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261579+0000", 2024-09-17T21:21:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001193431 2024-09-17T21:21:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261608+0000", 2024-09-17T21:21:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8858999999999999e-05 2024-09-17T21:21:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266505+0000", 2024-09-17T21:21:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004896999 2024-09-17T21:21:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266631+0000", 2024-09-17T21:21:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012655300000000001 2024-09-17T21:21:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5751 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'480, mlcod=71'473)", 2024-09-17T21:21:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 107.285631849, 2024-09-17T21:21:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075756299999999999, 2024-09-17T21:21:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261037+0000", 2024-09-17T21:21:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:21:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261049+0000", 2024-09-17T21:21:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1063999999999999e-05 2024-09-17T21:21:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261072+0000", 2024-09-17T21:21:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3102e-05 2024-09-17T21:21:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261133+0000", 2024-09-17T21:21:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1276000000000001e-05 2024-09-17T21:21:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:21:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015334319999999999 2024-09-17T21:21:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262702+0000", 2024-09-17T21:21:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5281999999999999e-05 2024-09-17T21:21:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268527+0000", 2024-09-17T21:21:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058250389999999997 2024-09-17T21:21:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268611+0000", 2024-09-17T21:21:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4724999999999999e-05 2024-09-17T21:21:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5756 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'481, mlcod=71'473)", 2024-09-17T21:21:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 107.284770993, 2024-09-17T21:21:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006772386, 2024-09-17T21:21:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261899+0000", 2024-09-17T21:21:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.903e-06 2024-09-17T21:21:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261912+0000", 2024-09-17T21:21:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-05 2024-09-17T21:21:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261938+0000", 2024-09-17T21:21:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6098999999999999e-05 2024-09-17T21:21:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261955+0000", 2024-09-17T21:21:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.696e-05 2024-09-17T21:21:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263845+0000", 2024-09-17T21:21:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001890187 2024-09-17T21:21:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263907+0000", 2024-09-17T21:21:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1600999999999995e-05 2024-09-17T21:21:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268615+0000", 2024-09-17T21:21:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047086070000000001 2024-09-17T21:21:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268669+0000", 2024-09-17T21:21:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3588999999999998e-05 2024-09-17T21:21:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4978.0:598 3.9 e80/23 3:9701cc17:::rbd_data.1152724ff2a3.0000000000000019:head v 80'202, mlcod=80'198)", 2024-09-17T21:21:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:21:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 96.783196610000005, 2024-09-17T21:21:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066473340000000004, 2024-09-17T21:21:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:21:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:21:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763472+0000", 2024-09-17T21:21:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6899999999999997e-07 2024-09-17T21:21:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763478+0000", 2024-09-17T21:21:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:21:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763489+0000", 2024-09-17T21:21:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0628999999999999e-05 2024-09-17T21:21:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763495+0000", 2024-09-17T21:21:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6649999999999997e-06 2024-09-17T21:21:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763907+0000", 2024-09-17T21:21:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041163899999999999 2024-09-17T21:21:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763976+0000", 2024-09-17T21:21:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8534999999999993e-05 2024-09-17T21:21:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.769900+0000", 2024-09-17T21:21:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059243270000000001 2024-09-17T21:21:03.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.770118+0000", 2024-09-17T21:21:03.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021842500000000001 2024-09-17T21:21:03.622 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.622 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.622 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:468 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'94, mlcod=89'87)", 2024-09-17T21:21:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.860451311000006, 2024-09-17T21:21:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066528790000000004, 2024-09-17T21:21:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686218+0000", 2024-09-17T21:21:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3060000000000001e-06 2024-09-17T21:21:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686256+0000", 2024-09-17T21:21:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8333e-05 2024-09-17T21:21:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686272+0000", 2024-09-17T21:21:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:21:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686283+0000", 2024-09-17T21:21:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0903e-05 2024-09-17T21:21:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686682+0000", 2024-09-17T21:21:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039935099999999999 2024-09-17T21:21:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686785+0000", 2024-09-17T21:21:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010275199999999999 2024-09-17T21:21:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692816+0000", 2024-09-17T21:21:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060317579999999999 2024-09-17T21:21:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692869+0000", 2024-09-17T21:21:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2775999999999998e-05 2024-09-17T21:21:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:755 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'127, mlcod=89'122)", 2024-09-17T21:21:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.723996096999997, 2024-09-17T21:21:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084014080000000008, 2024-09-17T21:21:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:21:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822683+0000", 2024-09-17T21:21:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0674e-05 2024-09-17T21:21:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822698+0000", 2024-09-17T21:21:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5189e-05 2024-09-17T21:21:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822708+0000", 2024-09-17T21:21:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9119999999999993e-06 2024-09-17T21:21:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822748+0000", 2024-09-17T21:21:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9727000000000001e-05 2024-09-17T21:21:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822759+0000", 2024-09-17T21:21:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0865e-05 2024-09-17T21:21:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831034+0000", 2024-09-17T21:21:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082750710000000002 2024-09-17T21:21:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831073+0000", 2024-09-17T21:21:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9063999999999997e-05 2024-09-17T21:21:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:756 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'128, mlcod=89'124)", 2024-09-17T21:21:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.722073464999994, 2024-09-17T21:21:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0076300270000000002, 2024-09-17T21:21:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824595+0000", 2024-09-17T21:21:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-17T21:21:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824600+0000", 2024-09-17T21:21:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:21:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824612+0000", 2024-09-17T21:21:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.181e-05 2024-09-17T21:21:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824619+0000", 2024-09-17T21:21:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:21:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824671+0000", 2024-09-17T21:21:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2262999999999998e-05 2024-09-17T21:21:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824695+0000", 2024-09-17T21:21:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3883000000000002e-05 2024-09-17T21:21:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832185+0000", 2024-09-17T21:21:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007490048 2024-09-17T21:21:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832224+0000", 2024-09-17T21:21:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8957e-05 2024-09-17T21:21:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2617 3.d e90/23 3:b0143718:::rbd_data.1152a8ae2019.0000000000000016:head v 90'249, mlcod=90'246)", 2024-09-17T21:21:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:21:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 82.702222465000006, 2024-09-17T21:21:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063715559999999996, 2024-09-17T21:21:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:21:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:21:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844455+0000", 2024-09-17T21:21:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4259999999999992e-06 2024-09-17T21:21:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844470+0000", 2024-09-17T21:21:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5386000000000001e-05 2024-09-17T21:21:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844499+0000", 2024-09-17T21:21:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.883e-05 2024-09-17T21:21:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844518+0000", 2024-09-17T21:21:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8830000000000001e-05 2024-09-17T21:21:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844649+0000", 2024-09-17T21:21:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013143399999999999 2024-09-17T21:21:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844702+0000", 2024-09-17T21:21:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2876e-05 2024-09-17T21:21:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850696+0000", 2024-09-17T21:21:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059943729999999999 2024-09-17T21:21:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850817+0000", 2024-09-17T21:21:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120401 2024-09-17T21:21:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2635 3.d e90/23 3:b3d1e920:::rbd_data.1152a8ae2019.000000000000001f:head v 90'252, mlcod=90'246)", 2024-09-17T21:21:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 82.699514394999994, 2024-09-17T21:21:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0077265190000000003, 2024-09-17T21:21:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847155+0000", 2024-09-17T21:21:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.544e-06 2024-09-17T21:21:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847165+0000", 2024-09-17T21:21:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.043e-05 2024-09-17T21:21:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847189+0000", 2024-09-17T21:21:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3659000000000002e-05 2024-09-17T21:21:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847210+0000", 2024-09-17T21:21:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1163e-05 2024-09-17T21:21:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848856+0000", 2024-09-17T21:21:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001645521 2024-09-17T21:21:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848905+0000", 2024-09-17T21:21:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9048999999999997e-05 2024-09-17T21:21:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854770+0000", 2024-09-17T21:21:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058652879999999998 2024-09-17T21:21:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854880+0000", 2024-09-17T21:21:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010986500000000001 2024-09-17T21:21:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:03.647 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:03.649 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:21:03.650 INFO:teuthology.orchestra.run.smithi184.stdout:5566 punch from 0xba1a200 to 0xba1a800, (0x600 bytes) 2024-09-17T21:21:03.650 INFO:teuthology.orchestra.run.smithi184.stdout:5570 punch from 0x2ea2800 to 0x2eb0a00, (0xe200 bytes) 2024-09-17T21:21:03.650 INFO:teuthology.orchestra.run.smithi184.stdout:5572 write 0xa287e00 thru 0xa296e59 (0xf05a bytes) 2024-09-17T21:21:03.650 INFO:teuthology.orchestra.run.smithi184.stdout:5573 write 0xbd89800 thru 0xbd8c155 (0x2956 bytes) 2024-09-17T21:21:03.650 INFO:teuthology.orchestra.run.smithi184.stdout:5574 read 0x3b59200 thru 0x3b66f97 (0xdd98 bytes) 2024-09-17T21:21:03.650 INFO:teuthology.orchestra.run.smithi184.stdout:5575 write 0xb134800 thru 0xb137fdd (0x37de bytes) 2024-09-17T21:21:03.650 INFO:teuthology.orchestra.run.smithi184.stdout:5577 write 0x3a1bc00 thru 0x3a1f0a9 (0x34aa bytes) 2024-09-17T21:21:03.650 INFO:teuthology.orchestra.run.smithi184.stdout:5580 write 0x8159800 thru 0x81635cf (0x9dd0 bytes) 2024-09-17T21:21:03.651 INFO:teuthology.orchestra.run.smithi184.stdout:5582 punch from 0x72b0e00 to 0x72b9000, (0x8200 bytes) 2024-09-17T21:21:03.651 INFO:teuthology.orchestra.run.smithi184.stdout:5583 punch from 0x637d200 to 0x6385800, (0x8600 bytes) 2024-09-17T21:21:03.651 INFO:teuthology.orchestra.run.smithi184.stdout:5587 write 0x8b28600 thru 0x8b2af6a (0x296b bytes) 2024-09-17T21:21:03.651 INFO:teuthology.orchestra.run.smithi184.stdout:5588 trunc from 0xbf01a00 to 0xb741800 2024-09-17T21:21:03.651 INFO:teuthology.orchestra.run.smithi184.stdout:5589 read 0x8990800 thru 0x8991f2e (0x172f bytes) 2024-09-17T21:21:03.651 INFO:teuthology.orchestra.run.smithi184.stdout:5590 write 0x877ee00 thru 0x87897dc (0xa9dd bytes) 2024-09-17T21:21:03.651 INFO:teuthology.orchestra.run.smithi184.stdout:5592 trunc from 0xb741800 to 0xca91a00 2024-09-17T21:21:03.651 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:03.642+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:03.664 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-17T21:21:03.664 INFO:teuthology.orchestra.run.smithi083.stderr:marked in osd.2. 2024-09-17T21:21:03.675 INFO:teuthology.orchestra.run.smithi184.stdout:5593 punch from 0xa06fe00 to 0xa078a00, (0x8c00 bytes) 2024-09-17T21:21:03.676 INFO:teuthology.orchestra.run.smithi184.stdout:5595 write 0x3597200 thru 0x35a37c5 (0xc5c6 bytes) 2024-09-17T21:21:03.689 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-17T21:21:03.749 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:03.744+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:03.784 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:03.784 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:03.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:03.785 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5890.0:20032 3.8 3.9024ef48 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:21:03.785 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.675239+0000", 2024-09-17T21:21:03.785 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.107363156, 2024-09-17T21:21:03.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.107407608, 2024-09-17T21:21:03.785 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:03.785 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:03.785 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "waiting for peered", 2024-09-17T21:21:03.785 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:03.785 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5890", 2024-09-17T21:21:03.785 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3064344947", 2024-09-17T21:21:03.785 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20032 2024-09-17T21:21:03.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.675239+0000", 2024-09-17T21:21:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.675239+0000", 2024-09-17T21:21:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.675242+0000", 2024-09-17T21:21:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4380000000000002e-06 2024-09-17T21:21:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.675250+0000", 2024-09-17T21:21:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4090000000000003e-06 2024-09-17T21:21:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.675276+0000", 2024-09-17T21:21:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5965999999999998e-05 2024-09-17T21:21:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.675288+0000", 2024-09-17T21:21:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.239e-05 2024-09-17T21:21:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.675384+0000", 2024-09-17T21:21:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5247000000000005e-05 2024-09-17T21:21:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.675396+0000", 2024-09-17T21:21:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.225e-05 2024-09-17T21:21:03.789 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:03.789 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:03.789 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:03.789 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:03.789 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-09-17T21:21:03.789 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 1 2024-09-17T21:21:03.789 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:03.798 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:21:03.849 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:03.844+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:03.919 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:03.919 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:03.919 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:03.919 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:03.919 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:03.932 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:21:03.979 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:03.974+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:04.051 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:04.046+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:04.057 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:04.057 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:04.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:04.057 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:04.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:21:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 187.03892086900001, 2024-09-17T21:21:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:21:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:21:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:21:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:21:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:21:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:21:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:21:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:21:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:21:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:21:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:21:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:21:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:21:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:21:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:21:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:21:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:21:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:21:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:21:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:21:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:21:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:21:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:21:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:21:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:21:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:21:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:21:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:21:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:21:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:21:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:21:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 167.45875658899999, 2024-09-17T21:21:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:21:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:21:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:21:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:21:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:21:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:21:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:21:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:21:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:21:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:21:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:21:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:21:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:21:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:21:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:21:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:21:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:21:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:21:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:21:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:21:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:21:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:21:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:21:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:21:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:21:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:21:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 167.45865927700001, 2024-09-17T21:21:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:21:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:21:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:21:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:21:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:21:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:21:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:21:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:21:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:21:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:21:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:21:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:21:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:21:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:21:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:21:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:21:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:21:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:21:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:21:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:21:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:21:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:21:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:21:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:21:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:21:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:21:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:21:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:21:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:21:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 167.45657247599999, 2024-09-17T21:21:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:21:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:21:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:21:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:21:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:21:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:21:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:21:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:21:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:21:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:21:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:21:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:21:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:21:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:21:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:21:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:21:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:21:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:21:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:21:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:21:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:21:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:21:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:21:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:21:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:21:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:21:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:21:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:21:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:21:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:21:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:21:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:21:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:21:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:21:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:21:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 134.77267369500001, 2024-09-17T21:21:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:21:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:21:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:21:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:21:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:21:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:21:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:21:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:21:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:21:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:21:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:21:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:21:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:21:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:21:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:21:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:21:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:21:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:21:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:21:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:21:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:21:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:21:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:21:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:21:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 134.76358198400001, 2024-09-17T21:21:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:21:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:21:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:21:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:21:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:21:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:21:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:21:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:21:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:21:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:21:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:21:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:21:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:21:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:21:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:21:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:21:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:21:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:21:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:21:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:21:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 134.76340993299999, 2024-09-17T21:21:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:21:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:21:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:21:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:21:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:21:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:21:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:21:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:21:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:21:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:21:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:21:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:21:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:21:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:21:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:21:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:21:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:21:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:21:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:21:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:21:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 134.58847718600001, 2024-09-17T21:21:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:21:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:21:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:21:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:21:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:21:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:21:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:21:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:21:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:21:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:21:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:21:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:21:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:21:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:21:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:21:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:21:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:21:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:21:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:21:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:21:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 134.588392629, 2024-09-17T21:21:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:21:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:21:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:21:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:21:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:21:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:21:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:21:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:21:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:21:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:21:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:21:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:21:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:21:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:21:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:21:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:21:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:21:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:21:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:21:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 134.58832699199999, 2024-09-17T21:21:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:21:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:21:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:21:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:21:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:21:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:21:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:21:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:21:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:21:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:21:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:21:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:21:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:21:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:21:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:21:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:21:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:21:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:21:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:21:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:21:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.214751895000006, 2024-09-17T21:21:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:21:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:21:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:21:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:21:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:21:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:21:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:21:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:21:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:21:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:21:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:21:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:21:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:21:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:21:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:21:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:21:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:21:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.214632382999994, 2024-09-17T21:21:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:21:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:21:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:21:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:21:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:21:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:21:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:21:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:21:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:21:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:21:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:21:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:21:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:21:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:21:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:21:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:21:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:21:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:994 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 571392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:21:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.621660409, 2024-09-17T21:21:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083898465000000005, 2024-09-17T21:21:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 994 2024-09-17T21:21:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:21:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:21:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:21:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:21:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431058+0000", 2024-09-17T21:21:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1667e-05 2024-09-17T21:21:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431067+0000", 2024-09-17T21:21:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0720000000000006e-06 2024-09-17T21:21:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431075+0000", 2024-09-17T21:21:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9200000000000004e-06 2024-09-17T21:21:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514671+0000", 2024-09-17T21:21:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083596229999999994 2024-09-17T21:21:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514741+0000", 2024-09-17T21:21:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9929000000000001e-05 2024-09-17T21:21:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514924+0000", 2024-09-17T21:21:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000183122 2024-09-17T21:21:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:995 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 575488~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:21:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.621601392999999, 2024-09-17T21:21:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084214699000000004, 2024-09-17T21:21:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 995 2024-09-17T21:21:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:21:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:21:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431086+0000", 2024-09-17T21:21:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7999999999999995e-07 2024-09-17T21:21:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431088+0000", 2024-09-17T21:21:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4729999999999999e-06 2024-09-17T21:21:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431126+0000", 2024-09-17T21:21:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7583999999999998e-05 2024-09-17T21:21:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431137+0000", 2024-09-17T21:21:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1768e-05 2024-09-17T21:21:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515003+0000", 2024-09-17T21:21:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083865097 2024-09-17T21:21:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515115+0000", 2024-09-17T21:21:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112561 2024-09-17T21:21:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515300+0000", 2024-09-17T21:21:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018453600000000001 2024-09-17T21:21:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:996 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.621519012, 2024-09-17T21:21:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084493736999999999, 2024-09-17T21:21:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 996 2024-09-17T21:21:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431168+0000", 2024-09-17T21:21:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4e-07 2024-09-17T21:21:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431172+0000", 2024-09-17T21:21:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.608e-06 2024-09-17T21:21:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431180+0000", 2024-09-17T21:21:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5650000000000002e-06 2024-09-17T21:21:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431188+0000", 2024-09-17T21:21:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:21:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515354+0000", 2024-09-17T21:21:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084165977000000003 2024-09-17T21:21:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515451+0000", 2024-09-17T21:21:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6669000000000006e-05 2024-09-17T21:21:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515661+0000", 2024-09-17T21:21:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021021599999999999 2024-09-17T21:21:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:997 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 583680~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.621487617, 2024-09-17T21:21:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084772419000000002, 2024-09-17T21:21:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 997 2024-09-17T21:21:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0400000000000005e-07 2024-09-17T21:21:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431202+0000", 2024-09-17T21:21:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:21:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431208+0000", 2024-09-17T21:21:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4740000000000002e-06 2024-09-17T21:21:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431214+0000", 2024-09-17T21:21:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6550000000000001e-06 2024-09-17T21:21:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515741+0000", 2024-09-17T21:21:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084526904 2024-09-17T21:21:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515826+0000", 2024-09-17T21:21:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4643999999999998e-05 2024-09-17T21:21:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515971+0000", 2024-09-17T21:21:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145444 2024-09-17T21:21:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:998 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 587776~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.621464645000003, 2024-09-17T21:21:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085108565999999997, 2024-09-17T21:21:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 998 2024-09-17T21:21:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:21:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431224+0000", 2024-09-17T21:21:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:21:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431231+0000", 2024-09-17T21:21:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000002e-06 2024-09-17T21:21:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431236+0000", 2024-09-17T21:21:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.648e-06 2024-09-17T21:21:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516024+0000", 2024-09-17T21:21:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084787584999999999 2024-09-17T21:21:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516122+0000", 2024-09-17T21:21:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8076000000000001e-05 2024-09-17T21:21:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516330+0000", 2024-09-17T21:21:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020808 2024-09-17T21:21:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1000 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 591872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.621442530000003, 2024-09-17T21:21:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085389065, 2024-09-17T21:21:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1000 2024-09-17T21:21:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:21:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431246+0000", 2024-09-17T21:21:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1749999999999999e-06 2024-09-17T21:21:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431253+0000", 2024-09-17T21:21:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2790000000000004e-06 2024-09-17T21:21:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431258+0000", 2024-09-17T21:21:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3299999999999998e-06 2024-09-17T21:21:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516384+0000", 2024-09-17T21:21:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085125549999999994 2024-09-17T21:21:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516460+0000", 2024-09-17T21:21:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6359999999999994e-05 2024-09-17T21:21:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516633+0000", 2024-09-17T21:21:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017285900000000001 2024-09-17T21:21:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1001 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 595968~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.621419123999999, 2024-09-17T21:21:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085614715999999993, 2024-09-17T21:21:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1001 2024-09-17T21:21:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438066+0000", 2024-09-17T21:21:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067989490000000003 2024-09-17T21:21:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438073+0000", 2024-09-17T21:21:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8419999999999999e-06 2024-09-17T21:21:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438090+0000", 2024-09-17T21:21:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6657e-05 2024-09-17T21:21:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438099+0000", 2024-09-17T21:21:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0529999999999996e-06 2024-09-17T21:21:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516678+0000", 2024-09-17T21:21:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078579422999999995 2024-09-17T21:21:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516739+0000", 2024-09-17T21:21:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1315000000000005e-05 2024-09-17T21:21:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516882+0000", 2024-09-17T21:21:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014247700000000001 2024-09-17T21:21:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:19998 3.5 3:a4885450:::rbd_object_map.1152d139f9f9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-17T21:21:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.650151878999999, 2024-09-17T21:21:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90715489299999996, 2024-09-17T21:21:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19998 2024-09-17T21:21:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402537+0000", 2024-09-17T21:21:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5600000000000001e-06 2024-09-17T21:21:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402549+0000", 2024-09-17T21:21:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1929999999999999e-05 2024-09-17T21:21:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402579+0000", 2024-09-17T21:21:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0093000000000001e-05 2024-09-17T21:21:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402595+0000", 2024-09-17T21:21:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5855999999999999e-05 2024-09-17T21:21:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402731+0000", 2024-09-17T21:21:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013610799999999999 2024-09-17T21:21:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402750+0000", 2024-09-17T21:21:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8737e-05 2024-09-17T21:21:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.305977+0000", 2024-09-17T21:21:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90322768399999998 2024-09-17T21:21:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306236+0000", 2024-09-17T21:21:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000258111 2024-09-17T21:21:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306557+0000", 2024-09-17T21:21:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032159799999999998 2024-09-17T21:21:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.308286+0000", 2024-09-17T21:21:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017292150000000001 2024-09-17T21:21:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309530+0000", 2024-09-17T21:21:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00124409 2024-09-17T21:21:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309630+0000", 2024-09-17T21:21:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9689e-05 2024-09-17T21:21:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309689+0000", 2024-09-17T21:21:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9222000000000003e-05 2024-09-17T21:21:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:04.163 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:04.163 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:04.163 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:21:04.165 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:21:04.165 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:04.158+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:04.265 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:04.258+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:04.311 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:04.311 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:04.329 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:04.329 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:04.333 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:04.333 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:04.333 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:04.333 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:04.346 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:21:04.346 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:21:04.347 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:21:04.363 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:04.363 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:04.382 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:04.383 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:04.391 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:04.386+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:04.401 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:04.402 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:04.466 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:04.460+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:04.512 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:04.512 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:04.527 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:21:04.568 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:04.564+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:04.651 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:04.652 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:04.666 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:21:04.668 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:04.662+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:04.769 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:04.764+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:04.784 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:04.784 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:04.797 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:21:04.870 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:04.862+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:04.913 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:04.913 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:04.927 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:21:04.971 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:04.966+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:05.072 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:05.064+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:05.094 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:05.094 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:05.108 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:21:05.172 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:05.166+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:05.238 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:05.238 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:05.252 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:21:05.297 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:05.292+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:05.370 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:05.370 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:05.374 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:05.370+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:05.382 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:21:05.475 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:05.468+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:05.551 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:05.551 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:05.565 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:21:05.609 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:05.604+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:05.683 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:05.672+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:05.693 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:21:05.717 INFO:teuthology.orchestra.run.smithi184.stdout:5596 write 0x947b600 thru 0x947c9cc (0x13cd bytes) 2024-09-17T21:21:05.719 INFO:teuthology.orchestra.run.smithi184.stdout:5597 read 0x84f3600 thru 0x84fd5c3 (0x9fc4 bytes) 2024-09-17T21:21:05.721 INFO:teuthology.orchestra.run.smithi184.stdout:5598 write 0x57c1200 thru 0x57c8785 (0x7586 bytes) 2024-09-17T21:21:05.724 INFO:teuthology.orchestra.run.smithi184.stdout:5600 read 0x40fb600 thru 0x4102297 (0x6c98 bytes) 2024-09-17T21:21:05.753 INFO:teuthology.orchestra.run.smithi184.stdout:5601 write 0xdc6ac00 thru 0xdc6fe78 (0x5279 bytes) 2024-09-17T21:21:05.755 INFO:teuthology.orchestra.run.smithi184.stdout:5602 write 0x1795000 thru 0x17a27c4 (0xd7c5 bytes) 2024-09-17T21:21:05.763 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:05.763 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:05.778 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:05.770+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:05.782 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:21:05.845 INFO:teuthology.orchestra.run.smithi184.stdout:5603 trunc from 0xdc70000 to 0x4c0400 2024-09-17T21:21:05.878 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:05.872+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:05.913 INFO:teuthology.orchestra.run.smithi184.stdout:5607 read 0x109e00 thru 0x113fff (0xa200 bytes) 2024-09-17T21:21:05.915 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:05.915 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:05.932 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:21:05.948 INFO:teuthology.orchestra.run.smithi184.stdout:5608 write 0x5810600 thru 0x5814c0c (0x460d bytes) 2024-09-17T21:21:05.980 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:05.974+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:05.991 INFO:teuthology.orchestra.run.smithi184.stdout:5610 write 0xb8b3200 thru 0xb8c1891 (0xe692 bytes) 2024-09-17T21:21:05.994 INFO:teuthology.orchestra.run.smithi184.stdout:5612 read 0x2d3e600 thru 0x2d3e667 (0x68 bytes) 2024-09-17T21:21:05.994 INFO:teuthology.orchestra.run.smithi184.stdout:5614 read 0x2434800 thru 0x243a89c (0x609d bytes) 2024-09-17T21:21:05.995 INFO:teuthology.orchestra.run.smithi184.stdout:5618 write 0x9308c00 thru 0x9314c7a (0xc07b bytes) 2024-09-17T21:21:05.998 INFO:teuthology.orchestra.run.smithi184.stdout:5619 trunc from 0xb8c1a00 to 0x9db5400 2024-09-17T21:21:06.036 INFO:teuthology.orchestra.run.smithi184.stdout:5621 trunc from 0x9db5400 to 0x8188a00 2024-09-17T21:21:06.056 INFO:teuthology.orchestra.run.smithi184.stdout:5624 read 0x4ac4000 thru 0x4acb636 (0x7637 bytes) 2024-09-17T21:21:06.058 INFO:teuthology.orchestra.run.smithi184.stdout:5625 write 0x1603800 thru 0x1606cf6 (0x34f7 bytes) 2024-09-17T21:21:06.060 INFO:teuthology.orchestra.run.smithi184.stdout:5627 read 0x41ed200 thru 0x41f3821 (0x6622 bytes) 2024-09-17T21:21:06.061 INFO:teuthology.orchestra.run.smithi184.stdout:5633 read 0x23ef200 thru 0x23f6347 (0x7148 bytes) 2024-09-17T21:21:06.062 INFO:teuthology.orchestra.run.smithi184.stdout:5634 read 0x2015400 thru 0x201942b (0x402c bytes) 2024-09-17T21:21:06.080 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:06.072+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:06.100 INFO:teuthology.orchestra.run.smithi184.stdout:5638 write 0xe1f2000 thru 0xe1f7d97 (0x5d98 bytes) 2024-09-17T21:21:06.102 INFO:teuthology.orchestra.run.smithi184.stdout:5640 write 0x4b82c00 thru 0x4b86777 (0x3b78 bytes) 2024-09-17T21:21:06.103 INFO:teuthology.orchestra.run.smithi184.stdout:5641 write 0xb78b200 thru 0xb78bab8 (0x8b9 bytes) 2024-09-17T21:21:06.113 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:06.113 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:06.113 INFO:teuthology.orchestra.run.smithi184.stdout:5644 trunc from 0xe1f7e00 to 0xa0f3e00 2024-09-17T21:21:06.128 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:21:06.168 INFO:teuthology.orchestra.run.smithi184.stdout:5647 write 0x8032a00 thru 0x803dc11 (0xb212 bytes) 2024-09-17T21:21:06.170 INFO:teuthology.orchestra.run.smithi184.stdout:5648 write 0x8c15400 thru 0x8c212ca (0xbecb bytes) 2024-09-17T21:21:06.170 INFO:teuthology.orchestra.run.smithi184.stdout:5649 read 0x5c28c00 thru 0x5c351d7 (0xc5d8 bytes) 2024-09-17T21:21:06.172 INFO:teuthology.orchestra.run.smithi184.stdout:5650 read 0x2595000 thru 0x259b2aa (0x62ab bytes) 2024-09-17T21:21:06.173 INFO:teuthology.orchestra.run.smithi184.stdout:5652 read 0x2ad9a00 thru 0x2ae6a54 (0xd055 bytes) 2024-09-17T21:21:06.173 INFO:teuthology.orchestra.run.smithi184.stdout:5653 read 0x26bbe00 thru 0x26c9a40 (0xdc41 bytes) 2024-09-17T21:21:06.174 INFO:teuthology.orchestra.run.smithi184.stdout:5654 read 0x6381600 thru 0x638da8a (0xc48b bytes) 2024-09-17T21:21:06.175 INFO:teuthology.orchestra.run.smithi184.stdout:5656 write 0x8a83a00 thru 0x8a83d3e (0x33f bytes) 2024-09-17T21:21:06.176 INFO:teuthology.orchestra.run.smithi184.stdout:5658 read 0x4575a00 thru 0x4582805 (0xce06 bytes) 2024-09-17T21:21:06.177 INFO:teuthology.orchestra.run.smithi184.stdout:5659 write 0x85bda00 thru 0x85beed6 (0x14d7 bytes) 2024-09-17T21:21:06.179 INFO:teuthology.orchestra.run.smithi184.stdout:5660 punch from 0x9e0ca00 to 0x9e1a000, (0xd600 bytes) 2024-09-17T21:21:06.179 INFO:teuthology.orchestra.run.smithi184.stdout:5661 read 0x4fabc00 thru 0x4fb30df (0x74e0 bytes) 2024-09-17T21:21:06.180 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:06.174+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:06.181 INFO:teuthology.orchestra.run.smithi184.stdout:5662 write 0x890e200 thru 0x89119db (0x37dc bytes) 2024-09-17T21:21:06.182 INFO:teuthology.orchestra.run.smithi184.stdout:5666 write 0xb7d400 thru 0xb823b0 (0x4fb1 bytes) 2024-09-17T21:21:06.216 INFO:teuthology.orchestra.run.smithi184.stdout:5671 write 0xe779600 thru 0xe787817 (0xe218 bytes) 2024-09-17T21:21:06.217 INFO:teuthology.orchestra.run.smithi184.stdout:5674 write 0x890000 thru 0x894e7d (0x4e7e bytes) 2024-09-17T21:21:06.218 INFO:teuthology.orchestra.run.smithi184.stdout:5675 write 0xdc38400 thru 0xdc432fd (0xaefe bytes) 2024-09-17T21:21:06.218 INFO:teuthology.orchestra.run.smithi184.stdout:5676 read 0x76a1400 thru 0x76a8157 (0x6d58 bytes) 2024-09-17T21:21:06.218 INFO:teuthology.orchestra.run.smithi184.stdout:5677 write 0x2b69600 thru 0x2b6f6b6 (0x60b7 bytes) 2024-09-17T21:21:06.220 INFO:teuthology.orchestra.run.smithi184.stdout:5679 read 0xd804800 thru 0xd80b737 (0x6f38 bytes) 2024-09-17T21:21:06.220 INFO:teuthology.orchestra.run.smithi184.stdout:5681 write 0x2a56000 thru 0x2a60e14 (0xae15 bytes) 2024-09-17T21:21:06.221 INFO:teuthology.orchestra.run.smithi184.stdout:5683 write 0x30f2a00 thru 0x30f9781 (0x6d82 bytes) 2024-09-17T21:21:06.222 INFO:teuthology.orchestra.run.smithi184.stdout:5686 write 0xc84fe00 thru 0xc85e52b (0xe72c bytes) 2024-09-17T21:21:06.222 INFO:teuthology.orchestra.run.smithi184.stdout:5688 read 0x4de2c00 thru 0x4de505d (0x245e bytes) 2024-09-17T21:21:06.223 INFO:teuthology.orchestra.run.smithi184.stdout:5690 read 0x413c800 thru 0x4149c1e (0xd41f bytes) 2024-09-17T21:21:06.223 INFO:teuthology.orchestra.run.smithi184.stdout:5691 punch from 0x681a000 to 0x6821400, (0x7400 bytes) 2024-09-17T21:21:06.224 INFO:teuthology.orchestra.run.smithi184.stdout:5692 read 0x311cc00 thru 0x311fa14 (0x2e15 bytes) 2024-09-17T21:21:06.246 INFO:teuthology.orchestra.run.smithi184.stdout:5697 trunc from 0xe787a00 to 0x5670e00 2024-09-17T21:21:06.281 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:06.276+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:06.305 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:06.306 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:06.308 INFO:teuthology.orchestra.run.smithi184.stdout:5698 write 0x7cd7c00 thru 0x7cddaf9 (0x5efa bytes) 2024-09-17T21:21:06.308 INFO:teuthology.orchestra.run.smithi184.stdout:5702 punch from 0xcfd000 to 0xd01000, (0x4000 bytes) 2024-09-17T21:21:06.309 INFO:teuthology.orchestra.run.smithi184.stdout:5703 punch from 0x208600 to 0x20ba00, (0x3400 bytes) 2024-09-17T21:21:06.309 INFO:teuthology.orchestra.run.smithi184.stdout:5704 read 0x61e5000 thru 0x61ece3f (0x7e40 bytes) 2024-09-17T21:21:06.310 INFO:teuthology.orchestra.run.smithi184.stdout:5705 read 0x1487000 thru 0x148a65d (0x365e bytes) 2024-09-17T21:21:06.321 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:21:06.322 INFO:teuthology.orchestra.run.smithi184.stdout:5707 trunc from 0x7cddc00 to 0x4b8d000 2024-09-17T21:21:06.368 INFO:teuthology.orchestra.run.smithi184.stdout:5708 trunc from 0x4b8d000 to 0xc3df600 2024-09-17T21:21:06.386 INFO:teuthology.orchestra.run.smithi184.stdout:5709 write 0x35fa200 thru 0x3607ba6 (0xd9a7 bytes) 2024-09-17T21:21:06.389 INFO:teuthology.orchestra.run.smithi184.stdout:5710 punch from 0x7b38400 to 0x7b45600, (0xd200 bytes) 2024-09-17T21:21:06.390 INFO:teuthology.orchestra.run.smithi184.stdout:5713 read 0x7917600 thru 0x79205e4 (0x8fe5 bytes) 2024-09-17T21:21:06.391 INFO:teuthology.orchestra.run.smithi184.stdout:5714 trunc from 0xc3df600 to 0xcb65800 2024-09-17T21:21:06.409 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:06.404+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:06.414 INFO:teuthology.orchestra.run.smithi184.stdout:5715 read 0x91a9c00 thru 0x91b63bd (0xc7be bytes) 2024-09-17T21:21:06.416 INFO:teuthology.orchestra.run.smithi184.stdout:5716 read 0x916f200 thru 0x916f700 (0x501 bytes) 2024-09-17T21:21:06.418 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:21:06.431 INFO:teuthology.orchestra.run.smithi184.stdout:5717 trunc from 0xcb65800 to 0x4cdc800 2024-09-17T21:21:06.460 INFO:teuthology.orchestra.run.smithi184.stdout:5718 read 0x34b5600 thru 0x34b8919 (0x331a bytes) 2024-09-17T21:21:06.483 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:06.476+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:06.487 INFO:teuthology.orchestra.run.smithi184.stdout:5719 write 0x7fb6600 thru 0x7fc4bbb (0xe5bc bytes) 2024-09-17T21:21:06.490 INFO:teuthology.orchestra.run.smithi184.stdout:skipping zero length punch hole 2024-09-17T21:21:06.490 INFO:teuthology.orchestra.run.smithi184.stdout:5721 read 0x5d7400 thru 0x5de2f0 (0x6ef1 bytes) 2024-09-17T21:21:06.491 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:06.491 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:06.491 INFO:teuthology.orchestra.run.smithi184.stdout:5722 punch from 0x614d600 to 0x6154200, (0x6c00 bytes) 2024-09-17T21:21:06.492 INFO:teuthology.orchestra.run.smithi184.stdout:5723 punch from 0xd62600 to 0xd6e800, (0xc200 bytes) 2024-09-17T21:21:06.501 INFO:teuthology.orchestra.run.smithi184.stdout:5724 trunc from 0x7fc4c00 to 0xc199e00 2024-09-17T21:21:06.507 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:21:06.517 INFO:teuthology.orchestra.run.smithi184.stdout:5726 read 0x8fa7000 thru 0x8fb5127 (0xe128 bytes) 2024-09-17T21:21:06.519 INFO:teuthology.orchestra.run.smithi184.stdout:5727 read 0x5416c00 thru 0x541b216 (0x4617 bytes) 2024-09-17T21:21:06.520 INFO:teuthology.orchestra.run.smithi184.stdout:5729 read 0x5ef1400 thru 0x5efa1dc (0x8ddd bytes) 2024-09-17T21:21:06.521 INFO:teuthology.orchestra.run.smithi184.stdout:5731 read 0x2a64e00 thru 0x2a6fd93 (0xaf94 bytes) 2024-09-17T21:21:06.523 INFO:teuthology.orchestra.run.smithi184.stdout:5734 read 0x974de00 thru 0x9759929 (0xbb2a bytes) 2024-09-17T21:21:06.540 INFO:teuthology.orchestra.run.smithi184.stdout:5736 write 0xc6a3800 thru 0xc6b3449 (0xfc4a bytes) 2024-09-17T21:21:06.541 INFO:teuthology.orchestra.run.smithi184.stdout:5740 write 0x7ad9a00 thru 0x7ae529f (0xb8a0 bytes) 2024-09-17T21:21:06.543 INFO:teuthology.orchestra.run.smithi184.stdout:5742 write 0x5966e00 thru 0x5975947 (0xeb48 bytes) 2024-09-17T21:21:06.544 INFO:teuthology.orchestra.run.smithi184.stdout:5743 write 0x6377600 thru 0x637afd0 (0x39d1 bytes) 2024-09-17T21:21:06.545 INFO:teuthology.orchestra.run.smithi184.stdout:5744 write 0x9f30800 thru 0x9f406b5 (0xfeb6 bytes) 2024-09-17T21:21:06.561 INFO:teuthology.orchestra.run.smithi184.stdout:5745 trunc from 0xc6b3600 to 0x69ba600 2024-09-17T21:21:06.572 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:06.572 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:06.583 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:06.578+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:06.591 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:06.611 INFO:teuthology.orchestra.run.smithi184.stdout:5747 punch from 0x34ef800 to 0x34f6400, (0x6c00 bytes) 2024-09-17T21:21:06.611 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:21:06.611 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:21:06.616 INFO:teuthology.orchestra.run.smithi184.stdout:5750 trunc from 0x69ba600 to 0x57a7000 2024-09-17T21:21:06.637 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:06.637 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:06.641 INFO:teuthology.orchestra.run.smithi184.stdout:5751 read 0x30d4a00 thru 0x30d7c2b (0x322c bytes) 2024-09-17T21:21:06.643 INFO:teuthology.orchestra.run.smithi184.stdout:5752 write 0x3689400 thru 0x368cc44 (0x3845 bytes) 2024-09-17T21:21:06.644 INFO:teuthology.orchestra.run.smithi184.stdout:5753 write 0x2fe1400 thru 0x2fe3211 (0x1e12 bytes) 2024-09-17T21:21:06.663 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:06.663 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:06.670 INFO:teuthology.orchestra.run.smithi184.stdout:5754 write 0x6970600 thru 0x6979e72 (0x9873 bytes) 2024-09-17T21:21:06.684 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-17T21:21:06.684 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:06.678+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:06.690 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:06.690 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:06.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:06.690 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:21:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:21:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 168.46992965000001, 2024-09-17T21:21:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:21:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:21:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:21:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:21:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:21:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:21:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:21:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:21:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:21:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:21:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:21:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:21:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:21:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:21:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:21:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:21:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:21:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:21:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:21:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:21:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 163.819628986, 2024-09-17T21:21:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:21:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:21:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:21:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:21:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:21:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:21:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:21:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:21:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:21:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:21:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:21:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:21:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:21:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:21:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:21:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:21:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:21:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:21:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 163.81836390300001, 2024-09-17T21:21:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:21:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:21:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:21:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:21:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:21:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:21:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:21:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:21:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:21:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:21:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:21:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:21:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:21:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:21:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:21:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:21:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:21:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:21:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 163.80858449300001, 2024-09-17T21:21:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:21:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:06.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:06.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:21:06.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:21:06.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:06.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:21:06.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:21:06.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:21:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:21:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:21:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:21:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:21:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:21:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:21:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:21:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:21:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:21:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:21:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:21:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:21:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 163.79986209, 2024-09-17T21:21:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:21:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:21:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:21:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:21:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:21:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:21:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:21:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:21:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:21:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:21:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:21:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:21:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:21:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:21:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:21:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:21:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:21:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:21:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 150.41477152600001, 2024-09-17T21:21:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:21:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:06.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:21:06.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:21:06.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:06.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:21:06.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:21:06.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:06.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:21:06.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:21:06.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:06.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:21:06.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:21:06.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:06.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:21:06.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:21:06.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:06.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:21:06.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:21:06.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:06.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:21:06.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:21:06.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:06.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:21:06.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:21:06.718 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.718 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:06.718 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.719 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1780 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'195, mlcod=69'190)", 2024-09-17T21:21:06.719 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:06.719 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 113.436915922, 2024-09-17T21:21:06.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075761769999999999, 2024-09-17T21:21:06.719 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:06.719 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:06.719 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:06.719 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:06.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:06.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:06.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.720 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:06.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:06.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.720 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:06.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250375+0000", 2024-09-17T21:21:06.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:21:06.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.720 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:06.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250383+0000", 2024-09-17T21:21:06.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7870000000000002e-06 2024-09-17T21:21:06.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:06.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250402+0000", 2024-09-17T21:21:06.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9346000000000001e-05 2024-09-17T21:21:06.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:06.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250413+0000", 2024-09-17T21:21:06.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.039e-05 2024-09-17T21:21:06.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:06.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251353+0000", 2024-09-17T21:21:06.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000940091 2024-09-17T21:21:06.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:06.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251407+0000", 2024-09-17T21:21:06.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4701999999999999e-05 2024-09-17T21:21:06.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:06.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257829+0000", 2024-09-17T21:21:06.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064212779999999999 2024-09-17T21:21:06.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:06.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257950+0000", 2024-09-17T21:21:06.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121623 2024-09-17T21:21:06.723 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.723 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:06.723 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.723 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1781 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'196, mlcod=69'190)", 2024-09-17T21:21:06.724 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:21:06.724 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 113.435832971, 2024-09-17T21:21:06.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065632939999999999, 2024-09-17T21:21:06.724 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:06.724 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:06.724 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:06.724 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:06.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:06.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:21:06.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:06.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:21:06.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:06.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251459+0000", 2024-09-17T21:21:06.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.624e-06 2024-09-17T21:21:06.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:06.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251471+0000", 2024-09-17T21:21:06.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2636000000000001e-05 2024-09-17T21:21:06.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:06.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251498+0000", 2024-09-17T21:21:06.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:21:06.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:06.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251515+0000", 2024-09-17T21:21:06.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6939000000000002e-05 2024-09-17T21:21:06.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:06.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252938+0000", 2024-09-17T21:21:06.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001423082 2024-09-17T21:21:06.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:06.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252999+0000", 2024-09-17T21:21:06.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1138999999999999e-05 2024-09-17T21:21:06.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:06.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257955+0000", 2024-09-17T21:21:06.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049559640000000002 2024-09-17T21:21:06.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:06.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.258020+0000", 2024-09-17T21:21:06.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5504000000000002e-05 2024-09-17T21:21:06.728 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.728 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:06.728 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.728 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1788 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'198, mlcod=69'191)", 2024-09-17T21:21:06.728 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:06.728 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 113.43380748200001, 2024-09-17T21:21:06.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073690370000000002, 2024-09-17T21:21:06.729 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:06.729 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:06.729 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:06.729 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:06.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:06.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:06.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:06.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:06.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:06.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253484+0000", 2024-09-17T21:21:06.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2470000000000001e-06 2024-09-17T21:21:06.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:06.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253492+0000", 2024-09-17T21:21:06.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7159999999999992e-06 2024-09-17T21:21:06.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:06.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253509+0000", 2024-09-17T21:21:06.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.666e-05 2024-09-17T21:21:06.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:06.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253522+0000", 2024-09-17T21:21:06.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2643000000000001e-05 2024-09-17T21:21:06.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:06.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254853+0000", 2024-09-17T21:21:06.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331238 2024-09-17T21:21:06.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:06.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254885+0000", 2024-09-17T21:21:06.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1562e-05 2024-09-17T21:21:06.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:06.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260788+0000", 2024-09-17T21:21:06.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059030749999999998 2024-09-17T21:21:06.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:06.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260852+0000", 2024-09-17T21:21:06.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3895999999999999e-05 2024-09-17T21:21:06.732 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.732 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:06.732 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.733 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5746 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'477, mlcod=71'473)", 2024-09-17T21:21:06.733 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:06.733 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.428379924, 2024-09-17T21:21:06.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074182709999999997, 2024-09-17T21:21:06.733 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:06.733 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:06.733 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:06.733 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:06.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:06.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:06.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:06.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:06.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:06.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258912+0000", 2024-09-17T21:21:06.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:21:06.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:06.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258926+0000", 2024-09-17T21:21:06.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4249e-05 2024-09-17T21:21:06.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:06.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258946+0000", 2024-09-17T21:21:06.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9879999999999999e-05 2024-09-17T21:21:06.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:06.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258956+0000", 2024-09-17T21:21:06.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:21:06.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:06.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259380+0000", 2024-09-17T21:21:06.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042337599999999999 2024-09-17T21:21:06.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:06.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259412+0000", 2024-09-17T21:21:06.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:21:06.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:06.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266191+0000", 2024-09-17T21:21:06.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067793269999999999 2024-09-17T21:21:06.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:06.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266328+0000", 2024-09-17T21:21:06.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013690999999999999 2024-09-17T21:21:06.737 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.737 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:06.737 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.737 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5747 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'478, mlcod=71'473)", 2024-09-17T21:21:06.737 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:06.737 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.427673575, 2024-09-17T21:21:06.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068779440000000004, 2024-09-17T21:21:06.738 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:06.738 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:06.738 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:06.738 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:06.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:06.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:06.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:06.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:06.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:06.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259618+0000", 2024-09-17T21:21:06.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6199999999999999e-06 2024-09-17T21:21:06.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:06.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259630+0000", 2024-09-17T21:21:06.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1884e-05 2024-09-17T21:21:06.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:06.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259657+0000", 2024-09-17T21:21:06.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7339e-05 2024-09-17T21:21:06.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:06.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259677+0000", 2024-09-17T21:21:06.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0156000000000001e-05 2024-09-17T21:21:06.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:06.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260521+0000", 2024-09-17T21:21:06.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084312899999999997 2024-09-17T21:21:06.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:06.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260585+0000", 2024-09-17T21:21:06.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4558000000000001e-05 2024-09-17T21:21:06.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:06.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266336+0000", 2024-09-17T21:21:06.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057505020000000002 2024-09-17T21:21:06.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:06.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266494+0000", 2024-09-17T21:21:06.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158756 2024-09-17T21:21:06.742 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.742 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:06.742 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.742 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5750 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'479, mlcod=71'473)", 2024-09-17T21:21:06.742 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:21:06.742 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.426961382, 2024-09-17T21:21:06.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006302874, 2024-09-17T21:21:06.742 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:06.742 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:06.743 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:06.743 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:06.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:06.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:21:06.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:06.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260329+0000", 2024-09-17T21:21:06.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:06.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260330+0000", 2024-09-17T21:21:06.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.567e-06 2024-09-17T21:21:06.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:06.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260343+0000", 2024-09-17T21:21:06.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2862e-05 2024-09-17T21:21:06.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:06.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260367+0000", 2024-09-17T21:21:06.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3723e-05 2024-09-17T21:21:06.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:06.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260386+0000", 2024-09-17T21:21:06.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8879999999999999e-05 2024-09-17T21:21:06.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:06.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261579+0000", 2024-09-17T21:21:06.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001193431 2024-09-17T21:21:06.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:06.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261608+0000", 2024-09-17T21:21:06.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8858999999999999e-05 2024-09-17T21:21:06.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:06.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266505+0000", 2024-09-17T21:21:06.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004896999 2024-09-17T21:21:06.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:06.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266631+0000", 2024-09-17T21:21:06.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012655300000000001 2024-09-17T21:21:06.747 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.747 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:06.747 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.747 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5751 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'480, mlcod=71'473)", 2024-09-17T21:21:06.747 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:06.747 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.42625416999999, 2024-09-17T21:21:06.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075756299999999999, 2024-09-17T21:21:06.747 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:06.747 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:06.747 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:06.747 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:06.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:06.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:06.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:06.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:06.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:06.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261037+0000", 2024-09-17T21:21:06.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:21:06.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:06.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261049+0000", 2024-09-17T21:21:06.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1063999999999999e-05 2024-09-17T21:21:06.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:06.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261072+0000", 2024-09-17T21:21:06.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3102e-05 2024-09-17T21:21:06.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:06.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261133+0000", 2024-09-17T21:21:06.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1276000000000001e-05 2024-09-17T21:21:06.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:06.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:21:06.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015334319999999999 2024-09-17T21:21:06.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:06.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262702+0000", 2024-09-17T21:21:06.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5281999999999999e-05 2024-09-17T21:21:06.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:06.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268527+0000", 2024-09-17T21:21:06.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058250389999999997 2024-09-17T21:21:06.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:06.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268611+0000", 2024-09-17T21:21:06.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4724999999999999e-05 2024-09-17T21:21:06.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.752 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:06.752 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.752 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5756 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'481, mlcod=71'473)", 2024-09-17T21:21:06.752 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:06.752 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.425393314, 2024-09-17T21:21:06.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006772386, 2024-09-17T21:21:06.752 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:06.752 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:06.752 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:06.753 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:06.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:06.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:06.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:06.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:06.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:06.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261899+0000", 2024-09-17T21:21:06.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.903e-06 2024-09-17T21:21:06.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:06.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261912+0000", 2024-09-17T21:21:06.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-05 2024-09-17T21:21:06.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:06.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261938+0000", 2024-09-17T21:21:06.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6098999999999999e-05 2024-09-17T21:21:06.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:06.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261955+0000", 2024-09-17T21:21:06.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.696e-05 2024-09-17T21:21:06.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:06.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263845+0000", 2024-09-17T21:21:06.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001890187 2024-09-17T21:21:06.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:06.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263907+0000", 2024-09-17T21:21:06.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1600999999999995e-05 2024-09-17T21:21:06.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:06.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268615+0000", 2024-09-17T21:21:06.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047086070000000001 2024-09-17T21:21:06.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:06.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268669+0000", 2024-09-17T21:21:06.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3588999999999998e-05 2024-09-17T21:21:06.756 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.756 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:06.756 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.756 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4978.0:598 3.9 e80/23 3:9701cc17:::rbd_data.1152724ff2a3.0000000000000019:head v 80'202, mlcod=80'198)", 2024-09-17T21:21:06.757 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:21:06.757 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 99.923818931, 2024-09-17T21:21:06.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066473340000000004, 2024-09-17T21:21:06.757 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:06.757 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:06.757 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:06.757 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:06.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:06.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:21:06.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:06.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:21:06.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:06.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763472+0000", 2024-09-17T21:21:06.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6899999999999997e-07 2024-09-17T21:21:06.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:06.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763478+0000", 2024-09-17T21:21:06.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:21:06.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:06.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763489+0000", 2024-09-17T21:21:06.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0628999999999999e-05 2024-09-17T21:21:06.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:06.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763495+0000", 2024-09-17T21:21:06.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6649999999999997e-06 2024-09-17T21:21:06.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:06.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763907+0000", 2024-09-17T21:21:06.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041163899999999999 2024-09-17T21:21:06.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:06.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763976+0000", 2024-09-17T21:21:06.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8534999999999993e-05 2024-09-17T21:21:06.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:06.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.769900+0000", 2024-09-17T21:21:06.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059243270000000001 2024-09-17T21:21:06.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:06.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.770118+0000", 2024-09-17T21:21:06.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021842500000000001 2024-09-17T21:21:06.760 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.761 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:06.761 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.761 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:468 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'94, mlcod=89'87)", 2024-09-17T21:21:06.761 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:06.761 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 87.001073632000001, 2024-09-17T21:21:06.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066528790000000004, 2024-09-17T21:21:06.761 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:06.761 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:06.761 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:06.761 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:06.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:06.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:06.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:06.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:06.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:06.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686218+0000", 2024-09-17T21:21:06.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3060000000000001e-06 2024-09-17T21:21:06.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:06.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686256+0000", 2024-09-17T21:21:06.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8333e-05 2024-09-17T21:21:06.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:06.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686272+0000", 2024-09-17T21:21:06.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:21:06.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:06.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686283+0000", 2024-09-17T21:21:06.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0903e-05 2024-09-17T21:21:06.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:06.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686682+0000", 2024-09-17T21:21:06.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039935099999999999 2024-09-17T21:21:06.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:06.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686785+0000", 2024-09-17T21:21:06.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010275199999999999 2024-09-17T21:21:06.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:06.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692816+0000", 2024-09-17T21:21:06.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060317579999999999 2024-09-17T21:21:06.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:06.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692869+0000", 2024-09-17T21:21:06.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2775999999999998e-05 2024-09-17T21:21:06.765 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.765 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:06.765 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.766 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:755 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'127, mlcod=89'122)", 2024-09-17T21:21:06.766 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:06.766 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 86.864618418000006, 2024-09-17T21:21:06.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084014080000000008, 2024-09-17T21:21:06.766 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:06.766 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:06.766 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:06.766 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:06.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:06.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:06.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:06.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:06.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:06.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:06.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:21:06.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:06.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822683+0000", 2024-09-17T21:21:06.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0674e-05 2024-09-17T21:21:06.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:06.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822698+0000", 2024-09-17T21:21:06.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5189e-05 2024-09-17T21:21:06.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:06.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822708+0000", 2024-09-17T21:21:06.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9119999999999993e-06 2024-09-17T21:21:06.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:06.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822748+0000", 2024-09-17T21:21:06.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9727000000000001e-05 2024-09-17T21:21:06.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:06.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822759+0000", 2024-09-17T21:21:06.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0865e-05 2024-09-17T21:21:06.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:06.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831034+0000", 2024-09-17T21:21:06.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082750710000000002 2024-09-17T21:21:06.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:06.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831073+0000", 2024-09-17T21:21:06.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9063999999999997e-05 2024-09-17T21:21:06.770 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.770 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:06.770 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.770 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:756 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'128, mlcod=89'124)", 2024-09-17T21:21:06.770 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:06.770 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 86.862695786000003, 2024-09-17T21:21:06.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0076300270000000002, 2024-09-17T21:21:06.770 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:06.770 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:06.770 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:06.770 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:06.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:06.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:06.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:06.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:06.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:06.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824595+0000", 2024-09-17T21:21:06.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-17T21:21:06.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:06.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824600+0000", 2024-09-17T21:21:06.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:21:06.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:06.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824612+0000", 2024-09-17T21:21:06.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.181e-05 2024-09-17T21:21:06.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:06.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824619+0000", 2024-09-17T21:21:06.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:21:06.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:06.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824671+0000", 2024-09-17T21:21:06.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2262999999999998e-05 2024-09-17T21:21:06.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:06.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824695+0000", 2024-09-17T21:21:06.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3883000000000002e-05 2024-09-17T21:21:06.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:06.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832185+0000", 2024-09-17T21:21:06.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007490048 2024-09-17T21:21:06.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:06.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832224+0000", 2024-09-17T21:21:06.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8957e-05 2024-09-17T21:21:06.774 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.774 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:06.774 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.774 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2617 3.d e90/23 3:b0143718:::rbd_data.1152a8ae2019.0000000000000016:head v 90'249, mlcod=90'246)", 2024-09-17T21:21:06.775 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:21:06.775 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 85.842844786000001, 2024-09-17T21:21:06.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063715559999999996, 2024-09-17T21:21:06.775 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:06.775 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:06.775 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:06.775 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:06.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:06.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:21:06.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:06.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:21:06.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:06.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844455+0000", 2024-09-17T21:21:06.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4259999999999992e-06 2024-09-17T21:21:06.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:06.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844470+0000", 2024-09-17T21:21:06.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5386000000000001e-05 2024-09-17T21:21:06.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:06.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844499+0000", 2024-09-17T21:21:06.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.883e-05 2024-09-17T21:21:06.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:06.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844518+0000", 2024-09-17T21:21:06.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8830000000000001e-05 2024-09-17T21:21:06.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:06.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844649+0000", 2024-09-17T21:21:06.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013143399999999999 2024-09-17T21:21:06.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:06.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844702+0000", 2024-09-17T21:21:06.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2876e-05 2024-09-17T21:21:06.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:06.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850696+0000", 2024-09-17T21:21:06.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059943729999999999 2024-09-17T21:21:06.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:06.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850817+0000", 2024-09-17T21:21:06.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120401 2024-09-17T21:21:06.779 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.779 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:06.779 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.779 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2635 3.d e90/23 3:b3d1e920:::rbd_data.1152a8ae2019.000000000000001f:head v 90'252, mlcod=90'246)", 2024-09-17T21:21:06.779 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:06.779 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 85.840136716000004, 2024-09-17T21:21:06.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0077265190000000003, 2024-09-17T21:21:06.779 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:06.779 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:06.780 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:06.780 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:06.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:06.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:06.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:06.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:06.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:06.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:06.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847155+0000", 2024-09-17T21:21:06.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.544e-06 2024-09-17T21:21:06.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:06.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847165+0000", 2024-09-17T21:21:06.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.043e-05 2024-09-17T21:21:06.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:06.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847189+0000", 2024-09-17T21:21:06.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3659000000000002e-05 2024-09-17T21:21:06.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:06.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847210+0000", 2024-09-17T21:21:06.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1163e-05 2024-09-17T21:21:06.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:06.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848856+0000", 2024-09-17T21:21:06.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001645521 2024-09-17T21:21:06.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:06.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848905+0000", 2024-09-17T21:21:06.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9048999999999997e-05 2024-09-17T21:21:06.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:06.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854770+0000", 2024-09-17T21:21:06.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058652879999999998 2024-09-17T21:21:06.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:06.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:06.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:06.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854880+0000", 2024-09-17T21:21:06.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010986500000000001 2024-09-17T21:21:06.783 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.783 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:06.783 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.783 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:06.783 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:06.783 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:06.784 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:06.784 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:06.786 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:21:06.788 INFO:teuthology.orchestra.run.smithi184.stdout:5755 write 0x8951c00 thru 0x89560c0 (0x44c1 bytes) 2024-09-17T21:21:06.788 INFO:teuthology.orchestra.run.smithi184.stdout:5757 write 0x40c1800 thru 0x40ca291 (0x8a92 bytes) 2024-09-17T21:21:06.788 INFO:teuthology.orchestra.run.smithi184.stdout:5759 punch from 0x36cde00 to 0x36d9e00, (0xc000 bytes) 2024-09-17T21:21:06.788 INFO:teuthology.orchestra.run.smithi184.stdout:5762 write 0x5eff000 thru 0x5f0519f (0x61a0 bytes) 2024-09-17T21:21:06.788 INFO:teuthology.orchestra.run.smithi184.stdout:5763 write 0xe71f800 thru 0xe72a1cb (0xa9cc bytes) 2024-09-17T21:21:06.788 INFO:teuthology.orchestra.run.smithi184.stdout:5766 read 0x966ba00 thru 0x966d48c (0x1a8d bytes) 2024-09-17T21:21:06.788 INFO:teuthology.orchestra.run.smithi184.stdout:5767 read 0x4059a00 thru 0x406116b (0x776c bytes) 2024-09-17T21:21:06.788 INFO:teuthology.orchestra.run.smithi184.stdout:5768 read 0x1834200 thru 0x183b4ce (0x72cf bytes) 2024-09-17T21:21:06.789 INFO:teuthology.orchestra.run.smithi184.stdout:5769 write 0x6d5d600 thru 0x6d67827 (0xa228 bytes) 2024-09-17T21:21:06.789 INFO:teuthology.orchestra.run.smithi184.stdout:5770 write 0xe4a4c00 thru 0xe4a5457 (0x858 bytes) 2024-09-17T21:21:06.789 INFO:teuthology.orchestra.run.smithi184.stdout:5771 write 0x1f70a00 thru 0x1f74a2a (0x402b bytes) 2024-09-17T21:21:06.789 INFO:teuthology.orchestra.run.smithi184.stdout:5772 write 0x910ac00 thru 0x911709d (0xc49e bytes) 2024-09-17T21:21:06.789 INFO:teuthology.orchestra.run.smithi184.stdout:5775 punch from 0x6442200 to 0x6443400, (0x1200 bytes) 2024-09-17T21:21:06.789 INFO:teuthology.orchestra.run.smithi184.stdout:5776 write 0x85ddc00 thru 0x85e87e2 (0xabe3 bytes) 2024-09-17T21:21:06.789 INFO:teuthology.orchestra.run.smithi184.stdout:5777 read 0x703ae00 thru 0x7047412 (0xc613 bytes) 2024-09-17T21:21:06.789 INFO:teuthology.orchestra.run.smithi184.stdout:5779 read 0x9533e00 thru 0x95357aa (0x19ab bytes) 2024-09-17T21:21:06.789 INFO:teuthology.orchestra.run.smithi184.stdout:5781 read 0xc50a000 thru 0xc517695 (0xd696 bytes) 2024-09-17T21:21:06.789 INFO:teuthology.orchestra.run.smithi184.stdout:5783 write 0x950c00 thru 0x95d9c7 (0xcdc8 bytes) 2024-09-17T21:21:06.789 INFO:teuthology.orchestra.run.smithi184.stdout:5785 read 0x9f68c00 thru 0x9f71414 (0x8815 bytes) 2024-09-17T21:21:06.789 INFO:teuthology.orchestra.run.smithi184.stdout:5786 trunc from 0xe72a200 to 0x3159800 2024-09-17T21:21:06.823 INFO:teuthology.orchestra.run.smithi184.stdout:5787 punch from 0x134e400 to 0x1355800, (0x7400 bytes) 2024-09-17T21:21:06.829 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:06.824+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:06.861 INFO:teuthology.orchestra.run.smithi184.stdout:5789 write 0x9868c00 thru 0x986aab8 (0x1eb9 bytes) 2024-09-17T21:21:06.863 INFO:teuthology.orchestra.run.smithi184.stdout:5790 read 0x2afea00 thru 0x2b045cd (0x5bce bytes) 2024-09-17T21:21:06.891 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:06.884+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:06.892 INFO:teuthology.orchestra.run.smithi184.stdout:5791 write 0xb880200 thru 0xb88ba9c (0xb89d bytes) 2024-09-17T21:21:06.924 INFO:teuthology.orchestra.run.smithi184.stdout:5792 trunc from 0xb88bc00 to 0x40c800 2024-09-17T21:21:06.932 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:21736 3.a e151/85 3:553e7ea9:::rbd_data.1152f056b73f.0000000000000009:head v 151'303, mlcod=150'301)", 2024-09-17T21:21:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:06.928935+0000", 2024-09-17T21:21:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.00210772, 2024-09-17T21:21:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021549160000000002, 2024-09-17T21:21:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:06.928935+0000", 2024-09-17T21:21:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:06.928935+0000", 2024-09-17T21:21:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:06.928936+0000", 2024-09-17T21:21:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.513e-06 2024-09-17T21:21:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:06.928960+0000", 2024-09-17T21:21:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3918999999999999e-05 2024-09-17T21:21:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:06.928982+0000", 2024-09-17T21:21:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1322999999999999e-05 2024-09-17T21:21:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:06.928992+0000", 2024-09-17T21:21:06.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0047999999999999e-05 2024-09-17T21:21:06.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:06.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:06.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:06.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:06.929106+0000", 2024-09-17T21:21:06.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011435899999999999 2024-09-17T21:21:06.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:06.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:06.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:06.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:06.929147+0000", 2024-09-17T21:21:06.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0748999999999999e-05 2024-09-17T21:21:06.936 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:06.937 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:06.937 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:06.937 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:06.937 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-09-17T21:21:06.937 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 1 2024-09-17T21:21:06.937 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:06.950 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:21:06.980 INFO:teuthology.orchestra.run.smithi184.stdout:5794 punch from 0xd5600 to 0xd7a00, (0x2400 bytes) 2024-09-17T21:21:06.995 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:06.990+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:07.002 INFO:teuthology.orchestra.run.smithi184.stdout:5797 trunc from 0x40c800 to 0x6ebca00 2024-09-17T21:21:07.027 INFO:teuthology.orchestra.run.smithi184.stdout:5798 read 0x47fd800 thru 0x480932a (0xbb2b bytes) 2024-09-17T21:21:07.043 INFO:teuthology.orchestra.run.smithi184.stdout:5801 trunc from 0x6ebca00 to 0x1bcb400 2024-09-17T21:21:07.072 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:07.072 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:07.073 INFO:teuthology.orchestra.run.smithi184.stdout:5802 read 0xdc7a00 thru 0xdd6e98 (0xf499 bytes) 2024-09-17T21:21:07.087 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:21:07.120 INFO:teuthology.orchestra.run.smithi184.stdout:5804 write 0xc07ae00 thru 0xc07f425 (0x4626 bytes) 2024-09-17T21:21:07.121 INFO:teuthology.orchestra.run.smithi184.stdout:5806 write 0x22a2e00 thru 0x22b08ed (0xdaee bytes) 2024-09-17T21:21:07.123 INFO:teuthology.orchestra.run.smithi184.stdout:5807 read 0x1113a00 thru 0x111bc14 (0x8215 bytes) 2024-09-17T21:21:07.125 INFO:teuthology.orchestra.run.smithi184.stdout:5810 read 0xb5bba00 thru 0xb5c80ab (0xc6ac bytes) 2024-09-17T21:21:07.125 INFO:teuthology.orchestra.run.smithi184.stdout:5811 punch from 0x509400 to 0x50e800, (0x5400 bytes) 2024-09-17T21:21:07.126 INFO:teuthology.orchestra.run.smithi184.stdout:5812 write 0x23ffc00 thru 0x240ae50 (0xb251 bytes) 2024-09-17T21:21:07.130 INFO:teuthology.orchestra.run.smithi184.stdout:5813 punch from 0xb0f4a00 to 0xb100800, (0xbe00 bytes) 2024-09-17T21:21:07.131 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:07.126+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:07.159 INFO:teuthology.orchestra.run.smithi184.stdout:5814 write 0xe091600 thru 0xe09a0d8 (0x8ad9 bytes) 2024-09-17T21:21:07.183 INFO:teuthology.orchestra.run.smithi184.stdout:5815 trunc from 0xe09a200 to 0xc1800 2024-09-17T21:21:07.193 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:07.188+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:07.239 INFO:teuthology.orchestra.run.smithi184.stdout:5818 read 0x60800 thru 0x69848 (0x9049 bytes) 2024-09-17T21:21:07.262 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:07.262 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:07.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:07.263 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:07.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.263 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:21:07.263 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:07.263 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 190.245360826, 2024-09-17T21:21:07.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:21:07.263 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.263 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.263 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:07.263 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.263 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:21:07.264 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:21:07.264 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:21:07.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.264 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:07.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:07.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:21:07.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:21:07.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:21:07.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:21:07.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:21:07.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:21:07.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:21:07.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:21:07.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:21:07.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:21:07.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:07.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:21:07.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:21:07.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:21:07.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:21:07.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:21:07.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:21:07.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:21:07.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:21:07.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:21:07.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:07.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:21:07.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:21:07.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:07.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:21:07.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:21:07.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:07.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:21:07.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:21:07.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:21:07.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:21:07.269 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.270 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.270 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.270 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:07.270 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:07.270 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 170.665196546, 2024-09-17T21:21:07.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:21:07.270 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.270 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.270 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.270 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.270 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:21:07.271 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:21:07.271 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:21:07.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.271 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:07.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:07.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:21:07.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:21:07.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:21:07.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:21:07.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:21:07.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:21:07.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:21:07.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:21:07.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:21:07.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:21:07.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:07.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:21:07.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:21:07.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:21:07.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:21:07.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:07.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:21:07.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:21:07.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:21:07.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:21:07.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:21:07.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:21:07.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:21:07.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:21:07.276 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.276 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.276 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.276 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:07.276 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:07.276 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 170.665099234, 2024-09-17T21:21:07.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:21:07.276 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.276 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.276 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.277 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.277 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:07.277 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:07.277 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:21:07.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.277 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:07.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:07.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:21:07.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:21:07.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:21:07.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:21:07.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:21:07.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:21:07.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:21:07.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:21:07.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:21:07.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:21:07.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:07.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:21:07.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:21:07.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:21:07.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:21:07.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:07.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:21:07.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:21:07.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:21:07.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:21:07.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:21:07.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:21:07.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:21:07.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:21:07.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:21:07.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:21:07.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:21:07.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:21:07.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:21:07.283 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.283 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.283 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.283 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:07.283 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:07.283 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 170.66301243300001, 2024-09-17T21:21:07.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:21:07.283 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.284 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.284 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:07.284 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.284 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:21:07.284 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:21:07.284 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:21:07.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.284 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:07.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:07.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:21:07.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:21:07.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:21:07.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:21:07.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:21:07.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:21:07.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:21:07.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:21:07.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:21:07.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:21:07.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:21:07.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:21:07.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:21:07.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:21:07.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:21:07.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:07.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:21:07.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:21:07.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:21:07.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:21:07.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:21:07.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:21:07.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:07.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:21:07.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:21:07.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:07.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:21:07.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:21:07.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:07.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:21:07.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:21:07.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:07.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:21:07.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:21:07.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:21:07.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:21:07.290 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.290 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.291 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.291 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:07.291 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:07.291 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 137.979113652, 2024-09-17T21:21:07.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:21:07.291 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.291 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.291 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.291 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.291 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:07.291 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:07.291 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:21:07.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.292 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:07.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:07.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:21:07.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:21:07.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:21:07.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:21:07.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:21:07.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:21:07.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:21:07.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:21:07.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:21:07.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:21:07.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:07.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:21:07.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:21:07.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:21:07.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:21:07.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:07.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:21:07.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:21:07.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:21:07.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:21:07.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:21:07.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:21:07.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:21:07.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:21:07.298 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.298 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.298 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.298 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:07.298 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:07.298 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 137.970021941, 2024-09-17T21:21:07.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:21:07.298 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.298 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.298 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.298 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.298 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:07.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:07.299 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:21:07.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.299 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:07.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:07.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:21:07.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:21:07.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:21:07.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:21:07.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:21:07.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:21:07.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:21:07.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:21:07.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:21:07.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:21:07.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:07.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:21:07.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:21:07.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:21:07.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:21:07.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:21:07.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:21:07.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:21:07.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:21:07.303 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.303 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.303 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.303 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:07.303 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:07.303 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 137.96984989000001, 2024-09-17T21:21:07.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:21:07.303 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.303 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.303 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.303 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.303 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:07.303 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:07.304 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:21:07.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.304 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:07.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:07.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:21:07.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:21:07.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:21:07.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:21:07.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:21:07.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:21:07.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:21:07.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:21:07.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:21:07.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:21:07.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:07.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:21:07.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:21:07.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:21:07.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:21:07.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:21:07.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:21:07.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:21:07.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:21:07.308 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.308 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.308 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.309 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:07.309 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:07.309 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 137.79491714299999, 2024-09-17T21:21:07.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:21:07.309 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.309 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.309 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.309 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.309 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:07.309 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:07.310 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:21:07.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.310 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:07.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:07.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:21:07.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:21:07.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:21:07.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:21:07.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:21:07.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:21:07.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:21:07.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:21:07.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:21:07.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:21:07.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:07.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:21:07.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:21:07.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:21:07.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:21:07.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:21:07.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:21:07.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:21:07.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:21:07.314 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.314 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.314 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.314 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:07.314 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:07.314 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 137.79483258600001, 2024-09-17T21:21:07.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:21:07.314 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.315 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.315 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.315 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.315 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:07.315 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:07.315 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:21:07.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.315 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:07.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:07.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:07.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:21:07.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:21:07.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:21:07.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:21:07.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:21:07.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:21:07.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:21:07.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:21:07.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:21:07.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:07.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:21:07.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:21:07.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:21:07.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:21:07.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:21:07.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:21:07.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:21:07.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:21:07.319 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.319 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.319 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.320 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:07.320 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:07.320 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 137.79476694900001, 2024-09-17T21:21:07.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:21:07.320 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.320 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.320 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.320 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.320 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:07.320 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:07.320 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:21:07.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.320 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:07.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:07.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:21:07.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:21:07.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:21:07.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:21:07.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:21:07.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:21:07.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:21:07.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:21:07.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:21:07.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:21:07.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:07.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:21:07.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:21:07.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:21:07.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:21:07.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:21:07.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:21:07.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:21:07.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:21:07.324 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.324 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.325 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.325 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:07.325 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:07.325 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 86.421191852000007, 2024-09-17T21:21:07.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:21:07.325 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.325 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.325 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.325 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.325 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:21:07.325 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:07.325 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:21:07.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.326 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:07.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:07.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:21:07.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:21:07.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:21:07.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:21:07.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:21:07.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:21:07.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:21:07.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:21:07.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:21:07.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:21:07.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:21:07.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:21:07.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:21:07.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:21:07.329 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.329 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.329 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.329 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:07.329 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:07.329 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 86.421072339999995, 2024-09-17T21:21:07.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:21:07.329 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.329 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.330 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.330 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.330 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:21:07.330 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:07.330 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:21:07.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.330 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:07.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:07.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:21:07.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:21:07.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:21:07.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:21:07.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:21:07.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:21:07.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:21:07.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:21:07.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:21:07.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:21:07.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:21:07.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:21:07.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:21:07.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:21:07.334 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.334 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.334 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.334 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:994 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 571392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:07.334 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:21:07.334 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.828100366000001, 2024-09-17T21:21:07.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083898465000000005, 2024-09-17T21:21:07.334 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.335 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.335 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.335 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.335 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:07.335 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:07.335 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 994 2024-09-17T21:21:07.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.335 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:21:07.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:21:07.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:21:07.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:21:07.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431058+0000", 2024-09-17T21:21:07.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1667e-05 2024-09-17T21:21:07.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431067+0000", 2024-09-17T21:21:07.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0720000000000006e-06 2024-09-17T21:21:07.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431075+0000", 2024-09-17T21:21:07.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9200000000000004e-06 2024-09-17T21:21:07.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514671+0000", 2024-09-17T21:21:07.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083596229999999994 2024-09-17T21:21:07.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514741+0000", 2024-09-17T21:21:07.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9929000000000001e-05 2024-09-17T21:21:07.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514924+0000", 2024-09-17T21:21:07.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000183122 2024-09-17T21:21:07.338 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.338 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.338 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.339 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:995 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 575488~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:07.339 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:21:07.339 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.828041349999999, 2024-09-17T21:21:07.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084214699000000004, 2024-09-17T21:21:07.339 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.339 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.339 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.339 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.339 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:07.339 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:07.339 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 995 2024-09-17T21:21:07.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.339 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:21:07.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:21:07.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431086+0000", 2024-09-17T21:21:07.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7999999999999995e-07 2024-09-17T21:21:07.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431088+0000", 2024-09-17T21:21:07.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4729999999999999e-06 2024-09-17T21:21:07.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431126+0000", 2024-09-17T21:21:07.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7583999999999998e-05 2024-09-17T21:21:07.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431137+0000", 2024-09-17T21:21:07.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1768e-05 2024-09-17T21:21:07.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515003+0000", 2024-09-17T21:21:07.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083865097 2024-09-17T21:21:07.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515115+0000", 2024-09-17T21:21:07.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112561 2024-09-17T21:21:07.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515300+0000", 2024-09-17T21:21:07.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018453600000000001 2024-09-17T21:21:07.343 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.343 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.343 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.343 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:996 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:07.343 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:07.343 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.827958969000001, 2024-09-17T21:21:07.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084493736999999999, 2024-09-17T21:21:07.343 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.343 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.343 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.343 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.343 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:07.344 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:07.344 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 996 2024-09-17T21:21:07.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.344 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:07.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:07.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431168+0000", 2024-09-17T21:21:07.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4e-07 2024-09-17T21:21:07.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431172+0000", 2024-09-17T21:21:07.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.608e-06 2024-09-17T21:21:07.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431180+0000", 2024-09-17T21:21:07.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5650000000000002e-06 2024-09-17T21:21:07.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431188+0000", 2024-09-17T21:21:07.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:21:07.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515354+0000", 2024-09-17T21:21:07.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084165977000000003 2024-09-17T21:21:07.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515451+0000", 2024-09-17T21:21:07.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6669000000000006e-05 2024-09-17T21:21:07.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515661+0000", 2024-09-17T21:21:07.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021021599999999999 2024-09-17T21:21:07.347 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.347 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.347 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.348 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:997 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 583680~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:07.348 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:07.348 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.827927574, 2024-09-17T21:21:07.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084772419000000002, 2024-09-17T21:21:07.348 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.348 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.348 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.348 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.348 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:07.348 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:07.348 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 997 2024-09-17T21:21:07.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.349 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:07.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:07.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:07.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0400000000000005e-07 2024-09-17T21:21:07.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431202+0000", 2024-09-17T21:21:07.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:21:07.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431208+0000", 2024-09-17T21:21:07.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4740000000000002e-06 2024-09-17T21:21:07.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431214+0000", 2024-09-17T21:21:07.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6550000000000001e-06 2024-09-17T21:21:07.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515741+0000", 2024-09-17T21:21:07.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084526904 2024-09-17T21:21:07.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515826+0000", 2024-09-17T21:21:07.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4643999999999998e-05 2024-09-17T21:21:07.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515971+0000", 2024-09-17T21:21:07.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145444 2024-09-17T21:21:07.352 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.352 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.352 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.352 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:998 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 587776~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:07.352 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:07.352 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.827904601999997, 2024-09-17T21:21:07.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085108565999999997, 2024-09-17T21:21:07.353 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.353 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.353 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:07.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:07.353 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 998 2024-09-17T21:21:07.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.353 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:07.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:07.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:07.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:21:07.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431224+0000", 2024-09-17T21:21:07.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:21:07.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431231+0000", 2024-09-17T21:21:07.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000002e-06 2024-09-17T21:21:07.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431236+0000", 2024-09-17T21:21:07.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.648e-06 2024-09-17T21:21:07.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516024+0000", 2024-09-17T21:21:07.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084787584999999999 2024-09-17T21:21:07.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516122+0000", 2024-09-17T21:21:07.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8076000000000001e-05 2024-09-17T21:21:07.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516330+0000", 2024-09-17T21:21:07.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020808 2024-09-17T21:21:07.357 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.357 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.357 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.357 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1000 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 591872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:07.357 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:07.357 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.827882486999997, 2024-09-17T21:21:07.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085389065, 2024-09-17T21:21:07.357 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.357 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.357 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.357 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.358 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:07.358 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:07.358 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1000 2024-09-17T21:21:07.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.358 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:07.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:07.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:07.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:21:07.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431246+0000", 2024-09-17T21:21:07.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1749999999999999e-06 2024-09-17T21:21:07.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431253+0000", 2024-09-17T21:21:07.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2790000000000004e-06 2024-09-17T21:21:07.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431258+0000", 2024-09-17T21:21:07.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3299999999999998e-06 2024-09-17T21:21:07.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516384+0000", 2024-09-17T21:21:07.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085125549999999994 2024-09-17T21:21:07.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516460+0000", 2024-09-17T21:21:07.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6359999999999994e-05 2024-09-17T21:21:07.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516633+0000", 2024-09-17T21:21:07.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017285900000000001 2024-09-17T21:21:07.361 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.361 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.361 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.362 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1001 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 595968~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:07.362 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:07.362 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.827859081, 2024-09-17T21:21:07.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085614715999999993, 2024-09-17T21:21:07.362 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.362 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.362 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.362 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.362 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:07.362 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:07.362 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1001 2024-09-17T21:21:07.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.362 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:07.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:07.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438066+0000", 2024-09-17T21:21:07.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067989490000000003 2024-09-17T21:21:07.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438073+0000", 2024-09-17T21:21:07.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8419999999999999e-06 2024-09-17T21:21:07.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438090+0000", 2024-09-17T21:21:07.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6657e-05 2024-09-17T21:21:07.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438099+0000", 2024-09-17T21:21:07.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0529999999999996e-06 2024-09-17T21:21:07.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516678+0000", 2024-09-17T21:21:07.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078579422999999995 2024-09-17T21:21:07.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516739+0000", 2024-09-17T21:21:07.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1315000000000005e-05 2024-09-17T21:21:07.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516882+0000", 2024-09-17T21:21:07.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014247700000000001 2024-09-17T21:21:07.365 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.365 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.365 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.366 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:19998 3.5 3:a4885450:::rbd_object_map.1152d139f9f9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-17T21:21:07.366 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:07.366 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.856591836, 2024-09-17T21:21:07.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90715489299999996, 2024-09-17T21:21:07.366 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.366 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.366 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:07.366 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.366 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:07.366 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:07.366 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19998 2024-09-17T21:21:07.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.367 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:07.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:07.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402537+0000", 2024-09-17T21:21:07.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5600000000000001e-06 2024-09-17T21:21:07.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402549+0000", 2024-09-17T21:21:07.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1929999999999999e-05 2024-09-17T21:21:07.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402579+0000", 2024-09-17T21:21:07.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0093000000000001e-05 2024-09-17T21:21:07.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402595+0000", 2024-09-17T21:21:07.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5855999999999999e-05 2024-09-17T21:21:07.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402731+0000", 2024-09-17T21:21:07.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013610799999999999 2024-09-17T21:21:07.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:07.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402750+0000", 2024-09-17T21:21:07.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8737e-05 2024-09-17T21:21:07.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.305977+0000", 2024-09-17T21:21:07.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90322768399999998 2024-09-17T21:21:07.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306236+0000", 2024-09-17T21:21:07.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000258111 2024-09-17T21:21:07.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:07.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306557+0000", 2024-09-17T21:21:07.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032159799999999998 2024-09-17T21:21:07.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:07.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.308286+0000", 2024-09-17T21:21:07.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017292150000000001 2024-09-17T21:21:07.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:07.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309530+0000", 2024-09-17T21:21:07.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00124409 2024-09-17T21:21:07.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:07.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309630+0000", 2024-09-17T21:21:07.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9689e-05 2024-09-17T21:21:07.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309689+0000", 2024-09-17T21:21:07.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9222000000000003e-05 2024-09-17T21:21:07.372 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.372 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.372 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.372 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.372 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.373 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:07.375 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:21:07.375 INFO:teuthology.orchestra.run.smithi184.stdout:5819 write 0xa332c00 thru 0xa339d72 (0x7173 bytes) 2024-09-17T21:21:07.375 INFO:teuthology.orchestra.run.smithi184.stdout:5821 write 0x89b2600 thru 0x89c2017 (0xfa18 bytes) 2024-09-17T21:21:07.375 INFO:teuthology.orchestra.run.smithi184.stdout:5822 read 0x17b3000 thru 0x17bf046 (0xc047 bytes) 2024-09-17T21:21:07.376 INFO:teuthology.orchestra.run.smithi184.stdout:5823 trunc from 0xa339e00 to 0x1a5a200 2024-09-17T21:21:07.376 INFO:teuthology.orchestra.run.smithi184.stdout:5825 write 0xd33000 thru 0xd4037c (0xd37d bytes) 2024-09-17T21:21:07.376 INFO:teuthology.orchestra.run.smithi184.stdout:5826 punch from 0x8b7e00 to 0x8c0000, (0x8200 bytes) 2024-09-17T21:21:07.376 INFO:teuthology.orchestra.run.smithi184.stdout:5828 read 0x80ae00 thru 0x81222e (0x742f bytes) 2024-09-17T21:21:07.376 INFO:teuthology.orchestra.run.smithi184.stdout:5830 punch from 0x138d400 to 0x138e800, (0x1400 bytes) 2024-09-17T21:21:07.376 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:07.368+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:07.405 INFO:teuthology.orchestra.run.smithi184.stdout:5832 write 0x5760400 thru 0x5762fcb (0x2bcc bytes) 2024-09-17T21:21:07.406 INFO:teuthology.orchestra.run.smithi184.stdout:5834 punch from 0x1a67e00 to 0x1a75c00, (0xde00 bytes) 2024-09-17T21:21:07.407 INFO:teuthology.orchestra.run.smithi184.stdout:5835 read 0x1e3ae00 thru 0x1e3f84e (0x4a4f bytes) 2024-09-17T21:21:07.449 INFO:teuthology.orchestra.run.smithi184.stdout:5837 write 0xc420c00 thru 0xc42da31 (0xce32 bytes) 2024-09-17T21:21:07.453 INFO:teuthology.orchestra.run.smithi184.stdout:5842 trunc from 0xc42dc00 to 0xd79fa00 2024-09-17T21:21:07.474 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:07.468+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:07.478 INFO:teuthology.orchestra.run.smithi184.stdout:5844 clone 42 order 20 su 32768 sc 12 2024-09-17T21:21:07.511 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:07.511 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:07.511 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:07.511 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:07.525 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:21:07.575 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:07.568+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:07.641 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:07.641 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:07.641 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:07.641 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:07.641 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:07.655 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:21:07.676 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:07.668+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:07.777 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:07.777 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:07.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:07.777 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:07.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.777 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:18003 3.1d 3:b9765580:::rbd_header.11527d2b15f6:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T21:21:07.777 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:07.777 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.414466687999997, 2024-09-17T21:21:07.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94165346900000002, 2024-09-17T21:21:07.777 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18003 2024-09-17T21:21:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357880+0000", 2024-09-17T21:21:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:21:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357891+0000", 2024-09-17T21:21:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0417999999999999e-05 2024-09-17T21:21:07.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357913+0000", 2024-09-17T21:21:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2215e-05 2024-09-17T21:21:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357922+0000", 2024-09-17T21:21:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4779999999999997e-06 2024-09-17T21:21:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358023+0000", 2024-09-17T21:21:07.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101025 2024-09-17T21:21:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358040+0000", 2024-09-17T21:21:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6291999999999999e-05 2024-09-17T21:21:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298203+0000", 2024-09-17T21:21:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94016297500000001 2024-09-17T21:21:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298626+0000", 2024-09-17T21:21:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000422853 2024-09-17T21:21:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.299532+0000", 2024-09-17T21:21:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00090657300000000004 2024-09-17T21:21:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15706 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 977920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:07.782 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:21:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.251364897000002, 2024-09-17T21:21:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091952328, 2024-09-17T21:21:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15706 2024-09-17T21:21:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:21:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:21:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520982+0000", 2024-09-17T21:21:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.277e-06 2024-09-17T21:21:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520988+0000", 2024-09-17T21:21:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4470000000000003e-06 2024-09-17T21:21:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521008+0000", 2024-09-17T21:21:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9309000000000001e-05 2024-09-17T21:21:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521023+0000", 2024-09-17T21:21:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5119999999999999e-05 2024-09-17T21:21:07.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612638+0000", 2024-09-17T21:21:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091615328999999995 2024-09-17T21:21:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612738+0000", 2024-09-17T21:21:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100466 2024-09-17T21:21:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612933+0000", 2024-09-17T21:21:07.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019437999999999999 2024-09-17T21:21:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15707 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 982016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.251301441000003, 2024-09-17T21:21:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092305228000000003, 2024-09-17T21:21:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.787 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15707 2024-09-17T21:21:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521045+0000", 2024-09-17T21:21:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4899999999999999e-06 2024-09-17T21:21:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521066+0000", 2024-09-17T21:21:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0236000000000001e-05 2024-09-17T21:21:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521085+0000", 2024-09-17T21:21:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8819999999999999e-05 2024-09-17T21:21:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521100+0000", 2024-09-17T21:21:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5146e-05 2024-09-17T21:21:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612978+0000", 2024-09-17T21:21:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091878847 2024-09-17T21:21:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613079+0000", 2024-09-17T21:21:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100503 2024-09-17T21:21:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613349+0000", 2024-09-17T21:21:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000270186 2024-09-17T21:21:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15708 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 986112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.251223265999997, 2024-09-17T21:21:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092547217000000001, 2024-09-17T21:21:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15708 2024-09-17T21:21:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521123+0000", 2024-09-17T21:21:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.353e-06 2024-09-17T21:21:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521129+0000", 2024-09-17T21:21:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:21:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521145+0000", 2024-09-17T21:21:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.611e-05 2024-09-17T21:21:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521154+0000", 2024-09-17T21:21:07.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4500000000000004e-06 2024-09-17T21:21:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613397+0000", 2024-09-17T21:21:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092243275 2024-09-17T21:21:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613503+0000", 2024-09-17T21:21:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106013 2024-09-17T21:21:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613669+0000", 2024-09-17T21:21:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016642299999999999 2024-09-17T21:21:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15709 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 990208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.251180249999997, 2024-09-17T21:21:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092728678999999994, 2024-09-17T21:21:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15709 2024-09-17T21:21:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521166+0000", 2024-09-17T21:21:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7000000000000004e-07 2024-09-17T21:21:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521169+0000", 2024-09-17T21:21:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2660000000000002e-06 2024-09-17T21:21:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521181+0000", 2024-09-17T21:21:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.208e-05 2024-09-17T21:21:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521194+0000", 2024-09-17T21:21:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2707e-05 2024-09-17T21:21:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613695+0000", 2024-09-17T21:21:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092500709 2024-09-17T21:21:07.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613750+0000", 2024-09-17T21:21:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.503e-05 2024-09-17T21:21:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613894+0000", 2024-09-17T21:21:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144117 2024-09-17T21:21:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15710 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 994304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.251135683999998, 2024-09-17T21:21:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092957729000000003, 2024-09-17T21:21:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15710 2024-09-17T21:21:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8000000000000005e-07 2024-09-17T21:21:07.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521216+0000", 2024-09-17T21:21:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3859999999999998e-06 2024-09-17T21:21:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521234+0000", 2024-09-17T21:21:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:21:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521248+0000", 2024-09-17T21:21:07.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:21:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613926+0000", 2024-09-17T21:21:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092678324000000006 2024-09-17T21:21:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613974+0000", 2024-09-17T21:21:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7754999999999999e-05 2024-09-17T21:21:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614167+0000", 2024-09-17T21:21:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193508 2024-09-17T21:21:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15711 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 998400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.251078302000003, 2024-09-17T21:21:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093268000000000004, 2024-09-17T21:21:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.805 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15711 2024-09-17T21:21:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:07.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521268+0000", 2024-09-17T21:21:07.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-17T21:21:07.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521273+0000", 2024-09-17T21:21:07.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:21:07.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521290+0000", 2024-09-17T21:21:07.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7269999999999999e-05 2024-09-17T21:21:07.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521304+0000", 2024-09-17T21:21:07.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3626e-05 2024-09-17T21:21:07.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614220+0000", 2024-09-17T21:21:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092916367 2024-09-17T21:21:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614322+0000", 2024-09-17T21:21:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102212 2024-09-17T21:21:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614535+0000", 2024-09-17T21:21:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021266199999999999 2024-09-17T21:21:07.809 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15712 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1002496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.251022806999998, 2024-09-17T21:21:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093568707000000001, 2024-09-17T21:21:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.810 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15712 2024-09-17T21:21:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521324+0000", 2024-09-17T21:21:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.17e-06 2024-09-17T21:21:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521334+0000", 2024-09-17T21:21:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9369999999999999e-06 2024-09-17T21:21:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521356+0000", 2024-09-17T21:21:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2045999999999999e-05 2024-09-17T21:21:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521370+0000", 2024-09-17T21:21:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4270000000000001e-05 2024-09-17T21:21:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614583+0000", 2024-09-17T21:21:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093213220999999999 2024-09-17T21:21:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614680+0000", 2024-09-17T21:21:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6900999999999999e-05 2024-09-17T21:21:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614891+0000", 2024-09-17T21:21:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021116200000000001 2024-09-17T21:21:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.814 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15713 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1006592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.250956105, 2024-09-17T21:21:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093834641999999996, 2024-09-17T21:21:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15713 2024-09-17T21:21:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521390+0000", 2024-09-17T21:21:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:21:07.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521396+0000", 2024-09-17T21:21:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:21:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521417+0000", 2024-09-17T21:21:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1236000000000001e-05 2024-09-17T21:21:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521431+0000", 2024-09-17T21:21:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3873000000000001e-05 2024-09-17T21:21:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614938+0000", 2024-09-17T21:21:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093506522999999994 2024-09-17T21:21:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615025+0000", 2024-09-17T21:21:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7635999999999999e-05 2024-09-17T21:21:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615224+0000", 2024-09-17T21:21:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019844700000000001 2024-09-17T21:21:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15714 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.250892849000003, 2024-09-17T21:21:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094071925000000001, 2024-09-17T21:21:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.819 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.820 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.820 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:07.820 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:07.820 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15714 2024-09-17T21:21:07.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.820 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:07.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:07.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521454+0000", 2024-09-17T21:21:07.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.097e-06 2024-09-17T21:21:07.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521459+0000", 2024-09-17T21:21:07.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2660000000000001e-06 2024-09-17T21:21:07.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521481+0000", 2024-09-17T21:21:07.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1878999999999998e-05 2024-09-17T21:21:07.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521498+0000", 2024-09-17T21:21:07.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6807e-05 2024-09-17T21:21:07.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615271+0000", 2024-09-17T21:21:07.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093773286999999997 2024-09-17T21:21:07.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615366+0000", 2024-09-17T21:21:07.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5322000000000007e-05 2024-09-17T21:21:07.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615524+0000", 2024-09-17T21:21:07.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015826699999999999 2024-09-17T21:21:07.823 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.823 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.823 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.824 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15715 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1014784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:07.824 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:07.824 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.250828513999998, 2024-09-17T21:21:07.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094243840999999995, 2024-09-17T21:21:07.824 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.824 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.824 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.824 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.824 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:07.824 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:07.824 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15715 2024-09-17T21:21:07.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.825 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:07.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:07.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521518+0000", 2024-09-17T21:21:07.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4300000000000001e-06 2024-09-17T21:21:07.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521524+0000", 2024-09-17T21:21:07.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5860000000000004e-06 2024-09-17T21:21:07.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521541+0000", 2024-09-17T21:21:07.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7574000000000001e-05 2024-09-17T21:21:07.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521555+0000", 2024-09-17T21:21:07.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3540000000000001e-05 2024-09-17T21:21:07.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615560+0000", 2024-09-17T21:21:07.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.09400472 2024-09-17T21:21:07.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615614+0000", 2024-09-17T21:21:07.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4693e-05 2024-09-17T21:21:07.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615761+0000", 2024-09-17T21:21:07.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000146298 2024-09-17T21:21:07.828 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.828 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.828 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.828 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15716 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1018880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:07.828 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:07.828 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.250772169000001, 2024-09-17T21:21:07.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094400676000000003, 2024-09-17T21:21:07.829 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.829 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.829 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.829 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.829 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:07.829 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:07.829 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15716 2024-09-17T21:21:07.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.829 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:07.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:07.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521574+0000", 2024-09-17T21:21:07.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1939999999999999e-06 2024-09-17T21:21:07.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521579+0000", 2024-09-17T21:21:07.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:21:07.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521597+0000", 2024-09-17T21:21:07.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8074e-05 2024-09-17T21:21:07.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521613+0000", 2024-09-17T21:21:07.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5316e-05 2024-09-17T21:21:07.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615780+0000", 2024-09-17T21:21:07.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094166874999999997 2024-09-17T21:21:07.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615836+0000", 2024-09-17T21:21:07.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6926999999999999e-05 2024-09-17T21:21:07.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615974+0000", 2024-09-17T21:21:07.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013741400000000001 2024-09-17T21:21:07.833 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.833 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.833 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.833 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15717 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1022976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:07.833 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:07.833 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.250713472999998, 2024-09-17T21:21:07.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094586723999999997, 2024-09-17T21:21:07.833 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.833 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.833 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.834 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.834 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:07.834 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:07.834 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15717 2024-09-17T21:21:07.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.834 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:07.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:07.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521633+0000", 2024-09-17T21:21:07.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:21:07.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521638+0000", 2024-09-17T21:21:07.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:21:07.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521659+0000", 2024-09-17T21:21:07.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-05 2024-09-17T21:21:07.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521675+0000", 2024-09-17T21:21:07.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5688999999999998e-05 2024-09-17T21:21:07.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616008+0000", 2024-09-17T21:21:07.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094333314000000001 2024-09-17T21:21:07.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616066+0000", 2024-09-17T21:21:07.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7797000000000002e-05 2024-09-17T21:21:07.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616219+0000", 2024-09-17T21:21:07.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015281100000000001 2024-09-17T21:21:07.837 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.838 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.838 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.838 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15718 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:07.838 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:07.838 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.250613397999999, 2024-09-17T21:21:07.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094703674000000002, 2024-09-17T21:21:07.838 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.838 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.838 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.838 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.838 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:07.839 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:07.839 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15718 2024-09-17T21:21:07.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.839 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:07.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:07.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521734+0000", 2024-09-17T21:21:07.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.61e-06 2024-09-17T21:21:07.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521740+0000", 2024-09-17T21:21:07.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:21:07.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521754+0000", 2024-09-17T21:21:07.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3793e-05 2024-09-17T21:21:07.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521763+0000", 2024-09-17T21:21:07.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5129999999999996e-06 2024-09-17T21:21:07.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616256+0000", 2024-09-17T21:21:07.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094492919999999994 2024-09-17T21:21:07.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616312+0000", 2024-09-17T21:21:07.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6737000000000002e-05 2024-09-17T21:21:07.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616436+0000", 2024-09-17T21:21:07.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012335800000000001 2024-09-17T21:21:07.842 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.842 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.842 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.843 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15719 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1031168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:07.843 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:07.843 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.250559993000003, 2024-09-17T21:21:07.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094888963000000007, 2024-09-17T21:21:07.843 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.843 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.843 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.843 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.843 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:07.843 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:07.843 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15719 2024-09-17T21:21:07.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.844 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:07.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:07.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521786+0000", 2024-09-17T21:21:07.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.074e-06 2024-09-17T21:21:07.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521791+0000", 2024-09-17T21:21:07.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3030000000000001e-06 2024-09-17T21:21:07.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521804+0000", 2024-09-17T21:21:07.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3027000000000001e-05 2024-09-17T21:21:07.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521811+0000", 2024-09-17T21:21:07.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6090000000000003e-06 2024-09-17T21:21:07.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616466+0000", 2024-09-17T21:21:07.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094654294 2024-09-17T21:21:07.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616523+0000", 2024-09-17T21:21:07.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6953999999999997e-05 2024-09-17T21:21:07.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616674+0000", 2024-09-17T21:21:07.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015170200000000001 2024-09-17T21:21:07.847 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.847 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.847 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.847 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15720 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1035264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:07.847 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:07.847 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.250523153000003, 2024-09-17T21:21:07.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095067061999999994, 2024-09-17T21:21:07.848 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.848 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.848 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.848 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.848 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:07.848 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:07.848 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15720 2024-09-17T21:21:07.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.848 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:07.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:07.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521823+0000", 2024-09-17T21:21:07.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:21:07.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521826+0000", 2024-09-17T21:21:07.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2100000000000002e-06 2024-09-17T21:21:07.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521835+0000", 2024-09-17T21:21:07.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1630000000000002e-06 2024-09-17T21:21:07.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521842+0000", 2024-09-17T21:21:07.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:21:07.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616711+0000", 2024-09-17T21:21:07.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094868859 2024-09-17T21:21:07.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616764+0000", 2024-09-17T21:21:07.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2778999999999997e-05 2024-09-17T21:21:07.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616889+0000", 2024-09-17T21:21:07.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012533100000000001 2024-09-17T21:21:07.852 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.852 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.852 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.852 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15721 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1039360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:07.852 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:07.852 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.250476044000003, 2024-09-17T21:21:07.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095277219999999996, 2024-09-17T21:21:07.853 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.853 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.853 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.853 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.853 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:07.853 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:07.853 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15721 2024-09-17T21:21:07.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.853 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:07.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:07.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521871+0000", 2024-09-17T21:21:07.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.443e-06 2024-09-17T21:21:07.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521876+0000", 2024-09-17T21:21:07.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0300000000000001e-06 2024-09-17T21:21:07.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521886+0000", 2024-09-17T21:21:07.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186e-05 2024-09-17T21:21:07.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521893+0000", 2024-09-17T21:21:07.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.08e-06 2024-09-17T21:21:07.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616918+0000", 2024-09-17T21:21:07.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095025223000000006 2024-09-17T21:21:07.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616977+0000", 2024-09-17T21:21:07.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8403999999999999e-05 2024-09-17T21:21:07.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617147+0000", 2024-09-17T21:21:07.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016985399999999999 2024-09-17T21:21:07.857 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.857 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.857 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.857 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15722 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:07.857 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:07.857 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.250441594999998, 2024-09-17T21:21:07.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095459783000000006, 2024-09-17T21:21:07.857 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.857 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.857 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.858 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.858 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:07.858 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:07.858 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15722 2024-09-17T21:21:07.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.858 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:07.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:07.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:07.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7700000000000004e-07 2024-09-17T21:21:07.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521907+0000", 2024-09-17T21:21:07.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.813e-06 2024-09-17T21:21:07.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521922+0000", 2024-09-17T21:21:07.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4533e-05 2024-09-17T21:21:07.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521934+0000", 2024-09-17T21:21:07.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2469999999999999e-05 2024-09-17T21:21:07.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617187+0000", 2024-09-17T21:21:07.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095252311000000006 2024-09-17T21:21:07.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617240+0000", 2024-09-17T21:21:07.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3740999999999999e-05 2024-09-17T21:21:07.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617364+0000", 2024-09-17T21:21:07.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-17T21:21:07.861 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.861 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.861 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.862 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15723 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:07.862 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:07.862 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.250397728999999, 2024-09-17T21:21:07.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095653563999999996, 2024-09-17T21:21:07.862 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.862 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.862 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.862 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.862 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:07.862 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:07.862 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15723 2024-09-17T21:21:07.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.862 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521949+0000", 2024-09-17T21:21:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:21:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521953+0000", 2024-09-17T21:21:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5870000000000003e-06 2024-09-17T21:21:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521970+0000", 2024-09-17T21:21:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7282000000000002e-05 2024-09-17T21:21:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521984+0000", 2024-09-17T21:21:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:21:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617383+0000", 2024-09-17T21:21:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095399063000000006 2024-09-17T21:21:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617437+0000", 2024-09-17T21:21:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3931999999999998e-05 2024-09-17T21:21:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617601+0000", 2024-09-17T21:21:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016407300000000001 2024-09-17T21:21:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5629.0:2384 3.1c 3:38c1263e:::rbd_data.1152d37183c3.000000000000000b:head [read 11780096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-17T21:21:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.347566899, 2024-09-17T21:21:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0246185320000001, 2024-09-17T21:21:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5629", 2024-09-17T21:21:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3798711329", 2024-09-17T21:21:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2384 2024-09-17T21:21:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424780+0000", 2024-09-17T21:21:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.13e-06 2024-09-17T21:21:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424785+0000", 2024-09-17T21:21:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4739999999999996e-06 2024-09-17T21:21:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424798+0000", 2024-09-17T21:21:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2704000000000001e-05 2024-09-17T21:21:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424806+0000", 2024-09-17T21:21:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9899999999999997e-06 2024-09-17T21:21:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424868+0000", 2024-09-17T21:21:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999997e-05 2024-09-17T21:21:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424875+0000", 2024-09-17T21:21:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0870000000000001e-06 2024-09-17T21:21:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428897+0000", 2024-09-17T21:21:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040226080000000004 2024-09-17T21:21:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428913+0000", 2024-09-17T21:21:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:21:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.448963+0000", 2024-09-17T21:21:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0200498659999999 2024-09-17T21:21:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449158+0000", 2024-09-17T21:21:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019581000000000001 2024-09-17T21:21:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449397+0000", 2024-09-17T21:21:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023871 2024-09-17T21:21:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:07.872 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:07.873 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:07.772+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:07.874 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:21:07.919 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:07.913+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:07.978 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:07.972+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:07.997 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:07.997 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:07.997 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:07.997 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:08.012 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:21:08.099 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:08.093+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:08.176 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:08.177 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:08.177 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:08.177 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:08.177 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:08.180 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:08.172+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:08.190 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:21:08.281 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:08.273+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:08.369 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:08.369 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:08.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:08.369 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:08.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.370 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:21:08.370 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:08.370 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 166.760854397, 2024-09-17T21:21:08.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:21:08.370 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:08.370 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:08.370 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:08.370 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:08.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:08.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:08.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:08.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:08.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:08.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:21:08.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:21:08.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:08.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:21:08.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:21:08.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:08.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:21:08.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:21:08.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:21:08.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:21:08.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:08.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:21:08.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:21:08.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:08.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:21:08.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:21:08.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:08.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:21:08.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:21:08.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:08.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:21:08.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:21:08.374 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.374 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:08.374 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.374 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:21:08.374 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:08.374 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 166.75090776100001, 2024-09-17T21:21:08.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:21:08.374 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:08.374 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:08.375 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:08.375 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:08.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:08.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:08.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:08.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:08.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:08.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:21:08.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:21:08.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:08.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:21:08.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:21:08.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:08.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:21:08.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:21:08.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:21:08.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:21:08.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:08.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:21:08.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:21:08.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:08.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:21:08.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:21:08.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:08.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:21:08.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:21:08.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:08.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:21:08.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:21:08.378 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.378 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:08.378 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.378 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:21:08.379 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:08.379 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 165.50775692100001, 2024-09-17T21:21:08.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:21:08.379 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:08.379 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:08.379 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:08.379 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:08.379 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:21:08.379 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:21:08.379 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:21:08.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.379 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:08.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:08.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:08.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:08.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:08.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:08.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:21:08.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:21:08.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:08.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:21:08.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:21:08.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:08.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:21:08.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:21:08.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:21:08.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:21:08.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:08.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:21:08.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:21:08.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:08.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:21:08.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:21:08.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:08.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:21:08.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:21:08.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:08.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:21:08.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:21:08.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:08.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:21:08.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:21:08.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:08.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:21:08.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:21:08.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:08.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:21:08.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:21:08.384 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.384 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:08.384 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.385 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:21:08.385 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:08.385 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 165.498111435, 2024-09-17T21:21:08.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:21:08.385 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:08.385 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:08.385 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:08.385 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:08.385 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:21:08.385 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:21:08.385 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:21:08.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.385 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:08.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:08.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:08.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:08.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:08.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:08.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:21:08.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:21:08.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.387 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.387 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:08.387 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:21:08.387 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:21:08.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.387 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.387 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:08.387 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:21:08.387 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:21:08.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.387 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.387 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.387 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:21:08.387 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:21:08.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.388 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.388 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:08.388 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:21:08.388 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:21:08.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.388 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.388 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:08.388 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:21:08.388 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:21:08.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.388 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.388 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:08.388 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:21:08.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:21:08.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.389 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:08.389 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:21:08.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:21:08.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.389 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:08.389 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:21:08.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:21:08.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.389 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:08.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:21:08.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:21:08.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.390 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:08.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:21:08.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:21:08.390 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.390 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:08.390 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.390 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:21:08.390 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:08.391 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 152.12917540199999, 2024-09-17T21:21:08.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:21:08.391 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:08.391 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:08.391 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:08.391 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:08.391 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:21:08.391 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:21:08.391 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:21:08.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.391 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:08.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:08.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:08.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.392 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:08.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:08.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.392 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:08.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:21:08.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:21:08.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.392 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:08.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:21:08.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:21:08.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:08.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:21:08.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:21:08.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:21:08.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:21:08.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:08.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:21:08.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:21:08.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:08.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:21:08.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:21:08.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:08.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:21:08.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:21:08.394 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.394 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:08.395 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.395 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:08.395 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:08.395 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 133.92749083999999, 2024-09-17T21:21:08.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:21:08.395 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:08.395 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:08.395 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:08.395 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:08.395 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:08.395 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:08.395 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:21:08.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.396 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:08.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:08.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:08.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:08.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:08.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:08.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:21:08.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:21:08.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:08.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:21:08.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:21:08.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:08.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:21:08.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:21:08.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:21:08.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:21:08.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:08.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:21:08.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:21:08.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:08.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:21:08.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:21:08.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:08.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:21:08.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:21:08.399 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.399 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:08.399 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.399 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:08.399 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:08.399 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 133.92699626300001, 2024-09-17T21:21:08.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:21:08.399 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:08.400 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:08.400 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:08.400 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:08.400 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:08.400 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:08.400 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:21:08.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.400 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:08.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:08.400 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:08.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:08.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:08.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:08.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:21:08.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:21:08.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:08.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:21:08.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:21:08.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:08.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:21:08.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:21:08.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:21:08.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:21:08.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:08.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:21:08.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:21:08.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:08.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:21:08.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:21:08.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:08.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:21:08.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:21:08.403 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.403 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:08.404 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.404 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:08.404 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:08.404 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 133.926940882, 2024-09-17T21:21:08.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:21:08.404 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:08.404 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:08.404 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:08.404 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:08.404 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:08.404 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:08.404 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:21:08.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.405 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:08.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:08.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:08.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:08.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:08.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:08.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:21:08.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:21:08.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:08.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:21:08.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:21:08.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:08.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:21:08.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:21:08.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:21:08.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:21:08.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:08.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:21:08.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:21:08.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:08.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:21:08.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:21:08.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.408 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:08.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:21:08.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:21:08.408 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.408 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:08.408 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.408 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:08.408 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:08.408 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 133.92688460400001, 2024-09-17T21:21:08.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:21:08.409 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:08.409 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:08.409 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:08.409 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:08.409 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:08.409 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:08.409 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:21:08.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.409 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:08.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:08.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:08.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:08.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:08.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:08.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:21:08.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:21:08.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:08.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:21:08.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:21:08.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:08.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:21:08.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:21:08.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:21:08.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:21:08.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.412 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:08.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:21:08.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:21:08.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.412 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:08.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:21:08.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:21:08.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.412 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:08.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:21:08.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:21:08.413 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.413 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:08.413 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.413 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:08.413 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:08.413 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 133.92593496500001, 2024-09-17T21:21:08.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:21:08.413 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:08.413 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:08.413 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:08.414 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:08.414 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:08.414 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:08.414 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:21:08.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.414 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:08.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:08.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:08.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:08.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:08.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:08.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:21:08.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:21:08.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:08.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:21:08.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:21:08.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:08.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:21:08.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:21:08.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:21:08.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:21:08.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:08.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:21:08.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:21:08.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:08.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:21:08.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:21:08.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:08.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:21:08.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:21:08.417 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.417 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:08.417 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.418 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:08.418 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:08.418 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 133.925856345, 2024-09-17T21:21:08.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:21:08.418 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:08.418 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:08.418 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:08.418 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:08.418 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:08.418 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:08.418 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:21:08.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.419 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:08.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:08.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:08.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:08.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:08.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:08.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:21:08.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:21:08.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:08.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:21:08.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:21:08.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:08.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:21:08.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:21:08.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:21:08.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:21:08.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:08.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:21:08.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:21:08.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:08.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:21:08.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:21:08.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:08.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:21:08.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:21:08.422 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.422 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:08.422 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.422 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:08.422 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:08.422 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 133.92574340600001, 2024-09-17T21:21:08.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:21:08.422 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:08.422 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:08.422 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:08.422 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:08.422 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:08.423 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:08.423 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:21:08.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.423 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:08.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:08.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:08.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:08.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:08.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:08.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:21:08.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:21:08.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:08.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:21:08.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:21:08.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:08.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:21:08.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:21:08.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:21:08.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:21:08.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:08.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:21:08.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:21:08.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:08.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:21:08.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:21:08.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.426 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:08.426 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:21:08.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:21:08.426 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.426 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:08.426 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.426 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:08.426 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:08.427 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 133.925082352, 2024-09-17T21:21:08.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:21:08.427 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:08.427 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:08.427 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:08.427 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:08.427 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:08.427 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:08.427 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:21:08.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.427 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:08.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:08.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:08.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:08.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:08.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:08.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:21:08.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:21:08.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:08.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:21:08.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:21:08.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:08.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:21:08.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:21:08.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:21:08.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:21:08.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:08.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:21:08.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:21:08.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:08.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:21:08.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:21:08.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:08.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:21:08.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:21:08.431 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.431 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:08.431 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.431 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:08.431 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:08.431 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 133.92501754099999, 2024-09-17T21:21:08.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:21:08.431 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:08.431 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:08.431 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:08.431 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:08.431 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:08.431 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:08.432 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:21:08.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.432 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:08.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:08.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:08.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:08.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:08.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:08.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:21:08.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:21:08.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:08.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:21:08.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:21:08.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:08.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:21:08.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:21:08.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:21:08.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:21:08.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:08.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:21:08.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:21:08.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:08.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:21:08.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:21:08.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:08.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:21:08.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:21:08.435 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.435 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:08.435 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.435 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:08.435 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:08.436 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 133.92495656400001, 2024-09-17T21:21:08.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:21:08.436 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:08.436 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:08.436 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:08.436 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:08.436 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:08.436 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:08.436 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:21:08.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.436 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:08.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:08.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:08.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:08.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:08.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:08.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:21:08.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:21:08.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:08.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:21:08.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:21:08.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:08.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:21:08.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:21:08.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:21:08.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:21:08.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:08.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:21:08.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:21:08.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:08.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:21:08.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:21:08.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:08.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:21:08.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:21:08.440 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.440 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:08.440 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.440 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:21:08.440 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:08.440 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 79.027385378999995, 2024-09-17T21:21:08.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:21:08.440 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:08.440 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:08.440 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:08.440 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:08.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:08.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:08.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:08.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:08.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:08.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:21:08.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:21:08.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:08.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:21:08.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:21:08.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:08.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:21:08.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:21:08.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:21:08.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:21:08.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:08.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:21:08.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:21:08.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:08.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:21:08.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:21:08.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:08.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:21:08.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:21:08.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:08.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:21:08.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:21:08.444 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.444 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:08.444 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.445 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:21:08.445 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:08.445 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 78.818726984999998, 2024-09-17T21:21:08.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:21:08.445 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:08.445 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:08.445 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:08.445 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:08.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:08.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:08.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:08.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:08.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:08.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:21:08.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:21:08.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:08.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:21:08.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:21:08.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:08.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:21:08.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:21:08.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:21:08.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:21:08.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:08.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:21:08.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:21:08.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:08.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:21:08.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:21:08.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:08.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:21:08.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:21:08.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:08.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:21:08.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:21:08.449 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.449 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:08.449 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.449 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:9547 3.2 e147/132 3:42b518d9:::rbd_data.1152f056b73f.0000000000000001:head v 147'513, mlcod=142'511)", 2024-09-17T21:21:08.449 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:08.449 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.8937538629999997, 2024-09-17T21:21:08.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.031059566, 2024-09-17T21:21:08.449 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:08.449 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:08.449 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:08.449 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:08.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:08.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:08.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:08.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:08.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:08.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471293+0000", 2024-09-17T21:21:08.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2029999999999999e-06 2024-09-17T21:21:08.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:08.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480606+0000", 2024-09-17T21:21:08.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093123839999999999 2024-09-17T21:21:08.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:08.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480653+0000", 2024-09-17T21:21:08.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7852000000000002e-05 2024-09-17T21:21:08.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480673+0000", 2024-09-17T21:21:08.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0013e-05 2024-09-17T21:21:08.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:08.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480862+0000", 2024-09-17T21:21:08.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018833599999999999 2024-09-17T21:21:08.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:08.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480906+0000", 2024-09-17T21:21:08.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4162000000000002e-05 2024-09-17T21:21:08.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:08.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502259+0000", 2024-09-17T21:21:08.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021353154999999999 2024-09-17T21:21:08.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:08.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502351+0000", 2024-09-17T21:21:08.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1460999999999997e-05 2024-09-17T21:21:08.453 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.453 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:08.453 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.453 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:15234 3.1 e147/134 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head v 147'917, mlcod=145'914)", 2024-09-17T21:21:08.454 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:08.454 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.2610932449999996, 2024-09-17T21:21:08.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045303695999999997, 2024-09-17T21:21:08.454 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:08.454 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:08.454 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:08.454 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:08.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:08.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:08.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:08.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:08.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:08.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103954+0000", 2024-09-17T21:21:08.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2259999999999999e-06 2024-09-17T21:21:08.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:08.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119105+0000", 2024-09-17T21:21:08.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015150871 2024-09-17T21:21:08.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:08.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119141+0000", 2024-09-17T21:21:08.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6239000000000003e-05 2024-09-17T21:21:08.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119159+0000", 2024-09-17T21:21:08.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7946999999999999e-05 2024-09-17T21:21:08.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:08.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119344+0000", 2024-09-17T21:21:08.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018524599999999999 2024-09-17T21:21:08.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:08.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119399+0000", 2024-09-17T21:21:08.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4922e-05 2024-09-17T21:21:08.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:08.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149165+0000", 2024-09-17T21:21:08.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029765697000000001 2024-09-17T21:21:08.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:08.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149255+0000", 2024-09-17T21:21:08.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0548000000000001e-05 2024-09-17T21:21:08.458 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.458 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:08.458 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.458 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:19859 3.8 e147/133 3:12f72409:::rbd_data.1152f056b73f.0000000000000003:head v 147'482, mlcod=145'479)", 2024-09-17T21:21:08.458 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:08.458 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8522021779999998, 2024-09-17T21:21:08.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033476895999999999, 2024-09-17T21:21:08.458 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:08.458 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:08.458 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:08.458 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:08.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:08.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:08.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:08.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:08.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:08.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:08.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512845+0000", 2024-09-17T21:21:08.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.886e-06 2024-09-17T21:21:08.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:08.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524104+0000", 2024-09-17T21:21:08.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011259668 2024-09-17T21:21:08.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:08.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524148+0000", 2024-09-17T21:21:08.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3386000000000001e-05 2024-09-17T21:21:08.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524172+0000", 2024-09-17T21:21:08.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4762000000000001e-05 2024-09-17T21:21:08.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:08.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524379+0000", 2024-09-17T21:21:08.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020671599999999999 2024-09-17T21:21:08.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:08.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524421+0000", 2024-09-17T21:21:08.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2182999999999999e-05 2024-09-17T21:21:08.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:08.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546216+0000", 2024-09-17T21:21:08.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021795101000000001 2024-09-17T21:21:08.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:08.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:08.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:08.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546320+0000", 2024-09-17T21:21:08.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103194 2024-09-17T21:21:08.462 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.462 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:08.462 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.462 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:08.462 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:08.462 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:08.464 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:21:08.464 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:08.456+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:08.564 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:08.557+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:08.617 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:08.617 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:08.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.617 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:23467 3.1 3.6a5ad301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:21:08.617 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:08.525105+0000", 2024-09-17T21:21:08.617 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.090708122000000002, 2024-09-17T21:21:08.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090749201000000002, 2024-09-17T21:21:08.617 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:08.617 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:08.617 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for peered", 2024-09-17T21:21:08.618 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:08.618 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:21:08.618 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:08.618 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 23467 2024-09-17T21:21:08.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.618 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:08.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:08.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:08.525105+0000", 2024-09-17T21:21:08.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:08.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:08.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:08.525105+0000", 2024-09-17T21:21:08.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:08.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:08.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:08.525108+0000", 2024-09-17T21:21:08.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:21:08.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:08.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:08.525122+0000", 2024-09-17T21:21:08.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.418e-05 2024-09-17T21:21:08.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:08.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:08.525164+0000", 2024-09-17T21:21:08.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2067999999999998e-05 2024-09-17T21:21:08.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:08.525187+0000", 2024-09-17T21:21:08.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3073000000000001e-05 2024-09-17T21:21:08.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:08.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:08.525294+0000", 2024-09-17T21:21:08.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106913 2024-09-17T21:21:08.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:08.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:08.525303+0000", 2024-09-17T21:21:08.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5669999999999995e-06 2024-09-17T21:21:08.621 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:08.621 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:08.621 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:08.621 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:08.621 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:21:08.621 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 1 2024-09-17T21:21:08.621 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:08.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:21:08.677 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:08.672+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:08.690 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 0, 5, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:21:08.690 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:21:08.691 INFO:tasks.thrashosds.thrasher:Reweighting osd 3 to 0.9648701079504248 2024-09-17T21:21:08.691 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 0.9648701079504248 2024-09-17T21:21:08.766 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:08.760+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:08.769 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:08.769 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:08.769 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:21:08.769 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:21:08.769 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:08.786 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:21:08.787 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:21:08.788 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:21:08.867 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:08.861+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:08.968 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:08.960+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:08.974 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:08.974 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:08.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:08.974 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:08.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.974 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:21:08.974 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:21:08.974 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 162.770136476, 2024-09-17T21:21:08.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:21:08.975 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:08.975 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:08.975 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:08.975 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:08.975 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:21:08.975 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:21:08.975 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:21:08.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.975 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:08.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:08.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:21:08.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:08.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:08.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:21:08.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:08.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:08.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:21:08.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:21:08.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:21:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:21:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:21:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:21:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:21:08.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:21:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:21:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:21:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:21:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:21:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:08.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:21:08.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:21:08.979 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:08.979 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:08.979 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:08.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.979 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:21:08.979 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:21:08.979 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 162.76995779399999, 2024-09-17T21:21:08.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:21:08.979 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:08.979 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:08.979 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:08.979 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:08.979 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:21:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:21:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:21:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:21:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:21:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:21:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:21:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:21:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:21:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:21:08.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:21:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:21:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:21:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:21:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:21:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:21:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:21:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:21:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:21:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:21:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:21:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 162.76989852099999, 2024-09-17T21:21:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:21:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:21:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:21:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:21:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:08.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:21:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:21:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:21:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:21:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:21:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:21:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:21:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:21:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:21:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:21:08.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:21:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:21:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:21:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:21:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:08.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:21:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:21:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:21:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:21:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 162.76975632599999, 2024-09-17T21:21:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:21:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:08.988 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:21:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:21:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:21:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:21:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:21:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:08.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:21:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:21:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:21:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:21:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:21:08.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:21:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:21:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:21:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:21:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:21:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:21:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:21:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:21:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:21:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.992 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:21:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:21:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 162.769700403, 2024-09-17T21:21:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:21:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:21:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:21:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:21:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:21:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:21:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:21:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:21:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:21:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:21:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:21:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:21:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:21:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:21:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:21:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:21:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:21:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:21:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:21:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:21:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:21:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:21:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 162.76964678600001, 2024-09-17T21:21:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:21:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:08.997 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:21:08.998 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:21:08.998 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:21:08.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.998 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:08.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:08.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:21:08.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:08.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:08.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:21:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:21:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:21:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:21:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:21:08.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:09.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:21:09.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:21:09.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:09.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:21:09.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:21:09.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:21:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:21:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:21:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:21:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:21:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:21:09.001 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:21:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 112.46041854400001, 2024-09-17T21:21:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:21:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:21:09.002 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:21:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:21:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:21:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:21:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:21:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:21:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:21:09.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:21:09.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:09.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:21:09.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:21:09.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:09.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:21:09.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:21:09.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:21:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:21:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:21:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:21:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:21:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:21:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.006 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2597 3.1f 3:fb0e9615:::rbd_data.1152a8ae2019.000000000000000c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 88.131982116000003, 2024-09-17T21:21:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097064994000000002, 2024-09-17T21:21:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2597 2024-09-17T21:21:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838628+0000", 2024-09-17T21:21:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.187e-06 2024-09-17T21:21:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838654+0000", 2024-09-17T21:21:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:21:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838696+0000", 2024-09-17T21:21:09.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1226000000000001e-05 2024-09-17T21:21:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838715+0000", 2024-09-17T21:21:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9646000000000001e-05 2024-09-17T21:21:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838937+0000", 2024-09-17T21:21:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022163 2024-09-17T21:21:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:09.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.839284+0000", 2024-09-17T21:21:09.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034753799999999999 2024-09-17T21:21:09.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.011 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:09.011 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935691+0000", 2024-09-17T21:21:09.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.096406360999999996 2024-09-17T21:21:09.011 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.011 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:09.011 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.011 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2641 3.12 3:488a729f:::rbd_data.1152a8ae2019.0000000000000022:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 88.128525343000007, 2024-09-17T21:21:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098280953000000004, 2024-09-17T21:21:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2641 2024-09-17T21:21:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:09.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842083+0000", 2024-09-17T21:21:09.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5300000000000002e-07 2024-09-17T21:21:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842088+0000", 2024-09-17T21:21:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:21:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842105+0000", 2024-09-17T21:21:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6773e-05 2024-09-17T21:21:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:09.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842119+0000", 2024-09-17T21:21:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:21:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842320+0000", 2024-09-17T21:21:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020052700000000001 2024-09-17T21:21:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842845+0000", 2024-09-17T21:21:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052508600000000002 2024-09-17T21:21:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940363+0000", 2024-09-17T21:21:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097518795000000005 2024-09-17T21:21:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2674 3.12 3:4c87e3ac:::rbd_data.1152a8ae2019.0000000000000033:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 88.128162705999998, 2024-09-17T21:21:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098025125000000005, 2024-09-17T21:21:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:09.016 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2674 2024-09-17T21:21:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.017 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842447+0000", 2024-09-17T21:21:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3999999999999999e-06 2024-09-17T21:21:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842453+0000", 2024-09-17T21:21:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:21:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842472+0000", 2024-09-17T21:21:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9006000000000001e-05 2024-09-17T21:21:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842487+0000", 2024-09-17T21:21:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4866e-05 2024-09-17T21:21:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843594+0000", 2024-09-17T21:21:09.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011073249999999999 2024-09-17T21:21:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843668+0000", 2024-09-17T21:21:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3188000000000005e-05 2024-09-17T21:21:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843893+0000", 2024-09-17T21:21:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022503399999999999 2024-09-17T21:21:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850439+0000", 2024-09-17T21:21:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065467880000000004 2024-09-17T21:21:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940409+0000", 2024-09-17T21:21:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089969511000000002 2024-09-17T21:21:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940444+0000", 2024-09-17T21:21:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5345999999999997e-05 2024-09-17T21:21:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940470+0000", 2024-09-17T21:21:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5964000000000001e-05 2024-09-17T21:21:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2707 3.1f 3:f805e65e:::rbd_data.1152a8ae2019.0000000000000049:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 88.123116045000003, 2024-09-17T21:21:09.022 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088327300999999997, 2024-09-17T21:21:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2707 2024-09-17T21:21:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847494+0000", 2024-09-17T21:21:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:21:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847504+0000", 2024-09-17T21:21:09.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:21:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847526+0000", 2024-09-17T21:21:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1976000000000001e-05 2024-09-17T21:21:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847541+0000", 2024-09-17T21:21:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5277e-05 2024-09-17T21:21:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856142+0000", 2024-09-17T21:21:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086006710000000007 2024-09-17T21:21:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856435+0000", 2024-09-17T21:21:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029301499999999999 2024-09-17T21:21:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935819+0000", 2024-09-17T21:21:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079384448999999996 2024-09-17T21:21:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2730 3.12 3:48b3422a:::rbd_data.1152a8ae2019.0000000000000057:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 88.113923146000005, 2024-09-17T21:21:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083879975999999995, 2024-09-17T21:21:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:09.027 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2730 2024-09-17T21:21:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856687+0000", 2024-09-17T21:21:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:21:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856697+0000", 2024-09-17T21:21:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0207000000000001e-05 2024-09-17T21:21:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856808+0000", 2024-09-17T21:21:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011071200000000001 2024-09-17T21:21:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856828+0000", 2024-09-17T21:21:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0219000000000001e-05 2024-09-17T21:21:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856907+0000", 2024-09-17T21:21:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9661999999999997e-05 2024-09-17T21:21:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.857019+0000", 2024-09-17T21:21:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011148499999999999 2024-09-17T21:21:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940565+0000", 2024-09-17T21:21:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083545791999999994 2024-09-17T21:21:09.032 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.032 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:09.032 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.032 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2743 3.12 3:4d036e62:::rbd_data.1152a8ae2019.000000000000005f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:09.032 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:09.032 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 88.109977645000001, 2024-09-17T21:21:09.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080059962999999998, 2024-09-17T21:21:09.032 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:09.032 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2743 2024-09-17T21:21:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.034 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:09.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:09.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.034 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:09.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860631+0000", 2024-09-17T21:21:09.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:21:09.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.034 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:09.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860636+0000", 2024-09-17T21:21:09.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5279999999999997e-06 2024-09-17T21:21:09.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:09.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860649+0000", 2024-09-17T21:21:09.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-17T21:21:09.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860655+0000", 2024-09-17T21:21:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:21:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860731+0000", 2024-09-17T21:21:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5920000000000005e-05 2024-09-17T21:21:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.861070+0000", 2024-09-17T21:21:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338535 2024-09-17T21:21:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940690+0000", 2024-09-17T21:21:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079620503999999995 2024-09-17T21:21:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3378 3.12 3:4832a51b:::rbd_data.1152a8ae2019.000000000000006c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:09.037 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 88.081613794999996, 2024-09-17T21:21:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051812516000000003, 2024-09-17T21:21:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3378 2024-09-17T21:21:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888995+0000", 2024-09-17T21:21:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:21:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889006+0000", 2024-09-17T21:21:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1012999999999999e-05 2024-09-17T21:21:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889032+0000", 2024-09-17T21:21:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6441e-05 2024-09-17T21:21:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889040+0000", 2024-09-17T21:21:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0639999999999994e-06 2024-09-17T21:21:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889278+0000", 2024-09-17T21:21:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023785899999999999 2024-09-17T21:21:09.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889408+0000", 2024-09-17T21:21:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129219 2024-09-17T21:21:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889723+0000", 2024-09-17T21:21:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000315264 2024-09-17T21:21:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.891972+0000", 2024-09-17T21:21:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002249633 2024-09-17T21:21:09.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940733+0000", 2024-09-17T21:21:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048760626000000001 2024-09-17T21:21:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940785+0000", 2024-09-17T21:21:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1906000000000001e-05 2024-09-17T21:21:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940807+0000", 2024-09-17T21:21:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1554999999999999e-05 2024-09-17T21:21:09.042 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3379 3.12 3:4e83fd71:::rbd_data.1152a8ae2019.000000000000006d:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 88.081529587999995, 2024-09-17T21:21:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051804144000000003, 2024-09-17T21:21:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:09.043 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3379 2024-09-17T21:21:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889079+0000", 2024-09-17T21:21:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1800000000000004e-07 2024-09-17T21:21:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889083+0000", 2024-09-17T21:21:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:21:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889094+0000", 2024-09-17T21:21:09.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1562999999999999e-05 2024-09-17T21:21:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889102+0000", 2024-09-17T21:21:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.401e-06 2024-09-17T21:21:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:21:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014279259999999999 2024-09-17T21:21:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:09.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890609+0000", 2024-09-17T21:21:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9566000000000004e-05 2024-09-17T21:21:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890789+0000", 2024-09-17T21:21:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017949700000000001 2024-09-17T21:21:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.893929+0000", 2024-09-17T21:21:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003140538 2024-09-17T21:21:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940834+0000", 2024-09-17T21:21:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046904811999999997 2024-09-17T21:21:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940860+0000", 2024-09-17T21:21:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5341999999999999e-05 2024-09-17T21:21:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940882+0000", 2024-09-17T21:21:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2878999999999999e-05 2024-09-17T21:21:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:09.048 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.049 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:2644 3.1e 3:79615585:::rbd_data.115266d0b9f2.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2475520~14848,write 3106304~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:21:09.049 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:09.049 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 79.458901185000002, 2024-09-17T21:21:09.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050733484000000002, 2024-09-17T21:21:09.049 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:09.049 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:09.049 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:09.049 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:09.049 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:21:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:21:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2644 2024-09-17T21:21:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511708+0000", 2024-09-17T21:21:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-17T21:21:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522196+0000", 2024-09-17T21:21:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010487712999999999 2024-09-17T21:21:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522223+0000", 2024-09-17T21:21:09.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7064999999999999e-05 2024-09-17T21:21:09.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:09.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522236+0000", 2024-09-17T21:21:09.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2931e-05 2024-09-17T21:21:09.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:09.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522348+0000", 2024-09-17T21:21:09.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111906 2024-09-17T21:21:09.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:09.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522554+0000", 2024-09-17T21:21:09.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020632699999999999 2024-09-17T21:21:09.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:21:09.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.523185+0000", 2024-09-17T21:21:09.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063081400000000005 2024-09-17T21:21:09.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:09.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.547011+0000", 2024-09-17T21:21:09.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023825570000000001 2024-09-17T21:21:09.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:09.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562268+0000", 2024-09-17T21:21:09.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015257514 2024-09-17T21:21:09.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:09.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562367+0000", 2024-09-17T21:21:09.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8678000000000006e-05 2024-09-17T21:21:09.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:09.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562440+0000", 2024-09-17T21:21:09.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3337000000000006e-05 2024-09-17T21:21:09.055 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.055 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:09.055 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.055 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:231 3.1 3:856fed78:::rbd_data.115220dd393e.000000000000000b:head [read 0~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:09.055 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:09.055 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.016145915999999, 2024-09-17T21:21:09.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34254251699999999, 2024-09-17T21:21:09.055 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:09.055 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:09.055 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:09.055 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:09.056 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:09.056 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:09.056 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 231 2024-09-17T21:21:09.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.056 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:09.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:09.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:09.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:09.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:09.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.057 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:09.057 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954464+0000", 2024-09-17T21:21:09.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:21:09.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.057 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:09.057 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954475+0000", 2024-09-17T21:21:09.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1146e-05 2024-09-17T21:21:09.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:09.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954506+0000", 2024-09-17T21:21:09.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0292999999999999e-05 2024-09-17T21:21:09.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:09.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954524+0000", 2024-09-17T21:21:09.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8430000000000001e-05 2024-09-17T21:21:09.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:09.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954661+0000", 2024-09-17T21:21:09.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136403 2024-09-17T21:21:09.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:09.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954680+0000", 2024-09-17T21:21:09.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9743000000000001e-05 2024-09-17T21:21:09.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:09.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.296896+0000", 2024-09-17T21:21:09.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34221550499999998 2024-09-17T21:21:09.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:09.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297004+0000", 2024-09-17T21:21:09.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010865399999999999 2024-09-17T21:21:09.059 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.060 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:09.060 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.060 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.060 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:56 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:09.060 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:09.060 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.860512219, 2024-09-17T21:21:09.060 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18718773899999999, 2024-09-17T21:21:09.060 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:09.060 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:09.060 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:09.060 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:09.060 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:21:09.060 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:21:09.061 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 56 2024-09-17T21:21:09.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.061 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:09.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.061 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:09.061 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:09.061 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.061 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:09.061 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:09.061 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.062 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:09.062 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110098+0000", 2024-09-17T21:21:09.062 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:21:09.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.062 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.062 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:09.062 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110111+0000", 2024-09-17T21:21:09.062 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2289999999999999e-05 2024-09-17T21:21:09.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.062 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.062 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:09.062 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110143+0000", 2024-09-17T21:21:09.062 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1979e-05 2024-09-17T21:21:09.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.062 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:09.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110161+0000", 2024-09-17T21:21:09.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7839e-05 2024-09-17T21:21:09.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:09.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110319+0000", 2024-09-17T21:21:09.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015835400000000001 2024-09-17T21:21:09.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:09.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110353+0000", 2024-09-17T21:21:09.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3849000000000001e-05 2024-09-17T21:21:09.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.064 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:09.064 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297041+0000", 2024-09-17T21:21:09.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18668842299999999 2024-09-17T21:21:09.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.064 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:09.064 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297169+0000", 2024-09-17T21:21:09.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012786199999999999 2024-09-17T21:21:09.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.064 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:09.064 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297283+0000", 2024-09-17T21:21:09.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011430300000000001 2024-09-17T21:21:09.064 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.065 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:09.065 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.065 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.065 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:68 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:09.065 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:09.065 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.851971288000001, 2024-09-17T21:21:09.065 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17885582999999999, 2024-09-17T21:21:09.065 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:09.065 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:09.065 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:09.065 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:09.065 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:21:09.065 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:21:09.066 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 68 2024-09-17T21:21:09.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.066 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:09.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.066 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:09.066 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:09.066 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.066 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:09.066 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:09.066 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118639+0000", 2024-09-17T21:21:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.306e-06 2024-09-17T21:21:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118651+0000", 2024-09-17T21:21:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2567e-05 2024-09-17T21:21:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118682+0000", 2024-09-17T21:21:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0502e-05 2024-09-17T21:21:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118707+0000", 2024-09-17T21:21:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5463e-05 2024-09-17T21:21:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118889+0000", 2024-09-17T21:21:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018150999999999999 2024-09-17T21:21:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118910+0000", 2024-09-17T21:21:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:21:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297313+0000", 2024-09-17T21:21:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17840366699999999 2024-09-17T21:21:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297389+0000", 2024-09-17T21:21:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-17T21:21:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297492+0000", 2024-09-17T21:21:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103019 2024-09-17T21:21:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5890.0:15234 3.1 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8451072~29696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-17T21:21:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.8773224759999998, 2024-09-17T21:21:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.057026052000000001, 2024-09-17T21:21:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5890", 2024-09-17T21:21:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3064344947", 2024-09-17T21:21:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 15234 2024-09-17T21:21:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093289+0000", 2024-09-17T21:21:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4939999999999999e-06 2024-09-17T21:21:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103074+0000", 2024-09-17T21:21:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00978564 2024-09-17T21:21:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103112+0000", 2024-09-17T21:21:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7246999999999998e-05 2024-09-17T21:21:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103128+0000", 2024-09-17T21:21:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5812999999999999e-05 2024-09-17T21:21:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103207+0000", 2024-09-17T21:21:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9785999999999998e-05 2024-09-17T21:21:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103315+0000", 2024-09-17T21:21:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107396 2024-09-17T21:21:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103957+0000", 2024-09-17T21:21:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000642524 2024-09-17T21:21:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.123856+0000", 2024-09-17T21:21:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019898376999999998 2024-09-17T21:21:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150190+0000", 2024-09-17T21:21:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026334682000000002 2024-09-17T21:21:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150255+0000", 2024-09-17T21:21:09.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4770999999999994e-05 2024-09-17T21:21:09.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:09.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:09.075 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:09.075 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150311+0000", 2024-09-17T21:21:09.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6322e-05 2024-09-17T21:21:09.075 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.075 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:09.075 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.075 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:09.075 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:09.075 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:09.077 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:21:09.077 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:09.078 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:09.078 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:09.078 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:09.078 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:21:09.079 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:21:09.079 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:09.079 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:09.079 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:09.072+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:09.094 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:09.095 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:09.117 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:09.117 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:09.178 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:09.173+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:09.209 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:09.210 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:09.223 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:21:09.279 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:09.272+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:09.303 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.3 to 0.96487 (f701) 2024-09-17T21:21:09.309 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-17T21:21:09.348 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:09.348 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:09.362 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:21:09.390 INFO:teuthology.orchestra.run.smithi184.stdout:flattening image image_client.0-clone41 2024-09-17T21:21:09.409 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:09.404+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:09.481 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:09.473+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:09.527 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:09.528 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:09.543 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:21:09.582 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:09.573+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:09.666 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:09.666 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:09.681 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:21:09.682 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:09.673+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:09.783 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:09.776+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:09.799 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:09.799 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:09.813 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:21:09.901 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:09.896+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:09.979 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:09.979 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:09.985 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:09.981+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:09.993 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:21:10.085 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:10.084+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:10.124 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:10.125 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:10.138 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:21:10.186 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:10.184+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:10.261 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:10.261 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:10.275 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:21:10.287 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:10.284+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:10.389 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:10.388+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:10.405 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:10.405 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:10.419 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:21:10.489 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:10.488+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:10.536 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:10.536 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:10.550 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:21:10.599 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:10.593+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:10.669 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:10.669 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:10.683 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:21:10.727 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:10.721+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:10.791 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:10.785+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:10.849 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:10.849 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:10.863 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:21:10.907 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:10.901+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:10.994 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:10.992+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:11.032 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:11.032 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:11.046 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:21:11.094 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:11.085+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:11.137 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:21:11.165 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:11.166 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:11.180 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:21:11.227 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:11.221+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:11.287 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:11.288 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:11.296 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:11.292+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:11.304 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:11.304 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:11.322 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:21:11.323 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:21:11.337 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:21:11.344 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:11.344 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:11.356 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:11.356 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:11.364 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:11.365 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:11.370 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:21:11.384 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:11.384 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:11.397 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:11.396+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:11.493 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:11.494 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:11.494 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:21:11.494 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:11.498 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:11.496+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:11.508 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:21:11.601 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:11.600+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:11.680 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:11.681 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:11.681 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:21:11.681 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:21:11.681 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:11.696 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:21:11.699 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:11.693+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:11.801 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:11.793+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:11.858 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:11.858 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:11.859 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:11.859 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:11.859 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.859 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:533 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:21:11.859 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:21:11.859 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 165.654030823, 2024-09-17T21:21:11.859 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046893477000000003, 2024-09-17T21:21:11.859 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:11.859 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:11.859 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:11.859 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:11.859 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:21:11.859 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:21:11.860 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 533 2024-09-17T21:21:11.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.860 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:11.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:11.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:21:11.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:11.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200471+0000", 2024-09-17T21:21:11.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:11.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200473+0000", 2024-09-17T21:21:11.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.217e-06 2024-09-17T21:21:11.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:11.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200477+0000", 2024-09-17T21:21:11.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T21:21:11.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:11.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200493+0000", 2024-09-17T21:21:11.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6246999999999999e-05 2024-09-17T21:21:11.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.862 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:11.862 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200506+0000", 2024-09-17T21:21:11.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.26e-05 2024-09-17T21:21:11.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.862 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.862 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.246958+0000", 2024-09-17T21:21:11.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046452357999999999 2024-09-17T21:21:11.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.862 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:11.862 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247149+0000", 2024-09-17T21:21:11.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190423 2024-09-17T21:21:11.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.863 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:11.863 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247365+0000", 2024-09-17T21:21:11.863 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000216116 2024-09-17T21:21:11.863 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.863 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:11.863 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.864 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:545 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:21:11.864 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:21:11.864 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 165.65385214099999, 2024-09-17T21:21:11.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047182541000000001, 2024-09-17T21:21:11.864 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:11.864 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:11.864 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:11.864 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:11.864 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:21:11.864 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:21:11.864 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 545 2024-09-17T21:21:11.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.865 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:11.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:11.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:21:11.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:11.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200650+0000", 2024-09-17T21:21:11.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:11.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200652+0000", 2024-09-17T21:21:11.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.48e-06 2024-09-17T21:21:11.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.866 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:11.866 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200659+0000", 2024-09-17T21:21:11.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:21:11.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.866 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:11.866 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200675+0000", 2024-09-17T21:21:11.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6183000000000001e-05 2024-09-17T21:21:11.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.867 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:11.867 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200689+0000", 2024-09-17T21:21:11.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4090000000000001e-05 2024-09-17T21:21:11.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.867 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.867 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247511+0000", 2024-09-17T21:21:11.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046821890999999997 2024-09-17T21:21:11.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.867 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:11.867 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247670+0000", 2024-09-17T21:21:11.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159271 2024-09-17T21:21:11.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:11.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247833+0000", 2024-09-17T21:21:11.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016209899999999999 2024-09-17T21:21:11.868 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.868 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:11.868 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.868 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:546 3.1 3:864e0ba1:::rbd_data.1152dfd5dee5.0000000000000001:6 [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:21:11.868 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:21:11.869 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 165.65379286800001, 2024-09-17T21:21:11.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048005894, 2024-09-17T21:21:11.869 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:11.869 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:11.869 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:11.869 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:11.869 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:21:11.869 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:21:11.869 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 546 2024-09-17T21:21:11.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.869 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:11.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:11.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:21:11.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:11.870 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200709+0000", 2024-09-17T21:21:11.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:11.870 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200711+0000", 2024-09-17T21:21:11.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:21:11.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:11.871 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200717+0000", 2024-09-17T21:21:11.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:21:11.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.871 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:11.871 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200734+0000", 2024-09-17T21:21:11.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-17T21:21:11.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.871 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:11.871 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200747+0000", 2024-09-17T21:21:11.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3523000000000001e-05 2024-09-17T21:21:11.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247897+0000", 2024-09-17T21:21:11.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047149861000000001 2024-09-17T21:21:11.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:11.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.247995+0000", 2024-09-17T21:21:11.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7947000000000003e-05 2024-09-17T21:21:11.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:11.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248715+0000", 2024-09-17T21:21:11.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072004999999999997 2024-09-17T21:21:11.873 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.873 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:11.873 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.873 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:555 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:21:11.873 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:21:11.873 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 165.65365067299999, 2024-09-17T21:21:11.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048251937000000002, 2024-09-17T21:21:11.873 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:11.873 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:11.873 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:11.874 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:11.874 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:21:11.874 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:21:11.874 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 555 2024-09-17T21:21:11.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.874 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:11.874 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.874 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:11.874 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:21:11.874 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.874 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.874 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:11.874 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200852+0000", 2024-09-17T21:21:11.875 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.875 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.875 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:11.875 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200853+0000", 2024-09-17T21:21:11.875 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.314e-06 2024-09-17T21:21:11.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.875 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.875 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:11.875 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200858+0000", 2024-09-17T21:21:11.875 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1560000000000003e-06 2024-09-17T21:21:11.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.876 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:11.876 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200875+0000", 2024-09-17T21:21:11.876 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.681e-05 2024-09-17T21:21:11.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.876 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:11.876 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200888+0000", 2024-09-17T21:21:11.876 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2877000000000001e-05 2024-09-17T21:21:11.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.876 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.876 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248816+0000", 2024-09-17T21:21:11.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047928614000000001 2024-09-17T21:21:11.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:11.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.248930+0000", 2024-09-17T21:21:11.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113701 2024-09-17T21:21:11.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:11.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249103+0000", 2024-09-17T21:21:11.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017346500000000001 2024-09-17T21:21:11.877 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.877 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:11.877 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.878 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:556 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:21:11.878 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:21:11.878 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 165.65359475, 2024-09-17T21:21:11.878 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048657830999999999, 2024-09-17T21:21:11.878 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:11.878 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:11.878 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:11.878 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:11.878 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:21:11.878 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:21:11.878 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 556 2024-09-17T21:21:11.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.879 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:11.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.879 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:11.879 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:21:11.879 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.879 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:11.879 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200907+0000", 2024-09-17T21:21:11.879 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.879 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:11.880 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200909+0000", 2024-09-17T21:21:11.880 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:21:11.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.880 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:11.880 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200913+0000", 2024-09-17T21:21:11.880 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6430000000000003e-06 2024-09-17T21:21:11.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.880 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:11.880 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200928+0000", 2024-09-17T21:21:11.880 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5279999999999999e-05 2024-09-17T21:21:11.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:11.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200942+0000", 2024-09-17T21:21:11.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3020000000000001e-05 2024-09-17T21:21:11.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249214+0000", 2024-09-17T21:21:11.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048272892999999997 2024-09-17T21:21:11.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:11.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249382+0000", 2024-09-17T21:21:11.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016788799999999999 2024-09-17T21:21:11.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.882 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:11.882 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249565+0000", 2024-09-17T21:21:11.882 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018297300000000001 2024-09-17T21:21:11.882 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.882 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:11.882 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.882 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4498.0:557 3.1 3:8717dc6f:::rbd_data.1152dfd5dee5.0000000000000008:6 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:21:11.882 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:21:11.883 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 165.653541133, 2024-09-17T21:21:11.883 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048999532999999998, 2024-09-17T21:21:11.883 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:11.883 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:11.883 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:11.883 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:11.883 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4498", 2024-09-17T21:21:11.883 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/772961625", 2024-09-17T21:21:11.883 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 557 2024-09-17T21:21:11.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.883 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:11.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.884 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:11.884 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:21:11.884 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.884 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:11.884 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200961+0000", 2024-09-17T21:21:11.884 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.884 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:11.884 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200962+0000", 2024-09-17T21:21:11.884 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:21:11.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.885 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:11.885 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200967+0000", 2024-09-17T21:21:11.885 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0429999999999998e-06 2024-09-17T21:21:11.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.885 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:11.885 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.200984+0000", 2024-09-17T21:21:11.885 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6779999999999999e-05 2024-09-17T21:21:11.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.885 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:11.885 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.201014+0000", 2024-09-17T21:21:11.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9765999999999999e-05 2024-09-17T21:21:11.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.886 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.886 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249643+0000", 2024-09-17T21:21:11.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048629009000000001 2024-09-17T21:21:11.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.886 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:11.886 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249788+0000", 2024-09-17T21:21:11.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000145685 2024-09-17T21:21:11.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.886 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:11.887 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:26.249961+0000", 2024-09-17T21:21:11.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017226700000000001 2024-09-17T21:21:11.887 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.887 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:11.887 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.887 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:21:11.887 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:11.887 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 115.344312891, 2024-09-17T21:21:11.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:21:11.887 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:11.887 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:11.887 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:11.887 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:11.888 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:21:11.888 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:21:11.888 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:21:11.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.888 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:11.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:11.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:11.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:11.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:11.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:11.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:21:11.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:21:11.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:11.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:21:11.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:21:11.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:11.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:21:11.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:21:11.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:11.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:21:11.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:21:11.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:21:11.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:21:11.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:11.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:21:11.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:21:11.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:21:11.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:21:11.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:11.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:21:11.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:21:11.891 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.891 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:11.891 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.892 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2597 3.1f 3:fb0e9615:::rbd_data.1152a8ae2019.000000000000000c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:11.892 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:11.892 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.015876462999998, 2024-09-17T21:21:11.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097064994000000002, 2024-09-17T21:21:11.892 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:11.892 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:11.892 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:11.892 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:11.892 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:11.892 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:11.892 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2597 2024-09-17T21:21:11.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.892 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:11.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.893 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:11.893 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:11.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.893 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:11.893 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:11.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.893 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:11.893 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838628+0000", 2024-09-17T21:21:11.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.187e-06 2024-09-17T21:21:11.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:11.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838654+0000", 2024-09-17T21:21:11.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:21:11.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:11.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838696+0000", 2024-09-17T21:21:11.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1226000000000001e-05 2024-09-17T21:21:11.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:11.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838715+0000", 2024-09-17T21:21:11.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9646000000000001e-05 2024-09-17T21:21:11.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838937+0000", 2024-09-17T21:21:11.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022163 2024-09-17T21:21:11.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:11.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.839284+0000", 2024-09-17T21:21:11.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034753799999999999 2024-09-17T21:21:11.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:11.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935691+0000", 2024-09-17T21:21:11.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.096406360999999996 2024-09-17T21:21:11.896 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.896 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:11.896 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.896 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2641 3.12 3:488a729f:::rbd_data.1152a8ae2019.0000000000000022:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:11.896 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:11.896 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.012419690000002, 2024-09-17T21:21:11.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098280953000000004, 2024-09-17T21:21:11.896 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:11.897 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:11.897 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:11.897 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:11.897 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:11.897 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:11.897 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2641 2024-09-17T21:21:11.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.897 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:11.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:11.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:11.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:11.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:11.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:11.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842083+0000", 2024-09-17T21:21:11.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5300000000000002e-07 2024-09-17T21:21:11.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:11.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842088+0000", 2024-09-17T21:21:11.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:21:11.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:11.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842105+0000", 2024-09-17T21:21:11.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6773e-05 2024-09-17T21:21:11.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:11.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842119+0000", 2024-09-17T21:21:11.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:21:11.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842320+0000", 2024-09-17T21:21:11.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020052700000000001 2024-09-17T21:21:11.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:11.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842845+0000", 2024-09-17T21:21:11.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052508600000000002 2024-09-17T21:21:11.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:11.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940363+0000", 2024-09-17T21:21:11.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097518795000000005 2024-09-17T21:21:11.901 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.901 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:11.901 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.902 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2674 3.12 3:4c87e3ac:::rbd_data.1152a8ae2019.0000000000000033:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:11.902 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:11.902 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.012057053000007, 2024-09-17T21:21:11.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098025125000000005, 2024-09-17T21:21:11.902 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:11.902 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:11.902 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:11.902 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:11.902 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:11.902 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:11.902 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2674 2024-09-17T21:21:11.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.903 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:11.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:11.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:11.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:11.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:11.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:11.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842447+0000", 2024-09-17T21:21:11.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3999999999999999e-06 2024-09-17T21:21:11.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:11.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842453+0000", 2024-09-17T21:21:11.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:21:11.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:11.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842472+0000", 2024-09-17T21:21:11.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9006000000000001e-05 2024-09-17T21:21:11.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:11.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842487+0000", 2024-09-17T21:21:11.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4866e-05 2024-09-17T21:21:11.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843594+0000", 2024-09-17T21:21:11.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011073249999999999 2024-09-17T21:21:11.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:11.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843668+0000", 2024-09-17T21:21:11.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3188000000000005e-05 2024-09-17T21:21:11.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:11.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843893+0000", 2024-09-17T21:21:11.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022503399999999999 2024-09-17T21:21:11.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:11.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850439+0000", 2024-09-17T21:21:11.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065467880000000004 2024-09-17T21:21:11.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:11.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940409+0000", 2024-09-17T21:21:11.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089969511000000002 2024-09-17T21:21:11.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:11.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940444+0000", 2024-09-17T21:21:11.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5345999999999997e-05 2024-09-17T21:21:11.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:11.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940470+0000", 2024-09-17T21:21:11.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5964000000000001e-05 2024-09-17T21:21:11.908 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.908 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:11.908 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.908 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2707 3.1f 3:f805e65e:::rbd_data.1152a8ae2019.0000000000000049:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:11.908 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:11.908 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.007010391999998, 2024-09-17T21:21:11.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088327300999999997, 2024-09-17T21:21:11.909 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:11.909 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:11.909 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:11.909 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:11.909 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:11.909 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:11.909 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2707 2024-09-17T21:21:11.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.909 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:11.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:11.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:11.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:11.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:11.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:11.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847494+0000", 2024-09-17T21:21:11.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:21:11.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:11.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847504+0000", 2024-09-17T21:21:11.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:21:11.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:11.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847526+0000", 2024-09-17T21:21:11.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1976000000000001e-05 2024-09-17T21:21:11.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:11.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847541+0000", 2024-09-17T21:21:11.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5277e-05 2024-09-17T21:21:11.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856142+0000", 2024-09-17T21:21:11.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086006710000000007 2024-09-17T21:21:11.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:11.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856435+0000", 2024-09-17T21:21:11.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029301499999999999 2024-09-17T21:21:11.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:11.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935819+0000", 2024-09-17T21:21:11.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079384448999999996 2024-09-17T21:21:11.913 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.913 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:11.913 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.913 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2730 3.12 3:48b3422a:::rbd_data.1152a8ae2019.0000000000000057:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:11.913 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:11.913 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 90.997817492999999, 2024-09-17T21:21:11.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083879975999999995, 2024-09-17T21:21:11.913 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:11.913 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:11.913 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:11.914 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:11.914 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:11.914 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:11.914 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2730 2024-09-17T21:21:11.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.914 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:11.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:11.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:11.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:11.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:11.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:11.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856687+0000", 2024-09-17T21:21:11.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:21:11.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:11.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856697+0000", 2024-09-17T21:21:11.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0207000000000001e-05 2024-09-17T21:21:11.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:11.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856808+0000", 2024-09-17T21:21:11.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011071200000000001 2024-09-17T21:21:11.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:11.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856828+0000", 2024-09-17T21:21:11.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0219000000000001e-05 2024-09-17T21:21:11.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856907+0000", 2024-09-17T21:21:11.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9661999999999997e-05 2024-09-17T21:21:11.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:11.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.857019+0000", 2024-09-17T21:21:11.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011148499999999999 2024-09-17T21:21:11.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:11.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940565+0000", 2024-09-17T21:21:11.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083545791999999994 2024-09-17T21:21:11.917 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.917 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:11.918 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.918 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2743 3.12 3:4d036e62:::rbd_data.1152a8ae2019.000000000000005f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:11.918 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:11.918 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 90.993871991999995, 2024-09-17T21:21:11.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080059962999999998, 2024-09-17T21:21:11.918 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:11.918 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:11.918 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:11.918 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:11.918 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:11.918 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:11.919 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2743 2024-09-17T21:21:11.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.919 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:11.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:11.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:11.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:11.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:11.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:11.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860631+0000", 2024-09-17T21:21:11.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:21:11.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:11.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860636+0000", 2024-09-17T21:21:11.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5279999999999997e-06 2024-09-17T21:21:11.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:11.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860649+0000", 2024-09-17T21:21:11.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-17T21:21:11.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:11.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860655+0000", 2024-09-17T21:21:11.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:21:11.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860731+0000", 2024-09-17T21:21:11.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5920000000000005e-05 2024-09-17T21:21:11.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:11.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.861070+0000", 2024-09-17T21:21:11.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338535 2024-09-17T21:21:11.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:11.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940690+0000", 2024-09-17T21:21:11.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079620503999999995 2024-09-17T21:21:11.922 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.922 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:11.922 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.922 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3378 3.12 3:4832a51b:::rbd_data.1152a8ae2019.000000000000006c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:11.923 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:11.923 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 90.965508142000004, 2024-09-17T21:21:11.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051812516000000003, 2024-09-17T21:21:11.923 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:11.923 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:11.923 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:11.923 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:11.923 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:11.923 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:11.923 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3378 2024-09-17T21:21:11.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.923 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:11.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:11.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:11.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:11.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:11.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:11.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888995+0000", 2024-09-17T21:21:11.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:21:11.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:11.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889006+0000", 2024-09-17T21:21:11.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1012999999999999e-05 2024-09-17T21:21:11.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:11.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889032+0000", 2024-09-17T21:21:11.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6441e-05 2024-09-17T21:21:11.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:11.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889040+0000", 2024-09-17T21:21:11.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0639999999999994e-06 2024-09-17T21:21:11.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889278+0000", 2024-09-17T21:21:11.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023785899999999999 2024-09-17T21:21:11.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:11.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889408+0000", 2024-09-17T21:21:11.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129219 2024-09-17T21:21:11.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:11.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889723+0000", 2024-09-17T21:21:11.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000315264 2024-09-17T21:21:11.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:11.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.891972+0000", 2024-09-17T21:21:11.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002249633 2024-09-17T21:21:11.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:11.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940733+0000", 2024-09-17T21:21:11.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048760626000000001 2024-09-17T21:21:11.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:11.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940785+0000", 2024-09-17T21:21:11.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1906000000000001e-05 2024-09-17T21:21:11.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:11.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940807+0000", 2024-09-17T21:21:11.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1554999999999999e-05 2024-09-17T21:21:11.928 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.928 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:11.928 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.929 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3379 3.12 3:4e83fd71:::rbd_data.1152a8ae2019.000000000000006d:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:11.929 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:11.929 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 90.965423935000004, 2024-09-17T21:21:11.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051804144000000003, 2024-09-17T21:21:11.929 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:11.929 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:11.929 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:11.929 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:11.929 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:11.929 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:11.929 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3379 2024-09-17T21:21:11.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.929 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:11.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:11.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:11.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:11.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:11.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:11.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889079+0000", 2024-09-17T21:21:11.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1800000000000004e-07 2024-09-17T21:21:11.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:11.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889083+0000", 2024-09-17T21:21:11.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:21:11.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:11.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889094+0000", 2024-09-17T21:21:11.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1562999999999999e-05 2024-09-17T21:21:11.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:11.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889102+0000", 2024-09-17T21:21:11.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.401e-06 2024-09-17T21:21:11.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:21:11.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014279259999999999 2024-09-17T21:21:11.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:11.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890609+0000", 2024-09-17T21:21:11.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9566000000000004e-05 2024-09-17T21:21:11.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:11.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890789+0000", 2024-09-17T21:21:11.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017949700000000001 2024-09-17T21:21:11.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:11.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.893929+0000", 2024-09-17T21:21:11.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003140538 2024-09-17T21:21:11.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:11.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940834+0000", 2024-09-17T21:21:11.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046904811999999997 2024-09-17T21:21:11.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:11.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940860+0000", 2024-09-17T21:21:11.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5341999999999999e-05 2024-09-17T21:21:11.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:11.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940882+0000", 2024-09-17T21:21:11.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2878999999999999e-05 2024-09-17T21:21:11.934 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.934 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:11.934 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.935 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:2644 3.1e 3:79615585:::rbd_data.115266d0b9f2.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2475520~14848,write 3106304~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:21:11.935 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:11.935 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 82.342795531999997, 2024-09-17T21:21:11.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050733484000000002, 2024-09-17T21:21:11.935 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:11.935 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:11.935 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:11.935 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:11.935 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:21:11.935 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:21:11.935 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2644 2024-09-17T21:21:11.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.936 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:11.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:11.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:11.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:11.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:11.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:11.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511708+0000", 2024-09-17T21:21:11.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-17T21:21:11.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:11.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522196+0000", 2024-09-17T21:21:11.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010487712999999999 2024-09-17T21:21:11.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:11.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522223+0000", 2024-09-17T21:21:11.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7064999999999999e-05 2024-09-17T21:21:11.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:11.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522236+0000", 2024-09-17T21:21:11.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2931e-05 2024-09-17T21:21:11.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522348+0000", 2024-09-17T21:21:11.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111906 2024-09-17T21:21:11.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:11.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522554+0000", 2024-09-17T21:21:11.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020632699999999999 2024-09-17T21:21:11.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:21:11.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.523185+0000", 2024-09-17T21:21:11.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063081400000000005 2024-09-17T21:21:11.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:11.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.547011+0000", 2024-09-17T21:21:11.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023825570000000001 2024-09-17T21:21:11.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:11.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562268+0000", 2024-09-17T21:21:11.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015257514 2024-09-17T21:21:11.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:11.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562367+0000", 2024-09-17T21:21:11.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8678000000000006e-05 2024-09-17T21:21:11.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:11.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562440+0000", 2024-09-17T21:21:11.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3337000000000006e-05 2024-09-17T21:21:11.941 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.941 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:11.941 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.941 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:231 3.1 3:856fed78:::rbd_data.115220dd393e.000000000000000b:head [read 0~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:11.942 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:11.942 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 53.900040263000001, 2024-09-17T21:21:11.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34254251699999999, 2024-09-17T21:21:11.942 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:11.942 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:11.942 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:11.942 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:11.942 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:11.942 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:11.942 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 231 2024-09-17T21:21:11.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.942 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:11.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:11.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:11.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:11.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:11.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:11.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954464+0000", 2024-09-17T21:21:11.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:21:11.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:11.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954475+0000", 2024-09-17T21:21:11.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1146e-05 2024-09-17T21:21:11.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:11.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954506+0000", 2024-09-17T21:21:11.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0292999999999999e-05 2024-09-17T21:21:11.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:11.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954524+0000", 2024-09-17T21:21:11.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8430000000000001e-05 2024-09-17T21:21:11.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954661+0000", 2024-09-17T21:21:11.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136403 2024-09-17T21:21:11.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:11.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954680+0000", 2024-09-17T21:21:11.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9743000000000001e-05 2024-09-17T21:21:11.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.296896+0000", 2024-09-17T21:21:11.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34221550499999998 2024-09-17T21:21:11.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:11.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297004+0000", 2024-09-17T21:21:11.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010865399999999999 2024-09-17T21:21:11.946 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.946 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:11.946 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.946 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:56 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:11.947 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:11.947 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 53.744406566000002, 2024-09-17T21:21:11.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18718773899999999, 2024-09-17T21:21:11.947 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:11.947 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:11.947 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:11.947 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:11.947 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:21:11.947 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:21:11.947 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 56 2024-09-17T21:21:11.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.947 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:11.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:11.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:11.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:11.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:11.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:11.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110098+0000", 2024-09-17T21:21:11.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:21:11.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:11.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110111+0000", 2024-09-17T21:21:11.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2289999999999999e-05 2024-09-17T21:21:11.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:11.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110143+0000", 2024-09-17T21:21:11.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1979e-05 2024-09-17T21:21:11.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:11.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110161+0000", 2024-09-17T21:21:11.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7839e-05 2024-09-17T21:21:11.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110319+0000", 2024-09-17T21:21:11.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015835400000000001 2024-09-17T21:21:11.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:11.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110353+0000", 2024-09-17T21:21:11.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3849000000000001e-05 2024-09-17T21:21:11.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297041+0000", 2024-09-17T21:21:11.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18668842299999999 2024-09-17T21:21:11.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:11.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297169+0000", 2024-09-17T21:21:11.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012786199999999999 2024-09-17T21:21:11.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:11.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297283+0000", 2024-09-17T21:21:11.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011430300000000001 2024-09-17T21:21:11.952 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.952 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:11.952 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.952 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:68 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:11.952 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:11.952 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 53.735865635000003, 2024-09-17T21:21:11.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17885582999999999, 2024-09-17T21:21:11.952 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:11.952 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:11.952 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:11.952 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:11.952 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:21:11.953 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:21:11.953 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 68 2024-09-17T21:21:11.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.953 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:11.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:11.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:11.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:11.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:11.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:11.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118639+0000", 2024-09-17T21:21:11.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.306e-06 2024-09-17T21:21:11.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:11.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118651+0000", 2024-09-17T21:21:11.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2567e-05 2024-09-17T21:21:11.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:11.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118682+0000", 2024-09-17T21:21:11.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0502e-05 2024-09-17T21:21:11.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:11.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118707+0000", 2024-09-17T21:21:11.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5463e-05 2024-09-17T21:21:11.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118889+0000", 2024-09-17T21:21:11.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018150999999999999 2024-09-17T21:21:11.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:11.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118910+0000", 2024-09-17T21:21:11.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:21:11.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297313+0000", 2024-09-17T21:21:11.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17840366699999999 2024-09-17T21:21:11.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:11.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297389+0000", 2024-09-17T21:21:11.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-17T21:21:11.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:11.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297492+0000", 2024-09-17T21:21:11.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103019 2024-09-17T21:21:11.957 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.957 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:11.957 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.958 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5890.0:15234 3.1 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8451072~29696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-17T21:21:11.958 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:11.958 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.7612168229999998, 2024-09-17T21:21:11.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.057026052000000001, 2024-09-17T21:21:11.958 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:11.958 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:11.958 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:11.958 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:11.958 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5890", 2024-09-17T21:21:11.958 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3064344947", 2024-09-17T21:21:11.958 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 15234 2024-09-17T21:21:11.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.958 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:11.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:11.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:11.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:11.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:11.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:11.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:11.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093289+0000", 2024-09-17T21:21:11.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4939999999999999e-06 2024-09-17T21:21:11.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:11.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103074+0000", 2024-09-17T21:21:11.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00978564 2024-09-17T21:21:11.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:11.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103112+0000", 2024-09-17T21:21:11.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7246999999999998e-05 2024-09-17T21:21:11.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:11.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103128+0000", 2024-09-17T21:21:11.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5812999999999999e-05 2024-09-17T21:21:11.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:11.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103207+0000", 2024-09-17T21:21:11.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9785999999999998e-05 2024-09-17T21:21:11.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:11.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103315+0000", 2024-09-17T21:21:11.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107396 2024-09-17T21:21:11.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:11.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103957+0000", 2024-09-17T21:21:11.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000642524 2024-09-17T21:21:11.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:11.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.123856+0000", 2024-09-17T21:21:11.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019898376999999998 2024-09-17T21:21:11.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:11.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150190+0000", 2024-09-17T21:21:11.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026334682000000002 2024-09-17T21:21:11.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:11.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150255+0000", 2024-09-17T21:21:11.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4770999999999994e-05 2024-09-17T21:21:11.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:11.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:11.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:11.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150311+0000", 2024-09-17T21:21:11.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6322e-05 2024-09-17T21:21:11.963 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.963 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:11.963 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.963 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:11.963 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:11.963 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:11.965 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:21:11.966 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:11.957+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:12.046 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-17T21:21:12.065 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:12.064+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:12.094 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:12.094 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:12.095 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:21:12.095 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:12.109 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:21:12.197 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:12.196+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:12.268 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:12.264+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:12.287 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:12.287 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:12.287 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:21:12.287 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:21:12.287 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:12.301 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-17T21:21:12.311 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:21:12.370 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:12.368+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:12.470 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:12.468+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:12.493 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:12.493 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:12.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:12.493 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:12.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.493 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:21:12.494 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:12.494 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 152.67968003199999, 2024-09-17T21:21:12.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:21:12.494 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:12.494 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:12.494 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:12.494 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:12.494 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:21:12.494 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:21:12.494 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:21:12.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.494 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:12.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.495 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:12.495 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:12.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.495 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:12.495 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:12.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.495 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:12.495 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:21:12.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:21:12.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.496 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:12.496 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:21:12.496 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:21:12.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.496 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:12.496 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:21:12.496 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:21:12.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.497 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:12.497 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:21:12.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:21:12.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.497 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.497 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:21:12.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:21:12.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.497 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:12.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:21:12.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:21:12.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:21:12.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:21:12.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:12.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:21:12.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:21:12.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:12.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:21:12.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:21:12.499 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.499 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:12.499 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.499 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:21:12.499 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:12.499 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 139.124578246, 2024-09-17T21:21:12.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:21:12.500 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:12.500 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:12.500 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:12.500 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:12.500 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:21:12.500 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:12.500 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:21:12.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.500 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:12.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:12.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:12.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:12.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:12.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:12.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:21:12.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:21:12.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:12.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:21:12.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:21:12.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:12.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:21:12.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:21:12.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:12.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:21:12.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:21:12.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:21:12.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:21:12.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:12.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:21:12.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:21:12.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.504 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:21:12.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:21:12.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:12.504 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:21:12.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:21:12.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:12.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:21:12.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:21:12.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.505 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:12.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:21:12.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:21:12.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.505 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:12.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:21:12.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:21:12.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:12.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:21:12.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:21:12.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:12.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:21:12.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:21:12.506 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.506 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:12.507 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.507 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:793 3.9 3:95f63599:::rbd_data.1152eb799094.0000000000000007:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:12.507 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:12.507 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 56.057522091000003, 2024-09-17T21:21:12.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88245766400000003, 2024-09-17T21:21:12.507 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:12.507 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:12.507 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:12.507 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:12.507 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:12.508 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:12.508 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 793 2024-09-17T21:21:12.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.508 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:12.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.508 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:12.508 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:12.508 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.508 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:12.508 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:12.508 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:12.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431941+0000", 2024-09-17T21:21:12.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.864e-06 2024-09-17T21:21:12.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:12.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431953+0000", 2024-09-17T21:21:12.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1486e-05 2024-09-17T21:21:12.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:12.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431984+0000", 2024-09-17T21:21:12.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1075999999999999e-05 2024-09-17T21:21:12.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:12.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431999+0000", 2024-09-17T21:21:12.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.503e-05 2024-09-17T21:21:12.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432095+0000", 2024-09-17T21:21:12.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5698000000000004e-05 2024-09-17T21:21:12.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:12.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432109+0000", 2024-09-17T21:21:12.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4188e-05 2024-09-17T21:21:12.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.309642+0000", 2024-09-17T21:21:12.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87753353000000001 2024-09-17T21:21:12.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:12.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310104+0000", 2024-09-17T21:21:12.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046103600000000001 2024-09-17T21:21:12.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.512 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:12.512 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310630+0000", 2024-09-17T21:21:12.512 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052682800000000002 2024-09-17T21:21:12.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.512 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:12.512 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313100+0000", 2024-09-17T21:21:12.512 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024693369999999998 2024-09-17T21:21:12.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.512 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:12.513 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313935+0000", 2024-09-17T21:21:12.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083540100000000005 2024-09-17T21:21:12.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.513 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:12.513 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314086+0000", 2024-09-17T21:21:12.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015053399999999999 2024-09-17T21:21:12.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.514 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:12.514 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314396+0000", 2024-09-17T21:21:12.514 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031065599999999997 2024-09-17T21:21:12.514 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.514 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:12.514 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.515 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:800 3.19 3:9dc17f12:::rbd_data.1152eb799094.000000000000000e:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:12.515 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:12.515 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 56.057274845999999, 2024-09-17T21:21:12.515 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88154663799999999, 2024-09-17T21:21:12.515 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:12.515 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:12.515 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:12.515 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:12.515 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:12.515 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:12.515 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 800 2024-09-17T21:21:12.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.516 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:12.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.516 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:12.516 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:12.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.516 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:12.516 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:12.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.516 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:12.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432187+0000", 2024-09-17T21:21:12.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4100000000000001e-06 2024-09-17T21:21:12.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:12.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432194+0000", 2024-09-17T21:21:12.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0389999999999999e-06 2024-09-17T21:21:12.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:12.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432216+0000", 2024-09-17T21:21:12.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1614e-05 2024-09-17T21:21:12.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.518 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:12.518 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432232+0000", 2024-09-17T21:21:12.518 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5739e-05 2024-09-17T21:21:12.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.518 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.518 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432405+0000", 2024-09-17T21:21:12.518 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017355300000000001 2024-09-17T21:21:12.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.518 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:12.518 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432432+0000", 2024-09-17T21:21:12.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6979e-05 2024-09-17T21:21:12.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.519 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.519 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.307767+0000", 2024-09-17T21:21:12.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87533472899999998 2024-09-17T21:21:12.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.519 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:12.519 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308192+0000", 2024-09-17T21:21:12.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042485999999999999 2024-09-17T21:21:12.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.519 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:12.520 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308727+0000", 2024-09-17T21:21:12.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00053550400000000001 2024-09-17T21:21:12.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.520 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:12.520 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310347+0000", 2024-09-17T21:21:12.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016198670000000001 2024-09-17T21:21:12.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.520 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:12.520 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313424+0000", 2024-09-17T21:21:12.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030764640000000001 2024-09-17T21:21:12.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:12.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313510+0000", 2024-09-17T21:21:12.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5938000000000003e-05 2024-09-17T21:21:12.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:12.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313733+0000", 2024-09-17T21:21:12.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022294200000000001 2024-09-17T21:21:12.521 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.521 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:12.521 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.522 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:805 3.19 3:9cab2ed6:::rbd_data.1152eb799094.0000000000000013:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:12.522 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:12.522 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 56.057070826999997, 2024-09-17T21:21:12.522 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88274355699999996, 2024-09-17T21:21:12.522 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:12.522 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:12.522 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:12.522 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:12.522 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:12.522 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:12.522 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 805 2024-09-17T21:21:12.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.522 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:12.522 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.523 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:12.523 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:12.523 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.523 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.523 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:12.523 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:12.523 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.523 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.523 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:12.523 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432391+0000", 2024-09-17T21:21:12.523 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3e-06 2024-09-17T21:21:12.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:12.524 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432398+0000", 2024-09-17T21:21:12.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4489999999999996e-06 2024-09-17T21:21:12.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:12.525 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432415+0000", 2024-09-17T21:21:12.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7750000000000001e-05 2024-09-17T21:21:12.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.525 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.525 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:12.525 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432430+0000", 2024-09-17T21:21:12.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.449e-05 2024-09-17T21:21:12.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.525 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.525 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432635+0000", 2024-09-17T21:21:12.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205031 2024-09-17T21:21:12.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:12.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432657+0000", 2024-09-17T21:21:12.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2064000000000001e-05 2024-09-17T21:21:12.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310444+0000", 2024-09-17T21:21:12.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87778699199999999 2024-09-17T21:21:12.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:12.527 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310849+0000", 2024-09-17T21:21:12.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00040497800000000001 2024-09-17T21:21:12.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:12.527 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.311396+0000", 2024-09-17T21:21:12.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00054729099999999999 2024-09-17T21:21:12.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.528 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:12.528 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313847+0000", 2024-09-17T21:21:12.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024504940000000001 2024-09-17T21:21:12.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.528 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:12.528 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314879+0000", 2024-09-17T21:21:12.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001031817 2024-09-17T21:21:12.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.529 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:12.529 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314964+0000", 2024-09-17T21:21:12.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5685000000000004e-05 2024-09-17T21:21:12.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.529 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.529 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:12.529 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.315133+0000", 2024-09-17T21:21:12.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000169216 2024-09-17T21:21:12.529 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.529 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:12.529 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.529 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.530 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12087 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 52224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:12.530 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:21:12.530 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.719982149000003, 2024-09-17T21:21:12.530 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087684460000000006, 2024-09-17T21:21:12.530 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:12.530 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:12.530 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:12.530 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:12.530 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:12.530 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:12.530 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12087 2024-09-17T21:21:12.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.530 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:12.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:12.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:21:12.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:12.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:21:12.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:12.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769480+0000", 2024-09-17T21:21:12.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:21:12.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:12.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769485+0000", 2024-09-17T21:21:12.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8880000000000002e-06 2024-09-17T21:21:12.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:12.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769509+0000", 2024-09-17T21:21:12.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3343e-05 2024-09-17T21:21:12.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:12.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769521+0000", 2024-09-17T21:21:12.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2497000000000001e-05 2024-09-17T21:21:12.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.856883+0000", 2024-09-17T21:21:12.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087362191000000006 2024-09-17T21:21:12.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:12.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857014+0000", 2024-09-17T21:21:12.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013090399999999999 2024-09-17T21:21:12.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:12.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857163+0000", 2024-09-17T21:21:12.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014864000000000001 2024-09-17T21:21:12.534 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.534 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:12.534 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.534 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12088 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 56320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:12.534 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:21:12.534 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.719898493999999, 2024-09-17T21:21:12.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087917772000000005, 2024-09-17T21:21:12.535 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:12.535 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:12.535 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:12.535 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:12.535 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:12.535 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:12.535 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12088 2024-09-17T21:21:12.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.535 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:12.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.535 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:12.535 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:21:12.535 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.536 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:12.536 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:21:12.536 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.536 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:12.536 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769569+0000", 2024-09-17T21:21:12.536 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:21:12.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.536 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:12.536 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769574+0000", 2024-09-17T21:21:12.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4659999999999998e-06 2024-09-17T21:21:12.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:12.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769586+0000", 2024-09-17T21:21:12.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2311e-05 2024-09-17T21:21:12.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:12.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769597+0000", 2024-09-17T21:21:12.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0548e-05 2024-09-17T21:21:12.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857217+0000", 2024-09-17T21:21:12.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087619896000000003 2024-09-17T21:21:12.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:12.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857335+0000", 2024-09-17T21:21:12.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118538 2024-09-17T21:21:12.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:12.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857480+0000", 2024-09-17T21:21:12.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014470999999999999 2024-09-17T21:21:12.539 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.539 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:12.539 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.539 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12089 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 60416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:12.539 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:12.539 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.719845794999998, 2024-09-17T21:21:12.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088293233999999998, 2024-09-17T21:21:12.539 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:12.539 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:12.539 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:12.539 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:12.540 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:12.540 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:12.540 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12089 2024-09-17T21:21:12.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.540 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:12.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:12.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:12.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:12.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:12.541 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.541 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:12.541 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769616+0000", 2024-09-17T21:21:12.541 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:21:12.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.541 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:12.542 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769620+0000", 2024-09-17T21:21:12.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5140000000000002e-06 2024-09-17T21:21:12.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.542 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:12.542 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769633+0000", 2024-09-17T21:21:12.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2849e-05 2024-09-17T21:21:12.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.542 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:12.542 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769644+0000", 2024-09-17T21:21:12.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1471000000000001e-05 2024-09-17T21:21:12.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.543 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857564+0000", 2024-09-17T21:21:12.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087919445999999998 2024-09-17T21:21:12.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:12.543 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857716+0000", 2024-09-17T21:21:12.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000152065 2024-09-17T21:21:12.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:12.543 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857908+0000", 2024-09-17T21:21:12.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019252599999999999 2024-09-17T21:21:12.544 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.544 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:12.544 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.544 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12090 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 64512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:12.544 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:12.544 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.71979941, 2024-09-17T21:21:12.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088582672000000001, 2024-09-17T21:21:12.544 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:12.544 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:12.545 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:12.545 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:12.545 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:12.545 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:12.545 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12090 2024-09-17T21:21:12.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.545 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:12.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:12.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:12.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:12.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:12.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:12.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769662+0000", 2024-09-17T21:21:12.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:21:12.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:12.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769683+0000", 2024-09-17T21:21:12.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0979e-05 2024-09-17T21:21:12.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:12.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769699+0000", 2024-09-17T21:21:12.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5446000000000001e-05 2024-09-17T21:21:12.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:12.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769708+0000", 2024-09-17T21:21:12.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1679999999999993e-06 2024-09-17T21:21:12.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857976+0000", 2024-09-17T21:21:12.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088267795999999996 2024-09-17T21:21:12.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:12.548 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858095+0000", 2024-09-17T21:21:12.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119147 2024-09-17T21:21:12.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:12.548 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858244+0000", 2024-09-17T21:21:12.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014917000000000001 2024-09-17T21:21:12.548 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.548 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:12.549 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.549 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12091 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 68608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:12.549 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:12.549 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.719728025999999, 2024-09-17T21:21:12.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088832752000000001, 2024-09-17T21:21:12.549 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:12.549 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:12.549 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:12.549 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:12.549 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:12.549 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:12.549 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12091 2024-09-17T21:21:12.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.550 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:12.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:12.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:12.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:12.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:12.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:12.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769734+0000", 2024-09-17T21:21:12.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:21:12.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:12.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769739+0000", 2024-09-17T21:21:12.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8080000000000004e-06 2024-09-17T21:21:12.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:12.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769764+0000", 2024-09-17T21:21:12.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4839999999999999e-05 2024-09-17T21:21:12.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:12.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769777+0000", 2024-09-17T21:21:12.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3037000000000001e-05 2024-09-17T21:21:12.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858295+0000", 2024-09-17T21:21:12.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088518475999999999 2024-09-17T21:21:12.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:12.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858419+0000", 2024-09-17T21:21:12.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012378700000000001 2024-09-17T21:21:12.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:12.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858565+0000", 2024-09-17T21:21:12.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014630099999999999 2024-09-17T21:21:12.553 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.553 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:12.553 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.553 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12092 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:12.553 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:12.554 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.719645702000001, 2024-09-17T21:21:12.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089191555000000006, 2024-09-17T21:21:12.554 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:12.554 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:12.554 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:12.554 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:12.554 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:12.554 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:12.554 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12092 2024-09-17T21:21:12.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.554 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:12.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:12.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:12.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:12.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:12.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:12.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769816+0000", 2024-09-17T21:21:12.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7199999999999997e-07 2024-09-17T21:21:12.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:12.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769822+0000", 2024-09-17T21:21:12.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:21:12.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:12.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769838+0000", 2024-09-17T21:21:12.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6674e-05 2024-09-17T21:21:12.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:12.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769851+0000", 2024-09-17T21:21:12.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2527999999999999e-05 2024-09-17T21:21:12.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858660+0000", 2024-09-17T21:21:12.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088808556999999996 2024-09-17T21:21:12.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:12.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858817+0000", 2024-09-17T21:21:12.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015730400000000001 2024-09-17T21:21:12.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:12.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859007+0000", 2024-09-17T21:21:12.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018974899999999999 2024-09-17T21:21:12.558 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.558 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:12.558 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.558 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12093 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:12.558 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:12.558 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.719537598000002, 2024-09-17T21:21:12.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089429710999999995, 2024-09-17T21:21:12.558 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:12.558 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:12.559 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:12.559 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:12.559 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:12.559 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:12.559 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12093 2024-09-17T21:21:12.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.559 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:12.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:12.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:12.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.560 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:12.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:12.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.560 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:12.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769924+0000", 2024-09-17T21:21:12.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.342e-06 2024-09-17T21:21:12.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.560 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:12.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769929+0000", 2024-09-17T21:21:12.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0350000000000001e-06 2024-09-17T21:21:12.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:12.561 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769946+0000", 2024-09-17T21:21:12.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6793999999999999e-05 2024-09-17T21:21:12.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:12.561 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769959+0000", 2024-09-17T21:21:12.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3062e-05 2024-09-17T21:21:12.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859075+0000", 2024-09-17T21:21:12.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089115696999999994 2024-09-17T21:21:12.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:12.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859202+0000", 2024-09-17T21:21:12.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012657 2024-09-17T21:21:12.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:12.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859353+0000", 2024-09-17T21:21:12.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151211 2024-09-17T21:21:12.562 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.562 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:12.563 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.563 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12094 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:12.563 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:12.563 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.719483422000003, 2024-09-17T21:21:12.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089695001999999996, 2024-09-17T21:21:12.563 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:12.563 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:12.563 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:12.563 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:12.563 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:12.563 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:12.563 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12094 2024-09-17T21:21:12.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.564 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:12.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:12.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:12.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:12.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:12.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:12.565 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769978+0000", 2024-09-17T21:21:12.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.142e-06 2024-09-17T21:21:12.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.565 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:12.565 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769983+0000", 2024-09-17T21:21:12.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.386e-06 2024-09-17T21:21:12.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.565 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:12.565 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769997+0000", 2024-09-17T21:21:12.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.416e-05 2024-09-17T21:21:12.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:12.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770012+0000", 2024-09-17T21:21:12.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5384999999999999e-05 2024-09-17T21:21:12.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859405+0000", 2024-09-17T21:21:12.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089392376999999995 2024-09-17T21:21:12.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.567 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:12.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859525+0000", 2024-09-17T21:21:12.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120555 2024-09-17T21:21:12.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.567 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:12.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859672+0000", 2024-09-17T21:21:12.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000146997 2024-09-17T21:21:12.567 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.567 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:12.567 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.568 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12095 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:12.568 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:12.568 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.719430934999998, 2024-09-17T21:21:12.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090073476, 2024-09-17T21:21:12.568 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:12.568 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:12.568 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:12.568 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:12.568 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:12.568 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:12.568 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12095 2024-09-17T21:21:12.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.568 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:12.569 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.569 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:12.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:12.569 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.569 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.569 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:12.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:12.569 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.569 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.569 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:12.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770031+0000", 2024-09-17T21:21:12.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1699999999999997e-07 2024-09-17T21:21:12.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.570 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:12.570 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770036+0000", 2024-09-17T21:21:12.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.023e-06 2024-09-17T21:21:12.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.570 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:12.570 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770051+0000", 2024-09-17T21:21:12.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4953999999999999e-05 2024-09-17T21:21:12.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.570 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:12.571 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770063+0000", 2024-09-17T21:21:12.571 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:21:12.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.571 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.571 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.571 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859756+0000", 2024-09-17T21:21:12.571 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089692980000000005 2024-09-17T21:21:12.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.571 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.571 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:12.571 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859891+0000", 2024-09-17T21:21:12.571 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013459499999999999 2024-09-17T21:21:12.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.572 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:12.572 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860103+0000", 2024-09-17T21:21:12.572 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021274400000000001 2024-09-17T21:21:12.572 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.572 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:12.572 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.572 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12096 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:12.572 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:12.572 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.719381249999998, 2024-09-17T21:21:12.572 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090358274000000002, 2024-09-17T21:21:12.573 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:12.573 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:12.573 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:12.573 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:12.573 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:12.573 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:12.573 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12096 2024-09-17T21:21:12.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.573 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:12.573 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.573 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:12.573 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:12.573 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.574 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:12.574 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:12.574 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.574 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:12.574 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770080+0000", 2024-09-17T21:21:12.574 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9400000000000004e-07 2024-09-17T21:21:12.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.574 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:12.574 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770085+0000", 2024-09-17T21:21:12.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2169999999999996e-06 2024-09-17T21:21:12.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.575 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.575 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:12.575 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770100+0000", 2024-09-17T21:21:12.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5077e-05 2024-09-17T21:21:12.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.575 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.575 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:12.575 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770112+0000", 2024-09-17T21:21:12.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1951e-05 2024-09-17T21:21:12.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.575 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.575 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.576 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860160+0000", 2024-09-17T21:21:12.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090048413999999993 2024-09-17T21:21:12.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.576 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.576 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:12.576 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860285+0000", 2024-09-17T21:21:12.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012466699999999999 2024-09-17T21:21:12.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.576 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.576 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:12.576 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860438+0000", 2024-09-17T21:21:12.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015305399999999999 2024-09-17T21:21:12.576 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.577 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:12.577 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.577 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12097 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:12.577 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:12.577 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.719333503000001, 2024-09-17T21:21:12.577 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090702375000000002, 2024-09-17T21:21:12.577 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:12.577 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:12.577 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:12.577 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:12.577 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:12.578 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:12.578 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12097 2024-09-17T21:21:12.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.578 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:12.578 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.578 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:12.578 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:12.578 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.578 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.578 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:12.578 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:12.579 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.579 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:12.579 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770128+0000", 2024-09-17T21:21:12.579 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.04e-06 2024-09-17T21:21:12.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.579 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:12.579 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770133+0000", 2024-09-17T21:21:12.579 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8999999999999997e-06 2024-09-17T21:21:12.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.579 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:12.580 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770152+0000", 2024-09-17T21:21:12.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8890999999999999e-05 2024-09-17T21:21:12.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.580 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.580 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:12.580 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770164+0000", 2024-09-17T21:21:12.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1769e-05 2024-09-17T21:21:12.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.580 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.580 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.580 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860511+0000", 2024-09-17T21:21:12.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090347527999999996 2024-09-17T21:21:12.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.581 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:12.581 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860652+0000", 2024-09-17T21:21:12.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140894 2024-09-17T21:21:12.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.581 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:12.581 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860830+0000", 2024-09-17T21:21:12.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017735299999999999 2024-09-17T21:21:12.581 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.581 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:12.581 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.582 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.582 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12098 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:12.582 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:12.582 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.719280034999997, 2024-09-17T21:21:12.582 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090997262999999995, 2024-09-17T21:21:12.582 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:12.582 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:12.582 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:12.582 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:12.582 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:12.582 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:12.582 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12098 2024-09-17T21:21:12.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.582 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:12.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.583 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:12.583 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:12.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.583 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:12.583 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:12.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.583 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:12.583 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775465+0000", 2024-09-17T21:21:12.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0052843350000000002 2024-09-17T21:21:12.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.584 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.584 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:12.584 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775476+0000", 2024-09-17T21:21:12.584 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0797e-05 2024-09-17T21:21:12.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.584 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.584 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:12.584 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775506+0000", 2024-09-17T21:21:12.584 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9709999999999998e-05 2024-09-17T21:21:12.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.584 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.584 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:12.585 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775530+0000", 2024-09-17T21:21:12.585 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4528999999999998e-05 2024-09-17T21:21:12.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.585 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.585 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.585 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860894+0000", 2024-09-17T21:21:12.585 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.085364195000000004 2024-09-17T21:21:12.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.585 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.585 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:12.585 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861019+0000", 2024-09-17T21:21:12.585 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012467399999999999 2024-09-17T21:21:12.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.586 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.586 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:12.586 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861178+0000", 2024-09-17T21:21:12.586 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159023 2024-09-17T21:21:12.586 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.586 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:12.586 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.586 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.586 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12104 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:12.586 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:12.586 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.713578925999997, 2024-09-17T21:21:12.586 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087969628999999994, 2024-09-17T21:21:12.587 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:12.587 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:12.587 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:12.587 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:12.587 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:12.587 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:12.587 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12104 2024-09-17T21:21:12.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.587 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:12.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.587 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:12.587 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:12.587 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.588 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:12.588 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:12.588 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.588 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:12.588 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775883+0000", 2024-09-17T21:21:12.588 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3399999999999998e-07 2024-09-17T21:21:12.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.588 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:12.588 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775886+0000", 2024-09-17T21:21:12.589 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2119999999999999e-06 2024-09-17T21:21:12.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.589 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.589 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:12.589 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775895+0000", 2024-09-17T21:21:12.589 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8710000000000003e-06 2024-09-17T21:21:12.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.589 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.589 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:12.589 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775901+0000", 2024-09-17T21:21:12.589 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:21:12.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.589 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.590 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863544+0000", 2024-09-17T21:21:12.590 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087642642000000007 2024-09-17T21:21:12.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:12.590 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863694+0000", 2024-09-17T21:21:12.590 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014952300000000001 2024-09-17T21:21:12.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:12.590 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863851+0000", 2024-09-17T21:21:12.590 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000157804 2024-09-17T21:21:12.591 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.591 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:12.591 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.591 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12105 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:12.591 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:12.591 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.713550032000001, 2024-09-17T21:21:12.591 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088330400000000003, 2024-09-17T21:21:12.591 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:12.591 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:12.591 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:12.591 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:12.591 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:12.592 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:12.592 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12105 2024-09-17T21:21:12.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.592 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:12.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.592 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:12.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:12.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.592 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:12.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:12.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.593 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:12.593 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:12.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9999999999999997e-07 2024-09-17T21:21:12.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.593 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:12.593 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775914+0000", 2024-09-17T21:21:12.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4710000000000001e-06 2024-09-17T21:21:12.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.594 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:12.594 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775921+0000", 2024-09-17T21:21:12.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:21:12.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.594 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:12.594 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775928+0000", 2024-09-17T21:21:12.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:21:12.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.594 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.594 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863923+0000", 2024-09-17T21:21:12.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087995126000000007 2024-09-17T21:21:12.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.595 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:12.595 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864059+0000", 2024-09-17T21:21:12.595 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135797 2024-09-17T21:21:12.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.595 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:12.595 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864241+0000", 2024-09-17T21:21:12.595 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018252599999999999 2024-09-17T21:21:12.595 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.595 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:12.595 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.596 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12106 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:12.596 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:12.596 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.713460853999997, 2024-09-17T21:21:12.596 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088586394999999998, 2024-09-17T21:21:12.596 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:12.596 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:12.596 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:12.596 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:12.596 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:12.596 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:12.596 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12106 2024-09-17T21:21:12.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.597 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:12.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:12.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:12.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:12.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:12.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:12.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:12.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776001+0000", 2024-09-17T21:21:12.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4300000000000001e-07 2024-09-17T21:21:12.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:12.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776004+0000", 2024-09-17T21:21:12.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8049999999999998e-06 2024-09-17T21:21:12.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:12.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776018+0000", 2024-09-17T21:21:12.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4613999999999999e-05 2024-09-17T21:21:12.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:12.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776025+0000", 2024-09-17T21:21:12.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:21:12.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:12.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864279+0000", 2024-09-17T21:21:12.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088253570000000003 2024-09-17T21:21:12.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:12.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864433+0000", 2024-09-17T21:21:12.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015469400000000001 2024-09-17T21:21:12.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:12.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:12.600 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:12.600 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864586+0000", 2024-09-17T21:21:12.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000153003 2024-09-17T21:21:12.600 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.600 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:12.600 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.600 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:12.600 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:12.600 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:12.603 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:21:12.604 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:12.593+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:12.703 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:12.697+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:12.778 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:12.778 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:12.778 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:21:12.779 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:12.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:21:12.837 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:12.836+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:12.904 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:12.897+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:12.959 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:12.959 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:12.959 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:21:12.959 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:21:12.959 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:12.979 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:21:13.021 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:13.020+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:13.106 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:13.104+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:13.141 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:13.141 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:13.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:13.142 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:13.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.142 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:21:13.142 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:21:13.142 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 174.921561336, 2024-09-17T21:21:13.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:21:13.142 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:13.142 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:13.142 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:13.142 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:13.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:13.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:21:13.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.143 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:13.143 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:21:13.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.143 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:13.143 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:21:13.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:21:13.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.143 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:13.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:21:13.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:21:13.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:13.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:21:13.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:21:13.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:13.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:21:13.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:21:13.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:13.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:21:13.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:21:13.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:13.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:21:13.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:21:13.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:13.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:21:13.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:21:13.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.146 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:13.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:21:13.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:21:13.146 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.146 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:13.146 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.146 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:21:13.146 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:13.146 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 170.27126067200001, 2024-09-17T21:21:13.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:21:13.147 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:13.147 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:13.147 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:13.147 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:13.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.147 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:13.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:13.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.147 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:13.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:13.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:13.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:21:13.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:21:13.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:13.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:21:13.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:21:13.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:13.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:21:13.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:21:13.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:13.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:21:13.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:21:13.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:13.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:21:13.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:21:13.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:13.150 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:21:13.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:21:13.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.150 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:13.150 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:21:13.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:21:13.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.150 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:13.150 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:21:13.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:21:13.150 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:21:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 170.26999558899999, 2024-09-17T21:21:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:21:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:21:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:21:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:13.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:21:13.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:21:13.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:13.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:21:13.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:21:13.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:13.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:21:13.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:21:13.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:21:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:21:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:21:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:21:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:21:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:21:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.155 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:13.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:21:13.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:21:13.155 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.155 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:13.155 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.155 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:21:13.155 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:13.155 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 170.260216179, 2024-09-17T21:21:13.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:21:13.155 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:13.156 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:13.156 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:13.156 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:13.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.156 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:13.156 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:13.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.156 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:13.156 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:13.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.157 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:13.157 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:21:13.157 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:21:13.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.157 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.157 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:13.157 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:21:13.157 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:21:13.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.157 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.157 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:13.157 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:21:13.157 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:21:13.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:13.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:21:13.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:21:13.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:13.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:21:13.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:21:13.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:13.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:21:13.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:21:13.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:13.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:21:13.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:21:13.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:13.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:21:13.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:21:13.159 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.159 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:13.160 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.160 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:21:13.160 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:13.160 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 170.25149377599999, 2024-09-17T21:21:13.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:21:13.160 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:13.160 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:13.160 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:13.160 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:13.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:13.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:13.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.161 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:13.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:13.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.161 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:13.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:21:13.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:21:13.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:13.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:21:13.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:21:13.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:13.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:21:13.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:21:13.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:13.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:21:13.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:21:13.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:21:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:21:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:21:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:21:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:21:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:21:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:21:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:21:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:21:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 156.86640321199999, 2024-09-17T21:21:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:21:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:13.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:21:13.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:21:13.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:13.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:21:13.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:21:13.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:13.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:21:13.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:21:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:21:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:21:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:21:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:21:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:21:13.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:21:13.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:13.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:21:13.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:21:13.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:13.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:21:13.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:21:13.168 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.168 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:13.169 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.169 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1780 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'195, mlcod=69'190)", 2024-09-17T21:21:13.169 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:13.169 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 119.888547608, 2024-09-17T21:21:13.169 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075761769999999999, 2024-09-17T21:21:13.169 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:13.169 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:13.169 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:13.169 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:13.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.169 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:13.170 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:13.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.170 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:13.170 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:13.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.170 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:13.170 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250375+0000", 2024-09-17T21:21:13.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:21:13.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250383+0000", 2024-09-17T21:21:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7870000000000002e-06 2024-09-17T21:21:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250402+0000", 2024-09-17T21:21:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9346000000000001e-05 2024-09-17T21:21:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250413+0000", 2024-09-17T21:21:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.039e-05 2024-09-17T21:21:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:13.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251353+0000", 2024-09-17T21:21:13.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000940091 2024-09-17T21:21:13.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:13.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251407+0000", 2024-09-17T21:21:13.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4701999999999999e-05 2024-09-17T21:21:13.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:13.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257829+0000", 2024-09-17T21:21:13.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064212779999999999 2024-09-17T21:21:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257950+0000", 2024-09-17T21:21:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121623 2024-09-17T21:21:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1781 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'196, mlcod=69'190)", 2024-09-17T21:21:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:21:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 119.887464657, 2024-09-17T21:21:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065632939999999999, 2024-09-17T21:21:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:21:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:21:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:13.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251459+0000", 2024-09-17T21:21:13.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.624e-06 2024-09-17T21:21:13.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:13.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251471+0000", 2024-09-17T21:21:13.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2636000000000001e-05 2024-09-17T21:21:13.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:13.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251498+0000", 2024-09-17T21:21:13.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:21:13.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.176 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:13.176 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251515+0000", 2024-09-17T21:21:13.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6939000000000002e-05 2024-09-17T21:21:13.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.176 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:13.176 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252938+0000", 2024-09-17T21:21:13.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001423082 2024-09-17T21:21:13.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.176 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:13.176 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252999+0000", 2024-09-17T21:21:13.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1138999999999999e-05 2024-09-17T21:21:13.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.177 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:13.177 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257955+0000", 2024-09-17T21:21:13.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049559640000000002 2024-09-17T21:21:13.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.177 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:13.177 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.258020+0000", 2024-09-17T21:21:13.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5504000000000002e-05 2024-09-17T21:21:13.177 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.177 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:13.177 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1788 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'198, mlcod=69'191)", 2024-09-17T21:21:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 119.885439168, 2024-09-17T21:21:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073690370000000002, 2024-09-17T21:21:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:13.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:13.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:13.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253484+0000", 2024-09-17T21:21:13.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2470000000000001e-06 2024-09-17T21:21:13.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:13.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253492+0000", 2024-09-17T21:21:13.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7159999999999992e-06 2024-09-17T21:21:13.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:13.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253509+0000", 2024-09-17T21:21:13.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.666e-05 2024-09-17T21:21:13.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:13.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253522+0000", 2024-09-17T21:21:13.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2643000000000001e-05 2024-09-17T21:21:13.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:13.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254853+0000", 2024-09-17T21:21:13.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331238 2024-09-17T21:21:13.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:13.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254885+0000", 2024-09-17T21:21:13.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1562e-05 2024-09-17T21:21:13.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:13.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260788+0000", 2024-09-17T21:21:13.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059030749999999998 2024-09-17T21:21:13.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.182 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:13.182 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260852+0000", 2024-09-17T21:21:13.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3895999999999999e-05 2024-09-17T21:21:13.182 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.182 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:13.182 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.182 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5746 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'477, mlcod=71'473)", 2024-09-17T21:21:13.182 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:13.182 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 116.88001161, 2024-09-17T21:21:13.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074182709999999997, 2024-09-17T21:21:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:13.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258912+0000", 2024-09-17T21:21:13.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:21:13.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:13.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258926+0000", 2024-09-17T21:21:13.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4249e-05 2024-09-17T21:21:13.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:13.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258946+0000", 2024-09-17T21:21:13.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9879999999999999e-05 2024-09-17T21:21:13.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:13.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258956+0000", 2024-09-17T21:21:13.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:21:13.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:13.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259380+0000", 2024-09-17T21:21:13.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042337599999999999 2024-09-17T21:21:13.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:13.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259412+0000", 2024-09-17T21:21:13.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:21:13.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:13.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266191+0000", 2024-09-17T21:21:13.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067793269999999999 2024-09-17T21:21:13.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:13.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266328+0000", 2024-09-17T21:21:13.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013690999999999999 2024-09-17T21:21:13.186 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.186 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:13.186 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5747 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'478, mlcod=71'473)", 2024-09-17T21:21:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 116.879305261, 2024-09-17T21:21:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068779440000000004, 2024-09-17T21:21:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259618+0000", 2024-09-17T21:21:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6199999999999999e-06 2024-09-17T21:21:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259630+0000", 2024-09-17T21:21:13.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1884e-05 2024-09-17T21:21:13.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:13.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259657+0000", 2024-09-17T21:21:13.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7339e-05 2024-09-17T21:21:13.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:13.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259677+0000", 2024-09-17T21:21:13.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0156000000000001e-05 2024-09-17T21:21:13.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.190 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:13.190 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260521+0000", 2024-09-17T21:21:13.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084312899999999997 2024-09-17T21:21:13.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.190 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:13.190 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260585+0000", 2024-09-17T21:21:13.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4558000000000001e-05 2024-09-17T21:21:13.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:13.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266336+0000", 2024-09-17T21:21:13.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057505020000000002 2024-09-17T21:21:13.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:13.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266494+0000", 2024-09-17T21:21:13.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158756 2024-09-17T21:21:13.191 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.191 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:13.191 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5751 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'480, mlcod=71'473)", 2024-09-17T21:21:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 116.87788585600001, 2024-09-17T21:21:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075756299999999999, 2024-09-17T21:21:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:13.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:13.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:13.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261037+0000", 2024-09-17T21:21:13.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:21:13.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261049+0000", 2024-09-17T21:21:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1063999999999999e-05 2024-09-17T21:21:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261072+0000", 2024-09-17T21:21:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3102e-05 2024-09-17T21:21:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261133+0000", 2024-09-17T21:21:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1276000000000001e-05 2024-09-17T21:21:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:13.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:21:13.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015334319999999999 2024-09-17T21:21:13.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:13.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262702+0000", 2024-09-17T21:21:13.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5281999999999999e-05 2024-09-17T21:21:13.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:13.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268527+0000", 2024-09-17T21:21:13.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058250389999999997 2024-09-17T21:21:13.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:13.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268611+0000", 2024-09-17T21:21:13.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4724999999999999e-05 2024-09-17T21:21:13.196 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.196 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:13.196 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.196 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5756 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'481, mlcod=71'473)", 2024-09-17T21:21:13.196 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:13.196 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 116.877025, 2024-09-17T21:21:13.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006772386, 2024-09-17T21:21:13.196 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:13.197 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:13.197 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:13.197 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:13.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:13.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:13.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:13.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:13.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:13.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261899+0000", 2024-09-17T21:21:13.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.903e-06 2024-09-17T21:21:13.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:13.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261912+0000", 2024-09-17T21:21:13.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-05 2024-09-17T21:21:13.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:13.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261938+0000", 2024-09-17T21:21:13.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6098999999999999e-05 2024-09-17T21:21:13.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:13.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261955+0000", 2024-09-17T21:21:13.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.696e-05 2024-09-17T21:21:13.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:13.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263845+0000", 2024-09-17T21:21:13.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001890187 2024-09-17T21:21:13.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:13.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263907+0000", 2024-09-17T21:21:13.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1600999999999995e-05 2024-09-17T21:21:13.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:13.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268615+0000", 2024-09-17T21:21:13.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047086070000000001 2024-09-17T21:21:13.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:13.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268669+0000", 2024-09-17T21:21:13.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3588999999999998e-05 2024-09-17T21:21:13.201 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.201 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:13.201 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.201 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4978.0:598 3.9 e80/23 3:9701cc17:::rbd_data.1152724ff2a3.0000000000000019:head v 80'202, mlcod=80'198)", 2024-09-17T21:21:13.201 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:21:13.201 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 106.375450617, 2024-09-17T21:21:13.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066473340000000004, 2024-09-17T21:21:13.201 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:13.201 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:13.201 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:13.201 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:13.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:13.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:21:13.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:13.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:21:13.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:13.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763472+0000", 2024-09-17T21:21:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6899999999999997e-07 2024-09-17T21:21:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763478+0000", 2024-09-17T21:21:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:21:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763489+0000", 2024-09-17T21:21:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0628999999999999e-05 2024-09-17T21:21:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763495+0000", 2024-09-17T21:21:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6649999999999997e-06 2024-09-17T21:21:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763907+0000", 2024-09-17T21:21:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041163899999999999 2024-09-17T21:21:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763976+0000", 2024-09-17T21:21:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8534999999999993e-05 2024-09-17T21:21:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.769900+0000", 2024-09-17T21:21:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059243270000000001 2024-09-17T21:21:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.770118+0000", 2024-09-17T21:21:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021842500000000001 2024-09-17T21:21:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:468 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'94, mlcod=89'87)", 2024-09-17T21:21:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 93.452705318, 2024-09-17T21:21:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066528790000000004, 2024-09-17T21:21:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686218+0000", 2024-09-17T21:21:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3060000000000001e-06 2024-09-17T21:21:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686256+0000", 2024-09-17T21:21:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8333e-05 2024-09-17T21:21:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686272+0000", 2024-09-17T21:21:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:21:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686283+0000", 2024-09-17T21:21:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0903e-05 2024-09-17T21:21:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686682+0000", 2024-09-17T21:21:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039935099999999999 2024-09-17T21:21:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686785+0000", 2024-09-17T21:21:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010275199999999999 2024-09-17T21:21:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692816+0000", 2024-09-17T21:21:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060317579999999999 2024-09-17T21:21:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692869+0000", 2024-09-17T21:21:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2775999999999998e-05 2024-09-17T21:21:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:755 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'127, mlcod=89'122)", 2024-09-17T21:21:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 93.316250104000005, 2024-09-17T21:21:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084014080000000008, 2024-09-17T21:21:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:21:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822683+0000", 2024-09-17T21:21:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0674e-05 2024-09-17T21:21:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822698+0000", 2024-09-17T21:21:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5189e-05 2024-09-17T21:21:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822708+0000", 2024-09-17T21:21:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9119999999999993e-06 2024-09-17T21:21:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822748+0000", 2024-09-17T21:21:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9727000000000001e-05 2024-09-17T21:21:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822759+0000", 2024-09-17T21:21:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0865e-05 2024-09-17T21:21:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831034+0000", 2024-09-17T21:21:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082750710000000002 2024-09-17T21:21:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831073+0000", 2024-09-17T21:21:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9063999999999997e-05 2024-09-17T21:21:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:756 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'128, mlcod=89'124)", 2024-09-17T21:21:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 93.314327472000002, 2024-09-17T21:21:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0076300270000000002, 2024-09-17T21:21:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824595+0000", 2024-09-17T21:21:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-17T21:21:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824600+0000", 2024-09-17T21:21:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:21:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824612+0000", 2024-09-17T21:21:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.181e-05 2024-09-17T21:21:13.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:13.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824619+0000", 2024-09-17T21:21:13.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:21:13.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:13.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824671+0000", 2024-09-17T21:21:13.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2262999999999998e-05 2024-09-17T21:21:13.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:13.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824695+0000", 2024-09-17T21:21:13.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3883000000000002e-05 2024-09-17T21:21:13.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:13.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832185+0000", 2024-09-17T21:21:13.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007490048 2024-09-17T21:21:13.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:13.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832224+0000", 2024-09-17T21:21:13.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8957e-05 2024-09-17T21:21:13.220 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.221 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:13.221 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.221 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2617 3.d e90/23 3:b0143718:::rbd_data.1152a8ae2019.0000000000000016:head v 90'249, mlcod=90'246)", 2024-09-17T21:21:13.221 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:21:13.221 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 92.294476471999999, 2024-09-17T21:21:13.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063715559999999996, 2024-09-17T21:21:13.221 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:13.221 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:13.221 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:13.221 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:13.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:13.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:21:13.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:13.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:21:13.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:13.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844455+0000", 2024-09-17T21:21:13.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4259999999999992e-06 2024-09-17T21:21:13.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844470+0000", 2024-09-17T21:21:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5386000000000001e-05 2024-09-17T21:21:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844499+0000", 2024-09-17T21:21:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.883e-05 2024-09-17T21:21:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844518+0000", 2024-09-17T21:21:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8830000000000001e-05 2024-09-17T21:21:13.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:13.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844649+0000", 2024-09-17T21:21:13.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013143399999999999 2024-09-17T21:21:13.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:13.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844702+0000", 2024-09-17T21:21:13.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2876e-05 2024-09-17T21:21:13.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:13.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850696+0000", 2024-09-17T21:21:13.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059943729999999999 2024-09-17T21:21:13.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:13.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850817+0000", 2024-09-17T21:21:13.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120401 2024-09-17T21:21:13.225 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.225 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:13.225 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.225 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2635 3.d e90/23 3:b3d1e920:::rbd_data.1152a8ae2019.000000000000001f:head v 90'252, mlcod=90'246)", 2024-09-17T21:21:13.225 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:13.226 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 92.291768402000002, 2024-09-17T21:21:13.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0077265190000000003, 2024-09-17T21:21:13.226 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:13.226 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:13.226 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:13.226 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:13.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:13.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:13.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:13.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847155+0000", 2024-09-17T21:21:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.544e-06 2024-09-17T21:21:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847165+0000", 2024-09-17T21:21:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.043e-05 2024-09-17T21:21:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:13.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847189+0000", 2024-09-17T21:21:13.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3659000000000002e-05 2024-09-17T21:21:13.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:13.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847210+0000", 2024-09-17T21:21:13.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1163e-05 2024-09-17T21:21:13.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:13.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848856+0000", 2024-09-17T21:21:13.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001645521 2024-09-17T21:21:13.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:13.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848905+0000", 2024-09-17T21:21:13.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9048999999999997e-05 2024-09-17T21:21:13.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:13.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854770+0000", 2024-09-17T21:21:13.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058652879999999998 2024-09-17T21:21:13.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:13.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854880+0000", 2024-09-17T21:21:13.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010986500000000001 2024-09-17T21:21:13.230 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.230 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:13.230 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.230 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "MOSDPGPush(3.5 151/149 [PushOp(3:a57cb214:::rbd_data.115262b8e217.0000000000000009:1d, version: 103'698, data_included: [196608~24576,229376~24576,262144~24576,294912~131072,524288~98304,819200~8192,843776~16384,876544~16384,909312~16384,942080~8192,1146880~16384,1179648~16384,1212416~16384,1474560~98304,1794048~8192,1826816~8192,1859584~8192,2105344~24576,2138112~24576,2170880~24576,2646016~8192,2678784~8192,2711552~8192,3072000~8192,3104768~8192,3137536~106496], data_size: 761856, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:a57cb214:::rbd_data.115262b8e217.0000000000000009:1d@103'698, size: 3276800, copy_subset: [(0, 3276800)], clone_subset: {}, snapset: 1d=[]:{1d: [1d]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3276800, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T21:21:13.230 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:07.006386+0000", 2024-09-17T21:21:13.230 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 6.1325359419999996, 2024-09-17T21:21:13.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082699420000000006, 2024-09-17T21:21:13.230 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:13.230 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:13.231 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:13.231 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:13.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:13.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.006386+0000", 2024-09-17T21:21:13.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:13.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.006386+0000", 2024-09-17T21:21:13.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:13.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:13.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.006388+0000", 2024-09-17T21:21:13.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.827e-06 2024-09-17T21:21:13.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:13.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.007671+0000", 2024-09-17T21:21:13.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0012836340000000001 2024-09-17T21:21:13.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:13.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.007737+0000", 2024-09-17T21:21:13.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5815000000000006e-05 2024-09-17T21:21:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.007782+0000", 2024-09-17T21:21:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5509000000000001e-05 2024-09-17T21:21:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.008023+0000", 2024-09-17T21:21:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000240184 2024-09-17T21:21:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.008065+0000", 2024-09-17T21:21:13.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1978999999999999e-05 2024-09-17T21:21:13.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:13.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:13.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:13.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.014656+0000", 2024-09-17T21:21:13.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065909940000000002 2024-09-17T21:21:13.234 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.234 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:13.234 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.234 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:13.234 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:13.234 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:13.236 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:21:13.236 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:13.232+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:13.337 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:13.336+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:13.379 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:13.379 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:13.380 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:13.380 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:13.393 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:21:13.404 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:21:13.453 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:13.452+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:13.512 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:13.512 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:13.526 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:21:13.539 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:13.536+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:13.585 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:13.585 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:13.605 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:13.605 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:13.622 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:21:13.622 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:21:13.639 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:13.636+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:13.643 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:13.643 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:13.663 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:13.663 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:13.666 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:13.666 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:13.682 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:21:13.683 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:13.741 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:13.740+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:13.829 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:13.829 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:13.841 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:13.833+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:13.843 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:21:13.941 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:13.937+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:14.010 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:14.010 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:14.024 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:21:14.042 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:14.040+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:14.143 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:14.137+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:14.225 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:14.225 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:14.244 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:21:14.244 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:14.240+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:14.329 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:21:14.335 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 0, 5, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:21:14.335 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:21:14.336 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-17T21:21:14.336 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T21:21:14.344 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:14.341+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:14.365 INFO:teuthology.orchestra.run.smithi184.stdout:5845 punch from 0xa1da800 to 0xa1e1a00, (0x7200 bytes) 2024-09-17T21:21:14.410 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:14.410 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:14.424 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:21:14.451 INFO:teuthology.orchestra.run.smithi184.stdout:5847 trunc from 0xd79fa00 to 0xcc55200 2024-09-17T21:21:14.467 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:14.461+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:14.521 INFO:teuthology.orchestra.run.smithi184.stdout:5848 read 0xa3baa00 thru 0xa3bf325 (0x4926 bytes) 2024-09-17T21:21:14.522 INFO:teuthology.orchestra.run.smithi184.stdout:5851 write 0xa533800 thru 0xa5339f3 (0x1f4 bytes) 2024-09-17T21:21:14.546 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:14.537+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:14.546 INFO:teuthology.orchestra.run.smithi184.stdout:5853 trunc from 0xcc55200 to 0x9dcb200 2024-09-17T21:21:14.600 INFO:teuthology.orchestra.run.smithi184.stdout:5855 write 0x8201e00 thru 0x821173f (0xf940 bytes) 2024-09-17T21:21:14.601 INFO:teuthology.orchestra.run.smithi184.stdout:5856 punch from 0x6aca600 to 0x6ad7a00, (0xd400 bytes) 2024-09-17T21:21:14.603 INFO:teuthology.orchestra.run.smithi184.stdout:5858 read 0x95b200 thru 0x9689b0 (0xd7b1 bytes) 2024-09-17T21:21:14.605 INFO:teuthology.orchestra.run.smithi184.stdout:5859 read 0x6967a00 thru 0x697330d (0xb90e bytes) 2024-09-17T21:21:14.605 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:14.605 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:14.610 INFO:teuthology.orchestra.run.smithi184.stdout:5860 read 0x989b600 thru 0x989ece4 (0x36e5 bytes) 2024-09-17T21:21:14.616 INFO:teuthology.orchestra.run.smithi184.stdout:5862 trunc from 0x9dcb200 to 0xcf7c400 2024-09-17T21:21:14.622 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:21:14.632 INFO:teuthology.orchestra.run.smithi184.stdout:5863 read 0x95ee00 thru 0x969216 (0xa417 bytes) 2024-09-17T21:21:14.647 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:14.644+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:14.679 INFO:teuthology.orchestra.run.smithi184.stdout:5866 trunc from 0xcf7c400 to 0x3c5f000 2024-09-17T21:21:14.709 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:21:14.709 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:14.709 INFO:teuthology.orchestra.run.smithi083.stdout: "epoch": 158, 2024-09-17T21:21:14.709 INFO:teuthology.orchestra.run.smithi083.stdout: "fsid": "224a7a29-e112-472e-8a13-f67345d6d123", 2024-09-17T21:21:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: "created": "2024-09-17T21:17:42.597312+0000", 2024-09-17T21:21:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: "modified": "2024-09-17T21:21:13.291698+0000", 2024-09-17T21:21:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: "last_up_change": "2024-09-17T21:17:53.976897+0000", 2024-09-17T21:21:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: "last_in_change": "2024-09-17T21:21:03.654444+0000", 2024-09-17T21:21:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T21:21:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_num": 5806080, 2024-09-17T21:21:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_set": [ 2024-09-17T21:21:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: "nodeep-scrub", 2024-09-17T21:21:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: "noscrub", 2024-09-17T21:21:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: "pglog_hardlimit", 2024-09-17T21:21:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: "purged_snapdirs", 2024-09-17T21:21:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_deletes", 2024-09-17T21:21:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: "sortbitwise" 2024-09-17T21:21:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:21:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_version": 5, 2024-09-17T21:21:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T21:21:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T21:21:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T21:21:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_snapshot": "", 2024-09-17T21:21:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_max": 3, 2024-09-17T21:21:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd": 6, 2024-09-17T21:21:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: "require_min_compat_client": "luminous", 2024-09-17T21:21:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: "min_compat_client": "luminous", 2024-09-17T21:21:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: "require_osd_release": "squid", 2024-09-17T21:21:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_crimson": false, 2024-09-17T21:21:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: "pools": [ 2024-09-17T21:21:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 1, 2024-09-17T21:21:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": ".mgr", 2024-09-17T21:21:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-17T21:17:55.581350+0000", 2024-09-17T21:21:14.712 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-17T21:21:14.712 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-17T21:21:14.712 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-17T21:21:14.712 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 2, 2024-09-17T21:21:14.712 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 1, 2024-09-17T21:21:14.712 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-17T21:21:14.712 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:21:14.712 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:21:14.712 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:21:14.712 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:21:14.712 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-17T21:21:14.712 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-17T21:21:14.712 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:21:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 1, 2024-09-17T21:21:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 1, 2024-09-17T21:21:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 1, 2024-09-17T21:21:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 1, 2024-09-17T21:21:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 1, 2024-09-17T21:21:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-17T21:21:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-17T21:21:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-17T21:21:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-17T21:21:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-17T21:21:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-17T21:21:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-17T21:21:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "17", 2024-09-17T21:21:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-17T21:21:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:21:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:21:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-17T21:21:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:21:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-17T21:21:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-17T21:21:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-17T21:21:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-17T21:21:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-17T21:21:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-17T21:21:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-17T21:21:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-17T21:21:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-17T21:21:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-17T21:21:14.715 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-17T21:21:14.715 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-17T21:21:14.715 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-17T21:21:14.715 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:21:14.715 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:21:14.715 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:21:14.715 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-17T21:21:14.715 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-17T21:21:14.715 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-17T21:21:14.715 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-17T21:21:14.715 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-17T21:21:14.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.715 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-17T21:21:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-17T21:21:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-17T21:21:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:21:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:21:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:21:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:21:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-17T21:21:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-17T21:21:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-17T21:21:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-17T21:21:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: "options": { 2024-09-17T21:21:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_max": 32, 2024-09-17T21:21:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_min": 1 2024-09-17T21:21:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-17T21:21:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": {} 2024-09-17T21:21:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-17T21:21:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-17T21:21:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 5, 2024-09-17T21:21:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 5, 2024-09-17T21:21:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 0.40000000596046448, 2024-09-17T21:21:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2, 2024-09-17T21:21:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2, 2024-09-17T21:21:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 0.53173065185546875, 2024-09-17T21:21:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 0.63999998569488525, 2024-09-17T21:21:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 0.52999997138977051 2024-09-17T21:21:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T21:21:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "rbd", 2024-09-17T21:21:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-17T21:17:59.603025+0000", 2024-09-17T21:21:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 8193, 2024-09-17T21:21:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:21:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-17T21:21:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 2, 2024-09-17T21:21:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 1, 2024-09-17T21:21:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-17T21:21:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:21:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:21:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:21:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:21:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-17T21:21:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-17T21:21:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:21:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 8, 2024-09-17T21:21:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 8, 2024-09-17T21:21:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 8, 2024-09-17T21:21:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 8, 2024-09-17T21:21:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 8, 2024-09-17T21:21:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-17T21:21:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-17T21:21:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-17T21:21:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-17T21:21:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-17T21:21:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-17T21:21:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-17T21:21:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "21", 2024-09-17T21:21:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-17T21:21:14.720 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:21:14.720 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:21:14.720 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-17T21:21:14.720 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:21:14.720 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 2, 2024-09-17T21:21:14.720 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 21, 2024-09-17T21:21:14.720 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-17T21:21:14.720 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-17T21:21:14.720 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-17T21:21:14.720 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-17T21:21:14.720 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-17T21:21:14.720 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-17T21:21:14.720 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-17T21:21:14.720 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-17T21:21:14.721 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-17T21:21:14.721 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-17T21:21:14.721 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-17T21:21:14.721 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:21:14.721 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:21:14.721 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:21:14.721 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-17T21:21:14.721 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-17T21:21:14.721 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-17T21:21:14.721 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-17T21:21:14.721 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-17T21:21:14.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.721 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-17T21:21:14.721 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-17T21:21:14.722 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-17T21:21:14.722 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:21:14.722 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:21:14.722 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:21:14.722 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:21:14.722 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-17T21:21:14.722 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-17T21:21:14.722 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-17T21:21:14.722 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-17T21:21:14.722 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-17T21:21:14.723 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-17T21:21:14.723 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd": {} 2024-09-17T21:21:14.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.723 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-17T21:21:14.723 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-17T21:21:14.723 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 2.5, 2024-09-17T21:21:14.723 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 2.5, 2024-09-17T21:21:14.723 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1.2000000476837158, 2024-09-17T21:21:14.723 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 3, 2024-09-17T21:21:14.723 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 3, 2024-09-17T21:21:14.723 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 0.53173065185546875, 2024-09-17T21:21:14.723 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 0.63999998569488525, 2024-09-17T21:21:14.723 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 0.52999997138977051 2024-09-17T21:21:14.724 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.724 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 3, 2024-09-17T21:21:14.724 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "pool_client.0", 2024-09-17T21:21:14.724 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-17T21:18:09.441137+0000", 2024-09-17T21:21:14.724 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 8193, 2024-09-17T21:21:14.724 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:21:14.724 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-17T21:21:14.724 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 2, 2024-09-17T21:21:14.724 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 1, 2024-09-17T21:21:14.724 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-17T21:21:14.724 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:21:14.725 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:21:14.725 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:21:14.725 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:21:14.725 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-17T21:21:14.725 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-17T21:21:14.725 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:21:14.725 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-17T21:21:14.725 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-17T21:21:14.725 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-17T21:21:14.725 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-17T21:21:14.726 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-17T21:21:14.726 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-17T21:21:14.726 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-17T21:21:14.726 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-17T21:21:14.726 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-17T21:21:14.726 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-17T21:21:14.726 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-17T21:21:14.726 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-17T21:21:14.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.726 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "153", 2024-09-17T21:21:14.726 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-17T21:21:14.726 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:21:14.726 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:21:14.726 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-17T21:21:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:21:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 44, 2024-09-17T21:21:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 153, 2024-09-17T21:21:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-17T21:21:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-17T21:21:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-17T21:21:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-17T21:21:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-17T21:21:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-17T21:21:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-17T21:21:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-17T21:21:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-17T21:21:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-17T21:21:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-17T21:21:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:21:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:21:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:21:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-17T21:21:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-17T21:21:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-17T21:21:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-17T21:21:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-17T21:21:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-17T21:21:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-17T21:21:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-17T21:21:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:21:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:21:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:21:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:21:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-17T21:21:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-17T21:21:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-17T21:21:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-17T21:21:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-17T21:21:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-17T21:21:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd": {} 2024-09-17T21:21:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-17T21:21:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-17T21:21:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 8.7700004577636719, 2024-09-17T21:21:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.4099999666213989, 2024-09-17T21:21:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1.2000000476837158, 2024-09-17T21:21:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 10.529999732971191, 2024-09-17T21:21:14.730 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.690000057220459, 2024-09-17T21:21:14.730 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 0.53173065185546875, 2024-09-17T21:21:14.730 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 0.63999998569488525, 2024-09-17T21:21:14.730 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 0.52999997138977051, 2024-09-17T21:21:14.730 INFO:teuthology.orchestra.run.smithi083.stdout: "error_message": "pool 3 has acting primaries on OSD(s) with primary affinity 0, read balance score is not accurate" 2024-09-17T21:21:14.730 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.730 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.730 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:21:14.730 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-17T21:21:14.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.730 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-17T21:21:14.730 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "5bfd41c9-0cac-47c4-9a6e-8dce774c3314", 2024-09-17T21:21:14.730 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:21:14.731 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:21:14.731 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:21:14.731 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 0.2524871826171875, 2024-09-17T21:21:14.731 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:21:14.731 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:21:14.731 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:21:14.731 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 153, 2024-09-17T21:21:14.731 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:21:14.731 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:21:14.731 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:21:14.732 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.732 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.732 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6816", 2024-09-17T21:21:14.732 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:21:14.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.732 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.732 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6817", 2024-09-17T21:21:14.732 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:21:14.732 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.732 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:21:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6818", 2024-09-17T21:21:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:21:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6819", 2024-09-17T21:21:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:21:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:21:14.734 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.734 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.734 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6822", 2024-09-17T21:21:14.734 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:21:14.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.734 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.734 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6823", 2024-09-17T21:21:14.734 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:21:14.734 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.734 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.735 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:21:14.735 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.735 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.735 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6820", 2024-09-17T21:21:14.735 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:21:14.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.735 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.735 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6821", 2024-09-17T21:21:14.735 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1111036754 2024-09-17T21:21:14.736 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.736 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.736 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6817/1111036754", 2024-09-17T21:21:14.736 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6819/1111036754", 2024-09-17T21:21:14.736 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6823/1111036754", 2024-09-17T21:21:14.736 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6821/1111036754", 2024-09-17T21:21:14.736 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:21:14.736 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:21:14.736 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:21:14.736 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.736 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-17T21:21:14.736 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "78404e0d-d397-4806-8e90-5f32182643bb", 2024-09-17T21:21:14.737 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:21:14.737 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:21:14.737 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:21:14.737 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-17T21:21:14.737 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:21:14.737 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:21:14.737 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:21:14.737 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 153, 2024-09-17T21:21:14.737 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:21:14.737 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:21:14.737 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:21:14.738 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.738 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.738 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6808", 2024-09-17T21:21:14.738 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:21:14.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.738 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.738 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6809", 2024-09-17T21:21:14.738 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:21:14.738 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.738 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.739 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:21:14.739 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.739 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.739 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6810", 2024-09-17T21:21:14.739 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:21:14.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.739 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.739 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6811", 2024-09-17T21:21:14.739 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:21:14.739 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.739 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.740 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:21:14.740 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.740 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.740 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6814", 2024-09-17T21:21:14.740 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:21:14.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.740 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.740 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6815", 2024-09-17T21:21:14.740 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:21:14.740 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.740 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.741 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:21:14.741 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.741 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.741 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6812", 2024-09-17T21:21:14.741 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:21:14.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.741 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.741 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6813", 2024-09-17T21:21:14.741 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1460363937 2024-09-17T21:21:14.741 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.741 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.742 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6809/1460363937", 2024-09-17T21:21:14.742 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6811/1460363937", 2024-09-17T21:21:14.742 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6815/1460363937", 2024-09-17T21:21:14.742 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6813/1460363937", 2024-09-17T21:21:14.742 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:21:14.742 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:21:14.742 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:21:14.742 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.742 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-17T21:21:14.742 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "286ed6e0-c121-4ebb-8cf6-37e506a397aa", 2024-09-17T21:21:14.742 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:21:14.742 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:21:14.743 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:21:14.743 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 0.2769012451171875, 2024-09-17T21:21:14.743 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:21:14.743 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:21:14.743 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 12, 2024-09-17T21:21:14.743 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 154, 2024-09-17T21:21:14.743 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:21:14.743 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:21:14.743 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:21:14.743 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.743 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.743 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6800", 2024-09-17T21:21:14.743 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:21:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6801", 2024-09-17T21:21:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:21:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:21:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.745 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6802", 2024-09-17T21:21:14.745 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:21:14.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.745 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.745 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6803", 2024-09-17T21:21:14.745 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:21:14.745 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.745 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.745 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:21:14.745 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.745 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.746 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6806", 2024-09-17T21:21:14.746 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:21:14.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.746 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.746 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6807", 2024-09-17T21:21:14.746 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:21:14.746 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.746 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.746 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:21:14.746 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.747 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.747 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6804", 2024-09-17T21:21:14.747 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:21:14.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.747 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.747 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6805", 2024-09-17T21:21:14.747 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2837444034 2024-09-17T21:21:14.747 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.747 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.747 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6801/2837444034", 2024-09-17T21:21:14.747 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6803/2837444034", 2024-09-17T21:21:14.747 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6807/2837444034", 2024-09-17T21:21:14.748 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6805/2837444034", 2024-09-17T21:21:14.748 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:21:14.748 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:21:14.748 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:21:14.748 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.748 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-17T21:21:14.748 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "b0af73a1-4bca-4892-a550-2ced87f4c357", 2024-09-17T21:21:14.748 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:21:14.748 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:21:14.748 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 0.9648590087890625, 2024-09-17T21:21:14.748 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 0.6609954833984375, 2024-09-17T21:21:14.748 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:21:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:21:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:21:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 154, 2024-09-17T21:21:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:21:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:21:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:21:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6808", 2024-09-17T21:21:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:21:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.750 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6809", 2024-09-17T21:21:14.750 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:21:14.750 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.750 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.750 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:21:14.750 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.750 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.750 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6810", 2024-09-17T21:21:14.750 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:21:14.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.750 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.751 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6811", 2024-09-17T21:21:14.751 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:21:14.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.751 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.751 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:21:14.751 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.751 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.751 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6814", 2024-09-17T21:21:14.751 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:21:14.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.751 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.752 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6815", 2024-09-17T21:21:14.752 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:21:14.752 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.752 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.752 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:21:14.752 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.752 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.752 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6812", 2024-09-17T21:21:14.752 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:21:14.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6813", 2024-09-17T21:21:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3637204865 2024-09-17T21:21:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.97:6809/3637204865", 2024-09-17T21:21:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.97:6811/3637204865", 2024-09-17T21:21:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.97:6815/3637204865", 2024-09-17T21:21:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.97:6813/3637204865", 2024-09-17T21:21:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:21:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:21:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:21:14.754 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.754 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-17T21:21:14.754 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "2188361c-e1a1-45fa-bc01-da5a984ffd9f", 2024-09-17T21:21:14.754 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:21:14.754 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:21:14.754 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 0.6880035400390625, 2024-09-17T21:21:14.754 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-17T21:21:14.754 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:21:14.754 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:21:14.754 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:21:14.754 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 157, 2024-09-17T21:21:14.755 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:21:14.755 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:21:14.755 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:21:14.755 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.755 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.755 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6800", 2024-09-17T21:21:14.755 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:21:14.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.755 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.755 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6801", 2024-09-17T21:21:14.755 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:21:14.755 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.756 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.756 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:21:14.756 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.756 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.756 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6802", 2024-09-17T21:21:14.756 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:21:14.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.756 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.756 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6803", 2024-09-17T21:21:14.756 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:21:14.757 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.757 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.757 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:21:14.757 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.757 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.757 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6806", 2024-09-17T21:21:14.757 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:21:14.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.757 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.757 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6807", 2024-09-17T21:21:14.758 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:21:14.758 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.758 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.758 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:21:14.758 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.758 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.758 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6804", 2024-09-17T21:21:14.758 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:21:14.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.758 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.758 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6805", 2024-09-17T21:21:14.759 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2391356911 2024-09-17T21:21:14.759 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.759 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.759 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.97:6801/2391356911", 2024-09-17T21:21:14.759 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.97:6803/2391356911", 2024-09-17T21:21:14.759 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.97:6807/2391356911", 2024-09-17T21:21:14.759 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.97:6805/2391356911", 2024-09-17T21:21:14.759 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:21:14.759 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:21:14.759 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:21:14.759 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.760 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-17T21:21:14.760 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "7feccf4d-d8e4-4d06-ac83-bbc1bc623a5c", 2024-09-17T21:21:14.760 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-17T21:21:14.760 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-17T21:21:14.760 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-17T21:21:14.760 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 0, 2024-09-17T21:21:14.760 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-17T21:21:14.760 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-17T21:21:14.760 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-17T21:21:14.760 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 152, 2024-09-17T21:21:14.760 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-17T21:21:14.760 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-17T21:21:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-17T21:21:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6816", 2024-09-17T21:21:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:21:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6817", 2024-09-17T21:21:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:21:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-17T21:21:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6818", 2024-09-17T21:21:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:21:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6819", 2024-09-17T21:21:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:21:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-17T21:21:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6822", 2024-09-17T21:21:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:21:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6823", 2024-09-17T21:21:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:21:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-17T21:21:14.764 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-17T21:21:14.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.764 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-17T21:21:14.764 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6820", 2024-09-17T21:21:14.764 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:21:14.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.764 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-17T21:21:14.764 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.97:6821", 2024-09-17T21:21:14.764 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2326316381 2024-09-17T21:21:14.764 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.764 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.765 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.97:6817/2326316381", 2024-09-17T21:21:14.765 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.97:6819/2326316381", 2024-09-17T21:21:14.765 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.97:6823/2326316381", 2024-09-17T21:21:14.765 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.97:6821/2326316381", 2024-09-17T21:21:14.765 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-17T21:21:14.765 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-17T21:21:14.765 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-17T21:21:14.765 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.765 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.765 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:21:14.765 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_xinfo": [ 2024-09-17T21:21:14.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.765 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-17T21:21:14.765 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:21:14.766 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:21:14.766 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:21:14.766 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:21:14.766 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:21:14.766 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:51.138101+0000", 2024-09-17T21:21:14.766 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:21:14.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.766 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-17T21:21:14.766 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:21:14.766 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:21:14.766 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:21:14.766 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:21:14.766 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:21:14.767 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:50.581632+0000", 2024-09-17T21:21:14.767 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:21:14.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.767 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-17T21:21:14.767 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:21:14.767 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:21:14.767 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:21:14.767 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:21:14.767 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:21:14.767 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:50.365193+0000", 2024-09-17T21:21:14.767 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:21:14.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.768 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-17T21:21:14.768 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:21:14.768 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:21:14.768 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:21:14.768 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:21:14.768 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:21:14.768 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:50.718974+0000", 2024-09-17T21:21:14.768 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:21:14.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.768 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-17T21:21:14.768 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:21:14.769 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:21:14.769 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:21:14.769 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:21:14.769 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:21:14.769 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:50.501878+0000", 2024-09-17T21:21:14.769 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:21:14.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.769 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-17T21:21:14.769 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-17T21:21:14.769 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-17T21:21:14.769 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-17T21:21:14.769 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-17T21:21:14.769 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-17T21:21:14.770 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-17T21:17:51.150747+0000", 2024-09-17T21:21:14.770 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-17T21:21:14.770 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.770 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:21:14.770 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap": [ 2024-09-17T21:21:14.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.770 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "3.8", 2024-09-17T21:21:14.770 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-17T21:21:14.770 INFO:teuthology.orchestra.run.smithi083.stdout: 2, 2024-09-17T21:21:14.770 INFO:teuthology.orchestra.run.smithi083.stdout: 3 2024-09-17T21:21:14.770 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.771 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "3.c", 2024-09-17T21:21:14.771 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-17T21:21:14.771 INFO:teuthology.orchestra.run.smithi083.stdout: 1, 2024-09-17T21:21:14.771 INFO:teuthology.orchestra.run.smithi083.stdout: 5 2024-09-17T21:21:14.771 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:14.771 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "3.1e", 2024-09-17T21:21:14.771 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-17T21:21:14.771 INFO:teuthology.orchestra.run.smithi083.stdout: 0, 2024-09-17T21:21:14.771 INFO:teuthology.orchestra.run.smithi083.stdout: 4 2024-09-17T21:21:14.771 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:14.771 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.771 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:21:14.772 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_items": [], 2024-09-17T21:21:14.772 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_primaries": [], 2024-09-17T21:21:14.772 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_temp": [], 2024-09-17T21:21:14.772 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_temp": [], 2024-09-17T21:21:14.772 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklist": {}, 2024-09-17T21:21:14.772 INFO:teuthology.orchestra.run.smithi083.stdout: "range_blocklist": {}, 2024-09-17T21:21:14.772 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profiles": { 2024-09-17T21:21:14.772 INFO:teuthology.orchestra.run.smithi083.stdout: "default": { 2024-09-17T21:21:14.772 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-failure-domain": "osd", 2024-09-17T21:21:14.772 INFO:teuthology.orchestra.run.smithi083.stdout: "k": "2", 2024-09-17T21:21:14.772 INFO:teuthology.orchestra.run.smithi083.stdout: "m": "1", 2024-09-17T21:21:14.772 INFO:teuthology.orchestra.run.smithi083.stdout: "plugin": "jerasure", 2024-09-17T21:21:14.772 INFO:teuthology.orchestra.run.smithi083.stdout: "technique": "reed_sol_van" 2024-09-17T21:21:14.772 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:14.773 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps_queue": [], 2024-09-17T21:21:14.773 INFO:teuthology.orchestra.run.smithi083.stdout: "new_removed_snaps": [], 2024-09-17T21:21:14.773 INFO:teuthology.orchestra.run.smithi083.stdout: "new_purged_snaps": [], 2024-09-17T21:21:14.773 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_node_flags": {}, 2024-09-17T21:21:14.773 INFO:teuthology.orchestra.run.smithi083.stdout: "device_class_flags": {}, 2024-09-17T21:21:14.773 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode": { 2024-09-17T21:21:14.773 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_enabled": false, 2024-09-17T21:21:14.773 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_bucket_count": 0, 2024-09-17T21:21:14.773 INFO:teuthology.orchestra.run.smithi083.stdout: "degraded_stretch_mode": 0, 2024-09-17T21:21:14.773 INFO:teuthology.orchestra.run.smithi083.stdout: "recovering_stretch_mode": 0, 2024-09-17T21:21:14.773 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_bucket": 0 2024-09-17T21:21:14.773 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:14.773 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:14.776 INFO:tasks.thrashosds.thrasher:j is {'epoch': 158, 'fsid': '224a7a29-e112-472e-8a13-f67345d6d123', 'created': '2024-09-17T21:17:42.597312+0000', 'modified': '2024-09-17T21:21:13.291698+0000', 'last_up_change': '2024-09-17T21:17:53.976897+0000', 'last_in_change': '2024-09-17T21:21:03.654444+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T21:17:55.581350+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5, 'score_stable': 5, 'optimal_score': 0.4000000059604645, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.5317306518554688, 'average_primary_affinity': 0.6399999856948853, 'average_primary_affinity_weighted': 0.5299999713897705}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T21:17:59.603025+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1.2000000476837158, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.5317306518554688, 'average_primary_affinity': 0.6399999856948853, 'average_primary_affinity_weighted': 0.5299999713897705}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T21:18:09.441137+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '153', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 44, 'snap_epoch': 153, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8.770000457763672, 'score_stable': 1.409999966621399, 'optimal_score': 1.2000000476837158, 'raw_score_acting': 10.529999732971191, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.5317306518554688, 'average_primary_affinity': 0.6399999856948853, 'average_primary_affinity_weighted': 0.5299999713897705, 'error_message': 'pool 3 has acting primaries on OSD(s) with primary affinity 0, read balance score is not accurate'}}], 'osds': [{'osd': 0, 'uuid': '5bfd41c9-0cac-47c4-9a6e-8dce774c3314', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.2524871826171875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 153, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6816', 'nonce': 1111036754}, {'type': 'v1', 'addr': '172.21.15.83:6817', 'nonce': 1111036754}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6818', 'nonce': 1111036754}, {'type': 'v1', 'addr': '172.21.15.83:6819', 'nonce': 1111036754}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6822', 'nonce': 1111036754}, {'type': 'v1', 'addr': '172.21.15.83:6823', 'nonce': 1111036754}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6820', 'nonce': 1111036754}, {'type': 'v1', 'addr': '172.21.15.83:6821', 'nonce': 1111036754}]}, 'public_addr': '172.21.15.83:6817/1111036754', 'cluster_addr': '172.21.15.83:6819/1111036754', 'heartbeat_back_addr': '172.21.15.83:6823/1111036754', 'heartbeat_front_addr': '172.21.15.83:6821/1111036754', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '78404e0d-d397-4806-8e90-5f32182643bb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 153, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6808', 'nonce': 1460363937}, {'type': 'v1', 'addr': '172.21.15.83:6809', 'nonce': 1460363937}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6810', 'nonce': 1460363937}, {'type': 'v1', 'addr': '172.21.15.83:6811', 'nonce': 1460363937}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6814', 'nonce': 1460363937}, {'type': 'v1', 'addr': '172.21.15.83:6815', 'nonce': 1460363937}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6812', 'nonce': 1460363937}, {'type': 'v1', 'addr': '172.21.15.83:6813', 'nonce': 1460363937}]}, 'public_addr': '172.21.15.83:6809/1460363937', 'cluster_addr': '172.21.15.83:6811/1460363937', 'heartbeat_back_addr': '172.21.15.83:6815/1460363937', 'heartbeat_front_addr': '172.21.15.83:6813/1460363937', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '286ed6e0-c121-4ebb-8cf6-37e506a397aa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.2769012451171875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 154, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6800', 'nonce': 2837444034}, {'type': 'v1', 'addr': '172.21.15.83:6801', 'nonce': 2837444034}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6802', 'nonce': 2837444034}, {'type': 'v1', 'addr': '172.21.15.83:6803', 'nonce': 2837444034}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6806', 'nonce': 2837444034}, {'type': 'v1', 'addr': '172.21.15.83:6807', 'nonce': 2837444034}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6804', 'nonce': 2837444034}, {'type': 'v1', 'addr': '172.21.15.83:6805', 'nonce': 2837444034}]}, 'public_addr': '172.21.15.83:6801/2837444034', 'cluster_addr': '172.21.15.83:6803/2837444034', 'heartbeat_back_addr': '172.21.15.83:6807/2837444034', 'heartbeat_front_addr': '172.21.15.83:6805/2837444034', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'b0af73a1-4bca-4892-a550-2ced87f4c357', 'up': 1, 'in': 1, 'weight': 0.9648590087890625, 'primary_affinity': 0.6609954833984375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 154, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6808', 'nonce': 3637204865}, {'type': 'v1', 'addr': '172.21.15.97:6809', 'nonce': 3637204865}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6810', 'nonce': 3637204865}, {'type': 'v1', 'addr': '172.21.15.97:6811', 'nonce': 3637204865}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6814', 'nonce': 3637204865}, {'type': 'v1', 'addr': '172.21.15.97:6815', 'nonce': 3637204865}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6812', 'nonce': 3637204865}, {'type': 'v1', 'addr': '172.21.15.97:6813', 'nonce': 3637204865}]}, 'public_addr': '172.21.15.97:6809/3637204865', 'cluster_addr': '172.21.15.97:6811/3637204865', 'heartbeat_back_addr': '172.21.15.97:6815/3637204865', 'heartbeat_front_addr': '172.21.15.97:6813/3637204865', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2188361c-e1a1-45fa-bc01-da5a984ffd9f', 'up': 1, 'in': 1, 'weight': 0.6880035400390625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 157, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6800', 'nonce': 2391356911}, {'type': 'v1', 'addr': '172.21.15.97:6801', 'nonce': 2391356911}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6802', 'nonce': 2391356911}, {'type': 'v1', 'addr': '172.21.15.97:6803', 'nonce': 2391356911}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6806', 'nonce': 2391356911}, {'type': 'v1', 'addr': '172.21.15.97:6807', 'nonce': 2391356911}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6804', 'nonce': 2391356911}, {'type': 'v1', 'addr': '172.21.15.97:6805', 'nonce': 2391356911}]}, 'public_addr': '172.21.15.97:6801/2391356911', 'cluster_addr': '172.21.15.97:6803/2391356911', 'heartbeat_back_addr': '172.21.15.97:6807/2391356911', 'heartbeat_front_addr': '172.21.15.97:6805/2391356911', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '7feccf4d-d8e4-4d06-ac83-bbc1bc623a5c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 152, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6816', 'nonce': 2326316381}, {'type': 'v1', 'addr': '172.21.15.97:6817', 'nonce': 2326316381}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6818', 'nonce': 2326316381}, {'type': 'v1', 'addr': '172.21.15.97:6819', 'nonce': 2326316381}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6822', 'nonce': 2326316381}, {'type': 'v1', 'addr': '172.21.15.97:6823', 'nonce': 2326316381}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6820', 'nonce': 2326316381}, {'type': 'v1', 'addr': '172.21.15.97:6821', 'nonce': 2326316381}]}, 'public_addr': '172.21.15.97:6817/2326316381', 'cluster_addr': '172.21.15.97:6819/2326316381', 'heartbeat_back_addr': '172.21.15.97:6823/2326316381', 'heartbeat_front_addr': '172.21.15.97:6821/2326316381', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:51.138101+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:50.581632+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:50.365193+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:50.718974+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:50.501878+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:17:51.150747+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.8', 'osds': [2, 3]}, {'pgid': '3.c', 'osds': [1, 5]}, {'pgid': '3.1e', 'osds': [0, 4]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T21:21:14.776 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:21:14.777 INFO:teuthology.orchestra.run.smithi184.stdout:5868 read 0x3098400 thru 0x30a6a82 (0xe683 bytes) 2024-09-17T21:21:14.777 INFO:teuthology.orchestra.run.smithi184.stdout:5870 read 0xde7600 thru 0xdf21c0 (0xabc1 bytes) 2024-09-17T21:21:14.777 INFO:teuthology.orchestra.run.smithi184.stdout:5873 read 0x9cf600 thru 0x9d4fdb (0x59dc bytes) 2024-09-17T21:21:14.777 INFO:teuthology.orchestra.run.smithi184.stdout:5874 write 0x4a9b200 thru 0x4aa1716 (0x6517 bytes) 2024-09-17T21:21:14.777 INFO:teuthology.orchestra.run.smithi184.stdout:5877 read 0x35d4c00 thru 0x35d6f4e (0x234f bytes) 2024-09-17T21:21:14.777 INFO:teuthology.orchestra.run.smithi184.stdout:5882 read 0x2372a00 thru 0x23773f2 (0x49f3 bytes) 2024-09-17T21:21:14.777 INFO:teuthology.orchestra.run.smithi184.stdout:5883 punch from 0x126ba00 to 0x126d000, (0x1600 bytes) 2024-09-17T21:21:14.777 INFO:teuthology.orchestra.run.smithi184.stdout:5884 read 0x5b7000 thru 0x5c394f (0xc950 bytes) 2024-09-17T21:21:14.778 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:14.765+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:14.779 INFO:teuthology.orchestra.run.smithi184.stdout:5887 trunc from 0x4aa1800 to 0x8ec7200 2024-09-17T21:21:14.795 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:14.795 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:14.797 INFO:teuthology.orchestra.run.smithi184.stdout:5888 read 0x3255200 thru 0x325a627 (0x5428 bytes) 2024-09-17T21:21:14.801 INFO:teuthology.orchestra.run.smithi184.stdout:5889 punch from 0x1ce3200 to 0x1ceb000, (0x7e00 bytes) 2024-09-17T21:21:14.808 INFO:teuthology.orchestra.run.smithi184.stdout:5891 trunc from 0x8ec7200 to 0xb664000 2024-09-17T21:21:14.810 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:21:14.829 INFO:teuthology.orchestra.run.smithi184.stdout:5892 write 0xb53000 thru 0xb5648d (0x348e bytes) 2024-09-17T21:21:14.831 INFO:teuthology.orchestra.run.smithi184.stdout:5896 write 0x6767400 thru 0x6768db6 (0x19b7 bytes) 2024-09-17T21:21:14.831 INFO:teuthology.orchestra.run.smithi184.stdout:5897 write 0x8de3e00 thru 0x8de9191 (0x5392 bytes) 2024-09-17T21:21:14.875 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:14.872+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:14.878 INFO:teuthology.orchestra.run.smithi184.stdout:5898 write 0xec1fe00 thru 0xec2a374 (0xa575 bytes) 2024-09-17T21:21:14.880 INFO:teuthology.orchestra.run.smithi184.stdout:5899 write 0x574da00 thru 0x5751b12 (0x4113 bytes) 2024-09-17T21:21:14.880 INFO:teuthology.orchestra.run.smithi184.stdout:5900 read 0xab32000 thru 0xab3fd7f (0xdd80 bytes) 2024-09-17T21:21:14.880 INFO:teuthology.orchestra.run.smithi184.stdout:5901 write 0x6356e00 thru 0x6364b0a (0xdd0b bytes) 2024-09-17T21:21:14.880 INFO:teuthology.orchestra.run.smithi184.stdout:5905 write 0x8623200 thru 0x862fc46 (0xca47 bytes) 2024-09-17T21:21:14.913 INFO:teuthology.orchestra.run.smithi184.stdout:5909 trunc from 0xec2a400 to 0x3681800 2024-09-17T21:21:14.946 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:14.946 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:14.966 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:21:14.976 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:14.969+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:15.070 INFO:teuthology.orchestra.run.smithi184.stdout:5910 write 0xbda7e00 thru 0xbdaa9ac (0x2bad bytes) 2024-09-17T21:21:15.077 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:15.069+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:15.127 INFO:teuthology.orchestra.run.smithi184.stdout:5911 trunc from 0xbdaaa00 to 0xc7da00 2024-09-17T21:21:15.130 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:15.130 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:15.145 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:21:15.168 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:21:15.169 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T21:21:15.177 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:15.173+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:15.196 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2024-09-17T21:21:13.584321+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1562992899,"num_objects":1805,"num_object_clones":495,"num_object_copies":3610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1805,"num_whiteouts":606,"num_read":318379,"num_read_kb":1065969,"num_write":17262,"num_write_kb":383166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":308080175,"num_keys_recovered":804,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14655,"ondisk_log_size":14655,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":793360,"kb_used_data":468816,"kb_used_omap":9,"kb_used_meta":324406,"kb_avail":565437680,"statfs":{"total":579820584960,"available":579008184320,"internally_reserved":0,"allocated":480067584,"data_stored":709259430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9574,"internal_metadata":332192410},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":102,"num_read_kb":87,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":4863320,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001303"},"pg_stats":[{"pgid":"3.1f","version":"154'505","reported_seq":2443,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.307614+0000","last_change":"2024-09-17T21:20:53.517639+0000","last_active":"2024-09-17T21:21:09.307614+0000","last_peered":"2024-09-17T21:21:09.307614+0000","last_clean":"2024-09-17T21:21:09.307614+0000","last_became_active":"2024-09-17T21:20:53.515713+0000","last_became_peered":"2024-09-17T21:20:53.515713+0000","last_unstale":"2024-09-17T21:21:09.307614+0000","last_undegraded":"2024-09-17T21:21:09.307614+0000","last_fullsized":"2024-09-17T21:21:09.307614+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:39:43.272505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088525599999999996,"stat_sum":{"num_bytes":11127741,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":15,"num_read":4548,"num_read_kb":15437,"num_write":689,"num_write_kb":3233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":568483,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"151'708","reported_seq":8294,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.314005+0000","last_change":"2024-09-17T21:20:53.519851+0000","last_active":"2024-09-17T21:21:09.314005+0000","last_peered":"2024-09-17T21:21:09.314005+0000","last_clean":"2024-09-17T21:21:09.314005+0000","last_became_active":"2024-09-17T21:20:53.516962+0000","last_became_peered":"2024-09-17T21:20:53.516962+0000","last_unstale":"2024-09-17T21:21:09.314005+0000","last_undegraded":"2024-09-17T21:21:09.314005+0000","last_fullsized":"2024-09-17T21:21:09.314005+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:35:18.108413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013256229999999999,"stat_sum":{"num_bytes":51642143,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":14,"num_read":10112,"num_read_kb":51439,"num_write":967,"num_write_kb":18671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":390798,"num_keys_recovered":27,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"153'672","reported_seq":7654,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.313043+0000","last_change":"2024-09-17T21:20:37.309498+0000","last_active":"2024-09-17T21:21:09.313043+0000","last_peered":"2024-09-17T21:21:09.313043+0000","last_clean":"2024-09-17T21:21:09.313043+0000","last_became_active":"2024-09-17T21:20:37.307252+0000","last_became_peered":"2024-09-17T21:20:37.307252+0000","last_unstale":"2024-09-17T21:21:09.313043+0000","last_undegraded":"2024-09-17T21:21:09.313043+0000","last_fullsized":"2024-09-17T21:21:09.313043+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:23:33.966116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018973079999999999,"stat_sum":{"num_bytes":38933737,"num_objects":63,"num_object_clones":16,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":26,"num_read":9554,"num_read_kb":26298,"num_write":864,"num_write_kb":10373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":8289602,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"154'280","reported_seq":3135,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.314460+0000","last_change":"2024-09-17T21:20:36.449746+0000","last_active":"2024-09-17T21:21:09.314460+0000","last_peered":"2024-09-17T21:21:09.314460+0000","last_clean":"2024-09-17T21:21:09.314460+0000","last_became_active":"2024-09-17T21:20:36.447759+0000","last_became_peered":"2024-09-17T21:20:36.447759+0000","last_unstale":"2024-09-17T21:21:09.314460+0000","last_undegraded":"2024-09-17T21:21:09.314460+0000","last_fullsized":"2024-09-17T21:21:09.314460+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:47:47.088603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034186630000000002,"stat_sum":{"num_bytes":27721744,"num_objects":48,"num_object_clones":15,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":15,"num_read":2404,"num_read_kb":23420,"num_write":273,"num_write_kb":5911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":5917728,"num_keys_recovered":132,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"145'514","reported_seq":6968,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.310153+0000","last_change":"2024-09-17T21:19:20.291675+0000","last_active":"2024-09-17T21:21:09.310153+0000","last_peered":"2024-09-17T21:21:09.310153+0000","last_clean":"2024-09-17T21:21:09.310153+0000","last_became_active":"2024-09-17T21:19:20.290844+0000","last_became_peered":"2024-09-17T21:19:20.290844+0000","last_unstale":"2024-09-17T21:21:09.310153+0000","last_undegraded":"2024-09-17T21:21:09.310153+0000","last_fullsized":"2024-09-17T21:21:09.310153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:10:38.361300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089504999999999999,"stat_sum":{"num_bytes":29107728,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":21,"num_read":10240,"num_read_kb":19908,"num_write":406,"num_write_kb":4647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":1443328,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"146'574","reported_seq":11878,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.406202+0000","last_change":"2024-09-17T21:18:13.314063+0000","last_active":"2024-09-17T21:21:09.406202+0000","last_peered":"2024-09-17T21:21:09.406202+0000","last_clean":"2024-09-17T21:21:09.406202+0000","last_became_active":"2024-09-17T21:18:11.301426+0000","last_became_peered":"2024-09-17T21:18:11.301426+0000","last_unstale":"2024-09-17T21:21:09.406202+0000","last_undegraded":"2024-09-17T21:21:09.406202+0000","last_fullsized":"2024-09-17T21:21:09.406202+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:21:34.122539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027366090000000001,"stat_sum":{"num_bytes":54786158,"num_objects":61,"num_object_clones":19,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":14,"num_read":15326,"num_read_kb":50575,"num_write":562,"num_write_kb":13494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"154'370","reported_seq":10624,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.415609+0000","last_change":"2024-09-17T21:20:19.296133+0000","last_active":"2024-09-17T21:21:09.415609+0000","last_peered":"2024-09-17T21:21:09.415609+0000","last_clean":"2024-09-17T21:21:09.415609+0000","last_became_active":"2024-09-17T21:20:19.295432+0000","last_became_peered":"2024-09-17T21:20:19.295432+0000","last_unstale":"2024-09-17T21:21:09.415609+0000","last_undegraded":"2024-09-17T21:21:09.415609+0000","last_fullsized":"2024-09-17T21:21:09.415609+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:08:04.170270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047673000000000002,"stat_sum":{"num_bytes":40430143,"num_objects":60,"num_object_clones":19,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":19,"num_read":11155,"num_read_kb":25382,"num_write":323,"num_write_kb":5552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":3100207,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"146'258","reported_seq":2073,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-17T21:21:13.338012+0000","last_change":"2024-09-17T21:21:07.308106+0000","last_active":"2024-09-17T21:21:13.338012+0000","last_peered":"2024-09-17T21:21:13.338012+0000","last_clean":"2024-09-17T21:21:13.338012+0000","last_became_active":"2024-09-17T21:21:07.305917+0000","last_became_peered":"2024-09-17T21:21:07.305917+0000","last_unstale":"2024-09-17T21:21:13.338012+0000","last_undegraded":"2024-09-17T21:21:13.338012+0000","last_fullsized":"2024-09-17T21:21:13.338012+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:49:49.542316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26157907,"num_objects":59,"num_object_clones":16,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":26,"num_read":2234,"num_read_kb":8952,"num_write":274,"num_write_kb":2893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":3791606,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"146'302","reported_seq":5387,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.310017+0000","last_change":"2024-09-17T21:20:37.308092+0000","last_active":"2024-09-17T21:21:09.310017+0000","last_peered":"2024-09-17T21:21:09.310017+0000","last_clean":"2024-09-17T21:21:09.310017+0000","last_became_active":"2024-09-17T21:20:37.307341+0000","last_became_peered":"2024-09-17T21:20:37.307341+0000","last_unstale":"2024-09-17T21:21:09.310017+0000","last_undegraded":"2024-09-17T21:21:09.310017+0000","last_fullsized":"2024-09-17T21:21:09.310017+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:31:14.478630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088962999999999998,"stat_sum":{"num_bytes":50984602,"num_objects":56,"num_object_clones":18,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":21,"num_read":5936,"num_read_kb":20107,"num_write":424,"num_write_kb":6905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":11001140,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"142'517","reported_seq":20569,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.306711+0000","last_change":"2024-09-17T21:20:39.310051+0000","last_active":"2024-09-17T21:21:09.306711+0000","last_peered":"2024-09-17T21:21:09.306711+0000","last_clean":"2024-09-17T21:21:09.306711+0000","last_became_active":"2024-09-17T21:20:39.307497+0000","last_became_peered":"2024-09-17T21:20:39.307497+0000","last_unstale":"2024-09-17T21:21:09.306711+0000","last_undegraded":"2024-09-17T21:21:09.306711+0000","last_fullsized":"2024-09-17T21:21:09.306711+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:25:37.703766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001825658,"stat_sum":{"num_bytes":31594864,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":14,"num_read":21315,"num_read_kb":26392,"num_write":793,"num_write_kb":12977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":2260704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"146'648","reported_seq":6073,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.306689+0000","last_change":"2024-09-17T21:19:20.288548+0000","last_active":"2024-09-17T21:21:09.306689+0000","last_peered":"2024-09-17T21:21:09.306689+0000","last_clean":"2024-09-17T21:21:09.306689+0000","last_became_active":"2024-09-17T21:19:20.287118+0000","last_became_peered":"2024-09-17T21:19:20.287118+0000","last_unstale":"2024-09-17T21:21:09.306689+0000","last_undegraded":"2024-09-17T21:21:09.306689+0000","last_fullsized":"2024-09-17T21:21:09.306689+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":648,"log_dups_size":0,"ondisk_log_size":648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:52:30.156493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027112149999999999,"stat_sum":{"num_bytes":63157872,"num_objects":58,"num_object_clones":16,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":19,"num_read":8100,"num_read_kb":29018,"num_write":882,"num_write_kb":5515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":3174038,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"142'352","reported_seq":3880,"reported_epoch":154,"state":"unknown","last_fresh":"2024-09-17T21:21:09.313699+0000","last_change":"2024-09-17T21:21:09.522159+0000","last_active":"2024-09-17T21:21:09.313699+0000","last_peered":"2024-09-17T21:20:51.766046+0000","last_clean":"2024-09-17T21:20:51.766046+0000","last_became_active":"2024-09-17T21:18:11.299817+0000","last_became_peered":"2024-09-17T21:18:11.299817+0000","last_unstale":"2024-09-17T21:21:09.313699+0000","last_undegraded":"2024-09-17T21:21:09.313699+0000","last_fullsized":"2024-09-17T21:21:09.313699+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002281113,"stat_sum":{"num_bytes":48974139,"num_objects":54,"num_object_clones":16,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":22,"num_read":4753,"num_read_kb":24343,"num_write":373,"num_write_kb":14906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":287,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.307814+0000","last_change":"2024-09-17T21:18:03.059715+0000","last_active":"2024-09-17T21:21:09.307814+0000","last_peered":"2024-09-17T21:21:09.307814+0000","last_clean":"2024-09-17T21:21:09.307814+0000","last_became_active":"2024-09-17T21:18:01.213047+0000","last_became_peered":"2024-09-17T21:18:01.213047+0000","last_unstale":"2024-09-17T21:21:09.307814+0000","last_undegraded":"2024-09-17T21:21:09.307814+0000","last_fullsized":"2024-09-17T21:21:09.307814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:46:35.222373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"146'461","reported_seq":11838,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.309766+0000","last_change":"2024-09-17T21:19:20.290612+0000","last_active":"2024-09-17T21:21:09.309766+0000","last_peered":"2024-09-17T21:21:09.309766+0000","last_clean":"2024-09-17T21:21:09.309766+0000","last_became_active":"2024-09-17T21:19:20.289862+0000","last_became_peered":"2024-09-17T21:19:20.289862+0000","last_unstale":"2024-09-17T21:21:09.309766+0000","last_undegraded":"2024-09-17T21:21:09.309766+0000","last_fullsized":"2024-09-17T21:21:09.309766+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:35:22.676528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084327200000000005,"stat_sum":{"num_bytes":57410914,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":18,"num_read":13489,"num_read_kb":59784,"num_write":525,"num_write_kb":23533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":17516408,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":287,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.305831+0000","last_change":"2024-09-17T21:18:03.073440+0000","last_active":"2024-09-17T21:21:09.305831+0000","last_peered":"2024-09-17T21:21:09.305831+0000","last_clean":"2024-09-17T21:21:09.305831+0000","last_became_active":"2024-09-17T21:18:01.048806+0000","last_became_peered":"2024-09-17T21:18:01.048806+0000","last_unstale":"2024-09-17T21:21:09.305831+0000","last_undegraded":"2024-09-17T21:21:09.305831+0000","last_fullsized":"2024-09-17T21:21:09.305831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:26.487163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094161400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"154'286","reported_seq":4634,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.347506+0000","last_change":"2024-09-17T21:21:09.312014+0000","last_active":"2024-09-17T21:21:09.347506+0000","last_peered":"2024-09-17T21:21:09.347506+0000","last_clean":"2024-09-17T21:21:09.347506+0000","last_became_active":"2024-09-17T21:21:09.311210+0000","last_became_peered":"2024-09-17T21:21:09.311210+0000","last_unstale":"2024-09-17T21:21:09.347506+0000","last_undegraded":"2024-09-17T21:21:09.347506+0000","last_fullsized":"2024-09-17T21:21:09.347506+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:47:42.951453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40553128,"num_objects":46,"num_object_clones":9,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":17,"num_read":4254,"num_read_kb":34875,"num_write":342,"num_write_kb":13832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":3382007,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":299,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.307266+0000","last_change":"2024-09-17T21:19:17.294067+0000","last_active":"2024-09-17T21:21:09.307266+0000","last_peered":"2024-09-17T21:21:09.307266+0000","last_clean":"2024-09-17T21:21:09.307266+0000","last_became_active":"2024-09-17T21:19:17.293357+0000","last_became_peered":"2024-09-17T21:19:17.293357+0000","last_unstale":"2024-09-17T21:21:09.307266+0000","last_undegraded":"2024-09-17T21:21:09.307266+0000","last_fullsized":"2024-09-17T21:21:09.307266+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:13:10.070543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099219900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"142'536","reported_seq":13886,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.305550+0000","last_change":"2024-09-17T21:21:07.306629+0000","last_active":"2024-09-17T21:21:09.305550+0000","last_peered":"2024-09-17T21:21:09.305550+0000","last_clean":"2024-09-17T21:21:09.305550+0000","last_became_active":"2024-09-17T21:21:07.304291+0000","last_became_peered":"2024-09-17T21:21:07.304291+0000","last_unstale":"2024-09-17T21:21:09.305550+0000","last_undegraded":"2024-09-17T21:21:09.305550+0000","last_fullsized":"2024-09-17T21:21:09.305550+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:26:14.542027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078465700000000002,"stat_sum":{"num_bytes":46015614,"num_objects":67,"num_object_clones":18,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":24,"num_read":11581,"num_read_kb":27630,"num_write":496,"num_write_kb":10682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":13838498,"num_keys_recovered":71,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":299,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.305909+0000","last_change":"2024-09-17T21:19:17.294907+0000","last_active":"2024-09-17T21:21:09.305909+0000","last_peered":"2024-09-17T21:21:09.305909+0000","last_clean":"2024-09-17T21:21:09.305909+0000","last_became_active":"2024-09-17T21:19:17.293525+0000","last_became_peered":"2024-09-17T21:19:17.293525+0000","last_unstale":"2024-09-17T21:21:09.305909+0000","last_undegraded":"2024-09-17T21:21:09.305909+0000","last_fullsized":"2024-09-17T21:21:09.305909+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:32:46.627308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006222,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"146'891","reported_seq":32351,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.311744+0000","last_change":"2024-09-17T21:21:08.322784+0000","last_active":"2024-09-17T21:21:09.311744+0000","last_peered":"2024-09-17T21:21:09.311744+0000","last_clean":"2024-09-17T21:21:09.311744+0000","last_became_active":"2024-09-17T21:21:08.320190+0000","last_became_peered":"2024-09-17T21:21:08.320190+0000","last_unstale":"2024-09-17T21:21:09.311744+0000","last_undegraded":"2024-09-17T21:21:09.311744+0000","last_fullsized":"2024-09-17T21:21:09.311744+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:03:40.335268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036493720000000001,"stat_sum":{"num_bytes":107471485,"num_objects":80,"num_object_clones":26,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":21,"num_read":36275,"num_read_kb":60675,"num_write":772,"num_write_kb":21820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":385,"num_bytes_recovered":37210446,"num_keys_recovered":136,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"146'272","reported_seq":2470,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.314097+0000","last_change":"2024-09-17T21:19:17.416920+0000","last_active":"2024-09-17T21:21:09.314097+0000","last_peered":"2024-09-17T21:21:09.314097+0000","last_clean":"2024-09-17T21:21:09.314097+0000","last_became_active":"2024-09-17T21:19:17.294385+0000","last_became_peered":"2024-09-17T21:19:17.294385+0000","last_unstale":"2024-09-17T21:21:09.314097+0000","last_undegraded":"2024-09-17T21:21:09.314097+0000","last_fullsized":"2024-09-17T21:21:09.314097+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:25:50.365916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084988700000000002,"stat_sum":{"num_bytes":22119602,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":17,"num_read":2428,"num_read_kb":12724,"num_write":274,"num_write_kb":4669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":2000896,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":229,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.305298+0000","last_change":"2024-09-17T21:20:33.408746+0000","last_active":"2024-09-17T21:21:09.305298+0000","last_peered":"2024-09-17T21:21:09.305298+0000","last_clean":"2024-09-17T21:21:09.305298+0000","last_became_active":"2024-09-17T21:20:33.401029+0000","last_became_peered":"2024-09-17T21:20:33.401029+0000","last_unstale":"2024-09-17T21:21:09.305298+0000","last_undegraded":"2024-09-17T21:21:09.305298+0000","last_fullsized":"2024-09-17T21:21:09.305298+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:23:16.392612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"146'188","reported_seq":7533,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.316789+0000","last_change":"2024-09-17T21:21:09.312194+0000","last_active":"2024-09-17T21:21:09.316789+0000","last_peered":"2024-09-17T21:21:09.316789+0000","last_clean":"2024-09-17T21:21:09.316789+0000","last_became_active":"2024-09-17T21:21:09.310418+0000","last_became_peered":"2024-09-17T21:21:09.310418+0000","last_unstale":"2024-09-17T21:21:09.316789+0000","last_undegraded":"2024-09-17T21:21:09.316789+0000","last_fullsized":"2024-09-17T21:21:09.316789+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:53.055102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049125000000000004,"stat_sum":{"num_bytes":53067355,"num_objects":56,"num_object_clones":19,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":15,"num_read":4271,"num_read_kb":21541,"num_write":193,"num_write_kb":6424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":11145883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":277,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.306550+0000","last_change":"2024-09-17T21:21:04.681469+0000","last_active":"2024-09-17T21:21:09.306550+0000","last_peered":"2024-09-17T21:21:09.306550+0000","last_clean":"2024-09-17T21:21:09.306550+0000","last_became_active":"2024-09-17T21:21:04.680896+0000","last_became_peered":"2024-09-17T21:21:04.680896+0000","last_unstale":"2024-09-17T21:21:09.306550+0000","last_undegraded":"2024-09-17T21:21:09.306550+0000","last_fullsized":"2024-09-17T21:21:09.306550+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:17:30.091194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"152'932","reported_seq":5745,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.312886+0000","last_change":"2024-09-17T21:21:09.312807+0000","last_active":"2024-09-17T21:21:09.312886+0000","last_peered":"2024-09-17T21:21:09.312886+0000","last_clean":"2024-09-17T21:21:09.312886+0000","last_became_active":"2024-09-17T21:21:09.311397+0000","last_became_peered":"2024-09-17T21:21:09.311397+0000","last_unstale":"2024-09-17T21:21:09.312886+0000","last_undegraded":"2024-09-17T21:21:09.312886+0000","last_fullsized":"2024-09-17T21:21:09.312886+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":932,"log_dups_size":0,"ondisk_log_size":932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:14:41.154904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066398399999999995,"stat_sum":{"num_bytes":30031040,"num_objects":59,"num_object_clones":16,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":14,"num_read":7803,"num_read_kb":48357,"num_write":1498,"num_write_kb":18080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":25989698,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":287,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.309397+0000","last_change":"2024-09-17T21:18:03.059156+0000","last_active":"2024-09-17T21:21:09.309397+0000","last_peered":"2024-09-17T21:21:09.309397+0000","last_clean":"2024-09-17T21:21:09.309397+0000","last_became_active":"2024-09-17T21:18:01.049180+0000","last_became_peered":"2024-09-17T21:18:01.049180+0000","last_unstale":"2024-09-17T21:21:09.309397+0000","last_undegraded":"2024-09-17T21:21:09.309397+0000","last_fullsized":"2024-09-17T21:21:09.309397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:27.069794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077408,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"152'543","reported_seq":12850,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.305145+0000","last_change":"2024-09-17T21:20:37.308643+0000","last_active":"2024-09-17T21:21:09.305145+0000","last_peered":"2024-09-17T21:21:09.305145+0000","last_clean":"2024-09-17T21:21:09.305145+0000","last_became_active":"2024-09-17T21:20:37.307411+0000","last_became_peered":"2024-09-17T21:20:37.307411+0000","last_unstale":"2024-09-17T21:21:09.305145+0000","last_undegraded":"2024-09-17T21:21:09.305145+0000","last_fullsized":"2024-09-17T21:21:09.305145+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:35:42.794499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049408099999999995,"stat_sum":{"num_bytes":72616869,"num_objects":56,"num_object_clones":16,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":23,"num_read":13769,"num_read_kb":30495,"num_write":675,"num_write_kb":20120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":158,"num_bytes_recovered":18131698,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":33,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-17T21:21:13.338683+0000","last_change":"2024-09-17T21:21:04.685863+0000","last_active":"2024-09-17T21:21:13.338683+0000","last_peered":"2024-09-17T21:21:13.338683+0000","last_clean":"2024-09-17T21:21:13.338683+0000","last_became_active":"2024-09-17T21:21:04.679420+0000","last_became_peered":"2024-09-17T21:21:04.679420+0000","last_unstale":"2024-09-17T21:21:13.338683+0000","last_undegraded":"2024-09-17T21:21:13.338683+0000","last_fullsized":"2024-09-17T21:21:13.338683+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:26:11.829570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":402,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.309637+0000","last_change":"2024-09-17T21:19:17.427717+0000","last_active":"2024-09-17T21:21:09.309637+0000","last_peered":"2024-09-17T21:21:09.309637+0000","last_clean":"2024-09-17T21:21:09.309637+0000","last_became_active":"2024-09-17T21:19:17.297187+0000","last_became_peered":"2024-09-17T21:19:17.297187+0000","last_unstale":"2024-09-17T21:21:09.309637+0000","last_undegraded":"2024-09-17T21:21:09.309637+0000","last_fullsized":"2024-09-17T21:21:09.309637+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_clean_scrub_stamp":"2024-09-17T21:17:55.982339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:19:07.636190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"152'495","reported_seq":11416,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-09-17T21:21:09.307862+0000","last_change":"2024-09-17T21:21:09.307525+0000","last_active":"2024-09-17T21:21:09.307525+0000","last_peered":"2024-09-17T21:18:10.267877+0000","last_clean":"2024-09-17T21:18:10.267877+0000","last_became_active":"2024-09-17T21:21:05.713362+0000","last_became_peered":"2024-09-17T21:21:05.713362+0000","last_unstale":"2024-09-17T21:21:09.307862+0000","last_undegraded":"2024-09-17T21:21:09.307862+0000","last_fullsized":"2024-09-17T21:21:09.307862+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71689301,"num_objects":70,"num_object_clones":21,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":23,"num_read":12065,"num_read_kb":61737,"num_write":516,"num_write_kb":29098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":19296952,"num_keys_recovered":88,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"142'603","reported_seq":10113,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-17T21:21:13.338146+0000","last_change":"2024-09-17T21:21:10.321108+0000","last_active":"2024-09-17T21:21:13.338146+0000","last_peered":"2024-09-17T21:21:13.338146+0000","last_clean":"2024-09-17T21:21:13.338146+0000","last_became_active":"2024-09-17T21:21:10.318396+0000","last_became_peered":"2024-09-17T21:21:10.318396+0000","last_unstale":"2024-09-17T21:21:13.338146+0000","last_undegraded":"2024-09-17T21:21:13.338146+0000","last_fullsized":"2024-09-17T21:21:13.338146+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:32:46.001334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023749890000000001,"stat_sum":{"num_bytes":56035672,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":15,"num_read":11359,"num_read_kb":43358,"num_write":1019,"num_write_kb":7250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":20486637,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"154'308","reported_seq":6643,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.367142+0000","last_change":"2024-09-17T21:19:34.613370+0000","last_active":"2024-09-17T21:21:09.367142+0000","last_peered":"2024-09-17T21:21:09.367142+0000","last_clean":"2024-09-17T21:21:09.367142+0000","last_became_active":"2024-09-17T21:19:34.611967+0000","last_became_peered":"2024-09-17T21:19:34.611967+0000","last_unstale":"2024-09-17T21:21:09.367142+0000","last_undegraded":"2024-09-17T21:21:09.367142+0000","last_fullsized":"2024-09-17T21:21:09.367142+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:04:12.231084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021951829999999999,"stat_sum":{"num_bytes":64961170,"num_objects":58,"num_object_clones":17,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":22,"num_read":6144,"num_read_kb":39412,"num_write":286,"num_write_kb":16635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"145'293","reported_seq":3735,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.305839+0000","last_change":"2024-09-17T21:19:20.292113+0000","last_active":"2024-09-17T21:21:09.305839+0000","last_peered":"2024-09-17T21:21:09.305839+0000","last_clean":"2024-09-17T21:21:09.305839+0000","last_became_active":"2024-09-17T21:19:20.290717+0000","last_became_peered":"2024-09-17T21:19:20.290717+0000","last_unstale":"2024-09-17T21:21:09.305839+0000","last_undegraded":"2024-09-17T21:21:09.305839+0000","last_fullsized":"2024-09-17T21:21:09.305839+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:45:34.450255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048704800000000002,"stat_sum":{"num_bytes":48478950,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":4132,"num_read_kb":23000,"num_write":267,"num_write_kb":13753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":1215535,"num_keys_recovered":36,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"152'333","reported_seq":10133,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.305407+0000","last_change":"2024-09-17T21:20:39.301546+0000","last_active":"2024-09-17T21:21:09.305407+0000","last_peered":"2024-09-17T21:21:09.305407+0000","last_clean":"2024-09-17T21:21:09.305407+0000","last_became_active":"2024-09-17T21:20:39.300478+0000","last_became_peered":"2024-09-17T21:20:39.300478+0000","last_unstale":"2024-09-17T21:21:09.305407+0000","last_undegraded":"2024-09-17T21:21:09.305407+0000","last_fullsized":"2024-09-17T21:21:09.305407+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:03:00.761983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015319789999999999,"stat_sum":{"num_bytes":49816616,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":22,"num_read":12205,"num_read_kb":22394,"num_write":499,"num_write_kb":14245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":3255808,"num_keys_recovered":72,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"146'425","reported_seq":13852,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.307015+0000","last_change":"2024-09-17T21:21:07.303861+0000","last_active":"2024-09-17T21:21:09.307015+0000","last_peered":"2024-09-17T21:21:09.307015+0000","last_clean":"2024-09-17T21:21:09.307015+0000","last_became_active":"2024-09-17T21:21:07.303122+0000","last_became_peered":"2024-09-17T21:21:07.303122+0000","last_unstale":"2024-09-17T21:21:09.307015+0000","last_undegraded":"2024-09-17T21:21:09.307015+0000","last_fullsized":"2024-09-17T21:21:09.307015+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:04:27.639087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015238949999999999,"stat_sum":{"num_bytes":39850633,"num_objects":57,"num_object_clones":19,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":16,"num_read":14714,"num_read_kb":41407,"num_write":597,"num_write_kb":16372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":14464679,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"153'492","reported_seq":13043,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.305718+0000","last_change":"2024-09-17T21:20:39.301233+0000","last_active":"2024-09-17T21:21:09.305718+0000","last_peered":"2024-09-17T21:21:09.305718+0000","last_clean":"2024-09-17T21:21:09.305718+0000","last_became_active":"2024-09-17T21:20:39.300521+0000","last_became_peered":"2024-09-17T21:20:39.300521+0000","last_unstale":"2024-09-17T21:21:09.305718+0000","last_undegraded":"2024-09-17T21:21:09.305718+0000","last_fullsized":"2024-09-17T21:21:09.305718+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:02:45.147783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003632209,"stat_sum":{"num_bytes":62563553,"num_objects":62,"num_object_clones":18,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":17,"num_read":14172,"num_read_kb":68575,"num_write":458,"num_write_kb":35777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":66488694,"num_keys_recovered":44,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"152'340","reported_seq":5554,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.307529+0000","last_change":"2024-09-17T21:20:37.309394+0000","last_active":"2024-09-17T21:21:09.307529+0000","last_peered":"2024-09-17T21:21:09.307529+0000","last_clean":"2024-09-17T21:21:09.307529+0000","last_became_active":"2024-09-17T21:20:37.307385+0000","last_became_peered":"2024-09-17T21:20:37.307385+0000","last_unstale":"2024-09-17T21:21:09.307529+0000","last_undegraded":"2024-09-17T21:21:09.307529+0000","last_fullsized":"2024-09-17T21:21:09.307529+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:26:30.777210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002454758,"stat_sum":{"num_bytes":42973317,"num_objects":61,"num_object_clones":17,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":22,"num_read":6226,"num_read_kb":25820,"num_write":360,"num_write_kb":4171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":3102970,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"152'210","reported_seq":5242,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.309523+0000","last_change":"2024-09-17T21:20:36.441857+0000","last_active":"2024-09-17T21:21:09.309523+0000","last_peered":"2024-09-17T21:21:09.309523+0000","last_clean":"2024-09-17T21:21:09.309523+0000","last_became_active":"2024-09-17T21:20:36.440601+0000","last_became_peered":"2024-09-17T21:20:36.440601+0000","last_unstale":"2024-09-17T21:21:09.309523+0000","last_undegraded":"2024-09-17T21:21:09.309523+0000","last_fullsized":"2024-09-17T21:21:09.309523+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:30:27.019368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001519261,"stat_sum":{"num_bytes":24747157,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":13,"num_read":6157,"num_read_kb":16278,"num_write":177,"num_write_kb":3252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":3300798,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"146'326","reported_seq":5250,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.306693+0000","last_change":"2024-09-17T21:18:13.305680+0000","last_active":"2024-09-17T21:21:09.306693+0000","last_peered":"2024-09-17T21:21:09.306693+0000","last_clean":"2024-09-17T21:21:09.306693+0000","last_became_active":"2024-09-17T21:18:11.299946+0000","last_became_peered":"2024-09-17T21:18:11.299946+0000","last_unstale":"2024-09-17T21:21:09.306693+0000","last_undegraded":"2024-09-17T21:21:09.306693+0000","last_fullsized":"2024-09-17T21:21:09.306693+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:48:36.321766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225884,"stat_sum":{"num_bytes":57364908,"num_objects":53,"num_object_clones":15,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":16,"num_read":5051,"num_read_kb":21256,"num_write":332,"num_write_kb":4437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"154'400","reported_seq":12830,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.363702+0000","last_change":"2024-09-17T21:20:53.517521+0000","last_active":"2024-09-17T21:21:09.363702+0000","last_peered":"2024-09-17T21:21:09.363702+0000","last_clean":"2024-09-17T21:21:09.363702+0000","last_became_active":"2024-09-17T21:20:53.515536+0000","last_became_peered":"2024-09-17T21:20:53.515536+0000","last_unstale":"2024-09-17T21:21:09.363702+0000","last_undegraded":"2024-09-17T21:21:09.363702+0000","last_fullsized":"2024-09-17T21:21:09.363702+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:30:27.609105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013045489999999999,"stat_sum":{"num_bytes":77827274,"num_objects":70,"num_object_clones":21,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":25,"num_read":15108,"num_read_kb":35446,"num_write":487,"num_write_kb":6896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":2706590,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"150'546","reported_seq":10986,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:21:09.306707+0000","last_change":"2024-09-17T21:20:38.299786+0000","last_active":"2024-09-17T21:21:09.306707+0000","last_peered":"2024-09-17T21:21:09.306707+0000","last_clean":"2024-09-17T21:21:09.306707+0000","last_became_active":"2024-09-17T21:20:38.298489+0000","last_became_peered":"2024-09-17T21:20:38.298489+0000","last_unstale":"2024-09-17T21:21:09.306707+0000","last_undegraded":"2024-09-17T21:21:09.306707+0000","last_fullsized":"2024-09-17T21:21:09.306707+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:52:21.570250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019790290000000002,"stat_sum":{"num_bytes":62189176,"num_objects":65,"num_object_clones":18,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":26,"num_read":11433,"num_read_kb":49879,"num_write":556,"num_write_kb":5413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":3427570,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1562402512,"num_objects":1801,"num_object_clones":495,"num_object_copies":3602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1801,"num_whiteouts":606,"num_read":318313,"num_read_kb":1065914,"num_write":17159,"num_write_kb":381536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4062,"num_bytes_recovered":306899401,"num_keys_recovered":804,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":447971328,"data_stored":678115234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14580,"ondisk_log_size":14580,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":165524,"kb_used_data":107968,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94206316,"statfs":{"total":96636764160,"available":96467267584,"internally_reserved":0,"allocated":110559232,"data_stored":156297229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":58915268},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574890,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140488,"kb_used_data":64884,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94231352,"statfs":{"total":96636764160,"available":96492904448,"internally_reserved":0,"allocated":66441216,"data_stored":104506150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":77396420},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574890,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":120280,"kb_used_data":62720,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94251560,"statfs":{"total":96636764160,"available":96513597440,"internally_reserved":0,"allocated":64225280,"data_stored":91536346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136784,"kb_used_data":101432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94235056,"statfs":{"total":96636764160,"available":96496697344,"internally_reserved":0,"allocated":103866368,"data_stored":160097780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":116744,"kb_used_data":67508,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94255096,"statfs":{"total":96636764160,"available":96517218304,"internally_reserved":0,"allocated":69128192,"data_stored":103092802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":50395588},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607596,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":113540,"kb_used_data":64304,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94258300,"statfs":{"total":96636764160,"available":96520499200,"internally_reserved":0,"allocated":65847296,"data_stored":93729123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":50395588},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66703360,"data_stored":101745762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":100159488,"data_stored":156923601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":63979520,"data_stored":92957255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61800448,"data_stored":90189306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":62140416,"data_stored":100566209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":93188096,"data_stored":135733101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:21:15.198 INFO:tasks.thrashosds.thrasher:Setting 1.0 to [4, 3] 2024-09-17T21:21:15.198 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '1.0', '4', '3'] 2024-09-17T21:21:15.198 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 1.0 4 3 2024-09-17T21:21:15.226 INFO:teuthology.orchestra.run.smithi184.stdout:5912 read 0x2e2800 thru 0x2e982c (0x702d bytes) 2024-09-17T21:21:15.230 INFO:teuthology.orchestra.run.smithi184.stdout:5915 punch from 0x38de00 to 0x393000, (0x5200 bytes) 2024-09-17T21:21:15.270 INFO:teuthology.orchestra.run.smithi184.stdout:5917 write 0x5253400 thru 0x52557bc (0x23bd bytes) 2024-09-17T21:21:15.272 INFO:teuthology.orchestra.run.smithi184.stdout:5920 punch from 0x609c00 to 0x611800, (0x7c00 bytes) 2024-09-17T21:21:15.278 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:15.273+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:15.280 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:15.280 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:15.295 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:21:15.314 INFO:teuthology.orchestra.run.smithi184.stdout:5927 write 0xd24e800 thru 0xd25aa9a (0xc29b bytes) 2024-09-17T21:21:15.315 INFO:teuthology.orchestra.run.smithi184.stdout:5929 read 0x2ca600 thru 0x2d8dbb (0xe7bc bytes) 2024-09-17T21:21:15.316 INFO:teuthology.orchestra.run.smithi184.stdout:5931 read 0x4bff000 thru 0x4c01e5d (0x2e5e bytes) 2024-09-17T21:21:15.317 INFO:teuthology.orchestra.run.smithi184.stdout:5933 clone 43 order 20 su 262144 sc 4 2024-09-17T21:21:15.320 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-17T21:21:15.379 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:15.373+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:15.443 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:15.443 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:15.457 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:21:15.501 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:15.499+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:15.572 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:15.572 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:15.580 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:15.575+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:15.586 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:21:15.681 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:15.677+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:15.700 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:21:15.755 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:15.755 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:15.771 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:21:15.782 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:15.777+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:15.844 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:15.844 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:15.861 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:15.861 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:15.880 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:21:15.880 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:21:15.882 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:15.879+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:15.891 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:15.891 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:15.891 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:21:15.891 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:15.900 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:15.900 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:15.905 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:21:15.922 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:15.922 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:15.942 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:15.943 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:15.984 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:15.981+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:16.041 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:16.041 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:16.041 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:21:16.041 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:21:16.041 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:16.054 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:21:16.101 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:16.099+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:16.178 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:16.178 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:16.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:16.178 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:16.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.178 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:21:16.178 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:21:16.178 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 177.95719231499999, 2024-09-17T21:21:16.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:21:16.179 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:16.179 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:16.179 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:16.179 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:16.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:16.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:21:16.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:16.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:21:16.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:16.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:21:16.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:21:16.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:16.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:21:16.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:21:16.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:16.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:21:16.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:21:16.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:21:16.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:21:16.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:16.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:21:16.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:21:16.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.182 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:16.182 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:21:16.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:21:16.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.182 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:16.182 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:21:16.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:21:16.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.182 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:16.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:21:16.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:21:16.183 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.183 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:16.183 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.183 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:21:16.183 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:16.183 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 173.306891651, 2024-09-17T21:21:16.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:21:16.183 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:16.183 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:16.183 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:16.184 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:16.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:16.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:16.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:16.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:16.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:16.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:21:16.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:21:16.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:16.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:21:16.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:21:16.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:16.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:21:16.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:21:16.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:21:16.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:21:16.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:16.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:21:16.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:21:16.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:16.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:21:16.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:21:16.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:16.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:21:16.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:21:16.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:16.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:21:16.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:21:16.187 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.187 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:16.187 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.188 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:21:16.188 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:16.188 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 173.30562656800001, 2024-09-17T21:21:16.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:21:16.188 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:16.188 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:16.188 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:16.188 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:16.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:16.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:16.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:16.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:16.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:16.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:21:16.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:21:16.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.190 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:16.190 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:21:16.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:21:16.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.190 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:16.190 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:21:16.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:21:16.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.190 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.190 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:21:16.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:21:16.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:16.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:21:16.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:21:16.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:16.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:21:16.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:21:16.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:16.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:21:16.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:21:16.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:16.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:21:16.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:21:16.192 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.192 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:16.192 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.192 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:21:16.193 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:16.193 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 173.29584715799999, 2024-09-17T21:21:16.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:21:16.193 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:16.193 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:16.193 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:16.193 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:16.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:16.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:16.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:16.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:16.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:16.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:21:16.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:21:16.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:16.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:21:16.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:21:16.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:16.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:21:16.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:21:16.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:21:16.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:21:16.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:16.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:21:16.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:21:16.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:16.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:21:16.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:21:16.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:16.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:21:16.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:21:16.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:16.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:21:16.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:21:16.196 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.197 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:16.197 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.197 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:21:16.197 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:16.197 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 173.28712475500001, 2024-09-17T21:21:16.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:21:16.197 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:16.197 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:16.197 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:16.197 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:16.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:16.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:16.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:16.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:16.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:16.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:21:16.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:21:16.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:16.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:21:16.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:21:16.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:16.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:21:16.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:21:16.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:21:16.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:21:16.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:16.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:21:16.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:21:16.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:16.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:21:16.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:21:16.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:16.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:21:16.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:21:16.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:16.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:21:16.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:21:16.201 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.201 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:16.201 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.201 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:21:16.201 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:16.201 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 159.90203419100001, 2024-09-17T21:21:16.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:21:16.202 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:16.202 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:16.202 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:16.202 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:16.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:16.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:16.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:16.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:16.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:16.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:21:16.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:21:16.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:16.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:21:16.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:21:16.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:16.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:21:16.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:21:16.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:21:16.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:21:16.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:16.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:21:16.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:21:16.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:16.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:21:16.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:21:16.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:16.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:21:16.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:21:16.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:16.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:21:16.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:21:16.205 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.205 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:16.205 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.206 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1780 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'195, mlcod=69'190)", 2024-09-17T21:21:16.206 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:16.206 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 122.924178587, 2024-09-17T21:21:16.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075761769999999999, 2024-09-17T21:21:16.206 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:16.206 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:16.206 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:16.206 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:16.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:16.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:16.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:16.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:16.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:16.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250375+0000", 2024-09-17T21:21:16.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:21:16.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:16.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250383+0000", 2024-09-17T21:21:16.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7870000000000002e-06 2024-09-17T21:21:16.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:16.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250402+0000", 2024-09-17T21:21:16.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9346000000000001e-05 2024-09-17T21:21:16.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250413+0000", 2024-09-17T21:21:16.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.039e-05 2024-09-17T21:21:16.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:16.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251353+0000", 2024-09-17T21:21:16.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000940091 2024-09-17T21:21:16.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:16.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251407+0000", 2024-09-17T21:21:16.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4701999999999999e-05 2024-09-17T21:21:16.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:16.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257829+0000", 2024-09-17T21:21:16.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064212779999999999 2024-09-17T21:21:16.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:16.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257950+0000", 2024-09-17T21:21:16.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121623 2024-09-17T21:21:16.210 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.210 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:16.210 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.210 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1781 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'196, mlcod=69'190)", 2024-09-17T21:21:16.210 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:21:16.210 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 122.923095636, 2024-09-17T21:21:16.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065632939999999999, 2024-09-17T21:21:16.210 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:16.211 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:16.211 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:16.211 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:16.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:16.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:21:16.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:16.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:21:16.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:16.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251459+0000", 2024-09-17T21:21:16.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.624e-06 2024-09-17T21:21:16.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:16.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251471+0000", 2024-09-17T21:21:16.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2636000000000001e-05 2024-09-17T21:21:16.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:16.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251498+0000", 2024-09-17T21:21:16.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:21:16.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251515+0000", 2024-09-17T21:21:16.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6939000000000002e-05 2024-09-17T21:21:16.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:16.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252938+0000", 2024-09-17T21:21:16.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001423082 2024-09-17T21:21:16.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:16.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252999+0000", 2024-09-17T21:21:16.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1138999999999999e-05 2024-09-17T21:21:16.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:16.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257955+0000", 2024-09-17T21:21:16.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049559640000000002 2024-09-17T21:21:16.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:16.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.258020+0000", 2024-09-17T21:21:16.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5504000000000002e-05 2024-09-17T21:21:16.214 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.214 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:16.215 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.215 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1788 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'198, mlcod=69'191)", 2024-09-17T21:21:16.215 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:16.215 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 122.92107014699999, 2024-09-17T21:21:16.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073690370000000002, 2024-09-17T21:21:16.215 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:16.215 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:16.215 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:16.215 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:16.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:16.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:16.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:16.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:16.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:16.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253484+0000", 2024-09-17T21:21:16.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2470000000000001e-06 2024-09-17T21:21:16.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:16.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253492+0000", 2024-09-17T21:21:16.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7159999999999992e-06 2024-09-17T21:21:16.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:16.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253509+0000", 2024-09-17T21:21:16.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.666e-05 2024-09-17T21:21:16.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253522+0000", 2024-09-17T21:21:16.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2643000000000001e-05 2024-09-17T21:21:16.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:16.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254853+0000", 2024-09-17T21:21:16.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331238 2024-09-17T21:21:16.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:16.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254885+0000", 2024-09-17T21:21:16.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1562e-05 2024-09-17T21:21:16.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:16.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260788+0000", 2024-09-17T21:21:16.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059030749999999998 2024-09-17T21:21:16.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:16.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260852+0000", 2024-09-17T21:21:16.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3895999999999999e-05 2024-09-17T21:21:16.219 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.219 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:16.219 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.219 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5746 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'477, mlcod=71'473)", 2024-09-17T21:21:16.219 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:16.219 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 119.915642589, 2024-09-17T21:21:16.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074182709999999997, 2024-09-17T21:21:16.220 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:16.220 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:16.220 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:16.220 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:16.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:16.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:16.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:16.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:16.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:16.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258912+0000", 2024-09-17T21:21:16.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:21:16.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:16.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258926+0000", 2024-09-17T21:21:16.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4249e-05 2024-09-17T21:21:16.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:16.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258946+0000", 2024-09-17T21:21:16.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9879999999999999e-05 2024-09-17T21:21:16.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258956+0000", 2024-09-17T21:21:16.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:21:16.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:16.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259380+0000", 2024-09-17T21:21:16.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042337599999999999 2024-09-17T21:21:16.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:16.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259412+0000", 2024-09-17T21:21:16.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:21:16.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:16.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266191+0000", 2024-09-17T21:21:16.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067793269999999999 2024-09-17T21:21:16.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:16.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266328+0000", 2024-09-17T21:21:16.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013690999999999999 2024-09-17T21:21:16.224 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.224 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:16.224 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.224 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5747 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'478, mlcod=71'473)", 2024-09-17T21:21:16.224 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:16.224 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 119.91493624, 2024-09-17T21:21:16.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068779440000000004, 2024-09-17T21:21:16.224 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:16.224 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:16.224 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:16.224 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:16.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:16.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:16.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:16.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:16.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:16.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259618+0000", 2024-09-17T21:21:16.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6199999999999999e-06 2024-09-17T21:21:16.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:16.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259630+0000", 2024-09-17T21:21:16.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1884e-05 2024-09-17T21:21:16.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:16.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259657+0000", 2024-09-17T21:21:16.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7339e-05 2024-09-17T21:21:16.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259677+0000", 2024-09-17T21:21:16.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0156000000000001e-05 2024-09-17T21:21:16.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:16.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260521+0000", 2024-09-17T21:21:16.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084312899999999997 2024-09-17T21:21:16.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:16.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260585+0000", 2024-09-17T21:21:16.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4558000000000001e-05 2024-09-17T21:21:16.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:16.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266336+0000", 2024-09-17T21:21:16.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057505020000000002 2024-09-17T21:21:16.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:16.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266494+0000", 2024-09-17T21:21:16.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158756 2024-09-17T21:21:16.228 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.228 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:16.228 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.229 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5751 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'480, mlcod=71'473)", 2024-09-17T21:21:16.229 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:16.229 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 119.913516835, 2024-09-17T21:21:16.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075756299999999999, 2024-09-17T21:21:16.229 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:16.229 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:16.229 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:16.229 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:16.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:16.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:16.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:16.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:16.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:16.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261037+0000", 2024-09-17T21:21:16.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:21:16.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:16.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261049+0000", 2024-09-17T21:21:16.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1063999999999999e-05 2024-09-17T21:21:16.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:16.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261072+0000", 2024-09-17T21:21:16.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3102e-05 2024-09-17T21:21:16.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261133+0000", 2024-09-17T21:21:16.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1276000000000001e-05 2024-09-17T21:21:16.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:16.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:21:16.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015334319999999999 2024-09-17T21:21:16.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:16.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262702+0000", 2024-09-17T21:21:16.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5281999999999999e-05 2024-09-17T21:21:16.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:16.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268527+0000", 2024-09-17T21:21:16.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058250389999999997 2024-09-17T21:21:16.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:16.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268611+0000", 2024-09-17T21:21:16.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4724999999999999e-05 2024-09-17T21:21:16.233 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.233 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:16.233 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.233 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5756 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'481, mlcod=71'473)", 2024-09-17T21:21:16.233 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:16.233 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 119.91265597899999, 2024-09-17T21:21:16.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006772386, 2024-09-17T21:21:16.234 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:16.234 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:16.234 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:16.234 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:16.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:16.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:16.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:16.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:16.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:16.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261899+0000", 2024-09-17T21:21:16.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.903e-06 2024-09-17T21:21:16.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:16.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261912+0000", 2024-09-17T21:21:16.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-05 2024-09-17T21:21:16.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:16.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261938+0000", 2024-09-17T21:21:16.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6098999999999999e-05 2024-09-17T21:21:16.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261955+0000", 2024-09-17T21:21:16.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.696e-05 2024-09-17T21:21:16.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:16.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263845+0000", 2024-09-17T21:21:16.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001890187 2024-09-17T21:21:16.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:16.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263907+0000", 2024-09-17T21:21:16.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1600999999999995e-05 2024-09-17T21:21:16.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:16.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268615+0000", 2024-09-17T21:21:16.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047086070000000001 2024-09-17T21:21:16.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:16.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268669+0000", 2024-09-17T21:21:16.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3588999999999998e-05 2024-09-17T21:21:16.237 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.237 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:16.238 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.238 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4978.0:598 3.9 e80/23 3:9701cc17:::rbd_data.1152724ff2a3.0000000000000019:head v 80'202, mlcod=80'198)", 2024-09-17T21:21:16.238 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:21:16.238 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 109.411081596, 2024-09-17T21:21:16.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066473340000000004, 2024-09-17T21:21:16.238 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:16.238 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:16.238 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:16.238 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:16.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:16.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:21:16.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:16.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:21:16.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:16.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763472+0000", 2024-09-17T21:21:16.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6899999999999997e-07 2024-09-17T21:21:16.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:16.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763478+0000", 2024-09-17T21:21:16.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:21:16.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:16.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763489+0000", 2024-09-17T21:21:16.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0628999999999999e-05 2024-09-17T21:21:16.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763495+0000", 2024-09-17T21:21:16.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6649999999999997e-06 2024-09-17T21:21:16.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:16.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763907+0000", 2024-09-17T21:21:16.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041163899999999999 2024-09-17T21:21:16.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:16.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763976+0000", 2024-09-17T21:21:16.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8534999999999993e-05 2024-09-17T21:21:16.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:16.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.769900+0000", 2024-09-17T21:21:16.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059243270000000001 2024-09-17T21:21:16.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:16.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.770118+0000", 2024-09-17T21:21:16.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021842500000000001 2024-09-17T21:21:16.242 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.242 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:16.242 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.242 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:468 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'94, mlcod=89'87)", 2024-09-17T21:21:16.242 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:16.242 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 96.488336297000004, 2024-09-17T21:21:16.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066528790000000004, 2024-09-17T21:21:16.243 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:16.243 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:16.243 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:16.243 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:16.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:16.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:16.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:16.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:16.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.244 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:16.244 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686218+0000", 2024-09-17T21:21:16.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3060000000000001e-06 2024-09-17T21:21:16.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.244 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:16.244 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686256+0000", 2024-09-17T21:21:16.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8333e-05 2024-09-17T21:21:16.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.244 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:16.244 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686272+0000", 2024-09-17T21:21:16.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:21:16.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686283+0000", 2024-09-17T21:21:16.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0903e-05 2024-09-17T21:21:16.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:16.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686682+0000", 2024-09-17T21:21:16.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039935099999999999 2024-09-17T21:21:16.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:16.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686785+0000", 2024-09-17T21:21:16.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010275199999999999 2024-09-17T21:21:16.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:16.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692816+0000", 2024-09-17T21:21:16.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060317579999999999 2024-09-17T21:21:16.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:16.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692869+0000", 2024-09-17T21:21:16.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2775999999999998e-05 2024-09-17T21:21:16.247 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.247 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:16.247 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.247 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:755 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'127, mlcod=89'122)", 2024-09-17T21:21:16.247 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:16.247 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 96.351881082999995, 2024-09-17T21:21:16.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084014080000000008, 2024-09-17T21:21:16.247 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:16.247 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:16.247 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:16.248 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:16.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:16.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:16.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:16.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:16.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:16.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:16.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:21:16.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:16.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822683+0000", 2024-09-17T21:21:16.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0674e-05 2024-09-17T21:21:16.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:16.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822698+0000", 2024-09-17T21:21:16.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5189e-05 2024-09-17T21:21:16.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822708+0000", 2024-09-17T21:21:16.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9119999999999993e-06 2024-09-17T21:21:16.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:16.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822748+0000", 2024-09-17T21:21:16.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9727000000000001e-05 2024-09-17T21:21:16.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:16.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822759+0000", 2024-09-17T21:21:16.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0865e-05 2024-09-17T21:21:16.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:16.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831034+0000", 2024-09-17T21:21:16.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082750710000000002 2024-09-17T21:21:16.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:16.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831073+0000", 2024-09-17T21:21:16.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9063999999999997e-05 2024-09-17T21:21:16.251 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.251 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:16.251 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.252 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:756 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'128, mlcod=89'124)", 2024-09-17T21:21:16.252 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:16.252 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 96.349958451000006, 2024-09-17T21:21:16.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0076300270000000002, 2024-09-17T21:21:16.252 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:16.252 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:16.252 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:16.252 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:16.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:16.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:16.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:16.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:16.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:16.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824595+0000", 2024-09-17T21:21:16.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-17T21:21:16.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:16.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824600+0000", 2024-09-17T21:21:16.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:21:16.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:16.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824612+0000", 2024-09-17T21:21:16.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.181e-05 2024-09-17T21:21:16.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824619+0000", 2024-09-17T21:21:16.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:21:16.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:16.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824671+0000", 2024-09-17T21:21:16.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2262999999999998e-05 2024-09-17T21:21:16.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:16.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824695+0000", 2024-09-17T21:21:16.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3883000000000002e-05 2024-09-17T21:21:16.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:16.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832185+0000", 2024-09-17T21:21:16.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007490048 2024-09-17T21:21:16.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:16.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832224+0000", 2024-09-17T21:21:16.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8957e-05 2024-09-17T21:21:16.256 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.256 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:16.256 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.256 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2617 3.d e90/23 3:b0143718:::rbd_data.1152a8ae2019.0000000000000016:head v 90'249, mlcod=90'246)", 2024-09-17T21:21:16.256 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:21:16.256 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 95.330107451000003, 2024-09-17T21:21:16.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063715559999999996, 2024-09-17T21:21:16.257 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:16.257 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:16.257 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:16.257 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:16.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:16.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:21:16.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:16.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844445+0000", 2024-09-17T21:21:16.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:16.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844455+0000", 2024-09-17T21:21:16.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4259999999999992e-06 2024-09-17T21:21:16.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:16.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844470+0000", 2024-09-17T21:21:16.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5386000000000001e-05 2024-09-17T21:21:16.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:16.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844499+0000", 2024-09-17T21:21:16.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.883e-05 2024-09-17T21:21:16.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844518+0000", 2024-09-17T21:21:16.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8830000000000001e-05 2024-09-17T21:21:16.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:16.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844649+0000", 2024-09-17T21:21:16.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013143399999999999 2024-09-17T21:21:16.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:16.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.844702+0000", 2024-09-17T21:21:16.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2876e-05 2024-09-17T21:21:16.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:16.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850696+0000", 2024-09-17T21:21:16.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059943729999999999 2024-09-17T21:21:16.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:16.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850817+0000", 2024-09-17T21:21:16.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120401 2024-09-17T21:21:16.260 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.260 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:16.261 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.261 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2635 3.d e90/23 3:b3d1e920:::rbd_data.1152a8ae2019.000000000000001f:head v 90'252, mlcod=90'246)", 2024-09-17T21:21:16.261 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:16.261 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 95.327399381000006, 2024-09-17T21:21:16.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0077265190000000003, 2024-09-17T21:21:16.261 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:16.261 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:16.261 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:16.261 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:16.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:16.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:16.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:16.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:16.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:16.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847155+0000", 2024-09-17T21:21:16.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.544e-06 2024-09-17T21:21:16.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:16.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847165+0000", 2024-09-17T21:21:16.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.043e-05 2024-09-17T21:21:16.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:16.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847189+0000", 2024-09-17T21:21:16.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3659000000000002e-05 2024-09-17T21:21:16.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847210+0000", 2024-09-17T21:21:16.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1163e-05 2024-09-17T21:21:16.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:16.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848856+0000", 2024-09-17T21:21:16.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001645521 2024-09-17T21:21:16.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:16.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848905+0000", 2024-09-17T21:21:16.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9048999999999997e-05 2024-09-17T21:21:16.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:16.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854770+0000", 2024-09-17T21:21:16.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058652879999999998 2024-09-17T21:21:16.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:16.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854880+0000", 2024-09-17T21:21:16.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010986500000000001 2024-09-17T21:21:16.265 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.265 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:16.265 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.265 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "MOSDPGPush(3.5 151/149 [PushOp(3:a57cb214:::rbd_data.115262b8e217.0000000000000009:1d, version: 103'698, data_included: [196608~24576,229376~24576,262144~24576,294912~131072,524288~98304,819200~8192,843776~16384,876544~16384,909312~16384,942080~8192,1146880~16384,1179648~16384,1212416~16384,1474560~98304,1794048~8192,1826816~8192,1859584~8192,2105344~24576,2138112~24576,2170880~24576,2646016~8192,2678784~8192,2711552~8192,3072000~8192,3104768~8192,3137536~106496], data_size: 761856, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:a57cb214:::rbd_data.115262b8e217.0000000000000009:1d@103'698, size: 3276800, copy_subset: [(0, 3276800)], clone_subset: {}, snapset: 1d=[]:{1d: [1d]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3276800, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T21:21:16.265 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:07.006386+0000", 2024-09-17T21:21:16.266 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.1681669209999992, 2024-09-17T21:21:16.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082699420000000006, 2024-09-17T21:21:16.266 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:16.266 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:16.266 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:16.266 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:16.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:16.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.006386+0000", 2024-09-17T21:21:16.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:16.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.006386+0000", 2024-09-17T21:21:16.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:16.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:16.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.006388+0000", 2024-09-17T21:21:16.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.827e-06 2024-09-17T21:21:16.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:16.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.007671+0000", 2024-09-17T21:21:16.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0012836340000000001 2024-09-17T21:21:16.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:16.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.007737+0000", 2024-09-17T21:21:16.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5815000000000006e-05 2024-09-17T21:21:16.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.007782+0000", 2024-09-17T21:21:16.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5509000000000001e-05 2024-09-17T21:21:16.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:16.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.008023+0000", 2024-09-17T21:21:16.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000240184 2024-09-17T21:21:16.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:16.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.008065+0000", 2024-09-17T21:21:16.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1978999999999999e-05 2024-09-17T21:21:16.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:16.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:16.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:16.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.014656+0000", 2024-09-17T21:21:16.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065909940000000002 2024-09-17T21:21:16.269 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.269 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:16.269 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.269 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:16.270 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:16.270 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:16.271 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:21:16.272 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:16.269+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:16.334 INFO:teuthology.orchestra.run.smithi083.stderr:set 1.0 pg_upmap mapping to [4,3] 2024-09-17T21:21:16.372 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:16.367+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:16.412 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:16.412 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:16.412 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:16.412 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:16.427 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:21:16.471 INFO:teuthology.orchestra.run.smithi184.stdout:truncating image image_client.0-clone42 from 0xd25ac00 (overlap 0xc7da00) to 0x8be400 2024-09-17T21:21:16.472 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:16.467+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:16.555 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:16.556 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:16.556 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:16.556 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:16.556 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:16.570 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:21:16.573 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:16.571+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:16.674 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:16.669+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:16.691 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:16.691 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:16.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:16.691 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:16.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.691 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:21:16.691 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:16.691 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 199.67245777100001, 2024-09-17T21:21:16.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:21:16.691 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:16.692 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:16.692 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:16.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:16.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:21:16.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:21:16.692 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:21:16.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.692 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:16.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:16.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:16.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:16.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:16.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:16.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:21:16.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:21:16.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:16.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:21:16.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:21:16.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:16.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:21:16.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:21:16.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:21:16.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:21:16.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:21:16.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:21:16.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:16.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:21:16.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:21:16.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:21:16.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:21:16.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:16.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:21:16.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:21:16.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:21:16.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:21:16.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:21:16.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:16.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:21:16.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:21:16.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:16.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:21:16.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:21:16.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:16.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:21:16.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:21:16.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:16.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:21:16.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:21:16.698 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.698 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:16.698 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.698 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:16.698 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:16.698 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 180.09229349099999, 2024-09-17T21:21:16.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:21:16.698 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:16.699 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:16.699 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:16.699 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:16.699 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:21:16.699 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:21:16.699 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:21:16.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.699 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:16.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:16.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:16.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:16.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:16.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:16.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:21:16.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:21:16.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:16.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:21:16.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:21:16.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:16.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:21:16.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:21:16.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:21:16.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:21:16.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:21:16.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:21:16.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:16.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:21:16.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:21:16.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:21:16.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:21:16.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:16.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:21:16.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:21:16.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:21:16.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:21:16.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:16.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:21:16.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:21:16.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:16.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:21:16.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:21:16.707 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.707 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:16.707 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.707 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:16.707 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:16.708 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 180.09219617900001, 2024-09-17T21:21:16.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:21:16.708 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:16.708 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:16.708 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:16.708 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:16.708 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:16.708 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:16.708 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:21:16.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.708 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:16.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:16.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:16.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:16.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:16.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:16.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:21:16.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:21:16.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:16.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:21:16.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:21:16.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:16.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:21:16.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:21:16.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:21:16.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:21:16.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:21:16.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:21:16.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:16.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:21:16.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:21:16.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:21:16.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:21:16.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:16.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:21:16.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:21:16.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:21:16.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:21:16.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:21:16.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:21:16.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:21:16.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:21:16.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:21:16.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:16.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:21:16.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:21:16.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:16.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:21:16.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:21:16.714 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.714 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:16.714 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.715 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:16.715 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:16.715 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 180.09010937799999, 2024-09-17T21:21:16.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:21:16.715 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:16.715 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:16.715 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:16.715 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:16.715 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:21:16.715 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:21:16.716 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:21:16.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.716 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:16.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:16.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:16.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:16.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:16.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:16.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:21:16.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:21:16.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:16.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:21:16.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:21:16.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:16.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:21:16.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:21:16.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:21:16.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:21:16.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:21:16.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:21:16.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:21:16.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:21:16.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:21:16.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:21:16.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:21:16.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:16.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:21:16.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:21:16.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:21:16.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:21:16.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:16.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:21:16.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:21:16.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:16.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:21:16.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:21:16.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:16.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:21:16.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:21:16.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:16.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:21:16.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:21:16.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:16.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:21:16.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:21:16.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:16.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:21:16.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:21:16.722 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.722 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:16.722 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.722 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:16.722 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:16.722 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 147.40621059700001, 2024-09-17T21:21:16.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:21:16.723 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:16.723 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:16.723 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:16.723 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:16.723 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:16.723 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:16.723 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:21:16.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.723 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:16.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:16.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:16.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:16.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:16.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:16.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:21:16.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:21:16.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:16.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:21:16.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:21:16.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:16.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:21:16.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:21:16.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:21:16.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:21:16.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:21:16.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:21:16.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:16.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:21:16.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:21:16.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:21:16.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:21:16.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:16.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:21:16.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:21:16.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:21:16.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:21:16.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:16.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:21:16.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:21:16.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:16.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:21:16.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:21:16.729 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.729 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:16.729 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.729 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:16.729 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:16.729 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 147.39711888599999, 2024-09-17T21:21:16.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:21:16.729 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:16.729 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:16.729 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:16.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:16.730 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:16.730 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:16.730 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:21:16.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.730 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:16.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:16.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:16.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:16.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:16.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:16.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:21:16.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:21:16.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:16.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:21:16.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:21:16.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:16.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:21:16.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:21:16.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:21:16.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:21:16.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:21:16.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:21:16.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:16.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:21:16.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:21:16.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:21:16.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:21:16.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:16.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:21:16.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:21:16.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:16.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:21:16.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:21:16.734 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.734 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:16.734 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.734 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:16.734 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:16.734 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 147.39694683499999, 2024-09-17T21:21:16.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:21:16.735 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:16.735 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:16.735 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:16.735 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:16.735 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:16.735 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:16.735 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:21:16.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.735 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:16.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:16.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:16.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:16.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:16.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:16.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:21:16.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:21:16.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:16.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:21:16.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:21:16.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:16.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:21:16.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:21:16.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:21:16.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:21:16.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:21:16.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:21:16.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:16.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:21:16.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:21:16.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:21:16.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:21:16.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:16.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:21:16.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:21:16.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:16.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:21:16.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:21:16.739 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.739 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:16.739 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.740 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:16.740 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:16.740 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 147.22201408800001, 2024-09-17T21:21:16.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:21:16.740 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:16.740 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:16.740 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:16.740 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:16.740 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:16.740 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:16.740 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:21:16.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.741 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:16.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:16.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:16.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:16.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:16.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:16.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:21:16.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:21:16.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:16.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:21:16.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:21:16.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:16.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:21:16.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:21:16.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:21:16.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:21:16.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:21:16.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:21:16.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:16.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:21:16.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:21:16.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:21:16.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:21:16.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:16.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:21:16.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:21:16.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:16.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:21:16.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:21:16.745 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.745 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:16.745 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.745 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:16.745 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:16.745 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 147.221929531, 2024-09-17T21:21:16.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:21:16.745 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:16.746 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:16.746 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:16.746 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:16.746 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:16.746 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:16.746 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:21:16.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.746 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:16.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:16.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:16.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:21:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:21:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:21:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:21:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:21:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:21:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:21:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:21:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:21:16.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:21:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:21:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:21:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:21:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:21:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:21:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:21:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:21:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:16.750 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 147.22186389399999, 2024-09-17T21:21:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:21:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:21:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:21:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:21:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:21:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:21:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:21:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:21:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:21:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:21:16.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:21:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:21:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:21:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:21:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:21:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:21:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:21:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:21:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:21:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:21:16.755 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 95.848288796999995, 2024-09-17T21:21:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:21:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:21:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:21:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:16.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:21:16.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:21:16.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:16.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:21:16.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:21:16.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:16.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:21:16.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:21:16.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:21:16.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:21:16.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:21:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:21:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:21:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:21:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:21:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:21:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.760 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 95.848169284999997, 2024-09-17T21:21:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:21:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:21:16.761 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:21:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:21:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:21:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:21:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:21:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:21:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:21:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:21:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:21:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:21:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:21:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:21:16.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:21:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:21:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:21:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:994 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 571392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:21:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 51.255197311000003, 2024-09-17T21:21:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083898465000000005, 2024-09-17T21:21:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 994 2024-09-17T21:21:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.766 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:21:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:21:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:21:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:21:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431058+0000", 2024-09-17T21:21:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1667e-05 2024-09-17T21:21:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431067+0000", 2024-09-17T21:21:16.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0720000000000006e-06 2024-09-17T21:21:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431075+0000", 2024-09-17T21:21:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9200000000000004e-06 2024-09-17T21:21:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514671+0000", 2024-09-17T21:21:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083596229999999994 2024-09-17T21:21:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:16.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514741+0000", 2024-09-17T21:21:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9929000000000001e-05 2024-09-17T21:21:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514924+0000", 2024-09-17T21:21:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000183122 2024-09-17T21:21:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.770 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:995 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 575488~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:21:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 51.255138295000002, 2024-09-17T21:21:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084214699000000004, 2024-09-17T21:21:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 995 2024-09-17T21:21:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:21:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:21:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431086+0000", 2024-09-17T21:21:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7999999999999995e-07 2024-09-17T21:21:16.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431088+0000", 2024-09-17T21:21:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4729999999999999e-06 2024-09-17T21:21:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431126+0000", 2024-09-17T21:21:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7583999999999998e-05 2024-09-17T21:21:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431137+0000", 2024-09-17T21:21:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1768e-05 2024-09-17T21:21:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515003+0000", 2024-09-17T21:21:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083865097 2024-09-17T21:21:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:16.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515115+0000", 2024-09-17T21:21:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112561 2024-09-17T21:21:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515300+0000", 2024-09-17T21:21:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018453600000000001 2024-09-17T21:21:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:996 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:16.775 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 51.255055914000003, 2024-09-17T21:21:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084493736999999999, 2024-09-17T21:21:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 996 2024-09-17T21:21:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:16.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431168+0000", 2024-09-17T21:21:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4e-07 2024-09-17T21:21:16.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431172+0000", 2024-09-17T21:21:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.608e-06 2024-09-17T21:21:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431180+0000", 2024-09-17T21:21:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5650000000000002e-06 2024-09-17T21:21:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431188+0000", 2024-09-17T21:21:16.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:21:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515354+0000", 2024-09-17T21:21:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084165977000000003 2024-09-17T21:21:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515451+0000", 2024-09-17T21:21:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6669000000000006e-05 2024-09-17T21:21:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515661+0000", 2024-09-17T21:21:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021021599999999999 2024-09-17T21:21:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:997 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 583680~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 51.255024519000003, 2024-09-17T21:21:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084772419000000002, 2024-09-17T21:21:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:16.780 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 997 2024-09-17T21:21:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:16.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0400000000000005e-07 2024-09-17T21:21:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431202+0000", 2024-09-17T21:21:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:21:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431208+0000", 2024-09-17T21:21:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4740000000000002e-06 2024-09-17T21:21:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431214+0000", 2024-09-17T21:21:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6550000000000001e-06 2024-09-17T21:21:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515741+0000", 2024-09-17T21:21:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084526904 2024-09-17T21:21:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515826+0000", 2024-09-17T21:21:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4643999999999998e-05 2024-09-17T21:21:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515971+0000", 2024-09-17T21:21:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145444 2024-09-17T21:21:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.784 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:998 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 587776~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 51.255001546999999, 2024-09-17T21:21:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085108565999999997, 2024-09-17T21:21:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:16.785 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 998 2024-09-17T21:21:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:21:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431224+0000", 2024-09-17T21:21:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:21:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431231+0000", 2024-09-17T21:21:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000002e-06 2024-09-17T21:21:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431236+0000", 2024-09-17T21:21:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.648e-06 2024-09-17T21:21:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516024+0000", 2024-09-17T21:21:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084787584999999999 2024-09-17T21:21:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516122+0000", 2024-09-17T21:21:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8076000000000001e-05 2024-09-17T21:21:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516330+0000", 2024-09-17T21:21:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020808 2024-09-17T21:21:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1000 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 591872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 51.254979431999999, 2024-09-17T21:21:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085389065, 2024-09-17T21:21:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:16.789 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1000 2024-09-17T21:21:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:21:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:16.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431246+0000", 2024-09-17T21:21:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1749999999999999e-06 2024-09-17T21:21:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431253+0000", 2024-09-17T21:21:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2790000000000004e-06 2024-09-17T21:21:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431258+0000", 2024-09-17T21:21:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3299999999999998e-06 2024-09-17T21:21:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516384+0000", 2024-09-17T21:21:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085125549999999994 2024-09-17T21:21:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516460+0000", 2024-09-17T21:21:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6359999999999994e-05 2024-09-17T21:21:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516633+0000", 2024-09-17T21:21:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017285900000000001 2024-09-17T21:21:16.793 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1001 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 595968~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 51.254956026000002, 2024-09-17T21:21:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085614715999999993, 2024-09-17T21:21:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:16.794 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1001 2024-09-17T21:21:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:16.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438066+0000", 2024-09-17T21:21:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067989490000000003 2024-09-17T21:21:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438073+0000", 2024-09-17T21:21:16.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8419999999999999e-06 2024-09-17T21:21:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438090+0000", 2024-09-17T21:21:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6657e-05 2024-09-17T21:21:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438099+0000", 2024-09-17T21:21:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0529999999999996e-06 2024-09-17T21:21:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516678+0000", 2024-09-17T21:21:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078579422999999995 2024-09-17T21:21:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516739+0000", 2024-09-17T21:21:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1315000000000005e-05 2024-09-17T21:21:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516882+0000", 2024-09-17T21:21:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014247700000000001 2024-09-17T21:21:16.798 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:19998 3.5 3:a4885450:::rbd_object_map.1152d139f9f9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-17T21:21:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 37.283688781000002, 2024-09-17T21:21:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90715489299999996, 2024-09-17T21:21:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:16.799 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19998 2024-09-17T21:21:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402537+0000", 2024-09-17T21:21:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5600000000000001e-06 2024-09-17T21:21:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402549+0000", 2024-09-17T21:21:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1929999999999999e-05 2024-09-17T21:21:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402579+0000", 2024-09-17T21:21:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0093000000000001e-05 2024-09-17T21:21:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402595+0000", 2024-09-17T21:21:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5855999999999999e-05 2024-09-17T21:21:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402731+0000", 2024-09-17T21:21:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013610799999999999 2024-09-17T21:21:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402750+0000", 2024-09-17T21:21:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8737e-05 2024-09-17T21:21:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.305977+0000", 2024-09-17T21:21:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90322768399999998 2024-09-17T21:21:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306236+0000", 2024-09-17T21:21:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000258111 2024-09-17T21:21:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306557+0000", 2024-09-17T21:21:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032159799999999998 2024-09-17T21:21:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.308286+0000", 2024-09-17T21:21:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017292150000000001 2024-09-17T21:21:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309530+0000", 2024-09-17T21:21:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00124409 2024-09-17T21:21:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309630+0000", 2024-09-17T21:21:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9689e-05 2024-09-17T21:21:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309689+0000", 2024-09-17T21:21:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9222000000000003e-05 2024-09-17T21:21:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:16.809 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:16.811 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:21:16.812 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:16.807+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:16.912 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:16.907+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:16.987 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:16.987 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:16.987 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:16.987 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:17.002 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:21:17.013 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:17.011+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:17.113 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:17.111+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:17.126 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:17.126 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:17.126 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:17.126 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:17.126 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:17.140 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:21:17.207 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-17T21:21:17.214 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:17.209+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:17.260 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:17.260 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:17.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:17.261 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:17.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.261 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:18003 3.1d 3:b9765580:::rbd_header.11527d2b15f6:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T21:21:17.261 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:17.261 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.898617889999997, 2024-09-17T21:21:17.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94165346900000002, 2024-09-17T21:21:17.261 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.261 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.261 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.261 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.261 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:17.261 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:17.262 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18003 2024-09-17T21:21:17.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.262 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:17.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:17.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357880+0000", 2024-09-17T21:21:17.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:21:17.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357891+0000", 2024-09-17T21:21:17.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0417999999999999e-05 2024-09-17T21:21:17.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357913+0000", 2024-09-17T21:21:17.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2215e-05 2024-09-17T21:21:17.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357922+0000", 2024-09-17T21:21:17.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4779999999999997e-06 2024-09-17T21:21:17.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358023+0000", 2024-09-17T21:21:17.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101025 2024-09-17T21:21:17.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:17.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358040+0000", 2024-09-17T21:21:17.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6291999999999999e-05 2024-09-17T21:21:17.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298203+0000", 2024-09-17T21:21:17.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94016297500000001 2024-09-17T21:21:17.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298626+0000", 2024-09-17T21:21:17.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000422853 2024-09-17T21:21:17.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.299532+0000", 2024-09-17T21:21:17.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00090657300000000004 2024-09-17T21:21:17.266 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.266 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.266 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.266 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15706 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 977920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:17.266 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:21:17.266 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.735516099000002, 2024-09-17T21:21:17.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091952328, 2024-09-17T21:21:17.266 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.267 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.267 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.267 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.267 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:17.267 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:17.267 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15706 2024-09-17T21:21:17.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.267 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:21:17.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:21:17.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520982+0000", 2024-09-17T21:21:17.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.277e-06 2024-09-17T21:21:17.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520988+0000", 2024-09-17T21:21:17.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4470000000000003e-06 2024-09-17T21:21:17.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521008+0000", 2024-09-17T21:21:17.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9309000000000001e-05 2024-09-17T21:21:17.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521023+0000", 2024-09-17T21:21:17.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5119999999999999e-05 2024-09-17T21:21:17.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612638+0000", 2024-09-17T21:21:17.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091615328999999995 2024-09-17T21:21:17.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612738+0000", 2024-09-17T21:21:17.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100466 2024-09-17T21:21:17.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612933+0000", 2024-09-17T21:21:17.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019437999999999999 2024-09-17T21:21:17.270 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.270 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.270 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.271 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15707 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 982016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:17.271 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:17.271 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.735452643000002, 2024-09-17T21:21:17.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092305228000000003, 2024-09-17T21:21:17.271 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.271 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.271 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.271 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.271 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:17.271 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:17.271 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15707 2024-09-17T21:21:17.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.272 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:17.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:17.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521045+0000", 2024-09-17T21:21:17.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4899999999999999e-06 2024-09-17T21:21:17.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521066+0000", 2024-09-17T21:21:17.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0236000000000001e-05 2024-09-17T21:21:17.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521085+0000", 2024-09-17T21:21:17.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8819999999999999e-05 2024-09-17T21:21:17.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521100+0000", 2024-09-17T21:21:17.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5146e-05 2024-09-17T21:21:17.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612978+0000", 2024-09-17T21:21:17.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091878847 2024-09-17T21:21:17.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613079+0000", 2024-09-17T21:21:17.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100503 2024-09-17T21:21:17.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613349+0000", 2024-09-17T21:21:17.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000270186 2024-09-17T21:21:17.275 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.275 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.275 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.275 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15708 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 986112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:17.275 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:17.276 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.735374468000003, 2024-09-17T21:21:17.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092547217000000001, 2024-09-17T21:21:17.276 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.276 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.276 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.276 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.276 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:17.276 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:17.276 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15708 2024-09-17T21:21:17.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.276 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:17.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:17.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521123+0000", 2024-09-17T21:21:17.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.353e-06 2024-09-17T21:21:17.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521129+0000", 2024-09-17T21:21:17.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:21:17.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521145+0000", 2024-09-17T21:21:17.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.611e-05 2024-09-17T21:21:17.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521154+0000", 2024-09-17T21:21:17.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4500000000000004e-06 2024-09-17T21:21:17.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613397+0000", 2024-09-17T21:21:17.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092243275 2024-09-17T21:21:17.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613503+0000", 2024-09-17T21:21:17.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106013 2024-09-17T21:21:17.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613669+0000", 2024-09-17T21:21:17.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016642299999999999 2024-09-17T21:21:17.280 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.280 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.280 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.280 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15709 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 990208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:17.280 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:17.280 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.735331451999997, 2024-09-17T21:21:17.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092728678999999994, 2024-09-17T21:21:17.280 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.280 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.280 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.280 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.281 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:17.281 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:17.281 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15709 2024-09-17T21:21:17.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.281 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:17.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:17.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521166+0000", 2024-09-17T21:21:17.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7000000000000004e-07 2024-09-17T21:21:17.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521169+0000", 2024-09-17T21:21:17.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2660000000000002e-06 2024-09-17T21:21:17.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521181+0000", 2024-09-17T21:21:17.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.208e-05 2024-09-17T21:21:17.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521194+0000", 2024-09-17T21:21:17.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2707e-05 2024-09-17T21:21:17.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613695+0000", 2024-09-17T21:21:17.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092500709 2024-09-17T21:21:17.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613750+0000", 2024-09-17T21:21:17.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.503e-05 2024-09-17T21:21:17.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613894+0000", 2024-09-17T21:21:17.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144117 2024-09-17T21:21:17.284 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.284 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.284 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.285 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15710 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 994304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:17.285 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:17.285 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.735286885999997, 2024-09-17T21:21:17.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092957729000000003, 2024-09-17T21:21:17.285 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.285 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.285 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.285 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.285 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:17.286 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:17.286 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15710 2024-09-17T21:21:17.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.286 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:17.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:17.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:17.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8000000000000005e-07 2024-09-17T21:21:17.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521216+0000", 2024-09-17T21:21:17.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3859999999999998e-06 2024-09-17T21:21:17.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521234+0000", 2024-09-17T21:21:17.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:21:17.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521248+0000", 2024-09-17T21:21:17.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:21:17.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613926+0000", 2024-09-17T21:21:17.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092678324000000006 2024-09-17T21:21:17.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613974+0000", 2024-09-17T21:21:17.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7754999999999999e-05 2024-09-17T21:21:17.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614167+0000", 2024-09-17T21:21:17.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193508 2024-09-17T21:21:17.289 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.289 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.289 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.289 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15711 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 998400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:17.289 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:17.289 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.735229504000003, 2024-09-17T21:21:17.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093268000000000004, 2024-09-17T21:21:17.290 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.290 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.290 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.290 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.290 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:17.290 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:17.290 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15711 2024-09-17T21:21:17.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.290 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:17.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:17.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521268+0000", 2024-09-17T21:21:17.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-17T21:21:17.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521273+0000", 2024-09-17T21:21:17.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:21:17.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521290+0000", 2024-09-17T21:21:17.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7269999999999999e-05 2024-09-17T21:21:17.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521304+0000", 2024-09-17T21:21:17.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3626e-05 2024-09-17T21:21:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614220+0000", 2024-09-17T21:21:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092916367 2024-09-17T21:21:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614322+0000", 2024-09-17T21:21:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102212 2024-09-17T21:21:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614535+0000", 2024-09-17T21:21:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021266199999999999 2024-09-17T21:21:17.294 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.294 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.294 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.294 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15712 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1002496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:17.294 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:17.294 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.735174008999998, 2024-09-17T21:21:17.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093568707000000001, 2024-09-17T21:21:17.294 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.294 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.294 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.294 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.295 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:17.295 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:17.295 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15712 2024-09-17T21:21:17.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.295 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:17.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:17.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521324+0000", 2024-09-17T21:21:17.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.17e-06 2024-09-17T21:21:17.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521334+0000", 2024-09-17T21:21:17.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9369999999999999e-06 2024-09-17T21:21:17.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521356+0000", 2024-09-17T21:21:17.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2045999999999999e-05 2024-09-17T21:21:17.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521370+0000", 2024-09-17T21:21:17.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4270000000000001e-05 2024-09-17T21:21:17.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614583+0000", 2024-09-17T21:21:17.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093213220999999999 2024-09-17T21:21:17.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614680+0000", 2024-09-17T21:21:17.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6900999999999999e-05 2024-09-17T21:21:17.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614891+0000", 2024-09-17T21:21:17.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021116200000000001 2024-09-17T21:21:17.298 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.298 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.298 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.298 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15713 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1006592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:17.298 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:17.298 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.735107307, 2024-09-17T21:21:17.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093834641999999996, 2024-09-17T21:21:17.299 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.299 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.299 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:17.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:17.299 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15713 2024-09-17T21:21:17.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.299 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:17.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:17.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521390+0000", 2024-09-17T21:21:17.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:21:17.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521396+0000", 2024-09-17T21:21:17.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:21:17.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521417+0000", 2024-09-17T21:21:17.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1236000000000001e-05 2024-09-17T21:21:17.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521431+0000", 2024-09-17T21:21:17.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3873000000000001e-05 2024-09-17T21:21:17.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614938+0000", 2024-09-17T21:21:17.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093506522999999994 2024-09-17T21:21:17.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615025+0000", 2024-09-17T21:21:17.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7635999999999999e-05 2024-09-17T21:21:17.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615224+0000", 2024-09-17T21:21:17.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019844700000000001 2024-09-17T21:21:17.302 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.302 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.302 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.303 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15714 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:17.303 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:17.303 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.735044051000003, 2024-09-17T21:21:17.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094071925000000001, 2024-09-17T21:21:17.303 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.303 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.303 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.303 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.303 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:17.303 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:17.303 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15714 2024-09-17T21:21:17.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.303 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:17.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:17.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521454+0000", 2024-09-17T21:21:17.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.097e-06 2024-09-17T21:21:17.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521459+0000", 2024-09-17T21:21:17.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2660000000000001e-06 2024-09-17T21:21:17.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521481+0000", 2024-09-17T21:21:17.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1878999999999998e-05 2024-09-17T21:21:17.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521498+0000", 2024-09-17T21:21:17.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6807e-05 2024-09-17T21:21:17.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615271+0000", 2024-09-17T21:21:17.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093773286999999997 2024-09-17T21:21:17.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615366+0000", 2024-09-17T21:21:17.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5322000000000007e-05 2024-09-17T21:21:17.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615524+0000", 2024-09-17T21:21:17.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015826699999999999 2024-09-17T21:21:17.307 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.307 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.307 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.307 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15715 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1014784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:17.307 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:17.307 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.734979715999998, 2024-09-17T21:21:17.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094243840999999995, 2024-09-17T21:21:17.307 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.307 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.308 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.308 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.308 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:17.308 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:17.308 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15715 2024-09-17T21:21:17.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.308 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:17.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:17.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521518+0000", 2024-09-17T21:21:17.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4300000000000001e-06 2024-09-17T21:21:17.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521524+0000", 2024-09-17T21:21:17.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5860000000000004e-06 2024-09-17T21:21:17.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521541+0000", 2024-09-17T21:21:17.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7574000000000001e-05 2024-09-17T21:21:17.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521555+0000", 2024-09-17T21:21:17.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3540000000000001e-05 2024-09-17T21:21:17.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615560+0000", 2024-09-17T21:21:17.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.09400472 2024-09-17T21:21:17.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615614+0000", 2024-09-17T21:21:17.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4693e-05 2024-09-17T21:21:17.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615761+0000", 2024-09-17T21:21:17.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000146298 2024-09-17T21:21:17.311 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.311 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.311 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.311 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15716 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1018880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:17.312 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:17.312 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.734923371000001, 2024-09-17T21:21:17.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094400676000000003, 2024-09-17T21:21:17.312 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.312 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.312 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.312 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.312 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:17.312 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:17.312 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15716 2024-09-17T21:21:17.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.312 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:17.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:17.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521574+0000", 2024-09-17T21:21:17.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1939999999999999e-06 2024-09-17T21:21:17.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521579+0000", 2024-09-17T21:21:17.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:21:17.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521597+0000", 2024-09-17T21:21:17.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8074e-05 2024-09-17T21:21:17.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521613+0000", 2024-09-17T21:21:17.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5316e-05 2024-09-17T21:21:17.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615780+0000", 2024-09-17T21:21:17.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094166874999999997 2024-09-17T21:21:17.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615836+0000", 2024-09-17T21:21:17.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6926999999999999e-05 2024-09-17T21:21:17.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615974+0000", 2024-09-17T21:21:17.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013741400000000001 2024-09-17T21:21:17.316 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.316 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.316 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.316 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15717 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1022976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:17.316 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:17.316 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.734864674999997, 2024-09-17T21:21:17.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094586723999999997, 2024-09-17T21:21:17.317 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.317 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.317 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.317 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.317 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:17.317 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:17.317 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15717 2024-09-17T21:21:17.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.317 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:17.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:17.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521633+0000", 2024-09-17T21:21:17.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:21:17.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521638+0000", 2024-09-17T21:21:17.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:21:17.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521659+0000", 2024-09-17T21:21:17.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-05 2024-09-17T21:21:17.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521675+0000", 2024-09-17T21:21:17.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5688999999999998e-05 2024-09-17T21:21:17.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616008+0000", 2024-09-17T21:21:17.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094333314000000001 2024-09-17T21:21:17.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616066+0000", 2024-09-17T21:21:17.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7797000000000002e-05 2024-09-17T21:21:17.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616219+0000", 2024-09-17T21:21:17.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015281100000000001 2024-09-17T21:21:17.323 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.323 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.324 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.324 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15718 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:17.324 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:17.324 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.734764599999998, 2024-09-17T21:21:17.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094703674000000002, 2024-09-17T21:21:17.324 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.324 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.324 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.324 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.324 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:17.324 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:17.325 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15718 2024-09-17T21:21:17.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.325 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:17.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:17.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521734+0000", 2024-09-17T21:21:17.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.61e-06 2024-09-17T21:21:17.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521740+0000", 2024-09-17T21:21:17.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:21:17.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521754+0000", 2024-09-17T21:21:17.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3793e-05 2024-09-17T21:21:17.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521763+0000", 2024-09-17T21:21:17.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5129999999999996e-06 2024-09-17T21:21:17.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616256+0000", 2024-09-17T21:21:17.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094492919999999994 2024-09-17T21:21:17.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616312+0000", 2024-09-17T21:21:17.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6737000000000002e-05 2024-09-17T21:21:17.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616436+0000", 2024-09-17T21:21:17.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012335800000000001 2024-09-17T21:21:17.328 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.328 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.328 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.328 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15719 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1031168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:17.328 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:17.328 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.734711195000003, 2024-09-17T21:21:17.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094888963000000007, 2024-09-17T21:21:17.329 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.329 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.329 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.329 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.329 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:17.329 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:17.329 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15719 2024-09-17T21:21:17.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.329 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:17.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:17.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521786+0000", 2024-09-17T21:21:17.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.074e-06 2024-09-17T21:21:17.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521791+0000", 2024-09-17T21:21:17.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3030000000000001e-06 2024-09-17T21:21:17.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521804+0000", 2024-09-17T21:21:17.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3027000000000001e-05 2024-09-17T21:21:17.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521811+0000", 2024-09-17T21:21:17.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6090000000000003e-06 2024-09-17T21:21:17.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616466+0000", 2024-09-17T21:21:17.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094654294 2024-09-17T21:21:17.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616523+0000", 2024-09-17T21:21:17.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6953999999999997e-05 2024-09-17T21:21:17.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616674+0000", 2024-09-17T21:21:17.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015170200000000001 2024-09-17T21:21:17.333 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.333 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.333 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.333 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15720 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1035264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:17.333 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:17.333 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.734674355000003, 2024-09-17T21:21:17.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095067061999999994, 2024-09-17T21:21:17.333 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.333 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.333 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.333 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.334 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:17.334 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:17.334 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15720 2024-09-17T21:21:17.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.334 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:17.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:17.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521823+0000", 2024-09-17T21:21:17.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:21:17.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521826+0000", 2024-09-17T21:21:17.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2100000000000002e-06 2024-09-17T21:21:17.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521835+0000", 2024-09-17T21:21:17.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1630000000000002e-06 2024-09-17T21:21:17.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521842+0000", 2024-09-17T21:21:17.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:21:17.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616711+0000", 2024-09-17T21:21:17.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094868859 2024-09-17T21:21:17.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616764+0000", 2024-09-17T21:21:17.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2778999999999997e-05 2024-09-17T21:21:17.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616889+0000", 2024-09-17T21:21:17.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012533100000000001 2024-09-17T21:21:17.337 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.337 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.337 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.338 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15721 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1039360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:17.338 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:17.338 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.734627246000002, 2024-09-17T21:21:17.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095277219999999996, 2024-09-17T21:21:17.338 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.338 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.338 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.338 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.338 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:17.338 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:17.338 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15721 2024-09-17T21:21:17.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.338 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:17.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:17.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521871+0000", 2024-09-17T21:21:17.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.443e-06 2024-09-17T21:21:17.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521876+0000", 2024-09-17T21:21:17.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0300000000000001e-06 2024-09-17T21:21:17.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521886+0000", 2024-09-17T21:21:17.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186e-05 2024-09-17T21:21:17.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521893+0000", 2024-09-17T21:21:17.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.08e-06 2024-09-17T21:21:17.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616918+0000", 2024-09-17T21:21:17.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095025223000000006 2024-09-17T21:21:17.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616977+0000", 2024-09-17T21:21:17.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8403999999999999e-05 2024-09-17T21:21:17.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617147+0000", 2024-09-17T21:21:17.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016985399999999999 2024-09-17T21:21:17.342 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.342 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.342 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.342 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15722 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:17.342 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:17.342 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.734592796999998, 2024-09-17T21:21:17.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095459783000000006, 2024-09-17T21:21:17.342 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.343 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.343 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.343 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.343 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:17.343 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:17.343 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15722 2024-09-17T21:21:17.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.343 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:17.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:17.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:17.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7700000000000004e-07 2024-09-17T21:21:17.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521907+0000", 2024-09-17T21:21:17.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.813e-06 2024-09-17T21:21:17.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521922+0000", 2024-09-17T21:21:17.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4533e-05 2024-09-17T21:21:17.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521934+0000", 2024-09-17T21:21:17.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2469999999999999e-05 2024-09-17T21:21:17.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617187+0000", 2024-09-17T21:21:17.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095252311000000006 2024-09-17T21:21:17.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617240+0000", 2024-09-17T21:21:17.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3740999999999999e-05 2024-09-17T21:21:17.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617364+0000", 2024-09-17T21:21:17.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-17T21:21:17.346 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.346 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.346 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.347 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15723 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:17.347 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:17.347 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.734548930999999, 2024-09-17T21:21:17.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095653563999999996, 2024-09-17T21:21:17.347 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.347 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.347 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.347 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.347 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:17.347 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:17.347 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15723 2024-09-17T21:21:17.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.347 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:17.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:17.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521949+0000", 2024-09-17T21:21:17.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:21:17.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521953+0000", 2024-09-17T21:21:17.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5870000000000003e-06 2024-09-17T21:21:17.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521970+0000", 2024-09-17T21:21:17.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7282000000000002e-05 2024-09-17T21:21:17.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521984+0000", 2024-09-17T21:21:17.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:21:17.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617383+0000", 2024-09-17T21:21:17.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095399063000000006 2024-09-17T21:21:17.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617437+0000", 2024-09-17T21:21:17.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3931999999999998e-05 2024-09-17T21:21:17.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617601+0000", 2024-09-17T21:21:17.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016407300000000001 2024-09-17T21:21:17.351 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.351 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.351 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.351 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5629.0:2384 3.1c 3:38c1263e:::rbd_data.1152d37183c3.000000000000000b:head [read 11780096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-17T21:21:17.351 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:17.351 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.831718101, 2024-09-17T21:21:17.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0246185320000001, 2024-09-17T21:21:17.351 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.351 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.352 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.352 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.352 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5629", 2024-09-17T21:21:17.352 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3798711329", 2024-09-17T21:21:17.352 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2384 2024-09-17T21:21:17.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.352 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:17.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:17.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424780+0000", 2024-09-17T21:21:17.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.13e-06 2024-09-17T21:21:17.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424785+0000", 2024-09-17T21:21:17.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4739999999999996e-06 2024-09-17T21:21:17.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424798+0000", 2024-09-17T21:21:17.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2704000000000001e-05 2024-09-17T21:21:17.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424806+0000", 2024-09-17T21:21:17.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9899999999999997e-06 2024-09-17T21:21:17.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424868+0000", 2024-09-17T21:21:17.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999997e-05 2024-09-17T21:21:17.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:17.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424875+0000", 2024-09-17T21:21:17.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0870000000000001e-06 2024-09-17T21:21:17.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428897+0000", 2024-09-17T21:21:17.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040226080000000004 2024-09-17T21:21:17.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:17.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428913+0000", 2024-09-17T21:21:17.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:21:17.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.448963+0000", 2024-09-17T21:21:17.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0200498659999999 2024-09-17T21:21:17.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449158+0000", 2024-09-17T21:21:17.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019581000000000001 2024-09-17T21:21:17.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449397+0000", 2024-09-17T21:21:17.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023871 2024-09-17T21:21:17.357 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.357 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.357 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.357 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.357 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.357 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:17.358 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:21:17.359 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:17.355+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:17.360 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:21:17.459 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:17.455+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:17.537 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:17.537 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:17.537 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:17.537 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:17.552 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:21:17.560 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:17.555+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:17.660 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:17.655+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:17.673 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:17.674 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:17.689 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:21:17.761 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:17.757+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:17.811 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:17.811 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:17.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:17.811 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:17.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.811 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:21:17.811 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:17.811 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 176.20202681200001, 2024-09-17T21:21:17.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:21:17.812 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.812 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.812 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:17.812 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:17.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:17.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:21:17.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:21:17.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:21:17.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:21:17.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:21:17.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:21:17.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:21:17.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:21:17.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:21:17.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:21:17.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:21:17.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:21:17.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:17.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:21:17.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:21:17.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:21:17.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:21:17.816 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.816 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.816 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.816 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:21:17.816 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:17.816 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 176.19208017599999, 2024-09-17T21:21:17.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:21:17.816 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.816 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.816 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:17.817 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:17.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:17.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:21:17.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:21:17.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:21:17.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:21:17.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:21:17.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:21:17.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:21:17.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:21:17.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:21:17.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:21:17.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:21:17.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:21:17.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:17.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:21:17.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:21:17.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:21:17.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:21:17.820 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.820 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.820 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.820 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:21:17.821 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:17.821 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 174.94892933599999, 2024-09-17T21:21:17.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:21:17.821 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.821 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.821 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:17.821 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.821 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:21:17.821 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:21:17.821 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:21:17.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.821 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:17.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:17.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:21:17.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:21:17.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:21:17.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:21:17.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:21:17.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:21:17.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:21:17.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:21:17.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:21:17.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:21:17.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:21:17.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:21:17.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:17.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:21:17.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:21:17.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:17.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:21:17.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:21:17.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:17.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:21:17.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:21:17.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:17.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:21:17.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:21:17.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:21:17.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:21:17.826 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.826 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.826 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.826 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:21:17.827 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:17.827 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 174.93928385000001, 2024-09-17T21:21:17.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:21:17.827 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.827 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.827 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:17.827 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.827 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:21:17.827 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:21:17.827 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:21:17.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.827 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:17.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:17.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:21:17.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:21:17.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:21:17.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:21:17.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:21:17.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:21:17.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:21:17.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:21:17.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:21:17.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:21:17.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:21:17.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:21:17.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:17.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:21:17.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:21:17.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:17.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:21:17.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:21:17.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:17.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:21:17.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:21:17.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:17.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:21:17.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:21:17.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:21:17.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:21:17.832 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.832 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.832 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.832 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:21:17.832 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:17.832 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 161.570347817, 2024-09-17T21:21:17.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:21:17.833 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.833 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.833 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.833 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.833 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:21:17.833 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:21:17.833 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:21:17.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.833 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.833 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.833 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:17.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:17.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:21:17.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:21:17.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:21:17.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:21:17.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:21:17.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:21:17.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:21:17.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:21:17.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:21:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:21:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:21:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:21:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:21:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:21:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.837 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:17.837 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:17.837 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 143.368663255, 2024-09-17T21:21:17.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:21:17.837 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.837 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.837 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.837 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.837 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:17.837 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:17.837 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:21:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:21:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:21:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:21:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:21:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:21:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:21:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:21:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:21:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:21:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:21:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:21:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:21:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:21:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:21:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 143.36816867799999, 2024-09-17T21:21:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:21:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:21:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:21:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:21:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:21:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:21:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:21:17.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:21:17.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:21:17.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:21:17.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:21:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:21:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:21:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:21:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:21:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:21:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 143.36811329700001, 2024-09-17T21:21:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:21:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:21:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:21:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:21:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:21:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:21:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:21:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:21:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:21:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:21:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:21:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:21:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:21:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:21:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:21:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:21:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 143.36805701899999, 2024-09-17T21:21:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:21:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:21:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:17.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:21:17.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:21:17.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:21:17.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:21:17.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:21:17.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:21:17.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:21:17.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:21:17.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:21:17.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:21:17.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:21:17.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:21:17.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:21:17.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:21:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 143.36710737999999, 2024-09-17T21:21:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:21:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:21:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:21:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:21:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:21:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:21:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:21:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:21:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:21:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:21:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:21:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:21:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:21:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:21:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:21:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:21:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 143.36702876000001, 2024-09-17T21:21:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:21:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:21:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:21:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:21:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:21:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:21:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:21:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:21:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:21:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:21:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:21:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:21:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:21:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:21:17.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:21:17.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:21:17.864 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.864 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.864 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.864 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:17.864 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:17.864 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 143.36691582099999, 2024-09-17T21:21:17.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:21:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:21:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:21:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:21:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:21:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:21:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:21:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:21:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:21:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:21:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:21:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:21:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:21:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:21:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:21:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:21:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 143.36625476699999, 2024-09-17T21:21:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:21:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:21:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:21:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:21:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:21:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:21:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:21:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:21:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:21:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:21:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:21:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:21:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:21:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:21:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:21:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:21:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 143.366189956, 2024-09-17T21:21:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:21:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:17.875 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:17.875 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:21:17.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.875 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:17.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:21:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:21:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:21:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:21:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:21:17.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:21:17.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:21:17.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:21:17.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:21:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:21:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:21:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:21:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:21:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:21:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 143.366128979, 2024-09-17T21:21:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:21:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:21:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:17.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:21:17.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:21:17.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:21:17.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:21:17.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:21:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:21:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:21:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:21:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:21:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:21:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:21:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:21:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:21:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:21:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:21:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 88.468557794000006, 2024-09-17T21:21:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:21:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:21:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:21:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:21:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:21:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:21:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:21:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:21:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:21:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:21:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:21:17.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:21:17.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:21:17.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:17.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:21:17.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:21:17.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.888 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:21:17.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:21:17.888 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.888 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.888 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.888 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:21:17.888 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:17.888 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 88.259899399999995, 2024-09-17T21:21:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:21:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:21:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:21:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:21:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:21:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:21:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:21:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:21:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:21:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:21:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:21:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:21:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:21:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:21:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:21:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:21:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:21:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:9547 3.2 e147/132 3:42b518d9:::rbd_data.1152f056b73f.0000000000000001:head v 147'513, mlcod=142'511)", 2024-09-17T21:21:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.334926277999999, 2024-09-17T21:21:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.031059566, 2024-09-17T21:21:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471293+0000", 2024-09-17T21:21:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2029999999999999e-06 2024-09-17T21:21:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480606+0000", 2024-09-17T21:21:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093123839999999999 2024-09-17T21:21:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480653+0000", 2024-09-17T21:21:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7852000000000002e-05 2024-09-17T21:21:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480673+0000", 2024-09-17T21:21:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0013e-05 2024-09-17T21:21:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480862+0000", 2024-09-17T21:21:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018833599999999999 2024-09-17T21:21:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480906+0000", 2024-09-17T21:21:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4162000000000002e-05 2024-09-17T21:21:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502259+0000", 2024-09-17T21:21:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021353154999999999 2024-09-17T21:21:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502351+0000", 2024-09-17T21:21:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1460999999999997e-05 2024-09-17T21:21:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:15234 3.1 e147/134 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head v 147'917, mlcod=145'914)", 2024-09-17T21:21:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.70226566, 2024-09-17T21:21:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045303695999999997, 2024-09-17T21:21:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103954+0000", 2024-09-17T21:21:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2259999999999999e-06 2024-09-17T21:21:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119105+0000", 2024-09-17T21:21:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015150871 2024-09-17T21:21:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119141+0000", 2024-09-17T21:21:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6239000000000003e-05 2024-09-17T21:21:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119159+0000", 2024-09-17T21:21:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7946999999999999e-05 2024-09-17T21:21:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119344+0000", 2024-09-17T21:21:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018524599999999999 2024-09-17T21:21:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119399+0000", 2024-09-17T21:21:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4922e-05 2024-09-17T21:21:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149165+0000", 2024-09-17T21:21:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029765697000000001 2024-09-17T21:21:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149255+0000", 2024-09-17T21:21:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0548000000000001e-05 2024-09-17T21:21:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:19859 3.8 e147/133 3:12f72409:::rbd_data.1152f056b73f.0000000000000003:head v 147'482, mlcod=145'479)", 2024-09-17T21:21:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.293374592999999, 2024-09-17T21:21:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033476895999999999, 2024-09-17T21:21:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512845+0000", 2024-09-17T21:21:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.886e-06 2024-09-17T21:21:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524104+0000", 2024-09-17T21:21:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011259668 2024-09-17T21:21:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524148+0000", 2024-09-17T21:21:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3386000000000001e-05 2024-09-17T21:21:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524172+0000", 2024-09-17T21:21:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4762000000000001e-05 2024-09-17T21:21:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524379+0000", 2024-09-17T21:21:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020671599999999999 2024-09-17T21:21:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524421+0000", 2024-09-17T21:21:17.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2182999999999999e-05 2024-09-17T21:21:17.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:17.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546216+0000", 2024-09-17T21:21:17.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021795101000000001 2024-09-17T21:21:17.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:17.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:17.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:17.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546320+0000", 2024-09-17T21:21:17.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103194 2024-09-17T21:21:17.906 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.906 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.906 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.906 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:17.906 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:17.906 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:17.908 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:17.905+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:17.908 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:21:17.961 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:21:18.008 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:18.003+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:18.042 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:18.042 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:18.042 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:21:18.042 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:18.057 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:21:18.109 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:18.109 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:18.110 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:18.105+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:18.126 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:18.126 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:18.144 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:21:18.144 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:21:18.166 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:18.166 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:18.188 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:18.188 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:18.209 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:18.209 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:18.210 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:18.205+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:18.218 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:18.218 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:18.234 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:21:18.311 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:18.305+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:18.343 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-17T21:21:18.400 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:18.400 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:18.412 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:21:18.413 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:18.409+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:18.513 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:18.511+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:18.576 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:18.576 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:18.591 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:21:18.603 INFO:teuthology.orchestra.run.smithi184.stdout:5935 write 0x531c800 thru 0x532266f (0x5e70 bytes) 2024-09-17T21:21:18.613 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:18.609+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:18.646 INFO:teuthology.orchestra.run.smithi184.stdout:5936 trunc from 0xd25ac00 to 0x5744400 2024-09-17T21:21:18.714 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:18.711+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:18.725 INFO:teuthology.orchestra.run.smithi184.stdout:5938 write 0xcdd5600 thru 0xcddb0be (0x5abf bytes) 2024-09-17T21:21:18.759 INFO:teuthology.orchestra.run.smithi184.stdout:5940 write 0xd3a8e00 thru 0xd3ab0d4 (0x22d5 bytes) 2024-09-17T21:21:18.760 INFO:teuthology.orchestra.run.smithi184.stdout:5942 write 0xb5ddc00 thru 0xb5e098e (0x2d8f bytes) 2024-09-17T21:21:18.760 INFO:teuthology.orchestra.run.smithi184.stdout:5944 write 0xb03e600 thru 0xb04e2d1 (0xfcd2 bytes) 2024-09-17T21:21:18.761 INFO:teuthology.orchestra.run.smithi184.stdout:5946 read 0x9fb4000 thru 0x9fb7dd2 (0x3dd3 bytes) 2024-09-17T21:21:18.761 INFO:teuthology.orchestra.run.smithi184.stdout:5947 read 0x4f52800 thru 0x4f5539c (0x2b9d bytes) 2024-09-17T21:21:18.761 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:18.761 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:18.762 INFO:teuthology.orchestra.run.smithi184.stdout:5948 punch from 0x612ec00 to 0x6133600, (0x4a00 bytes) 2024-09-17T21:21:18.777 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:21:18.805 INFO:teuthology.orchestra.run.smithi184.stdout:5951 write 0xdadbe00 thru 0xdaead8b (0xef8c bytes) 2024-09-17T21:21:18.807 INFO:teuthology.orchestra.run.smithi184.stdout:5954 read 0xab92400 thru 0xab95a4f (0x3650 bytes) 2024-09-17T21:21:18.808 INFO:teuthology.orchestra.run.smithi184.stdout:5955 write 0x535c600 thru 0x535d29d (0xc9e bytes) 2024-09-17T21:21:18.808 INFO:teuthology.orchestra.run.smithi184.stdout:5956 read 0xd87200 thru 0xd8e42a (0x722b bytes) 2024-09-17T21:21:18.809 INFO:teuthology.orchestra.run.smithi184.stdout:5959 write 0x68d2400 thru 0x68db031 (0x8c32 bytes) 2024-09-17T21:21:18.809 INFO:teuthology.orchestra.run.smithi184.stdout:5960 read 0x46ada00 thru 0x46bafa2 (0xd5a3 bytes) 2024-09-17T21:21:18.809 INFO:teuthology.orchestra.run.smithi184.stdout:5961 punch from 0x48cf400 to 0x48d4600, (0x5200 bytes) 2024-09-17T21:21:18.810 INFO:teuthology.orchestra.run.smithi184.stdout:5962 read 0xa024a00 thru 0xa028ea5 (0x44a6 bytes) 2024-09-17T21:21:18.810 INFO:teuthology.orchestra.run.smithi184.stdout:5966 punch from 0x5f6000 to 0x5fcc00, (0x6c00 bytes) 2024-09-17T21:21:18.820 INFO:teuthology.orchestra.run.smithi184.stdout:5968 trunc from 0xdaeae00 to 0x9210600 2024-09-17T21:21:18.821 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:18.819+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:18.863 INFO:teuthology.orchestra.run.smithi184.stdout:5970 read 0x30fca00 thru 0x30fce47 (0x448 bytes) 2024-09-17T21:21:18.883 INFO:teuthology.orchestra.run.smithi184.stdout:5971 write 0xae54e00 thru 0xae62e9e (0xe09f bytes) 2024-09-17T21:21:18.917 INFO:teuthology.orchestra.run.smithi184.stdout:5972 write 0xe9b4600 thru 0xe9b4a35 (0x436 bytes) 2024-09-17T21:21:18.917 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:18.913+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:18.919 INFO:teuthology.orchestra.run.smithi184.stdout:5973 read 0x5f08200 thru 0x5f10320 (0x8121 bytes) 2024-09-17T21:21:18.920 INFO:teuthology.orchestra.run.smithi184.stdout:5974 write 0xa656000 thru 0xa663440 (0xd441 bytes) 2024-09-17T21:21:18.920 INFO:teuthology.orchestra.run.smithi184.stdout:5977 write 0x689a200 thru 0x68a7a0b (0xd80c bytes) 2024-09-17T21:21:18.920 INFO:teuthology.orchestra.run.smithi184.stdout:5979 read 0x37e4c00 thru 0x37e794a (0x2d4b bytes) 2024-09-17T21:21:18.920 INFO:teuthology.orchestra.run.smithi184.stdout:5982 read 0xad7c800 thru 0xad7f21e (0x2a1f bytes) 2024-09-17T21:21:18.921 INFO:teuthology.orchestra.run.smithi184.stdout:5986 write 0x98a6600 thru 0x98a83c2 (0x1dc3 bytes) 2024-09-17T21:21:18.921 INFO:teuthology.orchestra.run.smithi184.stdout:5991 read 0x3afae00 thru 0x3b07001 (0xc202 bytes) 2024-09-17T21:21:18.922 INFO:teuthology.orchestra.run.smithi184.stdout:5992 write 0x5e9200 thru 0x5f416a (0xaf6b bytes) 2024-09-17T21:21:18.924 INFO:teuthology.orchestra.run.smithi184.stdout:5996 read 0x78c000 thru 0x7921c0 (0x61c1 bytes) 2024-09-17T21:21:18.926 INFO:teuthology.orchestra.run.smithi184.stdout:5997 write 0x86d8800 thru 0x86d9e5c (0x165d bytes) 2024-09-17T21:21:18.926 INFO:teuthology.orchestra.run.smithi184.stdout:5999 read 0x230b800 thru 0x230ed3b (0x353c bytes) 2024-09-17T21:21:18.956 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:18.956 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:18.968 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:21:19.018 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:19.015+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:19.087 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:19.088 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:19.103 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:21:19.119 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:19.113+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:19.219 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:19.215+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:19.240 INFO:teuthology.orchestra.run.smithi184.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-17T21:21:19.269 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:19.269 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:19.284 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:21:19.319 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:19.317+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:19.421 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:19.417+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:19.450 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:19.450 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:19.464 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:21:19.521 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:19.519+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:19.603 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:19.603 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:19.619 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:21:19.622 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:19.617+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:19.723 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:19.717+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:19.738 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:19.739 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:19.752 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:21:19.824 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:19.819+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:19.925 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:19.925 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:19.925 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:19.921+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:19.938 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:21:20.026 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:20.023+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:20.110 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:20.111 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:20.125 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:21:20.126 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:20.121+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:20.227 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:21:20.228 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:20.223+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:20.246 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:20.246 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:20.260 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:21:20.328 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:20.323+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:20.364 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:21:20.386 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:20.386 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:20.388 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:20.388 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:20.401 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:21:20.406 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:20.406 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:20.424 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:21:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:21:20.442 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:20.442 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:20.447 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:20.445+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:20.460 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:20.460 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:20.479 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:20.480 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:20.525 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:20.525 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:20.539 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:21:20.549 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:20.547+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:20.630 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:20.627+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:20.707 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:20.707 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:20.707 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:20.707 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:20.707 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:20.721 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:21:20.767 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:20.765+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:20.832 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:20.829+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:20.844 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:20.844 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:20.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:20.844 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:20.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.844 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:21:20.844 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:20.844 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 179.235420933, 2024-09-17T21:21:20.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:21:20.844 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:20.844 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:20.845 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:20.845 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:20.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:20.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:20.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:20.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:20.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:20.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:21:20.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:21:20.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:20.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:21:20.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:21:20.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:20.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:21:20.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:21:20.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:20.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:21:20.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:21:20.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:20.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:21:20.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:21:20.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:20.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:21:20.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:21:20.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:20.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:21:20.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:21:20.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:20.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:21:20.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:21:20.849 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.849 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:20.849 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.849 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:21:20.849 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:20.849 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 179.22547429700001, 2024-09-17T21:21:20.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:21:20.849 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:20.849 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:20.849 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:20.849 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:20.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:20.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:20.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:20.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:20.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:20.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:21:20.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:21:20.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:20.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:21:20.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:21:20.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:20.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:21:20.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:21:20.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:20.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:21:20.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:21:20.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:20.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:21:20.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:21:20.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:20.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:21:20.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:21:20.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:20.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:21:20.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:21:20.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:20.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:21:20.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:21:20.853 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.853 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:20.853 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.854 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:21:20.854 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:20.854 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 177.98232345700001, 2024-09-17T21:21:20.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:21:20.854 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:20.854 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:20.854 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:20.854 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:20.854 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:21:20.854 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:21:20.854 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:21:20.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.855 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:20.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:20.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:20.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:20.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:20.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:20.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:21:20.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:21:20.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:20.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:21:20.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:21:20.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:20.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:21:20.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:21:20.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:20.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:21:20.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:21:20.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:20.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:21:20.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:21:20.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:20.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:21:20.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:21:20.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:20.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:21:20.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:21:20.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:20.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:21:20.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:21:20.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:20.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:21:20.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:21:20.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:20.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:21:20.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:21:20.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:20.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:21:20.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:21:20.860 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.860 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:20.860 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.860 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:21:20.860 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:20.860 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 177.972677971, 2024-09-17T21:21:20.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:21:20.860 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:20.860 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:20.860 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:20.861 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:20.861 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:21:20.861 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:21:20.861 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:21:20.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.861 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:20.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:20.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:20.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:20.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:20.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:20.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:21:20.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:21:20.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:20.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:21:20.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:21:20.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:20.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:21:20.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:21:20.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:20.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:21:20.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:21:20.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:20.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:21:20.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:21:20.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:20.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:21:20.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:21:20.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:20.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:21:20.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:21:20.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:20.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:21:20.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:21:20.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:20.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:21:20.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:21:20.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:20.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:21:20.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:21:20.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:20.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:21:20.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:21:20.866 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.866 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:20.866 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.866 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:21:20.866 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:20.866 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 164.60374193800001, 2024-09-17T21:21:20.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:21:20.866 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:20.867 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:20.867 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:20.867 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:20.867 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:21:20.867 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:21:20.867 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:21:20.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.867 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:20.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:20.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:20.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:20.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:20.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:20.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:21:20.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:21:20.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:20.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:21:20.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:21:20.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.869 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:20.869 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:21:20.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:21:20.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.869 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:20.869 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:21:20.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:21:20.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:20.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:21:20.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:21:20.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:20.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:21:20.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:21:20.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:20.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:21:20.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:21:20.871 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.871 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:20.871 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.871 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:20.871 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:20.871 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 146.40205737599999, 2024-09-17T21:21:20.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:21:20.871 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:20.871 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:20.871 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:20.871 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:20.872 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:20.872 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:20.872 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:21:20.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.872 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:20.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:20.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:20.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:20.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:20.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:20.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:21:20.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:21:20.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:20.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:21:20.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:21:20.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.874 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:20.874 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:21:20.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:21:20.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.874 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:20.874 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:21:20.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:21:20.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.874 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:20.874 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:21:20.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:21:20.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:20.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:21:20.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:21:20.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:20.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:21:20.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:21:20.875 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.875 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:20.875 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.876 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:20.876 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:20.876 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 146.401562799, 2024-09-17T21:21:20.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:21:20.876 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:20.876 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:20.876 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:20.876 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:20.876 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:20.876 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:20.877 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:21:20.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.877 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:20.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:20.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:20.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:20.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:20.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:20.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:21:20.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:21:20.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:20.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:21:20.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:21:20.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:20.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:21:20.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:21:20.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:20.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:21:20.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:21:20.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:20.880 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:21:20.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:21:20.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:20.880 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:21:20.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:21:20.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:20.880 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:21:20.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:21:20.880 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.881 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:20.881 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.881 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:20.881 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:20.881 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 146.40150741799999, 2024-09-17T21:21:20.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:21:20.881 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:20.881 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:20.881 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:20.881 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:20.881 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:20.881 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:20.882 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:21:20.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.882 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:20.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:20.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:20.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:20.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:20.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:20.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:21:20.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:21:20.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:20.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:21:20.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:21:20.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:20.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:21:20.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:21:20.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:20.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:21:20.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:21:20.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:20.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:21:20.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:21:20.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:20.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:21:20.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:21:20.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:20.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:21:20.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:21:20.885 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.885 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:20.885 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.886 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:20.886 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:20.886 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 146.40145114000001, 2024-09-17T21:21:20.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:21:20.886 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:20.886 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:20.886 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:20.886 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:20.886 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:20.886 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:20.886 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:21:20.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.886 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:20.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:20.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:20.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:20.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:20.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:20.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:21:20.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:21:20.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:20.888 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:21:20.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:21:20.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:20.888 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:21:20.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:21:20.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:20.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:21:20.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:21:20.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:20.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:21:20.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:21:20.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:20.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:21:20.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:21:20.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:20.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:21:20.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:21:20.890 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.890 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:20.890 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.890 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:20.890 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:20.891 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 146.40050150100001, 2024-09-17T21:21:20.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:21:20.891 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:20.891 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:20.891 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:20.891 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:20.891 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:20.891 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:20.891 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:21:20.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.891 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:20.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:20.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:20.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.892 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:20.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:20.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.892 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:20.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:21:20.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:21:20.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:20.893 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:21:20.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:21:20.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:20.893 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:21:20.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:21:20.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:20.893 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:21:20.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:21:20.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:20.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:21:20.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:21:20.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:20.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:21:20.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:21:20.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:20.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:21:20.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:21:20.895 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.895 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:20.895 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.895 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:20.895 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:20.895 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 146.400422881, 2024-09-17T21:21:20.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:21:20.895 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:20.895 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:20.896 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:20.896 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:20.896 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:20.896 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:20.896 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:21:20.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.896 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:20.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:20.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:20.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:20.897 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:20.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.897 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:20.897 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:21:20.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:21:20.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.897 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:20.897 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:21:20.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:21:20.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:20.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:21:20.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:21:20.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:20.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:21:20.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:21:20.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:20.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:21:20.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:21:20.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:20.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:21:20.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:21:20.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:20.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:21:20.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:21:20.899 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.899 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:20.899 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.900 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:20.900 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:20.900 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 146.40030994200001, 2024-09-17T21:21:20.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:21:20.900 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:20.900 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:20.900 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:20.900 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:20.900 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:20.900 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:20.900 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:21:20.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.901 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:20.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:20.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:20.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:20.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:20.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:20.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:21:20.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:21:20.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:20.902 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:21:20.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:21:20.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:20.902 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:21:20.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:21:20.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:20.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:21:20.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:21:20.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:20.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:21:20.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:21:20.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:20.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:21:20.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:21:20.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:20.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:21:20.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:21:20.904 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.904 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:20.904 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.904 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:20.904 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:20.904 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 146.399648888, 2024-09-17T21:21:20.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:21:20.905 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:20.905 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:20.905 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:20.905 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:20.905 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:20.905 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:20.905 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:21:20.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.905 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:20.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:20.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:20.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:20.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:20.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:20.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:21:20.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:21:20.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:20.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:21:20.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:21:20.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:20.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:21:20.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:21:20.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:20.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:21:20.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:21:20.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:20.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:21:20.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:21:20.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:20.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:21:20.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:21:20.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:20.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:21:20.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:21:20.908 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.909 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:20.909 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.909 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:20.909 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:20.909 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 146.39958407699999, 2024-09-17T21:21:20.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:21:20.909 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:20.909 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:20.909 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:20.909 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:20.909 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:20.909 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:20.910 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:21:20.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.910 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:20.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.910 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:20.910 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:20.910 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.910 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:20.910 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:20.910 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:20.911 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:21:20.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:21:20.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:20.911 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:21:20.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:21:20.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:20.911 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:21:20.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:21:20.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:20.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:21:20.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:21:20.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:20.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:21:20.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:21:20.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.913 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:20.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:21:20.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:21:20.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.913 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:20.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:21:20.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:21:20.913 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.913 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:20.913 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.914 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:20.914 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:20.914 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 146.39952310000001, 2024-09-17T21:21:20.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:21:20.914 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:20.914 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:20.914 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:20.914 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:20.914 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:20.914 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:20.914 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:21:20.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.914 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:20.914 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.915 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:20.915 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:20.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.915 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:20.915 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:20.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.915 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:20.915 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:21:20.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:21:20.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:20.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:21:20.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:21:20.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:20.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:21:20.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:21:20.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:20.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:21:20.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:21:20.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.917 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:20.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:21:20.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:21:20.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.917 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:20.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:21:20.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:21:20.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.917 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:20.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:21:20.918 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:21:20.918 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.918 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:20.918 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.918 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:21:20.918 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:20.918 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 91.501951915000006, 2024-09-17T21:21:20.918 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:21:20.918 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:20.918 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:20.918 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:20.919 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:20.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:20.919 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:20.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:20.919 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:20.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:20.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:21:20.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:21:20.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:20.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:21:20.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:21:20.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:20.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:21:20.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:21:20.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:20.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:21:20.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:21:20.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:20.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:21:20.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:21:20.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:20.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:21:20.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:21:20.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:20.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:21:20.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:21:20.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:20.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:21:20.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:21:20.922 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.922 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:20.922 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.923 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:21:20.923 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:20.923 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 91.293293520999995, 2024-09-17T21:21:20.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:21:20.923 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:20.923 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:20.923 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:20.923 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:20.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.923 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:20.923 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:20.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:20.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:20.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:20.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:21:20.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:21:20.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:20.925 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:21:20.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:21:20.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:20.925 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:21:20.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:21:20.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:20.925 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:21:20.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:21:20.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:20.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:21:20.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:21:20.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:20.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:21:20.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:21:20.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:20.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:21:20.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:21:20.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:20.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:21:20.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:21:20.927 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.927 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:20.927 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.927 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:9547 3.2 e147/132 3:42b518d9:::rbd_data.1152f056b73f.0000000000000001:head v 147'513, mlcod=142'511)", 2024-09-17T21:21:20.927 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:20.927 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.368320399000002, 2024-09-17T21:21:20.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.031059566, 2024-09-17T21:21:20.928 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:20.928 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:20.928 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:20.928 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:20.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:20.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:20.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:20.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:20.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:20.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471293+0000", 2024-09-17T21:21:20.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2029999999999999e-06 2024-09-17T21:21:20.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:20.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480606+0000", 2024-09-17T21:21:20.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093123839999999999 2024-09-17T21:21:20.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:20.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480653+0000", 2024-09-17T21:21:20.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7852000000000002e-05 2024-09-17T21:21:20.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.930 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:20.930 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480673+0000", 2024-09-17T21:21:20.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0013e-05 2024-09-17T21:21:20.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.930 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:20.930 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480862+0000", 2024-09-17T21:21:20.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018833599999999999 2024-09-17T21:21:20.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.930 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:20.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480906+0000", 2024-09-17T21:21:20.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4162000000000002e-05 2024-09-17T21:21:20.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:20.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502259+0000", 2024-09-17T21:21:20.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021353154999999999 2024-09-17T21:21:20.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:20.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502351+0000", 2024-09-17T21:21:20.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1460999999999997e-05 2024-09-17T21:21:20.931 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.932 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:20.932 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.932 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:15234 3.1 e147/134 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head v 147'917, mlcod=145'914)", 2024-09-17T21:21:20.932 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:20.932 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.735659780999999, 2024-09-17T21:21:20.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045303695999999997, 2024-09-17T21:21:20.932 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:20.932 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:20.932 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:20.932 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:20.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:20.933 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:20.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:20.933 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:20.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:20.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103954+0000", 2024-09-17T21:21:20.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2259999999999999e-06 2024-09-17T21:21:20.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:20.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119105+0000", 2024-09-17T21:21:20.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015150871 2024-09-17T21:21:20.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:20.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119141+0000", 2024-09-17T21:21:20.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6239000000000003e-05 2024-09-17T21:21:20.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:20.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119159+0000", 2024-09-17T21:21:20.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7946999999999999e-05 2024-09-17T21:21:20.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:20.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119344+0000", 2024-09-17T21:21:20.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018524599999999999 2024-09-17T21:21:20.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:20.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119399+0000", 2024-09-17T21:21:20.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4922e-05 2024-09-17T21:21:20.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:20.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149165+0000", 2024-09-17T21:21:20.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029765697000000001 2024-09-17T21:21:20.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:20.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149255+0000", 2024-09-17T21:21:20.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0548000000000001e-05 2024-09-17T21:21:20.936 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.936 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:20.936 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.937 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:19859 3.8 e147/133 3:12f72409:::rbd_data.1152f056b73f.0000000000000003:head v 147'482, mlcod=145'479)", 2024-09-17T21:21:20.937 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:20.937 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.326768714, 2024-09-17T21:21:20.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033476895999999999, 2024-09-17T21:21:20.937 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:20.937 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:20.937 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:20.937 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:20.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.937 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:20.937 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:20.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:20.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:20.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:20.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:20.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512845+0000", 2024-09-17T21:21:20.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.886e-06 2024-09-17T21:21:20.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:20.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524104+0000", 2024-09-17T21:21:20.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011259668 2024-09-17T21:21:20.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.939 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.939 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:20.939 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524148+0000", 2024-09-17T21:21:20.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3386000000000001e-05 2024-09-17T21:21:20.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.939 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.939 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:20.939 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524172+0000", 2024-09-17T21:21:20.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4762000000000001e-05 2024-09-17T21:21:20.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.939 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.939 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:20.940 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524379+0000", 2024-09-17T21:21:20.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020671599999999999 2024-09-17T21:21:20.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.940 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:20.940 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524421+0000", 2024-09-17T21:21:20.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2182999999999999e-05 2024-09-17T21:21:20.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.940 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:20.940 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546216+0000", 2024-09-17T21:21:20.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021795101000000001 2024-09-17T21:21:20.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:20.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:20.941 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:20.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546320+0000", 2024-09-17T21:21:20.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103194 2024-09-17T21:21:20.941 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.941 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:20.941 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.941 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:20.941 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:20.941 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:20.943 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:20.937+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:20.943 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:21:21.044 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:21.041+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:21.077 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:21.078 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:21.078 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:21:21.078 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:21.093 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:21:21.145 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:21.141+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:21.217 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:21.217 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:21.217 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:21:21.217 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:21:21.218 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:21.231 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:21:21.277 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:21.275+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:21.347 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:21.345+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:21.352 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:21.352 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:21.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:21.353 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:21.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.353 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:21:21.353 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:21.353 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 124.83728874800001, 2024-09-17T21:21:21.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:21:21.353 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.353 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.353 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:21.353 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.353 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:21:21.353 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:21:21.353 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:21:21.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.354 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:21.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:21.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:21:21.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:21:21.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:21:21.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:21:21.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:21:21.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:21:21.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:21:21.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:21:21.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:21:21.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:21:21.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:21.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:21:21.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:21:21.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:21:21.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:21:21.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:21:21.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:21:21.357 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.357 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.357 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.358 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2597 3.1f 3:fb0e9615:::rbd_data.1152a8ae2019.000000000000000c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:21.358 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:21.358 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 100.50885232, 2024-09-17T21:21:21.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097064994000000002, 2024-09-17T21:21:21.358 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.358 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.358 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:21.358 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.358 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:21.358 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:21.358 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2597 2024-09-17T21:21:21.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.358 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:21.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:21.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838628+0000", 2024-09-17T21:21:21.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.187e-06 2024-09-17T21:21:21.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838654+0000", 2024-09-17T21:21:21.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:21:21.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838696+0000", 2024-09-17T21:21:21.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1226000000000001e-05 2024-09-17T21:21:21.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838715+0000", 2024-09-17T21:21:21.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9646000000000001e-05 2024-09-17T21:21:21.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838937+0000", 2024-09-17T21:21:21.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022163 2024-09-17T21:21:21.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.839284+0000", 2024-09-17T21:21:21.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034753799999999999 2024-09-17T21:21:21.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935691+0000", 2024-09-17T21:21:21.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.096406360999999996 2024-09-17T21:21:21.362 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.362 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.362 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.362 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2641 3.12 3:488a729f:::rbd_data.1152a8ae2019.0000000000000022:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:21.362 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:21.362 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 100.50539554700001, 2024-09-17T21:21:21.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098280953000000004, 2024-09-17T21:21:21.362 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.362 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.362 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:21.363 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.363 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:21.363 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:21.363 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2641 2024-09-17T21:21:21.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.363 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:21.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:21.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842083+0000", 2024-09-17T21:21:21.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5300000000000002e-07 2024-09-17T21:21:21.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842088+0000", 2024-09-17T21:21:21.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:21:21.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842105+0000", 2024-09-17T21:21:21.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6773e-05 2024-09-17T21:21:21.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842119+0000", 2024-09-17T21:21:21.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:21:21.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842320+0000", 2024-09-17T21:21:21.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020052700000000001 2024-09-17T21:21:21.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842845+0000", 2024-09-17T21:21:21.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052508600000000002 2024-09-17T21:21:21.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940363+0000", 2024-09-17T21:21:21.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097518795000000005 2024-09-17T21:21:21.366 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.366 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.367 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.367 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2674 3.12 3:4c87e3ac:::rbd_data.1152a8ae2019.0000000000000033:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:21.367 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:21.367 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 100.50503291, 2024-09-17T21:21:21.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098025125000000005, 2024-09-17T21:21:21.367 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.367 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.367 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:21.367 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.367 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:21.367 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:21.367 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2674 2024-09-17T21:21:21.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.368 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:21.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:21.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842447+0000", 2024-09-17T21:21:21.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3999999999999999e-06 2024-09-17T21:21:21.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842453+0000", 2024-09-17T21:21:21.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:21:21.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842472+0000", 2024-09-17T21:21:21.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9006000000000001e-05 2024-09-17T21:21:21.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842487+0000", 2024-09-17T21:21:21.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4866e-05 2024-09-17T21:21:21.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843594+0000", 2024-09-17T21:21:21.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011073249999999999 2024-09-17T21:21:21.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843668+0000", 2024-09-17T21:21:21.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3188000000000005e-05 2024-09-17T21:21:21.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:21.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843893+0000", 2024-09-17T21:21:21.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022503399999999999 2024-09-17T21:21:21.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:21.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850439+0000", 2024-09-17T21:21:21.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065467880000000004 2024-09-17T21:21:21.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:21.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940409+0000", 2024-09-17T21:21:21.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089969511000000002 2024-09-17T21:21:21.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:21.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940444+0000", 2024-09-17T21:21:21.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5345999999999997e-05 2024-09-17T21:21:21.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940470+0000", 2024-09-17T21:21:21.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5964000000000001e-05 2024-09-17T21:21:21.372 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.372 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.373 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.373 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2707 3.1f 3:f805e65e:::rbd_data.1152a8ae2019.0000000000000049:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:21.373 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:21.373 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 100.499986249, 2024-09-17T21:21:21.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088327300999999997, 2024-09-17T21:21:21.373 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.373 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.373 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:21.373 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.373 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:21.373 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:21.373 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2707 2024-09-17T21:21:21.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.374 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:21.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:21.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847494+0000", 2024-09-17T21:21:21.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:21:21.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847504+0000", 2024-09-17T21:21:21.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:21:21.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847526+0000", 2024-09-17T21:21:21.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1976000000000001e-05 2024-09-17T21:21:21.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847541+0000", 2024-09-17T21:21:21.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5277e-05 2024-09-17T21:21:21.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856142+0000", 2024-09-17T21:21:21.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086006710000000007 2024-09-17T21:21:21.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856435+0000", 2024-09-17T21:21:21.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029301499999999999 2024-09-17T21:21:21.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935819+0000", 2024-09-17T21:21:21.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079384448999999996 2024-09-17T21:21:21.377 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.377 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.377 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.377 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2730 3.12 3:48b3422a:::rbd_data.1152a8ae2019.0000000000000057:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:21.377 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:21.377 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 100.49079335, 2024-09-17T21:21:21.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083879975999999995, 2024-09-17T21:21:21.378 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.378 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.378 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:21.378 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.378 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:21.378 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:21.378 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2730 2024-09-17T21:21:21.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.378 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:21.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:21.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856687+0000", 2024-09-17T21:21:21.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:21:21.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856697+0000", 2024-09-17T21:21:21.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0207000000000001e-05 2024-09-17T21:21:21.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856808+0000", 2024-09-17T21:21:21.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011071200000000001 2024-09-17T21:21:21.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856828+0000", 2024-09-17T21:21:21.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0219000000000001e-05 2024-09-17T21:21:21.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856907+0000", 2024-09-17T21:21:21.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9661999999999997e-05 2024-09-17T21:21:21.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.857019+0000", 2024-09-17T21:21:21.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011148499999999999 2024-09-17T21:21:21.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940565+0000", 2024-09-17T21:21:21.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083545791999999994 2024-09-17T21:21:21.381 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.382 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.382 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.382 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2743 3.12 3:4d036e62:::rbd_data.1152a8ae2019.000000000000005f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:21.382 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:21.382 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 100.486847849, 2024-09-17T21:21:21.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080059962999999998, 2024-09-17T21:21:21.382 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.382 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.382 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:21.382 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.382 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:21.382 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:21.383 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2743 2024-09-17T21:21:21.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.383 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:21.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:21.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860631+0000", 2024-09-17T21:21:21.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:21:21.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860636+0000", 2024-09-17T21:21:21.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5279999999999997e-06 2024-09-17T21:21:21.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860649+0000", 2024-09-17T21:21:21.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-17T21:21:21.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860655+0000", 2024-09-17T21:21:21.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:21:21.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860731+0000", 2024-09-17T21:21:21.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5920000000000005e-05 2024-09-17T21:21:21.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.861070+0000", 2024-09-17T21:21:21.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338535 2024-09-17T21:21:21.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940690+0000", 2024-09-17T21:21:21.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079620503999999995 2024-09-17T21:21:21.386 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.386 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.386 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.387 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3378 3.12 3:4832a51b:::rbd_data.1152a8ae2019.000000000000006c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:21.387 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:21.387 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 100.45848399899999, 2024-09-17T21:21:21.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051812516000000003, 2024-09-17T21:21:21.387 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.387 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.387 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:21.387 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.387 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:21.387 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:21.387 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3378 2024-09-17T21:21:21.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.387 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:21.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:21.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888995+0000", 2024-09-17T21:21:21.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:21:21.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889006+0000", 2024-09-17T21:21:21.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1012999999999999e-05 2024-09-17T21:21:21.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889032+0000", 2024-09-17T21:21:21.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6441e-05 2024-09-17T21:21:21.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889040+0000", 2024-09-17T21:21:21.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0639999999999994e-06 2024-09-17T21:21:21.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889278+0000", 2024-09-17T21:21:21.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023785899999999999 2024-09-17T21:21:21.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889408+0000", 2024-09-17T21:21:21.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129219 2024-09-17T21:21:21.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:21.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889723+0000", 2024-09-17T21:21:21.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000315264 2024-09-17T21:21:21.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:21.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.891972+0000", 2024-09-17T21:21:21.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002249633 2024-09-17T21:21:21.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:21.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940733+0000", 2024-09-17T21:21:21.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048760626000000001 2024-09-17T21:21:21.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:21.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940785+0000", 2024-09-17T21:21:21.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1906000000000001e-05 2024-09-17T21:21:21.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940807+0000", 2024-09-17T21:21:21.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1554999999999999e-05 2024-09-17T21:21:21.392 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.392 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.392 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.393 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3379 3.12 3:4e83fd71:::rbd_data.1152a8ae2019.000000000000006d:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:21.393 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:21.393 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 100.45839979199999, 2024-09-17T21:21:21.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051804144000000003, 2024-09-17T21:21:21.393 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.393 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.393 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:21.393 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.393 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:21.393 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:21.393 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3379 2024-09-17T21:21:21.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.393 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:21.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:21.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889079+0000", 2024-09-17T21:21:21.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1800000000000004e-07 2024-09-17T21:21:21.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889083+0000", 2024-09-17T21:21:21.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:21:21.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889094+0000", 2024-09-17T21:21:21.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1562999999999999e-05 2024-09-17T21:21:21.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889102+0000", 2024-09-17T21:21:21.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.401e-06 2024-09-17T21:21:21.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:21:21.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014279259999999999 2024-09-17T21:21:21.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890609+0000", 2024-09-17T21:21:21.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9566000000000004e-05 2024-09-17T21:21:21.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:21.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890789+0000", 2024-09-17T21:21:21.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017949700000000001 2024-09-17T21:21:21.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:21.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.893929+0000", 2024-09-17T21:21:21.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003140538 2024-09-17T21:21:21.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:21.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940834+0000", 2024-09-17T21:21:21.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046904811999999997 2024-09-17T21:21:21.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:21.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940860+0000", 2024-09-17T21:21:21.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5341999999999999e-05 2024-09-17T21:21:21.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940882+0000", 2024-09-17T21:21:21.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2878999999999999e-05 2024-09-17T21:21:21.398 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.398 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.398 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.398 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:2644 3.1e 3:79615585:::rbd_data.115266d0b9f2.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2475520~14848,write 3106304~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:21:21.399 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:21.399 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.835771389000001, 2024-09-17T21:21:21.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050733484000000002, 2024-09-17T21:21:21.399 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.399 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.399 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:21.399 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.399 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:21:21.399 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:21:21.399 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2644 2024-09-17T21:21:21.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.399 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:21.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:21.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511708+0000", 2024-09-17T21:21:21.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-17T21:21:21.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522196+0000", 2024-09-17T21:21:21.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010487712999999999 2024-09-17T21:21:21.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522223+0000", 2024-09-17T21:21:21.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7064999999999999e-05 2024-09-17T21:21:21.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522236+0000", 2024-09-17T21:21:21.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2931e-05 2024-09-17T21:21:21.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522348+0000", 2024-09-17T21:21:21.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111906 2024-09-17T21:21:21.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522554+0000", 2024-09-17T21:21:21.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020632699999999999 2024-09-17T21:21:21.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:21:21.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.523185+0000", 2024-09-17T21:21:21.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063081400000000005 2024-09-17T21:21:21.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:21.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.547011+0000", 2024-09-17T21:21:21.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023825570000000001 2024-09-17T21:21:21.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:21.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562268+0000", 2024-09-17T21:21:21.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015257514 2024-09-17T21:21:21.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:21.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562367+0000", 2024-09-17T21:21:21.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8678000000000006e-05 2024-09-17T21:21:21.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562440+0000", 2024-09-17T21:21:21.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3337000000000006e-05 2024-09-17T21:21:21.404 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.404 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.404 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.405 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:231 3.1 3:856fed78:::rbd_data.115220dd393e.000000000000000b:head [read 0~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:21.405 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:21.405 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.393016119999999, 2024-09-17T21:21:21.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34254251699999999, 2024-09-17T21:21:21.405 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.405 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.405 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:21.405 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.405 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:21.405 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:21.405 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 231 2024-09-17T21:21:21.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.406 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:21.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:21.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954464+0000", 2024-09-17T21:21:21.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:21:21.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954475+0000", 2024-09-17T21:21:21.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1146e-05 2024-09-17T21:21:21.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954506+0000", 2024-09-17T21:21:21.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0292999999999999e-05 2024-09-17T21:21:21.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954524+0000", 2024-09-17T21:21:21.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8430000000000001e-05 2024-09-17T21:21:21.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954661+0000", 2024-09-17T21:21:21.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136403 2024-09-17T21:21:21.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:21.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954680+0000", 2024-09-17T21:21:21.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9743000000000001e-05 2024-09-17T21:21:21.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.296896+0000", 2024-09-17T21:21:21.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34221550499999998 2024-09-17T21:21:21.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297004+0000", 2024-09-17T21:21:21.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010865399999999999 2024-09-17T21:21:21.409 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.409 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.409 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.410 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:56 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:21.410 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:21.410 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.237382423, 2024-09-17T21:21:21.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18718773899999999, 2024-09-17T21:21:21.410 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.410 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.410 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:21.410 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.410 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:21:21.410 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:21:21.410 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 56 2024-09-17T21:21:21.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.410 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:21.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:21.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110098+0000", 2024-09-17T21:21:21.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:21:21.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110111+0000", 2024-09-17T21:21:21.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2289999999999999e-05 2024-09-17T21:21:21.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110143+0000", 2024-09-17T21:21:21.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1979e-05 2024-09-17T21:21:21.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110161+0000", 2024-09-17T21:21:21.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7839e-05 2024-09-17T21:21:21.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.413 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.413 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110319+0000", 2024-09-17T21:21:21.413 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015835400000000001 2024-09-17T21:21:21.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.413 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:21.413 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110353+0000", 2024-09-17T21:21:21.413 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3849000000000001e-05 2024-09-17T21:21:21.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.413 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297041+0000", 2024-09-17T21:21:21.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18668842299999999 2024-09-17T21:21:21.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297169+0000", 2024-09-17T21:21:21.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012786199999999999 2024-09-17T21:21:21.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297283+0000", 2024-09-17T21:21:21.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011430300000000001 2024-09-17T21:21:21.414 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.414 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.414 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.415 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:68 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:21.415 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:21.415 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.228841492000001, 2024-09-17T21:21:21.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17885582999999999, 2024-09-17T21:21:21.415 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.415 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.415 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:21.415 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.415 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:21:21.415 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:21:21.415 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 68 2024-09-17T21:21:21.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.416 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:21.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:21.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.417 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118639+0000", 2024-09-17T21:21:21.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.306e-06 2024-09-17T21:21:21.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.417 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118651+0000", 2024-09-17T21:21:21.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2567e-05 2024-09-17T21:21:21.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.417 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118682+0000", 2024-09-17T21:21:21.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0502e-05 2024-09-17T21:21:21.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118707+0000", 2024-09-17T21:21:21.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5463e-05 2024-09-17T21:21:21.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118889+0000", 2024-09-17T21:21:21.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018150999999999999 2024-09-17T21:21:21.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:21.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118910+0000", 2024-09-17T21:21:21.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:21:21.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297313+0000", 2024-09-17T21:21:21.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17840366699999999 2024-09-17T21:21:21.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297389+0000", 2024-09-17T21:21:21.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-17T21:21:21.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297492+0000", 2024-09-17T21:21:21.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103019 2024-09-17T21:21:21.420 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.420 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.420 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.420 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5890.0:15234 3.1 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8451072~29696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-17T21:21:21.420 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:21.420 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.254192679999999, 2024-09-17T21:21:21.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.057026052000000001, 2024-09-17T21:21:21.420 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.420 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.421 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:21.421 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.421 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5890", 2024-09-17T21:21:21.421 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3064344947", 2024-09-17T21:21:21.421 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 15234 2024-09-17T21:21:21.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.421 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.421 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.421 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:21.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:21.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093289+0000", 2024-09-17T21:21:21.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4939999999999999e-06 2024-09-17T21:21:21.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103074+0000", 2024-09-17T21:21:21.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00978564 2024-09-17T21:21:21.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103112+0000", 2024-09-17T21:21:21.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7246999999999998e-05 2024-09-17T21:21:21.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103128+0000", 2024-09-17T21:21:21.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5812999999999999e-05 2024-09-17T21:21:21.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103207+0000", 2024-09-17T21:21:21.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9785999999999998e-05 2024-09-17T21:21:21.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103315+0000", 2024-09-17T21:21:21.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107396 2024-09-17T21:21:21.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:21.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103957+0000", 2024-09-17T21:21:21.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000642524 2024-09-17T21:21:21.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:21.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.123856+0000", 2024-09-17T21:21:21.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019898376999999998 2024-09-17T21:21:21.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:21.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150190+0000", 2024-09-17T21:21:21.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026334682000000002 2024-09-17T21:21:21.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:21.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150255+0000", 2024-09-17T21:21:21.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4770999999999994e-05 2024-09-17T21:21:21.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.426 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150311+0000", 2024-09-17T21:21:21.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6322e-05 2024-09-17T21:21:21.426 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.426 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.426 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.426 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4237 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 411648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:21.426 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753637+0000", 2024-09-17T21:21:21.426 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.5938410740000002, 2024-09-17T21:21:21.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049394367000000002, 2024-09-17T21:21:21.426 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.426 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.426 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4237 2024-09-17T21:21:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753637+0000", 2024-09-17T21:21:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753637+0000", 2024-09-17T21:21:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753638+0000", 2024-09-17T21:21:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5199999999999997e-07 2024-09-17T21:21:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753640+0000", 2024-09-17T21:21:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2819999999999999e-06 2024-09-17T21:21:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753647+0000", 2024-09-17T21:21:21.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8689999999999998e-06 2024-09-17T21:21:21.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753653+0000", 2024-09-17T21:21:21.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8799999999999996e-06 2024-09-17T21:21:21.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.802605+0000", 2024-09-17T21:21:21.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048952298999999998 2024-09-17T21:21:21.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.802795+0000", 2024-09-17T21:21:21.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190173 2024-09-17T21:21:21.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803031+0000", 2024-09-17T21:21:21.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000236312 2024-09-17T21:21:21.430 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.430 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.430 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.430 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4238 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 412672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:21.431 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753660+0000", 2024-09-17T21:21:21.431 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.5938178660000002, 2024-09-17T21:21:21.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049651503999999999, 2024-09-17T21:21:21.431 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.431 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.431 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:21.431 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.431 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:21.431 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:21.431 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4238 2024-09-17T21:21:21.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.431 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.431 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.432 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753660+0000", 2024-09-17T21:21:21.432 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.432 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753660+0000", 2024-09-17T21:21:21.432 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.432 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753661+0000", 2024-09-17T21:21:21.432 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6000000000000004e-07 2024-09-17T21:21:21.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753663+0000", 2024-09-17T21:21:21.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3489999999999999e-06 2024-09-17T21:21:21.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753670+0000", 2024-09-17T21:21:21.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9539999999999998e-06 2024-09-17T21:21:21.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753676+0000", 2024-09-17T21:21:21.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6169999999999998e-06 2024-09-17T21:21:21.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803132+0000", 2024-09-17T21:21:21.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049456583999999998 2024-09-17T21:21:21.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803224+0000", 2024-09-17T21:21:21.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2139000000000007e-05 2024-09-17T21:21:21.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803312+0000", 2024-09-17T21:21:21.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7300999999999996e-05 2024-09-17T21:21:21.435 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.435 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.435 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.435 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4239 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 424960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:21.435 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:21:21.435 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.5937950230000002, 2024-09-17T21:21:21.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050123167000000003, 2024-09-17T21:21:21.435 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.435 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.435 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:21.435 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.436 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:21.436 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:21.436 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4239 2024-09-17T21:21:21.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.436 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.436 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.436 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:21:21.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.436 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.436 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:21:21.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753684+0000", 2024-09-17T21:21:21.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.37e-07 2024-09-17T21:21:21.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753686+0000", 2024-09-17T21:21:21.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.322e-06 2024-09-17T21:21:21.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753693+0000", 2024-09-17T21:21:21.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0890000000000002e-06 2024-09-17T21:21:21.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753699+0000", 2024-09-17T21:21:21.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:21:21.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803390+0000", 2024-09-17T21:21:21.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049690506000000002 2024-09-17T21:21:21.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803569+0000", 2024-09-17T21:21:21.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179784 2024-09-17T21:21:21.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803806+0000", 2024-09-17T21:21:21.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023683499999999999 2024-09-17T21:21:21.439 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.439 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.439 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.439 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4240 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 413696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:21.440 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:21.440 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.5937712949999998, 2024-09-17T21:21:21.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050331353000000002, 2024-09-17T21:21:21.440 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.440 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.440 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:21.440 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.440 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:21.440 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:21.440 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4240 2024-09-17T21:21:21.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.440 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.441 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:21.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.441 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:21.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.441 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:21.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5700000000000002e-07 2024-09-17T21:21:21.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753710+0000", 2024-09-17T21:21:21.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.385e-06 2024-09-17T21:21:21.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753717+0000", 2024-09-17T21:21:21.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.832e-06 2024-09-17T21:21:21.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.443 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753722+0000", 2024-09-17T21:21:21.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8050000000000003e-06 2024-09-17T21:21:21.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.443 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.443 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803880+0000", 2024-09-17T21:21:21.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050157641000000003 2024-09-17T21:21:21.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.443 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.443 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803945+0000", 2024-09-17T21:21:21.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5357000000000005e-05 2024-09-17T21:21:21.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804038+0000", 2024-09-17T21:21:21.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2775999999999994e-05 2024-09-17T21:21:21.444 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.444 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.444 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.444 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4241 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 415744~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:21.444 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:21.444 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.5937481790000003, 2024-09-17T21:21:21.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050578790999999998, 2024-09-17T21:21:21.444 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.445 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.445 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:21.445 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.445 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:21.445 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:21.445 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4241 2024-09-17T21:21:21.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.445 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:21.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:21.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:21.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5700000000000002e-07 2024-09-17T21:21:21.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753733+0000", 2024-09-17T21:21:21.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2570000000000002e-06 2024-09-17T21:21:21.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753740+0000", 2024-09-17T21:21:21.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.849e-06 2024-09-17T21:21:21.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753745+0000", 2024-09-17T21:21:21.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7450000000000001e-06 2024-09-17T21:21:21.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804115+0000", 2024-09-17T21:21:21.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050370038999999998 2024-09-17T21:21:21.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804223+0000", 2024-09-17T21:21:21.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010786 2024-09-17T21:21:21.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804309+0000", 2024-09-17T21:21:21.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5483999999999997e-05 2024-09-17T21:21:21.448 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.448 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.448 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.449 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4242 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 414720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:21.449 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:21.449 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.5937252910000002, 2024-09-17T21:21:21.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050748148999999999, 2024-09-17T21:21:21.449 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.449 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.449 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:21.449 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.449 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:21.449 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:21.449 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4242 2024-09-17T21:21:21.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.449 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:21.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:21.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:21.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6599999999999996e-07 2024-09-17T21:21:21.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753756+0000", 2024-09-17T21:21:21.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.368e-06 2024-09-17T21:21:21.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753763+0000", 2024-09-17T21:21:21.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9349999999999997e-06 2024-09-17T21:21:21.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753768+0000", 2024-09-17T21:21:21.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7620000000000001e-06 2024-09-17T21:21:21.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804343+0000", 2024-09-17T21:21:21.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050575050000000003 2024-09-17T21:21:21.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804397+0000", 2024-09-17T21:21:21.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4070000000000002e-05 2024-09-17T21:21:21.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804501+0000", 2024-09-17T21:21:21.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103398 2024-09-17T21:21:21.453 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.453 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.453 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.453 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.453 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.453 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:21.453 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 0, 5, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:21:21.453 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:21:21.454 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [1, 3, 4, 0, 5, 2] 2024-09-17T21:21:21.454 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-09-17T21:21:21.455 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-17T21:21:21.456 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:21:21.457 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:21.451+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:21.556 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:21.553+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:21.639 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:21.639 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:21.639 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:21:21.639 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:21.657 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:21:21.658 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:21.655+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:21.758 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:21.755+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:21.816 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:21.817 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:21.817 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:21:21.817 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:21:21.817 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:21.831 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:21:21.834 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-09-17T21:21:21.831+0000 7fe37768f640 -1 mon.a@0(leader).osd e164 definitely_dead 0 2024-09-17T21:21:21.881 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:21.879+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:21.952 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:21.952 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:21.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:21.952 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:21.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.952 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:21:21.952 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:21.952 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 162.139626967, 2024-09-17T21:21:21.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:21:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:21:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:21:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:21:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:21.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:21:21.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:21:21.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:21:21.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:21:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:21:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:21:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:21:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:21:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:21:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:21:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:21:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:21:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:21:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:21:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:21:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:21:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:21:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:21:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.957 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:21:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 148.584525181, 2024-09-17T21:21:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:21:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:21:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:21:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:21:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:21:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:21:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:21:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:21:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:21:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:21:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:21:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:21:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:21:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:21:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:21:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:21:21.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:21:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:21:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:21:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:21:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:21:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:21.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:21:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:21:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:21:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:21:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:21:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:21:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:21:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:21:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:793 3.9 3:95f63599:::rbd_data.1152eb799094.0000000000000007:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 65.517469026000001, 2024-09-17T21:21:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88245766400000003, 2024-09-17T21:21:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.964 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 793 2024-09-17T21:21:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431941+0000", 2024-09-17T21:21:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.864e-06 2024-09-17T21:21:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431953+0000", 2024-09-17T21:21:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1486e-05 2024-09-17T21:21:21.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431984+0000", 2024-09-17T21:21:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1075999999999999e-05 2024-09-17T21:21:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431999+0000", 2024-09-17T21:21:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.503e-05 2024-09-17T21:21:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432095+0000", 2024-09-17T21:21:21.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5698000000000004e-05 2024-09-17T21:21:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432109+0000", 2024-09-17T21:21:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4188e-05 2024-09-17T21:21:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.309642+0000", 2024-09-17T21:21:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87753353000000001 2024-09-17T21:21:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310104+0000", 2024-09-17T21:21:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046103600000000001 2024-09-17T21:21:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310630+0000", 2024-09-17T21:21:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052682800000000002 2024-09-17T21:21:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313100+0000", 2024-09-17T21:21:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024693369999999998 2024-09-17T21:21:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313935+0000", 2024-09-17T21:21:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083540100000000005 2024-09-17T21:21:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314086+0000", 2024-09-17T21:21:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015053399999999999 2024-09-17T21:21:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314396+0000", 2024-09-17T21:21:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031065599999999997 2024-09-17T21:21:21.970 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:800 3.19 3:9dc17f12:::rbd_data.1152eb799094.000000000000000e:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 65.517221781000003, 2024-09-17T21:21:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88154663799999999, 2024-09-17T21:21:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 800 2024-09-17T21:21:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432187+0000", 2024-09-17T21:21:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4100000000000001e-06 2024-09-17T21:21:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432194+0000", 2024-09-17T21:21:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0389999999999999e-06 2024-09-17T21:21:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432216+0000", 2024-09-17T21:21:21.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1614e-05 2024-09-17T21:21:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432232+0000", 2024-09-17T21:21:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5739e-05 2024-09-17T21:21:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432405+0000", 2024-09-17T21:21:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017355300000000001 2024-09-17T21:21:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432432+0000", 2024-09-17T21:21:21.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6979e-05 2024-09-17T21:21:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.307767+0000", 2024-09-17T21:21:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87533472899999998 2024-09-17T21:21:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308192+0000", 2024-09-17T21:21:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042485999999999999 2024-09-17T21:21:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:21.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308727+0000", 2024-09-17T21:21:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00053550400000000001 2024-09-17T21:21:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310347+0000", 2024-09-17T21:21:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016198670000000001 2024-09-17T21:21:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313424+0000", 2024-09-17T21:21:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030764640000000001 2024-09-17T21:21:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313510+0000", 2024-09-17T21:21:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5938000000000003e-05 2024-09-17T21:21:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313733+0000", 2024-09-17T21:21:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022294200000000001 2024-09-17T21:21:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.977 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:805 3.19 3:9cab2ed6:::rbd_data.1152eb799094.0000000000000013:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 65.517017761999995, 2024-09-17T21:21:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88274355699999996, 2024-09-17T21:21:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 805 2024-09-17T21:21:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432391+0000", 2024-09-17T21:21:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3e-06 2024-09-17T21:21:21.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432398+0000", 2024-09-17T21:21:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4489999999999996e-06 2024-09-17T21:21:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432415+0000", 2024-09-17T21:21:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7750000000000001e-05 2024-09-17T21:21:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432430+0000", 2024-09-17T21:21:21.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.449e-05 2024-09-17T21:21:21.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432635+0000", 2024-09-17T21:21:21.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205031 2024-09-17T21:21:21.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:21.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432657+0000", 2024-09-17T21:21:21.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2064000000000001e-05 2024-09-17T21:21:21.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310444+0000", 2024-09-17T21:21:21.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87778699199999999 2024-09-17T21:21:21.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310849+0000", 2024-09-17T21:21:21.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00040497800000000001 2024-09-17T21:21:21.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:21.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.311396+0000", 2024-09-17T21:21:21.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00054729099999999999 2024-09-17T21:21:21.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:21.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313847+0000", 2024-09-17T21:21:21.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024504940000000001 2024-09-17T21:21:21.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:21.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314879+0000", 2024-09-17T21:21:21.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001031817 2024-09-17T21:21:21.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:21.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314964+0000", 2024-09-17T21:21:21.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5685000000000004e-05 2024-09-17T21:21:21.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.315133+0000", 2024-09-17T21:21:21.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000169216 2024-09-17T21:21:21.984 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.984 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.984 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.984 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12087 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 52224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:21.984 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:21:21.985 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.179929084000001, 2024-09-17T21:21:21.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087684460000000006, 2024-09-17T21:21:21.985 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.985 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.985 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:21.985 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.985 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:21.985 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:21.985 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12087 2024-09-17T21:21:21.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.985 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:21:21.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:21:21.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769480+0000", 2024-09-17T21:21:21.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:21:21.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769485+0000", 2024-09-17T21:21:21.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8880000000000002e-06 2024-09-17T21:21:21.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769509+0000", 2024-09-17T21:21:21.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3343e-05 2024-09-17T21:21:21.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769521+0000", 2024-09-17T21:21:21.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2497000000000001e-05 2024-09-17T21:21:21.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.856883+0000", 2024-09-17T21:21:21.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087362191000000006 2024-09-17T21:21:21.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857014+0000", 2024-09-17T21:21:21.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013090399999999999 2024-09-17T21:21:21.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857163+0000", 2024-09-17T21:21:21.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014864000000000001 2024-09-17T21:21:21.989 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.989 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.989 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.989 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12088 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 56320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:21.989 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:21:21.989 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.179845428999997, 2024-09-17T21:21:21.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087917772000000005, 2024-09-17T21:21:21.989 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.989 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.989 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:21.989 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.990 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:21.990 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:21.990 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12088 2024-09-17T21:21:21.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.990 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:21:21.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:21:21.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769569+0000", 2024-09-17T21:21:21.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:21:21.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769574+0000", 2024-09-17T21:21:21.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4659999999999998e-06 2024-09-17T21:21:21.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769586+0000", 2024-09-17T21:21:21.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2311e-05 2024-09-17T21:21:21.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769597+0000", 2024-09-17T21:21:21.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0548e-05 2024-09-17T21:21:21.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857217+0000", 2024-09-17T21:21:21.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087619896000000003 2024-09-17T21:21:21.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857335+0000", 2024-09-17T21:21:21.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118538 2024-09-17T21:21:21.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857480+0000", 2024-09-17T21:21:21.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014470999999999999 2024-09-17T21:21:21.993 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.993 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.993 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.994 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12089 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 60416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:21.994 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:21.994 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.179792730000003, 2024-09-17T21:21:21.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088293233999999998, 2024-09-17T21:21:21.994 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.994 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.994 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:21.994 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.994 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:21.994 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:21.994 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12089 2024-09-17T21:21:21.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.995 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:21.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:21.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:21.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:21.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769616+0000", 2024-09-17T21:21:21.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:21:21.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:21.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769620+0000", 2024-09-17T21:21:21.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5140000000000002e-06 2024-09-17T21:21:21.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:21.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769633+0000", 2024-09-17T21:21:21.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2849e-05 2024-09-17T21:21:21.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:21.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769644+0000", 2024-09-17T21:21:21.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1471000000000001e-05 2024-09-17T21:21:21.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:21.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857564+0000", 2024-09-17T21:21:21.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087919445999999998 2024-09-17T21:21:21.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:21.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857716+0000", 2024-09-17T21:21:21.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000152065 2024-09-17T21:21:21.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:21.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857908+0000", 2024-09-17T21:21:21.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019252599999999999 2024-09-17T21:21:21.998 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.998 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:21.998 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:21.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.998 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12090 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 64512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:21.998 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:21.998 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.179746344999998, 2024-09-17T21:21:21.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088582672000000001, 2024-09-17T21:21:21.998 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:21.999 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:21.999 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:21.999 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:21.999 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:21.999 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:21.999 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12090 2024-09-17T21:21:21.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.999 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:21.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:21.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:21.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:21.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:21.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:21.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:22.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769662+0000", 2024-09-17T21:21:22.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:21:22.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769683+0000", 2024-09-17T21:21:22.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0979e-05 2024-09-17T21:21:22.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769699+0000", 2024-09-17T21:21:22.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5446000000000001e-05 2024-09-17T21:21:22.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769708+0000", 2024-09-17T21:21:22.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1679999999999993e-06 2024-09-17T21:21:22.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857976+0000", 2024-09-17T21:21:22.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088267795999999996 2024-09-17T21:21:22.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858095+0000", 2024-09-17T21:21:22.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119147 2024-09-17T21:21:22.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858244+0000", 2024-09-17T21:21:22.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014917000000000001 2024-09-17T21:21:22.002 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.002 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.003 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.003 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12091 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 68608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:22.003 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:22.003 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.179674961000003, 2024-09-17T21:21:22.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088832752000000001, 2024-09-17T21:21:22.003 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.003 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.003 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:22.003 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:22.003 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:22.003 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:22.003 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12091 2024-09-17T21:21:22.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.004 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:22.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:22.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769734+0000", 2024-09-17T21:21:22.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:21:22.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769739+0000", 2024-09-17T21:21:22.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8080000000000004e-06 2024-09-17T21:21:22.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769764+0000", 2024-09-17T21:21:22.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4839999999999999e-05 2024-09-17T21:21:22.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769777+0000", 2024-09-17T21:21:22.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3037000000000001e-05 2024-09-17T21:21:22.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858295+0000", 2024-09-17T21:21:22.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088518475999999999 2024-09-17T21:21:22.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858419+0000", 2024-09-17T21:21:22.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012378700000000001 2024-09-17T21:21:22.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858565+0000", 2024-09-17T21:21:22.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014630099999999999 2024-09-17T21:21:22.007 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.007 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.007 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.007 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12092 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:22.007 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:22.007 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.179592636999999, 2024-09-17T21:21:22.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089191555000000006, 2024-09-17T21:21:22.008 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.008 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.008 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:22.008 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:22.008 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:22.008 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:22.008 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12092 2024-09-17T21:21:22.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.008 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.008 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.008 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:22.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:22.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769816+0000", 2024-09-17T21:21:22.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7199999999999997e-07 2024-09-17T21:21:22.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769822+0000", 2024-09-17T21:21:22.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:21:22.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769838+0000", 2024-09-17T21:21:22.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6674e-05 2024-09-17T21:21:22.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769851+0000", 2024-09-17T21:21:22.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2527999999999999e-05 2024-09-17T21:21:22.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.011 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858660+0000", 2024-09-17T21:21:22.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088808556999999996 2024-09-17T21:21:22.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.011 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.011 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858817+0000", 2024-09-17T21:21:22.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015730400000000001 2024-09-17T21:21:22.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.011 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.011 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859007+0000", 2024-09-17T21:21:22.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018974899999999999 2024-09-17T21:21:22.011 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.011 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.012 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.012 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12093 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:22.012 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:22.012 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.179484533, 2024-09-17T21:21:22.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089429710999999995, 2024-09-17T21:21:22.012 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.012 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.012 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:22.012 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:22.012 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:22.012 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:22.013 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12093 2024-09-17T21:21:22.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.013 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:22.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:22.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769924+0000", 2024-09-17T21:21:22.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.342e-06 2024-09-17T21:21:22.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769929+0000", 2024-09-17T21:21:22.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0350000000000001e-06 2024-09-17T21:21:22.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769946+0000", 2024-09-17T21:21:22.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6793999999999999e-05 2024-09-17T21:21:22.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769959+0000", 2024-09-17T21:21:22.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3062e-05 2024-09-17T21:21:22.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859075+0000", 2024-09-17T21:21:22.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089115696999999994 2024-09-17T21:21:22.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859202+0000", 2024-09-17T21:21:22.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012657 2024-09-17T21:21:22.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.016 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.016 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859353+0000", 2024-09-17T21:21:22.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151211 2024-09-17T21:21:22.016 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.016 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.016 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.016 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12094 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:22.016 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:22.016 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.179430357000001, 2024-09-17T21:21:22.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089695001999999996, 2024-09-17T21:21:22.017 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.017 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.017 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:22.017 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:22.017 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:22.017 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:22.017 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12094 2024-09-17T21:21:22.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.017 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.017 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.017 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:22.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:22.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769978+0000", 2024-09-17T21:21:22.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.142e-06 2024-09-17T21:21:22.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769983+0000", 2024-09-17T21:21:22.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.386e-06 2024-09-17T21:21:22.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769997+0000", 2024-09-17T21:21:22.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.416e-05 2024-09-17T21:21:22.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770012+0000", 2024-09-17T21:21:22.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5384999999999999e-05 2024-09-17T21:21:22.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859405+0000", 2024-09-17T21:21:22.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089392376999999995 2024-09-17T21:21:22.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859525+0000", 2024-09-17T21:21:22.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120555 2024-09-17T21:21:22.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859672+0000", 2024-09-17T21:21:22.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000146997 2024-09-17T21:21:22.021 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.021 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.021 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.021 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12095 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:22.021 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:22.021 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.179377870000003, 2024-09-17T21:21:22.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090073476, 2024-09-17T21:21:22.021 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.021 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.021 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:22.022 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:22.022 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:22.022 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:22.022 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12095 2024-09-17T21:21:22.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.022 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.022 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.022 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:22.022 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.022 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.022 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:22.023 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.023 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.023 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770031+0000", 2024-09-17T21:21:22.023 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1699999999999997e-07 2024-09-17T21:21:22.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.023 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.023 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770036+0000", 2024-09-17T21:21:22.023 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.023e-06 2024-09-17T21:21:22.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.023 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770051+0000", 2024-09-17T21:21:22.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4953999999999999e-05 2024-09-17T21:21:22.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770063+0000", 2024-09-17T21:21:22.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:21:22.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859756+0000", 2024-09-17T21:21:22.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089692980000000005 2024-09-17T21:21:22.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859891+0000", 2024-09-17T21:21:22.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013459499999999999 2024-09-17T21:21:22.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860103+0000", 2024-09-17T21:21:22.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021274400000000001 2024-09-17T21:21:22.025 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.025 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.025 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.026 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12096 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:22.026 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:22.026 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.179328185000003, 2024-09-17T21:21:22.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090358274000000002, 2024-09-17T21:21:22.026 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.026 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.026 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:22.026 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:22.026 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:22.026 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:22.026 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12096 2024-09-17T21:21:22.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.026 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.027 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.027 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:22.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.027 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.027 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:22.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.027 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.027 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770080+0000", 2024-09-17T21:21:22.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9400000000000004e-07 2024-09-17T21:21:22.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770085+0000", 2024-09-17T21:21:22.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2169999999999996e-06 2024-09-17T21:21:22.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770100+0000", 2024-09-17T21:21:22.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5077e-05 2024-09-17T21:21:22.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770112+0000", 2024-09-17T21:21:22.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1951e-05 2024-09-17T21:21:22.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860160+0000", 2024-09-17T21:21:22.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090048413999999993 2024-09-17T21:21:22.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860285+0000", 2024-09-17T21:21:22.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012466699999999999 2024-09-17T21:21:22.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860438+0000", 2024-09-17T21:21:22.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015305399999999999 2024-09-17T21:21:22.030 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.030 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.030 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.030 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12097 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:22.030 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:22.030 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.179280437999999, 2024-09-17T21:21:22.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090702375000000002, 2024-09-17T21:21:22.030 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.030 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.030 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:22.031 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:22.031 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:22.031 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:22.031 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12097 2024-09-17T21:21:22.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.031 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:22.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:22.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.032 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770128+0000", 2024-09-17T21:21:22.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.04e-06 2024-09-17T21:21:22.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.032 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770133+0000", 2024-09-17T21:21:22.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8999999999999997e-06 2024-09-17T21:21:22.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770152+0000", 2024-09-17T21:21:22.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8890999999999999e-05 2024-09-17T21:21:22.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770164+0000", 2024-09-17T21:21:22.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1769e-05 2024-09-17T21:21:22.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860511+0000", 2024-09-17T21:21:22.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090347527999999996 2024-09-17T21:21:22.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.034 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860652+0000", 2024-09-17T21:21:22.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140894 2024-09-17T21:21:22.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.034 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860830+0000", 2024-09-17T21:21:22.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017735299999999999 2024-09-17T21:21:22.034 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.034 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.034 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.034 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12098 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:22.035 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:22.035 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.179226970000002, 2024-09-17T21:21:22.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090997262999999995, 2024-09-17T21:21:22.035 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.035 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.035 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:22.035 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:22.035 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:22.035 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:22.035 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12098 2024-09-17T21:21:22.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.035 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:22.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:22.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775465+0000", 2024-09-17T21:21:22.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0052843350000000002 2024-09-17T21:21:22.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775476+0000", 2024-09-17T21:21:22.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0797e-05 2024-09-17T21:21:22.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775506+0000", 2024-09-17T21:21:22.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9709999999999998e-05 2024-09-17T21:21:22.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775530+0000", 2024-09-17T21:21:22.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4528999999999998e-05 2024-09-17T21:21:22.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.038 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.038 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860894+0000", 2024-09-17T21:21:22.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.085364195000000004 2024-09-17T21:21:22.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.038 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.038 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861019+0000", 2024-09-17T21:21:22.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012467399999999999 2024-09-17T21:21:22.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.038 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.038 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861178+0000", 2024-09-17T21:21:22.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159023 2024-09-17T21:21:22.039 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.039 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.039 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.039 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12104 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:22.039 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:22.039 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.173525861000002, 2024-09-17T21:21:22.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087969628999999994, 2024-09-17T21:21:22.039 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.039 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.039 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:22.039 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:22.039 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:22.040 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:22.040 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12104 2024-09-17T21:21:22.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.040 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:22.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:22.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775883+0000", 2024-09-17T21:21:22.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3399999999999998e-07 2024-09-17T21:21:22.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775886+0000", 2024-09-17T21:21:22.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2119999999999999e-06 2024-09-17T21:21:22.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775895+0000", 2024-09-17T21:21:22.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8710000000000003e-06 2024-09-17T21:21:22.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775901+0000", 2024-09-17T21:21:22.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:21:22.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863544+0000", 2024-09-17T21:21:22.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087642642000000007 2024-09-17T21:21:22.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863694+0000", 2024-09-17T21:21:22.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014952300000000001 2024-09-17T21:21:22.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863851+0000", 2024-09-17T21:21:22.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000157804 2024-09-17T21:21:22.043 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.043 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.043 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.043 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12105 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:22.044 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:22.044 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.173496966999998, 2024-09-17T21:21:22.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088330400000000003, 2024-09-17T21:21:22.044 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.044 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.044 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:22.044 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:22.044 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:22.044 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:22.044 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12105 2024-09-17T21:21:22.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.044 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:22.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:22.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:22.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9999999999999997e-07 2024-09-17T21:21:22.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775914+0000", 2024-09-17T21:21:22.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4710000000000001e-06 2024-09-17T21:21:22.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775921+0000", 2024-09-17T21:21:22.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:21:22.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775928+0000", 2024-09-17T21:21:22.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:21:22.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863923+0000", 2024-09-17T21:21:22.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087995126000000007 2024-09-17T21:21:22.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864059+0000", 2024-09-17T21:21:22.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135797 2024-09-17T21:21:22.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864241+0000", 2024-09-17T21:21:22.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018252599999999999 2024-09-17T21:21:22.048 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.048 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.048 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.048 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12106 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:22.048 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:22.048 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.173407789000002, 2024-09-17T21:21:22.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088586394999999998, 2024-09-17T21:21:22.048 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.048 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.048 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:22.048 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:22.048 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:22.049 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:22.049 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12106 2024-09-17T21:21:22.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.049 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.049 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.049 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:22.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.049 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.049 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:22.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.050 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.050 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776001+0000", 2024-09-17T21:21:22.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4300000000000001e-07 2024-09-17T21:21:22.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.050 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.050 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776004+0000", 2024-09-17T21:21:22.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8049999999999998e-06 2024-09-17T21:21:22.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.050 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.050 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776018+0000", 2024-09-17T21:21:22.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4613999999999999e-05 2024-09-17T21:21:22.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776025+0000", 2024-09-17T21:21:22.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:21:22.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864279+0000", 2024-09-17T21:21:22.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088253570000000003 2024-09-17T21:21:22.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864433+0000", 2024-09-17T21:21:22.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015469400000000001 2024-09-17T21:21:22.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864586+0000", 2024-09-17T21:21:22.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000153003 2024-09-17T21:21:22.052 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.052 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.052 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.052 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.052 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.052 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:22.054 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:21:22.054 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:22.049+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:22.155 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:22.151+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:22.224 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:22.224 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:22.224 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:21:22.224 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:22.238 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:21:22.255 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:22.253+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:22.356 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:22.355+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:22.380 INFO:teuthology.orchestra.run.smithi083.stderr:marked out osd.0. 2024-09-17T21:21:22.416 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:21:22.416 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:21:22.416 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:22.430 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:21:22.457 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:22.453+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:22.502 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:21:22.558 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:22.553+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:22.590 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:22.590 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:22.590 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:22.591 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:22.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.591 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4423.0:787 3.9 e28/23 3:9797fd44:::rbd_object_map.1152c6af092e:head v 28'67, mlcod=28'60)", 2024-09-17T21:21:22.591 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:21:22.591 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 184.371127401, 2024-09-17T21:21:22.591 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063811709999999997, 2024-09-17T21:21:22.591 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.591 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.591 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:22.591 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.591 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.591 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:21:22.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.592 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217360+0000", 2024-09-17T21:21:22.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.592 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217362+0000", 2024-09-17T21:21:22.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:21:22.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.593 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.593 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217377+0000", 2024-09-17T21:21:22.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5393e-05 2024-09-17T21:21:22.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.593 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.593 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217401+0000", 2024-09-17T21:21:22.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3289999999999999e-05 2024-09-17T21:21:22.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.593 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.593 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.217429+0000", 2024-09-17T21:21:22.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7973000000000001e-05 2024-09-17T21:21:22.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.594 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.594 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220009+0000", 2024-09-17T21:21:22.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025797630000000001 2024-09-17T21:21:22.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.594 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.594 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.220059+0000", 2024-09-17T21:21:22.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0157000000000001e-05 2024-09-17T21:21:22.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.595 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:22.595 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223626+0000", 2024-09-17T21:21:22.595 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035678060000000002 2024-09-17T21:21:22.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.595 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.595 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:18.223741+0000", 2024-09-17T21:21:22.595 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115009 2024-09-17T21:21:22.595 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.595 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.595 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.595 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:21:22.596 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:22.596 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 179.72082673700001, 2024-09-17T21:21:22.596 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:21:22.596 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.596 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.596 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:22.596 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.596 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.596 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:22.596 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.596 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:22.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:21:22.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:21:22.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:21:22.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:21:22.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:21:22.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:21:22.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:21:22.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:21:22.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:21:22.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:21:22.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:21:22.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:21:22.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:22.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:21:22.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:21:22.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:21:22.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:21:22.600 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.600 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.600 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.600 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:21:22.600 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:22.600 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 179.71956165399999, 2024-09-17T21:21:22.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:21:22.600 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.600 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.600 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:22.600 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.601 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:22.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.601 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:22.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.601 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:21:22.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:21:22.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.602 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:21:22.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:21:22.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.602 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:21:22.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:21:22.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.602 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:21:22.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:21:22.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:21:22.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:21:22.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:21:22.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:21:22.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:22.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:21:22.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:21:22.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:21:22.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:21:22.604 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.604 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.604 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.604 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:21:22.605 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:22.605 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 179.709782244, 2024-09-17T21:21:22.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:21:22.605 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.605 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.605 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:22.605 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.605 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.605 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:22.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:22.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:21:22.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:21:22.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:21:22.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:21:22.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:21:22.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:21:22.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:21:22.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:21:22.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:21:22.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:21:22.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:21:22.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:21:22.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:22.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:21:22.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:21:22.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:21:22.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:21:22.609 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.609 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.609 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.609 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:21:22.609 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:22.609 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 179.70105984099999, 2024-09-17T21:21:22.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:21:22.609 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.609 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.609 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:22.609 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:22.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:22.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:21:22.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:21:22.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:21:22.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:21:22.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:21:22.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:21:22.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:21:22.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:21:22.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:21:22.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:21:22.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:21:22.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:21:22.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:22.613 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:21:22.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:21:22.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.613 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:21:22.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:21:22.613 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.613 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.613 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.613 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:21:22.613 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:22.613 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 166.31596927699999, 2024-09-17T21:21:22.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:21:22.614 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.614 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.614 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:22.614 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:22.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:22.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:21:22.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:21:22.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:21:22.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:21:22.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:21:22.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:21:22.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:21:22.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:21:22.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:21:22.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:21:22.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:21:22.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:21:22.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:22.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:21:22.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:21:22.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:21:22.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:21:22.617 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.617 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.617 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.618 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1780 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'195, mlcod=69'190)", 2024-09-17T21:21:22.618 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:22.618 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 129.33811367300001, 2024-09-17T21:21:22.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075761769999999999, 2024-09-17T21:21:22.618 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.618 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.618 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:22.618 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:22.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:22.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250375+0000", 2024-09-17T21:21:22.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:21:22.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250383+0000", 2024-09-17T21:21:22.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7870000000000002e-06 2024-09-17T21:21:22.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250402+0000", 2024-09-17T21:21:22.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9346000000000001e-05 2024-09-17T21:21:22.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250413+0000", 2024-09-17T21:21:22.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.039e-05 2024-09-17T21:21:22.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251353+0000", 2024-09-17T21:21:22.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000940091 2024-09-17T21:21:22.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251407+0000", 2024-09-17T21:21:22.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4701999999999999e-05 2024-09-17T21:21:22.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:22.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257829+0000", 2024-09-17T21:21:22.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064212779999999999 2024-09-17T21:21:22.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257950+0000", 2024-09-17T21:21:22.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121623 2024-09-17T21:21:22.621 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.622 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.622 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.622 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1781 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'196, mlcod=69'190)", 2024-09-17T21:21:22.622 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:21:22.622 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 129.33703072200001, 2024-09-17T21:21:22.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065632939999999999, 2024-09-17T21:21:22.622 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.622 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.622 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:22.622 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:21:22.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:21:22.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251459+0000", 2024-09-17T21:21:22.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.624e-06 2024-09-17T21:21:22.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251471+0000", 2024-09-17T21:21:22.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2636000000000001e-05 2024-09-17T21:21:22.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251498+0000", 2024-09-17T21:21:22.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:21:22.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251515+0000", 2024-09-17T21:21:22.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6939000000000002e-05 2024-09-17T21:21:22.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252938+0000", 2024-09-17T21:21:22.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001423082 2024-09-17T21:21:22.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252999+0000", 2024-09-17T21:21:22.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1138999999999999e-05 2024-09-17T21:21:22.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:22.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257955+0000", 2024-09-17T21:21:22.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049559640000000002 2024-09-17T21:21:22.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.258020+0000", 2024-09-17T21:21:22.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5504000000000002e-05 2024-09-17T21:21:22.626 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.626 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.626 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.626 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1788 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'198, mlcod=69'191)", 2024-09-17T21:21:22.626 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:22.626 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 129.335005233, 2024-09-17T21:21:22.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073690370000000002, 2024-09-17T21:21:22.626 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.626 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.627 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:22.627 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:22.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:22.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253484+0000", 2024-09-17T21:21:22.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2470000000000001e-06 2024-09-17T21:21:22.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253492+0000", 2024-09-17T21:21:22.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7159999999999992e-06 2024-09-17T21:21:22.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253509+0000", 2024-09-17T21:21:22.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.666e-05 2024-09-17T21:21:22.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253522+0000", 2024-09-17T21:21:22.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2643000000000001e-05 2024-09-17T21:21:22.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254853+0000", 2024-09-17T21:21:22.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331238 2024-09-17T21:21:22.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254885+0000", 2024-09-17T21:21:22.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1562e-05 2024-09-17T21:21:22.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:22.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260788+0000", 2024-09-17T21:21:22.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059030749999999998 2024-09-17T21:21:22.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260852+0000", 2024-09-17T21:21:22.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3895999999999999e-05 2024-09-17T21:21:22.630 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.630 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.630 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.630 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5746 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'477, mlcod=71'473)", 2024-09-17T21:21:22.630 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:22.631 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 126.329577675, 2024-09-17T21:21:22.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074182709999999997, 2024-09-17T21:21:22.631 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.631 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.631 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:22.631 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:22.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:22.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258912+0000", 2024-09-17T21:21:22.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:21:22.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258926+0000", 2024-09-17T21:21:22.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4249e-05 2024-09-17T21:21:22.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258946+0000", 2024-09-17T21:21:22.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9879999999999999e-05 2024-09-17T21:21:22.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258956+0000", 2024-09-17T21:21:22.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:21:22.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259380+0000", 2024-09-17T21:21:22.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042337599999999999 2024-09-17T21:21:22.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259412+0000", 2024-09-17T21:21:22.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:21:22.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:22.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266191+0000", 2024-09-17T21:21:22.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067793269999999999 2024-09-17T21:21:22.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266328+0000", 2024-09-17T21:21:22.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013690999999999999 2024-09-17T21:21:22.634 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5747 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'478, mlcod=71'473)", 2024-09-17T21:21:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 126.328871326, 2024-09-17T21:21:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068779440000000004, 2024-09-17T21:21:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259618+0000", 2024-09-17T21:21:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6199999999999999e-06 2024-09-17T21:21:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259630+0000", 2024-09-17T21:21:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1884e-05 2024-09-17T21:21:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259657+0000", 2024-09-17T21:21:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7339e-05 2024-09-17T21:21:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259677+0000", 2024-09-17T21:21:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0156000000000001e-05 2024-09-17T21:21:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260521+0000", 2024-09-17T21:21:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084312899999999997 2024-09-17T21:21:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260585+0000", 2024-09-17T21:21:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4558000000000001e-05 2024-09-17T21:21:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266336+0000", 2024-09-17T21:21:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057505020000000002 2024-09-17T21:21:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266494+0000", 2024-09-17T21:21:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158756 2024-09-17T21:21:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5751 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'480, mlcod=71'473)", 2024-09-17T21:21:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 126.32745192100001, 2024-09-17T21:21:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075756299999999999, 2024-09-17T21:21:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261037+0000", 2024-09-17T21:21:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:21:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261049+0000", 2024-09-17T21:21:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1063999999999999e-05 2024-09-17T21:21:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261072+0000", 2024-09-17T21:21:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3102e-05 2024-09-17T21:21:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261133+0000", 2024-09-17T21:21:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1276000000000001e-05 2024-09-17T21:21:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:21:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015334319999999999 2024-09-17T21:21:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262702+0000", 2024-09-17T21:21:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5281999999999999e-05 2024-09-17T21:21:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268527+0000", 2024-09-17T21:21:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058250389999999997 2024-09-17T21:21:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268611+0000", 2024-09-17T21:21:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4724999999999999e-05 2024-09-17T21:21:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5756 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'481, mlcod=71'473)", 2024-09-17T21:21:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 126.326591065, 2024-09-17T21:21:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006772386, 2024-09-17T21:21:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261899+0000", 2024-09-17T21:21:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.903e-06 2024-09-17T21:21:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261912+0000", 2024-09-17T21:21:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-05 2024-09-17T21:21:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261938+0000", 2024-09-17T21:21:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6098999999999999e-05 2024-09-17T21:21:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261955+0000", 2024-09-17T21:21:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.696e-05 2024-09-17T21:21:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263845+0000", 2024-09-17T21:21:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001890187 2024-09-17T21:21:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263907+0000", 2024-09-17T21:21:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1600999999999995e-05 2024-09-17T21:21:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268615+0000", 2024-09-17T21:21:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047086070000000001 2024-09-17T21:21:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268669+0000", 2024-09-17T21:21:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3588999999999998e-05 2024-09-17T21:21:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4978.0:598 3.9 e80/23 3:9701cc17:::rbd_data.1152724ff2a3.0000000000000019:head v 80'202, mlcod=80'198)", 2024-09-17T21:21:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:21:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 115.825016682, 2024-09-17T21:21:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066473340000000004, 2024-09-17T21:21:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:21:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:21:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763472+0000", 2024-09-17T21:21:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6899999999999997e-07 2024-09-17T21:21:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763478+0000", 2024-09-17T21:21:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:21:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763489+0000", 2024-09-17T21:21:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0628999999999999e-05 2024-09-17T21:21:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763495+0000", 2024-09-17T21:21:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6649999999999997e-06 2024-09-17T21:21:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763907+0000", 2024-09-17T21:21:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041163899999999999 2024-09-17T21:21:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763976+0000", 2024-09-17T21:21:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8534999999999993e-05 2024-09-17T21:21:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.769900+0000", 2024-09-17T21:21:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059243270000000001 2024-09-17T21:21:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.770118+0000", 2024-09-17T21:21:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021842500000000001 2024-09-17T21:21:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:468 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'94, mlcod=89'87)", 2024-09-17T21:21:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 102.902271383, 2024-09-17T21:21:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066528790000000004, 2024-09-17T21:21:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686218+0000", 2024-09-17T21:21:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3060000000000001e-06 2024-09-17T21:21:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686256+0000", 2024-09-17T21:21:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8333e-05 2024-09-17T21:21:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686272+0000", 2024-09-17T21:21:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:21:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686283+0000", 2024-09-17T21:21:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0903e-05 2024-09-17T21:21:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686682+0000", 2024-09-17T21:21:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039935099999999999 2024-09-17T21:21:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686785+0000", 2024-09-17T21:21:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010275199999999999 2024-09-17T21:21:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692816+0000", 2024-09-17T21:21:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060317579999999999 2024-09-17T21:21:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692869+0000", 2024-09-17T21:21:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2775999999999998e-05 2024-09-17T21:21:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:755 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'127, mlcod=89'122)", 2024-09-17T21:21:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 102.765816169, 2024-09-17T21:21:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084014080000000008, 2024-09-17T21:21:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:21:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822683+0000", 2024-09-17T21:21:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0674e-05 2024-09-17T21:21:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822698+0000", 2024-09-17T21:21:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5189e-05 2024-09-17T21:21:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822708+0000", 2024-09-17T21:21:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9119999999999993e-06 2024-09-17T21:21:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822748+0000", 2024-09-17T21:21:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9727000000000001e-05 2024-09-17T21:21:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822759+0000", 2024-09-17T21:21:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0865e-05 2024-09-17T21:21:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831034+0000", 2024-09-17T21:21:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082750710000000002 2024-09-17T21:21:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831073+0000", 2024-09-17T21:21:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9063999999999997e-05 2024-09-17T21:21:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:756 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'128, mlcod=89'124)", 2024-09-17T21:21:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 102.763893537, 2024-09-17T21:21:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0076300270000000002, 2024-09-17T21:21:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824595+0000", 2024-09-17T21:21:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-17T21:21:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824600+0000", 2024-09-17T21:21:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:21:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824612+0000", 2024-09-17T21:21:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.181e-05 2024-09-17T21:21:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824619+0000", 2024-09-17T21:21:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:21:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824671+0000", 2024-09-17T21:21:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2262999999999998e-05 2024-09-17T21:21:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824695+0000", 2024-09-17T21:21:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3883000000000002e-05 2024-09-17T21:21:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832185+0000", 2024-09-17T21:21:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007490048 2024-09-17T21:21:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832224+0000", 2024-09-17T21:21:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8957e-05 2024-09-17T21:21:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2635 3.d e90/23 3:b3d1e920:::rbd_data.1152a8ae2019.000000000000001f:head v 90'252, mlcod=90'246)", 2024-09-17T21:21:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 101.741334467, 2024-09-17T21:21:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0077265190000000003, 2024-09-17T21:21:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847155+0000", 2024-09-17T21:21:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.544e-06 2024-09-17T21:21:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847165+0000", 2024-09-17T21:21:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.043e-05 2024-09-17T21:21:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847189+0000", 2024-09-17T21:21:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3659000000000002e-05 2024-09-17T21:21:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847210+0000", 2024-09-17T21:21:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1163e-05 2024-09-17T21:21:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848856+0000", 2024-09-17T21:21:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001645521 2024-09-17T21:21:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848905+0000", 2024-09-17T21:21:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9048999999999997e-05 2024-09-17T21:21:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854770+0000", 2024-09-17T21:21:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058652879999999998 2024-09-17T21:21:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854880+0000", 2024-09-17T21:21:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010986500000000001 2024-09-17T21:21:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "MOSDPGPush(3.5 151/149 [PushOp(3:a57cb214:::rbd_data.115262b8e217.0000000000000009:1d, version: 103'698, data_included: [196608~24576,229376~24576,262144~24576,294912~131072,524288~98304,819200~8192,843776~16384,876544~16384,909312~16384,942080~8192,1146880~16384,1179648~16384,1212416~16384,1474560~98304,1794048~8192,1826816~8192,1859584~8192,2105344~24576,2138112~24576,2170880~24576,2646016~8192,2678784~8192,2711552~8192,3072000~8192,3104768~8192,3137536~106496], data_size: 761856, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:a57cb214:::rbd_data.115262b8e217.0000000000000009:1d@103'698, size: 3276800, copy_subset: [(0, 3276800)], clone_subset: {}, snapset: 1d=[]:{1d: [1d]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3276800, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T21:21:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:07.006386+0000", 2024-09-17T21:21:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.582102007, 2024-09-17T21:21:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082699420000000006, 2024-09-17T21:21:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.006386+0000", 2024-09-17T21:21:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.006386+0000", 2024-09-17T21:21:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.006388+0000", 2024-09-17T21:21:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.827e-06 2024-09-17T21:21:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.007671+0000", 2024-09-17T21:21:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0012836340000000001 2024-09-17T21:21:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.007737+0000", 2024-09-17T21:21:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5815000000000006e-05 2024-09-17T21:21:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.007782+0000", 2024-09-17T21:21:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5509000000000001e-05 2024-09-17T21:21:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.008023+0000", 2024-09-17T21:21:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000240184 2024-09-17T21:21:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.008065+0000", 2024-09-17T21:21:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1978999999999999e-05 2024-09-17T21:21:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.014656+0000", 2024-09-17T21:21:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065909940000000002 2024-09-17T21:21:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(osd.3.0:3834 3.8 e165/154 3:11ef155a:::rbd_data.11527217c540.000000000000003f:2b v 165'498, mlcod=152'495)", 2024-09-17T21:21:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:22.375046+0000", 2024-09-17T21:21:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.213441822, 2024-09-17T21:21:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010228170999999999, 2024-09-17T21:21:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375046+0000", 2024-09-17T21:21:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375046+0000", 2024-09-17T21:21:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375047+0000", 2024-09-17T21:21:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.373e-06 2024-09-17T21:21:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375085+0000", 2024-09-17T21:21:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7376999999999998e-05 2024-09-17T21:21:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375127+0000", 2024-09-17T21:21:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1941999999999999e-05 2024-09-17T21:21:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375150+0000", 2024-09-17T21:21:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3113e-05 2024-09-17T21:21:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375261+0000", 2024-09-17T21:21:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111402 2024-09-17T21:21:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375287+0000", 2024-09-17T21:21:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5585e-05 2024-09-17T21:21:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.385224+0000", 2024-09-17T21:21:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0099369750000000007 2024-09-17T21:21:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:22.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:22.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.385274+0000", 2024-09-17T21:21:22.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0404000000000001e-05 2024-09-17T21:21:22.680 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.680 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.680 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.680 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:22.680 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:22.680 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:22.682 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:21:22.682 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:22.682 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:22.683 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:22.683 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:22.679+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:22.691 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:21:22.691 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:21:22.711 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:22.711 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:22.730 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:22.730 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:22.749 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:22.749 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:22.782 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:22.777+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:22.814 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:22.814 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:22.829 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:21:22.883 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:22.879+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:22.983 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:22.979+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:22.996 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:22.996 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:23.010 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:21:23.084 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:23.083+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:23.140 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:23.140 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:23.154 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:21:23.186 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:23.183+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:23.277 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:23.277 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:23.287 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:23.283+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:23.291 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:21:23.388 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:23.383+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:23.414 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:23.414 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:23.428 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:21:23.458 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:21:23.489 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:23.485+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:23.550 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:23.550 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:23.564 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:21:23.589 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:23.587+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:23.690 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:23.685+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:23.737 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:23.737 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:23.751 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:21:23.795 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:23.793+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:23.891 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:23.887+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:23.918 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:23.919 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:23.932 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:21:23.992 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:23.989+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:24.050 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:24.050 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:24.064 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:21:24.107 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:24.105+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:24.194 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:24.191+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:24.197 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:24.197 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:24.211 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:21:24.297 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:24.295+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:24.371 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-17T21:21:24.371 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:24.371 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:24.398 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:24.391+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:24.401 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:21:24.501 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:24.499+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:24.597 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:24.595+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:24.597 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:24.597 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:24.615 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:21:24.705 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:24.703+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:24.767 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:21:24.802 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:24.802 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:24.803 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:24.799+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:24.821 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:21:24.917 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:24.915+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:24.937 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:24.937 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:24.959 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:24.960 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:24.982 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:21:24.982 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:21:25.000 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:24.995+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:25.005 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:25.005 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:25.016 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:25.016 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:25.016 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:21:25.016 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:21:25.016 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:25.028 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:25.028 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:25.035 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:21:25.050 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:25.117 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:25.115+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:25.192 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:21:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 165.38028141500001, 2024-09-17T21:21:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:21:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:21:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:21:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:21:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:21:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:21:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:21:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:21:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:21:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:21:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:21:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:21:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:21:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:21:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:21:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:21:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:21:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:21:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:21:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:21:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:21:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:21:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:21:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 151.82517962899999, 2024-09-17T21:21:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:21:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:21:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:21:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:21:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:21:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:21:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:21:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:21:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:21:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:21:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:21:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:21:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:21:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:21:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:21:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:21:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:21:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:21:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:21:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:21:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:21:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:21:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:21:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:21:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:21:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:21:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:21:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:21:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:21:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:793 3.9 3:95f63599:::rbd_data.1152eb799094.0000000000000007:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 68.758123474000001, 2024-09-17T21:21:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88245766400000003, 2024-09-17T21:21:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 793 2024-09-17T21:21:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431941+0000", 2024-09-17T21:21:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.864e-06 2024-09-17T21:21:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431953+0000", 2024-09-17T21:21:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1486e-05 2024-09-17T21:21:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431984+0000", 2024-09-17T21:21:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1075999999999999e-05 2024-09-17T21:21:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431999+0000", 2024-09-17T21:21:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.503e-05 2024-09-17T21:21:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432095+0000", 2024-09-17T21:21:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5698000000000004e-05 2024-09-17T21:21:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432109+0000", 2024-09-17T21:21:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4188e-05 2024-09-17T21:21:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.309642+0000", 2024-09-17T21:21:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87753353000000001 2024-09-17T21:21:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310104+0000", 2024-09-17T21:21:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046103600000000001 2024-09-17T21:21:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310630+0000", 2024-09-17T21:21:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052682800000000002 2024-09-17T21:21:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313100+0000", 2024-09-17T21:21:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024693369999999998 2024-09-17T21:21:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313935+0000", 2024-09-17T21:21:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083540100000000005 2024-09-17T21:21:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314086+0000", 2024-09-17T21:21:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015053399999999999 2024-09-17T21:21:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314396+0000", 2024-09-17T21:21:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031065599999999997 2024-09-17T21:21:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:800 3.19 3:9dc17f12:::rbd_data.1152eb799094.000000000000000e:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 68.757876229000004, 2024-09-17T21:21:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88154663799999999, 2024-09-17T21:21:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 800 2024-09-17T21:21:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432187+0000", 2024-09-17T21:21:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4100000000000001e-06 2024-09-17T21:21:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432194+0000", 2024-09-17T21:21:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0389999999999999e-06 2024-09-17T21:21:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432216+0000", 2024-09-17T21:21:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1614e-05 2024-09-17T21:21:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432232+0000", 2024-09-17T21:21:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5739e-05 2024-09-17T21:21:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432405+0000", 2024-09-17T21:21:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017355300000000001 2024-09-17T21:21:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432432+0000", 2024-09-17T21:21:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6979e-05 2024-09-17T21:21:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.307767+0000", 2024-09-17T21:21:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87533472899999998 2024-09-17T21:21:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308192+0000", 2024-09-17T21:21:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042485999999999999 2024-09-17T21:21:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308727+0000", 2024-09-17T21:21:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00053550400000000001 2024-09-17T21:21:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310347+0000", 2024-09-17T21:21:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016198670000000001 2024-09-17T21:21:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313424+0000", 2024-09-17T21:21:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030764640000000001 2024-09-17T21:21:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313510+0000", 2024-09-17T21:21:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5938000000000003e-05 2024-09-17T21:21:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313733+0000", 2024-09-17T21:21:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022294200000000001 2024-09-17T21:21:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:805 3.19 3:9cab2ed6:::rbd_data.1152eb799094.0000000000000013:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 68.757672209999996, 2024-09-17T21:21:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88274355699999996, 2024-09-17T21:21:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 805 2024-09-17T21:21:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432391+0000", 2024-09-17T21:21:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3e-06 2024-09-17T21:21:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432398+0000", 2024-09-17T21:21:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4489999999999996e-06 2024-09-17T21:21:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432415+0000", 2024-09-17T21:21:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7750000000000001e-05 2024-09-17T21:21:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432430+0000", 2024-09-17T21:21:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.449e-05 2024-09-17T21:21:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432635+0000", 2024-09-17T21:21:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205031 2024-09-17T21:21:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432657+0000", 2024-09-17T21:21:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2064000000000001e-05 2024-09-17T21:21:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310444+0000", 2024-09-17T21:21:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87778699199999999 2024-09-17T21:21:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310849+0000", 2024-09-17T21:21:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00040497800000000001 2024-09-17T21:21:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.311396+0000", 2024-09-17T21:21:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00054729099999999999 2024-09-17T21:21:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313847+0000", 2024-09-17T21:21:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024504940000000001 2024-09-17T21:21:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314879+0000", 2024-09-17T21:21:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001031817 2024-09-17T21:21:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314964+0000", 2024-09-17T21:21:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5685000000000004e-05 2024-09-17T21:21:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.315133+0000", 2024-09-17T21:21:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000169216 2024-09-17T21:21:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12087 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 52224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:21:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.420583532000002, 2024-09-17T21:21:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087684460000000006, 2024-09-17T21:21:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12087 2024-09-17T21:21:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:21:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769479+0000", 2024-09-17T21:21:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769480+0000", 2024-09-17T21:21:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:21:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769485+0000", 2024-09-17T21:21:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8880000000000002e-06 2024-09-17T21:21:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769509+0000", 2024-09-17T21:21:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3343e-05 2024-09-17T21:21:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769521+0000", 2024-09-17T21:21:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2497000000000001e-05 2024-09-17T21:21:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.856883+0000", 2024-09-17T21:21:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087362191000000006 2024-09-17T21:21:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857014+0000", 2024-09-17T21:21:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013090399999999999 2024-09-17T21:21:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857163+0000", 2024-09-17T21:21:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014864000000000001 2024-09-17T21:21:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12088 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 56320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:21:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.420499876999997, 2024-09-17T21:21:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087917772000000005, 2024-09-17T21:21:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12088 2024-09-17T21:21:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:21:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769562+0000", 2024-09-17T21:21:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769569+0000", 2024-09-17T21:21:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:21:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769574+0000", 2024-09-17T21:21:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4659999999999998e-06 2024-09-17T21:21:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769586+0000", 2024-09-17T21:21:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2311e-05 2024-09-17T21:21:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769597+0000", 2024-09-17T21:21:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0548e-05 2024-09-17T21:21:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857217+0000", 2024-09-17T21:21:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087619896000000003 2024-09-17T21:21:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857335+0000", 2024-09-17T21:21:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118538 2024-09-17T21:21:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857480+0000", 2024-09-17T21:21:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014470999999999999 2024-09-17T21:21:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12089 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 60416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.420447178000003, 2024-09-17T21:21:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088293233999999998, 2024-09-17T21:21:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12089 2024-09-17T21:21:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769616+0000", 2024-09-17T21:21:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:21:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769620+0000", 2024-09-17T21:21:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5140000000000002e-06 2024-09-17T21:21:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769633+0000", 2024-09-17T21:21:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2849e-05 2024-09-17T21:21:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769644+0000", 2024-09-17T21:21:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1471000000000001e-05 2024-09-17T21:21:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857564+0000", 2024-09-17T21:21:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087919445999999998 2024-09-17T21:21:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857716+0000", 2024-09-17T21:21:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000152065 2024-09-17T21:21:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857908+0000", 2024-09-17T21:21:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019252599999999999 2024-09-17T21:21:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12090 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 64512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.420400792999999, 2024-09-17T21:21:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088582672000000001, 2024-09-17T21:21:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12090 2024-09-17T21:21:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769662+0000", 2024-09-17T21:21:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:21:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769683+0000", 2024-09-17T21:21:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0979e-05 2024-09-17T21:21:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769699+0000", 2024-09-17T21:21:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5446000000000001e-05 2024-09-17T21:21:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769708+0000", 2024-09-17T21:21:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1679999999999993e-06 2024-09-17T21:21:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857976+0000", 2024-09-17T21:21:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088267795999999996 2024-09-17T21:21:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858095+0000", 2024-09-17T21:21:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119147 2024-09-17T21:21:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858244+0000", 2024-09-17T21:21:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014917000000000001 2024-09-17T21:21:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12091 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 68608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.420329408999997, 2024-09-17T21:21:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088832752000000001, 2024-09-17T21:21:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12091 2024-09-17T21:21:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769734+0000", 2024-09-17T21:21:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:21:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769739+0000", 2024-09-17T21:21:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8080000000000004e-06 2024-09-17T21:21:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769764+0000", 2024-09-17T21:21:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4839999999999999e-05 2024-09-17T21:21:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769777+0000", 2024-09-17T21:21:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3037000000000001e-05 2024-09-17T21:21:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858295+0000", 2024-09-17T21:21:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088518475999999999 2024-09-17T21:21:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858419+0000", 2024-09-17T21:21:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012378700000000001 2024-09-17T21:21:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858565+0000", 2024-09-17T21:21:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014630099999999999 2024-09-17T21:21:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12092 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.420247085, 2024-09-17T21:21:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089191555000000006, 2024-09-17T21:21:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12092 2024-09-17T21:21:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769816+0000", 2024-09-17T21:21:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7199999999999997e-07 2024-09-17T21:21:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769822+0000", 2024-09-17T21:21:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:21:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769838+0000", 2024-09-17T21:21:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6674e-05 2024-09-17T21:21:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769851+0000", 2024-09-17T21:21:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2527999999999999e-05 2024-09-17T21:21:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858660+0000", 2024-09-17T21:21:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088808556999999996 2024-09-17T21:21:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858817+0000", 2024-09-17T21:21:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015730400000000001 2024-09-17T21:21:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859007+0000", 2024-09-17T21:21:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018974899999999999 2024-09-17T21:21:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12093 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.420138981000001, 2024-09-17T21:21:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089429710999999995, 2024-09-17T21:21:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12093 2024-09-17T21:21:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769924+0000", 2024-09-17T21:21:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.342e-06 2024-09-17T21:21:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769929+0000", 2024-09-17T21:21:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0350000000000001e-06 2024-09-17T21:21:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769946+0000", 2024-09-17T21:21:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6793999999999999e-05 2024-09-17T21:21:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769959+0000", 2024-09-17T21:21:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3062e-05 2024-09-17T21:21:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859075+0000", 2024-09-17T21:21:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089115696999999994 2024-09-17T21:21:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859202+0000", 2024-09-17T21:21:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012657 2024-09-17T21:21:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859353+0000", 2024-09-17T21:21:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151211 2024-09-17T21:21:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12094 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.420084805000002, 2024-09-17T21:21:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089695001999999996, 2024-09-17T21:21:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12094 2024-09-17T21:21:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769978+0000", 2024-09-17T21:21:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.142e-06 2024-09-17T21:21:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769983+0000", 2024-09-17T21:21:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.386e-06 2024-09-17T21:21:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769997+0000", 2024-09-17T21:21:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.416e-05 2024-09-17T21:21:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770012+0000", 2024-09-17T21:21:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5384999999999999e-05 2024-09-17T21:21:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859405+0000", 2024-09-17T21:21:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089392376999999995 2024-09-17T21:21:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859525+0000", 2024-09-17T21:21:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120555 2024-09-17T21:21:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859672+0000", 2024-09-17T21:21:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000146997 2024-09-17T21:21:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12095 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.420032317999997, 2024-09-17T21:21:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090073476, 2024-09-17T21:21:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12095 2024-09-17T21:21:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770031+0000", 2024-09-17T21:21:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1699999999999997e-07 2024-09-17T21:21:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770036+0000", 2024-09-17T21:21:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.023e-06 2024-09-17T21:21:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770051+0000", 2024-09-17T21:21:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4953999999999999e-05 2024-09-17T21:21:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770063+0000", 2024-09-17T21:21:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:21:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859756+0000", 2024-09-17T21:21:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089692980000000005 2024-09-17T21:21:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859891+0000", 2024-09-17T21:21:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013459499999999999 2024-09-17T21:21:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860103+0000", 2024-09-17T21:21:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021274400000000001 2024-09-17T21:21:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12096 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.419982632999996, 2024-09-17T21:21:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090358274000000002, 2024-09-17T21:21:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12096 2024-09-17T21:21:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770080+0000", 2024-09-17T21:21:25.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9400000000000004e-07 2024-09-17T21:21:25.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770085+0000", 2024-09-17T21:21:25.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2169999999999996e-06 2024-09-17T21:21:25.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770100+0000", 2024-09-17T21:21:25.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5077e-05 2024-09-17T21:21:25.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770112+0000", 2024-09-17T21:21:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1951e-05 2024-09-17T21:21:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860160+0000", 2024-09-17T21:21:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090048413999999993 2024-09-17T21:21:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860285+0000", 2024-09-17T21:21:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012466699999999999 2024-09-17T21:21:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860438+0000", 2024-09-17T21:21:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015305399999999999 2024-09-17T21:21:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12097 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.419934886, 2024-09-17T21:21:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090702375000000002, 2024-09-17T21:21:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12097 2024-09-17T21:21:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770128+0000", 2024-09-17T21:21:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.04e-06 2024-09-17T21:21:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770133+0000", 2024-09-17T21:21:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8999999999999997e-06 2024-09-17T21:21:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770152+0000", 2024-09-17T21:21:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8890999999999999e-05 2024-09-17T21:21:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770164+0000", 2024-09-17T21:21:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1769e-05 2024-09-17T21:21:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860511+0000", 2024-09-17T21:21:25.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090347527999999996 2024-09-17T21:21:25.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860652+0000", 2024-09-17T21:21:25.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140894 2024-09-17T21:21:25.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860830+0000", 2024-09-17T21:21:25.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017735299999999999 2024-09-17T21:21:25.275 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.276 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.276 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.276 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12098 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:25.276 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:25.276 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.419881418000003, 2024-09-17T21:21:25.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090997262999999995, 2024-09-17T21:21:25.276 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.276 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.276 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:25.276 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:25.276 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:25.277 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:25.277 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12098 2024-09-17T21:21:25.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.277 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:25.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:25.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775465+0000", 2024-09-17T21:21:25.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0052843350000000002 2024-09-17T21:21:25.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775476+0000", 2024-09-17T21:21:25.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0797e-05 2024-09-17T21:21:25.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775506+0000", 2024-09-17T21:21:25.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9709999999999998e-05 2024-09-17T21:21:25.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775530+0000", 2024-09-17T21:21:25.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4528999999999998e-05 2024-09-17T21:21:25.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860894+0000", 2024-09-17T21:21:25.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.085364195000000004 2024-09-17T21:21:25.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861019+0000", 2024-09-17T21:21:25.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012467399999999999 2024-09-17T21:21:25.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861178+0000", 2024-09-17T21:21:25.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159023 2024-09-17T21:21:25.280 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.280 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.280 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.280 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12104 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:25.280 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:25.280 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.414180309000002, 2024-09-17T21:21:25.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087969628999999994, 2024-09-17T21:21:25.281 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.281 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.281 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:25.281 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:25.281 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:25.281 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:25.281 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12104 2024-09-17T21:21:25.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.281 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:25.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:25.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775883+0000", 2024-09-17T21:21:25.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3399999999999998e-07 2024-09-17T21:21:25.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775886+0000", 2024-09-17T21:21:25.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2119999999999999e-06 2024-09-17T21:21:25.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775895+0000", 2024-09-17T21:21:25.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8710000000000003e-06 2024-09-17T21:21:25.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775901+0000", 2024-09-17T21:21:25.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:21:25.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863544+0000", 2024-09-17T21:21:25.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087642642000000007 2024-09-17T21:21:25.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863694+0000", 2024-09-17T21:21:25.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014952300000000001 2024-09-17T21:21:25.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863851+0000", 2024-09-17T21:21:25.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000157804 2024-09-17T21:21:25.285 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.285 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.285 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.285 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12105 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:25.285 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:25.285 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.414151414999999, 2024-09-17T21:21:25.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088330400000000003, 2024-09-17T21:21:25.285 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.285 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.286 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:25.286 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:25.286 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:25.286 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:25.286 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12105 2024-09-17T21:21:25.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.286 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:25.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:25.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:25.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9999999999999997e-07 2024-09-17T21:21:25.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775914+0000", 2024-09-17T21:21:25.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4710000000000001e-06 2024-09-17T21:21:25.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775921+0000", 2024-09-17T21:21:25.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:21:25.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775928+0000", 2024-09-17T21:21:25.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:21:25.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863923+0000", 2024-09-17T21:21:25.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087995126000000007 2024-09-17T21:21:25.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864059+0000", 2024-09-17T21:21:25.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135797 2024-09-17T21:21:25.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864241+0000", 2024-09-17T21:21:25.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018252599999999999 2024-09-17T21:21:25.290 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.290 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.290 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.290 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12106 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:25.290 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:25.290 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 63.414062237000003, 2024-09-17T21:21:25.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088586394999999998, 2024-09-17T21:21:25.290 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.290 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.290 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:25.290 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:25.291 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:25.291 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:25.291 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12106 2024-09-17T21:21:25.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.291 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:25.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:25.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776001+0000", 2024-09-17T21:21:25.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4300000000000001e-07 2024-09-17T21:21:25.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776004+0000", 2024-09-17T21:21:25.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8049999999999998e-06 2024-09-17T21:21:25.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776018+0000", 2024-09-17T21:21:25.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4613999999999999e-05 2024-09-17T21:21:25.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776025+0000", 2024-09-17T21:21:25.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:21:25.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864279+0000", 2024-09-17T21:21:25.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088253570000000003 2024-09-17T21:21:25.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864433+0000", 2024-09-17T21:21:25.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015469400000000001 2024-09-17T21:21:25.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864586+0000", 2024-09-17T21:21:25.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000153003 2024-09-17T21:21:25.294 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.294 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.295 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.295 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.295 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.295 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:25.297 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:21:25.298 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:25.295+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:25.397 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:25.395+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:25.488 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:25.489 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:25.489 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:21:25.489 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:25.498 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:25.493+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:25.508 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:21:25.599 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:25.597+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:25.682 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:25.682 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:25.682 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:21:25.682 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:21:25.682 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:25.700 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:21:25.700 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:25.695+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:25.800 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:25.799+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:25.885 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:21:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 183.013615071, 2024-09-17T21:21:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:21:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:21:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:21:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:21:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:21:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:21:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:21:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:21:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:21:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:21:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:21:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:21:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:21:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:21:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:21:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:21:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:21:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:21:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 183.01234998800001, 2024-09-17T21:21:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:21:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:21:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:21:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:21:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:21:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:21:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:21:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:21:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:21:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:21:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:21:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:21:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:21:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:21:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:21:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:21:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:21:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:21:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 183.00257057799999, 2024-09-17T21:21:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:21:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:21:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:21:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:21:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:21:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:21:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:21:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:21:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:21:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:21:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:21:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:21:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:21:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:21:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:21:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:21:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:21:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:21:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 182.99384817500001, 2024-09-17T21:21:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:21:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:21:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:21:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:21:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:21:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:21:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:21:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:21:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:21:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:21:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:21:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:21:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:21:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:21:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:21:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:21:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:21:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:21:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 169.60875761099999, 2024-09-17T21:21:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:21:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:21:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:21:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:21:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:21:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:21:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:21:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:21:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:21:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:21:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:21:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:21:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:21:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:21:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:21:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:21:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:21:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1780 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'195, mlcod=69'190)", 2024-09-17T21:21:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 132.630902007, 2024-09-17T21:21:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075761769999999999, 2024-09-17T21:21:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250375+0000", 2024-09-17T21:21:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:21:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250383+0000", 2024-09-17T21:21:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7870000000000002e-06 2024-09-17T21:21:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250402+0000", 2024-09-17T21:21:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9346000000000001e-05 2024-09-17T21:21:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250413+0000", 2024-09-17T21:21:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.039e-05 2024-09-17T21:21:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251353+0000", 2024-09-17T21:21:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000940091 2024-09-17T21:21:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251407+0000", 2024-09-17T21:21:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4701999999999999e-05 2024-09-17T21:21:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257829+0000", 2024-09-17T21:21:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064212779999999999 2024-09-17T21:21:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257950+0000", 2024-09-17T21:21:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121623 2024-09-17T21:21:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1781 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'196, mlcod=69'190)", 2024-09-17T21:21:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:21:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 132.629819056, 2024-09-17T21:21:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065632939999999999, 2024-09-17T21:21:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:21:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251457+0000", 2024-09-17T21:21:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251459+0000", 2024-09-17T21:21:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.624e-06 2024-09-17T21:21:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251471+0000", 2024-09-17T21:21:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2636000000000001e-05 2024-09-17T21:21:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251498+0000", 2024-09-17T21:21:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:21:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251515+0000", 2024-09-17T21:21:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6939000000000002e-05 2024-09-17T21:21:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252938+0000", 2024-09-17T21:21:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001423082 2024-09-17T21:21:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.252999+0000", 2024-09-17T21:21:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1138999999999999e-05 2024-09-17T21:21:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257955+0000", 2024-09-17T21:21:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049559640000000002 2024-09-17T21:21:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.258020+0000", 2024-09-17T21:21:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5504000000000002e-05 2024-09-17T21:21:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1788 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'198, mlcod=69'191)", 2024-09-17T21:21:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 132.627793567, 2024-09-17T21:21:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073690370000000002, 2024-09-17T21:21:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253484+0000", 2024-09-17T21:21:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2470000000000001e-06 2024-09-17T21:21:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253492+0000", 2024-09-17T21:21:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7159999999999992e-06 2024-09-17T21:21:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253509+0000", 2024-09-17T21:21:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.666e-05 2024-09-17T21:21:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253522+0000", 2024-09-17T21:21:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2643000000000001e-05 2024-09-17T21:21:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254853+0000", 2024-09-17T21:21:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331238 2024-09-17T21:21:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254885+0000", 2024-09-17T21:21:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1562e-05 2024-09-17T21:21:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260788+0000", 2024-09-17T21:21:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059030749999999998 2024-09-17T21:21:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260852+0000", 2024-09-17T21:21:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3895999999999999e-05 2024-09-17T21:21:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5746 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'477, mlcod=71'473)", 2024-09-17T21:21:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 129.62236600899999, 2024-09-17T21:21:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074182709999999997, 2024-09-17T21:21:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258912+0000", 2024-09-17T21:21:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:21:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258926+0000", 2024-09-17T21:21:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4249e-05 2024-09-17T21:21:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258946+0000", 2024-09-17T21:21:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9879999999999999e-05 2024-09-17T21:21:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258956+0000", 2024-09-17T21:21:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:21:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259380+0000", 2024-09-17T21:21:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042337599999999999 2024-09-17T21:21:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259412+0000", 2024-09-17T21:21:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:21:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266191+0000", 2024-09-17T21:21:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067793269999999999 2024-09-17T21:21:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266328+0000", 2024-09-17T21:21:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013690999999999999 2024-09-17T21:21:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5747 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'478, mlcod=71'473)", 2024-09-17T21:21:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 129.62165966000001, 2024-09-17T21:21:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068779440000000004, 2024-09-17T21:21:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259618+0000", 2024-09-17T21:21:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6199999999999999e-06 2024-09-17T21:21:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259630+0000", 2024-09-17T21:21:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1884e-05 2024-09-17T21:21:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259657+0000", 2024-09-17T21:21:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7339e-05 2024-09-17T21:21:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259677+0000", 2024-09-17T21:21:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0156000000000001e-05 2024-09-17T21:21:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260521+0000", 2024-09-17T21:21:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084312899999999997 2024-09-17T21:21:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260585+0000", 2024-09-17T21:21:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4558000000000001e-05 2024-09-17T21:21:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266336+0000", 2024-09-17T21:21:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057505020000000002 2024-09-17T21:21:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266494+0000", 2024-09-17T21:21:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158756 2024-09-17T21:21:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5751 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'480, mlcod=71'473)", 2024-09-17T21:21:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 129.620240255, 2024-09-17T21:21:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075756299999999999, 2024-09-17T21:21:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261037+0000", 2024-09-17T21:21:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:21:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261049+0000", 2024-09-17T21:21:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1063999999999999e-05 2024-09-17T21:21:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261072+0000", 2024-09-17T21:21:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3102e-05 2024-09-17T21:21:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261133+0000", 2024-09-17T21:21:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1276000000000001e-05 2024-09-17T21:21:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:21:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015334319999999999 2024-09-17T21:21:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262702+0000", 2024-09-17T21:21:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5281999999999999e-05 2024-09-17T21:21:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268527+0000", 2024-09-17T21:21:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058250389999999997 2024-09-17T21:21:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268611+0000", 2024-09-17T21:21:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4724999999999999e-05 2024-09-17T21:21:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5756 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'481, mlcod=71'473)", 2024-09-17T21:21:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 129.619379399, 2024-09-17T21:21:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006772386, 2024-09-17T21:21:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261899+0000", 2024-09-17T21:21:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.903e-06 2024-09-17T21:21:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261912+0000", 2024-09-17T21:21:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-05 2024-09-17T21:21:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261938+0000", 2024-09-17T21:21:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6098999999999999e-05 2024-09-17T21:21:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261955+0000", 2024-09-17T21:21:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.696e-05 2024-09-17T21:21:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263845+0000", 2024-09-17T21:21:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001890187 2024-09-17T21:21:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263907+0000", 2024-09-17T21:21:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1600999999999995e-05 2024-09-17T21:21:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268615+0000", 2024-09-17T21:21:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047086070000000001 2024-09-17T21:21:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268669+0000", 2024-09-17T21:21:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3588999999999998e-05 2024-09-17T21:21:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4978.0:598 3.9 e80/23 3:9701cc17:::rbd_data.1152724ff2a3.0000000000000019:head v 80'202, mlcod=80'198)", 2024-09-17T21:21:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:21:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 119.11780501600001, 2024-09-17T21:21:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066473340000000004, 2024-09-17T21:21:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:21:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:21:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763472+0000", 2024-09-17T21:21:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6899999999999997e-07 2024-09-17T21:21:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763478+0000", 2024-09-17T21:21:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:21:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763489+0000", 2024-09-17T21:21:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0628999999999999e-05 2024-09-17T21:21:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763495+0000", 2024-09-17T21:21:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6649999999999997e-06 2024-09-17T21:21:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763907+0000", 2024-09-17T21:21:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041163899999999999 2024-09-17T21:21:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763976+0000", 2024-09-17T21:21:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8534999999999993e-05 2024-09-17T21:21:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.769900+0000", 2024-09-17T21:21:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059243270000000001 2024-09-17T21:21:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.770118+0000", 2024-09-17T21:21:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021842500000000001 2024-09-17T21:21:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:468 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'94, mlcod=89'87)", 2024-09-17T21:21:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 106.19505971700001, 2024-09-17T21:21:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066528790000000004, 2024-09-17T21:21:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686218+0000", 2024-09-17T21:21:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3060000000000001e-06 2024-09-17T21:21:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686256+0000", 2024-09-17T21:21:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8333e-05 2024-09-17T21:21:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686272+0000", 2024-09-17T21:21:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:21:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686283+0000", 2024-09-17T21:21:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0903e-05 2024-09-17T21:21:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686682+0000", 2024-09-17T21:21:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039935099999999999 2024-09-17T21:21:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686785+0000", 2024-09-17T21:21:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010275199999999999 2024-09-17T21:21:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692816+0000", 2024-09-17T21:21:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060317579999999999 2024-09-17T21:21:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692869+0000", 2024-09-17T21:21:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2775999999999998e-05 2024-09-17T21:21:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:755 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'127, mlcod=89'122)", 2024-09-17T21:21:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 106.058604503, 2024-09-17T21:21:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084014080000000008, 2024-09-17T21:21:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:21:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822683+0000", 2024-09-17T21:21:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0674e-05 2024-09-17T21:21:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822698+0000", 2024-09-17T21:21:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5189e-05 2024-09-17T21:21:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822708+0000", 2024-09-17T21:21:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9119999999999993e-06 2024-09-17T21:21:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822748+0000", 2024-09-17T21:21:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9727000000000001e-05 2024-09-17T21:21:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822759+0000", 2024-09-17T21:21:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0865e-05 2024-09-17T21:21:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:25.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831034+0000", 2024-09-17T21:21:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082750710000000002 2024-09-17T21:21:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831073+0000", 2024-09-17T21:21:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9063999999999997e-05 2024-09-17T21:21:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:756 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'128, mlcod=89'124)", 2024-09-17T21:21:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:25.953 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 106.05668187099999, 2024-09-17T21:21:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0076300270000000002, 2024-09-17T21:21:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:25.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824595+0000", 2024-09-17T21:21:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-17T21:21:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824600+0000", 2024-09-17T21:21:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:21:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824612+0000", 2024-09-17T21:21:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.181e-05 2024-09-17T21:21:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824619+0000", 2024-09-17T21:21:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:21:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824671+0000", 2024-09-17T21:21:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2262999999999998e-05 2024-09-17T21:21:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824695+0000", 2024-09-17T21:21:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3883000000000002e-05 2024-09-17T21:21:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832185+0000", 2024-09-17T21:21:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007490048 2024-09-17T21:21:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832224+0000", 2024-09-17T21:21:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8957e-05 2024-09-17T21:21:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.957 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2635 3.d e90/23 3:b3d1e920:::rbd_data.1152a8ae2019.000000000000001f:head v 90'252, mlcod=90'246)", 2024-09-17T21:21:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 105.034122801, 2024-09-17T21:21:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0077265190000000003, 2024-09-17T21:21:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847155+0000", 2024-09-17T21:21:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.544e-06 2024-09-17T21:21:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847165+0000", 2024-09-17T21:21:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.043e-05 2024-09-17T21:21:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847189+0000", 2024-09-17T21:21:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3659000000000002e-05 2024-09-17T21:21:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847210+0000", 2024-09-17T21:21:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1163e-05 2024-09-17T21:21:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848856+0000", 2024-09-17T21:21:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001645521 2024-09-17T21:21:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848905+0000", 2024-09-17T21:21:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9048999999999997e-05 2024-09-17T21:21:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854770+0000", 2024-09-17T21:21:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058652879999999998 2024-09-17T21:21:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854880+0000", 2024-09-17T21:21:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010986500000000001 2024-09-17T21:21:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "MOSDPGPush(3.5 151/149 [PushOp(3:a57cb214:::rbd_data.115262b8e217.0000000000000009:1d, version: 103'698, data_included: [196608~24576,229376~24576,262144~24576,294912~131072,524288~98304,819200~8192,843776~16384,876544~16384,909312~16384,942080~8192,1146880~16384,1179648~16384,1212416~16384,1474560~98304,1794048~8192,1826816~8192,1859584~8192,2105344~24576,2138112~24576,2170880~24576,2646016~8192,2678784~8192,2711552~8192,3072000~8192,3104768~8192,3137536~106496], data_size: 761856, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:a57cb214:::rbd_data.115262b8e217.0000000000000009:1d@103'698, size: 3276800, copy_subset: [(0, 3276800)], clone_subset: {}, snapset: 1d=[]:{1d: [1d]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3276800, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T21:21:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:07.006386+0000", 2024-09-17T21:21:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.874890341, 2024-09-17T21:21:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082699420000000006, 2024-09-17T21:21:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.962 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.006386+0000", 2024-09-17T21:21:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.006386+0000", 2024-09-17T21:21:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.006388+0000", 2024-09-17T21:21:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.827e-06 2024-09-17T21:21:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.007671+0000", 2024-09-17T21:21:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0012836340000000001 2024-09-17T21:21:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.007737+0000", 2024-09-17T21:21:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5815000000000006e-05 2024-09-17T21:21:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.007782+0000", 2024-09-17T21:21:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5509000000000001e-05 2024-09-17T21:21:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.008023+0000", 2024-09-17T21:21:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000240184 2024-09-17T21:21:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.008065+0000", 2024-09-17T21:21:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1978999999999999e-05 2024-09-17T21:21:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.014656+0000", 2024-09-17T21:21:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065909940000000002 2024-09-17T21:21:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(osd.3.0:3834 3.8 e165/154 3:11ef155a:::rbd_data.11527217c540.000000000000003f:2b v 165'498, mlcod=152'495)", 2024-09-17T21:21:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:22.375046+0000", 2024-09-17T21:21:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.506230156, 2024-09-17T21:21:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010228170999999999, 2024-09-17T21:21:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.966 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375046+0000", 2024-09-17T21:21:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375046+0000", 2024-09-17T21:21:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375047+0000", 2024-09-17T21:21:25.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.373e-06 2024-09-17T21:21:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375085+0000", 2024-09-17T21:21:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7376999999999998e-05 2024-09-17T21:21:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375127+0000", 2024-09-17T21:21:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1941999999999999e-05 2024-09-17T21:21:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375150+0000", 2024-09-17T21:21:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3113e-05 2024-09-17T21:21:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375261+0000", 2024-09-17T21:21:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111402 2024-09-17T21:21:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375287+0000", 2024-09-17T21:21:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5585e-05 2024-09-17T21:21:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.385224+0000", 2024-09-17T21:21:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0099369750000000007 2024-09-17T21:21:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.385274+0000", 2024-09-17T21:21:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0404000000000001e-05 2024-09-17T21:21:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.970 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4434.0:25007 3.1e e168/166 3:7ce0ed33:::rbd_object_map.11528d8227e0:head v 168'728, mlcod=26'23)", 2024-09-17T21:21:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:25.445808+0000", 2024-09-17T21:21:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.43546764399999999, 2024-09-17T21:21:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074791850000000002, 2024-09-17T21:21:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445808+0000", 2024-09-17T21:21:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445808+0000", 2024-09-17T21:21:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445809+0000", 2024-09-17T21:21:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0299999999999997e-07 2024-09-17T21:21:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445816+0000", 2024-09-17T21:21:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.392e-06 2024-09-17T21:21:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445828+0000", 2024-09-17T21:21:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2187e-05 2024-09-17T21:21:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445834+0000", 2024-09-17T21:21:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5710000000000003e-06 2024-09-17T21:21:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445896+0000", 2024-09-17T21:21:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2024000000000001e-05 2024-09-17T21:21:25.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445919+0000", 2024-09-17T21:21:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2480000000000002e-05 2024-09-17T21:21:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.453224+0000", 2024-09-17T21:21:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073048729999999999 2024-09-17T21:21:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.453288+0000", 2024-09-17T21:21:25.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3755000000000002e-05 2024-09-17T21:21:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:25.975 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:25.976 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:21:25.977 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:25.971+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:26.078 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:26.073+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:26.109 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:26.109 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:26.109 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:26.122 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:21:26.178 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:26.175+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:26.279 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:26.275+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:26.285 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:26.286 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:26.300 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:21:26.381 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:26.379+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:26.414 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:21:26.426 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:26.426 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:26.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:26.427 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:26.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.427 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:21:26.427 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:26.427 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 209.40943488900001, 2024-09-17T21:21:26.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:21:26.427 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:26.427 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:26.427 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:26.427 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:26.427 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:21:26.427 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:21:26.427 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:21:26.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.428 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:26.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:26.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:26.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:26.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:26.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:26.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:21:26.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:21:26.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:26.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:21:26.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:21:26.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:26.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:21:26.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:21:26.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:26.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:21:26.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:21:26.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:21:26.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:21:26.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:26.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:21:26.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:21:26.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:21:26.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:21:26.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:26.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:21:26.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:21:26.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:21:26.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:21:26.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:21:26.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:26.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:21:26.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:21:26.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:26.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:21:26.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:21:26.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:26.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:21:26.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:21:26.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:26.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:21:26.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:21:26.433 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.433 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:26.433 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.433 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:26.434 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:26.434 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 189.82927060899999, 2024-09-17T21:21:26.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:21:26.434 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:26.434 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:26.434 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:26.434 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:26.434 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:21:26.434 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:21:26.434 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:21:26.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.434 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:26.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:26.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:26.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:26.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:26.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:26.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:21:26.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:21:26.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:26.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:21:26.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:21:26.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:26.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:21:26.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:21:26.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:26.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:21:26.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:21:26.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:21:26.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:21:26.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:26.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:21:26.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:21:26.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:21:26.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:21:26.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:26.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:21:26.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:21:26.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:21:26.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:21:26.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:26.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:21:26.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:21:26.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:26.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:21:26.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:21:26.439 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.439 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:26.439 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.439 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:26.439 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:26.439 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 189.82917329700001, 2024-09-17T21:21:26.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:21:26.440 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:26.440 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:26.440 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:26.440 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:26.440 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:26.440 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:26.440 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:21:26.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.440 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:26.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:26.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:26.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:26.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:26.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:26.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:21:26.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:21:26.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:26.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:21:26.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:21:26.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:26.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:21:26.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:21:26.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:26.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:21:26.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:21:26.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:21:26.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:21:26.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:26.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:21:26.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:21:26.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:21:26.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:21:26.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:26.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:21:26.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:21:26.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:21:26.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:21:26.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:21:26.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:21:26.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:21:26.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:21:26.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:21:26.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:26.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:21:26.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:21:26.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:26.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:21:26.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:21:26.445 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.446 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:26.446 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.446 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:26.446 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:26.446 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 189.82708649599999, 2024-09-17T21:21:26.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:21:26.446 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:26.446 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:26.446 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:26.446 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:26.446 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:21:26.446 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:21:26.446 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:21:26.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.447 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:26.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:26.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:26.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:26.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:26.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:26.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:21:26.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:21:26.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:26.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:21:26.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:21:26.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:26.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:21:26.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:21:26.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:26.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:21:26.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:21:26.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:21:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:21:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:21:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:21:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:21:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:21:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:21:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:21:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:21:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:21:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:21:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:21:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:21:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:21:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:21:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:21:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:21:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:21:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:21:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:21:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:21:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:21:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:21:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 157.14318771500001, 2024-09-17T21:21:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:21:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:21:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:21:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:21:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:21:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:21:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:21:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:21:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:21:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:21:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:21:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:21:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:21:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:21:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:21:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:21:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:21:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:21:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:21:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:21:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:21:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:21:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:21:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:21:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 157.13409600400001, 2024-09-17T21:21:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:21:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:21:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:21:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:21:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:21:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:21:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:21:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:21:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:21:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:21:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:21:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:21:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:21:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:21:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:21:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:21:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:21:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:21:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:21:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:21:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 157.13392395299999, 2024-09-17T21:21:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:21:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:21:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:21:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:21:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:21:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:21:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:21:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:21:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:21:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:21:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:21:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:21:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:21:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:21:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:21:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:21:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:21:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:21:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:21:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:21:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 156.95899120600001, 2024-09-17T21:21:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:21:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:21:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:21:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:21:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:21:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:21:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:21:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:21:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:21:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:21:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:21:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:21:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:21:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:21:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:21:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:21:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:21:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:21:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:21:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:21:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 156.958906649, 2024-09-17T21:21:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:21:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:21:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:21:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:21:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:21:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:21:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:21:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:21:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:21:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:21:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:21:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:21:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:21:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:21:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:21:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:21:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:21:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:21:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:21:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 156.95884101199999, 2024-09-17T21:21:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:21:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:21:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:21:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:21:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:21:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:21:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:21:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:21:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:21:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:21:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:21:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:21:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:21:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:21:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:21:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:21:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:21:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:21:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:21:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:21:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 105.58526591499999, 2024-09-17T21:21:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:21:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:21:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:21:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:21:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:21:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:21:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:21:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:21:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:21:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:21:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:21:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:21:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:21:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:21:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:21:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:21:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:21:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 105.585146403, 2024-09-17T21:21:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:21:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:21:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:21:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:21:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:21:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:21:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:21:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:21:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:21:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:21:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:21:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:21:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:21:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:21:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:21:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:21:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:21:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:994 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 571392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:21:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 60.992174429000002, 2024-09-17T21:21:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083898465000000005, 2024-09-17T21:21:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 994 2024-09-17T21:21:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:21:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:21:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:21:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:21:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431058+0000", 2024-09-17T21:21:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1667e-05 2024-09-17T21:21:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431067+0000", 2024-09-17T21:21:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0720000000000006e-06 2024-09-17T21:21:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431075+0000", 2024-09-17T21:21:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9200000000000004e-06 2024-09-17T21:21:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514671+0000", 2024-09-17T21:21:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083596229999999994 2024-09-17T21:21:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514741+0000", 2024-09-17T21:21:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9929000000000001e-05 2024-09-17T21:21:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514924+0000", 2024-09-17T21:21:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000183122 2024-09-17T21:21:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:995 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 575488~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:21:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 60.992115413000001, 2024-09-17T21:21:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084214699000000004, 2024-09-17T21:21:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 995 2024-09-17T21:21:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:21:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:21:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431086+0000", 2024-09-17T21:21:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7999999999999995e-07 2024-09-17T21:21:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431088+0000", 2024-09-17T21:21:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4729999999999999e-06 2024-09-17T21:21:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431126+0000", 2024-09-17T21:21:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7583999999999998e-05 2024-09-17T21:21:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431137+0000", 2024-09-17T21:21:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1768e-05 2024-09-17T21:21:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515003+0000", 2024-09-17T21:21:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083865097 2024-09-17T21:21:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515115+0000", 2024-09-17T21:21:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112561 2024-09-17T21:21:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515300+0000", 2024-09-17T21:21:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018453600000000001 2024-09-17T21:21:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:996 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 60.992033032000002, 2024-09-17T21:21:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084493736999999999, 2024-09-17T21:21:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 996 2024-09-17T21:21:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431168+0000", 2024-09-17T21:21:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4e-07 2024-09-17T21:21:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431172+0000", 2024-09-17T21:21:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.608e-06 2024-09-17T21:21:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431180+0000", 2024-09-17T21:21:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5650000000000002e-06 2024-09-17T21:21:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431188+0000", 2024-09-17T21:21:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:21:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515354+0000", 2024-09-17T21:21:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084165977000000003 2024-09-17T21:21:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515451+0000", 2024-09-17T21:21:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6669000000000006e-05 2024-09-17T21:21:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515661+0000", 2024-09-17T21:21:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021021599999999999 2024-09-17T21:21:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:997 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 583680~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 60.992001637000001, 2024-09-17T21:21:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084772419000000002, 2024-09-17T21:21:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 997 2024-09-17T21:21:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0400000000000005e-07 2024-09-17T21:21:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431202+0000", 2024-09-17T21:21:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:21:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431208+0000", 2024-09-17T21:21:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4740000000000002e-06 2024-09-17T21:21:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431214+0000", 2024-09-17T21:21:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6550000000000001e-06 2024-09-17T21:21:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515741+0000", 2024-09-17T21:21:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084526904 2024-09-17T21:21:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515826+0000", 2024-09-17T21:21:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4643999999999998e-05 2024-09-17T21:21:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515971+0000", 2024-09-17T21:21:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145444 2024-09-17T21:21:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:998 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 587776~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 60.991978664999998, 2024-09-17T21:21:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085108565999999997, 2024-09-17T21:21:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:26.509 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 998 2024-09-17T21:21:26.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.509 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:26.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:26.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:26.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:26.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:26.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:26.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:26.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:21:26.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:26.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431224+0000", 2024-09-17T21:21:26.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:21:26.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:26.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431231+0000", 2024-09-17T21:21:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000002e-06 2024-09-17T21:21:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431236+0000", 2024-09-17T21:21:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.648e-06 2024-09-17T21:21:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516024+0000", 2024-09-17T21:21:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084787584999999999 2024-09-17T21:21:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516122+0000", 2024-09-17T21:21:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8076000000000001e-05 2024-09-17T21:21:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516330+0000", 2024-09-17T21:21:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020808 2024-09-17T21:21:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1000 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 591872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 60.991956549999998, 2024-09-17T21:21:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085389065, 2024-09-17T21:21:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1000 2024-09-17T21:21:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:21:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431246+0000", 2024-09-17T21:21:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1749999999999999e-06 2024-09-17T21:21:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431253+0000", 2024-09-17T21:21:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2790000000000004e-06 2024-09-17T21:21:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431258+0000", 2024-09-17T21:21:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3299999999999998e-06 2024-09-17T21:21:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516384+0000", 2024-09-17T21:21:26.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085125549999999994 2024-09-17T21:21:26.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:26.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516460+0000", 2024-09-17T21:21:26.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6359999999999994e-05 2024-09-17T21:21:26.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:26.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516633+0000", 2024-09-17T21:21:26.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017285900000000001 2024-09-17T21:21:26.516 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.516 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:26.516 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.517 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1001 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 595968~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:26.517 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:26.517 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 60.991933144000001, 2024-09-17T21:21:26.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085614715999999993, 2024-09-17T21:21:26.517 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:26.517 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:26.517 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:26.517 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:26.517 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:26.517 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:26.517 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1001 2024-09-17T21:21:26.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.517 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:26.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:26.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:26.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:26.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:26.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:26.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438066+0000", 2024-09-17T21:21:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067989490000000003 2024-09-17T21:21:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438073+0000", 2024-09-17T21:21:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8419999999999999e-06 2024-09-17T21:21:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438090+0000", 2024-09-17T21:21:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6657e-05 2024-09-17T21:21:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438099+0000", 2024-09-17T21:21:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0529999999999996e-06 2024-09-17T21:21:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516678+0000", 2024-09-17T21:21:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078579422999999995 2024-09-17T21:21:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516739+0000", 2024-09-17T21:21:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1315000000000005e-05 2024-09-17T21:21:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516882+0000", 2024-09-17T21:21:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014247700000000001 2024-09-17T21:21:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:19998 3.5 3:a4885450:::rbd_object_map.1152d139f9f9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-17T21:21:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 47.020665899000001, 2024-09-17T21:21:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90715489299999996, 2024-09-17T21:21:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19998 2024-09-17T21:21:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402537+0000", 2024-09-17T21:21:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5600000000000001e-06 2024-09-17T21:21:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402549+0000", 2024-09-17T21:21:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1929999999999999e-05 2024-09-17T21:21:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402579+0000", 2024-09-17T21:21:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0093000000000001e-05 2024-09-17T21:21:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402595+0000", 2024-09-17T21:21:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5855999999999999e-05 2024-09-17T21:21:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402731+0000", 2024-09-17T21:21:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013610799999999999 2024-09-17T21:21:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:26.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402750+0000", 2024-09-17T21:21:26.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8737e-05 2024-09-17T21:21:26.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:26.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.305977+0000", 2024-09-17T21:21:26.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90322768399999998 2024-09-17T21:21:26.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:26.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306236+0000", 2024-09-17T21:21:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000258111 2024-09-17T21:21:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306557+0000", 2024-09-17T21:21:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032159799999999998 2024-09-17T21:21:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.308286+0000", 2024-09-17T21:21:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017292150000000001 2024-09-17T21:21:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309530+0000", 2024-09-17T21:21:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00124409 2024-09-17T21:21:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309630+0000", 2024-09-17T21:21:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9689e-05 2024-09-17T21:21:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309689+0000", 2024-09-17T21:21:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9222000000000003e-05 2024-09-17T21:21:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.528 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:26.528 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:26.528 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:26.529 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:21:26.529 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:26.525+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:26.630 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:26.627+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:26.699 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:26.699 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:26.699 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:26.699 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:26.713 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:21:26.759 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:26.757+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:26.833 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:26.829+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:26.837 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:26.837 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:26.837 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:26.837 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:26.837 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:26.851 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:21:26.939 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:26.937+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:27.020 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:27.020 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:27.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:27.020 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:27.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.020 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:18003 3.1d 3:b9765580:::rbd_header.11527d2b15f6:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T21:21:27.020 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:27.020 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.657849975999994, 2024-09-17T21:21:27.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94165346900000002, 2024-09-17T21:21:27.020 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18003 2024-09-17T21:21:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357880+0000", 2024-09-17T21:21:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:21:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357891+0000", 2024-09-17T21:21:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0417999999999999e-05 2024-09-17T21:21:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357913+0000", 2024-09-17T21:21:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2215e-05 2024-09-17T21:21:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357922+0000", 2024-09-17T21:21:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4779999999999997e-06 2024-09-17T21:21:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358023+0000", 2024-09-17T21:21:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101025 2024-09-17T21:21:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358040+0000", 2024-09-17T21:21:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6291999999999999e-05 2024-09-17T21:21:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298203+0000", 2024-09-17T21:21:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94016297500000001 2024-09-17T21:21:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298626+0000", 2024-09-17T21:21:27.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000422853 2024-09-17T21:21:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.299532+0000", 2024-09-17T21:21:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00090657300000000004 2024-09-17T21:21:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15706 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 977920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:21:27.025 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.494748185000006, 2024-09-17T21:21:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091952328, 2024-09-17T21:21:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15706 2024-09-17T21:21:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:21:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:21:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520982+0000", 2024-09-17T21:21:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.277e-06 2024-09-17T21:21:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520988+0000", 2024-09-17T21:21:27.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4470000000000003e-06 2024-09-17T21:21:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521008+0000", 2024-09-17T21:21:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9309000000000001e-05 2024-09-17T21:21:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521023+0000", 2024-09-17T21:21:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5119999999999999e-05 2024-09-17T21:21:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:27.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612638+0000", 2024-09-17T21:21:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091615328999999995 2024-09-17T21:21:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612738+0000", 2024-09-17T21:21:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100466 2024-09-17T21:21:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612933+0000", 2024-09-17T21:21:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019437999999999999 2024-09-17T21:21:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.029 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15707 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 982016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.494684728999999, 2024-09-17T21:21:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092305228000000003, 2024-09-17T21:21:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15707 2024-09-17T21:21:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.030 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521045+0000", 2024-09-17T21:21:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4899999999999999e-06 2024-09-17T21:21:27.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521066+0000", 2024-09-17T21:21:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0236000000000001e-05 2024-09-17T21:21:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521085+0000", 2024-09-17T21:21:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8819999999999999e-05 2024-09-17T21:21:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521100+0000", 2024-09-17T21:21:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5146e-05 2024-09-17T21:21:27.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612978+0000", 2024-09-17T21:21:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091878847 2024-09-17T21:21:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613079+0000", 2024-09-17T21:21:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100503 2024-09-17T21:21:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613349+0000", 2024-09-17T21:21:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000270186 2024-09-17T21:21:27.033 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15708 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 986112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.494606554000001, 2024-09-17T21:21:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092547217000000001, 2024-09-17T21:21:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15708 2024-09-17T21:21:27.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521123+0000", 2024-09-17T21:21:27.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.353e-06 2024-09-17T21:21:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521129+0000", 2024-09-17T21:21:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:21:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521145+0000", 2024-09-17T21:21:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.611e-05 2024-09-17T21:21:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521154+0000", 2024-09-17T21:21:27.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4500000000000004e-06 2024-09-17T21:21:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613397+0000", 2024-09-17T21:21:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092243275 2024-09-17T21:21:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613503+0000", 2024-09-17T21:21:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106013 2024-09-17T21:21:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613669+0000", 2024-09-17T21:21:27.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016642299999999999 2024-09-17T21:21:27.038 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.038 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:27.038 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.038 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15709 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 990208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:27.038 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:27.038 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.494563537999994, 2024-09-17T21:21:27.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092728678999999994, 2024-09-17T21:21:27.038 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:27.038 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:27.038 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:27.038 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:27.038 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:27.039 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:27.039 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15709 2024-09-17T21:21:27.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.039 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:27.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:27.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:27.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:27.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:27.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:27.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521166+0000", 2024-09-17T21:21:27.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7000000000000004e-07 2024-09-17T21:21:27.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:27.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521169+0000", 2024-09-17T21:21:27.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2660000000000002e-06 2024-09-17T21:21:27.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521181+0000", 2024-09-17T21:21:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.208e-05 2024-09-17T21:21:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521194+0000", 2024-09-17T21:21:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2707e-05 2024-09-17T21:21:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613695+0000", 2024-09-17T21:21:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092500709 2024-09-17T21:21:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613750+0000", 2024-09-17T21:21:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.503e-05 2024-09-17T21:21:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613894+0000", 2024-09-17T21:21:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144117 2024-09-17T21:21:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15710 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 994304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.494518971999995, 2024-09-17T21:21:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092957729000000003, 2024-09-17T21:21:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15710 2024-09-17T21:21:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:27.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8000000000000005e-07 2024-09-17T21:21:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521216+0000", 2024-09-17T21:21:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3859999999999998e-06 2024-09-17T21:21:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521234+0000", 2024-09-17T21:21:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:21:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521248+0000", 2024-09-17T21:21:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:21:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613926+0000", 2024-09-17T21:21:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092678324000000006 2024-09-17T21:21:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613974+0000", 2024-09-17T21:21:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7754999999999999e-05 2024-09-17T21:21:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614167+0000", 2024-09-17T21:21:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193508 2024-09-17T21:21:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:27.046 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15711 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 998400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.49446159, 2024-09-17T21:21:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093268000000000004, 2024-09-17T21:21:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15711 2024-09-17T21:21:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:27.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521268+0000", 2024-09-17T21:21:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-17T21:21:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521273+0000", 2024-09-17T21:21:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:21:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521290+0000", 2024-09-17T21:21:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7269999999999999e-05 2024-09-17T21:21:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521304+0000", 2024-09-17T21:21:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3626e-05 2024-09-17T21:21:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614220+0000", 2024-09-17T21:21:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092916367 2024-09-17T21:21:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614322+0000", 2024-09-17T21:21:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102212 2024-09-17T21:21:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614535+0000", 2024-09-17T21:21:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021266199999999999 2024-09-17T21:21:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.050 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15712 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1002496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.494406095000002, 2024-09-17T21:21:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093568707000000001, 2024-09-17T21:21:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:27.051 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15712 2024-09-17T21:21:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521324+0000", 2024-09-17T21:21:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.17e-06 2024-09-17T21:21:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521334+0000", 2024-09-17T21:21:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9369999999999999e-06 2024-09-17T21:21:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521356+0000", 2024-09-17T21:21:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2045999999999999e-05 2024-09-17T21:21:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521370+0000", 2024-09-17T21:21:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4270000000000001e-05 2024-09-17T21:21:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614583+0000", 2024-09-17T21:21:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093213220999999999 2024-09-17T21:21:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614680+0000", 2024-09-17T21:21:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6900999999999999e-05 2024-09-17T21:21:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614891+0000", 2024-09-17T21:21:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021116200000000001 2024-09-17T21:21:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15713 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1006592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.494339393000004, 2024-09-17T21:21:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093834641999999996, 2024-09-17T21:21:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15713 2024-09-17T21:21:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521390+0000", 2024-09-17T21:21:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:21:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521396+0000", 2024-09-17T21:21:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:21:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521417+0000", 2024-09-17T21:21:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1236000000000001e-05 2024-09-17T21:21:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521431+0000", 2024-09-17T21:21:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3873000000000001e-05 2024-09-17T21:21:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614938+0000", 2024-09-17T21:21:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093506522999999994 2024-09-17T21:21:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615025+0000", 2024-09-17T21:21:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7635999999999999e-05 2024-09-17T21:21:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615224+0000", 2024-09-17T21:21:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019844700000000001 2024-09-17T21:21:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15714 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:27.059 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.494276137, 2024-09-17T21:21:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094071925000000001, 2024-09-17T21:21:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15714 2024-09-17T21:21:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:27.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521454+0000", 2024-09-17T21:21:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.097e-06 2024-09-17T21:21:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:27.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521459+0000", 2024-09-17T21:21:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2660000000000001e-06 2024-09-17T21:21:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521481+0000", 2024-09-17T21:21:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1878999999999998e-05 2024-09-17T21:21:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521498+0000", 2024-09-17T21:21:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6807e-05 2024-09-17T21:21:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615271+0000", 2024-09-17T21:21:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093773286999999997 2024-09-17T21:21:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615366+0000", 2024-09-17T21:21:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5322000000000007e-05 2024-09-17T21:21:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615524+0000", 2024-09-17T21:21:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015826699999999999 2024-09-17T21:21:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15715 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1014784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.494211801999995, 2024-09-17T21:21:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094243840999999995, 2024-09-17T21:21:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15715 2024-09-17T21:21:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521518+0000", 2024-09-17T21:21:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4300000000000001e-06 2024-09-17T21:21:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521524+0000", 2024-09-17T21:21:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5860000000000004e-06 2024-09-17T21:21:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521541+0000", 2024-09-17T21:21:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7574000000000001e-05 2024-09-17T21:21:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521555+0000", 2024-09-17T21:21:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3540000000000001e-05 2024-09-17T21:21:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615560+0000", 2024-09-17T21:21:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.09400472 2024-09-17T21:21:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615614+0000", 2024-09-17T21:21:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4693e-05 2024-09-17T21:21:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615761+0000", 2024-09-17T21:21:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000146298 2024-09-17T21:21:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15716 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1018880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.494155457000005, 2024-09-17T21:21:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094400676000000003, 2024-09-17T21:21:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15716 2024-09-17T21:21:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.068 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521574+0000", 2024-09-17T21:21:27.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1939999999999999e-06 2024-09-17T21:21:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521579+0000", 2024-09-17T21:21:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:21:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521597+0000", 2024-09-17T21:21:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8074e-05 2024-09-17T21:21:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:27.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521613+0000", 2024-09-17T21:21:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5316e-05 2024-09-17T21:21:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615780+0000", 2024-09-17T21:21:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094166874999999997 2024-09-17T21:21:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615836+0000", 2024-09-17T21:21:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6926999999999999e-05 2024-09-17T21:21:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:27.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615974+0000", 2024-09-17T21:21:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013741400000000001 2024-09-17T21:21:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15717 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1022976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.494096760999994, 2024-09-17T21:21:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094586723999999997, 2024-09-17T21:21:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15717 2024-09-17T21:21:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521633+0000", 2024-09-17T21:21:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:21:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521638+0000", 2024-09-17T21:21:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:21:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521659+0000", 2024-09-17T21:21:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-05 2024-09-17T21:21:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521675+0000", 2024-09-17T21:21:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5688999999999998e-05 2024-09-17T21:21:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616008+0000", 2024-09-17T21:21:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094333314000000001 2024-09-17T21:21:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616066+0000", 2024-09-17T21:21:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7797000000000002e-05 2024-09-17T21:21:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:27.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616219+0000", 2024-09-17T21:21:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015281100000000001 2024-09-17T21:21:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15718 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.493996686000003, 2024-09-17T21:21:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094703674000000002, 2024-09-17T21:21:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:27.076 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15718 2024-09-17T21:21:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521734+0000", 2024-09-17T21:21:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.61e-06 2024-09-17T21:21:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521740+0000", 2024-09-17T21:21:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:21:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521754+0000", 2024-09-17T21:21:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3793e-05 2024-09-17T21:21:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521763+0000", 2024-09-17T21:21:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5129999999999996e-06 2024-09-17T21:21:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616256+0000", 2024-09-17T21:21:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094492919999999994 2024-09-17T21:21:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616312+0000", 2024-09-17T21:21:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6737000000000002e-05 2024-09-17T21:21:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616436+0000", 2024-09-17T21:21:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012335800000000001 2024-09-17T21:21:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15719 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1031168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.493943281, 2024-09-17T21:21:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094888963000000007, 2024-09-17T21:21:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:27.080 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15719 2024-09-17T21:21:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:27.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521786+0000", 2024-09-17T21:21:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.074e-06 2024-09-17T21:21:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521791+0000", 2024-09-17T21:21:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3030000000000001e-06 2024-09-17T21:21:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521804+0000", 2024-09-17T21:21:27.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3027000000000001e-05 2024-09-17T21:21:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521811+0000", 2024-09-17T21:21:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6090000000000003e-06 2024-09-17T21:21:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616466+0000", 2024-09-17T21:21:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094654294 2024-09-17T21:21:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616523+0000", 2024-09-17T21:21:27.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6953999999999997e-05 2024-09-17T21:21:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616674+0000", 2024-09-17T21:21:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015170200000000001 2024-09-17T21:21:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15720 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1035264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.493906440999993, 2024-09-17T21:21:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095067061999999994, 2024-09-17T21:21:27.084 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15720 2024-09-17T21:21:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521823+0000", 2024-09-17T21:21:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:21:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521826+0000", 2024-09-17T21:21:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2100000000000002e-06 2024-09-17T21:21:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521835+0000", 2024-09-17T21:21:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1630000000000002e-06 2024-09-17T21:21:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521842+0000", 2024-09-17T21:21:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:21:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616711+0000", 2024-09-17T21:21:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094868859 2024-09-17T21:21:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:27.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616764+0000", 2024-09-17T21:21:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2778999999999997e-05 2024-09-17T21:21:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616889+0000", 2024-09-17T21:21:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012533100000000001 2024-09-17T21:21:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15721 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1039360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.493859332, 2024-09-17T21:21:27.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095277219999999996, 2024-09-17T21:21:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15721 2024-09-17T21:21:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521871+0000", 2024-09-17T21:21:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.443e-06 2024-09-17T21:21:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521876+0000", 2024-09-17T21:21:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0300000000000001e-06 2024-09-17T21:21:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521886+0000", 2024-09-17T21:21:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186e-05 2024-09-17T21:21:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521893+0000", 2024-09-17T21:21:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.08e-06 2024-09-17T21:21:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616918+0000", 2024-09-17T21:21:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095025223000000006 2024-09-17T21:21:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616977+0000", 2024-09-17T21:21:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8403999999999999e-05 2024-09-17T21:21:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617147+0000", 2024-09-17T21:21:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016985399999999999 2024-09-17T21:21:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.092 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15722 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.493824883000002, 2024-09-17T21:21:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095459783000000006, 2024-09-17T21:21:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15722 2024-09-17T21:21:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7700000000000004e-07 2024-09-17T21:21:27.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521907+0000", 2024-09-17T21:21:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.813e-06 2024-09-17T21:21:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521922+0000", 2024-09-17T21:21:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4533e-05 2024-09-17T21:21:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521934+0000", 2024-09-17T21:21:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2469999999999999e-05 2024-09-17T21:21:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617187+0000", 2024-09-17T21:21:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095252311000000006 2024-09-17T21:21:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617240+0000", 2024-09-17T21:21:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3740999999999999e-05 2024-09-17T21:21:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617364+0000", 2024-09-17T21:21:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-17T21:21:27.096 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15723 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.493781017000003, 2024-09-17T21:21:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095653563999999996, 2024-09-17T21:21:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15723 2024-09-17T21:21:27.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521949+0000", 2024-09-17T21:21:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:21:27.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521953+0000", 2024-09-17T21:21:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5870000000000003e-06 2024-09-17T21:21:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521970+0000", 2024-09-17T21:21:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7282000000000002e-05 2024-09-17T21:21:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521984+0000", 2024-09-17T21:21:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:21:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617383+0000", 2024-09-17T21:21:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095399063000000006 2024-09-17T21:21:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617437+0000", 2024-09-17T21:21:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3931999999999998e-05 2024-09-17T21:21:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617601+0000", 2024-09-17T21:21:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016407300000000001 2024-09-17T21:21:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.100 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5629.0:2384 3.1c 3:38c1263e:::rbd_data.1152d37183c3.000000000000000b:head [read 11780096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-17T21:21:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 51.590950186999997, 2024-09-17T21:21:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0246185320000001, 2024-09-17T21:21:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5629", 2024-09-17T21:21:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3798711329", 2024-09-17T21:21:27.101 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2384 2024-09-17T21:21:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:27.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424780+0000", 2024-09-17T21:21:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.13e-06 2024-09-17T21:21:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424785+0000", 2024-09-17T21:21:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4739999999999996e-06 2024-09-17T21:21:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424798+0000", 2024-09-17T21:21:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2704000000000001e-05 2024-09-17T21:21:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:27.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424806+0000", 2024-09-17T21:21:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9899999999999997e-06 2024-09-17T21:21:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424868+0000", 2024-09-17T21:21:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999997e-05 2024-09-17T21:21:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424875+0000", 2024-09-17T21:21:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0870000000000001e-06 2024-09-17T21:21:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428897+0000", 2024-09-17T21:21:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040226080000000004 2024-09-17T21:21:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428913+0000", 2024-09-17T21:21:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:21:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.448963+0000", 2024-09-17T21:21:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0200498659999999 2024-09-17T21:21:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449158+0000", 2024-09-17T21:21:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019581000000000001 2024-09-17T21:21:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449397+0000", 2024-09-17T21:21:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023871 2024-09-17T21:21:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:27.106 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:27.106 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:27.107 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:21:27.108 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:21:27.109 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:27.105+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:27.209 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:27.207+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:27.259 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:27.260 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:27.276 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:27.276 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:27.278 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:27.278 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:27.278 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:27.278 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:27.292 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:21:27.293 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:21:27.294 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:21:27.299 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-17T21:21:27.309 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:27.307+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:27.322 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:27.323 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:27.342 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:27.342 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:27.363 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:27.363 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:27.406 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 2] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:21:27.406 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:21:27.407 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [1, 3, 4, 5, 2] 2024-09-17T21:21:27.407 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-17T21:21:27.410 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:27.405+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:27.428 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:27.428 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:27.444 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:21:27.511 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:27.507+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:27.612 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:27.607+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:27.613 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:27.613 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:27.628 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:21:27.713 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:27.711+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:27.739 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-09-17T21:21:27.735+0000 7fe37768f640 -1 mon.a@0(leader).osd e170 definitely_dead 0 2024-09-17T21:21:27.748 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:27.748 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:27.762 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:21:27.814 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:27.809+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:27.915 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:27.911+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:27.923 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:27.923 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:27.937 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:21:28.015 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:28.013+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:28.103 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:28.103 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:28.116 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:28.111+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:28.117 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:21:28.217 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:28.213+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:28.284 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:28.285 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:28.298 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:21:28.318 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:28.313+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:28.416 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:28.416 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:28.418 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:28.413+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:28.429 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:21:28.440 INFO:teuthology.orchestra.run.smithi083.stderr:marked out osd.2. 2024-09-17T21:21:28.519 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:28.515+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:28.554 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:28.554 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:28.568 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:21:28.653 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:28.651+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:28.766 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:28.766 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:28.767 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:21:28.767 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:28.761+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:28.867 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:28.863+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:28.933 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:28.933 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:28.948 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:21:28.967 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:28.965+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:29.069 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:29.065+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:29.113 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:29.113 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:29.127 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:21:29.175 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:29.173+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:29.258 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:29.259 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:29.270 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:29.265+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:29.273 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:21:29.324 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:21:29.371 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:29.369+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:29.380 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:21:29.437 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:29.438 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:29.452 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:21:29.471 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:29.467+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:29.528 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:29.528 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:29.545 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:29.545 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:29.561 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:21:29.561 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:21:29.573 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:29.569+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:29.581 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:29.581 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:29.601 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:29.602 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:29.618 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:29.618 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:29.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:29.619 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:29.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.619 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:21:29.619 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:29.619 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 212.60003317900001, 2024-09-17T21:21:29.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:21:29.619 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:29.619 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:29.619 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:29.619 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:29.619 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:21:29.620 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:21:29.620 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:21:29.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.620 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:29.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:29.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:29.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:29.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:29.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:29.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:21:29.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:21:29.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:29.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:21:29.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:21:29.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:29.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:21:29.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:21:29.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:29.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:21:29.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:21:29.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:21:29.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:21:29.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:29.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:21:29.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:21:29.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:21:29.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:21:29.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:29.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:21:29.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:21:29.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:21:29.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:21:29.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:21:29.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:29.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:21:29.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:21:29.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:29.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:21:29.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:21:29.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:29.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:21:29.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:21:29.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:29.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:21:29.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:21:29.625 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.625 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:29.625 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.626 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:29.626 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:29.626 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 193.01986889899999, 2024-09-17T21:21:29.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:21:29.626 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:29.626 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:29.626 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:29.626 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:29.626 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:21:29.626 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:21:29.626 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:21:29.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.626 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:29.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:29.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:29.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:29.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:29.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:29.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:21:29.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:21:29.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:29.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:21:29.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:21:29.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:29.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:21:29.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:21:29.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:29.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:21:29.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:21:29.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:21:29.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:21:29.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:29.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:21:29.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:21:29.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:21:29.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:21:29.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:29.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:21:29.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:21:29.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:21:29.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:21:29.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:29.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:21:29.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:21:29.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:29.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:21:29.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:21:29.631 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.631 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:29.631 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.631 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:29.632 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:29.632 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 193.01977158700001, 2024-09-17T21:21:29.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:21:29.632 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:29.632 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:29.632 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:29.632 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:29.632 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:29.632 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:29.632 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:21:29.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.632 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:29.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:29.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:29.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:29.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:29.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:29.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:21:29.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:21:29.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:29.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:21:29.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:21:29.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:29.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:21:29.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:21:29.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:29.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:21:29.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:21:29.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:21:29.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:21:29.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:29.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:21:29.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:21:29.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:21:29.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:21:29.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:29.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:21:29.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:21:29.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:21:29.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:21:29.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:21:29.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:21:29.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:21:29.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:21:29.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:21:29.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:29.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:21:29.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:21:29.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:29.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:21:29.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:21:29.638 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.638 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:29.638 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.638 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:29.638 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:29.638 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 193.01768478599999, 2024-09-17T21:21:29.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:21:29.638 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:29.638 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:29.639 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:29.639 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:29.639 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:21:29.639 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:21:29.639 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:21:29.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.639 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:29.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:29.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:29.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:29.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:29.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:29.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:21:29.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:21:29.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:29.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:21:29.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:21:29.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:29.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:21:29.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:21:29.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:29.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:21:29.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:21:29.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:21:29.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:21:29.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:21:29.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:21:29.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:21:29.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:21:29.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:21:29.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:29.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:21:29.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:21:29.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:21:29.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:21:29.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:29.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:21:29.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:21:29.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:29.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:21:29.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:21:29.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:29.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:21:29.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:21:29.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:29.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:21:29.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:21:29.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:29.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:21:29.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:21:29.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:29.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:21:29.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:21:29.645 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.645 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:29.645 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.645 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:29.645 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:29.645 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 160.33378600500001, 2024-09-17T21:21:29.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:21:29.645 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:29.646 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:29.646 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:29.646 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:29.646 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:29.646 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:29.646 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:21:29.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.646 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:29.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:29.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:29.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:29.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:29.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:29.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:21:29.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:21:29.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:29.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:21:29.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:21:29.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:29.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:21:29.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:21:29.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:29.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:21:29.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:21:29.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:21:29.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:21:29.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:29.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:21:29.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:21:29.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:21:29.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:21:29.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:29.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:21:29.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:21:29.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:21:29.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:21:29.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:29.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:21:29.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:21:29.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:29.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:21:29.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:21:29.651 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.651 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:29.651 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.651 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:29.651 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:29.651 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 160.32469429400001, 2024-09-17T21:21:29.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:21:29.651 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:29.651 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:29.651 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:29.651 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:29.652 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:29.652 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:29.652 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:21:29.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.652 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:29.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:29.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:29.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:29.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:29.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:29.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:21:29.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:21:29.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:29.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:21:29.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:21:29.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:29.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:21:29.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:21:29.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:29.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:21:29.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:21:29.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:21:29.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:21:29.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:29.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:21:29.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:21:29.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:21:29.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:21:29.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:29.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:21:29.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:21:29.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:29.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:21:29.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:21:29.656 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.656 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:29.656 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.656 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:29.656 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:29.656 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 160.32452224299999, 2024-09-17T21:21:29.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:21:29.656 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:29.657 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:29.657 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:29.657 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:29.657 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:29.657 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:29.657 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:21:29.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.657 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:29.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:29.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:29.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:29.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:29.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:29.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:21:29.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:21:29.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:29.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:21:29.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:21:29.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:29.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:21:29.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:21:29.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:29.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:21:29.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:21:29.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:21:29.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:21:29.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:29.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:21:29.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:21:29.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:21:29.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:21:29.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:29.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:21:29.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:21:29.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:29.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:21:29.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:21:29.661 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.661 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:29.661 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.661 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:29.662 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:29.662 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 160.149589496, 2024-09-17T21:21:29.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:21:29.662 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:29.662 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:29.662 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:29.662 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:29.662 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:29.662 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:29.662 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:21:29.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.662 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:29.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:29.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:29.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:29.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:29.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:29.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:21:29.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:21:29.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:29.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:21:29.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:21:29.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:29.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:21:29.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:21:29.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:29.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:21:29.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:21:29.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:21:29.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:21:29.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:29.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:21:29.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:21:29.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:21:29.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:21:29.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:29.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:21:29.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:21:29.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:29.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:21:29.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:21:29.667 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.667 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:29.667 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.667 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:29.667 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:29.667 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 160.149504939, 2024-09-17T21:21:29.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:21:29.667 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:29.667 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:29.668 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:29.668 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:29.668 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:29.668 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:29.668 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:21:29.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.668 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:29.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:29.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:29.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:29.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:29.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:29.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:29.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:21:29.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:29.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:21:29.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:21:29.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:29.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:21:29.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:21:29.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:29.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:21:29.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:21:29.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:21:29.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:21:29.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:29.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:21:29.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:21:29.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:21:29.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:21:29.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:29.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:21:29.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:21:29.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:29.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:21:29.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:21:29.672 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.672 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:29.672 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.672 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:29.672 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:29.672 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 160.14943930199999, 2024-09-17T21:21:29.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:21:29.672 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:29.672 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:29.673 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:29.673 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:29.673 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:29.673 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:29.673 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:21:29.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.673 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:29.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:29.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:29.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:29.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:29.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:29.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:21:29.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:21:29.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:29.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:21:29.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:21:29.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:29.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:21:29.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:21:29.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:29.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:21:29.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:21:29.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:21:29.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:21:29.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:29.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:21:29.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:21:29.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:21:29.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:21:29.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:29.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:21:29.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:21:29.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:29.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:21:29.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:21:29.677 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.677 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:29.677 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.678 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:29.678 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:29.678 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 108.775864205, 2024-09-17T21:21:29.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:21:29.678 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:29.678 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:29.678 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:29.678 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:29.678 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:21:29.678 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:29.678 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:21:29.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.679 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:29.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:29.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:29.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:29.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:29.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:29.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:21:29.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:21:29.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:29.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:21:29.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:21:29.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:29.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:21:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:21:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:21:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:21:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:21:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:21:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:21:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:21:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:21:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:21:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.683 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:29.683 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:29.683 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 108.77574469300001, 2024-09-17T21:21:29.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:21:29.683 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:29.683 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:29.683 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:29.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:29.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:21:29.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:29.683 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:21:29.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.683 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:29.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:29.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:29.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:29.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:29.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:29.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:21:29.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:21:29.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:29.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:21:29.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:21:29.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:29.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:21:29.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:21:29.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:29.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:21:29.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:21:29.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:21:29.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:21:29.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:29.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:21:29.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:21:29.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:29.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:21:29.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:21:29.687 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.687 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:29.687 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.687 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:994 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 571392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:29.687 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:21:29.687 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.182772718999999, 2024-09-17T21:21:29.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083898465000000005, 2024-09-17T21:21:29.687 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:29.687 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:29.688 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:29.688 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:29.688 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:29.688 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:29.688 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 994 2024-09-17T21:21:29.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.688 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:29.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:29.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:21:29.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:29.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:21:29.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:29.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431026+0000", 2024-09-17T21:21:29.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:21:29.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:29.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431058+0000", 2024-09-17T21:21:29.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1667e-05 2024-09-17T21:21:29.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:29.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431067+0000", 2024-09-17T21:21:29.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0720000000000006e-06 2024-09-17T21:21:29.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:29.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431075+0000", 2024-09-17T21:21:29.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9200000000000004e-06 2024-09-17T21:21:29.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514671+0000", 2024-09-17T21:21:29.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083596229999999994 2024-09-17T21:21:29.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:29.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514741+0000", 2024-09-17T21:21:29.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9929000000000001e-05 2024-09-17T21:21:29.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:29.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.514924+0000", 2024-09-17T21:21:29.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000183122 2024-09-17T21:21:29.691 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.691 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:29.691 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.692 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:995 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 575488~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:29.692 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:21:29.692 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.182713703000005, 2024-09-17T21:21:29.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084214699000000004, 2024-09-17T21:21:29.692 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:29.692 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:29.692 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:29.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:29.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:29.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:29.692 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 995 2024-09-17T21:21:29.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.693 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:29.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:29.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:21:29.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:29.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:21:29.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:29.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431086+0000", 2024-09-17T21:21:29.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7999999999999995e-07 2024-09-17T21:21:29.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:29.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431088+0000", 2024-09-17T21:21:29.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4729999999999999e-06 2024-09-17T21:21:29.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:29.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431126+0000", 2024-09-17T21:21:29.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7583999999999998e-05 2024-09-17T21:21:29.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:29.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431137+0000", 2024-09-17T21:21:29.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1768e-05 2024-09-17T21:21:29.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515003+0000", 2024-09-17T21:21:29.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083865097 2024-09-17T21:21:29.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:29.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515115+0000", 2024-09-17T21:21:29.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112561 2024-09-17T21:21:29.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:29.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515300+0000", 2024-09-17T21:21:29.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018453600000000001 2024-09-17T21:21:29.696 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.696 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:29.696 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.696 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:996 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:29.696 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:29.696 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.182631322000006, 2024-09-17T21:21:29.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084493736999999999, 2024-09-17T21:21:29.696 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:29.697 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:29.697 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:29.697 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:29.697 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:29.697 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:29.697 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 996 2024-09-17T21:21:29.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.697 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:29.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:29.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:29.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:29.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:29.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:29.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431168+0000", 2024-09-17T21:21:29.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4e-07 2024-09-17T21:21:29.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:29.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431172+0000", 2024-09-17T21:21:29.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.608e-06 2024-09-17T21:21:29.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:29.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431180+0000", 2024-09-17T21:21:29.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5650000000000002e-06 2024-09-17T21:21:29.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:29.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431188+0000", 2024-09-17T21:21:29.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:21:29.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515354+0000", 2024-09-17T21:21:29.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084165977000000003 2024-09-17T21:21:29.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:29.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515451+0000", 2024-09-17T21:21:29.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6669000000000006e-05 2024-09-17T21:21:29.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:29.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515661+0000", 2024-09-17T21:21:29.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021021599999999999 2024-09-17T21:21:29.700 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.700 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:29.700 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.701 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:997 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 583680~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:29.701 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:29.701 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.182599926999998, 2024-09-17T21:21:29.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084772419000000002, 2024-09-17T21:21:29.701 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:29.701 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:29.701 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:29.701 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:29.701 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:29.701 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:29.701 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 997 2024-09-17T21:21:29.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.702 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:29.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:29.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:29.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:29.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:29.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:29.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:29.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0400000000000005e-07 2024-09-17T21:21:29.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:29.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431202+0000", 2024-09-17T21:21:29.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:21:29.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:29.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431208+0000", 2024-09-17T21:21:29.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4740000000000002e-06 2024-09-17T21:21:29.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:29.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431214+0000", 2024-09-17T21:21:29.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6550000000000001e-06 2024-09-17T21:21:29.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515741+0000", 2024-09-17T21:21:29.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084526904 2024-09-17T21:21:29.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:29.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515826+0000", 2024-09-17T21:21:29.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4643999999999998e-05 2024-09-17T21:21:29.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:29.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515971+0000", 2024-09-17T21:21:29.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145444 2024-09-17T21:21:29.705 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.705 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:29.705 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.705 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:998 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 587776~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:29.705 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:29.705 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.182576955000002, 2024-09-17T21:21:29.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085108565999999997, 2024-09-17T21:21:29.706 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:29.706 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:29.706 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:29.706 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:29.706 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:29.706 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:29.706 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 998 2024-09-17T21:21:29.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.706 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:29.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:29.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:29.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:29.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:29.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:29.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:29.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:21:29.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:29.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431224+0000", 2024-09-17T21:21:29.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:21:29.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:29.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431231+0000", 2024-09-17T21:21:29.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000002e-06 2024-09-17T21:21:29.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:29.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431236+0000", 2024-09-17T21:21:29.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.648e-06 2024-09-17T21:21:29.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516024+0000", 2024-09-17T21:21:29.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084787584999999999 2024-09-17T21:21:29.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:29.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516122+0000", 2024-09-17T21:21:29.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8076000000000001e-05 2024-09-17T21:21:29.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:29.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516330+0000", 2024-09-17T21:21:29.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020808 2024-09-17T21:21:29.709 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.709 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:29.709 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.709 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1000 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 591872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:29.710 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:29.710 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.182554839999995, 2024-09-17T21:21:29.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085389065, 2024-09-17T21:21:29.710 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:29.710 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:29.710 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:29.710 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:29.710 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:29.710 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:29.710 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1000 2024-09-17T21:21:29.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.710 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:29.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:29.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:29.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:29.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:29.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:29.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:29.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:21:29.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:29.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431246+0000", 2024-09-17T21:21:29.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1749999999999999e-06 2024-09-17T21:21:29.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:29.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431253+0000", 2024-09-17T21:21:29.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2790000000000004e-06 2024-09-17T21:21:29.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:29.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431258+0000", 2024-09-17T21:21:29.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3299999999999998e-06 2024-09-17T21:21:29.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516384+0000", 2024-09-17T21:21:29.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085125549999999994 2024-09-17T21:21:29.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:29.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516460+0000", 2024-09-17T21:21:29.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6359999999999994e-05 2024-09-17T21:21:29.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:29.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516633+0000", 2024-09-17T21:21:29.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017285900000000001 2024-09-17T21:21:29.713 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.713 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:29.713 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.714 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1001 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 595968~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:29.714 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:29.714 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.182531433999998, 2024-09-17T21:21:29.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085614715999999993, 2024-09-17T21:21:29.714 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:29.714 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:29.714 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:29.714 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:29.714 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:29.714 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:29.714 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1001 2024-09-17T21:21:29.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.714 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:29.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:29.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:29.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:29.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:29.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:29.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438066+0000", 2024-09-17T21:21:29.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067989490000000003 2024-09-17T21:21:29.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:29.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438073+0000", 2024-09-17T21:21:29.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8419999999999999e-06 2024-09-17T21:21:29.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:29.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438090+0000", 2024-09-17T21:21:29.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6657e-05 2024-09-17T21:21:29.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:29.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438099+0000", 2024-09-17T21:21:29.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0529999999999996e-06 2024-09-17T21:21:29.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516678+0000", 2024-09-17T21:21:29.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078579422999999995 2024-09-17T21:21:29.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:29.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516739+0000", 2024-09-17T21:21:29.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1315000000000005e-05 2024-09-17T21:21:29.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:29.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516882+0000", 2024-09-17T21:21:29.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014247700000000001 2024-09-17T21:21:29.717 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.717 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:29.718 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.718 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:19998 3.5 3:a4885450:::rbd_object_map.1152d139f9f9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-17T21:21:29.718 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:29.718 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 50.211264188999998, 2024-09-17T21:21:29.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90715489299999996, 2024-09-17T21:21:29.718 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:29.718 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:29.718 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:29.718 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:29.718 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:29.718 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:29.718 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19998 2024-09-17T21:21:29.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.719 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:29.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:29.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:29.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:29.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:29.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:29.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:29.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402537+0000", 2024-09-17T21:21:29.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5600000000000001e-06 2024-09-17T21:21:29.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:29.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402549+0000", 2024-09-17T21:21:29.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1929999999999999e-05 2024-09-17T21:21:29.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:29.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402579+0000", 2024-09-17T21:21:29.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0093000000000001e-05 2024-09-17T21:21:29.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:29.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402595+0000", 2024-09-17T21:21:29.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5855999999999999e-05 2024-09-17T21:21:29.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402731+0000", 2024-09-17T21:21:29.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013610799999999999 2024-09-17T21:21:29.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:29.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402750+0000", 2024-09-17T21:21:29.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8737e-05 2024-09-17T21:21:29.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:29.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.305977+0000", 2024-09-17T21:21:29.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90322768399999998 2024-09-17T21:21:29.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:29.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306236+0000", 2024-09-17T21:21:29.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000258111 2024-09-17T21:21:29.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:29.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306557+0000", 2024-09-17T21:21:29.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032159799999999998 2024-09-17T21:21:29.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:29.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.308286+0000", 2024-09-17T21:21:29.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017292150000000001 2024-09-17T21:21:29.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:29.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309530+0000", 2024-09-17T21:21:29.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00124409 2024-09-17T21:21:29.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:29.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309630+0000", 2024-09-17T21:21:29.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9689e-05 2024-09-17T21:21:29.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:29.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:29.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:29.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309689+0000", 2024-09-17T21:21:29.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9222000000000003e-05 2024-09-17T21:21:29.724 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.724 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:29.724 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.724 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:29.724 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:29.724 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:29.725 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:29.726 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:21:29.727 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:29.721+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:29.727 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:29.826 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:29.821+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:29.894 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:29.894 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:29.894 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:29.894 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:29.908 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:21:29.951 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:29.949+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:30.028 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:30.023+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:30.074 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:30.074 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:30.074 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:30.074 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:30.074 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:30.088 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:21:30.129 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:30.127+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:30.207 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:30.207 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:30.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:30.207 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:30.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.207 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:18003 3.1d 3:b9765580:::rbd_header.11527d2b15f6:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T21:21:30.208 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:30.208 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.846482026000004, 2024-09-17T21:21:30.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94165346900000002, 2024-09-17T21:21:30.208 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.208 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.208 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.208 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.208 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:30.208 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:30.208 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18003 2024-09-17T21:21:30.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.208 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:30.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:30.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357880+0000", 2024-09-17T21:21:30.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:21:30.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357891+0000", 2024-09-17T21:21:30.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0417999999999999e-05 2024-09-17T21:21:30.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357913+0000", 2024-09-17T21:21:30.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2215e-05 2024-09-17T21:21:30.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357922+0000", 2024-09-17T21:21:30.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4779999999999997e-06 2024-09-17T21:21:30.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358023+0000", 2024-09-17T21:21:30.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101025 2024-09-17T21:21:30.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:30.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358040+0000", 2024-09-17T21:21:30.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6291999999999999e-05 2024-09-17T21:21:30.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298203+0000", 2024-09-17T21:21:30.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94016297500000001 2024-09-17T21:21:30.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298626+0000", 2024-09-17T21:21:30.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000422853 2024-09-17T21:21:30.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.299532+0000", 2024-09-17T21:21:30.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00090657300000000004 2024-09-17T21:21:30.212 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.212 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.212 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.213 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15706 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 977920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:30.213 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:21:30.213 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 67.683380235000001, 2024-09-17T21:21:30.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091952328, 2024-09-17T21:21:30.213 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.213 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.213 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.213 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.213 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:30.213 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:30.214 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15706 2024-09-17T21:21:30.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.214 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:21:30.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:21:30.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520982+0000", 2024-09-17T21:21:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.277e-06 2024-09-17T21:21:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520988+0000", 2024-09-17T21:21:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4470000000000003e-06 2024-09-17T21:21:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521008+0000", 2024-09-17T21:21:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9309000000000001e-05 2024-09-17T21:21:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521023+0000", 2024-09-17T21:21:30.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5119999999999999e-05 2024-09-17T21:21:30.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612638+0000", 2024-09-17T21:21:30.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091615328999999995 2024-09-17T21:21:30.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612738+0000", 2024-09-17T21:21:30.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100466 2024-09-17T21:21:30.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612933+0000", 2024-09-17T21:21:30.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019437999999999999 2024-09-17T21:21:30.217 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.217 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.217 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.217 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15707 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 982016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:30.217 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:30.217 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 67.683316778999995, 2024-09-17T21:21:30.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092305228000000003, 2024-09-17T21:21:30.217 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.217 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.218 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.218 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.218 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:30.218 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:30.218 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15707 2024-09-17T21:21:30.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.218 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:30.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:30.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521045+0000", 2024-09-17T21:21:30.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4899999999999999e-06 2024-09-17T21:21:30.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521066+0000", 2024-09-17T21:21:30.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0236000000000001e-05 2024-09-17T21:21:30.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521085+0000", 2024-09-17T21:21:30.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8819999999999999e-05 2024-09-17T21:21:30.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521100+0000", 2024-09-17T21:21:30.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5146e-05 2024-09-17T21:21:30.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612978+0000", 2024-09-17T21:21:30.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091878847 2024-09-17T21:21:30.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613079+0000", 2024-09-17T21:21:30.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100503 2024-09-17T21:21:30.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613349+0000", 2024-09-17T21:21:30.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000270186 2024-09-17T21:21:30.221 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.221 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.221 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.222 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15708 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 986112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:30.222 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:30.222 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 67.683238603999996, 2024-09-17T21:21:30.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092547217000000001, 2024-09-17T21:21:30.222 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.222 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.222 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.222 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.222 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:30.222 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:30.222 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15708 2024-09-17T21:21:30.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.223 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:30.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:30.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521123+0000", 2024-09-17T21:21:30.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.353e-06 2024-09-17T21:21:30.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521129+0000", 2024-09-17T21:21:30.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:21:30.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521145+0000", 2024-09-17T21:21:30.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.611e-05 2024-09-17T21:21:30.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521154+0000", 2024-09-17T21:21:30.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4500000000000004e-06 2024-09-17T21:21:30.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613397+0000", 2024-09-17T21:21:30.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092243275 2024-09-17T21:21:30.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613503+0000", 2024-09-17T21:21:30.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106013 2024-09-17T21:21:30.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613669+0000", 2024-09-17T21:21:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016642299999999999 2024-09-17T21:21:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15709 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 990208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 67.683195588000004, 2024-09-17T21:21:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092728678999999994, 2024-09-17T21:21:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.227 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.227 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:30.227 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:30.227 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15709 2024-09-17T21:21:30.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.227 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:30.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:30.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521166+0000", 2024-09-17T21:21:30.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7000000000000004e-07 2024-09-17T21:21:30.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521169+0000", 2024-09-17T21:21:30.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2660000000000002e-06 2024-09-17T21:21:30.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521181+0000", 2024-09-17T21:21:30.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.208e-05 2024-09-17T21:21:30.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521194+0000", 2024-09-17T21:21:30.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2707e-05 2024-09-17T21:21:30.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613695+0000", 2024-09-17T21:21:30.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092500709 2024-09-17T21:21:30.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613750+0000", 2024-09-17T21:21:30.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.503e-05 2024-09-17T21:21:30.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613894+0000", 2024-09-17T21:21:30.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144117 2024-09-17T21:21:30.230 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.230 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.230 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.230 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15710 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 994304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:30.230 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:30.230 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 67.683151022000004, 2024-09-17T21:21:30.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092957729000000003, 2024-09-17T21:21:30.231 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.231 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.231 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.231 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.231 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:30.231 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:30.231 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15710 2024-09-17T21:21:30.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.231 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:30.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:30.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:30.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8000000000000005e-07 2024-09-17T21:21:30.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521216+0000", 2024-09-17T21:21:30.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3859999999999998e-06 2024-09-17T21:21:30.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521234+0000", 2024-09-17T21:21:30.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:21:30.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521248+0000", 2024-09-17T21:21:30.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:21:30.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613926+0000", 2024-09-17T21:21:30.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092678324000000006 2024-09-17T21:21:30.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613974+0000", 2024-09-17T21:21:30.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7754999999999999e-05 2024-09-17T21:21:30.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614167+0000", 2024-09-17T21:21:30.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193508 2024-09-17T21:21:30.234 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.234 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.234 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.235 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15711 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 998400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:30.235 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:30.235 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 67.683093639999996, 2024-09-17T21:21:30.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093268000000000004, 2024-09-17T21:21:30.235 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.235 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.235 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.235 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.235 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:30.235 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:30.235 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15711 2024-09-17T21:21:30.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.235 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:30.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:30.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521268+0000", 2024-09-17T21:21:30.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-17T21:21:30.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521273+0000", 2024-09-17T21:21:30.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:21:30.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521290+0000", 2024-09-17T21:21:30.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7269999999999999e-05 2024-09-17T21:21:30.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521304+0000", 2024-09-17T21:21:30.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3626e-05 2024-09-17T21:21:30.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614220+0000", 2024-09-17T21:21:30.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092916367 2024-09-17T21:21:30.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614322+0000", 2024-09-17T21:21:30.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102212 2024-09-17T21:21:30.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614535+0000", 2024-09-17T21:21:30.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021266199999999999 2024-09-17T21:21:30.239 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.239 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.239 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.239 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15712 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1002496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:30.239 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:30.239 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 67.683038144999998, 2024-09-17T21:21:30.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093568707000000001, 2024-09-17T21:21:30.239 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.239 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.239 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.239 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.239 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:30.239 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:30.240 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15712 2024-09-17T21:21:30.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.240 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:30.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:30.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521324+0000", 2024-09-17T21:21:30.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.17e-06 2024-09-17T21:21:30.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521334+0000", 2024-09-17T21:21:30.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9369999999999999e-06 2024-09-17T21:21:30.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521356+0000", 2024-09-17T21:21:30.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2045999999999999e-05 2024-09-17T21:21:30.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521370+0000", 2024-09-17T21:21:30.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4270000000000001e-05 2024-09-17T21:21:30.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614583+0000", 2024-09-17T21:21:30.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093213220999999999 2024-09-17T21:21:30.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614680+0000", 2024-09-17T21:21:30.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6900999999999999e-05 2024-09-17T21:21:30.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614891+0000", 2024-09-17T21:21:30.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021116200000000001 2024-09-17T21:21:30.243 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.243 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.243 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.243 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15713 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1006592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:30.243 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:30.243 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 67.682971443, 2024-09-17T21:21:30.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093834641999999996, 2024-09-17T21:21:30.243 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.243 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.243 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.243 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.244 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:30.244 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:30.244 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15713 2024-09-17T21:21:30.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.244 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:30.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:30.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521390+0000", 2024-09-17T21:21:30.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:21:30.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521396+0000", 2024-09-17T21:21:30.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:21:30.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521417+0000", 2024-09-17T21:21:30.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1236000000000001e-05 2024-09-17T21:21:30.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521431+0000", 2024-09-17T21:21:30.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3873000000000001e-05 2024-09-17T21:21:30.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614938+0000", 2024-09-17T21:21:30.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093506522999999994 2024-09-17T21:21:30.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615025+0000", 2024-09-17T21:21:30.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7635999999999999e-05 2024-09-17T21:21:30.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615224+0000", 2024-09-17T21:21:30.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019844700000000001 2024-09-17T21:21:30.247 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.247 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.247 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.247 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15714 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:30.247 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:30.247 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 67.682908186999995, 2024-09-17T21:21:30.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094071925000000001, 2024-09-17T21:21:30.247 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.247 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.248 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.248 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.248 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:30.248 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:30.248 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15714 2024-09-17T21:21:30.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.248 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:30.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:30.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521454+0000", 2024-09-17T21:21:30.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.097e-06 2024-09-17T21:21:30.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521459+0000", 2024-09-17T21:21:30.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2660000000000001e-06 2024-09-17T21:21:30.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521481+0000", 2024-09-17T21:21:30.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1878999999999998e-05 2024-09-17T21:21:30.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521498+0000", 2024-09-17T21:21:30.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6807e-05 2024-09-17T21:21:30.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615271+0000", 2024-09-17T21:21:30.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093773286999999997 2024-09-17T21:21:30.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615366+0000", 2024-09-17T21:21:30.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5322000000000007e-05 2024-09-17T21:21:30.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615524+0000", 2024-09-17T21:21:30.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015826699999999999 2024-09-17T21:21:30.251 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.251 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.251 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.251 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15715 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1014784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:30.251 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:30.251 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 67.682843852000005, 2024-09-17T21:21:30.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094243840999999995, 2024-09-17T21:21:30.252 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.252 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.252 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.252 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.252 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:30.252 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:30.252 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15715 2024-09-17T21:21:30.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.252 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:30.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:30.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521518+0000", 2024-09-17T21:21:30.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4300000000000001e-06 2024-09-17T21:21:30.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521524+0000", 2024-09-17T21:21:30.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5860000000000004e-06 2024-09-17T21:21:30.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521541+0000", 2024-09-17T21:21:30.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7574000000000001e-05 2024-09-17T21:21:30.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521555+0000", 2024-09-17T21:21:30.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3540000000000001e-05 2024-09-17T21:21:30.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615560+0000", 2024-09-17T21:21:30.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.09400472 2024-09-17T21:21:30.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615614+0000", 2024-09-17T21:21:30.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4693e-05 2024-09-17T21:21:30.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615761+0000", 2024-09-17T21:21:30.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000146298 2024-09-17T21:21:30.255 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.255 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.255 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.256 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15716 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1018880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:30.256 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:30.256 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 67.682787507, 2024-09-17T21:21:30.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094400676000000003, 2024-09-17T21:21:30.256 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.256 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.256 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.256 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.256 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:30.256 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:30.256 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15716 2024-09-17T21:21:30.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.256 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:30.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:30.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521574+0000", 2024-09-17T21:21:30.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1939999999999999e-06 2024-09-17T21:21:30.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521579+0000", 2024-09-17T21:21:30.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:21:30.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521597+0000", 2024-09-17T21:21:30.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8074e-05 2024-09-17T21:21:30.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521613+0000", 2024-09-17T21:21:30.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5316e-05 2024-09-17T21:21:30.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615780+0000", 2024-09-17T21:21:30.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094166874999999997 2024-09-17T21:21:30.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615836+0000", 2024-09-17T21:21:30.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6926999999999999e-05 2024-09-17T21:21:30.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615974+0000", 2024-09-17T21:21:30.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013741400000000001 2024-09-17T21:21:30.259 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.260 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.260 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.260 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15717 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1022976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:30.260 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:30.260 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 67.682728811000004, 2024-09-17T21:21:30.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094586723999999997, 2024-09-17T21:21:30.260 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.260 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.260 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.260 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.260 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:30.260 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:30.261 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15717 2024-09-17T21:21:30.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.261 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:30.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:30.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521633+0000", 2024-09-17T21:21:30.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:21:30.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521638+0000", 2024-09-17T21:21:30.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:21:30.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521659+0000", 2024-09-17T21:21:30.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-05 2024-09-17T21:21:30.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521675+0000", 2024-09-17T21:21:30.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5688999999999998e-05 2024-09-17T21:21:30.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616008+0000", 2024-09-17T21:21:30.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094333314000000001 2024-09-17T21:21:30.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616066+0000", 2024-09-17T21:21:30.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7797000000000002e-05 2024-09-17T21:21:30.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616219+0000", 2024-09-17T21:21:30.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015281100000000001 2024-09-17T21:21:30.264 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.264 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.264 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.264 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15718 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:30.264 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:30.264 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 67.682628735999998, 2024-09-17T21:21:30.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094703674000000002, 2024-09-17T21:21:30.265 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.265 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.265 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.265 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.265 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:30.265 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:30.265 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15718 2024-09-17T21:21:30.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.265 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:30.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521734+0000", 2024-09-17T21:21:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.61e-06 2024-09-17T21:21:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521740+0000", 2024-09-17T21:21:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:21:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521754+0000", 2024-09-17T21:21:30.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3793e-05 2024-09-17T21:21:30.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521763+0000", 2024-09-17T21:21:30.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5129999999999996e-06 2024-09-17T21:21:30.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616256+0000", 2024-09-17T21:21:30.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094492919999999994 2024-09-17T21:21:30.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616312+0000", 2024-09-17T21:21:30.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6737000000000002e-05 2024-09-17T21:21:30.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616436+0000", 2024-09-17T21:21:30.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012335800000000001 2024-09-17T21:21:30.268 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.268 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.269 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.269 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15719 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1031168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:30.269 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:30.269 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 67.682575330999995, 2024-09-17T21:21:30.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094888963000000007, 2024-09-17T21:21:30.269 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.269 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.269 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.269 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.269 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:30.269 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:30.269 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15719 2024-09-17T21:21:30.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.270 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:30.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:30.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521786+0000", 2024-09-17T21:21:30.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.074e-06 2024-09-17T21:21:30.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521791+0000", 2024-09-17T21:21:30.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3030000000000001e-06 2024-09-17T21:21:30.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521804+0000", 2024-09-17T21:21:30.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3027000000000001e-05 2024-09-17T21:21:30.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521811+0000", 2024-09-17T21:21:30.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6090000000000003e-06 2024-09-17T21:21:30.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616466+0000", 2024-09-17T21:21:30.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094654294 2024-09-17T21:21:30.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616523+0000", 2024-09-17T21:21:30.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6953999999999997e-05 2024-09-17T21:21:30.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616674+0000", 2024-09-17T21:21:30.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015170200000000001 2024-09-17T21:21:30.273 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.273 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.273 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.273 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15720 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1035264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:30.273 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:30.273 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 67.682538491000003, 2024-09-17T21:21:30.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095067061999999994, 2024-09-17T21:21:30.273 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.274 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.274 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.274 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.274 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:30.274 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:30.274 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15720 2024-09-17T21:21:30.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.274 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:30.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:30.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521823+0000", 2024-09-17T21:21:30.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:21:30.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521826+0000", 2024-09-17T21:21:30.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2100000000000002e-06 2024-09-17T21:21:30.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521835+0000", 2024-09-17T21:21:30.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1630000000000002e-06 2024-09-17T21:21:30.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521842+0000", 2024-09-17T21:21:30.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:21:30.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616711+0000", 2024-09-17T21:21:30.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094868859 2024-09-17T21:21:30.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616764+0000", 2024-09-17T21:21:30.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2778999999999997e-05 2024-09-17T21:21:30.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616889+0000", 2024-09-17T21:21:30.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012533100000000001 2024-09-17T21:21:30.277 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.277 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.277 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.277 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15721 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1039360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:30.277 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:30.278 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 67.682491381999995, 2024-09-17T21:21:30.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095277219999999996, 2024-09-17T21:21:30.278 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.278 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.278 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:30.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:30.278 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15721 2024-09-17T21:21:30.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.278 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:30.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:30.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521871+0000", 2024-09-17T21:21:30.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.443e-06 2024-09-17T21:21:30.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521876+0000", 2024-09-17T21:21:30.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0300000000000001e-06 2024-09-17T21:21:30.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521886+0000", 2024-09-17T21:21:30.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186e-05 2024-09-17T21:21:30.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521893+0000", 2024-09-17T21:21:30.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.08e-06 2024-09-17T21:21:30.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616918+0000", 2024-09-17T21:21:30.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095025223000000006 2024-09-17T21:21:30.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616977+0000", 2024-09-17T21:21:30.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8403999999999999e-05 2024-09-17T21:21:30.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617147+0000", 2024-09-17T21:21:30.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016985399999999999 2024-09-17T21:21:30.281 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.281 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.282 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.282 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15722 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:30.282 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:30.282 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 67.682456932999997, 2024-09-17T21:21:30.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095459783000000006, 2024-09-17T21:21:30.282 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.282 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.282 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:30.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:30.282 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15722 2024-09-17T21:21:30.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.283 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:30.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:30.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:30.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7700000000000004e-07 2024-09-17T21:21:30.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521907+0000", 2024-09-17T21:21:30.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.813e-06 2024-09-17T21:21:30.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521922+0000", 2024-09-17T21:21:30.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4533e-05 2024-09-17T21:21:30.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521934+0000", 2024-09-17T21:21:30.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2469999999999999e-05 2024-09-17T21:21:30.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617187+0000", 2024-09-17T21:21:30.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095252311000000006 2024-09-17T21:21:30.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617240+0000", 2024-09-17T21:21:30.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3740999999999999e-05 2024-09-17T21:21:30.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617364+0000", 2024-09-17T21:21:30.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-17T21:21:30.286 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.286 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.286 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.286 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15723 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:30.286 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:30.286 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 67.682413066999999, 2024-09-17T21:21:30.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095653563999999996, 2024-09-17T21:21:30.286 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.286 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.286 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.286 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.287 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:30.287 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:30.287 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15723 2024-09-17T21:21:30.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.287 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:30.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:30.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521949+0000", 2024-09-17T21:21:30.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:21:30.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521953+0000", 2024-09-17T21:21:30.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5870000000000003e-06 2024-09-17T21:21:30.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521970+0000", 2024-09-17T21:21:30.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7282000000000002e-05 2024-09-17T21:21:30.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521984+0000", 2024-09-17T21:21:30.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:21:30.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617383+0000", 2024-09-17T21:21:30.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095399063000000006 2024-09-17T21:21:30.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617437+0000", 2024-09-17T21:21:30.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3931999999999998e-05 2024-09-17T21:21:30.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617601+0000", 2024-09-17T21:21:30.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016407300000000001 2024-09-17T21:21:30.290 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.290 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.290 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5629.0:2384 3.1c 3:38c1263e:::rbd_data.1152d37183c3.000000000000000b:head [read 11780096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-17T21:21:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.779582237, 2024-09-17T21:21:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0246185320000001, 2024-09-17T21:21:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5629", 2024-09-17T21:21:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3798711329", 2024-09-17T21:21:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2384 2024-09-17T21:21:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.291 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424780+0000", 2024-09-17T21:21:30.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.13e-06 2024-09-17T21:21:30.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424785+0000", 2024-09-17T21:21:30.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4739999999999996e-06 2024-09-17T21:21:30.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424798+0000", 2024-09-17T21:21:30.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2704000000000001e-05 2024-09-17T21:21:30.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424806+0000", 2024-09-17T21:21:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9899999999999997e-06 2024-09-17T21:21:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424868+0000", 2024-09-17T21:21:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999997e-05 2024-09-17T21:21:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424875+0000", 2024-09-17T21:21:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0870000000000001e-06 2024-09-17T21:21:30.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428897+0000", 2024-09-17T21:21:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040226080000000004 2024-09-17T21:21:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428913+0000", 2024-09-17T21:21:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:21:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.448963+0000", 2024-09-17T21:21:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0200498659999999 2024-09-17T21:21:30.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449158+0000", 2024-09-17T21:21:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019581000000000001 2024-09-17T21:21:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449397+0000", 2024-09-17T21:21:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023871 2024-09-17T21:21:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.296 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.296 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:30.298 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:21:30.298 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:30.295+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:30.399 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:30.397+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:30.423 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:30.423 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:30.423 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:30.423 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:30.437 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:21:30.460 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-17T21:21:30.528 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:30.525+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:30.601 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:30.599+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:30.619 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:30.619 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:30.619 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:30.619 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:30.619 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:30.635 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:21:30.701 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:30.699+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:30.802 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:30.799+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:30.822 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:21:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 189.21449859200001, 2024-09-17T21:21:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:21:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.822 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:21:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:21:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:21:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:21:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:21:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:21:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:21:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:21:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:21:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:21:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:21:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:21:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:21:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:21:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:21:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:21:30.826 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:21:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 189.20455195599999, 2024-09-17T21:21:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:21:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:21:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:21:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:21:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:21:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:21:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:21:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:21:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:21:30.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:21:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:21:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:21:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:21:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:21:30.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:21:30.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:21:30.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:21:30.831 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.831 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.831 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:21:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 187.96140111599999, 2024-09-17T21:21:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:21:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:21:30.832 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:21:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:21:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:21:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:21:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:21:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:21:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:21:30.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:21:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:21:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:21:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:21:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:21:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:21:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:21:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:21:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:21:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:21:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:21:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:21:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:21:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:21:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:21:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:21:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:21:30.837 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:21:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 187.95175563000001, 2024-09-17T21:21:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:21:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:21:30.838 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:21:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:21:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:21:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:21:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:21:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:21:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:21:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:21:30.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:21:30.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:21:30.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:21:30.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:21:30.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:21:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:21:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:21:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:21:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:30.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:21:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:21:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:21:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:21:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:21:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:21:30.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:21:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:21:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:21:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 174.582819597, 2024-09-17T21:21:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:21:30.844 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:21:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:21:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:21:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:21:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:21:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:21:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:21:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:21:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:21:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:21:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:21:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:21:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:21:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:21:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:21:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:21:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:21:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 156.381135035, 2024-09-17T21:21:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:21:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.849 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:21:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:21:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:21:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:21:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:21:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:21:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:21:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:21:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:21:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:21:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:21:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:21:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:21:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:21:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:21:30.853 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 156.38064045799999, 2024-09-17T21:21:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:21:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.854 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:21:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:21:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:21:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:21:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:21:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:21:30.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:21:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:21:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:21:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:21:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:21:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:21:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:21:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:21:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:21:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:30.858 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 156.38058507700001, 2024-09-17T21:21:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:21:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:21:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:30.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:21:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:21:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:21:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:21:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:21:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:21:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:21:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:21:30.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:21:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:21:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:21:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:21:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:21:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:21:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 156.38052879899999, 2024-09-17T21:21:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:21:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:21:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:30.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:21:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:21:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:21:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:21:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:21:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:21:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:21:30.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:21:30.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:21:30.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:21:30.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:21:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:21:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:21:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:21:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 156.37957915999999, 2024-09-17T21:21:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:21:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:21:30.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:21:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:21:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:21:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:21:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:21:30.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:21:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:21:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:21:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:21:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:21:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:21:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:21:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:21:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:21:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 156.37950054000001, 2024-09-17T21:21:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:21:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:21:30.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:21:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:21:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:21:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:21:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:21:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:21:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:21:30.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:21:30.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:21:30.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:21:30.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:21:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:21:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:21:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:21:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:30.878 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 156.37938760099999, 2024-09-17T21:21:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:21:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:21:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:21:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:21:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:21:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:21:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:21:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:21:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:21:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:21:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:21:30.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:21:30.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:21:30.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:21:30.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:21:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:21:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 156.37872654700001, 2024-09-17T21:21:30.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:21:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:21:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:21:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:21:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:21:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:21:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:21:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:21:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:21:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:21:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:21:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:21:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:21:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:21:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:21:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:21:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.887 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 156.378661736, 2024-09-17T21:21:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:21:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.888 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:21:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:21:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:21:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:21:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:21:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:21:30.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:21:30.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:21:30.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:21:30.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:21:30.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:21:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:21:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:21:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:21:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:21:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.892 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 156.37860075899999, 2024-09-17T21:21:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:21:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:21:30.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:21:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:21:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:21:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:21:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:21:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:21:30.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:21:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:21:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:21:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:21:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:21:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:21:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:21:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:21:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:21:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.481029574, 2024-09-17T21:21:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:21:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:21:30.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:21:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:21:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:21:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:21:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:21:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:21:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:21:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:21:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:21:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:21:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:21:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:21:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:21:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:21:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:21:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.902 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:21:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.27237117999999, 2024-09-17T21:21:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:21:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:21:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:21:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:21:30.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:21:30.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:21:30.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:21:30.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:21:30.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:21:30.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:21:30.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:21:30.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:21:30.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:21:30.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.909 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:30.909 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:21:30.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:21:30.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.910 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.910 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:21:30.910 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:21:30.910 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.910 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.910 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.911 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:9547 3.2 e147/132 3:42b518d9:::rbd_data.1152f056b73f.0000000000000001:head v 147'513, mlcod=142'511)", 2024-09-17T21:21:30.911 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:30.911 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.347398058, 2024-09-17T21:21:30.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.031059566, 2024-09-17T21:21:30.912 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.912 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.912 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:30.912 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.913 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:30.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.913 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.914 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:30.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.914 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.914 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.914 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471293+0000", 2024-09-17T21:21:30.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2029999999999999e-06 2024-09-17T21:21:30.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.915 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.915 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480606+0000", 2024-09-17T21:21:30.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093123839999999999 2024-09-17T21:21:30.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480653+0000", 2024-09-17T21:21:30.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7852000000000002e-05 2024-09-17T21:21:30.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.917 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480673+0000", 2024-09-17T21:21:30.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0013e-05 2024-09-17T21:21:30.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.918 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.918 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480862+0000", 2024-09-17T21:21:30.918 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018833599999999999 2024-09-17T21:21:30.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.918 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.919 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480906+0000", 2024-09-17T21:21:30.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4162000000000002e-05 2024-09-17T21:21:30.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:30.919 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502259+0000", 2024-09-17T21:21:30.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021353154999999999 2024-09-17T21:21:30.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502351+0000", 2024-09-17T21:21:30.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1460999999999997e-05 2024-09-17T21:21:30.921 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.921 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.921 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.922 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:15234 3.1 e147/134 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head v 147'917, mlcod=145'914)", 2024-09-17T21:21:30.922 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:30.922 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.71473744, 2024-09-17T21:21:30.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045303695999999997, 2024-09-17T21:21:30.922 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.922 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.923 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:30.923 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.923 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.923 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:30.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:30.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.925 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103954+0000", 2024-09-17T21:21:30.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2259999999999999e-06 2024-09-17T21:21:30.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119105+0000", 2024-09-17T21:21:30.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015150871 2024-09-17T21:21:30.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119141+0000", 2024-09-17T21:21:30.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6239000000000003e-05 2024-09-17T21:21:30.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119159+0000", 2024-09-17T21:21:30.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7946999999999999e-05 2024-09-17T21:21:30.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119344+0000", 2024-09-17T21:21:30.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018524599999999999 2024-09-17T21:21:30.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119399+0000", 2024-09-17T21:21:30.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4922e-05 2024-09-17T21:21:30.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:30.930 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149165+0000", 2024-09-17T21:21:30.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029765697000000001 2024-09-17T21:21:30.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.930 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.930 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149255+0000", 2024-09-17T21:21:30.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0548000000000001e-05 2024-09-17T21:21:30.930 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.930 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.930 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:19859 3.8 e147/133 3:12f72409:::rbd_data.1152f056b73f.0000000000000003:head v 147'482, mlcod=145'479)", 2024-09-17T21:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.305846373000001, 2024-09-17T21:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033476895999999999, 2024-09-17T21:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:30.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:30.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:30.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:30.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512845+0000", 2024-09-17T21:21:30.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.886e-06 2024-09-17T21:21:30.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:30.933 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524104+0000", 2024-09-17T21:21:30.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011259668 2024-09-17T21:21:30.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:30.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524148+0000", 2024-09-17T21:21:30.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3386000000000001e-05 2024-09-17T21:21:30.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:30.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524172+0000", 2024-09-17T21:21:30.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4762000000000001e-05 2024-09-17T21:21:30.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:30.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524379+0000", 2024-09-17T21:21:30.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020671599999999999 2024-09-17T21:21:30.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:30.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524421+0000", 2024-09-17T21:21:30.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2182999999999999e-05 2024-09-17T21:21:30.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:30.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546216+0000", 2024-09-17T21:21:30.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021795101000000001 2024-09-17T21:21:30.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:30.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:30.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:30.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546320+0000", 2024-09-17T21:21:30.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103194 2024-09-17T21:21:30.936 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.936 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.936 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.936 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:30.936 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:30.936 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:30.938 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:21:30.938 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:30.935+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:31.039 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:31.035+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:31.068 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:31.068 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:31.068 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:21:31.068 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:31.082 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:21:31.140 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:31.137+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:31.241 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:31.239+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:31.263 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:31.264 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:31.264 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:21:31.264 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:21:31.264 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:31.280 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:21:31.365 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:31.363+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:31.442 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:31.439+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:31.460 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:31.460 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:31.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:31.460 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:31.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.460 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:21:31.460 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:31.460 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 134.947435576, 2024-09-17T21:21:31.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:21:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:21:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:21:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:21:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:31.461 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:21:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:21:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:31.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:21:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:21:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:21:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:21:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:21:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:21:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:21:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:21:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:21:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:21:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:21:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:21:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:21:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:21:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2597 3.1f 3:fb0e9615:::rbd_data.1152a8ae2019.000000000000000c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.618999148, 2024-09-17T21:21:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097064994000000002, 2024-09-17T21:21:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2597 2024-09-17T21:21:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:31.466 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838628+0000", 2024-09-17T21:21:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.187e-06 2024-09-17T21:21:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:31.467 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838654+0000", 2024-09-17T21:21:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:21:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838696+0000", 2024-09-17T21:21:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1226000000000001e-05 2024-09-17T21:21:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838715+0000", 2024-09-17T21:21:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9646000000000001e-05 2024-09-17T21:21:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.468 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838937+0000", 2024-09-17T21:21:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022163 2024-09-17T21:21:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.839284+0000", 2024-09-17T21:21:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034753799999999999 2024-09-17T21:21:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:31.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935691+0000", 2024-09-17T21:21:31.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.096406360999999996 2024-09-17T21:21:31.470 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.470 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:31.470 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.470 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2641 3.12 3:488a729f:::rbd_data.1152a8ae2019.0000000000000022:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.615542375, 2024-09-17T21:21:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098280953000000004, 2024-09-17T21:21:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2641 2024-09-17T21:21:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842083+0000", 2024-09-17T21:21:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5300000000000002e-07 2024-09-17T21:21:31.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842088+0000", 2024-09-17T21:21:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:21:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842105+0000", 2024-09-17T21:21:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6773e-05 2024-09-17T21:21:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842119+0000", 2024-09-17T21:21:31.473 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:21:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842320+0000", 2024-09-17T21:21:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020052700000000001 2024-09-17T21:21:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842845+0000", 2024-09-17T21:21:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052508600000000002 2024-09-17T21:21:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940363+0000", 2024-09-17T21:21:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097518795000000005 2024-09-17T21:21:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2674 3.12 3:4c87e3ac:::rbd_data.1152a8ae2019.0000000000000033:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.61517973799999, 2024-09-17T21:21:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098025125000000005, 2024-09-17T21:21:31.475 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2674 2024-09-17T21:21:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:31.476 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842447+0000", 2024-09-17T21:21:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3999999999999999e-06 2024-09-17T21:21:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.477 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842453+0000", 2024-09-17T21:21:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:21:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842472+0000", 2024-09-17T21:21:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9006000000000001e-05 2024-09-17T21:21:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842487+0000", 2024-09-17T21:21:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4866e-05 2024-09-17T21:21:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843594+0000", 2024-09-17T21:21:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011073249999999999 2024-09-17T21:21:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843668+0000", 2024-09-17T21:21:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3188000000000005e-05 2024-09-17T21:21:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843893+0000", 2024-09-17T21:21:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022503399999999999 2024-09-17T21:21:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850439+0000", 2024-09-17T21:21:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065467880000000004 2024-09-17T21:21:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940409+0000", 2024-09-17T21:21:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089969511000000002 2024-09-17T21:21:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940444+0000", 2024-09-17T21:21:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5345999999999997e-05 2024-09-17T21:21:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940470+0000", 2024-09-17T21:21:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5964000000000001e-05 2024-09-17T21:21:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2707 3.1f 3:f805e65e:::rbd_data.1152a8ae2019.0000000000000049:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.610133077, 2024-09-17T21:21:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088327300999999997, 2024-09-17T21:21:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2707 2024-09-17T21:21:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.482 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847494+0000", 2024-09-17T21:21:31.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:21:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847504+0000", 2024-09-17T21:21:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:21:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847526+0000", 2024-09-17T21:21:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1976000000000001e-05 2024-09-17T21:21:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:31.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847541+0000", 2024-09-17T21:21:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5277e-05 2024-09-17T21:21:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856142+0000", 2024-09-17T21:21:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086006710000000007 2024-09-17T21:21:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856435+0000", 2024-09-17T21:21:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029301499999999999 2024-09-17T21:21:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935819+0000", 2024-09-17T21:21:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079384448999999996 2024-09-17T21:21:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2730 3.12 3:48b3422a:::rbd_data.1152a8ae2019.0000000000000057:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.600940178, 2024-09-17T21:21:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083879975999999995, 2024-09-17T21:21:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:31.486 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2730 2024-09-17T21:21:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856687+0000", 2024-09-17T21:21:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:21:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856697+0000", 2024-09-17T21:21:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0207000000000001e-05 2024-09-17T21:21:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856808+0000", 2024-09-17T21:21:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011071200000000001 2024-09-17T21:21:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856828+0000", 2024-09-17T21:21:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0219000000000001e-05 2024-09-17T21:21:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856907+0000", 2024-09-17T21:21:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9661999999999997e-05 2024-09-17T21:21:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.857019+0000", 2024-09-17T21:21:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011148499999999999 2024-09-17T21:21:31.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940565+0000", 2024-09-17T21:21:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083545791999999994 2024-09-17T21:21:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2743 3.12 3:4d036e62:::rbd_data.1152a8ae2019.000000000000005f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.596994677, 2024-09-17T21:21:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080059962999999998, 2024-09-17T21:21:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:31.490 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:31.491 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:31.491 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:31.491 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:31.491 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:31.491 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2743 2024-09-17T21:21:31.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.491 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:31.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:31.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:31.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:31.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:31.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:31.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860631+0000", 2024-09-17T21:21:31.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:21:31.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:31.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860636+0000", 2024-09-17T21:21:31.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5279999999999997e-06 2024-09-17T21:21:31.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:31.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860649+0000", 2024-09-17T21:21:31.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-17T21:21:31.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:31.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860655+0000", 2024-09-17T21:21:31.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:21:31.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860731+0000", 2024-09-17T21:21:31.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5920000000000005e-05 2024-09-17T21:21:31.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:31.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.861070+0000", 2024-09-17T21:21:31.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338535 2024-09-17T21:21:31.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:31.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940690+0000", 2024-09-17T21:21:31.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079620503999999995 2024-09-17T21:21:31.494 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.494 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:31.494 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.495 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3378 3.12 3:4832a51b:::rbd_data.1152a8ae2019.000000000000006c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:31.495 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:31.495 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.56863082700001, 2024-09-17T21:21:31.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051812516000000003, 2024-09-17T21:21:31.495 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:31.495 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:31.495 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:31.495 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:31.495 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:31.495 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:31.495 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3378 2024-09-17T21:21:31.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.496 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:31.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.496 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:31.496 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:31.496 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.496 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:31.496 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:31.496 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.496 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:31.497 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888995+0000", 2024-09-17T21:21:31.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:21:31.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.497 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:31.497 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889006+0000", 2024-09-17T21:21:31.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1012999999999999e-05 2024-09-17T21:21:31.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.497 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:31.497 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889032+0000", 2024-09-17T21:21:31.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6441e-05 2024-09-17T21:21:31.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:31.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889040+0000", 2024-09-17T21:21:31.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0639999999999994e-06 2024-09-17T21:21:31.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889278+0000", 2024-09-17T21:21:31.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023785899999999999 2024-09-17T21:21:31.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:31.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889408+0000", 2024-09-17T21:21:31.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129219 2024-09-17T21:21:31.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:31.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889723+0000", 2024-09-17T21:21:31.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000315264 2024-09-17T21:21:31.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:31.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.891972+0000", 2024-09-17T21:21:31.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002249633 2024-09-17T21:21:31.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:31.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940733+0000", 2024-09-17T21:21:31.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048760626000000001 2024-09-17T21:21:31.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:31.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940785+0000", 2024-09-17T21:21:31.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1906000000000001e-05 2024-09-17T21:21:31.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:31.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940807+0000", 2024-09-17T21:21:31.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1554999999999999e-05 2024-09-17T21:21:31.500 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.500 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:31.501 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.501 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3379 3.12 3:4e83fd71:::rbd_data.1152a8ae2019.000000000000006d:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:31.501 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:31.501 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.56854662000001, 2024-09-17T21:21:31.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051804144000000003, 2024-09-17T21:21:31.501 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:31.501 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:31.501 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:31.501 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:31.501 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:31.501 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:31.501 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3379 2024-09-17T21:21:31.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.502 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:31.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:31.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:31.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:31.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:31.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:31.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889079+0000", 2024-09-17T21:21:31.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1800000000000004e-07 2024-09-17T21:21:31.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:31.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889083+0000", 2024-09-17T21:21:31.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:21:31.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:31.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889094+0000", 2024-09-17T21:21:31.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1562999999999999e-05 2024-09-17T21:21:31.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:31.504 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889102+0000", 2024-09-17T21:21:31.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.401e-06 2024-09-17T21:21:31.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.504 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:21:31.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014279259999999999 2024-09-17T21:21:31.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:31.504 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890609+0000", 2024-09-17T21:21:31.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9566000000000004e-05 2024-09-17T21:21:31.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.505 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:31.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890789+0000", 2024-09-17T21:21:31.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017949700000000001 2024-09-17T21:21:31.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.505 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:31.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.893929+0000", 2024-09-17T21:21:31.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003140538 2024-09-17T21:21:31.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:31.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940834+0000", 2024-09-17T21:21:31.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046904811999999997 2024-09-17T21:21:31.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:31.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940860+0000", 2024-09-17T21:21:31.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5341999999999999e-05 2024-09-17T21:21:31.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.507 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:31.507 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940882+0000", 2024-09-17T21:21:31.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2878999999999999e-05 2024-09-17T21:21:31.507 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.507 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:31.507 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.507 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:2644 3.1e 3:79615585:::rbd_data.115266d0b9f2.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2475520~14848,write 3106304~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:21:31.507 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:31.507 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 101.945918217, 2024-09-17T21:21:31.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050733484000000002, 2024-09-17T21:21:31.507 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:31.507 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:31.507 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:31.508 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:31.508 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:21:31.508 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:21:31.508 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2644 2024-09-17T21:21:31.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.508 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:31.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.508 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:31.508 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:31.508 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:31.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:31.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:31.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511708+0000", 2024-09-17T21:21:31.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-17T21:21:31.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:31.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522196+0000", 2024-09-17T21:21:31.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010487712999999999 2024-09-17T21:21:31.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:31.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522223+0000", 2024-09-17T21:21:31.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7064999999999999e-05 2024-09-17T21:21:31.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:31.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522236+0000", 2024-09-17T21:21:31.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2931e-05 2024-09-17T21:21:31.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522348+0000", 2024-09-17T21:21:31.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111906 2024-09-17T21:21:31.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:31.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522554+0000", 2024-09-17T21:21:31.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020632699999999999 2024-09-17T21:21:31.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:21:31.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.523185+0000", 2024-09-17T21:21:31.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063081400000000005 2024-09-17T21:21:31.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:31.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.547011+0000", 2024-09-17T21:21:31.512 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023825570000000001 2024-09-17T21:21:31.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.512 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:31.512 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562268+0000", 2024-09-17T21:21:31.512 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015257514 2024-09-17T21:21:31.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.512 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:31.512 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562367+0000", 2024-09-17T21:21:31.512 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8678000000000006e-05 2024-09-17T21:21:31.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.513 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:31.513 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562440+0000", 2024-09-17T21:21:31.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3337000000000006e-05 2024-09-17T21:21:31.513 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.513 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:31.513 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.513 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:231 3.1 3:856fed78:::rbd_data.115220dd393e.000000000000000b:head [read 0~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:31.513 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:31.513 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.503162947999996, 2024-09-17T21:21:31.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34254251699999999, 2024-09-17T21:21:31.513 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:31.513 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:31.514 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:31.514 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:31.514 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:31.514 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:31.514 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 231 2024-09-17T21:21:31.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.514 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:31.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.514 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:31.514 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:31.514 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.514 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:31.515 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:31.515 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.515 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:31.515 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954464+0000", 2024-09-17T21:21:31.515 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:21:31.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.515 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:31.515 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954475+0000", 2024-09-17T21:21:31.515 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1146e-05 2024-09-17T21:21:31.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.515 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:31.516 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954506+0000", 2024-09-17T21:21:31.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0292999999999999e-05 2024-09-17T21:21:31.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.516 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:31.516 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954524+0000", 2024-09-17T21:21:31.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8430000000000001e-05 2024-09-17T21:21:31.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.516 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.516 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954661+0000", 2024-09-17T21:21:31.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136403 2024-09-17T21:21:31.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:31.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954680+0000", 2024-09-17T21:21:31.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9743000000000001e-05 2024-09-17T21:21:31.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.296896+0000", 2024-09-17T21:21:31.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34221550499999998 2024-09-17T21:21:31.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:31.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297004+0000", 2024-09-17T21:21:31.518 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010865399999999999 2024-09-17T21:21:31.518 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.518 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:31.518 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.518 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:56 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:31.518 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:31.518 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.347529250999997, 2024-09-17T21:21:31.518 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18718773899999999, 2024-09-17T21:21:31.518 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:31.518 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:31.518 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:31.518 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:31.519 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:21:31.519 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:21:31.519 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 56 2024-09-17T21:21:31.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.519 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:31.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.519 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:31.519 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:31.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.519 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:31.519 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:31.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.520 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:31.520 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110098+0000", 2024-09-17T21:21:31.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:21:31.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.520 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:31.520 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110111+0000", 2024-09-17T21:21:31.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2289999999999999e-05 2024-09-17T21:21:31.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.520 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:31.520 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110143+0000", 2024-09-17T21:21:31.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1979e-05 2024-09-17T21:21:31.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:31.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110161+0000", 2024-09-17T21:21:31.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7839e-05 2024-09-17T21:21:31.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110319+0000", 2024-09-17T21:21:31.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015835400000000001 2024-09-17T21:21:31.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.522 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.522 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:31.522 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110353+0000", 2024-09-17T21:21:31.522 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3849000000000001e-05 2024-09-17T21:21:31.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.522 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.522 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.522 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297041+0000", 2024-09-17T21:21:31.522 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18668842299999999 2024-09-17T21:21:31.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.522 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.522 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:31.522 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297169+0000", 2024-09-17T21:21:31.523 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012786199999999999 2024-09-17T21:21:31.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.523 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.523 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:31.523 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297283+0000", 2024-09-17T21:21:31.523 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011430300000000001 2024-09-17T21:21:31.523 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.523 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:31.523 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.523 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.523 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:68 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:31.523 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:31.523 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.338988319999999, 2024-09-17T21:21:31.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17885582999999999, 2024-09-17T21:21:31.524 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:31.524 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:31.524 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:31.524 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:31.524 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:21:31.524 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:21:31.524 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 68 2024-09-17T21:21:31.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.524 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:31.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:31.524 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:31.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.525 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.525 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:31.525 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:31.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.525 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.525 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:31.525 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118639+0000", 2024-09-17T21:21:31.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.306e-06 2024-09-17T21:21:31.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.525 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:31.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118651+0000", 2024-09-17T21:21:31.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2567e-05 2024-09-17T21:21:31.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:31.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118682+0000", 2024-09-17T21:21:31.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0502e-05 2024-09-17T21:21:31.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:31.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118707+0000", 2024-09-17T21:21:31.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5463e-05 2024-09-17T21:21:31.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.527 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118889+0000", 2024-09-17T21:21:31.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018150999999999999 2024-09-17T21:21:31.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:31.527 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118910+0000", 2024-09-17T21:21:31.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:21:31.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.527 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297313+0000", 2024-09-17T21:21:31.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17840366699999999 2024-09-17T21:21:31.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.528 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:31.528 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297389+0000", 2024-09-17T21:21:31.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-17T21:21:31.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.528 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:31.528 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297492+0000", 2024-09-17T21:21:31.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103019 2024-09-17T21:21:31.529 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.529 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:31.529 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.529 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.529 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5890.0:15234 3.1 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8451072~29696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-17T21:21:31.529 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:31.529 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.364339508, 2024-09-17T21:21:31.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.057026052000000001, 2024-09-17T21:21:31.529 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:31.529 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:31.529 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:31.529 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:31.529 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5890", 2024-09-17T21:21:31.529 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3064344947", 2024-09-17T21:21:31.530 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 15234 2024-09-17T21:21:31.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.530 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:31.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.530 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:31.530 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:31.530 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.530 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:31.530 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:31.530 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.530 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:31.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093289+0000", 2024-09-17T21:21:31.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4939999999999999e-06 2024-09-17T21:21:31.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:31.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103074+0000", 2024-09-17T21:21:31.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00978564 2024-09-17T21:21:31.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:31.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103112+0000", 2024-09-17T21:21:31.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7246999999999998e-05 2024-09-17T21:21:31.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:31.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103128+0000", 2024-09-17T21:21:31.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5812999999999999e-05 2024-09-17T21:21:31.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103207+0000", 2024-09-17T21:21:31.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9785999999999998e-05 2024-09-17T21:21:31.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:31.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103315+0000", 2024-09-17T21:21:31.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107396 2024-09-17T21:21:31.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:31.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103957+0000", 2024-09-17T21:21:31.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000642524 2024-09-17T21:21:31.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:31.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.123856+0000", 2024-09-17T21:21:31.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019898376999999998 2024-09-17T21:21:31.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:31.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150190+0000", 2024-09-17T21:21:31.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026334682000000002 2024-09-17T21:21:31.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:31.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150255+0000", 2024-09-17T21:21:31.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4770999999999994e-05 2024-09-17T21:21:31.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:31.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150311+0000", 2024-09-17T21:21:31.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6322e-05 2024-09-17T21:21:31.534 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.534 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:31.535 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.535 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4237 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 411648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:31.535 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753637+0000", 2024-09-17T21:21:31.535 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.703987902000001, 2024-09-17T21:21:31.535 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049394367000000002, 2024-09-17T21:21:31.535 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:31.535 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:31.535 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:31.535 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:31.535 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:31.535 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:31.535 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4237 2024-09-17T21:21:31.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.536 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:31.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.536 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:31.536 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753637+0000", 2024-09-17T21:21:31.536 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.536 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:31.536 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753637+0000", 2024-09-17T21:21:31.536 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.536 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:31.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753638+0000", 2024-09-17T21:21:31.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5199999999999997e-07 2024-09-17T21:21:31.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:31.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753640+0000", 2024-09-17T21:21:31.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2819999999999999e-06 2024-09-17T21:21:31.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:31.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753647+0000", 2024-09-17T21:21:31.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8689999999999998e-06 2024-09-17T21:21:31.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:31.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753653+0000", 2024-09-17T21:21:31.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8799999999999996e-06 2024-09-17T21:21:31.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.802605+0000", 2024-09-17T21:21:31.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048952298999999998 2024-09-17T21:21:31.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:31.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.802795+0000", 2024-09-17T21:21:31.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190173 2024-09-17T21:21:31.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:31.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803031+0000", 2024-09-17T21:21:31.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000236312 2024-09-17T21:21:31.539 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.539 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:31.539 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.539 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4238 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 412672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:31.539 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753660+0000", 2024-09-17T21:21:31.539 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.703964694, 2024-09-17T21:21:31.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049651503999999999, 2024-09-17T21:21:31.539 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:31.540 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:31.540 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:31.540 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:31.540 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:31.540 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:31.540 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4238 2024-09-17T21:21:31.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.540 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:31.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:31.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753660+0000", 2024-09-17T21:21:31.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.541 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:31.541 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753660+0000", 2024-09-17T21:21:31.541 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.541 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:31.541 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753661+0000", 2024-09-17T21:21:31.541 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6000000000000004e-07 2024-09-17T21:21:31.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.541 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:31.541 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753663+0000", 2024-09-17T21:21:31.541 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3489999999999999e-06 2024-09-17T21:21:31.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.542 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:31.542 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753670+0000", 2024-09-17T21:21:31.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9539999999999998e-06 2024-09-17T21:21:31.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.542 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:31.542 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753676+0000", 2024-09-17T21:21:31.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6169999999999998e-06 2024-09-17T21:21:31.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.542 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.542 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803132+0000", 2024-09-17T21:21:31.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049456583999999998 2024-09-17T21:21:31.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:31.543 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803224+0000", 2024-09-17T21:21:31.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2139000000000007e-05 2024-09-17T21:21:31.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:31.543 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803312+0000", 2024-09-17T21:21:31.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7300999999999996e-05 2024-09-17T21:21:31.544 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.544 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:31.544 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.544 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4239 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 424960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:31.544 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:21:31.544 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.703941851, 2024-09-17T21:21:31.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050123167000000003, 2024-09-17T21:21:31.544 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:31.544 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:31.544 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:31.545 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:31.545 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:31.545 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:31.545 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4239 2024-09-17T21:21:31.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.545 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:31.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:31.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:21:31.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:31.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:21:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753684+0000", 2024-09-17T21:21:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.37e-07 2024-09-17T21:21:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753686+0000", 2024-09-17T21:21:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.322e-06 2024-09-17T21:21:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753693+0000", 2024-09-17T21:21:31.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0890000000000002e-06 2024-09-17T21:21:31.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:31.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753699+0000", 2024-09-17T21:21:31.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:21:31.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803390+0000", 2024-09-17T21:21:31.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049690506000000002 2024-09-17T21:21:31.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:31.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803569+0000", 2024-09-17T21:21:31.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179784 2024-09-17T21:21:31.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:31.548 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803806+0000", 2024-09-17T21:21:31.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023683499999999999 2024-09-17T21:21:31.548 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.548 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:31.548 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.548 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4240 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 413696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:31.548 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:31.549 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.703918123000001, 2024-09-17T21:21:31.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050331353000000002, 2024-09-17T21:21:31.549 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:31.549 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:31.549 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:31.549 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:31.549 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:31.549 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:31.549 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4240 2024-09-17T21:21:31.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.550 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:31.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:31.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:31.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:31.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:31.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:31.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:31.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5700000000000002e-07 2024-09-17T21:21:31.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:31.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753710+0000", 2024-09-17T21:21:31.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.385e-06 2024-09-17T21:21:31.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:31.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753717+0000", 2024-09-17T21:21:31.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.832e-06 2024-09-17T21:21:31.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:31.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753722+0000", 2024-09-17T21:21:31.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8050000000000003e-06 2024-09-17T21:21:31.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803880+0000", 2024-09-17T21:21:31.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050157641000000003 2024-09-17T21:21:31.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:31.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803945+0000", 2024-09-17T21:21:31.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5357000000000005e-05 2024-09-17T21:21:31.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:31.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804038+0000", 2024-09-17T21:21:31.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2775999999999994e-05 2024-09-17T21:21:31.553 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.554 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:31.554 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.554 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4241 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 415744~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:31.554 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:31.554 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.703895007, 2024-09-17T21:21:31.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050578790999999998, 2024-09-17T21:21:31.554 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:31.554 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:31.554 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:31.554 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:31.554 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:31.554 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:31.554 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4241 2024-09-17T21:21:31.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.555 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:31.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:31.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:31.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:31.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:31.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:31.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:31.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5700000000000002e-07 2024-09-17T21:21:31.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:31.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753733+0000", 2024-09-17T21:21:31.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2570000000000002e-06 2024-09-17T21:21:31.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:31.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753740+0000", 2024-09-17T21:21:31.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.849e-06 2024-09-17T21:21:31.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:31.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753745+0000", 2024-09-17T21:21:31.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7450000000000001e-06 2024-09-17T21:21:31.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804115+0000", 2024-09-17T21:21:31.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050370038999999998 2024-09-17T21:21:31.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:31.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804223+0000", 2024-09-17T21:21:31.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010786 2024-09-17T21:21:31.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:31.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804309+0000", 2024-09-17T21:21:31.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5483999999999997e-05 2024-09-17T21:21:31.558 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.558 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:31.558 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.559 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4242 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 414720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:31.559 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:31.559 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.703872119, 2024-09-17T21:21:31.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050748148999999999, 2024-09-17T21:21:31.559 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:31.559 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:31.559 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:31.559 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:31.559 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:31.559 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:31.559 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4242 2024-09-17T21:21:31.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.559 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:31.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:31.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:31.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.560 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:31.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:31.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.560 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:31.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:31.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6599999999999996e-07 2024-09-17T21:21:31.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:31.561 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753756+0000", 2024-09-17T21:21:31.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.368e-06 2024-09-17T21:21:31.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:31.561 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753763+0000", 2024-09-17T21:21:31.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9349999999999997e-06 2024-09-17T21:21:31.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:31.561 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753768+0000", 2024-09-17T21:21:31.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7620000000000001e-06 2024-09-17T21:21:31.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804343+0000", 2024-09-17T21:21:31.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050575050000000003 2024-09-17T21:21:31.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:31.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804397+0000", 2024-09-17T21:21:31.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4070000000000002e-05 2024-09-17T21:21:31.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:31.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804501+0000", 2024-09-17T21:21:31.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103398 2024-09-17T21:21:31.563 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.563 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:31.563 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.563 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.563 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:31.563 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:31.565 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:21:31.565 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:31.563+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:31.666 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:31.661+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:31.728 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:21:31.750 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:31.751 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:31.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.751 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25488 3.9 3.3fb01fa9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-17T21:21:31.751 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:31.473013+0000", 2024-09-17T21:21:31.751 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.27677790400000002, 2024-09-17T21:21:31.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.27680666500000001, 2024-09-17T21:21:31.752 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:31.752 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:31.752 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for peered", 2024-09-17T21:21:31.752 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:31.752 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:21:31.752 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:31.753 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25488 2024-09-17T21:21:31.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.753 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:31.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:31.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:31.473013+0000", 2024-09-17T21:21:31.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:31.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:31.473013+0000", 2024-09-17T21:21:31.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:31.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:31.473016+0000", 2024-09-17T21:21:31.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5770000000000001e-06 2024-09-17T21:21:31.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:31.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:31.473025+0000", 2024-09-17T21:21:31.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0820000000000005e-06 2024-09-17T21:21:31.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:31.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:31.473051+0000", 2024-09-17T21:21:31.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5775999999999999e-05 2024-09-17T21:21:31.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:31.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:31.473066+0000", 2024-09-17T21:21:31.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5379e-05 2024-09-17T21:21:31.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:31.473188+0000", 2024-09-17T21:21:31.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000122065 2024-09-17T21:21:31.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-17T21:21:31.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:31.473204+0000", 2024-09-17T21:21:31.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5911000000000001e-05 2024-09-17T21:21:31.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:31.477302+0000", 2024-09-17T21:21:31.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0040979240000000002 2024-09-17T21:21:31.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:31.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:31.477312+0000", 2024-09-17T21:21:31.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8709999999999992e-06 2024-09-17T21:21:31.761 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.761 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:31.761 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.761 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25489 3.9 3.3fb01fa9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-17T21:21:31.762 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:31.479428+0000", 2024-09-17T21:21:31.762 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.27036268800000002, 2024-09-17T21:21:31.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.27049230299999999, 2024-09-17T21:21:31.762 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:31.762 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:31.763 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for peered", 2024-09-17T21:21:31.763 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:31.763 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:21:31.763 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:31.763 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25489 2024-09-17T21:21:31.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.764 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:31.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:31.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:31.479428+0000", 2024-09-17T21:21:31.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:31.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:31.479428+0000", 2024-09-17T21:21:31.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:31.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:31.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:31.479430+0000", 2024-09-17T21:21:31.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5990000000000001e-06 2024-09-17T21:21:31.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:31.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:31.479437+0000", 2024-09-17T21:21:31.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.674e-06 2024-09-17T21:21:31.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:31.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:31.479448+0000", 2024-09-17T21:21:31.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1334e-05 2024-09-17T21:21:31.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:31.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:31.479454+0000", 2024-09-17T21:21:31.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2450000000000003e-06 2024-09-17T21:21:31.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:31.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:31.479517+0000", 2024-09-17T21:21:31.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2356999999999999e-05 2024-09-17T21:21:31.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:31.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:31.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:31.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:31.479525+0000", 2024-09-17T21:21:31.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4309999999999994e-06 2024-09-17T21:21:31.770 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.770 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:31.771 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.771 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:31.771 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:21:31.771 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 2 2024-09-17T21:21:31.771 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:31.773 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:21:31.774 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:31.769+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:31.874 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:31.869+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:31.897 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:31.897 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:31.917 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:31.917 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:31.937 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:21:31.937 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:21:31.953 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:31.953 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:31.962 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:31.962 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:31.970 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:21:31.975 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:31.973+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:31.987 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:31.987 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:32.013 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:32.013 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:32.077 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:32.075+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:32.093 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:32.093 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:32.107 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:21:32.193 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:32.191+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:32.266 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:32.266 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:32.278 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:32.275+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:32.281 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:21:32.378 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:32.373+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:32.460 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:32.460 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:32.475 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:21:32.480 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:32.477+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:32.497 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:21:32.580 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:32.575+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:32.606 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:32.606 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:32.621 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:21:32.680 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:32.677+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:32.751 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:32.751 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:32.767 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:21:32.781 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:32.779+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:32.882 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:32.879+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:32.946 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:32.946 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:32.962 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:21:33.007 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:33.005+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:33.080 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:33.080 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:33.084 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:33.079+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:33.096 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:21:33.187 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:33.185+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:33.281 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:33.281 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:33.286 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:33.281+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:33.297 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:21:33.307 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-17T21:21:33.386 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:33.383+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:33.470 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5] out_osds: [0, 2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:21:33.471 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:21:33.471 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-09-17T21:21:33.471 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-17T21:21:33.478 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:33.478 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:33.487 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:33.485+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:33.494 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:21:33.588 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:33.585+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:33.666 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:33.666 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:33.681 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:21:33.723 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:33.721+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:33.789 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:33.787+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:33.822 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-09-17T21:21:33.819+0000 7fe37768f640 -1 mon.a@0(leader).osd e176 definitely_dead 0 2024-09-17T21:21:33.847 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:33.847 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:33.861 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:21:33.890 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:33.887+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:33.983 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:33.983 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:33.991 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:33.987+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:33.997 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:21:34.032 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:21:34.093 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:34.089+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:34.114 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:34.114 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:34.129 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:21:34.182 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:34.182 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:34.193 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:34.191+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:34.199 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:34.216 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:21:34.216 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:21:34.237 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:34.258 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:34.258 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:34.259 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:34.259 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:34.259 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:21:34.260 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:34.274 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:21:34.280 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:34.280 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:34.297 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:34.295+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:34.299 INFO:teuthology.orchestra.run.smithi083.stderr:marked in osd.0. 2024-09-17T21:21:34.326 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-09-17T21:21:34.395 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:34.391+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:34.412 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:34.413 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:34.413 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:21:34.413 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:21:34.413 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:34.427 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:21:34.497 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:34.493+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:34.593 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:34.594 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:34.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:34.594 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:34.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.594 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:21:34.594 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:34.594 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 138.07996809799999, 2024-09-17T21:21:34.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:21:34.594 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:34.594 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:34.595 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:34.595 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:34.595 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:21:34.595 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:21:34.595 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:21:34.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.595 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:34.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.595 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:34.595 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:34.595 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.595 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:34.596 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:34.596 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.596 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:34.596 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:21:34.596 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:21:34.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:34.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:21:34.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:21:34.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:34.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:21:34.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:21:34.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:34.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:21:34.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:21:34.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:21:34.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:21:34.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:34.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:21:34.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:21:34.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:21:34.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:21:34.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:34.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:21:34.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:21:34.599 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.599 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:34.599 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.600 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2597 3.1f 3:fb0e9615:::rbd_data.1152a8ae2019.000000000000000c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:34.600 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:34.600 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 113.75153167000001, 2024-09-17T21:21:34.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097064994000000002, 2024-09-17T21:21:34.600 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:34.600 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:34.600 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:34.600 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:34.600 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:34.600 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:34.601 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2597 2024-09-17T21:21:34.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.601 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:34.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:34.601 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:34.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:34.601 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:34.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.602 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:34.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838628+0000", 2024-09-17T21:21:34.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.187e-06 2024-09-17T21:21:34.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.602 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:34.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838654+0000", 2024-09-17T21:21:34.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:21:34.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.602 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:34.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838696+0000", 2024-09-17T21:21:34.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1226000000000001e-05 2024-09-17T21:21:34.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:34.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838715+0000", 2024-09-17T21:21:34.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9646000000000001e-05 2024-09-17T21:21:34.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838937+0000", 2024-09-17T21:21:34.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022163 2024-09-17T21:21:34.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:34.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.839284+0000", 2024-09-17T21:21:34.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034753799999999999 2024-09-17T21:21:34.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:34.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935691+0000", 2024-09-17T21:21:34.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.096406360999999996 2024-09-17T21:21:34.604 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.604 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:34.604 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.605 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2641 3.12 3:488a729f:::rbd_data.1152a8ae2019.0000000000000022:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:34.605 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:34.605 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 113.748074897, 2024-09-17T21:21:34.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098280953000000004, 2024-09-17T21:21:34.605 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:34.605 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:34.605 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:34.605 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:34.605 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:34.605 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:34.605 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2641 2024-09-17T21:21:34.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.606 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:34.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:34.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:34.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:34.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:34.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:34.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842083+0000", 2024-09-17T21:21:34.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5300000000000002e-07 2024-09-17T21:21:34.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:34.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842088+0000", 2024-09-17T21:21:34.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:21:34.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:34.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842105+0000", 2024-09-17T21:21:34.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6773e-05 2024-09-17T21:21:34.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:34.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842119+0000", 2024-09-17T21:21:34.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:21:34.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842320+0000", 2024-09-17T21:21:34.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020052700000000001 2024-09-17T21:21:34.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:34.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842845+0000", 2024-09-17T21:21:34.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052508600000000002 2024-09-17T21:21:34.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.609 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:34.609 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940363+0000", 2024-09-17T21:21:34.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097518795000000005 2024-09-17T21:21:34.609 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.609 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:34.609 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.609 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2674 3.12 3:4c87e3ac:::rbd_data.1152a8ae2019.0000000000000033:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:34.609 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:34.609 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 113.74771226, 2024-09-17T21:21:34.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098025125000000005, 2024-09-17T21:21:34.610 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:34.610 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:34.610 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:34.610 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:34.610 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:34.610 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:34.610 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2674 2024-09-17T21:21:34.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.610 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:34.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:34.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:34.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:34.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:34.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:34.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842447+0000", 2024-09-17T21:21:34.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3999999999999999e-06 2024-09-17T21:21:34.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:34.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842453+0000", 2024-09-17T21:21:34.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:21:34.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:34.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842472+0000", 2024-09-17T21:21:34.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9006000000000001e-05 2024-09-17T21:21:34.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:34.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842487+0000", 2024-09-17T21:21:34.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4866e-05 2024-09-17T21:21:34.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843594+0000", 2024-09-17T21:21:34.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011073249999999999 2024-09-17T21:21:34.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:34.613 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843668+0000", 2024-09-17T21:21:34.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3188000000000005e-05 2024-09-17T21:21:34.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:34.613 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843893+0000", 2024-09-17T21:21:34.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022503399999999999 2024-09-17T21:21:34.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:34.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850439+0000", 2024-09-17T21:21:34.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065467880000000004 2024-09-17T21:21:34.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:34.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940409+0000", 2024-09-17T21:21:34.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089969511000000002 2024-09-17T21:21:34.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:34.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940444+0000", 2024-09-17T21:21:34.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5345999999999997e-05 2024-09-17T21:21:34.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:34.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940470+0000", 2024-09-17T21:21:34.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5964000000000001e-05 2024-09-17T21:21:34.615 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.615 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:34.615 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.615 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2707 3.1f 3:f805e65e:::rbd_data.1152a8ae2019.0000000000000049:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:34.616 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:34.616 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 113.74266559900001, 2024-09-17T21:21:34.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088327300999999997, 2024-09-17T21:21:34.616 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:34.616 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:34.616 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:34.616 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:34.616 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:34.616 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:34.616 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2707 2024-09-17T21:21:34.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.616 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:34.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:34.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:34.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:34.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:34.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:34.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847494+0000", 2024-09-17T21:21:34.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:21:34.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:34.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847504+0000", 2024-09-17T21:21:34.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:21:34.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:34.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847526+0000", 2024-09-17T21:21:34.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1976000000000001e-05 2024-09-17T21:21:34.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:34.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847541+0000", 2024-09-17T21:21:34.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5277e-05 2024-09-17T21:21:34.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856142+0000", 2024-09-17T21:21:34.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086006710000000007 2024-09-17T21:21:34.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:34.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856435+0000", 2024-09-17T21:21:34.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029301499999999999 2024-09-17T21:21:34.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:34.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935819+0000", 2024-09-17T21:21:34.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079384448999999996 2024-09-17T21:21:34.620 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.620 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:34.620 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.620 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2730 3.12 3:48b3422a:::rbd_data.1152a8ae2019.0000000000000057:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:34.620 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:34.620 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 113.73347269999999, 2024-09-17T21:21:34.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083879975999999995, 2024-09-17T21:21:34.620 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:34.620 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:34.621 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:34.621 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:34.621 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:34.621 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:34.621 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2730 2024-09-17T21:21:34.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.621 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:34.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:34.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:34.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:34.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:34.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:34.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856687+0000", 2024-09-17T21:21:34.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:21:34.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:34.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856697+0000", 2024-09-17T21:21:34.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0207000000000001e-05 2024-09-17T21:21:34.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:34.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856808+0000", 2024-09-17T21:21:34.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011071200000000001 2024-09-17T21:21:34.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:34.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856828+0000", 2024-09-17T21:21:34.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0219000000000001e-05 2024-09-17T21:21:34.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856907+0000", 2024-09-17T21:21:34.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9661999999999997e-05 2024-09-17T21:21:34.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:34.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.857019+0000", 2024-09-17T21:21:34.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011148499999999999 2024-09-17T21:21:34.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:34.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940565+0000", 2024-09-17T21:21:34.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083545791999999994 2024-09-17T21:21:34.624 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.624 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:34.625 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.625 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2743 3.12 3:4d036e62:::rbd_data.1152a8ae2019.000000000000005f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:34.625 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:34.625 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 113.729527199, 2024-09-17T21:21:34.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080059962999999998, 2024-09-17T21:21:34.625 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:34.625 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:34.625 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:34.625 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:34.625 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:34.625 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:34.625 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2743 2024-09-17T21:21:34.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.626 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:34.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:34.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:34.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:34.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:34.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:34.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860631+0000", 2024-09-17T21:21:34.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:21:34.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:34.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860636+0000", 2024-09-17T21:21:34.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5279999999999997e-06 2024-09-17T21:21:34.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:34.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860649+0000", 2024-09-17T21:21:34.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-17T21:21:34.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:34.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860655+0000", 2024-09-17T21:21:34.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:21:34.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860731+0000", 2024-09-17T21:21:34.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5920000000000005e-05 2024-09-17T21:21:34.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:34.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.861070+0000", 2024-09-17T21:21:34.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338535 2024-09-17T21:21:34.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:34.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940690+0000", 2024-09-17T21:21:34.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079620503999999995 2024-09-17T21:21:34.629 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.629 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:34.629 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.629 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3378 3.12 3:4832a51b:::rbd_data.1152a8ae2019.000000000000006c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:34.629 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:34.630 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 113.701163349, 2024-09-17T21:21:34.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051812516000000003, 2024-09-17T21:21:34.630 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:34.630 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:34.630 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:34.630 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:34.630 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:34.630 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:34.630 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3378 2024-09-17T21:21:34.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.630 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:34.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:34.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:34.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:34.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:34.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:34.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888995+0000", 2024-09-17T21:21:34.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:21:34.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:34.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889006+0000", 2024-09-17T21:21:34.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1012999999999999e-05 2024-09-17T21:21:34.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:34.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889032+0000", 2024-09-17T21:21:34.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6441e-05 2024-09-17T21:21:34.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:34.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889040+0000", 2024-09-17T21:21:34.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0639999999999994e-06 2024-09-17T21:21:34.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889278+0000", 2024-09-17T21:21:34.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023785899999999999 2024-09-17T21:21:34.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:34.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889408+0000", 2024-09-17T21:21:34.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129219 2024-09-17T21:21:34.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:34.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889723+0000", 2024-09-17T21:21:34.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000315264 2024-09-17T21:21:34.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:34.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.891972+0000", 2024-09-17T21:21:34.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002249633 2024-09-17T21:21:34.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:34.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940733+0000", 2024-09-17T21:21:34.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048760626000000001 2024-09-17T21:21:34.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:34.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940785+0000", 2024-09-17T21:21:34.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1906000000000001e-05 2024-09-17T21:21:34.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:34.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940807+0000", 2024-09-17T21:21:34.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1554999999999999e-05 2024-09-17T21:21:34.635 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.635 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:34.635 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.636 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3379 3.12 3:4e83fd71:::rbd_data.1152a8ae2019.000000000000006d:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:34.636 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:34.636 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 113.701079142, 2024-09-17T21:21:34.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051804144000000003, 2024-09-17T21:21:34.636 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:34.636 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:34.636 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:34.636 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:34.636 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:34.636 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:34.636 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3379 2024-09-17T21:21:34.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.636 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:34.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:34.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:34.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:34.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:34.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:34.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889079+0000", 2024-09-17T21:21:34.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1800000000000004e-07 2024-09-17T21:21:34.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:34.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889083+0000", 2024-09-17T21:21:34.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:21:34.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:34.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889094+0000", 2024-09-17T21:21:34.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1562999999999999e-05 2024-09-17T21:21:34.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:34.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889102+0000", 2024-09-17T21:21:34.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.401e-06 2024-09-17T21:21:34.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:21:34.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014279259999999999 2024-09-17T21:21:34.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:34.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890609+0000", 2024-09-17T21:21:34.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9566000000000004e-05 2024-09-17T21:21:34.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:34.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890789+0000", 2024-09-17T21:21:34.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017949700000000001 2024-09-17T21:21:34.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:34.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.893929+0000", 2024-09-17T21:21:34.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003140538 2024-09-17T21:21:34.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:34.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940834+0000", 2024-09-17T21:21:34.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046904811999999997 2024-09-17T21:21:34.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:34.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940860+0000", 2024-09-17T21:21:34.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5341999999999999e-05 2024-09-17T21:21:34.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:34.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940882+0000", 2024-09-17T21:21:34.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2878999999999999e-05 2024-09-17T21:21:34.642 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.642 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:34.642 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.642 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:2644 3.1e 3:79615585:::rbd_data.115266d0b9f2.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2475520~14848,write 3106304~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:21:34.642 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:34.642 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 105.078450739, 2024-09-17T21:21:34.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050733484000000002, 2024-09-17T21:21:34.642 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:34.642 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:34.642 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:34.643 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:34.643 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:21:34.643 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:21:34.643 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2644 2024-09-17T21:21:34.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.643 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:34.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:34.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:34.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:34.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:34.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:34.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511708+0000", 2024-09-17T21:21:34.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-17T21:21:34.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:34.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522196+0000", 2024-09-17T21:21:34.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010487712999999999 2024-09-17T21:21:34.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:34.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522223+0000", 2024-09-17T21:21:34.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7064999999999999e-05 2024-09-17T21:21:34.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:34.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522236+0000", 2024-09-17T21:21:34.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2931e-05 2024-09-17T21:21:34.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522348+0000", 2024-09-17T21:21:34.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111906 2024-09-17T21:21:34.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:34.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522554+0000", 2024-09-17T21:21:34.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020632699999999999 2024-09-17T21:21:34.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:21:34.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.523185+0000", 2024-09-17T21:21:34.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063081400000000005 2024-09-17T21:21:34.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:34.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.547011+0000", 2024-09-17T21:21:34.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023825570000000001 2024-09-17T21:21:34.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:34.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562268+0000", 2024-09-17T21:21:34.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015257514 2024-09-17T21:21:34.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:34.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562367+0000", 2024-09-17T21:21:34.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8678000000000006e-05 2024-09-17T21:21:34.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:34.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562440+0000", 2024-09-17T21:21:34.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3337000000000006e-05 2024-09-17T21:21:34.648 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.648 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:34.648 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.648 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:231 3.1 3:856fed78:::rbd_data.115220dd393e.000000000000000b:head [read 0~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:34.648 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:34.648 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 76.635695470000002, 2024-09-17T21:21:34.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34254251699999999, 2024-09-17T21:21:34.648 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:34.648 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:34.649 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:34.649 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:34.649 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:34.649 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:34.649 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 231 2024-09-17T21:21:34.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.649 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:34.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.649 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:34.649 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:34.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:34.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:34.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:34.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954464+0000", 2024-09-17T21:21:34.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:21:34.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:34.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954475+0000", 2024-09-17T21:21:34.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1146e-05 2024-09-17T21:21:34.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:34.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954506+0000", 2024-09-17T21:21:34.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0292999999999999e-05 2024-09-17T21:21:34.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:34.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954524+0000", 2024-09-17T21:21:34.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8430000000000001e-05 2024-09-17T21:21:34.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954661+0000", 2024-09-17T21:21:34.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136403 2024-09-17T21:21:34.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:34.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954680+0000", 2024-09-17T21:21:34.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9743000000000001e-05 2024-09-17T21:21:34.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.296896+0000", 2024-09-17T21:21:34.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34221550499999998 2024-09-17T21:21:34.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:34.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297004+0000", 2024-09-17T21:21:34.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010865399999999999 2024-09-17T21:21:34.653 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.653 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:34.653 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.653 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:56 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:34.653 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:34.653 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 76.480061773000003, 2024-09-17T21:21:34.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18718773899999999, 2024-09-17T21:21:34.653 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:34.653 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:34.653 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:34.654 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:34.654 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:21:34.654 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:21:34.654 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 56 2024-09-17T21:21:34.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.654 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:34.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:34.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:34.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:34.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:34.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:34.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110098+0000", 2024-09-17T21:21:34.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:21:34.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:34.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110111+0000", 2024-09-17T21:21:34.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2289999999999999e-05 2024-09-17T21:21:34.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:34.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110143+0000", 2024-09-17T21:21:34.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1979e-05 2024-09-17T21:21:34.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:34.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110161+0000", 2024-09-17T21:21:34.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7839e-05 2024-09-17T21:21:34.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110319+0000", 2024-09-17T21:21:34.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015835400000000001 2024-09-17T21:21:34.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:34.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110353+0000", 2024-09-17T21:21:34.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3849000000000001e-05 2024-09-17T21:21:34.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297041+0000", 2024-09-17T21:21:34.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18668842299999999 2024-09-17T21:21:34.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:34.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297169+0000", 2024-09-17T21:21:34.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012786199999999999 2024-09-17T21:21:34.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:34.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297283+0000", 2024-09-17T21:21:34.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011430300000000001 2024-09-17T21:21:34.658 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.658 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:34.658 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.658 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:68 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:34.658 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:34.659 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 76.471520842000004, 2024-09-17T21:21:34.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17885582999999999, 2024-09-17T21:21:34.659 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:34.659 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:34.659 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:34.659 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:34.659 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:21:34.659 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:21:34.659 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 68 2024-09-17T21:21:34.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.659 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:34.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:34.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:34.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:34.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:34.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:34.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118639+0000", 2024-09-17T21:21:34.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.306e-06 2024-09-17T21:21:34.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:34.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118651+0000", 2024-09-17T21:21:34.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2567e-05 2024-09-17T21:21:34.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:34.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118682+0000", 2024-09-17T21:21:34.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0502e-05 2024-09-17T21:21:34.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:34.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118707+0000", 2024-09-17T21:21:34.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5463e-05 2024-09-17T21:21:34.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118889+0000", 2024-09-17T21:21:34.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018150999999999999 2024-09-17T21:21:34.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:34.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118910+0000", 2024-09-17T21:21:34.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:21:34.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297313+0000", 2024-09-17T21:21:34.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17840366699999999 2024-09-17T21:21:34.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:34.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297389+0000", 2024-09-17T21:21:34.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-17T21:21:34.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:34.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297492+0000", 2024-09-17T21:21:34.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103019 2024-09-17T21:21:34.663 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.664 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:34.664 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.664 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5890.0:15234 3.1 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8451072~29696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-17T21:21:34.664 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:34.664 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.496872029999999, 2024-09-17T21:21:34.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.057026052000000001, 2024-09-17T21:21:34.664 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:34.664 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:34.664 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:34.664 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:34.664 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5890", 2024-09-17T21:21:34.665 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3064344947", 2024-09-17T21:21:34.665 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 15234 2024-09-17T21:21:34.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.665 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:34.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:34.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:34.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:34.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:34.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:34.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093289+0000", 2024-09-17T21:21:34.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4939999999999999e-06 2024-09-17T21:21:34.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:34.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103074+0000", 2024-09-17T21:21:34.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00978564 2024-09-17T21:21:34.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:34.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103112+0000", 2024-09-17T21:21:34.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7246999999999998e-05 2024-09-17T21:21:34.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:34.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103128+0000", 2024-09-17T21:21:34.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5812999999999999e-05 2024-09-17T21:21:34.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103207+0000", 2024-09-17T21:21:34.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9785999999999998e-05 2024-09-17T21:21:34.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:34.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103315+0000", 2024-09-17T21:21:34.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107396 2024-09-17T21:21:34.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:34.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103957+0000", 2024-09-17T21:21:34.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000642524 2024-09-17T21:21:34.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:34.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.123856+0000", 2024-09-17T21:21:34.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019898376999999998 2024-09-17T21:21:34.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:34.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150190+0000", 2024-09-17T21:21:34.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026334682000000002 2024-09-17T21:21:34.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:34.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150255+0000", 2024-09-17T21:21:34.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4770999999999994e-05 2024-09-17T21:21:34.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:34.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150311+0000", 2024-09-17T21:21:34.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6322e-05 2024-09-17T21:21:34.670 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.670 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:34.670 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.670 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4237 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 411648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:34.670 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753637+0000", 2024-09-17T21:21:34.670 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 21.836520424, 2024-09-17T21:21:34.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049394367000000002, 2024-09-17T21:21:34.670 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:34.671 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:34.671 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:34.671 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:34.671 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:34.671 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:34.671 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4237 2024-09-17T21:21:34.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.671 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:34.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:34.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753637+0000", 2024-09-17T21:21:34.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.672 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:34.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753637+0000", 2024-09-17T21:21:34.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.672 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:34.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753638+0000", 2024-09-17T21:21:34.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5199999999999997e-07 2024-09-17T21:21:34.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.672 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:34.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753640+0000", 2024-09-17T21:21:34.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2819999999999999e-06 2024-09-17T21:21:34.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:34.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753647+0000", 2024-09-17T21:21:34.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8689999999999998e-06 2024-09-17T21:21:34.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:34.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753653+0000", 2024-09-17T21:21:34.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8799999999999996e-06 2024-09-17T21:21:34.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.802605+0000", 2024-09-17T21:21:34.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048952298999999998 2024-09-17T21:21:34.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:34.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.802795+0000", 2024-09-17T21:21:34.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000190173 2024-09-17T21:21:34.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:34.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803031+0000", 2024-09-17T21:21:34.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000236312 2024-09-17T21:21:34.674 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.674 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:34.674 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.675 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4238 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 412672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:34.675 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753660+0000", 2024-09-17T21:21:34.675 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 21.836497216000001, 2024-09-17T21:21:34.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049651503999999999, 2024-09-17T21:21:34.675 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:34.675 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:34.675 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:34.675 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:34.675 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:34.675 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:34.675 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4238 2024-09-17T21:21:34.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.675 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:34.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:34.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753660+0000", 2024-09-17T21:21:34.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:34.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753660+0000", 2024-09-17T21:21:34.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:34.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753661+0000", 2024-09-17T21:21:34.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6000000000000004e-07 2024-09-17T21:21:34.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:34.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753663+0000", 2024-09-17T21:21:34.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3489999999999999e-06 2024-09-17T21:21:34.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:34.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753670+0000", 2024-09-17T21:21:34.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9539999999999998e-06 2024-09-17T21:21:34.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:34.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753676+0000", 2024-09-17T21:21:34.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6169999999999998e-06 2024-09-17T21:21:34.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803132+0000", 2024-09-17T21:21:34.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049456583999999998 2024-09-17T21:21:34.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:34.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803224+0000", 2024-09-17T21:21:34.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2139000000000007e-05 2024-09-17T21:21:34.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:34.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803312+0000", 2024-09-17T21:21:34.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7300999999999996e-05 2024-09-17T21:21:34.679 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.679 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:34.679 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.679 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4239 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 424960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:34.679 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:21:34.679 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 21.836474373000001, 2024-09-17T21:21:34.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050123167000000003, 2024-09-17T21:21:34.680 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:34.680 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:34.680 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:34.680 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:34.680 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:34.680 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:34.680 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4239 2024-09-17T21:21:34.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.680 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:34.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:34.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:21:34.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:34.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:21:34.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:34.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753684+0000", 2024-09-17T21:21:34.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.37e-07 2024-09-17T21:21:34.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:34.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753686+0000", 2024-09-17T21:21:34.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.322e-06 2024-09-17T21:21:34.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:34.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753693+0000", 2024-09-17T21:21:34.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0890000000000002e-06 2024-09-17T21:21:34.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:34.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753699+0000", 2024-09-17T21:21:34.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:21:34.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803390+0000", 2024-09-17T21:21:34.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049690506000000002 2024-09-17T21:21:34.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:34.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803569+0000", 2024-09-17T21:21:34.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179784 2024-09-17T21:21:34.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:34.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803806+0000", 2024-09-17T21:21:34.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023683499999999999 2024-09-17T21:21:34.684 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.684 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:34.684 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.684 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4240 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 413696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:34.684 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:34.684 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 21.836450644999999, 2024-09-17T21:21:34.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050331353000000002, 2024-09-17T21:21:34.684 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:34.684 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:34.684 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:34.685 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:34.685 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:34.685 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:34.685 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4240 2024-09-17T21:21:34.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.685 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:34.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:34.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:34.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:34.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:34.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:34.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:34.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5700000000000002e-07 2024-09-17T21:21:34.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:34.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753710+0000", 2024-09-17T21:21:34.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.385e-06 2024-09-17T21:21:34.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:34.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753717+0000", 2024-09-17T21:21:34.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.832e-06 2024-09-17T21:21:34.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:34.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753722+0000", 2024-09-17T21:21:34.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8050000000000003e-06 2024-09-17T21:21:34.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803880+0000", 2024-09-17T21:21:34.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050157641000000003 2024-09-17T21:21:34.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:34.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803945+0000", 2024-09-17T21:21:34.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5357000000000005e-05 2024-09-17T21:21:34.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:34.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804038+0000", 2024-09-17T21:21:34.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2775999999999994e-05 2024-09-17T21:21:34.688 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.688 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:34.688 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.689 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4241 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 415744~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:34.689 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:34.689 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 21.836427529000002, 2024-09-17T21:21:34.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050578790999999998, 2024-09-17T21:21:34.689 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:34.689 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:34.689 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:34.689 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:34.689 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:34.689 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:34.689 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4241 2024-09-17T21:21:34.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.690 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:34.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:34.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:34.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:34.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:34.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:34.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:34.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5700000000000002e-07 2024-09-17T21:21:34.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:34.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753733+0000", 2024-09-17T21:21:34.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2570000000000002e-06 2024-09-17T21:21:34.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:34.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753740+0000", 2024-09-17T21:21:34.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.849e-06 2024-09-17T21:21:34.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:34.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753745+0000", 2024-09-17T21:21:34.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7450000000000001e-06 2024-09-17T21:21:34.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804115+0000", 2024-09-17T21:21:34.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050370038999999998 2024-09-17T21:21:34.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:34.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804223+0000", 2024-09-17T21:21:34.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010786 2024-09-17T21:21:34.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:34.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804309+0000", 2024-09-17T21:21:34.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5483999999999997e-05 2024-09-17T21:21:34.693 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.693 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:34.693 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.693 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4242 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 414720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:34.693 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:34.694 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 21.836404641000001, 2024-09-17T21:21:34.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050748148999999999, 2024-09-17T21:21:34.694 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:34.694 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:34.694 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:34.694 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:34.694 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:34.694 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:34.694 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4242 2024-09-17T21:21:34.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.694 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:34.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:34.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:34.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:34.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:34.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:34.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:34.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:34.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6599999999999996e-07 2024-09-17T21:21:34.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:34.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753756+0000", 2024-09-17T21:21:34.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.368e-06 2024-09-17T21:21:34.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:34.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753763+0000", 2024-09-17T21:21:34.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9349999999999997e-06 2024-09-17T21:21:34.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:34.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753768+0000", 2024-09-17T21:21:34.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7620000000000001e-06 2024-09-17T21:21:34.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:34.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804343+0000", 2024-09-17T21:21:34.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050575050000000003 2024-09-17T21:21:34.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:34.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804397+0000", 2024-09-17T21:21:34.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4070000000000002e-05 2024-09-17T21:21:34.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:34.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:34.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:34.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804501+0000", 2024-09-17T21:21:34.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103398 2024-09-17T21:21:34.698 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.698 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:34.698 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.698 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:34.698 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:34.698 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:34.700 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:21:34.700 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:34.695+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:34.800 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:34.797+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:34.867 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:34.867 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:34.867 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:21:34.867 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:34.882 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:21:34.929 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:34.927+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:34.999 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:35.000 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:35.000 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:21:35.000 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:21:35.000 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:35.002 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:34.997+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:35.013 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:21:35.103 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:35.099+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:35.184 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:35.185 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:35.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:35.185 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:35.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.185 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:21:35.185 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:35.185 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 175.371053426, 2024-09-17T21:21:35.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:21:35.185 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.185 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.185 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:35.185 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:35.185 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:21:35.185 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:21:35.186 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:21:35.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.186 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:35.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:35.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:21:35.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:21:35.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:21:35.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:21:35.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:21:35.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:21:35.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:21:35.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:21:35.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:21:35.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:21:35.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:35.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:21:35.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:21:35.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:21:35.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:21:35.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:21:35.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:21:35.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:21:35.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:21:35.189 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.189 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.189 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.190 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:21:35.190 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:35.190 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 161.81595164000001, 2024-09-17T21:21:35.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:21:35.190 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.190 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.190 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:35.190 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:35.190 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:21:35.190 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:35.190 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:21:35.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.191 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:35.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:35.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:21:35.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:21:35.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:21:35.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:21:35.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:21:35.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:21:35.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:21:35.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:21:35.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:21:35.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:21:35.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:35.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:21:35.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:21:35.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:21:35.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:21:35.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:21:35.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:21:35.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:35.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:21:35.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:21:35.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:35.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:21:35.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:21:35.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:35.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:21:35.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:21:35.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:35.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:21:35.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:21:35.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:21:35.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:21:35.196 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.196 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.196 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.196 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:793 3.9 3:95f63599:::rbd_data.1152eb799094.0000000000000007:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:35.196 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:35.196 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 78.748895485000006, 2024-09-17T21:21:35.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88245766400000003, 2024-09-17T21:21:35.196 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.196 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.196 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:35.196 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:35.197 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:35.197 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:35.197 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 793 2024-09-17T21:21:35.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.197 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:35.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:35.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431941+0000", 2024-09-17T21:21:35.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.864e-06 2024-09-17T21:21:35.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431953+0000", 2024-09-17T21:21:35.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1486e-05 2024-09-17T21:21:35.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431984+0000", 2024-09-17T21:21:35.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1075999999999999e-05 2024-09-17T21:21:35.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431999+0000", 2024-09-17T21:21:35.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.503e-05 2024-09-17T21:21:35.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432095+0000", 2024-09-17T21:21:35.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5698000000000004e-05 2024-09-17T21:21:35.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:35.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432109+0000", 2024-09-17T21:21:35.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4188e-05 2024-09-17T21:21:35.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.309642+0000", 2024-09-17T21:21:35.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87753353000000001 2024-09-17T21:21:35.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310104+0000", 2024-09-17T21:21:35.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046103600000000001 2024-09-17T21:21:35.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:35.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310630+0000", 2024-09-17T21:21:35.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052682800000000002 2024-09-17T21:21:35.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:35.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313100+0000", 2024-09-17T21:21:35.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024693369999999998 2024-09-17T21:21:35.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:35.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313935+0000", 2024-09-17T21:21:35.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083540100000000005 2024-09-17T21:21:35.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:35.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314086+0000", 2024-09-17T21:21:35.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015053399999999999 2024-09-17T21:21:35.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314396+0000", 2024-09-17T21:21:35.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031065599999999997 2024-09-17T21:21:35.202 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.202 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.202 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.202 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:800 3.19 3:9dc17f12:::rbd_data.1152eb799094.000000000000000e:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:35.202 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:35.202 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 78.748648239999994, 2024-09-17T21:21:35.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88154663799999999, 2024-09-17T21:21:35.202 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.203 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.203 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:35.203 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:35.203 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:35.203 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:35.203 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 800 2024-09-17T21:21:35.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.203 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:35.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:35.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432187+0000", 2024-09-17T21:21:35.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4100000000000001e-06 2024-09-17T21:21:35.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432194+0000", 2024-09-17T21:21:35.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0389999999999999e-06 2024-09-17T21:21:35.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432216+0000", 2024-09-17T21:21:35.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1614e-05 2024-09-17T21:21:35.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432232+0000", 2024-09-17T21:21:35.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5739e-05 2024-09-17T21:21:35.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432405+0000", 2024-09-17T21:21:35.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017355300000000001 2024-09-17T21:21:35.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:35.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432432+0000", 2024-09-17T21:21:35.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6979e-05 2024-09-17T21:21:35.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.307767+0000", 2024-09-17T21:21:35.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87533472899999998 2024-09-17T21:21:35.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308192+0000", 2024-09-17T21:21:35.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042485999999999999 2024-09-17T21:21:35.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:35.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308727+0000", 2024-09-17T21:21:35.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00053550400000000001 2024-09-17T21:21:35.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:35.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310347+0000", 2024-09-17T21:21:35.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016198670000000001 2024-09-17T21:21:35.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:35.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313424+0000", 2024-09-17T21:21:35.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030764640000000001 2024-09-17T21:21:35.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:35.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313510+0000", 2024-09-17T21:21:35.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5938000000000003e-05 2024-09-17T21:21:35.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313733+0000", 2024-09-17T21:21:35.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022294200000000001 2024-09-17T21:21:35.208 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.208 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.208 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.208 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:805 3.19 3:9cab2ed6:::rbd_data.1152eb799094.0000000000000013:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:35.208 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:35.208 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 78.748444221, 2024-09-17T21:21:35.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88274355699999996, 2024-09-17T21:21:35.209 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.209 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.209 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:35.209 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:35.209 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:35.209 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:35.209 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 805 2024-09-17T21:21:35.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.209 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:35.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:35.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432391+0000", 2024-09-17T21:21:35.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3e-06 2024-09-17T21:21:35.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432398+0000", 2024-09-17T21:21:35.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4489999999999996e-06 2024-09-17T21:21:35.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432415+0000", 2024-09-17T21:21:35.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7750000000000001e-05 2024-09-17T21:21:35.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432430+0000", 2024-09-17T21:21:35.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.449e-05 2024-09-17T21:21:35.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432635+0000", 2024-09-17T21:21:35.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205031 2024-09-17T21:21:35.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:35.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432657+0000", 2024-09-17T21:21:35.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2064000000000001e-05 2024-09-17T21:21:35.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310444+0000", 2024-09-17T21:21:35.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87778699199999999 2024-09-17T21:21:35.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310849+0000", 2024-09-17T21:21:35.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00040497800000000001 2024-09-17T21:21:35.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:35.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.311396+0000", 2024-09-17T21:21:35.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00054729099999999999 2024-09-17T21:21:35.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:35.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313847+0000", 2024-09-17T21:21:35.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024504940000000001 2024-09-17T21:21:35.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:35.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314879+0000", 2024-09-17T21:21:35.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001031817 2024-09-17T21:21:35.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:35.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314964+0000", 2024-09-17T21:21:35.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5685000000000004e-05 2024-09-17T21:21:35.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.315133+0000", 2024-09-17T21:21:35.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000169216 2024-09-17T21:21:35.214 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.214 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.214 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.215 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12089 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 60416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:35.215 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:35.215 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.411219188999993, 2024-09-17T21:21:35.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088293233999999998, 2024-09-17T21:21:35.215 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.215 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.215 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:35.215 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:35.215 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:35.215 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:35.215 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12089 2024-09-17T21:21:35.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.216 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:35.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:35.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769616+0000", 2024-09-17T21:21:35.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:21:35.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769620+0000", 2024-09-17T21:21:35.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5140000000000002e-06 2024-09-17T21:21:35.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769633+0000", 2024-09-17T21:21:35.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2849e-05 2024-09-17T21:21:35.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769644+0000", 2024-09-17T21:21:35.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1471000000000001e-05 2024-09-17T21:21:35.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857564+0000", 2024-09-17T21:21:35.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087919445999999998 2024-09-17T21:21:35.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857716+0000", 2024-09-17T21:21:35.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000152065 2024-09-17T21:21:35.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857908+0000", 2024-09-17T21:21:35.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019252599999999999 2024-09-17T21:21:35.219 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.219 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.219 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.219 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12090 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 64512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:35.219 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:35.219 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.411172804000003, 2024-09-17T21:21:35.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088582672000000001, 2024-09-17T21:21:35.219 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.219 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.219 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:35.219 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:35.220 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:35.220 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:35.220 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12090 2024-09-17T21:21:35.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.220 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:35.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:35.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769662+0000", 2024-09-17T21:21:35.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:21:35.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769683+0000", 2024-09-17T21:21:35.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0979e-05 2024-09-17T21:21:35.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769699+0000", 2024-09-17T21:21:35.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5446000000000001e-05 2024-09-17T21:21:35.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769708+0000", 2024-09-17T21:21:35.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1679999999999993e-06 2024-09-17T21:21:35.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857976+0000", 2024-09-17T21:21:35.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088267795999999996 2024-09-17T21:21:35.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858095+0000", 2024-09-17T21:21:35.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119147 2024-09-17T21:21:35.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858244+0000", 2024-09-17T21:21:35.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014917000000000001 2024-09-17T21:21:35.223 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.223 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.223 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.223 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12091 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 68608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:35.223 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:35.223 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.411101419999994, 2024-09-17T21:21:35.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088832752000000001, 2024-09-17T21:21:35.224 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.224 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.224 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:35.224 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:35.224 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:35.224 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:35.224 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12091 2024-09-17T21:21:35.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.224 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:35.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:35.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769734+0000", 2024-09-17T21:21:35.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:21:35.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769739+0000", 2024-09-17T21:21:35.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8080000000000004e-06 2024-09-17T21:21:35.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769764+0000", 2024-09-17T21:21:35.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4839999999999999e-05 2024-09-17T21:21:35.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769777+0000", 2024-09-17T21:21:35.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3037000000000001e-05 2024-09-17T21:21:35.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858295+0000", 2024-09-17T21:21:35.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088518475999999999 2024-09-17T21:21:35.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858419+0000", 2024-09-17T21:21:35.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012378700000000001 2024-09-17T21:21:35.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858565+0000", 2024-09-17T21:21:35.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014630099999999999 2024-09-17T21:21:35.227 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.227 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.227 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.228 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12092 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:35.228 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:35.228 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.411019096000004, 2024-09-17T21:21:35.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089191555000000006, 2024-09-17T21:21:35.228 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.228 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.228 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:35.228 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:35.228 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:35.228 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:35.228 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12092 2024-09-17T21:21:35.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.228 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:35.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:35.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769816+0000", 2024-09-17T21:21:35.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7199999999999997e-07 2024-09-17T21:21:35.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769822+0000", 2024-09-17T21:21:35.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:21:35.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769838+0000", 2024-09-17T21:21:35.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6674e-05 2024-09-17T21:21:35.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769851+0000", 2024-09-17T21:21:35.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2527999999999999e-05 2024-09-17T21:21:35.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858660+0000", 2024-09-17T21:21:35.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088808556999999996 2024-09-17T21:21:35.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858817+0000", 2024-09-17T21:21:35.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015730400000000001 2024-09-17T21:21:35.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859007+0000", 2024-09-17T21:21:35.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018974899999999999 2024-09-17T21:21:35.231 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.231 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.231 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.232 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12093 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:35.232 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:35.232 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.410910991999998, 2024-09-17T21:21:35.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089429710999999995, 2024-09-17T21:21:35.232 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.232 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.232 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:35.232 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:35.232 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:35.232 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:35.232 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12093 2024-09-17T21:21:35.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.233 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:35.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:35.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769924+0000", 2024-09-17T21:21:35.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.342e-06 2024-09-17T21:21:35.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769929+0000", 2024-09-17T21:21:35.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0350000000000001e-06 2024-09-17T21:21:35.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769946+0000", 2024-09-17T21:21:35.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6793999999999999e-05 2024-09-17T21:21:35.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769959+0000", 2024-09-17T21:21:35.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3062e-05 2024-09-17T21:21:35.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859075+0000", 2024-09-17T21:21:35.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089115696999999994 2024-09-17T21:21:35.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859202+0000", 2024-09-17T21:21:35.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012657 2024-09-17T21:21:35.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859353+0000", 2024-09-17T21:21:35.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151211 2024-09-17T21:21:35.236 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.236 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.236 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.236 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12094 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:35.236 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:35.236 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.410856816000006, 2024-09-17T21:21:35.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089695001999999996, 2024-09-17T21:21:35.236 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.236 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.236 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:35.237 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:35.237 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:35.237 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:35.237 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12094 2024-09-17T21:21:35.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.237 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:35.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:35.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769978+0000", 2024-09-17T21:21:35.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.142e-06 2024-09-17T21:21:35.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769983+0000", 2024-09-17T21:21:35.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.386e-06 2024-09-17T21:21:35.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769997+0000", 2024-09-17T21:21:35.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.416e-05 2024-09-17T21:21:35.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770012+0000", 2024-09-17T21:21:35.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5384999999999999e-05 2024-09-17T21:21:35.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859405+0000", 2024-09-17T21:21:35.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089392376999999995 2024-09-17T21:21:35.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859525+0000", 2024-09-17T21:21:35.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120555 2024-09-17T21:21:35.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859672+0000", 2024-09-17T21:21:35.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000146997 2024-09-17T21:21:35.240 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.240 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.240 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.240 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12095 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:35.240 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:35.240 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.410804329000001, 2024-09-17T21:21:35.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090073476, 2024-09-17T21:21:35.240 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.240 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.240 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:35.241 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:35.241 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:35.241 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:35.241 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12095 2024-09-17T21:21:35.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.241 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:35.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:35.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770031+0000", 2024-09-17T21:21:35.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1699999999999997e-07 2024-09-17T21:21:35.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770036+0000", 2024-09-17T21:21:35.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.023e-06 2024-09-17T21:21:35.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770051+0000", 2024-09-17T21:21:35.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4953999999999999e-05 2024-09-17T21:21:35.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770063+0000", 2024-09-17T21:21:35.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:21:35.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859756+0000", 2024-09-17T21:21:35.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089692980000000005 2024-09-17T21:21:35.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859891+0000", 2024-09-17T21:21:35.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013459499999999999 2024-09-17T21:21:35.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.244 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.244 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860103+0000", 2024-09-17T21:21:35.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021274400000000001 2024-09-17T21:21:35.244 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.244 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.244 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.244 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12096 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:35.244 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:35.244 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.410754643999994, 2024-09-17T21:21:35.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090358274000000002, 2024-09-17T21:21:35.244 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.245 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.245 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:35.245 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:35.245 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:35.245 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:35.245 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12096 2024-09-17T21:21:35.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.245 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:35.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:35.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770080+0000", 2024-09-17T21:21:35.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9400000000000004e-07 2024-09-17T21:21:35.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770085+0000", 2024-09-17T21:21:35.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2169999999999996e-06 2024-09-17T21:21:35.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770100+0000", 2024-09-17T21:21:35.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5077e-05 2024-09-17T21:21:35.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770112+0000", 2024-09-17T21:21:35.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1951e-05 2024-09-17T21:21:35.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860160+0000", 2024-09-17T21:21:35.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090048413999999993 2024-09-17T21:21:35.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860285+0000", 2024-09-17T21:21:35.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012466699999999999 2024-09-17T21:21:35.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860438+0000", 2024-09-17T21:21:35.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015305399999999999 2024-09-17T21:21:35.248 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.248 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.248 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.248 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12097 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:35.248 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:35.248 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.410706896999997, 2024-09-17T21:21:35.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090702375000000002, 2024-09-17T21:21:35.248 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.248 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.249 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:35.249 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:35.249 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:35.249 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:35.249 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12097 2024-09-17T21:21:35.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.249 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:35.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:35.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770128+0000", 2024-09-17T21:21:35.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.04e-06 2024-09-17T21:21:35.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770133+0000", 2024-09-17T21:21:35.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8999999999999997e-06 2024-09-17T21:21:35.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770152+0000", 2024-09-17T21:21:35.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8890999999999999e-05 2024-09-17T21:21:35.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770164+0000", 2024-09-17T21:21:35.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1769e-05 2024-09-17T21:21:35.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860511+0000", 2024-09-17T21:21:35.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090347527999999996 2024-09-17T21:21:35.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860652+0000", 2024-09-17T21:21:35.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140894 2024-09-17T21:21:35.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860830+0000", 2024-09-17T21:21:35.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017735299999999999 2024-09-17T21:21:35.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.252 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.253 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12098 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:35.253 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:35.253 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.410653429000007, 2024-09-17T21:21:35.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090997262999999995, 2024-09-17T21:21:35.253 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.253 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.253 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:35.253 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:35.253 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:35.253 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:35.253 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12098 2024-09-17T21:21:35.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.253 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:35.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:35.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775465+0000", 2024-09-17T21:21:35.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0052843350000000002 2024-09-17T21:21:35.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775476+0000", 2024-09-17T21:21:35.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0797e-05 2024-09-17T21:21:35.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775506+0000", 2024-09-17T21:21:35.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9709999999999998e-05 2024-09-17T21:21:35.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775530+0000", 2024-09-17T21:21:35.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4528999999999998e-05 2024-09-17T21:21:35.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860894+0000", 2024-09-17T21:21:35.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.085364195000000004 2024-09-17T21:21:35.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861019+0000", 2024-09-17T21:21:35.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012467399999999999 2024-09-17T21:21:35.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861178+0000", 2024-09-17T21:21:35.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159023 2024-09-17T21:21:35.256 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.257 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.257 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.257 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12104 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:35.257 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:35.257 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.404952320000007, 2024-09-17T21:21:35.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087969628999999994, 2024-09-17T21:21:35.257 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.257 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.257 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:35.257 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:35.257 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:35.257 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:35.257 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12104 2024-09-17T21:21:35.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.258 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:35.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:35.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775883+0000", 2024-09-17T21:21:35.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3399999999999998e-07 2024-09-17T21:21:35.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775886+0000", 2024-09-17T21:21:35.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2119999999999999e-06 2024-09-17T21:21:35.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775895+0000", 2024-09-17T21:21:35.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8710000000000003e-06 2024-09-17T21:21:35.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775901+0000", 2024-09-17T21:21:35.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:21:35.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863544+0000", 2024-09-17T21:21:35.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087642642000000007 2024-09-17T21:21:35.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863694+0000", 2024-09-17T21:21:35.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014952300000000001 2024-09-17T21:21:35.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863851+0000", 2024-09-17T21:21:35.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000157804 2024-09-17T21:21:35.260 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.261 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.261 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.261 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12105 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:35.261 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:35.261 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.404923425999996, 2024-09-17T21:21:35.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088330400000000003, 2024-09-17T21:21:35.261 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.261 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.261 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:35.261 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:35.261 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:35.262 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:35.262 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12105 2024-09-17T21:21:35.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.262 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:35.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:35.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:35.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9999999999999997e-07 2024-09-17T21:21:35.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775914+0000", 2024-09-17T21:21:35.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4710000000000001e-06 2024-09-17T21:21:35.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775921+0000", 2024-09-17T21:21:35.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:21:35.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775928+0000", 2024-09-17T21:21:35.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:21:35.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863923+0000", 2024-09-17T21:21:35.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087995126000000007 2024-09-17T21:21:35.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864059+0000", 2024-09-17T21:21:35.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135797 2024-09-17T21:21:35.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864241+0000", 2024-09-17T21:21:35.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018252599999999999 2024-09-17T21:21:35.265 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.265 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.265 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.265 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12106 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:35.265 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:35.265 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.404834248, 2024-09-17T21:21:35.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088586394999999998, 2024-09-17T21:21:35.265 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.266 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.266 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:35.266 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:35.266 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:35.266 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:35.266 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12106 2024-09-17T21:21:35.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.266 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:35.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:35.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776001+0000", 2024-09-17T21:21:35.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4300000000000001e-07 2024-09-17T21:21:35.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776004+0000", 2024-09-17T21:21:35.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8049999999999998e-06 2024-09-17T21:21:35.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776018+0000", 2024-09-17T21:21:35.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4613999999999999e-05 2024-09-17T21:21:35.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776025+0000", 2024-09-17T21:21:35.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:21:35.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864279+0000", 2024-09-17T21:21:35.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088253570000000003 2024-09-17T21:21:35.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864433+0000", 2024-09-17T21:21:35.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015469400000000001 2024-09-17T21:21:35.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864586+0000", 2024-09-17T21:21:35.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000153003 2024-09-17T21:21:35.269 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.269 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.269 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.269 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25033 3.6 3.b74753e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e168)", 2024-09-17T21:21:35.269 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:21:35.270 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.6713909840000003, 2024-09-17T21:21:35.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93781952099999999, 2024-09-17T21:21:35.270 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.270 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.270 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:35.270 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:35.270 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:21:35.270 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:35.270 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25033 2024-09-17T21:21:35.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.270 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:21:35.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:21:35.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509444+0000", 2024-09-17T21:21:35.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0020000000000001e-06 2024-09-17T21:21:35.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509450+0000", 2024-09-17T21:21:35.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4249999999999997e-06 2024-09-17T21:21:35.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509460+0000", 2024-09-17T21:21:35.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0183999999999999e-05 2024-09-17T21:21:35.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509466+0000", 2024-09-17T21:21:35.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0039999999999996e-06 2024-09-17T21:21:35.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509540+0000", 2024-09-17T21:21:35.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3847e-05 2024-09-17T21:21:35.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:35.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509551+0000", 2024-09-17T21:21:35.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1731e-05 2024-09-17T21:21:35.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.447243+0000", 2024-09-17T21:21:35.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93769166000000004 2024-09-17T21:21:35.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.447263+0000", 2024-09-17T21:21:35.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9667999999999999e-05 2024-09-17T21:21:35.273 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.274 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.274 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.274 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25033 3.6 3:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e168)", 2024-09-17T21:21:35.274 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:21:35.274 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.6698653869999998, 2024-09-17T21:21:35.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.937309383, 2024-09-17T21:21:35.274 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.274 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.274 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:35.274 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:35.274 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:21:35.274 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:35.275 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25033 2024-09-17T21:21:35.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.275 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:21:35.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:21:35.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510970+0000", 2024-09-17T21:21:35.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.564e-06 2024-09-17T21:21:35.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510978+0000", 2024-09-17T21:21:35.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.362e-06 2024-09-17T21:21:35.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510998+0000", 2024-09-17T21:21:35.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0336999999999999e-05 2024-09-17T21:21:35.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511010+0000", 2024-09-17T21:21:35.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1786e-05 2024-09-17T21:21:35.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511080+0000", 2024-09-17T21:21:35.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0686999999999996e-05 2024-09-17T21:21:35.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:35.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511087+0000", 2024-09-17T21:21:35.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4899999999999997e-06 2024-09-17T21:21:35.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.448050+0000", 2024-09-17T21:21:35.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93696351700000002 2024-09-17T21:21:35.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.448278+0000", 2024-09-17T21:21:35.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022764000000000001 2024-09-17T21:21:35.278 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.278 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.278 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.278 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.278 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.278 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:35.280 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:21:35.280 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:35.275+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:35.336 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:21:35.413 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:35.411+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:35.447 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25642 3.1 3.6a5ad301 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e178)", 2024-09-17T21:21:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:35.302708+0000", 2024-09-17T21:21:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.14427890900000001, 2024-09-17T21:21:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.14430653399999999, 2024-09-17T21:21:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for peered", 2024-09-17T21:21:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:21:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25642 2024-09-17T21:21:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.302708+0000", 2024-09-17T21:21:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.302708+0000", 2024-09-17T21:21:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.302712+0000", 2024-09-17T21:21:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9330000000000003e-06 2024-09-17T21:21:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.302730+0000", 2024-09-17T21:21:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8473000000000001e-05 2024-09-17T21:21:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.302770+0000", 2024-09-17T21:21:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9669000000000003e-05 2024-09-17T21:21:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.302855+0000", 2024-09-17T21:21:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4981999999999994e-05 2024-09-17T21:21:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.303034+0000", 2024-09-17T21:21:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017908400000000001 2024-09-17T21:21:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-17T21:21:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.303055+0000", 2024-09-17T21:21:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0713999999999999e-05 2024-09-17T21:21:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.313876+0000", 2024-09-17T21:21:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010821061 2024-09-17T21:21:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.313901+0000", 2024-09-17T21:21:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5242e-05 2024-09-17T21:21:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:21:35.453 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 1 2024-09-17T21:21:35.453 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:35.462 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:21:35.482 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:35.477+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:35.583 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:35.579+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:35.622 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:35.622 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:35.622 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:21:35.622 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:21:35.623 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:35.636 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:21:35.725 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:35.723+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:35.784 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:35.781+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:35.800 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:35.800 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:35.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:35.800 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:35.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.800 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:21:35.801 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:35.801 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 192.92909393599999, 2024-09-17T21:21:35.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:21:35.801 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.801 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.801 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:35.801 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:35.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:35.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:21:35.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:21:35.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:21:35.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:21:35.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:21:35.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:21:35.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:21:35.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:21:35.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:21:35.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:21:35.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:21:35.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:21:35.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:35.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:21:35.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:21:35.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:21:35.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:21:35.805 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.805 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.805 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.805 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:21:35.805 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:35.805 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 192.92782885299999, 2024-09-17T21:21:35.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:21:35.805 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.805 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.805 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:35.805 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:35.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:35.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:21:35.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:21:35.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:21:35.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:21:35.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:21:35.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:21:35.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:21:35.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:21:35.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:21:35.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:21:35.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:21:35.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:21:35.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:35.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:21:35.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:21:35.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:21:35.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:21:35.809 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.809 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.809 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.809 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:21:35.809 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:35.809 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 192.918049443, 2024-09-17T21:21:35.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:21:35.809 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.809 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.810 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:35.810 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:35.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:35.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:21:35.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:21:35.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:21:35.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:21:35.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:21:35.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:21:35.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:21:35.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:21:35.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:21:35.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:21:35.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:21:35.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:21:35.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:35.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:21:35.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:21:35.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:21:35.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:21:35.813 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.813 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.813 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.813 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:21:35.813 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:35.814 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 192.90932703999999, 2024-09-17T21:21:35.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:21:35.814 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.814 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.814 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:35.814 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:35.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:35.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:21:35.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:21:35.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:21:35.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:21:35.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:21:35.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:21:35.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:21:35.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:21:35.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:21:35.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:21:35.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:21:35.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:21:35.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:35.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:21:35.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:21:35.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:21:35.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:21:35.818 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.818 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.818 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.818 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:21:35.818 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:35.818 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 179.524236476, 2024-09-17T21:21:35.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:21:35.818 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.818 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.818 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:35.818 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:35.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:35.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:21:35.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:21:35.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.820 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.820 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:21:35.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:21:35.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.820 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.820 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:21:35.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:21:35.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.820 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.820 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:21:35.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:21:35.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.821 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.821 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.821 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:21:35.821 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:21:35.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.821 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.821 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.821 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:21:35.821 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:21:35.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.821 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.821 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:35.822 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:21:35.822 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:21:35.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.822 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.822 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:21:35.822 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:21:35.822 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.822 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.822 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.822 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1780 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'195, mlcod=69'190)", 2024-09-17T21:21:35.822 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:35.823 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.54638087199999, 2024-09-17T21:21:35.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075761769999999999, 2024-09-17T21:21:35.823 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.823 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.823 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:35.823 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:35.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:35.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250375+0000", 2024-09-17T21:21:35.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:21:35.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250383+0000", 2024-09-17T21:21:35.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7870000000000002e-06 2024-09-17T21:21:35.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250402+0000", 2024-09-17T21:21:35.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9346000000000001e-05 2024-09-17T21:21:35.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250413+0000", 2024-09-17T21:21:35.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.039e-05 2024-09-17T21:21:35.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251353+0000", 2024-09-17T21:21:35.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000940091 2024-09-17T21:21:35.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.826 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.826 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.826 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251407+0000", 2024-09-17T21:21:35.826 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4701999999999999e-05 2024-09-17T21:21:35.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.826 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.826 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:35.826 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257829+0000", 2024-09-17T21:21:35.826 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064212779999999999 2024-09-17T21:21:35.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.826 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.826 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.826 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257950+0000", 2024-09-17T21:21:35.826 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121623 2024-09-17T21:21:35.827 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.827 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.827 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.827 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1788 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'198, mlcod=69'191)", 2024-09-17T21:21:35.827 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:35.827 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.54327243200001, 2024-09-17T21:21:35.827 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073690370000000002, 2024-09-17T21:21:35.827 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.827 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.827 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:35.827 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.828 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.828 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:35.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.828 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.828 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:35.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.828 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.828 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253484+0000", 2024-09-17T21:21:35.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2470000000000001e-06 2024-09-17T21:21:35.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253492+0000", 2024-09-17T21:21:35.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7159999999999992e-06 2024-09-17T21:21:35.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253509+0000", 2024-09-17T21:21:35.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.666e-05 2024-09-17T21:21:35.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253522+0000", 2024-09-17T21:21:35.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2643000000000001e-05 2024-09-17T21:21:35.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254853+0000", 2024-09-17T21:21:35.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331238 2024-09-17T21:21:35.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254885+0000", 2024-09-17T21:21:35.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1562e-05 2024-09-17T21:21:35.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:35.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260788+0000", 2024-09-17T21:21:35.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059030749999999998 2024-09-17T21:21:35.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260852+0000", 2024-09-17T21:21:35.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3895999999999999e-05 2024-09-17T21:21:35.831 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.831 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.831 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.831 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5746 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'477, mlcod=71'473)", 2024-09-17T21:21:35.831 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:35.831 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 139.537844874, 2024-09-17T21:21:35.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074182709999999997, 2024-09-17T21:21:35.832 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.832 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.832 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:35.832 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.832 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.832 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:35.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.832 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.832 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:35.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258912+0000", 2024-09-17T21:21:35.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:21:35.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258926+0000", 2024-09-17T21:21:35.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4249e-05 2024-09-17T21:21:35.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258946+0000", 2024-09-17T21:21:35.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9879999999999999e-05 2024-09-17T21:21:35.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258956+0000", 2024-09-17T21:21:35.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:21:35.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259380+0000", 2024-09-17T21:21:35.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042337599999999999 2024-09-17T21:21:35.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259412+0000", 2024-09-17T21:21:35.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:21:35.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.835 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:35.835 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266191+0000", 2024-09-17T21:21:35.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067793269999999999 2024-09-17T21:21:35.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.835 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.835 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266328+0000", 2024-09-17T21:21:35.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013690999999999999 2024-09-17T21:21:35.835 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.835 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.835 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.836 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5747 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'478, mlcod=71'473)", 2024-09-17T21:21:35.836 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:35.836 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 139.53713852499999, 2024-09-17T21:21:35.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068779440000000004, 2024-09-17T21:21:35.836 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.836 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.836 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:35.836 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:35.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:35.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259618+0000", 2024-09-17T21:21:35.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6199999999999999e-06 2024-09-17T21:21:35.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259630+0000", 2024-09-17T21:21:35.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1884e-05 2024-09-17T21:21:35.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259657+0000", 2024-09-17T21:21:35.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7339e-05 2024-09-17T21:21:35.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259677+0000", 2024-09-17T21:21:35.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0156000000000001e-05 2024-09-17T21:21:35.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260521+0000", 2024-09-17T21:21:35.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084312899999999997 2024-09-17T21:21:35.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260585+0000", 2024-09-17T21:21:35.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4558000000000001e-05 2024-09-17T21:21:35.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:35.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266336+0000", 2024-09-17T21:21:35.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057505020000000002 2024-09-17T21:21:35.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266494+0000", 2024-09-17T21:21:35.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158756 2024-09-17T21:21:35.840 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.840 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.840 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.840 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5751 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'480, mlcod=71'473)", 2024-09-17T21:21:35.840 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:35.840 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 139.53571912000001, 2024-09-17T21:21:35.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075756299999999999, 2024-09-17T21:21:35.840 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.840 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.841 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:35.841 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.841 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.841 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:35.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.841 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.841 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:35.842 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.842 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.842 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261037+0000", 2024-09-17T21:21:35.842 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:21:35.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.842 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.842 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261049+0000", 2024-09-17T21:21:35.842 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1063999999999999e-05 2024-09-17T21:21:35.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.843 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261072+0000", 2024-09-17T21:21:35.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3102e-05 2024-09-17T21:21:35.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.843 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261133+0000", 2024-09-17T21:21:35.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1276000000000001e-05 2024-09-17T21:21:35.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.843 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:21:35.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015334319999999999 2024-09-17T21:21:35.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262702+0000", 2024-09-17T21:21:35.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5281999999999999e-05 2024-09-17T21:21:35.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:35.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268527+0000", 2024-09-17T21:21:35.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058250389999999997 2024-09-17T21:21:35.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268611+0000", 2024-09-17T21:21:35.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4724999999999999e-05 2024-09-17T21:21:35.845 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.845 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.845 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.845 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5756 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'481, mlcod=71'473)", 2024-09-17T21:21:35.845 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:35.845 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 139.53485826400001, 2024-09-17T21:21:35.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006772386, 2024-09-17T21:21:35.845 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.845 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.845 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:35.845 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:35.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:35.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261899+0000", 2024-09-17T21:21:35.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.903e-06 2024-09-17T21:21:35.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.847 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.847 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.847 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261912+0000", 2024-09-17T21:21:35.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-05 2024-09-17T21:21:35.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.847 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.847 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.847 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261938+0000", 2024-09-17T21:21:35.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6098999999999999e-05 2024-09-17T21:21:35.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.847 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.847 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.847 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261955+0000", 2024-09-17T21:21:35.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.696e-05 2024-09-17T21:21:35.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.848 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.848 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.848 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263845+0000", 2024-09-17T21:21:35.848 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001890187 2024-09-17T21:21:35.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.848 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.848 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.848 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263907+0000", 2024-09-17T21:21:35.848 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1600999999999995e-05 2024-09-17T21:21:35.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.848 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.848 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:35.848 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268615+0000", 2024-09-17T21:21:35.849 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047086070000000001 2024-09-17T21:21:35.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.849 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.849 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268669+0000", 2024-09-17T21:21:35.849 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3588999999999998e-05 2024-09-17T21:21:35.849 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.849 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.849 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.849 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4978.0:598 3.9 e80/23 3:9701cc17:::rbd_data.1152724ff2a3.0000000000000019:head v 80'202, mlcod=80'198)", 2024-09-17T21:21:35.849 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:21:35.849 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 129.03328388099999, 2024-09-17T21:21:35.849 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066473340000000004, 2024-09-17T21:21:35.850 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.850 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.850 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:35.850 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:21:35.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763471+0000", 2024-09-17T21:21:35.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.851 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763472+0000", 2024-09-17T21:21:35.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6899999999999997e-07 2024-09-17T21:21:35.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.851 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763478+0000", 2024-09-17T21:21:35.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:21:35.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.851 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763489+0000", 2024-09-17T21:21:35.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0628999999999999e-05 2024-09-17T21:21:35.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.852 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.852 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763495+0000", 2024-09-17T21:21:35.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6649999999999997e-06 2024-09-17T21:21:35.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.852 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.852 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763907+0000", 2024-09-17T21:21:35.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041163899999999999 2024-09-17T21:21:35.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.852 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.853 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.763976+0000", 2024-09-17T21:21:35.853 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8534999999999993e-05 2024-09-17T21:21:35.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.853 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.853 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:35.853 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.769900+0000", 2024-09-17T21:21:35.853 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059243270000000001 2024-09-17T21:21:35.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.853 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.853 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.853 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:26.770118+0000", 2024-09-17T21:21:35.853 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021842500000000001 2024-09-17T21:21:35.853 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.853 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.853 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.854 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.854 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:468 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'94, mlcod=89'87)", 2024-09-17T21:21:35.854 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:35.854 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 116.110538582, 2024-09-17T21:21:35.854 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066528790000000004, 2024-09-17T21:21:35.854 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.854 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.854 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:35.854 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.854 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.854 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.854 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:35.854 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.854 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.855 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:35.855 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.855 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686218+0000", 2024-09-17T21:21:35.855 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3060000000000001e-06 2024-09-17T21:21:35.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.855 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686256+0000", 2024-09-17T21:21:35.855 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8333e-05 2024-09-17T21:21:35.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.856 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.856 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686272+0000", 2024-09-17T21:21:35.856 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:21:35.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.856 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.856 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686283+0000", 2024-09-17T21:21:35.856 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0903e-05 2024-09-17T21:21:35.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.856 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.856 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686682+0000", 2024-09-17T21:21:35.856 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039935099999999999 2024-09-17T21:21:35.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.857 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.857 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.857 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686785+0000", 2024-09-17T21:21:35.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010275199999999999 2024-09-17T21:21:35.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.857 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.857 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:35.857 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692816+0000", 2024-09-17T21:21:35.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060317579999999999 2024-09-17T21:21:35.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.857 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.857 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.857 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692869+0000", 2024-09-17T21:21:35.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2775999999999998e-05 2024-09-17T21:21:35.858 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.858 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.858 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.858 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.858 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:755 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'127, mlcod=89'122)", 2024-09-17T21:21:35.858 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:35.858 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 115.974083368, 2024-09-17T21:21:35.858 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084014080000000008, 2024-09-17T21:21:35.858 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.858 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.858 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:35.858 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.858 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.858 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.859 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:35.859 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.859 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.859 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.859 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:35.859 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.859 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.859 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.859 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:35.859 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:21:35.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.859 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822683+0000", 2024-09-17T21:21:35.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0674e-05 2024-09-17T21:21:35.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822698+0000", 2024-09-17T21:21:35.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5189e-05 2024-09-17T21:21:35.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822708+0000", 2024-09-17T21:21:35.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9119999999999993e-06 2024-09-17T21:21:35.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822748+0000", 2024-09-17T21:21:35.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9727000000000001e-05 2024-09-17T21:21:35.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822759+0000", 2024-09-17T21:21:35.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0865e-05 2024-09-17T21:21:35.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:35.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831034+0000", 2024-09-17T21:21:35.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082750710000000002 2024-09-17T21:21:35.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.862 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.862 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831073+0000", 2024-09-17T21:21:35.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9063999999999997e-05 2024-09-17T21:21:35.862 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.862 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.862 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.862 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:756 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'128, mlcod=89'124)", 2024-09-17T21:21:35.862 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:35.863 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 115.97216073600001, 2024-09-17T21:21:35.863 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0076300270000000002, 2024-09-17T21:21:35.863 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.863 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.863 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:35.863 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.863 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.863 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:35.863 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.863 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.863 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:35.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824595+0000", 2024-09-17T21:21:35.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-17T21:21:35.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824600+0000", 2024-09-17T21:21:35.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:21:35.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824612+0000", 2024-09-17T21:21:35.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.181e-05 2024-09-17T21:21:35.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824619+0000", 2024-09-17T21:21:35.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:21:35.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824671+0000", 2024-09-17T21:21:35.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2262999999999998e-05 2024-09-17T21:21:35.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.866 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824695+0000", 2024-09-17T21:21:35.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3883000000000002e-05 2024-09-17T21:21:35.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.866 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:35.866 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832185+0000", 2024-09-17T21:21:35.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007490048 2024-09-17T21:21:35.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.866 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.866 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832224+0000", 2024-09-17T21:21:35.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8957e-05 2024-09-17T21:21:35.866 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.866 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.867 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.867 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2635 3.d e90/23 3:b3d1e920:::rbd_data.1152a8ae2019.000000000000001f:head v 90'252, mlcod=90'246)", 2024-09-17T21:21:35.867 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:35.867 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 114.94960166600001, 2024-09-17T21:21:35.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0077265190000000003, 2024-09-17T21:21:35.867 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.867 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.867 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:35.867 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.867 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.867 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:35.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:35.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847155+0000", 2024-09-17T21:21:35.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.544e-06 2024-09-17T21:21:35.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847165+0000", 2024-09-17T21:21:35.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.043e-05 2024-09-17T21:21:35.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847189+0000", 2024-09-17T21:21:35.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3659000000000002e-05 2024-09-17T21:21:35.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847210+0000", 2024-09-17T21:21:35.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1163e-05 2024-09-17T21:21:35.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848856+0000", 2024-09-17T21:21:35.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001645521 2024-09-17T21:21:35.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.870 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848905+0000", 2024-09-17T21:21:35.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9048999999999997e-05 2024-09-17T21:21:35.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:35.870 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854770+0000", 2024-09-17T21:21:35.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058652879999999998 2024-09-17T21:21:35.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.871 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854880+0000", 2024-09-17T21:21:35.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010986500000000001 2024-09-17T21:21:35.871 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.871 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.871 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.871 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "MOSDPGPush(3.5 151/149 [PushOp(3:a57cb214:::rbd_data.115262b8e217.0000000000000009:1d, version: 103'698, data_included: [196608~24576,229376~24576,262144~24576,294912~131072,524288~98304,819200~8192,843776~16384,876544~16384,909312~16384,942080~8192,1146880~16384,1179648~16384,1212416~16384,1474560~98304,1794048~8192,1826816~8192,1859584~8192,2105344~24576,2138112~24576,2170880~24576,2646016~8192,2678784~8192,2711552~8192,3072000~8192,3104768~8192,3137536~106496], data_size: 761856, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:a57cb214:::rbd_data.115262b8e217.0000000000000009:1d@103'698, size: 3276800, copy_subset: [(0, 3276800)], clone_subset: {}, snapset: 1d=[]:{1d: [1d]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3276800, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T21:21:35.871 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:07.006386+0000", 2024-09-17T21:21:35.871 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 28.790369206000001, 2024-09-17T21:21:35.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082699420000000006, 2024-09-17T21:21:35.871 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.872 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.872 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:35.872 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.006386+0000", 2024-09-17T21:21:35.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.006386+0000", 2024-09-17T21:21:35.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.873 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.006388+0000", 2024-09-17T21:21:35.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.827e-06 2024-09-17T21:21:35.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.873 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.007671+0000", 2024-09-17T21:21:35.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0012836340000000001 2024-09-17T21:21:35.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.873 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.007737+0000", 2024-09-17T21:21:35.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5815000000000006e-05 2024-09-17T21:21:35.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.874 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.874 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.874 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.007782+0000", 2024-09-17T21:21:35.874 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5509000000000001e-05 2024-09-17T21:21:35.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.874 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.874 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.874 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.008023+0000", 2024-09-17T21:21:35.874 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000240184 2024-09-17T21:21:35.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.874 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.874 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.875 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.008065+0000", 2024-09-17T21:21:35.875 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1978999999999999e-05 2024-09-17T21:21:35.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.875 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.875 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.875 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.014656+0000", 2024-09-17T21:21:35.875 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065909940000000002 2024-09-17T21:21:35.875 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.875 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.875 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.875 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.875 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(osd.3.0:3834 3.8 e165/154 3:11ef155a:::rbd_data.11527217c540.000000000000003f:2b v 165'498, mlcod=152'495)", 2024-09-17T21:21:35.876 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:22.375046+0000", 2024-09-17T21:21:35.876 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 13.421709021, 2024-09-17T21:21:35.876 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010228170999999999, 2024-09-17T21:21:35.876 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.876 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.876 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:35.876 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.876 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.876 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375046+0000", 2024-09-17T21:21:35.876 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.876 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375046+0000", 2024-09-17T21:21:35.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375047+0000", 2024-09-17T21:21:35.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.373e-06 2024-09-17T21:21:35.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375085+0000", 2024-09-17T21:21:35.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7376999999999998e-05 2024-09-17T21:21:35.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.878 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.878 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375127+0000", 2024-09-17T21:21:35.878 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1941999999999999e-05 2024-09-17T21:21:35.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.878 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.878 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375150+0000", 2024-09-17T21:21:35.878 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3113e-05 2024-09-17T21:21:35.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.878 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.878 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375261+0000", 2024-09-17T21:21:35.878 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111402 2024-09-17T21:21:35.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.879 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.879 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375287+0000", 2024-09-17T21:21:35.879 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5585e-05 2024-09-17T21:21:35.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.879 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:35.879 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.385224+0000", 2024-09-17T21:21:35.879 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0099369750000000007 2024-09-17T21:21:35.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.879 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.879 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.385274+0000", 2024-09-17T21:21:35.879 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0404000000000001e-05 2024-09-17T21:21:35.880 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.880 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.880 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.880 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4434.0:25007 3.1e e168/166 3:7ce0ed33:::rbd_object_map.11528d8227e0:head v 168'728, mlcod=26'23)", 2024-09-17T21:21:35.880 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:25.445808+0000", 2024-09-17T21:21:35.880 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 10.350946509, 2024-09-17T21:21:35.880 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074791850000000002, 2024-09-17T21:21:35.880 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.880 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.880 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:35.880 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445808+0000", 2024-09-17T21:21:35.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445808+0000", 2024-09-17T21:21:35.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445809+0000", 2024-09-17T21:21:35.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0299999999999997e-07 2024-09-17T21:21:35.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.882 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.882 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445816+0000", 2024-09-17T21:21:35.882 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.392e-06 2024-09-17T21:21:35.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.882 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.882 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445828+0000", 2024-09-17T21:21:35.882 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2187e-05 2024-09-17T21:21:35.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.882 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.882 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445834+0000", 2024-09-17T21:21:35.883 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5710000000000003e-06 2024-09-17T21:21:35.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.883 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.883 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445896+0000", 2024-09-17T21:21:35.883 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2024000000000001e-05 2024-09-17T21:21:35.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.883 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.883 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445919+0000", 2024-09-17T21:21:35.883 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2480000000000002e-05 2024-09-17T21:21:35.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.883 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:35.884 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.453224+0000", 2024-09-17T21:21:35.884 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073048729999999999 2024-09-17T21:21:35.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.884 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.884 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.453288+0000", 2024-09-17T21:21:35.884 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3755000000000002e-05 2024-09-17T21:21:35.884 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.884 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.884 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.884 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25377 3.8 3:1508d022:::rbd_data.11524f65d4ff.000000000000000a:head [delete] snapc 29=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-17T21:21:35.885 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:30.688594+0000", 2024-09-17T21:21:35.885 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.1081609710000002, 2024-09-17T21:21:35.885 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010688471, 2024-09-17T21:21:35.885 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:35.885 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:35.885 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:35.885 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:35.885 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:21:35.885 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:35.885 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25377 2024-09-17T21:21:35.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.885 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:35.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.885 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:35.886 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.688594+0000", 2024-09-17T21:21:35.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.886 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:35.886 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.688594+0000", 2024-09-17T21:21:35.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:35.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.886 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:35.886 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.688603+0000", 2024-09-17T21:21:35.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4469999999999995e-06 2024-09-17T21:21:35.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.887 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:35.887 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.688615+0000", 2024-09-17T21:21:35.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1515000000000001e-05 2024-09-17T21:21:35.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.887 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:35.887 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.688646+0000", 2024-09-17T21:21:35.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0936000000000003e-05 2024-09-17T21:21:35.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.887 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:35.887 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.688666+0000", 2024-09-17T21:21:35.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9729999999999999e-05 2024-09-17T21:21:35.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:35.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.688812+0000", 2024-09-17T21:21:35.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000146265 2024-09-17T21:21:35.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:35.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.689181+0000", 2024-09-17T21:21:35.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00036869300000000002 2024-09-17T21:21:35.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,5", 2024-09-17T21:21:35.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.689503+0000", 2024-09-17T21:21:35.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000322539 2024-09-17T21:21:35.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:35.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.691833+0000", 2024-09-17T21:21:35.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023295159999999998 2024-09-17T21:21:35.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:35.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.691880+0000", 2024-09-17T21:21:35.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7899000000000003e-05 2024-09-17T21:21:35.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:35.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.698812+0000", 2024-09-17T21:21:35.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0069317390000000001 2024-09-17T21:21:35.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:35.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.698953+0000", 2024-09-17T21:21:35.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014077100000000001 2024-09-17T21:21:35.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:35.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:35.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:35.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.699282+0000", 2024-09-17T21:21:35.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00032942099999999999 2024-09-17T21:21:35.890 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.890 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.890 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.891 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:35.891 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:35.891 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:35.892 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:21:35.893 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:35.889+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:35.993 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:35.991+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:36.024 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:36.024 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:36.038 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:21:36.094 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:36.089+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:36.160 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:36.160 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:36.160 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:36.160 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:36.160 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:36.174 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:21:36.215 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:36.213+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:36.295 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:36.291+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:36.297 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:21:36.312 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-17T21:21:36.357 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:21:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 219.338360198, 2024-09-17T21:21:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:21:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:36.357 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:21:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:21:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:21:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:36.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:21:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:21:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:21:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:21:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:21:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:21:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:21:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:21:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:21:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:21:36.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:21:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:21:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:21:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:21:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:36.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:21:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:21:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:21:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:21:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:21:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:21:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:21:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:21:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:21:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:21:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:21:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:21:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:21:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:36.363 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 199.75819591800001, 2024-09-17T21:21:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:21:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:21:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:21:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:21:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:21:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:21:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:21:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:21:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:21:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:21:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:21:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:21:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:21:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:21:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:21:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:21:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:21:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:21:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:21:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:21:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:21:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:21:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:21:36.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:21:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:21:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:21:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 199.758098606, 2024-09-17T21:21:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:21:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:21:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:21:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:21:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:21:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:21:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:21:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:21:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:21:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:21:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:21:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:21:36.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:21:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:21:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:21:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:21:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:21:36.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:21:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:21:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:21:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:21:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:21:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:21:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:21:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:21:36.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:21:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:21:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:21:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:21:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 199.75601180500001, 2024-09-17T21:21:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:21:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:21:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:21:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:21:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:21:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:21:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:21:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:21:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:21:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:21:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:21:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:21:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:21:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:21:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:21:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:21:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:21:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:21:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:21:36.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:21:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:21:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:21:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:21:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:21:36.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:21:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:21:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:21:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:21:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:21:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:21:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:21:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:21:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:21:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:21:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:21:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.382 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 167.072113024, 2024-09-17T21:21:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:21:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:21:36.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:21:36.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:21:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:21:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:21:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:21:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:21:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:36.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:21:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:21:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:21:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:21:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:21:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:21:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:21:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:21:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:21:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:21:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:21:36.387 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:21:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:21:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:21:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:21:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:21:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 167.06302131300001, 2024-09-17T21:21:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:21:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:21:36.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:21:36.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:21:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:21:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:21:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:21:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:21:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:36.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:21:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:21:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:21:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:21:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:21:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:21:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:21:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:21:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:21:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:21:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:21:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:21:36.393 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 167.06284926199999, 2024-09-17T21:21:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:21:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:36.394 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:21:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:21:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:21:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:21:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:21:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:21:36.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:21:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:21:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:21:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:21:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:21:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:36.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:21:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:21:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:21:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:21:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:21:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:21:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:21:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:21:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 166.887916515, 2024-09-17T21:21:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:21:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:36.399 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:21:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:36.400 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:36.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:36.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:21:36.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:21:36.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:36.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:21:36.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:21:36.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:36.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:21:36.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:21:36.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:36.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:21:36.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:21:36.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:21:36.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:21:36.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:36.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:21:36.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:21:36.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:21:36.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:21:36.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:36.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:21:36.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:21:36.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.404 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:36.404 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:21:36.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:21:36.404 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.404 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:36.404 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.404 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:36.404 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:36.404 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 166.88783195799999, 2024-09-17T21:21:36.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:21:36.404 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:36.404 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:36.405 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:36.405 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:36.405 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:36.405 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:36.405 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:21:36.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.405 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:36.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:36.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:36.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:36.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:36.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:36.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:36.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:21:36.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:36.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:21:36.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:21:36.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:36.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:21:36.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:21:36.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:36.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:21:36.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:21:36.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:21:36.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:21:36.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:36.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:21:36.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:21:36.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.408 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:21:36.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:21:36.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.408 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:36.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:21:36.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:21:36.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:36.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:21:36.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:21:36.409 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.409 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:36.409 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.409 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:36.409 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:36.409 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 166.88776632099999, 2024-09-17T21:21:36.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:21:36.409 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:36.410 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:36.410 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:36.410 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:36.410 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:36.410 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:36.410 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:21:36.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.410 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:36.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:36.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:36.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:36.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:36.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:36.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:21:36.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:21:36.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:36.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:21:36.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:21:36.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.412 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:36.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:21:36.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:21:36.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.412 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:36.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:21:36.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:21:36.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:21:36.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:21:36.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:36.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:21:36.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:21:36.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:21:36.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:21:36.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:36.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:21:36.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:21:36.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:36.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:21:36.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:21:36.414 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.414 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:36.414 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.415 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:36.415 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:36.415 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 115.514191224, 2024-09-17T21:21:36.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:21:36.415 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:36.415 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:36.415 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:36.415 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:36.415 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:21:36.415 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:36.415 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:21:36.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.415 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:36.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:36.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:36.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:36.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:36.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:36.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:21:36.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:21:36.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:36.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:21:36.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:21:36.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:36.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:21:36.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:21:36.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:36.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:21:36.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:21:36.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:21:36.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:21:36.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:36.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:21:36.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:21:36.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:36.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:21:36.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:21:36.419 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.419 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:36.419 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.419 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:36.419 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:36.419 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 115.514071712, 2024-09-17T21:21:36.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:21:36.419 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:36.419 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:36.419 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:36.419 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:36.419 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:21:36.419 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:36.420 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:21:36.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.420 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:36.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:36.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:36.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:36.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:36.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:36.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:21:36.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:21:36.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:36.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:21:36.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:21:36.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:36.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:21:36.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:21:36.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.422 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:36.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:21:36.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:21:36.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.422 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:21:36.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:21:36.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.422 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:36.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:21:36.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:21:36.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:36.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:21:36.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:21:36.423 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.423 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:36.423 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.423 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:995 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 575488~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:36.423 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:21:36.423 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 70.921040722000001, 2024-09-17T21:21:36.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084214699000000004, 2024-09-17T21:21:36.423 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:36.424 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:36.424 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:36.424 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:36.424 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:36.424 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:36.424 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 995 2024-09-17T21:21:36.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.424 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:36.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:36.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:21:36.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:36.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:21:36.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:36.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431086+0000", 2024-09-17T21:21:36.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7999999999999995e-07 2024-09-17T21:21:36.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:36.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431088+0000", 2024-09-17T21:21:36.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4729999999999999e-06 2024-09-17T21:21:36.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.426 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:36.426 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431126+0000", 2024-09-17T21:21:36.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7583999999999998e-05 2024-09-17T21:21:36.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.426 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:36.426 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431137+0000", 2024-09-17T21:21:36.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1768e-05 2024-09-17T21:21:36.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.426 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.426 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515003+0000", 2024-09-17T21:21:36.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083865097 2024-09-17T21:21:36.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:36.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515115+0000", 2024-09-17T21:21:36.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112561 2024-09-17T21:21:36.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:36.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515300+0000", 2024-09-17T21:21:36.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018453600000000001 2024-09-17T21:21:36.427 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.427 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:36.427 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.428 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:996 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:36.428 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:36.428 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 70.920958341000002, 2024-09-17T21:21:36.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084493736999999999, 2024-09-17T21:21:36.428 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:36.428 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:36.428 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:36.428 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:36.428 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:36.428 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:36.428 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 996 2024-09-17T21:21:36.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.429 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:36.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:36.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:36.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:36.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:36.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:36.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431168+0000", 2024-09-17T21:21:36.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4e-07 2024-09-17T21:21:36.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:36.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431172+0000", 2024-09-17T21:21:36.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.608e-06 2024-09-17T21:21:36.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:36.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431180+0000", 2024-09-17T21:21:36.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5650000000000002e-06 2024-09-17T21:21:36.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:36.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431188+0000", 2024-09-17T21:21:36.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:21:36.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515354+0000", 2024-09-17T21:21:36.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084165977000000003 2024-09-17T21:21:36.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:36.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515451+0000", 2024-09-17T21:21:36.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6669000000000006e-05 2024-09-17T21:21:36.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:36.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515661+0000", 2024-09-17T21:21:36.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021021599999999999 2024-09-17T21:21:36.432 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.432 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:36.432 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.432 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:997 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 583680~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:36.432 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:36.432 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 70.920926945999994, 2024-09-17T21:21:36.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084772419000000002, 2024-09-17T21:21:36.433 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:36.433 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:36.433 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:36.433 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:36.433 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:36.433 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:36.433 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 997 2024-09-17T21:21:36.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.433 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:36.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:36.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:36.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:36.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:36.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:36.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:36.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0400000000000005e-07 2024-09-17T21:21:36.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:36.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431202+0000", 2024-09-17T21:21:36.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:21:36.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:36.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431208+0000", 2024-09-17T21:21:36.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4740000000000002e-06 2024-09-17T21:21:36.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:36.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431214+0000", 2024-09-17T21:21:36.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6550000000000001e-06 2024-09-17T21:21:36.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515741+0000", 2024-09-17T21:21:36.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084526904 2024-09-17T21:21:36.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:36.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515826+0000", 2024-09-17T21:21:36.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4643999999999998e-05 2024-09-17T21:21:36.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:36.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515971+0000", 2024-09-17T21:21:36.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145444 2024-09-17T21:21:36.436 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.436 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:36.436 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.436 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:998 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 587776~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:36.437 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:36.437 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 70.920903973999998, 2024-09-17T21:21:36.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085108565999999997, 2024-09-17T21:21:36.437 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:36.437 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:36.437 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:36.437 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:36.437 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:36.437 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:36.437 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 998 2024-09-17T21:21:36.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.437 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:36.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:36.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:36.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:36.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:36.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:36.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:36.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:21:36.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:36.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431224+0000", 2024-09-17T21:21:36.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:21:36.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:36.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431231+0000", 2024-09-17T21:21:36.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000002e-06 2024-09-17T21:21:36.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:36.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431236+0000", 2024-09-17T21:21:36.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.648e-06 2024-09-17T21:21:36.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516024+0000", 2024-09-17T21:21:36.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084787584999999999 2024-09-17T21:21:36.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:36.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516122+0000", 2024-09-17T21:21:36.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8076000000000001e-05 2024-09-17T21:21:36.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:36.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516330+0000", 2024-09-17T21:21:36.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020808 2024-09-17T21:21:36.441 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.441 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:36.441 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.441 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1000 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 591872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:36.441 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:36.441 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 70.920881859000005, 2024-09-17T21:21:36.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085389065, 2024-09-17T21:21:36.441 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:36.441 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:36.441 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:36.441 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:36.441 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:36.441 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:36.442 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1000 2024-09-17T21:21:36.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.442 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:36.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:36.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:36.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:36.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:36.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:36.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:36.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:21:36.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:36.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431246+0000", 2024-09-17T21:21:36.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1749999999999999e-06 2024-09-17T21:21:36.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:36.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431253+0000", 2024-09-17T21:21:36.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2790000000000004e-06 2024-09-17T21:21:36.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:36.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431258+0000", 2024-09-17T21:21:36.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3299999999999998e-06 2024-09-17T21:21:36.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516384+0000", 2024-09-17T21:21:36.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085125549999999994 2024-09-17T21:21:36.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:36.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516460+0000", 2024-09-17T21:21:36.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6359999999999994e-05 2024-09-17T21:21:36.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:36.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516633+0000", 2024-09-17T21:21:36.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017285900000000001 2024-09-17T21:21:36.445 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.445 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:36.445 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.445 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1001 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 595968~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:36.445 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:36.445 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 70.920858452999994, 2024-09-17T21:21:36.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085614715999999993, 2024-09-17T21:21:36.446 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:36.446 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:36.446 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:36.446 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:36.446 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:36.446 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:36.446 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1001 2024-09-17T21:21:36.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.446 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:36.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:36.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:36.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:36.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:36.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:36.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438066+0000", 2024-09-17T21:21:36.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067989490000000003 2024-09-17T21:21:36.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:36.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438073+0000", 2024-09-17T21:21:36.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8419999999999999e-06 2024-09-17T21:21:36.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:36.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438090+0000", 2024-09-17T21:21:36.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6657e-05 2024-09-17T21:21:36.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:36.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438099+0000", 2024-09-17T21:21:36.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0529999999999996e-06 2024-09-17T21:21:36.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516678+0000", 2024-09-17T21:21:36.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078579422999999995 2024-09-17T21:21:36.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:36.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516739+0000", 2024-09-17T21:21:36.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1315000000000005e-05 2024-09-17T21:21:36.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:36.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516882+0000", 2024-09-17T21:21:36.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014247700000000001 2024-09-17T21:21:36.449 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.449 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:36.449 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.450 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:19998 3.5 3:a4885450:::rbd_object_map.1152d139f9f9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-17T21:21:36.450 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:36.450 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 56.949591208000001, 2024-09-17T21:21:36.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90715489299999996, 2024-09-17T21:21:36.450 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:36.450 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:36.450 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:36.450 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:36.450 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:36.450 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:36.450 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19998 2024-09-17T21:21:36.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.450 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:36.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:36.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:36.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:36.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:36.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:36.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402537+0000", 2024-09-17T21:21:36.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5600000000000001e-06 2024-09-17T21:21:36.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:36.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402549+0000", 2024-09-17T21:21:36.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1929999999999999e-05 2024-09-17T21:21:36.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:36.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402579+0000", 2024-09-17T21:21:36.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0093000000000001e-05 2024-09-17T21:21:36.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:36.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402595+0000", 2024-09-17T21:21:36.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5855999999999999e-05 2024-09-17T21:21:36.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402731+0000", 2024-09-17T21:21:36.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013610799999999999 2024-09-17T21:21:36.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:36.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402750+0000", 2024-09-17T21:21:36.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8737e-05 2024-09-17T21:21:36.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.305977+0000", 2024-09-17T21:21:36.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90322768399999998 2024-09-17T21:21:36.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:36.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306236+0000", 2024-09-17T21:21:36.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000258111 2024-09-17T21:21:36.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:36.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306557+0000", 2024-09-17T21:21:36.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032159799999999998 2024-09-17T21:21:36.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:36.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.308286+0000", 2024-09-17T21:21:36.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017292150000000001 2024-09-17T21:21:36.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:36.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309530+0000", 2024-09-17T21:21:36.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00124409 2024-09-17T21:21:36.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:36.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309630+0000", 2024-09-17T21:21:36.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9689e-05 2024-09-17T21:21:36.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:36.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309689+0000", 2024-09-17T21:21:36.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9222000000000003e-05 2024-09-17T21:21:36.456 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.456 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:36.456 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.456 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:25277 3.1f 3:ff5d7cd5:::rbd_header.11524f65d4ff:head [watch ping cookie 139860845478480 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-17T21:21:36.456 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:21:36.456 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.7666351389999999, 2024-09-17T21:21:36.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.92185720000000004, 2024-09-17T21:21:36.456 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:36.456 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:36.457 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:36.457 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:36.457 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:36.457 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:36.457 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 25277 2024-09-17T21:21:36.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.457 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:36.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:36.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:21:36.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:36.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:21:36.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:36.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:36.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585493+0000", 2024-09-17T21:21:36.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:21:36.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:36.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585505+0000", 2024-09-17T21:21:36.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2466000000000001e-05 2024-09-17T21:21:36.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:36.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585537+0000", 2024-09-17T21:21:36.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1183000000000003e-05 2024-09-17T21:21:36.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:36.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585553+0000", 2024-09-17T21:21:36.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.662e-05 2024-09-17T21:21:36.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585696+0000", 2024-09-17T21:21:36.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014293800000000001 2024-09-17T21:21:36.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:36.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585718+0000", 2024-09-17T21:21:36.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1860000000000001e-05 2024-09-17T21:21:36.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.501188+0000", 2024-09-17T21:21:36.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.91547051300000004 2024-09-17T21:21:36.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:21:36.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.501260+0000", 2024-09-17T21:21:36.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1571999999999998e-05 2024-09-17T21:21:36.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:36.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507002+0000", 2024-09-17T21:21:36.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057414850000000002 2024-09-17T21:21:36.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:36.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507143+0000", 2024-09-17T21:21:36.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014140299999999999 2024-09-17T21:21:36.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:36.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:36.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:36.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507348+0000", 2024-09-17T21:21:36.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000204773 2024-09-17T21:21:36.462 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.462 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:36.462 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.462 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:36.462 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:36.462 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:36.464 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:21:36.464 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:36.461+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:36.476 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:36.476 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:36.494 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:36.495 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:36.512 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:21:36.512 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:21:36.531 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:36.531 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:36.552 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:36.552 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:36.563 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:36.561+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:36.570 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:36.570 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:36.638 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:36.638 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:36.652 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:21:36.664 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:36.659+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:36.765 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:36.761+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:36.816 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:36.816 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:36.830 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:21:36.871 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:36.869+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:36.967 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:36.965+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:36.992 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:36.992 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:37.006 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:21:37.068 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:37.063+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:37.169 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:37.165+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:37.178 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:37.179 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:37.193 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:21:37.270 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:37.267+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:37.367 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:37.367 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:37.371 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:37.367+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:37.383 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:21:37.472 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:37.467+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:37.518 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:37.518 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:37.534 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:21:37.573 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:37.569+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:37.673 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:37.671+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:37.676 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:37.676 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:37.692 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:21:37.785 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:37.783+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:37.875 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:37.871+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:37.878 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:37.878 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:37.893 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:21:37.976 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:37.973+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:38.061 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:38.061 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:38.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:21:38.077 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:38.075+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:38.178 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:38.173+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:38.203 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:38.203 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:38.218 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:21:38.279 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:38.277+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:38.351 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:21:38.355 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:38.355 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:38.375 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:21:38.380 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:38.377+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:38.481 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:38.477+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:38.557 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:38.557 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:38.571 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:21:38.588 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:21:38.617 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:38.615+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:38.683 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:38.679+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:38.704 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:38.704 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:38.720 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:21:38.756 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:38.757 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:38.775 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:38.775 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:38.783 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:38.779+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:38.792 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:21:38.792 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:21:38.814 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:38.814 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:38.835 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:38.836 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:38.857 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:38.857 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:38.886 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:38.886 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:38.887 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:21:38.887 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:21:38.887 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:38.887 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:38.883+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:38.902 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:21:38.985 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:38.981+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:39.070 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:39.070 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:39.070 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:39.070 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:39.070 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.070 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:542 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'12, mlcod=29'9)", 2024-09-17T21:21:39.070 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:39.070 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 196.199641802, 2024-09-17T21:21:39.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026455586, 2024-09-17T21:21:39.071 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:39.071 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:39.071 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:39.071 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:39.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:39.071 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:39.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:39.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867661+0000", 2024-09-17T21:21:39.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:39.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.867663+0000", 2024-09-17T21:21:39.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.733e-06 2024-09-17T21:21:39.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:39.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871556+0000", 2024-09-17T21:21:39.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038933040000000002 2024-09-17T21:21:39.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:39.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871593+0000", 2024-09-17T21:21:39.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6959999999999998e-05 2024-09-17T21:21:39.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871606+0000", 2024-09-17T21:21:39.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3091000000000001e-05 2024-09-17T21:21:39.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:39.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871705+0000", 2024-09-17T21:21:39.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9312000000000001e-05 2024-09-17T21:21:39.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:39.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.871740+0000", 2024-09-17T21:21:39.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4758999999999997e-05 2024-09-17T21:21:39.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:39.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.893887+0000", 2024-09-17T21:21:39.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022147038000000001 2024-09-17T21:21:39.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.075 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:39.075 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.894117+0000", 2024-09-17T21:21:39.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938900000000001 2024-09-17T21:21:39.075 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.075 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:39.075 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.075 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:540 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'6, mlcod=29'4)", 2024-09-17T21:21:39.075 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:39.075 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 196.19837671900001, 2024-09-17T21:21:39.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029109640999999999, 2024-09-17T21:21:39.075 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:39.075 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:39.076 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:39.076 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:39.076 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.076 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:39.076 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:39.076 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.076 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.076 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:39.076 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868926+0000", 2024-09-17T21:21:39.076 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:39.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.868928+0000", 2024-09-17T21:21:39.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:21:39.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:39.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876399+0000", 2024-09-17T21:21:39.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074708200000000004 2024-09-17T21:21:39.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:39.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876422+0000", 2024-09-17T21:21:39.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3600000000000001e-05 2024-09-17T21:21:39.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876437+0000", 2024-09-17T21:21:39.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4881e-05 2024-09-17T21:21:39.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:39.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876532+0000", 2024-09-17T21:21:39.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4934999999999998e-05 2024-09-17T21:21:39.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.079 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:39.079 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.876560+0000", 2024-09-17T21:21:39.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7521e-05 2024-09-17T21:21:39.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.079 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:39.079 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897986+0000", 2024-09-17T21:21:39.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.021426332999999999 2024-09-17T21:21:39.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.079 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:39.079 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898036+0000", 2024-09-17T21:21:39.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9840999999999999e-05 2024-09-17T21:21:39.079 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.079 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:39.080 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.080 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:543 3.4 e30/23 3:2231c195:::rbd_data.1152772b7dab.0000000000000002:head v 30'14, mlcod=29'9)", 2024-09-17T21:21:39.080 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:39.080 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 196.18859730899999, 2024-09-17T21:21:39.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019300102999999999, 2024-09-17T21:21:39.080 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:39.080 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:39.080 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:39.080 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:39.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:39.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:39.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.081 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:39.081 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878705+0000", 2024-09-17T21:21:39.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.081 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:39.081 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878708+0000", 2024-09-17T21:21:39.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.593e-06 2024-09-17T21:21:39.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.081 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:39.081 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878724+0000", 2024-09-17T21:21:39.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6313000000000001e-05 2024-09-17T21:21:39.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.082 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:39.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878744+0000", 2024-09-17T21:21:39.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9313e-05 2024-09-17T21:21:39.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.082 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.878758+0000", 2024-09-17T21:21:39.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4358999999999999e-05 2024-09-17T21:21:39.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.082 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:39.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881261+0000", 2024-09-17T21:21:39.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002503164 2024-09-17T21:21:39.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.083 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:39.083 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.881289+0000", 2024-09-17T21:21:39.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7968999999999999e-05 2024-09-17T21:21:39.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.083 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:39.083 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.897952+0000", 2024-09-17T21:21:39.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016662983999999999 2024-09-17T21:21:39.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.083 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:39.083 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.898006+0000", 2024-09-17T21:21:39.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3408000000000001e-05 2024-09-17T21:21:39.084 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.084 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:39.084 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.084 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4447.0:541 3.5 e30/23 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head v 30'8, mlcod=29'4)", 2024-09-17T21:21:39.084 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:39.084 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 196.17987490600001, 2024-09-17T21:21:39.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011891307, 2024-09-17T21:21:39.084 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:39.084 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:39.084 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:39.085 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:39.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:39.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:39.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:39.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887428+0000", 2024-09-17T21:21:39.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:39.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887429+0000", 2024-09-17T21:21:39.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.553e-06 2024-09-17T21:21:39.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:39.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887444+0000", 2024-09-17T21:21:39.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4609e-05 2024-09-17T21:21:39.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:39.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887461+0000", 2024-09-17T21:21:39.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6691e-05 2024-09-17T21:21:39.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.087 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.887472+0000", 2024-09-17T21:21:39.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1088e-05 2024-09-17T21:21:39.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.087 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:39.087 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891431+0000", 2024-09-17T21:21:39.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039593400000000004 2024-09-17T21:21:39.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.087 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:39.087 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.891463+0000", 2024-09-17T21:21:39.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2085999999999997e-05 2024-09-17T21:21:39.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.088 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:39.088 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899264+0000", 2024-09-17T21:21:39.088 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078011759999999999 2024-09-17T21:21:39.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.088 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:39.088 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:22.899319+0000", 2024-09-17T21:21:39.088 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4763999999999999e-05 2024-09-17T21:21:39.088 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.088 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:39.088 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.089 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4737.0:682 3.9 e39/23 3:94c4d4c6:::rbd_data.11528a47c62f.0000000000000000:head v 39'150, mlcod=36'148)", 2024-09-17T21:21:39.089 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:39.089 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 182.79478434200001, 2024-09-17T21:21:39.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014140061000000001, 2024-09-17T21:21:39.089 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:39.089 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:39.089 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:39.089 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:39.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:39.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:39.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:39.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272518+0000", 2024-09-17T21:21:39.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:39.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.272521+0000", 2024-09-17T21:21:39.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:21:39.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:39.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275674+0000", 2024-09-17T21:21:39.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031531189999999998 2024-09-17T21:21:39.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.091 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:39.091 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275709+0000", 2024-09-17T21:21:39.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4932999999999999e-05 2024-09-17T21:21:39.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.091 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.091 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275730+0000", 2024-09-17T21:21:39.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1597e-05 2024-09-17T21:21:39.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.091 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:39.091 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275931+0000", 2024-09-17T21:21:39.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020114199999999999 2024-09-17T21:21:39.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.092 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.092 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:39.092 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.275973+0000", 2024-09-17T21:21:39.092 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1656000000000002e-05 2024-09-17T21:21:39.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.092 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.092 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:39.092 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286511+0000", 2024-09-17T21:21:39.092 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010538142 2024-09-17T21:21:39.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.092 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.093 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:39.093 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:36.286658+0000", 2024-09-17T21:21:39.093 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014723199999999999 2024-09-17T21:21:39.093 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.093 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:39.093 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.093 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1780 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'195, mlcod=69'190)", 2024-09-17T21:21:39.093 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:39.093 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 145.816928738, 2024-09-17T21:21:39.093 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075761769999999999, 2024-09-17T21:21:39.093 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:39.093 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:39.094 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:39.094 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:39.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:39.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:39.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:39.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250374+0000", 2024-09-17T21:21:39.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:39.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250375+0000", 2024-09-17T21:21:39.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:21:39.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:39.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250383+0000", 2024-09-17T21:21:39.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7870000000000002e-06 2024-09-17T21:21:39.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:39.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250402+0000", 2024-09-17T21:21:39.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9346000000000001e-05 2024-09-17T21:21:39.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.096 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.096 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.250413+0000", 2024-09-17T21:21:39.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.039e-05 2024-09-17T21:21:39.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.096 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:39.096 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251353+0000", 2024-09-17T21:21:39.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000940091 2024-09-17T21:21:39.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.096 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:39.096 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.251407+0000", 2024-09-17T21:21:39.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4701999999999999e-05 2024-09-17T21:21:39.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:39.097 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257829+0000", 2024-09-17T21:21:39.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064212779999999999 2024-09-17T21:21:39.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:39.097 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.257950+0000", 2024-09-17T21:21:39.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121623 2024-09-17T21:21:39.097 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.097 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:39.098 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.098 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:1788 3.1 e69/42 3:836f7552:::rbd_object_map.115265216c1c:head v 69'198, mlcod=69'191)", 2024-09-17T21:21:39.098 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:39.098 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 145.813820298, 2024-09-17T21:21:39.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073690370000000002, 2024-09-17T21:21:39.098 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:39.098 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:39.098 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:39.098 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:39.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:39.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:39.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:39.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253482+0000", 2024-09-17T21:21:39.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:39.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253484+0000", 2024-09-17T21:21:39.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2470000000000001e-06 2024-09-17T21:21:39.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:39.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253492+0000", 2024-09-17T21:21:39.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7159999999999992e-06 2024-09-17T21:21:39.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:39.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253509+0000", 2024-09-17T21:21:39.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.666e-05 2024-09-17T21:21:39.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.253522+0000", 2024-09-17T21:21:39.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2643000000000001e-05 2024-09-17T21:21:39.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.101 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:39.101 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254853+0000", 2024-09-17T21:21:39.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331238 2024-09-17T21:21:39.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.101 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:39.101 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.254885+0000", 2024-09-17T21:21:39.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1562e-05 2024-09-17T21:21:39.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.102 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:39.102 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260788+0000", 2024-09-17T21:21:39.102 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059030749999999998 2024-09-17T21:21:39.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.102 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:39.102 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:13.260852+0000", 2024-09-17T21:21:39.102 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3895999999999999e-05 2024-09-17T21:21:39.102 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.102 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:39.102 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.103 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5746 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'477, mlcod=71'473)", 2024-09-17T21:21:39.103 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:39.103 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.80839273999999, 2024-09-17T21:21:39.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074182709999999997, 2024-09-17T21:21:39.103 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:39.103 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:39.103 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:39.103 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:39.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:39.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:39.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:39.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258910+0000", 2024-09-17T21:21:39.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:39.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258912+0000", 2024-09-17T21:21:39.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:21:39.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:39.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258926+0000", 2024-09-17T21:21:39.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4249e-05 2024-09-17T21:21:39.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:39.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258946+0000", 2024-09-17T21:21:39.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9879999999999999e-05 2024-09-17T21:21:39.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.258956+0000", 2024-09-17T21:21:39.106 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:21:39.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.106 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:39.106 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259380+0000", 2024-09-17T21:21:39.106 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042337599999999999 2024-09-17T21:21:39.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.106 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:39.106 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259412+0000", 2024-09-17T21:21:39.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2462000000000002e-05 2024-09-17T21:21:39.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:39.107 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266191+0000", 2024-09-17T21:21:39.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067793269999999999 2024-09-17T21:21:39.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:39.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266328+0000", 2024-09-17T21:21:39.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013690999999999999 2024-09-17T21:21:39.108 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.108 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:39.108 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.108 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5747 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'478, mlcod=71'473)", 2024-09-17T21:21:39.108 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:39.108 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.807686391, 2024-09-17T21:21:39.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068779440000000004, 2024-09-17T21:21:39.109 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:39.109 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:39.109 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:39.109 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:39.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:39.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:39.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:39.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259616+0000", 2024-09-17T21:21:39.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.110 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:39.110 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259618+0000", 2024-09-17T21:21:39.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6199999999999999e-06 2024-09-17T21:21:39.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.110 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:39.110 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259630+0000", 2024-09-17T21:21:39.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1884e-05 2024-09-17T21:21:39.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:39.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259657+0000", 2024-09-17T21:21:39.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7339e-05 2024-09-17T21:21:39.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.259677+0000", 2024-09-17T21:21:39.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0156000000000001e-05 2024-09-17T21:21:39.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:39.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260521+0000", 2024-09-17T21:21:39.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084312899999999997 2024-09-17T21:21:39.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:39.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.260585+0000", 2024-09-17T21:21:39.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4558000000000001e-05 2024-09-17T21:21:39.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:39.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266336+0000", 2024-09-17T21:21:39.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057505020000000002 2024-09-17T21:21:39.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:39.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.266494+0000", 2024-09-17T21:21:39.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158756 2024-09-17T21:21:39.113 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.113 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:39.113 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.113 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5751 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'480, mlcod=71'473)", 2024-09-17T21:21:39.113 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:39.113 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.806266986, 2024-09-17T21:21:39.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0075756299999999999, 2024-09-17T21:21:39.113 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:39.113 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:39.113 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:39.114 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:39.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:39.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:39.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:39.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261036+0000", 2024-09-17T21:21:39.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:39.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261037+0000", 2024-09-17T21:21:39.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:21:39.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:39.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261049+0000", 2024-09-17T21:21:39.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1063999999999999e-05 2024-09-17T21:21:39.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:39.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261072+0000", 2024-09-17T21:21:39.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3102e-05 2024-09-17T21:21:39.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261133+0000", 2024-09-17T21:21:39.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1276000000000001e-05 2024-09-17T21:21:39.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:39.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262666+0000", 2024-09-17T21:21:39.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015334319999999999 2024-09-17T21:21:39.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:39.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.262702+0000", 2024-09-17T21:21:39.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5281999999999999e-05 2024-09-17T21:21:39.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:39.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268527+0000", 2024-09-17T21:21:39.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058250389999999997 2024-09-17T21:21:39.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:39.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268611+0000", 2024-09-17T21:21:39.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4724999999999999e-05 2024-09-17T21:21:39.117 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.117 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:39.117 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.118 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4871.0:5756 3.1 e71/42 3:836f7552:::rbd_object_map.115265216c1c:head v 71'481, mlcod=71'473)", 2024-09-17T21:21:39.118 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:39.118 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.80540612999999, 2024-09-17T21:21:39.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006772386, 2024-09-17T21:21:39.118 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:39.118 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:39.118 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:39.118 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:39.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:39.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:39.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.119 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:39.119 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261897+0000", 2024-09-17T21:21:39.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.119 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:39.119 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261899+0000", 2024-09-17T21:21:39.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.903e-06 2024-09-17T21:21:39.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.119 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:39.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261912+0000", 2024-09-17T21:21:39.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-05 2024-09-17T21:21:39.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:39.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261938+0000", 2024-09-17T21:21:39.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6098999999999999e-05 2024-09-17T21:21:39.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.261955+0000", 2024-09-17T21:21:39.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.696e-05 2024-09-17T21:21:39.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:39.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263845+0000", 2024-09-17T21:21:39.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001890187 2024-09-17T21:21:39.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:39.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.263907+0000", 2024-09-17T21:21:39.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1600999999999995e-05 2024-09-17T21:21:39.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:39.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268615+0000", 2024-09-17T21:21:39.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047086070000000001 2024-09-17T21:21:39.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:39.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.268669+0000", 2024-09-17T21:21:39.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3588999999999998e-05 2024-09-17T21:21:39.122 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.122 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:39.122 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.122 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:468 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'94, mlcod=89'87)", 2024-09-17T21:21:39.122 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:39.122 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 119.381086448, 2024-09-17T21:21:39.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066528790000000004, 2024-09-17T21:21:39.122 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:39.123 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:39.123 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:39.123 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:39.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.123 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:39.123 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:39.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.123 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:39.123 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686216+0000", 2024-09-17T21:21:39.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.124 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:39.124 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686218+0000", 2024-09-17T21:21:39.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3060000000000001e-06 2024-09-17T21:21:39.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.124 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:39.124 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686256+0000", 2024-09-17T21:21:39.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8333e-05 2024-09-17T21:21:39.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.124 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:39.124 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686272+0000", 2024-09-17T21:21:39.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5699999999999999e-05 2024-09-17T21:21:39.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686283+0000", 2024-09-17T21:21:39.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0903e-05 2024-09-17T21:21:39.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:39.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686682+0000", 2024-09-17T21:21:39.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039935099999999999 2024-09-17T21:21:39.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:39.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.686785+0000", 2024-09-17T21:21:39.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010275199999999999 2024-09-17T21:21:39.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:39.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692816+0000", 2024-09-17T21:21:39.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060317579999999999 2024-09-17T21:21:39.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:39.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.692869+0000", 2024-09-17T21:21:39.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2775999999999998e-05 2024-09-17T21:21:39.126 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.126 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:39.126 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.127 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:755 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'127, mlcod=89'122)", 2024-09-17T21:21:39.127 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:39.127 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 119.244631234, 2024-09-17T21:21:39.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084014080000000008, 2024-09-17T21:21:39.127 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:39.127 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:39.127 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:39.127 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:39.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:39.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:39.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:39.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:39.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:39.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822672+0000", 2024-09-17T21:21:39.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0599999999999999e-07 2024-09-17T21:21:39.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:39.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822683+0000", 2024-09-17T21:21:39.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0674e-05 2024-09-17T21:21:39.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.129 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:39.129 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822698+0000", 2024-09-17T21:21:39.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5189e-05 2024-09-17T21:21:39.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.129 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.129 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822708+0000", 2024-09-17T21:21:39.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9119999999999993e-06 2024-09-17T21:21:39.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.129 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:39.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822748+0000", 2024-09-17T21:21:39.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9727000000000001e-05 2024-09-17T21:21:39.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:39.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.822759+0000", 2024-09-17T21:21:39.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0865e-05 2024-09-17T21:21:39.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:39.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831034+0000", 2024-09-17T21:21:39.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082750710000000002 2024-09-17T21:21:39.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:39.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.831073+0000", 2024-09-17T21:21:39.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9063999999999997e-05 2024-09-17T21:21:39.131 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.131 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:39.131 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.131 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:756 3.d e89/23 3:b21fbb9a:::rbd_object_map.1152a8ae2019:head v 89'128, mlcod=89'124)", 2024-09-17T21:21:39.131 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:39.131 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 119.24270860199999, 2024-09-17T21:21:39.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0076300270000000002, 2024-09-17T21:21:39.131 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:39.132 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:39.132 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:39.132 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:39.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:39.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:39.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:39.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824594+0000", 2024-09-17T21:21:39.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.133 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:39.133 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824595+0000", 2024-09-17T21:21:39.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-17T21:21:39.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.133 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:39.133 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824600+0000", 2024-09-17T21:21:39.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:21:39.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.133 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:39.133 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824612+0000", 2024-09-17T21:21:39.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.181e-05 2024-09-17T21:21:39.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824619+0000", 2024-09-17T21:21:39.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:21:39.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:39.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824671+0000", 2024-09-17T21:21:39.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2262999999999998e-05 2024-09-17T21:21:39.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:39.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.824695+0000", 2024-09-17T21:21:39.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3883000000000002e-05 2024-09-17T21:21:39.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:39.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832185+0000", 2024-09-17T21:21:39.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007490048 2024-09-17T21:21:39.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:39.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:39.832224+0000", 2024-09-17T21:21:39.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8957e-05 2024-09-17T21:21:39.135 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.135 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:39.136 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.136 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5472.0:2635 3.d e90/23 3:b3d1e920:::rbd_data.1152a8ae2019.000000000000001f:head v 90'252, mlcod=90'246)", 2024-09-17T21:21:39.136 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:39.136 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 118.22014953199999, 2024-09-17T21:21:39.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0077265190000000003, 2024-09-17T21:21:39.136 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:39.136 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:39.136 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:39.136 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:39.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:39.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:39.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.137 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:39.137 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847153+0000", 2024-09-17T21:21:39.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.137 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:39.137 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847155+0000", 2024-09-17T21:21:39.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.544e-06 2024-09-17T21:21:39.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.137 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:39.138 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847165+0000", 2024-09-17T21:21:39.138 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.043e-05 2024-09-17T21:21:39.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.138 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.138 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:39.138 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847189+0000", 2024-09-17T21:21:39.138 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3659000000000002e-05 2024-09-17T21:21:39.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.138 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.138 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.138 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847210+0000", 2024-09-17T21:21:39.138 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1163e-05 2024-09-17T21:21:39.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.138 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:39.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848856+0000", 2024-09-17T21:21:39.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001645521 2024-09-17T21:21:39.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:39.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.848905+0000", 2024-09-17T21:21:39.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9048999999999997e-05 2024-09-17T21:21:39.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:39.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854770+0000", 2024-09-17T21:21:39.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058652879999999998 2024-09-17T21:21:39.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.140 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:39.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.854880+0000", 2024-09-17T21:21:39.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010986500000000001 2024-09-17T21:21:39.140 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.140 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:39.140 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.140 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "MOSDPGPush(3.5 151/149 [PushOp(3:a57cb214:::rbd_data.115262b8e217.0000000000000009:1d, version: 103'698, data_included: [196608~24576,229376~24576,262144~24576,294912~131072,524288~98304,819200~8192,843776~16384,876544~16384,909312~16384,942080~8192,1146880~16384,1179648~16384,1212416~16384,1474560~98304,1794048~8192,1826816~8192,1859584~8192,2105344~24576,2138112~24576,2170880~24576,2646016~8192,2678784~8192,2711552~8192,3072000~8192,3104768~8192,3137536~106496], data_size: 761856, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:a57cb214:::rbd_data.115262b8e217.0000000000000009:1d@103'698, size: 3276800, copy_subset: [(0, 3276800)], clone_subset: {}, snapset: 1d=[]:{1d: [1d]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3276800, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T21:21:39.140 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:07.006386+0000", 2024-09-17T21:21:39.140 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 32.060917072000002, 2024-09-17T21:21:39.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082699420000000006, 2024-09-17T21:21:39.141 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:39.141 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:39.141 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:39.141 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:39.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:39.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.006386+0000", 2024-09-17T21:21:39.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:39.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.006386+0000", 2024-09-17T21:21:39.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:39.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.006388+0000", 2024-09-17T21:21:39.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.827e-06 2024-09-17T21:21:39.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:39.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.007671+0000", 2024-09-17T21:21:39.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0012836340000000001 2024-09-17T21:21:39.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:39.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.007737+0000", 2024-09-17T21:21:39.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5815000000000006e-05 2024-09-17T21:21:39.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.143 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.143 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.007782+0000", 2024-09-17T21:21:39.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5509000000000001e-05 2024-09-17T21:21:39.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.143 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:39.143 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.008023+0000", 2024-09-17T21:21:39.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000240184 2024-09-17T21:21:39.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:39.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.008065+0000", 2024-09-17T21:21:39.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1978999999999999e-05 2024-09-17T21:21:39.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:39.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:07.014656+0000", 2024-09-17T21:21:39.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065909940000000002 2024-09-17T21:21:39.144 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.144 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:39.144 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.144 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(osd.3.0:3834 3.8 e165/154 3:11ef155a:::rbd_data.11527217c540.000000000000003f:2b v 165'498, mlcod=152'495)", 2024-09-17T21:21:39.145 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:22.375046+0000", 2024-09-17T21:21:39.145 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 16.692256886999999, 2024-09-17T21:21:39.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010228170999999999, 2024-09-17T21:21:39.145 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:39.145 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:39.145 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:39.145 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:39.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:39.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375046+0000", 2024-09-17T21:21:39.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:39.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375046+0000", 2024-09-17T21:21:39.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.146 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:39.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375047+0000", 2024-09-17T21:21:39.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.373e-06 2024-09-17T21:21:39.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.146 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:39.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375085+0000", 2024-09-17T21:21:39.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7376999999999998e-05 2024-09-17T21:21:39.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.147 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:39.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375127+0000", 2024-09-17T21:21:39.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1941999999999999e-05 2024-09-17T21:21:39.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.147 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375150+0000", 2024-09-17T21:21:39.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3113e-05 2024-09-17T21:21:39.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.147 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:39.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375261+0000", 2024-09-17T21:21:39.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111402 2024-09-17T21:21:39.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:39.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.375287+0000", 2024-09-17T21:21:39.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5585e-05 2024-09-17T21:21:39.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:39.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.385224+0000", 2024-09-17T21:21:39.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0099369750000000007 2024-09-17T21:21:39.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:39.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:22.385274+0000", 2024-09-17T21:21:39.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0404000000000001e-05 2024-09-17T21:21:39.149 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.149 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:39.149 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.149 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.4434.0:25007 3.1e e168/166 3:7ce0ed33:::rbd_object_map.11528d8227e0:head v 168'728, mlcod=26'23)", 2024-09-17T21:21:39.149 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:25.445808+0000", 2024-09-17T21:21:39.149 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 13.621494374999999, 2024-09-17T21:21:39.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074791850000000002, 2024-09-17T21:21:39.150 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:39.150 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:39.150 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:39.150 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:39.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.150 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:39.150 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445808+0000", 2024-09-17T21:21:39.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.150 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:39.150 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445808+0000", 2024-09-17T21:21:39.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.151 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:39.151 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445809+0000", 2024-09-17T21:21:39.151 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0299999999999997e-07 2024-09-17T21:21:39.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.151 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:39.151 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445816+0000", 2024-09-17T21:21:39.151 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.392e-06 2024-09-17T21:21:39.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:39.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445828+0000", 2024-09-17T21:21:39.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2187e-05 2024-09-17T21:21:39.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445834+0000", 2024-09-17T21:21:39.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5710000000000003e-06 2024-09-17T21:21:39.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:39.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445896+0000", 2024-09-17T21:21:39.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2024000000000001e-05 2024-09-17T21:21:39.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:39.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.445919+0000", 2024-09-17T21:21:39.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2480000000000002e-05 2024-09-17T21:21:39.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:39.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.453224+0000", 2024-09-17T21:21:39.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073048729999999999 2024-09-17T21:21:39.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:39.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.453288+0000", 2024-09-17T21:21:39.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3755000000000002e-05 2024-09-17T21:21:39.154 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.154 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:39.154 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.154 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25377 3.8 3:1508d022:::rbd_data.11524f65d4ff.000000000000000a:head [delete] snapc 29=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-17T21:21:39.154 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:30.688594+0000", 2024-09-17T21:21:39.154 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 8.3787088369999996, 2024-09-17T21:21:39.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010688471, 2024-09-17T21:21:39.155 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:39.155 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:39.155 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:39.155 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:39.155 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:21:39.155 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:39.155 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25377 2024-09-17T21:21:39.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.155 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:39.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.155 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:39.156 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.688594+0000", 2024-09-17T21:21:39.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.156 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:39.156 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.688594+0000", 2024-09-17T21:21:39.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.156 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:39.156 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.688603+0000", 2024-09-17T21:21:39.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4469999999999995e-06 2024-09-17T21:21:39.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.157 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:39.157 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.688615+0000", 2024-09-17T21:21:39.157 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1515000000000001e-05 2024-09-17T21:21:39.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.157 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.157 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:39.157 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.688646+0000", 2024-09-17T21:21:39.157 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0936000000000003e-05 2024-09-17T21:21:39.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.157 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.157 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.157 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.688666+0000", 2024-09-17T21:21:39.157 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9729999999999999e-05 2024-09-17T21:21:39.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:39.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.688812+0000", 2024-09-17T21:21:39.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000146265 2024-09-17T21:21:39.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:39.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.689181+0000", 2024-09-17T21:21:39.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00036869300000000002 2024-09-17T21:21:39.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,5", 2024-09-17T21:21:39.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.689503+0000", 2024-09-17T21:21:39.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000322539 2024-09-17T21:21:39.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:39.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.691833+0000", 2024-09-17T21:21:39.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023295159999999998 2024-09-17T21:21:39.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:39.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.691880+0000", 2024-09-17T21:21:39.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7899000000000003e-05 2024-09-17T21:21:39.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:39.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.698812+0000", 2024-09-17T21:21:39.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0069317390000000001 2024-09-17T21:21:39.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:39.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.698953+0000", 2024-09-17T21:21:39.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014077100000000001 2024-09-17T21:21:39.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:39.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:30.699282+0000", 2024-09-17T21:21:39.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00032942099999999999 2024-09-17T21:21:39.160 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.161 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:39.161 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.161 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25642 3.1 3:80cb5a56:::rbd_object_map.1152634cc0c6.0000000000000027:head [call] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e178)", 2024-09-17T21:21:39.161 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:35.302708+0000", 2024-09-17T21:21:39.161 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.7645950849999998, 2024-09-17T21:21:39.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0818678390000001, 2024-09-17T21:21:39.161 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:39.161 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:39.161 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:39.161 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:39.161 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:21:39.161 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:39.162 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25642 2024-09-17T21:21:39.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.162 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:39.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:39.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.302708+0000", 2024-09-17T21:21:39.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:39.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.302708+0000", 2024-09-17T21:21:39.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:39.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.163 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:39.163 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.302712+0000", 2024-09-17T21:21:39.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9330000000000003e-06 2024-09-17T21:21:39.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.163 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:39.163 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.302730+0000", 2024-09-17T21:21:39.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8473000000000001e-05 2024-09-17T21:21:39.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.163 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:39.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.302770+0000", 2024-09-17T21:21:39.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9669000000000003e-05 2024-09-17T21:21:39.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.302855+0000", 2024-09-17T21:21:39.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4981999999999994e-05 2024-09-17T21:21:39.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:39.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.303034+0000", 2024-09-17T21:21:39.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017908400000000001 2024-09-17T21:21:39.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-17T21:21:39.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.303055+0000", 2024-09-17T21:21:39.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0713999999999999e-05 2024-09-17T21:21:39.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:39.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.313876+0000", 2024-09-17T21:21:39.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010821061 2024-09-17T21:21:39.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:39.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.313901+0000", 2024-09-17T21:21:39.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5242e-05 2024-09-17T21:21:39.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:39.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:36.383917+0000", 2024-09-17T21:21:39.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.070015843 2024-09-17T21:21:39.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:39.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:36.384204+0000", 2024-09-17T21:21:39.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028758700000000001 2024-09-17T21:21:39.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:39.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:39.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:39.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:36.384576+0000", 2024-09-17T21:21:39.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00037125100000000001 2024-09-17T21:21:39.167 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.167 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:39.167 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.167 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:39.167 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:39.167 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:39.169 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:21:39.169 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:39.163+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:39.270 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:39.267+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:39.297 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:39.297 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:39.297 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:39.297 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:39.312 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-17T21:21:39.316 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:21:39.326 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 0] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:21:39.327 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:21:39.327 INFO:tasks.thrashosds.thrasher:Removing osd 5, in_osds are: [1, 3, 4, 5, 0] 2024-09-17T21:21:39.327 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 5 2024-09-17T21:21:39.370 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:39.365+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:39.450 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:39.450 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:39.450 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:39.450 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:39.450 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:39.466 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:21:39.471 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:39.467+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:39.572 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:39.567+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:39.651 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:21:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 222.63205262599999, 2024-09-17T21:21:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:21:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:21:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:21:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:21:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:21:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:21:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:21:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:21:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:21:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:21:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:21:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:21:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:21:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:21:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:21:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:21:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:21:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:21:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:21:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:21:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:21:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:21:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:21:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:21:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:21:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:21:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:21:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:21:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:21:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:21:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:21:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 203.051888346, 2024-09-17T21:21:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:21:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:21:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:21:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:21:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:21:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:21:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:21:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:21:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:21:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:21:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:21:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:21:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:21:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:21:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:21:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:21:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:21:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:21:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:21:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:21:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:21:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:21:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:21:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:21:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:21:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:21:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 203.05179103399999, 2024-09-17T21:21:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:21:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:21:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:21:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:21:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:21:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:21:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:21:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:21:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:21:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:21:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:21:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:21:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:21:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:21:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:21:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:21:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:21:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:21:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:21:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:21:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:21:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:21:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:21:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:21:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:21:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:21:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:21:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:21:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:21:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 203.049704233, 2024-09-17T21:21:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:21:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:21:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:21:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:21:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:21:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:21:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:21:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:21:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:21:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:21:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:21:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:21:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:21:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:21:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:21:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:21:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:21:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:21:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:21:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:21:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:21:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:21:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:21:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:21:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:21:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:21:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:21:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:21:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:21:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:21:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:21:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:21:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:21:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:39.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:21:39.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:21:39.679 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.679 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:39.679 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.679 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:39.679 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:39.679 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 170.36580545199999, 2024-09-17T21:21:39.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:21:39.679 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:39.679 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:39.680 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:39.680 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:39.680 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:39.680 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:39.680 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:21:39.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.680 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:39.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:39.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:39.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:21:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:21:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:21:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:21:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:39.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:21:39.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:21:39.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:21:39.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:21:39.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:21:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:21:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:21:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:21:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:21:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:21:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:21:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:21:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:21:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:21:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:21:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:21:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:21:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:21:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 170.35671374099999, 2024-09-17T21:21:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:21:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:21:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:21:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:21:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:21:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:21:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:21:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:21:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:21:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:21:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:21:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:21:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:21:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:21:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:21:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:21:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:21:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:21:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:21:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:21:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 170.35654169, 2024-09-17T21:21:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:21:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:21:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:21:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:21:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:21:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:21:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:21:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:21:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:21:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:21:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:21:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:21:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:21:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:21:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:21:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:21:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:21:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:21:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:21:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:21:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 170.18160894299999, 2024-09-17T21:21:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:21:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:21:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:21:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:21:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:21:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:21:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:21:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:21:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:21:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:21:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:21:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:21:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:21:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:21:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:21:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:21:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:21:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:21:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:21:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:21:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 170.18152438600001, 2024-09-17T21:21:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:21:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:21:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:21:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:21:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:21:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:21:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:21:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:21:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:21:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:21:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:21:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:21:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:21:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:21:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:21:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:21:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:21:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:21:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:21:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 170.181458749, 2024-09-17T21:21:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:21:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:21:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:21:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:21:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:39.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:21:39.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:21:39.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:39.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:21:39.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:21:39.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:21:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:21:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:21:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:21:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:21:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:21:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:21:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:21:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:21:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:21:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:21:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:21:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.712 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:39.712 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.712 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:39.712 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:39.712 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 118.807883652, 2024-09-17T21:21:39.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:21:39.712 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:39.712 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:39.712 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:39.712 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:39.712 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:21:39.713 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:39.713 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:21:39.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.713 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:39.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:39.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:39.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:39.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:39.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:39.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:21:39.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:21:39.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:39.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:21:39.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:21:39.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:39.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:21:39.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:21:39.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:21:39.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:21:39.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:21:39.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:21:39.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:39.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:21:39.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:21:39.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:39.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:21:39.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:21:39.716 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.716 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:39.716 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 118.80776414, 2024-09-17T21:21:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:21:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:21:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:21:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:21:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:21:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:21:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:21:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:21:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:21:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:21:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:21:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:21:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:21:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:21:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:21:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:21:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:21:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:995 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 575488~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:21:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.214733150000001, 2024-09-17T21:21:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084214699000000004, 2024-09-17T21:21:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 995 2024-09-17T21:21:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:21:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431085+0000", 2024-09-17T21:21:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431086+0000", 2024-09-17T21:21:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7999999999999995e-07 2024-09-17T21:21:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431088+0000", 2024-09-17T21:21:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4729999999999999e-06 2024-09-17T21:21:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:39.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431126+0000", 2024-09-17T21:21:39.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7583999999999998e-05 2024-09-17T21:21:39.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431137+0000", 2024-09-17T21:21:39.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1768e-05 2024-09-17T21:21:39.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515003+0000", 2024-09-17T21:21:39.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.083865097 2024-09-17T21:21:39.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:39.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515115+0000", 2024-09-17T21:21:39.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112561 2024-09-17T21:21:39.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:39.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515300+0000", 2024-09-17T21:21:39.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018453600000000001 2024-09-17T21:21:39.725 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.725 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:39.725 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.726 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:996 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:39.726 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:39.726 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.214650769000002, 2024-09-17T21:21:39.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084493736999999999, 2024-09-17T21:21:39.726 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:39.726 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:39.726 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:39.726 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:39.726 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:39.726 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:39.726 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 996 2024-09-17T21:21:39.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.726 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:39.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:39.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:39.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:39.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:39.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:39.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431168+0000", 2024-09-17T21:21:39.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4e-07 2024-09-17T21:21:39.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:39.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431172+0000", 2024-09-17T21:21:39.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.608e-06 2024-09-17T21:21:39.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:39.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431180+0000", 2024-09-17T21:21:39.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5650000000000002e-06 2024-09-17T21:21:39.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431188+0000", 2024-09-17T21:21:39.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:21:39.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515354+0000", 2024-09-17T21:21:39.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084165977000000003 2024-09-17T21:21:39.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:39.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515451+0000", 2024-09-17T21:21:39.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6669000000000006e-05 2024-09-17T21:21:39.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:39.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515661+0000", 2024-09-17T21:21:39.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021021599999999999 2024-09-17T21:21:39.730 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.730 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:39.730 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.730 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:997 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 583680~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:39.730 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:39.730 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.214619373999994, 2024-09-17T21:21:39.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084772419000000002, 2024-09-17T21:21:39.730 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:39.730 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:39.731 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:39.731 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:39.731 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:39.731 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:39.731 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 997 2024-09-17T21:21:39.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.731 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:39.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:39.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:39.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:39.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:39.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:39.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:39.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0400000000000005e-07 2024-09-17T21:21:39.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:39.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431202+0000", 2024-09-17T21:21:39.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:21:39.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:39.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431208+0000", 2024-09-17T21:21:39.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4740000000000002e-06 2024-09-17T21:21:39.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431214+0000", 2024-09-17T21:21:39.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6550000000000001e-06 2024-09-17T21:21:39.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515741+0000", 2024-09-17T21:21:39.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084526904 2024-09-17T21:21:39.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:39.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515826+0000", 2024-09-17T21:21:39.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4643999999999998e-05 2024-09-17T21:21:39.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:39.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515971+0000", 2024-09-17T21:21:39.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145444 2024-09-17T21:21:39.734 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.734 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:39.735 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.735 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:998 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 587776~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:39.735 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:39.735 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.214596401999998, 2024-09-17T21:21:39.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085108565999999997, 2024-09-17T21:21:39.735 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:39.735 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:39.735 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:39.735 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:39.735 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:39.735 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:39.735 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 998 2024-09-17T21:21:39.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.736 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:39.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:39.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:39.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:39.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:39.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:39.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:39.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:21:39.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:39.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431224+0000", 2024-09-17T21:21:39.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:21:39.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:39.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431231+0000", 2024-09-17T21:21:39.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000002e-06 2024-09-17T21:21:39.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431236+0000", 2024-09-17T21:21:39.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.648e-06 2024-09-17T21:21:39.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516024+0000", 2024-09-17T21:21:39.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084787584999999999 2024-09-17T21:21:39.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:39.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516122+0000", 2024-09-17T21:21:39.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8076000000000001e-05 2024-09-17T21:21:39.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:39.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516330+0000", 2024-09-17T21:21:39.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020808 2024-09-17T21:21:39.739 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.739 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:39.739 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.739 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1000 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 591872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:39.739 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:39.739 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.214574287000005, 2024-09-17T21:21:39.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085389065, 2024-09-17T21:21:39.739 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:39.740 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:39.740 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:39.740 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:39.740 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:39.740 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:39.740 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1000 2024-09-17T21:21:39.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.740 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:39.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:39.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:39.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:39.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:39.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:39.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:39.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:21:39.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:39.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431246+0000", 2024-09-17T21:21:39.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1749999999999999e-06 2024-09-17T21:21:39.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:39.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431253+0000", 2024-09-17T21:21:39.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2790000000000004e-06 2024-09-17T21:21:39.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431258+0000", 2024-09-17T21:21:39.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3299999999999998e-06 2024-09-17T21:21:39.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516384+0000", 2024-09-17T21:21:39.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085125549999999994 2024-09-17T21:21:39.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:39.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516460+0000", 2024-09-17T21:21:39.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6359999999999994e-05 2024-09-17T21:21:39.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:39.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516633+0000", 2024-09-17T21:21:39.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017285900000000001 2024-09-17T21:21:39.744 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.744 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:39.744 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.744 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1001 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 595968~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:39.744 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:39.744 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.214550880999994, 2024-09-17T21:21:39.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085614715999999993, 2024-09-17T21:21:39.744 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:39.745 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:39.745 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:39.745 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:39.745 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:39.745 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:39.745 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1001 2024-09-17T21:21:39.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.745 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:39.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:39.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:39.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:39.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:39.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:39.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438066+0000", 2024-09-17T21:21:39.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067989490000000003 2024-09-17T21:21:39.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:39.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438073+0000", 2024-09-17T21:21:39.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8419999999999999e-06 2024-09-17T21:21:39.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:39.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438090+0000", 2024-09-17T21:21:39.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6657e-05 2024-09-17T21:21:39.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438099+0000", 2024-09-17T21:21:39.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0529999999999996e-06 2024-09-17T21:21:39.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516678+0000", 2024-09-17T21:21:39.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078579422999999995 2024-09-17T21:21:39.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:39.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516739+0000", 2024-09-17T21:21:39.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1315000000000005e-05 2024-09-17T21:21:39.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:39.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516882+0000", 2024-09-17T21:21:39.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014247700000000001 2024-09-17T21:21:39.749 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.749 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:39.749 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.749 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:19998 3.5 3:a4885450:::rbd_object_map.1152d139f9f9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-17T21:21:39.749 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:39.749 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 60.243283636000001, 2024-09-17T21:21:39.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90715489299999996, 2024-09-17T21:21:39.749 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:39.749 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:39.750 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:39.750 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:39.750 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:39.750 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:39.750 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19998 2024-09-17T21:21:39.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.750 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:39.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:39.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:39.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:39.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:39.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:39.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402537+0000", 2024-09-17T21:21:39.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5600000000000001e-06 2024-09-17T21:21:39.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:39.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402549+0000", 2024-09-17T21:21:39.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1929999999999999e-05 2024-09-17T21:21:39.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:39.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402579+0000", 2024-09-17T21:21:39.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0093000000000001e-05 2024-09-17T21:21:39.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402595+0000", 2024-09-17T21:21:39.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5855999999999999e-05 2024-09-17T21:21:39.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402731+0000", 2024-09-17T21:21:39.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013610799999999999 2024-09-17T21:21:39.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:39.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402750+0000", 2024-09-17T21:21:39.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8737e-05 2024-09-17T21:21:39.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.305977+0000", 2024-09-17T21:21:39.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90322768399999998 2024-09-17T21:21:39.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:39.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306236+0000", 2024-09-17T21:21:39.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000258111 2024-09-17T21:21:39.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:39.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306557+0000", 2024-09-17T21:21:39.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032159799999999998 2024-09-17T21:21:39.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:39.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.308286+0000", 2024-09-17T21:21:39.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017292150000000001 2024-09-17T21:21:39.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:39.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309530+0000", 2024-09-17T21:21:39.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00124409 2024-09-17T21:21:39.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:39.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309630+0000", 2024-09-17T21:21:39.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9689e-05 2024-09-17T21:21:39.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:39.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309689+0000", 2024-09-17T21:21:39.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9222000000000003e-05 2024-09-17T21:21:39.755 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.755 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:39.755 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.756 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:25277 3.1f 3:ff5d7cd5:::rbd_header.11524f65d4ff:head [watch ping cookie 139860845478480 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-17T21:21:39.756 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:21:39.756 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.060327567, 2024-09-17T21:21:39.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.92185720000000004, 2024-09-17T21:21:39.756 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:39.756 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:39.756 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:39.756 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:39.756 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:39.756 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:39.756 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 25277 2024-09-17T21:21:39.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.757 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:39.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:39.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:21:39.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:39.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:21:39.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:39.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:39.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585493+0000", 2024-09-17T21:21:39.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:21:39.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:39.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585505+0000", 2024-09-17T21:21:39.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2466000000000001e-05 2024-09-17T21:21:39.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:39.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585537+0000", 2024-09-17T21:21:39.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1183000000000003e-05 2024-09-17T21:21:39.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:39.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585553+0000", 2024-09-17T21:21:39.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.662e-05 2024-09-17T21:21:39.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585696+0000", 2024-09-17T21:21:39.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014293800000000001 2024-09-17T21:21:39.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:39.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585718+0000", 2024-09-17T21:21:39.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1860000000000001e-05 2024-09-17T21:21:39.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.501188+0000", 2024-09-17T21:21:39.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.91547051300000004 2024-09-17T21:21:39.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:21:39.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.501260+0000", 2024-09-17T21:21:39.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1571999999999998e-05 2024-09-17T21:21:39.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:39.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507002+0000", 2024-09-17T21:21:39.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057414850000000002 2024-09-17T21:21:39.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:39.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507143+0000", 2024-09-17T21:21:39.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014140299999999999 2024-09-17T21:21:39.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:39.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:39.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:39.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507348+0000", 2024-09-17T21:21:39.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000204773 2024-09-17T21:21:39.761 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.761 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:39.761 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.761 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:39.761 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:39.762 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:39.763 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-09-17T21:21:39.671+0000 7fe37768f640 -1 mon.a@0(leader).osd e182 definitely_dead 0 2024-09-17T21:21:39.764 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:21:39.764 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:39.761+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:39.864 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:39.861+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:39.947 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:39.947 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:39.947 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:39.947 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:39.961 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:21:40.003 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:40.001+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:40.066 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:40.061+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:40.126 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:40.127 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:40.141 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:21:40.167 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:40.163+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:40.268 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:40.265+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:40.311 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:40.311 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:40.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:40.311 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:40.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.311 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:18003 3.1d 3:b9765580:::rbd_header.11527d2b15f6:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T21:21:40.312 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:40.312 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 87.949711233000002, 2024-09-17T21:21:40.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94165346900000002, 2024-09-17T21:21:40.312 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.312 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.312 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.312 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.312 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:40.312 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:40.312 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18003 2024-09-17T21:21:40.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.312 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:40.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:40.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357880+0000", 2024-09-17T21:21:40.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:21:40.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357891+0000", 2024-09-17T21:21:40.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0417999999999999e-05 2024-09-17T21:21:40.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357913+0000", 2024-09-17T21:21:40.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2215e-05 2024-09-17T21:21:40.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357922+0000", 2024-09-17T21:21:40.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4779999999999997e-06 2024-09-17T21:21:40.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358023+0000", 2024-09-17T21:21:40.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101025 2024-09-17T21:21:40.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:40.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358040+0000", 2024-09-17T21:21:40.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6291999999999999e-05 2024-09-17T21:21:40.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298203+0000", 2024-09-17T21:21:40.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94016297500000001 2024-09-17T21:21:40.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298626+0000", 2024-09-17T21:21:40.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000422853 2024-09-17T21:21:40.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.299532+0000", 2024-09-17T21:21:40.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00090657300000000004 2024-09-17T21:21:40.317 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.317 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.317 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.317 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15706 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 977920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:40.317 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:21:40.317 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.786609442, 2024-09-17T21:21:40.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091952328, 2024-09-17T21:21:40.318 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.318 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.318 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.318 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.318 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:40.318 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:40.318 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15706 2024-09-17T21:21:40.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.318 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:21:40.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:21:40.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520982+0000", 2024-09-17T21:21:40.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.277e-06 2024-09-17T21:21:40.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520988+0000", 2024-09-17T21:21:40.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4470000000000003e-06 2024-09-17T21:21:40.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521008+0000", 2024-09-17T21:21:40.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9309000000000001e-05 2024-09-17T21:21:40.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521023+0000", 2024-09-17T21:21:40.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5119999999999999e-05 2024-09-17T21:21:40.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612638+0000", 2024-09-17T21:21:40.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091615328999999995 2024-09-17T21:21:40.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612738+0000", 2024-09-17T21:21:40.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100466 2024-09-17T21:21:40.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612933+0000", 2024-09-17T21:21:40.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019437999999999999 2024-09-17T21:21:40.321 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.321 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.321 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.322 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15707 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 982016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:40.322 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:40.322 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.786545985999993, 2024-09-17T21:21:40.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092305228000000003, 2024-09-17T21:21:40.322 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.322 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.322 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.322 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.322 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:40.322 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:40.322 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15707 2024-09-17T21:21:40.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.322 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:40.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:40.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521045+0000", 2024-09-17T21:21:40.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4899999999999999e-06 2024-09-17T21:21:40.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521066+0000", 2024-09-17T21:21:40.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0236000000000001e-05 2024-09-17T21:21:40.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521085+0000", 2024-09-17T21:21:40.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8819999999999999e-05 2024-09-17T21:21:40.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521100+0000", 2024-09-17T21:21:40.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5146e-05 2024-09-17T21:21:40.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612978+0000", 2024-09-17T21:21:40.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091878847 2024-09-17T21:21:40.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613079+0000", 2024-09-17T21:21:40.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100503 2024-09-17T21:21:40.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613349+0000", 2024-09-17T21:21:40.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000270186 2024-09-17T21:21:40.326 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.326 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.326 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.326 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15708 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 986112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:40.326 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:40.326 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.786467810999994, 2024-09-17T21:21:40.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092547217000000001, 2024-09-17T21:21:40.326 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.327 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.327 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.327 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.327 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:40.327 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:40.327 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15708 2024-09-17T21:21:40.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.327 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:40.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:40.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521123+0000", 2024-09-17T21:21:40.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.353e-06 2024-09-17T21:21:40.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521129+0000", 2024-09-17T21:21:40.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:21:40.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521145+0000", 2024-09-17T21:21:40.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.611e-05 2024-09-17T21:21:40.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521154+0000", 2024-09-17T21:21:40.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4500000000000004e-06 2024-09-17T21:21:40.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613397+0000", 2024-09-17T21:21:40.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092243275 2024-09-17T21:21:40.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613503+0000", 2024-09-17T21:21:40.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106013 2024-09-17T21:21:40.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613669+0000", 2024-09-17T21:21:40.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016642299999999999 2024-09-17T21:21:40.331 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.331 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.331 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.331 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15709 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 990208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:40.331 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:40.331 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.786424795000002, 2024-09-17T21:21:40.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092728678999999994, 2024-09-17T21:21:40.331 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.331 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.332 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:40.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:40.332 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15709 2024-09-17T21:21:40.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.332 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:40.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:40.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521166+0000", 2024-09-17T21:21:40.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7000000000000004e-07 2024-09-17T21:21:40.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521169+0000", 2024-09-17T21:21:40.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2660000000000002e-06 2024-09-17T21:21:40.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521181+0000", 2024-09-17T21:21:40.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.208e-05 2024-09-17T21:21:40.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521194+0000", 2024-09-17T21:21:40.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2707e-05 2024-09-17T21:21:40.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613695+0000", 2024-09-17T21:21:40.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092500709 2024-09-17T21:21:40.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613750+0000", 2024-09-17T21:21:40.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.503e-05 2024-09-17T21:21:40.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613894+0000", 2024-09-17T21:21:40.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144117 2024-09-17T21:21:40.335 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.335 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.335 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.336 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15710 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 994304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:40.336 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:40.336 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.786380229000002, 2024-09-17T21:21:40.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092957729000000003, 2024-09-17T21:21:40.336 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.336 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.336 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.336 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.336 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:40.336 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:40.336 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15710 2024-09-17T21:21:40.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.337 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:40.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:40.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:40.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8000000000000005e-07 2024-09-17T21:21:40.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521216+0000", 2024-09-17T21:21:40.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3859999999999998e-06 2024-09-17T21:21:40.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521234+0000", 2024-09-17T21:21:40.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:21:40.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521248+0000", 2024-09-17T21:21:40.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:21:40.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613926+0000", 2024-09-17T21:21:40.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092678324000000006 2024-09-17T21:21:40.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613974+0000", 2024-09-17T21:21:40.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7754999999999999e-05 2024-09-17T21:21:40.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614167+0000", 2024-09-17T21:21:40.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193508 2024-09-17T21:21:40.340 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.340 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.340 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.340 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15711 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 998400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:40.340 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:40.340 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.786322846999994, 2024-09-17T21:21:40.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093268000000000004, 2024-09-17T21:21:40.341 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.341 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.341 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.341 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.341 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:40.341 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:40.341 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15711 2024-09-17T21:21:40.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.341 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521268+0000", 2024-09-17T21:21:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-17T21:21:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521273+0000", 2024-09-17T21:21:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:21:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521290+0000", 2024-09-17T21:21:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7269999999999999e-05 2024-09-17T21:21:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521304+0000", 2024-09-17T21:21:40.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3626e-05 2024-09-17T21:21:40.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614220+0000", 2024-09-17T21:21:40.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092916367 2024-09-17T21:21:40.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614322+0000", 2024-09-17T21:21:40.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102212 2024-09-17T21:21:40.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614535+0000", 2024-09-17T21:21:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021266199999999999 2024-09-17T21:21:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15712 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1002496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.786267351999996, 2024-09-17T21:21:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093568707000000001, 2024-09-17T21:21:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15712 2024-09-17T21:21:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521324+0000", 2024-09-17T21:21:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.17e-06 2024-09-17T21:21:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521334+0000", 2024-09-17T21:21:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9369999999999999e-06 2024-09-17T21:21:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521356+0000", 2024-09-17T21:21:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2045999999999999e-05 2024-09-17T21:21:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521370+0000", 2024-09-17T21:21:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4270000000000001e-05 2024-09-17T21:21:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614583+0000", 2024-09-17T21:21:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093213220999999999 2024-09-17T21:21:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614680+0000", 2024-09-17T21:21:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6900999999999999e-05 2024-09-17T21:21:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614891+0000", 2024-09-17T21:21:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021116200000000001 2024-09-17T21:21:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15713 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1006592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.786200649999998, 2024-09-17T21:21:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093834641999999996, 2024-09-17T21:21:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15713 2024-09-17T21:21:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521390+0000", 2024-09-17T21:21:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:21:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521396+0000", 2024-09-17T21:21:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:21:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521417+0000", 2024-09-17T21:21:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1236000000000001e-05 2024-09-17T21:21:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521431+0000", 2024-09-17T21:21:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3873000000000001e-05 2024-09-17T21:21:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614938+0000", 2024-09-17T21:21:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093506522999999994 2024-09-17T21:21:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615025+0000", 2024-09-17T21:21:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7635999999999999e-05 2024-09-17T21:21:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615224+0000", 2024-09-17T21:21:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019844700000000001 2024-09-17T21:21:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.354 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.354 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.354 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15714 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:40.354 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:40.354 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.786137393999994, 2024-09-17T21:21:40.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094071925000000001, 2024-09-17T21:21:40.354 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.354 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.354 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.354 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.354 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:40.354 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:40.354 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15714 2024-09-17T21:21:40.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.355 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:40.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:40.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521454+0000", 2024-09-17T21:21:40.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.097e-06 2024-09-17T21:21:40.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521459+0000", 2024-09-17T21:21:40.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2660000000000001e-06 2024-09-17T21:21:40.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521481+0000", 2024-09-17T21:21:40.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1878999999999998e-05 2024-09-17T21:21:40.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521498+0000", 2024-09-17T21:21:40.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6807e-05 2024-09-17T21:21:40.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615271+0000", 2024-09-17T21:21:40.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093773286999999997 2024-09-17T21:21:40.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615366+0000", 2024-09-17T21:21:40.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5322000000000007e-05 2024-09-17T21:21:40.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615524+0000", 2024-09-17T21:21:40.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015826699999999999 2024-09-17T21:21:40.358 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.358 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.358 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.358 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15715 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1014784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:40.358 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:40.358 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.786073059000003, 2024-09-17T21:21:40.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094243840999999995, 2024-09-17T21:21:40.359 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.359 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.359 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.359 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.359 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:40.359 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:40.359 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15715 2024-09-17T21:21:40.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.359 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:40.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:40.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521518+0000", 2024-09-17T21:21:40.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4300000000000001e-06 2024-09-17T21:21:40.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521524+0000", 2024-09-17T21:21:40.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5860000000000004e-06 2024-09-17T21:21:40.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521541+0000", 2024-09-17T21:21:40.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7574000000000001e-05 2024-09-17T21:21:40.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521555+0000", 2024-09-17T21:21:40.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3540000000000001e-05 2024-09-17T21:21:40.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615560+0000", 2024-09-17T21:21:40.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.09400472 2024-09-17T21:21:40.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615614+0000", 2024-09-17T21:21:40.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4693e-05 2024-09-17T21:21:40.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615761+0000", 2024-09-17T21:21:40.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000146298 2024-09-17T21:21:40.362 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.363 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.363 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.363 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15716 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1018880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:40.363 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:40.363 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.786016713999999, 2024-09-17T21:21:40.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094400676000000003, 2024-09-17T21:21:40.363 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.363 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.363 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.363 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.363 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:40.363 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:40.363 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15716 2024-09-17T21:21:40.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.364 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:40.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:40.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521574+0000", 2024-09-17T21:21:40.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1939999999999999e-06 2024-09-17T21:21:40.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521579+0000", 2024-09-17T21:21:40.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:21:40.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521597+0000", 2024-09-17T21:21:40.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8074e-05 2024-09-17T21:21:40.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521613+0000", 2024-09-17T21:21:40.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5316e-05 2024-09-17T21:21:40.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615780+0000", 2024-09-17T21:21:40.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094166874999999997 2024-09-17T21:21:40.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615836+0000", 2024-09-17T21:21:40.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6926999999999999e-05 2024-09-17T21:21:40.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615974+0000", 2024-09-17T21:21:40.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013741400000000001 2024-09-17T21:21:40.367 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.367 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.367 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.367 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15717 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1022976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:40.367 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:40.367 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.785958018000002, 2024-09-17T21:21:40.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094586723999999997, 2024-09-17T21:21:40.368 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.368 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.368 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.368 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.368 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:40.368 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:40.368 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15717 2024-09-17T21:21:40.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.368 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:40.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:40.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521633+0000", 2024-09-17T21:21:40.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:21:40.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521638+0000", 2024-09-17T21:21:40.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:21:40.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521659+0000", 2024-09-17T21:21:40.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-05 2024-09-17T21:21:40.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521675+0000", 2024-09-17T21:21:40.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5688999999999998e-05 2024-09-17T21:21:40.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616008+0000", 2024-09-17T21:21:40.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094333314000000001 2024-09-17T21:21:40.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616066+0000", 2024-09-17T21:21:40.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7797000000000002e-05 2024-09-17T21:21:40.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616219+0000", 2024-09-17T21:21:40.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015281100000000001 2024-09-17T21:21:40.372 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.372 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.372 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.373 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15718 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:40.373 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:40.373 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.785857942999996, 2024-09-17T21:21:40.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094703674000000002, 2024-09-17T21:21:40.373 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.373 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.373 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.373 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.373 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:40.373 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:40.373 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15718 2024-09-17T21:21:40.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521734+0000", 2024-09-17T21:21:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.61e-06 2024-09-17T21:21:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521740+0000", 2024-09-17T21:21:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:21:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521754+0000", 2024-09-17T21:21:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3793e-05 2024-09-17T21:21:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521763+0000", 2024-09-17T21:21:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5129999999999996e-06 2024-09-17T21:21:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616256+0000", 2024-09-17T21:21:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094492919999999994 2024-09-17T21:21:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616312+0000", 2024-09-17T21:21:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6737000000000002e-05 2024-09-17T21:21:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616436+0000", 2024-09-17T21:21:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012335800000000001 2024-09-17T21:21:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15719 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1031168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.785804537999994, 2024-09-17T21:21:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094888963000000007, 2024-09-17T21:21:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15719 2024-09-17T21:21:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521786+0000", 2024-09-17T21:21:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.074e-06 2024-09-17T21:21:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521791+0000", 2024-09-17T21:21:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3030000000000001e-06 2024-09-17T21:21:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521804+0000", 2024-09-17T21:21:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3027000000000001e-05 2024-09-17T21:21:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521811+0000", 2024-09-17T21:21:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6090000000000003e-06 2024-09-17T21:21:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616466+0000", 2024-09-17T21:21:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094654294 2024-09-17T21:21:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616523+0000", 2024-09-17T21:21:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6953999999999997e-05 2024-09-17T21:21:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616674+0000", 2024-09-17T21:21:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015170200000000001 2024-09-17T21:21:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15720 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1035264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.785767698000001, 2024-09-17T21:21:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095067061999999994, 2024-09-17T21:21:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15720 2024-09-17T21:21:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521823+0000", 2024-09-17T21:21:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:21:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521826+0000", 2024-09-17T21:21:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2100000000000002e-06 2024-09-17T21:21:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521835+0000", 2024-09-17T21:21:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1630000000000002e-06 2024-09-17T21:21:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521842+0000", 2024-09-17T21:21:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:21:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616711+0000", 2024-09-17T21:21:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094868859 2024-09-17T21:21:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616764+0000", 2024-09-17T21:21:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2778999999999997e-05 2024-09-17T21:21:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616889+0000", 2024-09-17T21:21:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012533100000000001 2024-09-17T21:21:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15721 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1039360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.785720588999993, 2024-09-17T21:21:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095277219999999996, 2024-09-17T21:21:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15721 2024-09-17T21:21:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521871+0000", 2024-09-17T21:21:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.443e-06 2024-09-17T21:21:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521876+0000", 2024-09-17T21:21:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0300000000000001e-06 2024-09-17T21:21:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521886+0000", 2024-09-17T21:21:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186e-05 2024-09-17T21:21:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521893+0000", 2024-09-17T21:21:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.08e-06 2024-09-17T21:21:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616918+0000", 2024-09-17T21:21:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095025223000000006 2024-09-17T21:21:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616977+0000", 2024-09-17T21:21:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8403999999999999e-05 2024-09-17T21:21:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617147+0000", 2024-09-17T21:21:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016985399999999999 2024-09-17T21:21:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15722 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.785686139999996, 2024-09-17T21:21:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095459783000000006, 2024-09-17T21:21:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15722 2024-09-17T21:21:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7700000000000004e-07 2024-09-17T21:21:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521907+0000", 2024-09-17T21:21:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.813e-06 2024-09-17T21:21:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521922+0000", 2024-09-17T21:21:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4533e-05 2024-09-17T21:21:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521934+0000", 2024-09-17T21:21:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2469999999999999e-05 2024-09-17T21:21:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617187+0000", 2024-09-17T21:21:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095252311000000006 2024-09-17T21:21:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617240+0000", 2024-09-17T21:21:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3740999999999999e-05 2024-09-17T21:21:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617364+0000", 2024-09-17T21:21:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-17T21:21:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15723 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.785642273999997, 2024-09-17T21:21:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095653563999999996, 2024-09-17T21:21:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15723 2024-09-17T21:21:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521949+0000", 2024-09-17T21:21:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:21:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521953+0000", 2024-09-17T21:21:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5870000000000003e-06 2024-09-17T21:21:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521970+0000", 2024-09-17T21:21:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7282000000000002e-05 2024-09-17T21:21:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521984+0000", 2024-09-17T21:21:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:21:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617383+0000", 2024-09-17T21:21:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095399063000000006 2024-09-17T21:21:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617437+0000", 2024-09-17T21:21:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3931999999999998e-05 2024-09-17T21:21:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617601+0000", 2024-09-17T21:21:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016407300000000001 2024-09-17T21:21:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5629.0:2384 3.1c 3:38c1263e:::rbd_data.1152d37183c3.000000000000000b:head [read 11780096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-17T21:21:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.882811443999998, 2024-09-17T21:21:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0246185320000001, 2024-09-17T21:21:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5629", 2024-09-17T21:21:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3798711329", 2024-09-17T21:21:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2384 2024-09-17T21:21:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424780+0000", 2024-09-17T21:21:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.13e-06 2024-09-17T21:21:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424785+0000", 2024-09-17T21:21:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4739999999999996e-06 2024-09-17T21:21:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424798+0000", 2024-09-17T21:21:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2704000000000001e-05 2024-09-17T21:21:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424806+0000", 2024-09-17T21:21:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9899999999999997e-06 2024-09-17T21:21:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424868+0000", 2024-09-17T21:21:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999997e-05 2024-09-17T21:21:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424875+0000", 2024-09-17T21:21:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0870000000000001e-06 2024-09-17T21:21:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428897+0000", 2024-09-17T21:21:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040226080000000004 2024-09-17T21:21:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428913+0000", 2024-09-17T21:21:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:21:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.448963+0000", 2024-09-17T21:21:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0200498659999999 2024-09-17T21:21:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449158+0000", 2024-09-17T21:21:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019581000000000001 2024-09-17T21:21:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449397+0000", 2024-09-17T21:21:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023871 2024-09-17T21:21:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.408 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.408 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.408 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.408 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.408 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:marked out osd.5. 2024-09-17T21:21:40.410 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:21:40.411 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:40.407+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:40.510 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:40.505+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:40.577 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:40.577 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:40.577 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:40.577 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:40.591 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:21:40.611 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:40.607+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:40.712 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:40.709+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:40.759 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:40.759 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:40.773 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:21:40.813 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:40.811+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:40.879 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:21:40.914 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:40.909+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:40.939 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:21:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 199.33123866400001, 2024-09-17T21:21:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:21:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:21:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:21:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:21:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:21:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:21:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:21:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:21:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:21:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:21:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:21:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:21:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:21:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:21:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:21:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:21:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:21:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:21:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 199.32129202799999, 2024-09-17T21:21:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:21:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:21:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:21:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:21:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:21:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:21:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:21:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:21:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:21:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:21:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:21:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:21:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:21:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:21:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:21:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:21:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:21:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:21:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 198.07814118799999, 2024-09-17T21:21:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:21:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:21:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:21:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:21:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:21:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:21:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:21:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:21:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:21:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:21:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:21:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:21:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:21:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:21:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:21:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:21:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:21:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:21:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:21:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:21:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:21:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:21:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:21:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:21:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:21:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:21:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:21:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 198.06849570200001, 2024-09-17T21:21:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:21:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:21:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:21:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:21:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:21:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:21:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:21:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:21:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:21:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:21:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:21:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:21:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:21:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:21:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:21:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:21:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:21:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:21:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:21:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:21:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:21:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:21:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:21:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:21:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:21:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:21:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:21:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 184.699559669, 2024-09-17T21:21:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:21:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:21:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:21:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:21:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:21:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:21:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:21:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:21:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:21:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:21:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:21:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:21:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:21:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:21:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:21:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:21:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:21:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:21:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 166.497875107, 2024-09-17T21:21:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:21:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:21:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:21:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:21:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:21:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:21:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:21:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:21:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:21:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:21:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:21:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:21:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:21:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:21:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:21:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:21:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 166.49738052999999, 2024-09-17T21:21:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:21:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:21:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:21:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:21:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:21:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:21:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:21:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:21:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:21:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:21:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:21:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:21:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:21:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:21:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:21:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:21:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 166.49732514900001, 2024-09-17T21:21:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:21:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:21:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:21:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:21:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:21:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:21:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:21:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:21:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:21:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:21:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:21:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:21:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:21:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:21:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:21:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:21:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 166.49726887099999, 2024-09-17T21:21:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:21:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:21:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:21:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:21:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:21:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:21:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:21:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:21:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:21:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:21:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:21:40.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:21:40.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:21:40.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:21:40.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:21:40.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:21:40.985 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 166.49631923199999, 2024-09-17T21:21:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:21:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:21:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:21:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:21:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:21:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:21:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:21:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:21:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:21:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:21:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:21:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:21:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:21:40.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:21:40.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:21:40.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:21:40.990 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.990 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.990 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.990 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 166.49624061200001, 2024-09-17T21:21:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:21:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:21:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:21:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:21:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:21:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:21:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:21:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:21:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:21:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:21:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:21:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:21:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:21:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:21:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:21:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:21:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 166.49612767299999, 2024-09-17T21:21:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:21:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:21:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:21:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:21:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:21:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:21:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:21:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:21:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:21:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:21:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:21:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:21:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:21:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:21:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:21:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:21:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 166.49546661900001, 2024-09-17T21:21:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:21:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:21:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:21:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:21:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:21:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:21:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:21:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:21:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:21:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:21:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:21:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:21:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:21:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:21:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:21:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:21:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 166.495401808, 2024-09-17T21:21:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:21:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:21:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:21:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:21:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:21:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:21:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:21:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:21:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:21:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:21:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:21:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:21:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:21:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:21:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:21:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:21:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 166.49534083099999, 2024-09-17T21:21:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:21:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:21:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:21:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:21:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:21:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:21:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:21:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:21:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:21:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:21:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:21:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:21:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:21:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:21:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:21:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:21:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:21:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.597769646, 2024-09-17T21:21:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:21:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:21:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:21:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:21:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:21:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:21:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:21:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:21:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:21:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:21:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:21:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:21:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:21:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:21:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:21:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:21:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:21:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:21:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.38911125200001, 2024-09-17T21:21:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:21:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:21:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:21:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:21:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:21:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:21:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:21:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:21:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:21:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:21:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:21:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:21:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:21:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:21:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:21:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:21:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:21:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:9547 3.2 e147/132 3:42b518d9:::rbd_data.1152f056b73f.0000000000000001:head v 147'513, mlcod=142'511)", 2024-09-17T21:21:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.464138130000002, 2024-09-17T21:21:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.031059566, 2024-09-17T21:21:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471293+0000", 2024-09-17T21:21:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2029999999999999e-06 2024-09-17T21:21:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480606+0000", 2024-09-17T21:21:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093123839999999999 2024-09-17T21:21:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480653+0000", 2024-09-17T21:21:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7852000000000002e-05 2024-09-17T21:21:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480673+0000", 2024-09-17T21:21:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0013e-05 2024-09-17T21:21:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480862+0000", 2024-09-17T21:21:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018833599999999999 2024-09-17T21:21:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480906+0000", 2024-09-17T21:21:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4162000000000002e-05 2024-09-17T21:21:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502259+0000", 2024-09-17T21:21:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021353154999999999 2024-09-17T21:21:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502351+0000", 2024-09-17T21:21:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1460999999999997e-05 2024-09-17T21:21:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:15234 3.1 e147/134 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head v 147'917, mlcod=145'914)", 2024-09-17T21:21:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 37.831477511999999, 2024-09-17T21:21:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045303695999999997, 2024-09-17T21:21:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103954+0000", 2024-09-17T21:21:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2259999999999999e-06 2024-09-17T21:21:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119105+0000", 2024-09-17T21:21:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015150871 2024-09-17T21:21:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119141+0000", 2024-09-17T21:21:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6239000000000003e-05 2024-09-17T21:21:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119159+0000", 2024-09-17T21:21:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7946999999999999e-05 2024-09-17T21:21:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119344+0000", 2024-09-17T21:21:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018524599999999999 2024-09-17T21:21:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119399+0000", 2024-09-17T21:21:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4922e-05 2024-09-17T21:21:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149165+0000", 2024-09-17T21:21:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029765697000000001 2024-09-17T21:21:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149255+0000", 2024-09-17T21:21:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0548000000000001e-05 2024-09-17T21:21:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:19859 3.8 e147/133 3:12f72409:::rbd_data.1152f056b73f.0000000000000003:head v 147'482, mlcod=145'479)", 2024-09-17T21:21:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 37.422586445, 2024-09-17T21:21:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033476895999999999, 2024-09-17T21:21:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512845+0000", 2024-09-17T21:21:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.886e-06 2024-09-17T21:21:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524104+0000", 2024-09-17T21:21:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011259668 2024-09-17T21:21:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524148+0000", 2024-09-17T21:21:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3386000000000001e-05 2024-09-17T21:21:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524172+0000", 2024-09-17T21:21:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4762000000000001e-05 2024-09-17T21:21:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524379+0000", 2024-09-17T21:21:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020671599999999999 2024-09-17T21:21:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524421+0000", 2024-09-17T21:21:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2182999999999999e-05 2024-09-17T21:21:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546216+0000", 2024-09-17T21:21:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021795101000000001 2024-09-17T21:21:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546320+0000", 2024-09-17T21:21:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103194 2024-09-17T21:21:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:41.037 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:41.037 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:41.038 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:41.038 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:41.039 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:41.035+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:41.039 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:21:41.048 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:41.048 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:41.065 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:21:41.065 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:21:41.096 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:41.096 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:41.117 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:41.117 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:41.138 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:41.138 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:41.139 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:41.135+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:41.163 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:41.163 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:41.177 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:21:41.265 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:41.263+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:41.340 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:21:41.341 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:41.337+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:41.345 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:41.345 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:41.363 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:21:41.453 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:41.451+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:41.521 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:41.521 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:41.535 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:21:41.542 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:41.537+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:41.643 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:41.641+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:41.692 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:41.692 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:41.706 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:21:41.744 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:41.741+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:41.822 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:41.822 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:41.836 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:21:41.845 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:41.841+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:41.946 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:41.943+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:41.956 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:41.956 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:41.971 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:21:42.057 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:42.055+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:42.137 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:42.137 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:42.148 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:42.145+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:42.151 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:21:42.249 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:42.245+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:42.299 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-17T21:21:42.328 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:42.329 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:42.349 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:42.347+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:42.355 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:21:42.451 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:42.447+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:42.534 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:42.534 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:42.552 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:21:42.552 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:42.549+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:42.653 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:42.651+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:42.692 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:42.693 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:42.708 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:21:42.753 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:42.751+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:42.835 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:42.835 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:42.851 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:21:42.854 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:42.849+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:42.955 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:42.951+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:42.987 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:42.988 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:43.003 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:21:43.056 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:43.051+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:43.137 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:43.137 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:43.153 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:21:43.157 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:21:43.157 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:43.155+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:43.258 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:43.253+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:43.274 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:43.274 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:43.289 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:21:43.324 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:43.325 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:43.344 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:43.344 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:43.363 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-17T21:21:43.363 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:21:43.375 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:43.373+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:43.388 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:43.410 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:43.410 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:43.434 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:43.434 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:43.459 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:43.455+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:43.484 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:43.484 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:43.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:43.484 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:43.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.484 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:18003 3.1d 3:b9765580:::rbd_header.11527d2b15f6:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T21:21:43.484 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:43.484 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 91.123261662000004, 2024-09-17T21:21:43.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94165346900000002, 2024-09-17T21:21:43.485 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:43.485 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:43.485 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:43.485 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:43.485 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:43.485 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:43.485 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18003 2024-09-17T21:21:43.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.485 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:43.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:43.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:43.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:43.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:43.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:43.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357880+0000", 2024-09-17T21:21:43.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:21:43.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:43.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357891+0000", 2024-09-17T21:21:43.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0417999999999999e-05 2024-09-17T21:21:43.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:43.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357913+0000", 2024-09-17T21:21:43.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2215e-05 2024-09-17T21:21:43.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:43.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357922+0000", 2024-09-17T21:21:43.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4779999999999997e-06 2024-09-17T21:21:43.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:43.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358023+0000", 2024-09-17T21:21:43.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101025 2024-09-17T21:21:43.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:43.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358040+0000", 2024-09-17T21:21:43.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6291999999999999e-05 2024-09-17T21:21:43.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:43.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298203+0000", 2024-09-17T21:21:43.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94016297500000001 2024-09-17T21:21:43.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:43.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298626+0000", 2024-09-17T21:21:43.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000422853 2024-09-17T21:21:43.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:43.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.299532+0000", 2024-09-17T21:21:43.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00090657300000000004 2024-09-17T21:21:43.489 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.489 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:43.489 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.489 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15706 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 977920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:43.490 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:21:43.490 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.960159871000002, 2024-09-17T21:21:43.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091952328, 2024-09-17T21:21:43.490 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:43.490 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:43.490 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:43.490 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:43.490 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:43.490 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:43.490 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15706 2024-09-17T21:21:43.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.490 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:43.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:43.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:21:43.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:43.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520980+0000", 2024-09-17T21:21:43.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:43.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520982+0000", 2024-09-17T21:21:43.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.277e-06 2024-09-17T21:21:43.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:43.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.520988+0000", 2024-09-17T21:21:43.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4470000000000003e-06 2024-09-17T21:21:43.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:43.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521008+0000", 2024-09-17T21:21:43.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9309000000000001e-05 2024-09-17T21:21:43.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:43.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521023+0000", 2024-09-17T21:21:43.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5119999999999999e-05 2024-09-17T21:21:43.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:43.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612638+0000", 2024-09-17T21:21:43.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091615328999999995 2024-09-17T21:21:43.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:43.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612738+0000", 2024-09-17T21:21:43.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100466 2024-09-17T21:21:43.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:43.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612933+0000", 2024-09-17T21:21:43.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019437999999999999 2024-09-17T21:21:43.494 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.494 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:43.494 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.494 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15707 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 982016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:43.494 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:43.494 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.960096414999995, 2024-09-17T21:21:43.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092305228000000003, 2024-09-17T21:21:43.495 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:43.495 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:43.495 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:43.495 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:43.495 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:43.495 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:43.495 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15707 2024-09-17T21:21:43.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.496 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:43.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:43.496 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:43.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:43.496 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:43.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:43.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521045+0000", 2024-09-17T21:21:43.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4899999999999999e-06 2024-09-17T21:21:43.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:43.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521066+0000", 2024-09-17T21:21:43.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0236000000000001e-05 2024-09-17T21:21:43.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:43.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521085+0000", 2024-09-17T21:21:43.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8819999999999999e-05 2024-09-17T21:21:43.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:43.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521100+0000", 2024-09-17T21:21:43.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5146e-05 2024-09-17T21:21:43.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:43.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612978+0000", 2024-09-17T21:21:43.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091878847 2024-09-17T21:21:43.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:43.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613079+0000", 2024-09-17T21:21:43.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100503 2024-09-17T21:21:43.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:43.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613349+0000", 2024-09-17T21:21:43.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000270186 2024-09-17T21:21:43.502 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.502 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:43.502 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.502 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15708 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 986112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:43.503 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:43.503 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.960018239999997, 2024-09-17T21:21:43.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092547217000000001, 2024-09-17T21:21:43.503 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:43.503 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:43.503 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:43.503 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:43.503 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:43.503 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:43.503 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15708 2024-09-17T21:21:43.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.504 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:43.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:43.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:43.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:43.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:43.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:43.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521123+0000", 2024-09-17T21:21:43.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.353e-06 2024-09-17T21:21:43.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:43.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521129+0000", 2024-09-17T21:21:43.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:21:43.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:43.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521145+0000", 2024-09-17T21:21:43.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.611e-05 2024-09-17T21:21:43.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:43.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521154+0000", 2024-09-17T21:21:43.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4500000000000004e-06 2024-09-17T21:21:43.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:43.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613397+0000", 2024-09-17T21:21:43.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092243275 2024-09-17T21:21:43.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:43.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613503+0000", 2024-09-17T21:21:43.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106013 2024-09-17T21:21:43.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:43.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613669+0000", 2024-09-17T21:21:43.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016642299999999999 2024-09-17T21:21:43.507 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.507 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:43.507 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.507 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15709 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 990208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:43.508 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:43.508 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.959975224000004, 2024-09-17T21:21:43.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092728678999999994, 2024-09-17T21:21:43.508 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:43.508 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:43.508 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:43.508 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:43.508 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:43.509 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:43.509 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15709 2024-09-17T21:21:43.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.509 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:43.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:43.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:43.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:43.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:43.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:43.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521166+0000", 2024-09-17T21:21:43.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7000000000000004e-07 2024-09-17T21:21:43.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:43.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521169+0000", 2024-09-17T21:21:43.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2660000000000002e-06 2024-09-17T21:21:43.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:43.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521181+0000", 2024-09-17T21:21:43.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.208e-05 2024-09-17T21:21:43.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:43.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521194+0000", 2024-09-17T21:21:43.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2707e-05 2024-09-17T21:21:43.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:43.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613695+0000", 2024-09-17T21:21:43.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092500709 2024-09-17T21:21:43.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:43.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613750+0000", 2024-09-17T21:21:43.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.503e-05 2024-09-17T21:21:43.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:43.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613894+0000", 2024-09-17T21:21:43.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144117 2024-09-17T21:21:43.512 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.512 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:43.512 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.513 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15710 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 994304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:43.513 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:43.513 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.959930658000005, 2024-09-17T21:21:43.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092957729000000003, 2024-09-17T21:21:43.513 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:43.513 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:43.513 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:43.513 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:43.513 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:43.513 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:43.513 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15710 2024-09-17T21:21:43.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.514 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:43.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:43.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:43.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:43.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:43.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:43.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:43.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8000000000000005e-07 2024-09-17T21:21:43.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:43.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521216+0000", 2024-09-17T21:21:43.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3859999999999998e-06 2024-09-17T21:21:43.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:43.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521234+0000", 2024-09-17T21:21:43.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:21:43.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:43.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521248+0000", 2024-09-17T21:21:43.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:21:43.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:43.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613926+0000", 2024-09-17T21:21:43.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092678324000000006 2024-09-17T21:21:43.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:43.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613974+0000", 2024-09-17T21:21:43.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7754999999999999e-05 2024-09-17T21:21:43.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:43.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614167+0000", 2024-09-17T21:21:43.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193508 2024-09-17T21:21:43.517 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.517 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:43.517 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.517 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15711 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 998400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:43.517 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:43.517 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.959873275999996, 2024-09-17T21:21:43.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093268000000000004, 2024-09-17T21:21:43.517 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:43.518 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:43.518 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:43.518 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:43.518 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:43.518 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:43.518 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15711 2024-09-17T21:21:43.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.518 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:43.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:43.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:43.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:43.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:43.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:43.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521268+0000", 2024-09-17T21:21:43.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-17T21:21:43.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:43.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521273+0000", 2024-09-17T21:21:43.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:21:43.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:43.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521290+0000", 2024-09-17T21:21:43.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7269999999999999e-05 2024-09-17T21:21:43.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:43.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521304+0000", 2024-09-17T21:21:43.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3626e-05 2024-09-17T21:21:43.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:43.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614220+0000", 2024-09-17T21:21:43.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092916367 2024-09-17T21:21:43.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:43.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614322+0000", 2024-09-17T21:21:43.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102212 2024-09-17T21:21:43.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:43.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614535+0000", 2024-09-17T21:21:43.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021266199999999999 2024-09-17T21:21:43.521 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.521 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:43.521 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.522 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15712 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1002496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:43.522 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:43.522 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.959817780999998, 2024-09-17T21:21:43.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093568707000000001, 2024-09-17T21:21:43.522 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:43.522 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:43.522 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:43.522 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:43.522 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:43.522 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:43.522 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15712 2024-09-17T21:21:43.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.522 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:43.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:43.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:43.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:43.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:43.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:43.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521324+0000", 2024-09-17T21:21:43.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.17e-06 2024-09-17T21:21:43.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:43.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521334+0000", 2024-09-17T21:21:43.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9369999999999999e-06 2024-09-17T21:21:43.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:43.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521356+0000", 2024-09-17T21:21:43.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2045999999999999e-05 2024-09-17T21:21:43.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:43.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521370+0000", 2024-09-17T21:21:43.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4270000000000001e-05 2024-09-17T21:21:43.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:43.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614583+0000", 2024-09-17T21:21:43.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093213220999999999 2024-09-17T21:21:43.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:43.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614680+0000", 2024-09-17T21:21:43.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6900999999999999e-05 2024-09-17T21:21:43.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:43.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614891+0000", 2024-09-17T21:21:43.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021116200000000001 2024-09-17T21:21:43.526 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.526 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:43.526 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.526 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15713 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1006592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:43.526 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:43.526 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.959751079, 2024-09-17T21:21:43.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093834641999999996, 2024-09-17T21:21:43.526 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:43.526 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:43.526 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:43.526 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:43.527 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:43.527 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:43.527 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15713 2024-09-17T21:21:43.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.527 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:43.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:43.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:43.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:43.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:43.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:43.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521390+0000", 2024-09-17T21:21:43.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:21:43.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:43.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521396+0000", 2024-09-17T21:21:43.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:21:43.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:43.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521417+0000", 2024-09-17T21:21:43.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1236000000000001e-05 2024-09-17T21:21:43.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:43.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521431+0000", 2024-09-17T21:21:43.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3873000000000001e-05 2024-09-17T21:21:43.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:43.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614938+0000", 2024-09-17T21:21:43.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093506522999999994 2024-09-17T21:21:43.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:43.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615025+0000", 2024-09-17T21:21:43.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7635999999999999e-05 2024-09-17T21:21:43.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:43.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615224+0000", 2024-09-17T21:21:43.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019844700000000001 2024-09-17T21:21:43.530 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.530 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:43.530 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.530 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15714 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:43.530 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:43.530 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.959687822999996, 2024-09-17T21:21:43.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094071925000000001, 2024-09-17T21:21:43.531 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:43.531 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:43.531 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:43.531 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:43.531 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:43.531 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:43.531 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15714 2024-09-17T21:21:43.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.531 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:43.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:43.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:43.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:43.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:43.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:43.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521454+0000", 2024-09-17T21:21:43.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.097e-06 2024-09-17T21:21:43.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:43.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521459+0000", 2024-09-17T21:21:43.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2660000000000001e-06 2024-09-17T21:21:43.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:43.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521481+0000", 2024-09-17T21:21:43.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1878999999999998e-05 2024-09-17T21:21:43.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:43.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521498+0000", 2024-09-17T21:21:43.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6807e-05 2024-09-17T21:21:43.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:43.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615271+0000", 2024-09-17T21:21:43.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093773286999999997 2024-09-17T21:21:43.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:43.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615366+0000", 2024-09-17T21:21:43.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5322000000000007e-05 2024-09-17T21:21:43.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:43.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615524+0000", 2024-09-17T21:21:43.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015826699999999999 2024-09-17T21:21:43.534 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.534 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:43.534 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.535 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15715 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1014784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:43.535 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:43.535 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.959623488000005, 2024-09-17T21:21:43.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094243840999999995, 2024-09-17T21:21:43.535 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:43.535 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:43.535 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:43.535 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:43.535 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:43.535 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:43.535 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15715 2024-09-17T21:21:43.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.536 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:43.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:43.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:43.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:43.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:43.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:43.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521518+0000", 2024-09-17T21:21:43.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4300000000000001e-06 2024-09-17T21:21:43.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:43.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521524+0000", 2024-09-17T21:21:43.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5860000000000004e-06 2024-09-17T21:21:43.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:43.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521541+0000", 2024-09-17T21:21:43.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7574000000000001e-05 2024-09-17T21:21:43.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:43.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521555+0000", 2024-09-17T21:21:43.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3540000000000001e-05 2024-09-17T21:21:43.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:43.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615560+0000", 2024-09-17T21:21:43.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.09400472 2024-09-17T21:21:43.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:43.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615614+0000", 2024-09-17T21:21:43.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4693e-05 2024-09-17T21:21:43.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:43.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615761+0000", 2024-09-17T21:21:43.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000146298 2024-09-17T21:21:43.539 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.539 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:43.539 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.539 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15716 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1018880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:43.539 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:43.539 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.959567143000001, 2024-09-17T21:21:43.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094400676000000003, 2024-09-17T21:21:43.539 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:43.539 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:43.540 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:43.540 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:43.540 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:43.540 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:43.540 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15716 2024-09-17T21:21:43.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.540 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:43.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:43.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:43.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:43.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:43.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:43.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521574+0000", 2024-09-17T21:21:43.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1939999999999999e-06 2024-09-17T21:21:43.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:43.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521579+0000", 2024-09-17T21:21:43.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:21:43.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:43.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521597+0000", 2024-09-17T21:21:43.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8074e-05 2024-09-17T21:21:43.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:43.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521613+0000", 2024-09-17T21:21:43.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5316e-05 2024-09-17T21:21:43.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:43.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615780+0000", 2024-09-17T21:21:43.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094166874999999997 2024-09-17T21:21:43.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:43.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615836+0000", 2024-09-17T21:21:43.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6926999999999999e-05 2024-09-17T21:21:43.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:43.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615974+0000", 2024-09-17T21:21:43.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013741400000000001 2024-09-17T21:21:43.543 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.543 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:43.543 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.543 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15717 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1022976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:43.544 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:43.544 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.959508447000005, 2024-09-17T21:21:43.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094586723999999997, 2024-09-17T21:21:43.544 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:43.544 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:43.544 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:43.544 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:43.544 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:43.544 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:43.544 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15717 2024-09-17T21:21:43.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.544 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:43.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:43.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:43.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:43.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:43.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:43.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521633+0000", 2024-09-17T21:21:43.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:21:43.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:43.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521638+0000", 2024-09-17T21:21:43.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:21:43.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:43.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521659+0000", 2024-09-17T21:21:43.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-05 2024-09-17T21:21:43.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:43.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521675+0000", 2024-09-17T21:21:43.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5688999999999998e-05 2024-09-17T21:21:43.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:43.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616008+0000", 2024-09-17T21:21:43.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094333314000000001 2024-09-17T21:21:43.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:43.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616066+0000", 2024-09-17T21:21:43.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7797000000000002e-05 2024-09-17T21:21:43.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:43.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616219+0000", 2024-09-17T21:21:43.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015281100000000001 2024-09-17T21:21:43.548 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.548 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:43.548 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.548 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15718 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:43.548 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:43.548 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.959408371999999, 2024-09-17T21:21:43.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094703674000000002, 2024-09-17T21:21:43.548 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:43.549 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:43.549 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:43.549 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:43.549 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:43.549 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:43.549 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15718 2024-09-17T21:21:43.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.549 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:43.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:43.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:43.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:43.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:43.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:43.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521734+0000", 2024-09-17T21:21:43.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.61e-06 2024-09-17T21:21:43.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:43.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521740+0000", 2024-09-17T21:21:43.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:21:43.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:43.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521754+0000", 2024-09-17T21:21:43.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3793e-05 2024-09-17T21:21:43.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:43.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521763+0000", 2024-09-17T21:21:43.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5129999999999996e-06 2024-09-17T21:21:43.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:43.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616256+0000", 2024-09-17T21:21:43.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094492919999999994 2024-09-17T21:21:43.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:43.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616312+0000", 2024-09-17T21:21:43.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6737000000000002e-05 2024-09-17T21:21:43.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:43.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616436+0000", 2024-09-17T21:21:43.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012335800000000001 2024-09-17T21:21:43.552 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.553 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:43.553 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.553 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15719 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1031168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:43.553 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:43.553 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.959354966999996, 2024-09-17T21:21:43.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094888963000000007, 2024-09-17T21:21:43.553 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:43.553 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:43.553 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:43.553 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:43.553 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:43.553 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:43.554 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15719 2024-09-17T21:21:43.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.554 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:43.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:43.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:43.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:43.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:43.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:43.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521786+0000", 2024-09-17T21:21:43.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.074e-06 2024-09-17T21:21:43.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:43.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521791+0000", 2024-09-17T21:21:43.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3030000000000001e-06 2024-09-17T21:21:43.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:43.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521804+0000", 2024-09-17T21:21:43.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3027000000000001e-05 2024-09-17T21:21:43.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:43.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521811+0000", 2024-09-17T21:21:43.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6090000000000003e-06 2024-09-17T21:21:43.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:43.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616466+0000", 2024-09-17T21:21:43.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094654294 2024-09-17T21:21:43.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:43.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616523+0000", 2024-09-17T21:21:43.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6953999999999997e-05 2024-09-17T21:21:43.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.557 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:43.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616674+0000", 2024-09-17T21:21:43.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015170200000000001 2024-09-17T21:21:43.557 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.557 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:43.557 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.557 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15720 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1035264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:43.557 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:43.558 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.959318127000003, 2024-09-17T21:21:43.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095067061999999994, 2024-09-17T21:21:43.558 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:43.558 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:43.558 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:43.558 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:43.558 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:43.558 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:43.558 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15720 2024-09-17T21:21:43.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.558 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:43.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:43.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:43.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:43.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:43.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:43.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521823+0000", 2024-09-17T21:21:43.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:21:43.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:43.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521826+0000", 2024-09-17T21:21:43.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2100000000000002e-06 2024-09-17T21:21:43.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:43.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521835+0000", 2024-09-17T21:21:43.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1630000000000002e-06 2024-09-17T21:21:43.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:43.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521842+0000", 2024-09-17T21:21:43.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:21:43.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:43.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616711+0000", 2024-09-17T21:21:43.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094868859 2024-09-17T21:21:43.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:43.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616764+0000", 2024-09-17T21:21:43.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2778999999999997e-05 2024-09-17T21:21:43.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:43.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616889+0000", 2024-09-17T21:21:43.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012533100000000001 2024-09-17T21:21:43.562 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.562 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:43.562 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.562 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15721 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1039360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:43.562 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:43.562 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.959271017999995, 2024-09-17T21:21:43.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095277219999999996, 2024-09-17T21:21:43.562 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:43.562 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:43.562 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:43.563 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:43.563 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:43.563 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:43.563 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15721 2024-09-17T21:21:43.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.563 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:43.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:43.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:43.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:43.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:43.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:43.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521871+0000", 2024-09-17T21:21:43.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.443e-06 2024-09-17T21:21:43.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:43.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521876+0000", 2024-09-17T21:21:43.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0300000000000001e-06 2024-09-17T21:21:43.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:43.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521886+0000", 2024-09-17T21:21:43.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186e-05 2024-09-17T21:21:43.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:43.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521893+0000", 2024-09-17T21:21:43.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.08e-06 2024-09-17T21:21:43.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:43.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616918+0000", 2024-09-17T21:21:43.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095025223000000006 2024-09-17T21:21:43.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:43.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616977+0000", 2024-09-17T21:21:43.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8403999999999999e-05 2024-09-17T21:21:43.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:43.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617147+0000", 2024-09-17T21:21:43.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016985399999999999 2024-09-17T21:21:43.566 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.567 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:43.567 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.567 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15722 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:43.567 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:43.567 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.959236568999998, 2024-09-17T21:21:43.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095459783000000006, 2024-09-17T21:21:43.567 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:43.567 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:43.567 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:43.567 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:43.567 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:43.567 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:43.568 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15722 2024-09-17T21:21:43.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.568 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:43.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:43.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:43.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:43.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:43.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:43.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:43.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7700000000000004e-07 2024-09-17T21:21:43.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:43.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521907+0000", 2024-09-17T21:21:43.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.813e-06 2024-09-17T21:21:43.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:43.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521922+0000", 2024-09-17T21:21:43.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4533e-05 2024-09-17T21:21:43.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:43.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521934+0000", 2024-09-17T21:21:43.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2469999999999999e-05 2024-09-17T21:21:43.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:43.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617187+0000", 2024-09-17T21:21:43.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095252311000000006 2024-09-17T21:21:43.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:43.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617240+0000", 2024-09-17T21:21:43.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3740999999999999e-05 2024-09-17T21:21:43.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:43.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617364+0000", 2024-09-17T21:21:43.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-17T21:21:43.571 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.571 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:43.571 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.571 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15723 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:43.572 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:43.572 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.959192702999999, 2024-09-17T21:21:43.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095653563999999996, 2024-09-17T21:21:43.572 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:43.572 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:43.572 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:43.572 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:43.572 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:43.572 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:43.572 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15723 2024-09-17T21:21:43.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.572 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:43.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:43.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:43.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:43.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:43.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:43.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521949+0000", 2024-09-17T21:21:43.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:21:43.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:43.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521953+0000", 2024-09-17T21:21:43.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5870000000000003e-06 2024-09-17T21:21:43.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:43.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521970+0000", 2024-09-17T21:21:43.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7282000000000002e-05 2024-09-17T21:21:43.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:43.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521984+0000", 2024-09-17T21:21:43.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:21:43.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:43.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617383+0000", 2024-09-17T21:21:43.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095399063000000006 2024-09-17T21:21:43.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:43.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617437+0000", 2024-09-17T21:21:43.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3931999999999998e-05 2024-09-17T21:21:43.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:43.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617601+0000", 2024-09-17T21:21:43.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016407300000000001 2024-09-17T21:21:43.576 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.576 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:43.576 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.576 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5629.0:2384 3.1c 3:38c1263e:::rbd_data.1152d37183c3.000000000000000b:head [read 11780096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-17T21:21:43.576 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:43.576 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.056361873, 2024-09-17T21:21:43.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0246185320000001, 2024-09-17T21:21:43.576 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:43.577 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:43.577 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:43.577 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:43.577 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5629", 2024-09-17T21:21:43.577 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3798711329", 2024-09-17T21:21:43.577 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2384 2024-09-17T21:21:43.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.577 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:43.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:43.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:43.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:43.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:43.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:43.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:43.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424780+0000", 2024-09-17T21:21:43.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.13e-06 2024-09-17T21:21:43.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:43.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424785+0000", 2024-09-17T21:21:43.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4739999999999996e-06 2024-09-17T21:21:43.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:43.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424798+0000", 2024-09-17T21:21:43.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2704000000000001e-05 2024-09-17T21:21:43.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:43.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424806+0000", 2024-09-17T21:21:43.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9899999999999997e-06 2024-09-17T21:21:43.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:43.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424868+0000", 2024-09-17T21:21:43.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999997e-05 2024-09-17T21:21:43.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:43.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424875+0000", 2024-09-17T21:21:43.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0870000000000001e-06 2024-09-17T21:21:43.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:43.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428897+0000", 2024-09-17T21:21:43.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040226080000000004 2024-09-17T21:21:43.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:43.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428913+0000", 2024-09-17T21:21:43.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:21:43.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:43.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.448963+0000", 2024-09-17T21:21:43.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0200498659999999 2024-09-17T21:21:43.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:43.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449158+0000", 2024-09-17T21:21:43.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019581000000000001 2024-09-17T21:21:43.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:43.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:43.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:43.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449397+0000", 2024-09-17T21:21:43.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023871 2024-09-17T21:21:43.582 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.582 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:43.582 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.582 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:43.582 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:43.582 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:43.584 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:21:43.584 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:43.581+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:43.684 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:43.681+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:43.764 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:43.764 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:43.779 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:21:43.785 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:43.783+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:43.886 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:43.881+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:43.953 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:43.953 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:43.969 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:21:44.011 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:44.009+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:44.087 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:44.085+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:44.139 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:44.140 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:44.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:44.140 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:44.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.140 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:21:44.140 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:44.140 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 202.532387583, 2024-09-17T21:21:44.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:21:44.140 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:44.140 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:44.140 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:44.141 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:44.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:44.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:44.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:44.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:44.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:44.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:21:44.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:21:44.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:44.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:21:44.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:21:44.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:44.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:21:44.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:21:44.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:21:44.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:21:44.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:44.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:21:44.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:21:44.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:44.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:21:44.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:21:44.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:44.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:21:44.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:21:44.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:44.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:21:44.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:21:44.144 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.144 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:44.144 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.144 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:21:44.145 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:44.145 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 202.52244094700001, 2024-09-17T21:21:44.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:21:44.145 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:44.145 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:44.145 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:44.145 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:44.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:44.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:44.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:44.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:44.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:44.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:21:44.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:21:44.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:44.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:21:44.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:21:44.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:44.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:21:44.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:21:44.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:21:44.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:21:44.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:44.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:21:44.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:21:44.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:44.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:21:44.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:21:44.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:44.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:21:44.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:21:44.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:44.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:21:44.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:21:44.148 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.149 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:44.149 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.149 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:21:44.149 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:44.149 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 201.27929010700001, 2024-09-17T21:21:44.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:21:44.149 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:44.149 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:44.149 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:44.149 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:44.149 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:21:44.149 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:21:44.149 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:21:44.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.150 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:44.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:44.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:44.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:44.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:44.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:44.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:21:44.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:21:44.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:44.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:21:44.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:21:44.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:44.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:21:44.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:21:44.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:21:44.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:21:44.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:44.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:21:44.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:21:44.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:44.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:21:44.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:21:44.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:44.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:21:44.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:21:44.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:44.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:21:44.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:21:44.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:44.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:21:44.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:21:44.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:44.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:21:44.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:21:44.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:44.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:21:44.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:21:44.154 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.154 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:44.154 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.154 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:21:44.154 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:44.154 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 201.269644621, 2024-09-17T21:21:44.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:21:44.154 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:44.155 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:44.155 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:44.155 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:44.155 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:21:44.155 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:21:44.155 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:21:44.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.155 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:44.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:44.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:44.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:44.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:44.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:44.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:21:44.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:21:44.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:44.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:21:44.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:21:44.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:44.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:21:44.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:21:44.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:21:44.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:21:44.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:44.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:21:44.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:21:44.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:44.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:21:44.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:21:44.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:44.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:21:44.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:21:44.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:44.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:21:44.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:21:44.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:44.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:21:44.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:21:44.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:44.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:21:44.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:21:44.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:44.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:21:44.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:21:44.159 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.159 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:44.159 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.159 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:21:44.160 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:44.160 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 187.90070858799999, 2024-09-17T21:21:44.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:21:44.160 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:44.160 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:44.160 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:44.160 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:44.160 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:21:44.160 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:21:44.160 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:21:44.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.160 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:44.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:44.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:44.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:44.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:44.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:44.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:21:44.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:21:44.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:44.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:21:44.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:21:44.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:44.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:21:44.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:21:44.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:21:44.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:21:44.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:44.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:21:44.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:21:44.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:44.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:21:44.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:21:44.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:44.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:21:44.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:21:44.163 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.163 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:44.164 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.164 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:44.164 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:44.164 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 169.69902402599999, 2024-09-17T21:21:44.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:21:44.164 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:44.164 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:44.164 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:44.164 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:44.164 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:44.164 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:44.164 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:21:44.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.165 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:44.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:44.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:44.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:44.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:44.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:44.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:21:44.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:21:44.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:44.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:21:44.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:21:44.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:44.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:21:44.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:21:44.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:21:44.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:21:44.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:44.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:21:44.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:21:44.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:44.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:21:44.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:21:44.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:44.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:21:44.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:21:44.168 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.168 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:44.168 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.168 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:44.168 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:44.168 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 169.69852944900001, 2024-09-17T21:21:44.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:21:44.168 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:44.168 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:44.169 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:44.169 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:44.169 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:44.169 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:44.169 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:21:44.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.169 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:44.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:44.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:44.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:44.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:44.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:44.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:21:44.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:21:44.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:44.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:21:44.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:21:44.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:44.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:21:44.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:21:44.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:21:44.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:21:44.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:44.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:21:44.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:21:44.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:44.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:21:44.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:21:44.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:44.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:21:44.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:21:44.172 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.172 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:44.172 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.173 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:44.173 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:44.173 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 169.698474068, 2024-09-17T21:21:44.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:21:44.173 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:44.173 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:44.173 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:44.173 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:44.173 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:44.173 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:44.173 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:21:44.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.173 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:44.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:44.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:44.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:44.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:44.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:44.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:21:44.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:21:44.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:44.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:21:44.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:21:44.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:44.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:21:44.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:21:44.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:21:44.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:21:44.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:44.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:21:44.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:21:44.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:44.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:21:44.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:21:44.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:44.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:21:44.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:21:44.177 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.177 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:44.177 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.177 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:44.177 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:44.177 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 169.69841779000001, 2024-09-17T21:21:44.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:21:44.177 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:44.177 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:44.177 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:44.177 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:44.178 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:44.178 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:44.178 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:21:44.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.178 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:44.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:44.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:44.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:44.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:44.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:44.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:21:44.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:21:44.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:44.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:21:44.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:21:44.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:44.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:21:44.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:21:44.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:21:44.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:21:44.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:44.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:21:44.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:21:44.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:44.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:21:44.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:21:44.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:44.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:21:44.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:21:44.181 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.181 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:44.181 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.181 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:44.181 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:44.181 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 169.69746815100001, 2024-09-17T21:21:44.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:21:44.182 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:44.182 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:44.182 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:44.182 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:44.182 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:44.182 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:44.182 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:21:44.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.182 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:44.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:44.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:44.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:44.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:44.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:44.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:21:44.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:21:44.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:44.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:21:44.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:21:44.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:44.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:21:44.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:21:44.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:21:44.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:21:44.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:44.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:21:44.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:21:44.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:44.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:21:44.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:21:44.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:44.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:21:44.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:21:44.185 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.185 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:44.185 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.186 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:44.186 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:44.186 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 169.697389531, 2024-09-17T21:21:44.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:21:44.186 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:44.186 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:44.186 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:44.186 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:44.186 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:44.186 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:44.186 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:21:44.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.186 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:44.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:44.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:44.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:44.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:44.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:44.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:21:44.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:21:44.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:44.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:21:44.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:21:44.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:44.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:21:44.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:21:44.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:21:44.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:21:44.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:44.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:21:44.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:21:44.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:44.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:21:44.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:21:44.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:44.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:21:44.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:21:44.189 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.189 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:44.190 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.190 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:44.190 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:44.190 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 169.69727659200001, 2024-09-17T21:21:44.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:21:44.190 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:44.190 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:44.190 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:44.190 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:44.190 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:44.190 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:44.190 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:21:44.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.191 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:44.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:44.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:44.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:44.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:44.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:44.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:21:44.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:21:44.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:44.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:21:44.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:21:44.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:44.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:21:44.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:21:44.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:21:44.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:21:44.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:44.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:21:44.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:21:44.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:44.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:21:44.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:21:44.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:44.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:21:44.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:21:44.194 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.194 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:44.194 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.194 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:44.194 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:44.194 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 169.696615538, 2024-09-17T21:21:44.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:21:44.194 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:44.194 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:44.195 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:44.195 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:44.195 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:44.195 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:44.195 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:21:44.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.195 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:44.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:44.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:44.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:44.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:44.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:44.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:21:44.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:21:44.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:44.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:21:44.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:21:44.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:44.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:21:44.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:21:44.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:21:44.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:21:44.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:44.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:21:44.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:21:44.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:44.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:21:44.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:21:44.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:44.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:21:44.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:21:44.198 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.198 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:44.198 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.199 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:44.199 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:44.199 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 169.69655072699999, 2024-09-17T21:21:44.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:21:44.199 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:44.199 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:44.199 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:44.199 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:44.199 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:44.199 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:44.199 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:21:44.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.199 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:44.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:44.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:44.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:44.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:44.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:44.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:21:44.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:21:44.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:44.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:21:44.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:21:44.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:44.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:21:44.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:21:44.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:21:44.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:21:44.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:44.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:21:44.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:21:44.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:44.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:21:44.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:21:44.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:44.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:21:44.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:21:44.203 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.203 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:44.203 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.203 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:44.203 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:44.203 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 169.69648975000001, 2024-09-17T21:21:44.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:21:44.203 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:44.203 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:44.203 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:44.204 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:44.204 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:44.204 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:44.204 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:21:44.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.204 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:44.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:44.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:44.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:44.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:44.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:44.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:21:44.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:21:44.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:44.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:21:44.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:21:44.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:44.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:21:44.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:21:44.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:21:44.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:21:44.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:44.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:21:44.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:21:44.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:44.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:21:44.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:21:44.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:44.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:21:44.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:21:44.207 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.207 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:44.207 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.208 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:21:44.208 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:44.208 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 114.79891856499999, 2024-09-17T21:21:44.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:21:44.208 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:44.208 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:44.208 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:44.208 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:44.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:44.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:44.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:44.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:44.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:44.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:21:44.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:21:44.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:44.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:21:44.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:21:44.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:44.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:21:44.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:21:44.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:21:44.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:21:44.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:44.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:21:44.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:21:44.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:44.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:21:44.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:21:44.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:44.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:21:44.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:21:44.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:44.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:21:44.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:21:44.212 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.212 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:44.212 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.212 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:21:44.212 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:44.212 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 114.590260171, 2024-09-17T21:21:44.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:21:44.212 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:44.212 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:44.213 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:44.213 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:44.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:44.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:44.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:44.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:44.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:44.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:21:44.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:21:44.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:44.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:21:44.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:21:44.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:44.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:21:44.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:21:44.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:21:44.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:21:44.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:44.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:21:44.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:21:44.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:44.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:21:44.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:21:44.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:44.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:21:44.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:21:44.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:44.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:21:44.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:21:44.216 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.216 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:44.216 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.217 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:9547 3.2 e147/132 3:42b518d9:::rbd_data.1152f056b73f.0000000000000001:head v 147'513, mlcod=142'511)", 2024-09-17T21:21:44.217 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:44.217 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.665287049, 2024-09-17T21:21:44.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.031059566, 2024-09-17T21:21:44.217 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:44.217 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:44.217 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:44.217 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:44.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:44.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:44.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:44.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:44.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:44.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471293+0000", 2024-09-17T21:21:44.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2029999999999999e-06 2024-09-17T21:21:44.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:44.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480606+0000", 2024-09-17T21:21:44.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093123839999999999 2024-09-17T21:21:44.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:44.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480653+0000", 2024-09-17T21:21:44.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7852000000000002e-05 2024-09-17T21:21:44.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480673+0000", 2024-09-17T21:21:44.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0013e-05 2024-09-17T21:21:44.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:44.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480862+0000", 2024-09-17T21:21:44.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018833599999999999 2024-09-17T21:21:44.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:44.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480906+0000", 2024-09-17T21:21:44.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4162000000000002e-05 2024-09-17T21:21:44.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:44.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502259+0000", 2024-09-17T21:21:44.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021353154999999999 2024-09-17T21:21:44.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:44.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502351+0000", 2024-09-17T21:21:44.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1460999999999997e-05 2024-09-17T21:21:44.220 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.220 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:44.221 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.221 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:15234 3.1 e147/134 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head v 147'917, mlcod=145'914)", 2024-09-17T21:21:44.221 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:44.221 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.032626430999997, 2024-09-17T21:21:44.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045303695999999997, 2024-09-17T21:21:44.221 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:44.221 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:44.221 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:44.221 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:44.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:44.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:44.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:44.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:44.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:44.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103954+0000", 2024-09-17T21:21:44.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2259999999999999e-06 2024-09-17T21:21:44.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:44.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119105+0000", 2024-09-17T21:21:44.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015150871 2024-09-17T21:21:44.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:44.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119141+0000", 2024-09-17T21:21:44.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6239000000000003e-05 2024-09-17T21:21:44.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119159+0000", 2024-09-17T21:21:44.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7946999999999999e-05 2024-09-17T21:21:44.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:44.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119344+0000", 2024-09-17T21:21:44.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018524599999999999 2024-09-17T21:21:44.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:44.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119399+0000", 2024-09-17T21:21:44.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4922e-05 2024-09-17T21:21:44.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:44.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149165+0000", 2024-09-17T21:21:44.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029765697000000001 2024-09-17T21:21:44.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:44.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149255+0000", 2024-09-17T21:21:44.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0548000000000001e-05 2024-09-17T21:21:44.225 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.225 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:44.225 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.225 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:19859 3.8 e147/133 3:12f72409:::rbd_data.1152f056b73f.0000000000000003:head v 147'482, mlcod=145'479)", 2024-09-17T21:21:44.225 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:44.225 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 40.623735363999998, 2024-09-17T21:21:44.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033476895999999999, 2024-09-17T21:21:44.225 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:44.225 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:44.225 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:44.225 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:44.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:44.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:44.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:44.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:44.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:44.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:44.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512845+0000", 2024-09-17T21:21:44.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.886e-06 2024-09-17T21:21:44.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:44.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524104+0000", 2024-09-17T21:21:44.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011259668 2024-09-17T21:21:44.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:44.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524148+0000", 2024-09-17T21:21:44.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3386000000000001e-05 2024-09-17T21:21:44.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524172+0000", 2024-09-17T21:21:44.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4762000000000001e-05 2024-09-17T21:21:44.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:44.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524379+0000", 2024-09-17T21:21:44.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020671599999999999 2024-09-17T21:21:44.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:44.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524421+0000", 2024-09-17T21:21:44.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2182999999999999e-05 2024-09-17T21:21:44.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:44.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546216+0000", 2024-09-17T21:21:44.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021795101000000001 2024-09-17T21:21:44.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:44.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:44.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:44.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546320+0000", 2024-09-17T21:21:44.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103194 2024-09-17T21:21:44.229 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.229 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:44.229 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.229 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:44.229 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:44.229 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:44.231 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:44.225+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:44.231 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:21:44.331 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:44.327+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:44.336 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:21:44.363 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:44.363 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:44.363 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:21:44.363 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:44.382 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:21:44.432 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:44.429+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:44.532 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:21:44.531+0000 7f36beebf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25232) UID: 0 2024-09-17T21:21:44.542 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:44.542 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:44.542 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:21:44.542 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:21:44.543 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:44.557 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:21:44.633 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:44.629+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:44.726 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:44.726 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:44.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:44.726 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:44.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.726 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:21:44.726 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:44.726 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 148.21288559999999, 2024-09-17T21:21:44.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:21:44.726 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:44.727 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:44.727 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:44.727 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:44.727 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:21:44.727 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:21:44.727 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:21:44.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.727 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:44.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:44.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:44.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:44.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:44.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:44.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:21:44.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:21:44.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:44.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:21:44.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:21:44.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:44.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:21:44.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:21:44.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:21:44.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:21:44.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:21:44.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:21:44.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:44.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:21:44.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:21:44.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:21:44.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:21:44.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:44.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:21:44.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:21:44.731 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.731 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:44.731 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.731 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2597 3.1f 3:fb0e9615:::rbd_data.1152a8ae2019.000000000000000c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:44.731 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:44.732 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.884449172, 2024-09-17T21:21:44.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097064994000000002, 2024-09-17T21:21:44.732 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:44.732 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:44.732 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:44.732 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:44.732 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:44.732 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:44.732 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2597 2024-09-17T21:21:44.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.732 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:44.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:44.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:44.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:44.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:44.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:44.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838628+0000", 2024-09-17T21:21:44.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.187e-06 2024-09-17T21:21:44.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:44.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838654+0000", 2024-09-17T21:21:44.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:21:44.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:44.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838696+0000", 2024-09-17T21:21:44.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1226000000000001e-05 2024-09-17T21:21:44.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838715+0000", 2024-09-17T21:21:44.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9646000000000001e-05 2024-09-17T21:21:44.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838937+0000", 2024-09-17T21:21:44.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022163 2024-09-17T21:21:44.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:44.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.839284+0000", 2024-09-17T21:21:44.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034753799999999999 2024-09-17T21:21:44.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:44.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935691+0000", 2024-09-17T21:21:44.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.096406360999999996 2024-09-17T21:21:44.736 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.736 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:44.736 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.736 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2641 3.12 3:488a729f:::rbd_data.1152a8ae2019.0000000000000022:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:44.736 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:44.736 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.88099239899999, 2024-09-17T21:21:44.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098280953000000004, 2024-09-17T21:21:44.736 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:44.736 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:44.737 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:44.737 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:44.737 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:44.737 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:44.737 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2641 2024-09-17T21:21:44.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.737 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:44.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:44.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:44.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:44.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:44.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:44.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842083+0000", 2024-09-17T21:21:44.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5300000000000002e-07 2024-09-17T21:21:44.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:44.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842088+0000", 2024-09-17T21:21:44.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:21:44.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:44.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842105+0000", 2024-09-17T21:21:44.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6773e-05 2024-09-17T21:21:44.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842119+0000", 2024-09-17T21:21:44.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:21:44.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842320+0000", 2024-09-17T21:21:44.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020052700000000001 2024-09-17T21:21:44.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:44.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842845+0000", 2024-09-17T21:21:44.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052508600000000002 2024-09-17T21:21:44.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:44.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940363+0000", 2024-09-17T21:21:44.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097518795000000005 2024-09-17T21:21:44.740 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.740 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:44.740 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.741 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2674 3.12 3:4c87e3ac:::rbd_data.1152a8ae2019.0000000000000033:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:44.741 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:44.741 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.880629762, 2024-09-17T21:21:44.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098025125000000005, 2024-09-17T21:21:44.741 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:44.741 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:44.741 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:44.741 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:44.741 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:44.741 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:44.741 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2674 2024-09-17T21:21:44.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.742 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:44.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:44.742 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:44.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:44.742 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:44.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:44.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842447+0000", 2024-09-17T21:21:44.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3999999999999999e-06 2024-09-17T21:21:44.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:44.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842453+0000", 2024-09-17T21:21:44.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:21:44.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:44.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842472+0000", 2024-09-17T21:21:44.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9006000000000001e-05 2024-09-17T21:21:44.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842487+0000", 2024-09-17T21:21:44.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4866e-05 2024-09-17T21:21:44.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843594+0000", 2024-09-17T21:21:44.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011073249999999999 2024-09-17T21:21:44.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:44.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843668+0000", 2024-09-17T21:21:44.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3188000000000005e-05 2024-09-17T21:21:44.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:44.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843893+0000", 2024-09-17T21:21:44.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022503399999999999 2024-09-17T21:21:44.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:44.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850439+0000", 2024-09-17T21:21:44.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065467880000000004 2024-09-17T21:21:44.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:44.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940409+0000", 2024-09-17T21:21:44.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089969511000000002 2024-09-17T21:21:44.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:44.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940444+0000", 2024-09-17T21:21:44.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5345999999999997e-05 2024-09-17T21:21:44.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:44.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940470+0000", 2024-09-17T21:21:44.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5964000000000001e-05 2024-09-17T21:21:44.746 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.747 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:44.747 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.747 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2707 3.1f 3:f805e65e:::rbd_data.1152a8ae2019.0000000000000049:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:44.747 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:44.747 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.875583101, 2024-09-17T21:21:44.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088327300999999997, 2024-09-17T21:21:44.747 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:44.747 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:44.747 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:44.747 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:44.747 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:44.747 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:44.748 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2707 2024-09-17T21:21:44.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.748 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:44.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:44.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:44.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:44.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:44.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:44.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847494+0000", 2024-09-17T21:21:44.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:21:44.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:44.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847504+0000", 2024-09-17T21:21:44.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:21:44.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:44.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847526+0000", 2024-09-17T21:21:44.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1976000000000001e-05 2024-09-17T21:21:44.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847541+0000", 2024-09-17T21:21:44.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5277e-05 2024-09-17T21:21:44.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856142+0000", 2024-09-17T21:21:44.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086006710000000007 2024-09-17T21:21:44.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:44.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856435+0000", 2024-09-17T21:21:44.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029301499999999999 2024-09-17T21:21:44.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:44.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935819+0000", 2024-09-17T21:21:44.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079384448999999996 2024-09-17T21:21:44.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.751 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:44.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.751 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2730 3.12 3:48b3422a:::rbd_data.1152a8ae2019.0000000000000057:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:44.751 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:44.751 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.86639020200001, 2024-09-17T21:21:44.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083879975999999995, 2024-09-17T21:21:44.751 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:44.752 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:44.752 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:44.752 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:44.752 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:44.752 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:44.752 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2730 2024-09-17T21:21:44.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.752 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:44.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.752 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:44.752 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:44.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:44.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:44.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:44.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856687+0000", 2024-09-17T21:21:44.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:21:44.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:44.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856697+0000", 2024-09-17T21:21:44.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0207000000000001e-05 2024-09-17T21:21:44.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:44.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856808+0000", 2024-09-17T21:21:44.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011071200000000001 2024-09-17T21:21:44.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856828+0000", 2024-09-17T21:21:44.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0219000000000001e-05 2024-09-17T21:21:44.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856907+0000", 2024-09-17T21:21:44.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9661999999999997e-05 2024-09-17T21:21:44.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:44.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.857019+0000", 2024-09-17T21:21:44.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011148499999999999 2024-09-17T21:21:44.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:44.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940565+0000", 2024-09-17T21:21:44.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083545791999999994 2024-09-17T21:21:44.755 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.755 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:44.755 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.755 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2743 3.12 3:4d036e62:::rbd_data.1152a8ae2019.000000000000005f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:44.756 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:44.756 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.862444701, 2024-09-17T21:21:44.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080059962999999998, 2024-09-17T21:21:44.756 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:44.756 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:44.756 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:44.756 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:44.756 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:44.756 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:44.756 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2743 2024-09-17T21:21:44.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.756 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:44.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:44.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:44.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:44.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:44.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:44.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860631+0000", 2024-09-17T21:21:44.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:21:44.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:44.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860636+0000", 2024-09-17T21:21:44.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5279999999999997e-06 2024-09-17T21:21:44.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:44.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860649+0000", 2024-09-17T21:21:44.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-17T21:21:44.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860655+0000", 2024-09-17T21:21:44.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:21:44.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860731+0000", 2024-09-17T21:21:44.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5920000000000005e-05 2024-09-17T21:21:44.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:44.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.861070+0000", 2024-09-17T21:21:44.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338535 2024-09-17T21:21:44.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:44.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940690+0000", 2024-09-17T21:21:44.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079620503999999995 2024-09-17T21:21:44.760 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.760 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:44.760 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.760 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3378 3.12 3:4832a51b:::rbd_data.1152a8ae2019.000000000000006c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:44.760 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:44.760 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.834080851, 2024-09-17T21:21:44.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051812516000000003, 2024-09-17T21:21:44.760 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:44.760 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:44.761 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:44.761 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:44.761 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:44.761 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:44.761 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3378 2024-09-17T21:21:44.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.761 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:44.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:44.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:44.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:44.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:44.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:44.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888995+0000", 2024-09-17T21:21:44.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:21:44.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:44.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889006+0000", 2024-09-17T21:21:44.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1012999999999999e-05 2024-09-17T21:21:44.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:44.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889032+0000", 2024-09-17T21:21:44.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6441e-05 2024-09-17T21:21:44.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889040+0000", 2024-09-17T21:21:44.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0639999999999994e-06 2024-09-17T21:21:44.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889278+0000", 2024-09-17T21:21:44.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023785899999999999 2024-09-17T21:21:44.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:44.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889408+0000", 2024-09-17T21:21:44.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129219 2024-09-17T21:21:44.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:44.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889723+0000", 2024-09-17T21:21:44.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000315264 2024-09-17T21:21:44.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:44.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.891972+0000", 2024-09-17T21:21:44.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002249633 2024-09-17T21:21:44.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:44.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940733+0000", 2024-09-17T21:21:44.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048760626000000001 2024-09-17T21:21:44.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:44.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940785+0000", 2024-09-17T21:21:44.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1906000000000001e-05 2024-09-17T21:21:44.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:44.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940807+0000", 2024-09-17T21:21:44.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1554999999999999e-05 2024-09-17T21:21:44.766 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.766 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:44.766 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.766 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3379 3.12 3:4e83fd71:::rbd_data.1152a8ae2019.000000000000006d:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:44.766 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:44.766 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.833996644, 2024-09-17T21:21:44.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051804144000000003, 2024-09-17T21:21:44.766 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:44.766 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:44.766 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:44.766 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:44.766 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:44.766 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:44.767 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3379 2024-09-17T21:21:44.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.767 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:44.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:44.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:44.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:44.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:44.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:44.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889079+0000", 2024-09-17T21:21:44.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1800000000000004e-07 2024-09-17T21:21:44.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:44.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889083+0000", 2024-09-17T21:21:44.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:21:44.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:44.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889094+0000", 2024-09-17T21:21:44.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1562999999999999e-05 2024-09-17T21:21:44.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889102+0000", 2024-09-17T21:21:44.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.401e-06 2024-09-17T21:21:44.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:21:44.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014279259999999999 2024-09-17T21:21:44.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:44.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890609+0000", 2024-09-17T21:21:44.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9566000000000004e-05 2024-09-17T21:21:44.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:44.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890789+0000", 2024-09-17T21:21:44.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017949700000000001 2024-09-17T21:21:44.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:44.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.893929+0000", 2024-09-17T21:21:44.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003140538 2024-09-17T21:21:44.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:44.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940834+0000", 2024-09-17T21:21:44.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046904811999999997 2024-09-17T21:21:44.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:44.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940860+0000", 2024-09-17T21:21:44.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5341999999999999e-05 2024-09-17T21:21:44.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:44.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940882+0000", 2024-09-17T21:21:44.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2878999999999999e-05 2024-09-17T21:21:44.771 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.771 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:44.771 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.772 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:2644 3.1e 3:79615585:::rbd_data.115266d0b9f2.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2475520~14848,write 3106304~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:21:44.772 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:44.772 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 115.211368241, 2024-09-17T21:21:44.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050733484000000002, 2024-09-17T21:21:44.772 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:44.772 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:44.772 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:44.772 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:44.772 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:21:44.772 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:21:44.772 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2644 2024-09-17T21:21:44.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.773 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:44.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:44.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:44.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:44.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:44.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:44.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511708+0000", 2024-09-17T21:21:44.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-17T21:21:44.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:44.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522196+0000", 2024-09-17T21:21:44.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010487712999999999 2024-09-17T21:21:44.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:44.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522223+0000", 2024-09-17T21:21:44.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7064999999999999e-05 2024-09-17T21:21:44.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522236+0000", 2024-09-17T21:21:44.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2931e-05 2024-09-17T21:21:44.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522348+0000", 2024-09-17T21:21:44.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111906 2024-09-17T21:21:44.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:44.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522554+0000", 2024-09-17T21:21:44.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020632699999999999 2024-09-17T21:21:44.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:21:44.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.523185+0000", 2024-09-17T21:21:44.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063081400000000005 2024-09-17T21:21:44.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:44.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.547011+0000", 2024-09-17T21:21:44.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023825570000000001 2024-09-17T21:21:44.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:44.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562268+0000", 2024-09-17T21:21:44.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015257514 2024-09-17T21:21:44.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:44.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562367+0000", 2024-09-17T21:21:44.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8678000000000006e-05 2024-09-17T21:21:44.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:44.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562440+0000", 2024-09-17T21:21:44.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3337000000000006e-05 2024-09-17T21:21:44.777 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.777 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:44.777 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.778 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:231 3.1 3:856fed78:::rbd_data.115220dd393e.000000000000000b:head [read 0~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:44.778 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:44.778 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 86.768612972, 2024-09-17T21:21:44.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34254251699999999, 2024-09-17T21:21:44.778 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:44.778 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:44.778 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:44.778 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:44.778 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:44.778 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:44.778 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 231 2024-09-17T21:21:44.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.778 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:44.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:44.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:44.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:44.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:44.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:44.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954464+0000", 2024-09-17T21:21:44.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:21:44.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:44.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954475+0000", 2024-09-17T21:21:44.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1146e-05 2024-09-17T21:21:44.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:44.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954506+0000", 2024-09-17T21:21:44.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0292999999999999e-05 2024-09-17T21:21:44.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954524+0000", 2024-09-17T21:21:44.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8430000000000001e-05 2024-09-17T21:21:44.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954661+0000", 2024-09-17T21:21:44.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136403 2024-09-17T21:21:44.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:44.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954680+0000", 2024-09-17T21:21:44.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9743000000000001e-05 2024-09-17T21:21:44.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.296896+0000", 2024-09-17T21:21:44.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34221550499999998 2024-09-17T21:21:44.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:44.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297004+0000", 2024-09-17T21:21:44.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010865399999999999 2024-09-17T21:21:44.782 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.782 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:44.782 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.782 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:56 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:44.783 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:44.783 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 86.612979275000001, 2024-09-17T21:21:44.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18718773899999999, 2024-09-17T21:21:44.783 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:44.783 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:44.783 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:44.783 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:44.783 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:21:44.783 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:21:44.783 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 56 2024-09-17T21:21:44.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.783 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:44.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:44.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:44.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:44.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:44.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:44.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110098+0000", 2024-09-17T21:21:44.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:21:44.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:44.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110111+0000", 2024-09-17T21:21:44.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2289999999999999e-05 2024-09-17T21:21:44.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:44.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110143+0000", 2024-09-17T21:21:44.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1979e-05 2024-09-17T21:21:44.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110161+0000", 2024-09-17T21:21:44.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7839e-05 2024-09-17T21:21:44.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110319+0000", 2024-09-17T21:21:44.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015835400000000001 2024-09-17T21:21:44.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:44.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110353+0000", 2024-09-17T21:21:44.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3849000000000001e-05 2024-09-17T21:21:44.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297041+0000", 2024-09-17T21:21:44.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18668842299999999 2024-09-17T21:21:44.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:44.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297169+0000", 2024-09-17T21:21:44.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012786199999999999 2024-09-17T21:21:44.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:44.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297283+0000", 2024-09-17T21:21:44.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011430300000000001 2024-09-17T21:21:44.787 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.787 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:44.787 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.787 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:68 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:44.787 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:44.788 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 86.604438344000002, 2024-09-17T21:21:44.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17885582999999999, 2024-09-17T21:21:44.788 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:44.788 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:44.788 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:44.788 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:44.788 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:21:44.788 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:21:44.788 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 68 2024-09-17T21:21:44.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.788 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:44.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:44.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:44.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:44.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:44.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:44.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118639+0000", 2024-09-17T21:21:44.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.306e-06 2024-09-17T21:21:44.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:44.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118651+0000", 2024-09-17T21:21:44.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2567e-05 2024-09-17T21:21:44.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:44.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118682+0000", 2024-09-17T21:21:44.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0502e-05 2024-09-17T21:21:44.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118707+0000", 2024-09-17T21:21:44.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5463e-05 2024-09-17T21:21:44.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118889+0000", 2024-09-17T21:21:44.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018150999999999999 2024-09-17T21:21:44.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:44.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118910+0000", 2024-09-17T21:21:44.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:21:44.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297313+0000", 2024-09-17T21:21:44.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17840366699999999 2024-09-17T21:21:44.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:44.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297389+0000", 2024-09-17T21:21:44.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-17T21:21:44.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:44.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297492+0000", 2024-09-17T21:21:44.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103019 2024-09-17T21:21:44.792 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.792 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:44.792 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.793 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5890.0:15234 3.1 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8451072~29696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-17T21:21:44.793 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:44.793 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.629789531999997, 2024-09-17T21:21:44.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.057026052000000001, 2024-09-17T21:21:44.793 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:44.793 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:44.793 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:44.793 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:44.793 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5890", 2024-09-17T21:21:44.793 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3064344947", 2024-09-17T21:21:44.793 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 15234 2024-09-17T21:21:44.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.793 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:44.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:44.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:44.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:44.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:44.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:44.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093289+0000", 2024-09-17T21:21:44.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4939999999999999e-06 2024-09-17T21:21:44.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:44.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103074+0000", 2024-09-17T21:21:44.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00978564 2024-09-17T21:21:44.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:44.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103112+0000", 2024-09-17T21:21:44.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7246999999999998e-05 2024-09-17T21:21:44.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103128+0000", 2024-09-17T21:21:44.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5812999999999999e-05 2024-09-17T21:21:44.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103207+0000", 2024-09-17T21:21:44.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9785999999999998e-05 2024-09-17T21:21:44.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:44.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103315+0000", 2024-09-17T21:21:44.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107396 2024-09-17T21:21:44.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:44.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103957+0000", 2024-09-17T21:21:44.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000642524 2024-09-17T21:21:44.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:44.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.123856+0000", 2024-09-17T21:21:44.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019898376999999998 2024-09-17T21:21:44.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:44.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150190+0000", 2024-09-17T21:21:44.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026334682000000002 2024-09-17T21:21:44.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:44.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150255+0000", 2024-09-17T21:21:44.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4770999999999994e-05 2024-09-17T21:21:44.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:44.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150311+0000", 2024-09-17T21:21:44.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6322e-05 2024-09-17T21:21:44.798 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.798 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:44.798 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.798 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4239 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 424960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:44.798 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:21:44.798 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.969391874999999, 2024-09-17T21:21:44.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050123167000000003, 2024-09-17T21:21:44.798 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:44.799 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:44.799 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:44.799 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:44.799 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:44.799 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:44.799 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4239 2024-09-17T21:21:44.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.799 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:44.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:44.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:21:44.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:44.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:21:44.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:44.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753684+0000", 2024-09-17T21:21:44.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.37e-07 2024-09-17T21:21:44.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:44.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753686+0000", 2024-09-17T21:21:44.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.322e-06 2024-09-17T21:21:44.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:44.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753693+0000", 2024-09-17T21:21:44.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0890000000000002e-06 2024-09-17T21:21:44.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753699+0000", 2024-09-17T21:21:44.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:21:44.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803390+0000", 2024-09-17T21:21:44.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049690506000000002 2024-09-17T21:21:44.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:44.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803569+0000", 2024-09-17T21:21:44.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179784 2024-09-17T21:21:44.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:44.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803806+0000", 2024-09-17T21:21:44.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023683499999999999 2024-09-17T21:21:44.802 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.802 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:44.802 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.803 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4240 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 413696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:44.803 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:44.803 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.969368147000001, 2024-09-17T21:21:44.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050331353000000002, 2024-09-17T21:21:44.803 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:44.803 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:44.803 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:44.803 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:44.803 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:44.803 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:44.803 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4240 2024-09-17T21:21:44.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.803 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:44.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:44.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:44.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:44.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:44.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:44.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:44.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5700000000000002e-07 2024-09-17T21:21:44.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:44.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753710+0000", 2024-09-17T21:21:44.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.385e-06 2024-09-17T21:21:44.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:44.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753717+0000", 2024-09-17T21:21:44.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.832e-06 2024-09-17T21:21:44.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753722+0000", 2024-09-17T21:21:44.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8050000000000003e-06 2024-09-17T21:21:44.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803880+0000", 2024-09-17T21:21:44.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050157641000000003 2024-09-17T21:21:44.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:44.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803945+0000", 2024-09-17T21:21:44.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5357000000000005e-05 2024-09-17T21:21:44.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:44.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804038+0000", 2024-09-17T21:21:44.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2775999999999994e-05 2024-09-17T21:21:44.806 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.807 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:44.807 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.807 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4241 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 415744~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:44.807 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:44.807 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.969345031, 2024-09-17T21:21:44.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050578790999999998, 2024-09-17T21:21:44.807 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:44.807 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:44.807 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:44.807 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:44.807 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:44.807 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:44.808 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4241 2024-09-17T21:21:44.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.808 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:44.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:44.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:44.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:44.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:44.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:44.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:44.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5700000000000002e-07 2024-09-17T21:21:44.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:44.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753733+0000", 2024-09-17T21:21:44.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2570000000000002e-06 2024-09-17T21:21:44.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:44.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753740+0000", 2024-09-17T21:21:44.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.849e-06 2024-09-17T21:21:44.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753745+0000", 2024-09-17T21:21:44.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7450000000000001e-06 2024-09-17T21:21:44.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804115+0000", 2024-09-17T21:21:44.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050370038999999998 2024-09-17T21:21:44.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:44.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804223+0000", 2024-09-17T21:21:44.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010786 2024-09-17T21:21:44.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:44.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804309+0000", 2024-09-17T21:21:44.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5483999999999997e-05 2024-09-17T21:21:44.811 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.811 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:44.811 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.811 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4242 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 414720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:44.811 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:44.811 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.969322142999999, 2024-09-17T21:21:44.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050748148999999999, 2024-09-17T21:21:44.811 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:44.811 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:44.812 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:44.812 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:44.812 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:44.812 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:44.812 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4242 2024-09-17T21:21:44.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.812 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:44.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:44.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:44.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:44.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:44.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:44.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:44.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6599999999999996e-07 2024-09-17T21:21:44.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:44.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753756+0000", 2024-09-17T21:21:44.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.368e-06 2024-09-17T21:21:44.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:44.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753763+0000", 2024-09-17T21:21:44.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9349999999999997e-06 2024-09-17T21:21:44.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753768+0000", 2024-09-17T21:21:44.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7620000000000001e-06 2024-09-17T21:21:44.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804343+0000", 2024-09-17T21:21:44.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050575050000000003 2024-09-17T21:21:44.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:44.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804397+0000", 2024-09-17T21:21:44.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4070000000000002e-05 2024-09-17T21:21:44.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:44.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804501+0000", 2024-09-17T21:21:44.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103398 2024-09-17T21:21:44.815 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.815 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:44.815 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.816 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25642 3.1 3.6a5ad301 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e177)", 2024-09-17T21:21:44.816 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:34.316616+0000", 2024-09-17T21:21:44.816 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 10.406458560000001, 2024-09-17T21:21:44.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99065658300000003, 2024-09-17T21:21:44.816 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:44.816 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:44.816 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:44.816 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:44.816 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:21:44.816 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:44.816 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25642 2024-09-17T21:21:44.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.817 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:44.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:44.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316616+0000", 2024-09-17T21:21:44.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:44.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316616+0000", 2024-09-17T21:21:44.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:44.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316618+0000", 2024-09-17T21:21:44.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9309999999999998e-06 2024-09-17T21:21:44.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:44.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316624+0000", 2024-09-17T21:21:44.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2789999999999998e-06 2024-09-17T21:21:44.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:44.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316634+0000", 2024-09-17T21:21:44.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0774000000000001e-05 2024-09-17T21:21:44.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316644+0000", 2024-09-17T21:21:44.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0043e-05 2024-09-17T21:21:44.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316767+0000", 2024-09-17T21:21:44.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012289199999999999 2024-09-17T21:21:44.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:44.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316777+0000", 2024-09-17T21:21:44.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6199999999999994e-06 2024-09-17T21:21:44.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.820 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.307241+0000", 2024-09-17T21:21:44.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99046395099999995 2024-09-17T21:21:44.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.820 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:44.820 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.307273+0000", 2024-09-17T21:21:44.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2092999999999999e-05 2024-09-17T21:21:44.820 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.820 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:44.820 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.820 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25905 3.6 3:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-17T21:21:44.820 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:39.359572+0000", 2024-09-17T21:21:44.821 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 5.3635032779999996, 2024-09-17T21:21:44.821 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96065702100000006, 2024-09-17T21:21:44.821 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:44.821 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:44.821 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:44.821 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:44.821 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:21:44.821 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:44.821 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25905 2024-09-17T21:21:44.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.821 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:44.821 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.821 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:44.821 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359572+0000", 2024-09-17T21:21:44.821 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.822 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:44.822 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359572+0000", 2024-09-17T21:21:44.822 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:44.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.822 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:44.822 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359573+0000", 2024-09-17T21:21:44.822 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5370000000000001e-06 2024-09-17T21:21:44.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:44.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359579+0000", 2024-09-17T21:21:44.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8039999999999998e-06 2024-09-17T21:21:44.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:44.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359603+0000", 2024-09-17T21:21:44.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4533999999999999e-05 2024-09-17T21:21:44.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:44.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359613+0000", 2024-09-17T21:21:44.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2820000000000002e-06 2024-09-17T21:21:44.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359686+0000", 2024-09-17T21:21:44.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3301999999999997e-05 2024-09-17T21:21:44.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:44.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359693+0000", 2024-09-17T21:21:44.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3679999999999996e-06 2024-09-17T21:21:44.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:44.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:40.319999+0000", 2024-09-17T21:21:44.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96030524500000003 2024-09-17T21:21:44.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:44.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:44.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:44.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:40.320229+0000", 2024-09-17T21:21:44.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022994899999999999 2024-09-17T21:21:44.825 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.825 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:44.825 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.825 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:44.825 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:44.825 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:44.827 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:21:44.827 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:44.823+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:44.927 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:44.923+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:45.000 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:45.000 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:45.000 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:21:45.000 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:45.015 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:21:45.061 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:45.059+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:45.129 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:45.125+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:45.183 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:45.183 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:45.183 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:21:45.183 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:21:45.183 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:45.197 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:21:45.230 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:45.225+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:45.330 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:45.325+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:45.332 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:45.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:45.332 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:45.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.332 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:21:45.332 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:45.332 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 185.51853312899999, 2024-09-17T21:21:45.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:21:45.332 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:45.332 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:45.333 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:45.333 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:45.333 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:21:45.333 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:21:45.333 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:21:45.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.333 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:45.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:45.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:45.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:45.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:45.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:45.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:21:45.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:21:45.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:45.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:21:45.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:21:45.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:45.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:21:45.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:21:45.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:21:45.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:21:45.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:21:45.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:21:45.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:45.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:21:45.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:21:45.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:21:45.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:21:45.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:45.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:21:45.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:21:45.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:45.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:21:45.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:21:45.337 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.337 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:45.337 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.338 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:21:45.338 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:45.338 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 171.963431343, 2024-09-17T21:21:45.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:21:45.338 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:45.338 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:45.338 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:45.338 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:45.338 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:21:45.338 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:45.338 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:21:45.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.338 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:45.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:45.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:45.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:45.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:45.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:45.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:21:45.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:21:45.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:45.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:21:45.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:21:45.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:45.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:21:45.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:21:45.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:21:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:21:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:21:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:21:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:21:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:21:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:21:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:21:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:21:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:21:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:21:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:21:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:21:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:21:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:21:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:21:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:21:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:21:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:45.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:21:45.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:21:45.344 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.344 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:45.344 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.344 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:793 3.9 3:95f63599:::rbd_data.1152eb799094.0000000000000007:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:45.344 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:45.344 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 88.896375187999993, 2024-09-17T21:21:45.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88245766400000003, 2024-09-17T21:21:45.345 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:45.345 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:45.345 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:45.345 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:45.345 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:45.345 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:45.345 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 793 2024-09-17T21:21:45.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.345 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:45.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:45.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:45.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:45.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:45.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:45.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431941+0000", 2024-09-17T21:21:45.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.864e-06 2024-09-17T21:21:45.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:45.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431953+0000", 2024-09-17T21:21:45.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1486e-05 2024-09-17T21:21:45.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:45.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431984+0000", 2024-09-17T21:21:45.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1075999999999999e-05 2024-09-17T21:21:45.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431999+0000", 2024-09-17T21:21:45.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.503e-05 2024-09-17T21:21:45.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432095+0000", 2024-09-17T21:21:45.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5698000000000004e-05 2024-09-17T21:21:45.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:45.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432109+0000", 2024-09-17T21:21:45.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4188e-05 2024-09-17T21:21:45.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.309642+0000", 2024-09-17T21:21:45.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87753353000000001 2024-09-17T21:21:45.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:45.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310104+0000", 2024-09-17T21:21:45.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046103600000000001 2024-09-17T21:21:45.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:45.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310630+0000", 2024-09-17T21:21:45.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052682800000000002 2024-09-17T21:21:45.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:45.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313100+0000", 2024-09-17T21:21:45.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024693369999999998 2024-09-17T21:21:45.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:45.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313935+0000", 2024-09-17T21:21:45.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083540100000000005 2024-09-17T21:21:45.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:45.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314086+0000", 2024-09-17T21:21:45.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015053399999999999 2024-09-17T21:21:45.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:45.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314396+0000", 2024-09-17T21:21:45.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031065599999999997 2024-09-17T21:21:45.353 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.353 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:45.353 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.353 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:800 3.19 3:9dc17f12:::rbd_data.1152eb799094.000000000000000e:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:45.353 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:45.353 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 88.896127942999996, 2024-09-17T21:21:45.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88154663799999999, 2024-09-17T21:21:45.353 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:45.353 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:45.353 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:45.354 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:45.354 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:45.354 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:45.354 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 800 2024-09-17T21:21:45.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.354 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:45.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:45.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:45.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:45.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432187+0000", 2024-09-17T21:21:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4100000000000001e-06 2024-09-17T21:21:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432194+0000", 2024-09-17T21:21:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0389999999999999e-06 2024-09-17T21:21:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432216+0000", 2024-09-17T21:21:45.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1614e-05 2024-09-17T21:21:45.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432232+0000", 2024-09-17T21:21:45.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5739e-05 2024-09-17T21:21:45.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432405+0000", 2024-09-17T21:21:45.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017355300000000001 2024-09-17T21:21:45.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:45.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432432+0000", 2024-09-17T21:21:45.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6979e-05 2024-09-17T21:21:45.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.307767+0000", 2024-09-17T21:21:45.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87533472899999998 2024-09-17T21:21:45.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:45.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308192+0000", 2024-09-17T21:21:45.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042485999999999999 2024-09-17T21:21:45.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:45.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308727+0000", 2024-09-17T21:21:45.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00053550400000000001 2024-09-17T21:21:45.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:45.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310347+0000", 2024-09-17T21:21:45.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016198670000000001 2024-09-17T21:21:45.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:45.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313424+0000", 2024-09-17T21:21:45.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030764640000000001 2024-09-17T21:21:45.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:45.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313510+0000", 2024-09-17T21:21:45.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5938000000000003e-05 2024-09-17T21:21:45.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:45.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313733+0000", 2024-09-17T21:21:45.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022294200000000001 2024-09-17T21:21:45.359 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.359 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:45.359 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.360 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:805 3.19 3:9cab2ed6:::rbd_data.1152eb799094.0000000000000013:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:45.360 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:45.360 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 88.895923924000002, 2024-09-17T21:21:45.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88274355699999996, 2024-09-17T21:21:45.360 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:45.360 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:45.360 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:45.360 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:45.360 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:45.360 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:45.361 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 805 2024-09-17T21:21:45.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.361 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:45.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:45.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:45.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:45.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:45.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:45.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432391+0000", 2024-09-17T21:21:45.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3e-06 2024-09-17T21:21:45.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:45.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432398+0000", 2024-09-17T21:21:45.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4489999999999996e-06 2024-09-17T21:21:45.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:45.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432415+0000", 2024-09-17T21:21:45.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7750000000000001e-05 2024-09-17T21:21:45.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432430+0000", 2024-09-17T21:21:45.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.449e-05 2024-09-17T21:21:45.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432635+0000", 2024-09-17T21:21:45.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205031 2024-09-17T21:21:45.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:45.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432657+0000", 2024-09-17T21:21:45.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2064000000000001e-05 2024-09-17T21:21:45.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310444+0000", 2024-09-17T21:21:45.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87778699199999999 2024-09-17T21:21:45.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:45.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310849+0000", 2024-09-17T21:21:45.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00040497800000000001 2024-09-17T21:21:45.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:45.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.311396+0000", 2024-09-17T21:21:45.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00054729099999999999 2024-09-17T21:21:45.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:45.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313847+0000", 2024-09-17T21:21:45.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024504940000000001 2024-09-17T21:21:45.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:45.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314879+0000", 2024-09-17T21:21:45.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001031817 2024-09-17T21:21:45.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:45.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314964+0000", 2024-09-17T21:21:45.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5685000000000004e-05 2024-09-17T21:21:45.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:45.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.315133+0000", 2024-09-17T21:21:45.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000169216 2024-09-17T21:21:45.366 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.366 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:45.366 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.367 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12089 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 60416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:45.367 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:45.367 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.558698891999995, 2024-09-17T21:21:45.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088293233999999998, 2024-09-17T21:21:45.367 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:45.367 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:45.367 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:45.367 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:45.367 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:45.367 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:45.367 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12089 2024-09-17T21:21:45.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.367 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:45.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:45.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:45.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:45.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:45.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:45.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769616+0000", 2024-09-17T21:21:45.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:21:45.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:45.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769620+0000", 2024-09-17T21:21:45.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5140000000000002e-06 2024-09-17T21:21:45.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:45.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769633+0000", 2024-09-17T21:21:45.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2849e-05 2024-09-17T21:21:45.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769644+0000", 2024-09-17T21:21:45.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1471000000000001e-05 2024-09-17T21:21:45.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857564+0000", 2024-09-17T21:21:45.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087919445999999998 2024-09-17T21:21:45.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:45.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857716+0000", 2024-09-17T21:21:45.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000152065 2024-09-17T21:21:45.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:45.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857908+0000", 2024-09-17T21:21:45.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019252599999999999 2024-09-17T21:21:45.371 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.371 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:45.371 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.371 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12090 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 64512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:45.371 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:45.371 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.558652507000005, 2024-09-17T21:21:45.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088582672000000001, 2024-09-17T21:21:45.371 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:45.371 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:45.372 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:45.372 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:45.372 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:45.372 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:45.372 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12090 2024-09-17T21:21:45.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.372 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:45.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:45.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:45.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:45.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:45.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:45.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769662+0000", 2024-09-17T21:21:45.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:21:45.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:45.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769683+0000", 2024-09-17T21:21:45.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0979e-05 2024-09-17T21:21:45.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:45.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769699+0000", 2024-09-17T21:21:45.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5446000000000001e-05 2024-09-17T21:21:45.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769708+0000", 2024-09-17T21:21:45.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1679999999999993e-06 2024-09-17T21:21:45.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857976+0000", 2024-09-17T21:21:45.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088267795999999996 2024-09-17T21:21:45.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:45.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858095+0000", 2024-09-17T21:21:45.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119147 2024-09-17T21:21:45.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:45.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858244+0000", 2024-09-17T21:21:45.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014917000000000001 2024-09-17T21:21:45.375 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.375 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:45.375 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.376 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12091 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 68608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:45.376 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:45.376 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.558581122999996, 2024-09-17T21:21:45.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088832752000000001, 2024-09-17T21:21:45.376 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:45.376 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:45.376 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:45.376 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:45.376 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:45.376 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:45.377 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12091 2024-09-17T21:21:45.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.377 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:45.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:45.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:45.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:45.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:45.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:45.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769734+0000", 2024-09-17T21:21:45.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:21:45.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:45.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769739+0000", 2024-09-17T21:21:45.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8080000000000004e-06 2024-09-17T21:21:45.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:45.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769764+0000", 2024-09-17T21:21:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4839999999999999e-05 2024-09-17T21:21:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769777+0000", 2024-09-17T21:21:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3037000000000001e-05 2024-09-17T21:21:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858295+0000", 2024-09-17T21:21:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088518475999999999 2024-09-17T21:21:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:45.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858419+0000", 2024-09-17T21:21:45.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012378700000000001 2024-09-17T21:21:45.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:45.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858565+0000", 2024-09-17T21:21:45.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014630099999999999 2024-09-17T21:21:45.380 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.380 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:45.380 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.380 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12092 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:45.380 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:45.381 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.558498799000006, 2024-09-17T21:21:45.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089191555000000006, 2024-09-17T21:21:45.381 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:45.381 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:45.381 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:45.381 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:45.381 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:45.381 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:45.381 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12092 2024-09-17T21:21:45.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.381 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:45.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:45.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:45.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:45.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:45.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:45.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769816+0000", 2024-09-17T21:21:45.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7199999999999997e-07 2024-09-17T21:21:45.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:45.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769822+0000", 2024-09-17T21:21:45.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:21:45.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:45.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769838+0000", 2024-09-17T21:21:45.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6674e-05 2024-09-17T21:21:45.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769851+0000", 2024-09-17T21:21:45.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2527999999999999e-05 2024-09-17T21:21:45.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858660+0000", 2024-09-17T21:21:45.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088808556999999996 2024-09-17T21:21:45.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:45.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858817+0000", 2024-09-17T21:21:45.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015730400000000001 2024-09-17T21:21:45.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:45.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859007+0000", 2024-09-17T21:21:45.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018974899999999999 2024-09-17T21:21:45.385 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.385 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:45.385 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.385 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12093 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:45.385 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:45.385 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.558390695, 2024-09-17T21:21:45.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089429710999999995, 2024-09-17T21:21:45.385 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:45.385 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:45.386 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:45.386 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:45.386 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:45.386 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:45.386 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12093 2024-09-17T21:21:45.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.386 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:45.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:45.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:45.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:45.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:45.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:45.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769924+0000", 2024-09-17T21:21:45.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.342e-06 2024-09-17T21:21:45.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:45.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769929+0000", 2024-09-17T21:21:45.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0350000000000001e-06 2024-09-17T21:21:45.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:45.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769946+0000", 2024-09-17T21:21:45.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6793999999999999e-05 2024-09-17T21:21:45.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769959+0000", 2024-09-17T21:21:45.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3062e-05 2024-09-17T21:21:45.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859075+0000", 2024-09-17T21:21:45.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089115696999999994 2024-09-17T21:21:45.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:45.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859202+0000", 2024-09-17T21:21:45.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012657 2024-09-17T21:21:45.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:45.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859353+0000", 2024-09-17T21:21:45.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151211 2024-09-17T21:21:45.389 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.390 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:45.390 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.390 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12094 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:45.390 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:45.390 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.558336518999994, 2024-09-17T21:21:45.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089695001999999996, 2024-09-17T21:21:45.390 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:45.390 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:45.390 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:45.390 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:45.390 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:45.391 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:45.391 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12094 2024-09-17T21:21:45.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.391 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:45.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:45.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:45.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:45.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:45.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:45.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769978+0000", 2024-09-17T21:21:45.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.142e-06 2024-09-17T21:21:45.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:45.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769983+0000", 2024-09-17T21:21:45.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.386e-06 2024-09-17T21:21:45.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:45.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769997+0000", 2024-09-17T21:21:45.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.416e-05 2024-09-17T21:21:45.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770012+0000", 2024-09-17T21:21:45.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5384999999999999e-05 2024-09-17T21:21:45.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859405+0000", 2024-09-17T21:21:45.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089392376999999995 2024-09-17T21:21:45.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:45.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859525+0000", 2024-09-17T21:21:45.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120555 2024-09-17T21:21:45.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:45.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859672+0000", 2024-09-17T21:21:45.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000146997 2024-09-17T21:21:45.394 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.394 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:45.394 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12095 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.558284032000003, 2024-09-17T21:21:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090073476, 2024-09-17T21:21:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12095 2024-09-17T21:21:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:45.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:45.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:45.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:45.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:45.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770031+0000", 2024-09-17T21:21:45.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1699999999999997e-07 2024-09-17T21:21:45.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:45.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770036+0000", 2024-09-17T21:21:45.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.023e-06 2024-09-17T21:21:45.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:45.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770051+0000", 2024-09-17T21:21:45.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4953999999999999e-05 2024-09-17T21:21:45.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770063+0000", 2024-09-17T21:21:45.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:21:45.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859756+0000", 2024-09-17T21:21:45.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089692980000000005 2024-09-17T21:21:45.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:45.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859891+0000", 2024-09-17T21:21:45.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013459499999999999 2024-09-17T21:21:45.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:45.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860103+0000", 2024-09-17T21:21:45.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021274400000000001 2024-09-17T21:21:45.399 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.399 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:45.399 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.399 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12096 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:45.399 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:45.399 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.558234346999996, 2024-09-17T21:21:45.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090358274000000002, 2024-09-17T21:21:45.399 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:45.399 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:45.399 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:45.399 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:45.399 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:45.400 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:45.400 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12096 2024-09-17T21:21:45.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.400 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:45.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:45.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:45.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:45.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:45.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:45.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770080+0000", 2024-09-17T21:21:45.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9400000000000004e-07 2024-09-17T21:21:45.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:45.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770085+0000", 2024-09-17T21:21:45.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2169999999999996e-06 2024-09-17T21:21:45.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:45.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770100+0000", 2024-09-17T21:21:45.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5077e-05 2024-09-17T21:21:45.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770112+0000", 2024-09-17T21:21:45.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1951e-05 2024-09-17T21:21:45.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860160+0000", 2024-09-17T21:21:45.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090048413999999993 2024-09-17T21:21:45.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:45.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860285+0000", 2024-09-17T21:21:45.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012466699999999999 2024-09-17T21:21:45.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:45.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860438+0000", 2024-09-17T21:21:45.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015305399999999999 2024-09-17T21:21:45.403 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.403 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:45.403 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.403 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12097 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:45.403 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:45.403 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.558186599999999, 2024-09-17T21:21:45.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090702375000000002, 2024-09-17T21:21:45.404 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:45.404 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:45.404 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:45.404 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:45.404 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:45.404 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:45.404 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12097 2024-09-17T21:21:45.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.404 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:45.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:45.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:45.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:45.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:45.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:45.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770128+0000", 2024-09-17T21:21:45.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.04e-06 2024-09-17T21:21:45.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:45.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770133+0000", 2024-09-17T21:21:45.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8999999999999997e-06 2024-09-17T21:21:45.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:45.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770152+0000", 2024-09-17T21:21:45.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8890999999999999e-05 2024-09-17T21:21:45.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770164+0000", 2024-09-17T21:21:45.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1769e-05 2024-09-17T21:21:45.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860511+0000", 2024-09-17T21:21:45.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090347527999999996 2024-09-17T21:21:45.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:45.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860652+0000", 2024-09-17T21:21:45.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140894 2024-09-17T21:21:45.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:45.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860830+0000", 2024-09-17T21:21:45.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017735299999999999 2024-09-17T21:21:45.408 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.408 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:45.408 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.408 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12098 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:45.408 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:45.408 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.558133131999995, 2024-09-17T21:21:45.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090997262999999995, 2024-09-17T21:21:45.408 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:45.408 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:45.409 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:45.409 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:45.409 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:45.409 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:45.409 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12098 2024-09-17T21:21:45.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.409 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:45.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:45.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:45.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:45.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:45.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:45.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775465+0000", 2024-09-17T21:21:45.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0052843350000000002 2024-09-17T21:21:45.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:45.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775476+0000", 2024-09-17T21:21:45.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0797e-05 2024-09-17T21:21:45.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:45.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775506+0000", 2024-09-17T21:21:45.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9709999999999998e-05 2024-09-17T21:21:45.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775530+0000", 2024-09-17T21:21:45.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4528999999999998e-05 2024-09-17T21:21:45.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860894+0000", 2024-09-17T21:21:45.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.085364195000000004 2024-09-17T21:21:45.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:45.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861019+0000", 2024-09-17T21:21:45.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012467399999999999 2024-09-17T21:21:45.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:45.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861178+0000", 2024-09-17T21:21:45.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159023 2024-09-17T21:21:45.412 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.412 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:45.412 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12104 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.552432022999994, 2024-09-17T21:21:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087969628999999994, 2024-09-17T21:21:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12104 2024-09-17T21:21:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:45.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:45.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:45.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:45.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:45.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775883+0000", 2024-09-17T21:21:45.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3399999999999998e-07 2024-09-17T21:21:45.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775886+0000", 2024-09-17T21:21:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2119999999999999e-06 2024-09-17T21:21:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775895+0000", 2024-09-17T21:21:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8710000000000003e-06 2024-09-17T21:21:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775901+0000", 2024-09-17T21:21:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:21:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863544+0000", 2024-09-17T21:21:45.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087642642000000007 2024-09-17T21:21:45.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:45.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863694+0000", 2024-09-17T21:21:45.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014952300000000001 2024-09-17T21:21:45.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:45.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863851+0000", 2024-09-17T21:21:45.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000157804 2024-09-17T21:21:45.417 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.417 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:45.417 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.417 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12105 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:45.417 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:45.417 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.552403128999998, 2024-09-17T21:21:45.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088330400000000003, 2024-09-17T21:21:45.417 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:45.417 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:45.417 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:45.417 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:45.417 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:45.418 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:45.418 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12105 2024-09-17T21:21:45.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.418 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:45.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:45.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:45.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:45.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:45.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9999999999999997e-07 2024-09-17T21:21:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775914+0000", 2024-09-17T21:21:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4710000000000001e-06 2024-09-17T21:21:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775921+0000", 2024-09-17T21:21:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:21:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775928+0000", 2024-09-17T21:21:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:21:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863923+0000", 2024-09-17T21:21:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087995126000000007 2024-09-17T21:21:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864059+0000", 2024-09-17T21:21:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135797 2024-09-17T21:21:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.421 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:45.421 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864241+0000", 2024-09-17T21:21:45.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018252599999999999 2024-09-17T21:21:45.421 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.421 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:45.421 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.421 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12106 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:45.421 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:45.421 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.552313951000002, 2024-09-17T21:21:45.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088586394999999998, 2024-09-17T21:21:45.421 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:45.422 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:45.422 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:45.422 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:45.422 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:45.422 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:45.422 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12106 2024-09-17T21:21:45.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.422 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:45.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:45.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:45.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:45.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:45.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:45.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776001+0000", 2024-09-17T21:21:45.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4300000000000001e-07 2024-09-17T21:21:45.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:45.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776004+0000", 2024-09-17T21:21:45.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8049999999999998e-06 2024-09-17T21:21:45.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:45.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776018+0000", 2024-09-17T21:21:45.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4613999999999999e-05 2024-09-17T21:21:45.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776025+0000", 2024-09-17T21:21:45.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:21:45.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864279+0000", 2024-09-17T21:21:45.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088253570000000003 2024-09-17T21:21:45.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:45.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864433+0000", 2024-09-17T21:21:45.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015469400000000001 2024-09-17T21:21:45.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.426 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:45.426 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864586+0000", 2024-09-17T21:21:45.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000153003 2024-09-17T21:21:45.426 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.426 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:45.426 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.426 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25033 3.6 3.b74753e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e168)", 2024-09-17T21:21:45.426 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:21:45.426 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 19.818870687, 2024-09-17T21:21:45.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93781952099999999, 2024-09-17T21:21:45.427 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:45.427 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:45.427 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:45.427 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:45.427 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:21:45.427 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:45.427 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25033 2024-09-17T21:21:45.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.427 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:45.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:45.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:21:45.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:45.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:21:45.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:45.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509444+0000", 2024-09-17T21:21:45.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0020000000000001e-06 2024-09-17T21:21:45.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:45.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509450+0000", 2024-09-17T21:21:45.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4249999999999997e-06 2024-09-17T21:21:45.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:45.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509460+0000", 2024-09-17T21:21:45.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0183999999999999e-05 2024-09-17T21:21:45.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509466+0000", 2024-09-17T21:21:45.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0039999999999996e-06 2024-09-17T21:21:45.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509540+0000", 2024-09-17T21:21:45.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3847e-05 2024-09-17T21:21:45.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:45.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509551+0000", 2024-09-17T21:21:45.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1731e-05 2024-09-17T21:21:45.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.431 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.447243+0000", 2024-09-17T21:21:45.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93769166000000004 2024-09-17T21:21:45.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.431 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:45.431 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.447263+0000", 2024-09-17T21:21:45.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9667999999999999e-05 2024-09-17T21:21:45.431 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.431 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:45.431 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.431 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25033 3.6 3:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e168)", 2024-09-17T21:21:45.431 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:21:45.432 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 19.81734509, 2024-09-17T21:21:45.432 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.937309383, 2024-09-17T21:21:45.432 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:45.432 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:45.432 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:45.432 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:45.432 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:21:45.432 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:45.432 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25033 2024-09-17T21:21:45.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.432 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:45.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:45.432 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:21:45.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:45.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:21:45.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:45.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:45.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510970+0000", 2024-09-17T21:21:45.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.564e-06 2024-09-17T21:21:45.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:45.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510978+0000", 2024-09-17T21:21:45.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.362e-06 2024-09-17T21:21:45.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:45.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510998+0000", 2024-09-17T21:21:45.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0336999999999999e-05 2024-09-17T21:21:45.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511010+0000", 2024-09-17T21:21:45.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1786e-05 2024-09-17T21:21:45.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511080+0000", 2024-09-17T21:21:45.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0686999999999996e-05 2024-09-17T21:21:45.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:45.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511087+0000", 2024-09-17T21:21:45.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4899999999999997e-06 2024-09-17T21:21:45.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:45.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.448050+0000", 2024-09-17T21:21:45.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93696351700000002 2024-09-17T21:21:45.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:45.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:45.436 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:45.436 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.448278+0000", 2024-09-17T21:21:45.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022764000000000001 2024-09-17T21:21:45.436 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.436 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:45.436 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.436 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:45.436 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:45.436 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:45.436 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 0] out_osds: [2, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:21:45.436 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:21:45.437 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [0, 1, 2, 3, 4, 5] 2024-09-17T21:21:45.437 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-17T21:21:45.437 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:21:45.439 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:21:45.439 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-17T21:21:45.440 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:45.435+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:45.452 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:21:45.517 INFO:tasks.ceph.osd.2:Stopped 2024-09-17T21:21:45.517 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.2 2024-09-17T21:21:45.517 INFO:tasks.thrashosds.thrasher:remote for osd 2 is ubuntu@smithi083.front.sepia.ceph.com 2024-09-17T21:21:45.518 DEBUG:teuthology.orchestra.run.smithi083:> type ceph-objectstore-tool 2024-09-17T21:21:45.565 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:45.563+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:45.568 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T21:21:45.568 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --op list-pgs 2024-09-17T21:21:45.570 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:45.570 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:45.570 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:45.570 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:45.584 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:21:45.631 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:45.632 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:45.640 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:45.635+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:45.649 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:45.706 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:45.706 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:45.706 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:45.706 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:45.706 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:45.720 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:21:45.741 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:45.739+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:45.842 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:45.837+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:45.843 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:45.843 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:45.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:45.843 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:45.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.843 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:21:45.843 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:45.843 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 228.826178, 2024-09-17T21:21:45.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:21:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:21:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:21:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:21:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:45.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:45.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:45.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:45.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:21:45.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:21:45.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:45.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:21:45.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:21:45.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:45.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:21:45.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:21:45.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:21:45.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:21:45.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:21:45.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:21:45.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:45.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:21:45.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:21:45.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:21:45.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:21:45.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:45.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:21:45.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:21:45.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:21:45.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:21:45.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:21:45.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:45.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:21:45.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:21:45.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:45.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:21:45.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:21:45.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:45.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:21:45.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:21:45.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:45.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:21:45.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:21:45.850 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.851 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:45.851 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.851 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:45.851 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:45.851 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 209.24601372000001, 2024-09-17T21:21:45.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:21:45.851 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:45.851 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:45.851 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:45.851 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:45.852 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:21:45.852 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:21:45.852 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:21:45.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.852 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:45.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:45.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:45.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:45.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:45.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:45.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:21:45.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:21:45.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:45.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:21:45.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:21:45.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:45.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:21:45.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:21:45.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:21:45.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:21:45.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:21:45.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:21:45.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:45.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:21:45.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:21:45.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:21:45.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:21:45.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:45.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:21:45.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:21:45.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:21:45.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:21:45.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:45.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:21:45.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:21:45.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:45.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:21:45.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:21:45.857 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.857 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:45.857 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.858 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:45.858 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:45.858 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 209.245916408, 2024-09-17T21:21:45.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:21:45.858 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:45.858 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:45.858 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:45.858 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:45.858 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:45.858 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:45.858 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:21:45.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.859 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:45.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:45.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:45.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:45.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:45.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:45.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:21:45.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:21:45.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:45.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:21:45.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:21:45.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:45.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:21:45.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:21:45.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:21:45.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:21:45.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:21:45.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:21:45.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:45.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:21:45.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:21:45.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:21:45.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:21:45.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:45.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:21:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:21:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:21:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:21:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:21:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:21:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:21:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:21:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:21:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:21:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:21:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:21:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:21:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 209.24382960700001, 2024-09-17T21:21:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:21:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:21:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:21:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:21:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:21:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:21:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:21:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:21:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:21:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:21:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:21:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:21:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:21:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:21:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:21:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:21:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:21:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:21:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:21:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:45.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:21:45.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:21:45.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:21:45.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:21:45.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:45.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:21:45.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:21:45.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:45.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:21:45.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:21:45.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:45.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:21:45.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:21:45.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:45.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:21:45.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:21:45.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:45.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:21:45.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:21:45.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:45.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:21:45.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:21:45.873 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.873 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:45.873 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.873 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:45.873 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:45.873 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 176.559930826, 2024-09-17T21:21:45.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:21:45.873 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:45.873 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:45.873 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:45.874 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:45.874 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:45.874 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:45.874 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:21:45.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.874 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:45.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.874 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:45.874 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:45.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.874 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:45.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:45.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:45.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:21:45.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:21:45.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:45.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:21:45.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:21:45.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:45.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:21:45.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:21:45.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:21:45.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:21:45.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:21:45.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:21:45.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:45.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:21:45.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:21:45.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:21:45.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:21:45.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:45.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:21:45.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:21:45.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:21:45.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:21:45.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:45.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:21:45.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:21:45.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:45.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:21:45.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:21:45.879 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.879 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:45.879 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.880 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:45.880 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:45.880 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 176.550839115, 2024-09-17T21:21:45.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:21:45.880 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:45.880 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:45.880 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:45.880 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:45.880 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:45.880 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:45.880 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:21:45.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.881 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:45.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:45.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:45.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:45.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:45.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:45.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:21:45.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:21:45.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:45.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:21:45.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:21:45.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:45.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:21:45.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:21:45.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:21:45.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:21:45.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:21:45.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:21:45.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:45.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:21:45.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:21:45.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:21:45.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:21:45.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:45.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:21:45.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:21:45.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:45.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:21:45.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:21:45.885 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.885 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:45.885 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.885 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:45.886 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:45.886 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 176.55066706400001, 2024-09-17T21:21:45.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:21:45.886 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:45.886 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:45.886 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:45.886 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:45.886 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:45.886 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:45.886 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:21:45.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.886 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:45.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:45.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:45.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:45.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:45.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:45.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:21:45.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:21:45.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:45.888 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:21:45.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:21:45.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:45.888 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:21:45.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:21:45.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:21:45.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:21:45.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:21:45.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:21:45.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:45.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:21:45.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:21:45.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:21:45.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:21:45.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:45.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:21:45.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:21:45.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:45.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:21:45.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:21:45.891 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.891 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:45.891 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.891 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:45.891 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:45.892 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 176.375734317, 2024-09-17T21:21:45.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:21:45.892 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:45.892 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:45.892 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:45.892 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:45.892 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:45.892 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:45.892 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:21:45.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.892 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:45.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:45.893 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:45.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:45.893 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:45.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:45.893 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:21:45.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:21:45.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:45.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:21:45.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:21:45.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:45.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:21:45.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:21:45.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:21:45.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:21:45.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:21:45.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:21:45.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:45.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:21:45.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:21:45.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:21:45.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:21:45.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:45.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:21:45.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:21:45.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.897 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:45.897 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:21:45.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:21:45.897 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.897 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:45.897 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.897 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:45.897 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:45.898 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 176.37564975999999, 2024-09-17T21:21:45.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:21:45.898 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:45.898 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:45.898 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:45.898 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:45.898 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:45.898 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:45.898 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:21:45.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.898 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:45.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:45.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:45.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:45.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:45.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:45.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:45.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:21:45.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:45.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:21:45.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:21:45.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:45.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:21:45.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:21:45.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:21:45.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:21:45.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:21:45.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:21:45.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:45.902 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:21:45.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:21:45.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.902 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:21:45.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:21:45.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:45.902 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:21:45.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:21:45.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:45.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:21:45.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:21:45.903 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.903 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:45.903 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.903 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:45.903 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:45.904 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 176.37558412300001, 2024-09-17T21:21:45.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:21:45.904 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:45.904 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:45.904 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:45.904 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:45.904 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:45.904 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:45.904 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:21:45.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.904 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:45.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:45.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:45.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:45.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:45.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:45.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:21:45.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:21:45.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:45.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:21:45.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:21:45.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:45.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:21:45.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:21:45.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:21:45.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:21:45.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:21:45.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:21:45.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:45.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:21:45.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:21:45.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:21:45.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:21:45.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.909 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:45.909 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:21:45.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:21:45.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.909 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:45.909 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:21:45.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:21:45.909 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.909 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:45.910 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.910 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:45.910 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:45.910 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 125.002009026, 2024-09-17T21:21:45.910 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:21:45.910 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:45.910 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:45.910 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:45.910 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:45.910 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:21:45.910 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:45.911 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:21:45.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.911 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:45.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:45.911 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:45.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:45.911 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:45.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:45.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:21:45.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:21:45.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:45.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:21:45.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:21:45.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:45.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:21:45.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:21:45.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.913 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:21:45.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:21:45.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.913 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:21:45.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:21:45.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.914 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.914 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:45.914 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:21:45.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:21:45.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.914 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.914 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:45.914 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:21:45.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:21:45.914 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.914 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:45.914 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.915 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:45.915 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:45.915 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 125.001889514, 2024-09-17T21:21:45.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:21:45.915 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:45.915 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:45.915 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:45.915 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:45.915 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:21:45.915 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:45.915 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:21:45.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.916 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:45.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:45.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:45.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:45.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:45.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.917 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:45.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:21:45.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:21:45.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.917 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:45.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:21:45.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:21:45.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.918 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:45.918 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:21:45.918 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:21:45.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.918 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.918 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:21:45.918 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:21:45.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.918 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.918 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:21:45.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:21:45.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:45.919 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:21:45.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:21:45.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:45.919 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:21:45.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:21:45.919 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.919 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:45.920 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.920 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:996 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:45.920 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:45.920 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.408776142999997, 2024-09-17T21:21:45.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084493736999999999, 2024-09-17T21:21:45.920 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:45.920 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:45.920 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:45.920 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:45.920 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:45.920 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:45.921 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 996 2024-09-17T21:21:45.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.921 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:45.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:45.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:45.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:45.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:45.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:45.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431168+0000", 2024-09-17T21:21:45.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4e-07 2024-09-17T21:21:45.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:45.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431172+0000", 2024-09-17T21:21:45.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.608e-06 2024-09-17T21:21:45.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.923 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:45.923 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431180+0000", 2024-09-17T21:21:45.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5650000000000002e-06 2024-09-17T21:21:45.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.923 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.923 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431188+0000", 2024-09-17T21:21:45.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:21:45.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.923 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.923 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515354+0000", 2024-09-17T21:21:45.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084165977000000003 2024-09-17T21:21:45.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:45.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515451+0000", 2024-09-17T21:21:45.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6669000000000006e-05 2024-09-17T21:21:45.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:45.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515661+0000", 2024-09-17T21:21:45.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021021599999999999 2024-09-17T21:21:45.924 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.924 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:45.925 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.925 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:997 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 583680~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:45.925 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:45.925 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.408744748000004, 2024-09-17T21:21:45.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084772419000000002, 2024-09-17T21:21:45.925 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:45.925 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:45.925 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:45.925 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:45.925 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:45.925 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:45.926 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 997 2024-09-17T21:21:45.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.926 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:45.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:45.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:45.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:45.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:45.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:45.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:45.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0400000000000005e-07 2024-09-17T21:21:45.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:45.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431202+0000", 2024-09-17T21:21:45.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:21:45.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:45.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431208+0000", 2024-09-17T21:21:45.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4740000000000002e-06 2024-09-17T21:21:45.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431214+0000", 2024-09-17T21:21:45.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6550000000000001e-06 2024-09-17T21:21:45.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515741+0000", 2024-09-17T21:21:45.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084526904 2024-09-17T21:21:45.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:45.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515826+0000", 2024-09-17T21:21:45.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4643999999999998e-05 2024-09-17T21:21:45.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:45.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515971+0000", 2024-09-17T21:21:45.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145444 2024-09-17T21:21:45.930 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.930 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:45.930 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.930 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:998 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 587776~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:45.930 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:45.930 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.408721775999993, 2024-09-17T21:21:45.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085108565999999997, 2024-09-17T21:21:45.930 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:45.930 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:45.930 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:45.931 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:45.931 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:45.931 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:45.931 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 998 2024-09-17T21:21:45.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.931 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:45.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:45.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:45.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:45.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:45.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:45.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:45.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:21:45.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:45.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431224+0000", 2024-09-17T21:21:45.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:21:45.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:45.933 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431231+0000", 2024-09-17T21:21:45.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000002e-06 2024-09-17T21:21:45.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.933 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431236+0000", 2024-09-17T21:21:45.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.648e-06 2024-09-17T21:21:45.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516024+0000", 2024-09-17T21:21:45.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084787584999999999 2024-09-17T21:21:45.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:45.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516122+0000", 2024-09-17T21:21:45.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8076000000000001e-05 2024-09-17T21:21:45.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:45.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516330+0000", 2024-09-17T21:21:45.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020808 2024-09-17T21:21:45.935 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.935 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:45.935 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.935 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1000 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 591872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:45.935 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:45.935 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.408699661, 2024-09-17T21:21:45.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085389065, 2024-09-17T21:21:45.936 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:45.936 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:45.936 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:45.936 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:45.936 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:45.936 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:45.936 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1000 2024-09-17T21:21:45.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.936 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:45.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:45.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:45.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.937 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:45.937 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:45.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.937 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:45.937 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:45.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:21:45.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:45.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431246+0000", 2024-09-17T21:21:45.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1749999999999999e-06 2024-09-17T21:21:45.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:45.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431253+0000", 2024-09-17T21:21:45.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2790000000000004e-06 2024-09-17T21:21:45.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431258+0000", 2024-09-17T21:21:45.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3299999999999998e-06 2024-09-17T21:21:45.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.939 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.939 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.939 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516384+0000", 2024-09-17T21:21:45.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085125549999999994 2024-09-17T21:21:45.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.939 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.939 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:45.939 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516460+0000", 2024-09-17T21:21:45.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6359999999999994e-05 2024-09-17T21:21:45.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.939 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.940 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:45.940 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516633+0000", 2024-09-17T21:21:45.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017285900000000001 2024-09-17T21:21:45.940 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.940 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:45.940 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.940 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1001 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 595968~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:45.940 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:45.940 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.408676255000003, 2024-09-17T21:21:45.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085614715999999993, 2024-09-17T21:21:45.940 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:45.941 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:45.941 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:45.941 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:45.941 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:45.941 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:45.941 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1001 2024-09-17T21:21:45.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.941 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:45.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.941 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:45.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:45.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:45.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:45.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:45.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438066+0000", 2024-09-17T21:21:45.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067989490000000003 2024-09-17T21:21:45.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:45.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438073+0000", 2024-09-17T21:21:45.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8419999999999999e-06 2024-09-17T21:21:45.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:45.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438090+0000", 2024-09-17T21:21:45.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6657e-05 2024-09-17T21:21:45.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438099+0000", 2024-09-17T21:21:45.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0529999999999996e-06 2024-09-17T21:21:45.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.944 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.944 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516678+0000", 2024-09-17T21:21:45.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078579422999999995 2024-09-17T21:21:45.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.944 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:45.944 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516739+0000", 2024-09-17T21:21:45.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1315000000000005e-05 2024-09-17T21:21:45.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.944 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:45.944 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516882+0000", 2024-09-17T21:21:45.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014247700000000001 2024-09-17T21:21:45.945 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.945 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:45.945 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.945 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.945 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:19998 3.5 3:a4885450:::rbd_object_map.1152d139f9f9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-17T21:21:45.945 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:45.945 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 66.437409009999996, 2024-09-17T21:21:45.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90715489299999996, 2024-09-17T21:21:45.945 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:45.945 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:45.946 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:45.946 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:45.946 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:45.946 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:45.946 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19998 2024-09-17T21:21:45.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.946 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:45.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:45.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:45.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:45.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:45.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:45.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402537+0000", 2024-09-17T21:21:45.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5600000000000001e-06 2024-09-17T21:21:45.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:45.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402549+0000", 2024-09-17T21:21:45.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1929999999999999e-05 2024-09-17T21:21:45.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:45.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402579+0000", 2024-09-17T21:21:45.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0093000000000001e-05 2024-09-17T21:21:45.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402595+0000", 2024-09-17T21:21:45.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5855999999999999e-05 2024-09-17T21:21:45.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402731+0000", 2024-09-17T21:21:45.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013610799999999999 2024-09-17T21:21:45.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:45.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402750+0000", 2024-09-17T21:21:45.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8737e-05 2024-09-17T21:21:45.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.305977+0000", 2024-09-17T21:21:45.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90322768399999998 2024-09-17T21:21:45.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.950 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.950 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:45.950 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306236+0000", 2024-09-17T21:21:45.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000258111 2024-09-17T21:21:45.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.950 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.950 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:45.950 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306557+0000", 2024-09-17T21:21:45.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032159799999999998 2024-09-17T21:21:45.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:45.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.308286+0000", 2024-09-17T21:21:45.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017292150000000001 2024-09-17T21:21:45.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:45.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309530+0000", 2024-09-17T21:21:45.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00124409 2024-09-17T21:21:45.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:45.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309630+0000", 2024-09-17T21:21:45.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9689e-05 2024-09-17T21:21:45.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:45.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309689+0000", 2024-09-17T21:21:45.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9222000000000003e-05 2024-09-17T21:21:45.952 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.952 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:45.952 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.953 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:25277 3.1f 3:ff5d7cd5:::rbd_header.11524f65d4ff:head [watch ping cookie 139860845478480 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-17T21:21:45.953 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:21:45.953 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 16.254452941, 2024-09-17T21:21:45.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.92185720000000004, 2024-09-17T21:21:45.953 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:45.953 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:45.953 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:45.953 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:45.953 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:45.953 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:45.953 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 25277 2024-09-17T21:21:45.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.954 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:45.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.954 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:45.954 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:21:45.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.954 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:45.954 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:21:45.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.954 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:45.955 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585493+0000", 2024-09-17T21:21:45.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:21:45.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.955 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:45.955 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585505+0000", 2024-09-17T21:21:45.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2466000000000001e-05 2024-09-17T21:21:45.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.955 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:45.955 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585537+0000", 2024-09-17T21:21:45.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1183000000000003e-05 2024-09-17T21:21:45.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.956 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.956 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585553+0000", 2024-09-17T21:21:45.956 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.662e-05 2024-09-17T21:21:45.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.956 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.956 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585696+0000", 2024-09-17T21:21:45.956 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014293800000000001 2024-09-17T21:21:45.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.956 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:45.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585718+0000", 2024-09-17T21:21:45.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1860000000000001e-05 2024-09-17T21:21:45.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.501188+0000", 2024-09-17T21:21:45.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.91547051300000004 2024-09-17T21:21:45.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:21:45.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.501260+0000", 2024-09-17T21:21:45.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1571999999999998e-05 2024-09-17T21:21:45.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507002+0000", 2024-09-17T21:21:45.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057414850000000002 2024-09-17T21:21:45.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:45.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507143+0000", 2024-09-17T21:21:45.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014140299999999999 2024-09-17T21:21:45.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:45.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507348+0000", 2024-09-17T21:21:45.959 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000204773 2024-09-17T21:21:45.959 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.959 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:45.959 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.959 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:26042 3.f 3:f24134c1:::rbd_id.image_client.0-clone34:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e186)", 2024-09-17T21:21:45.959 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:21:45.959 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.3537645729999999, 2024-09-17T21:21:45.959 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.91185481000000002, 2024-09-17T21:21:45.959 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:45.959 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:45.959 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:45.960 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:45.960 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:45.960 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:45.960 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 26042 2024-09-17T21:21:45.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.960 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:45.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.960 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:45.960 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:21:45.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.960 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:45.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:21:45.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:45.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:45.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486180+0000", 2024-09-17T21:21:45.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5990000000000001e-06 2024-09-17T21:21:45.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:45.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486187+0000", 2024-09-17T21:21:45.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7859999999999999e-06 2024-09-17T21:21:45.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.962 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:45.962 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486206+0000", 2024-09-17T21:21:45.962 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8641e-05 2024-09-17T21:21:45.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.962 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:45.962 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486219+0000", 2024-09-17T21:21:45.962 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.308e-05 2024-09-17T21:21:45.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.963 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486283+0000", 2024-09-17T21:21:45.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4084000000000005e-05 2024-09-17T21:21:45.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:45.963 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486290+0000", 2024-09-17T21:21:45.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.584e-06 2024-09-17T21:21:45.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:45.963 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392182+0000", 2024-09-17T21:21:45.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90589206499999997 2024-09-17T21:21:45.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:45.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392518+0000", 2024-09-17T21:21:45.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033596 2024-09-17T21:21:45.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:21:45.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392802+0000", 2024-09-17T21:21:45.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028394999999999999 2024-09-17T21:21:45.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.965 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:45.965 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397705+0000", 2024-09-17T21:21:45.965 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0049031769999999999 2024-09-17T21:21:45.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.965 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:45.965 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397770+0000", 2024-09-17T21:21:45.965 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4765999999999996e-05 2024-09-17T21:21:45.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.965 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:45.965 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397927+0000", 2024-09-17T21:21:45.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015699900000000001 2024-09-17T21:21:45.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:45.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:45.966 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:45.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.398034+0000", 2024-09-17T21:21:45.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000107119 2024-09-17T21:21:45.966 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.966 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:45.966 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.966 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:45.966 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:45.966 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:45.968 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:21:45.968 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:45.965+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:46.068 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:46.065+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:46.138 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:46.138 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:46.138 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:46.138 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:46.153 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:21:46.195 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:46.193+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:46.271 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:46.267+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:46.317 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:46.317 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:46.317 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:46.317 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:46.317 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:46.332 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:21:46.379 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:46.377+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:46.456 INFO:tasks.thrashosds.thrasher:No PGs found for osd.2 2024-09-17T21:21:46.461 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:46.461 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:46.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:46.461 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:46.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.462 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:18003 3.1d 3:b9765580:::rbd_header.11527d2b15f6:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T21:21:46.462 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:46.462 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 94.098980202999996, 2024-09-17T21:21:46.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94165346900000002, 2024-09-17T21:21:46.462 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:46.462 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:46.462 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:46.462 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:46.462 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:46.462 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:46.462 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18003 2024-09-17T21:21:46.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.462 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:46.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:46.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:46.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:46.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:46.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:46.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357880+0000", 2024-09-17T21:21:46.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:21:46.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:46.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357891+0000", 2024-09-17T21:21:46.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0417999999999999e-05 2024-09-17T21:21:46.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:46.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357913+0000", 2024-09-17T21:21:46.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2215e-05 2024-09-17T21:21:46.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:46.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357922+0000", 2024-09-17T21:21:46.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4779999999999997e-06 2024-09-17T21:21:46.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358023+0000", 2024-09-17T21:21:46.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101025 2024-09-17T21:21:46.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:46.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358040+0000", 2024-09-17T21:21:46.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6291999999999999e-05 2024-09-17T21:21:46.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298203+0000", 2024-09-17T21:21:46.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94016297500000001 2024-09-17T21:21:46.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:46.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298626+0000", 2024-09-17T21:21:46.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000422853 2024-09-17T21:21:46.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:46.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.299532+0000", 2024-09-17T21:21:46.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00090657300000000004 2024-09-17T21:21:46.466 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.467 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:46.467 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.467 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15707 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 982016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:46.467 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:46.467 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.935814956000002, 2024-09-17T21:21:46.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092305228000000003, 2024-09-17T21:21:46.467 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:46.467 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:46.467 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:46.467 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:46.467 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:46.468 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:46.468 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15707 2024-09-17T21:21:46.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.468 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:46.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:46.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:46.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:46.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:46.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:46.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521045+0000", 2024-09-17T21:21:46.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4899999999999999e-06 2024-09-17T21:21:46.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:46.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521066+0000", 2024-09-17T21:21:46.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0236000000000001e-05 2024-09-17T21:21:46.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:46.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521085+0000", 2024-09-17T21:21:46.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8819999999999999e-05 2024-09-17T21:21:46.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:46.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521100+0000", 2024-09-17T21:21:46.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5146e-05 2024-09-17T21:21:46.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612978+0000", 2024-09-17T21:21:46.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091878847 2024-09-17T21:21:46.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:46.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613079+0000", 2024-09-17T21:21:46.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100503 2024-09-17T21:21:46.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:46.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613349+0000", 2024-09-17T21:21:46.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000270186 2024-09-17T21:21:46.471 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.471 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:46.471 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.472 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15708 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 986112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:46.472 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:46.472 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.935736781000003, 2024-09-17T21:21:46.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092547217000000001, 2024-09-17T21:21:46.472 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:46.472 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:46.472 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:46.472 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:46.472 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:46.472 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:46.472 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15708 2024-09-17T21:21:46.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.473 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:46.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:46.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:46.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:46.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:46.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:46.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521123+0000", 2024-09-17T21:21:46.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.353e-06 2024-09-17T21:21:46.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:46.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521129+0000", 2024-09-17T21:21:46.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:21:46.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:46.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521145+0000", 2024-09-17T21:21:46.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.611e-05 2024-09-17T21:21:46.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:46.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521154+0000", 2024-09-17T21:21:46.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4500000000000004e-06 2024-09-17T21:21:46.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613397+0000", 2024-09-17T21:21:46.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092243275 2024-09-17T21:21:46.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:46.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613503+0000", 2024-09-17T21:21:46.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106013 2024-09-17T21:21:46.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:46.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613669+0000", 2024-09-17T21:21:46.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016642299999999999 2024-09-17T21:21:46.476 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.476 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:46.476 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.476 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15709 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 990208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:46.476 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:46.477 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.935693764999996, 2024-09-17T21:21:46.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092728678999999994, 2024-09-17T21:21:46.477 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:46.477 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:46.477 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:46.477 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:46.477 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:46.477 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:46.477 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15709 2024-09-17T21:21:46.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.477 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:46.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:46.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:46.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:46.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:46.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:46.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521166+0000", 2024-09-17T21:21:46.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7000000000000004e-07 2024-09-17T21:21:46.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:46.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521169+0000", 2024-09-17T21:21:46.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2660000000000002e-06 2024-09-17T21:21:46.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:46.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521181+0000", 2024-09-17T21:21:46.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.208e-05 2024-09-17T21:21:46.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:46.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521194+0000", 2024-09-17T21:21:46.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2707e-05 2024-09-17T21:21:46.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613695+0000", 2024-09-17T21:21:46.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092500709 2024-09-17T21:21:46.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:46.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613750+0000", 2024-09-17T21:21:46.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.503e-05 2024-09-17T21:21:46.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:46.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613894+0000", 2024-09-17T21:21:46.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144117 2024-09-17T21:21:46.481 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.481 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:46.481 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.481 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15710 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 994304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:46.481 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:46.481 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.935649198999997, 2024-09-17T21:21:46.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092957729000000003, 2024-09-17T21:21:46.482 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:46.482 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:46.482 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:46.482 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:46.482 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:46.482 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:46.482 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15710 2024-09-17T21:21:46.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.482 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:46.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:46.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:46.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:46.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:46.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:46.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:46.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8000000000000005e-07 2024-09-17T21:21:46.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:46.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521216+0000", 2024-09-17T21:21:46.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3859999999999998e-06 2024-09-17T21:21:46.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:46.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521234+0000", 2024-09-17T21:21:46.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:21:46.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:46.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521248+0000", 2024-09-17T21:21:46.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:21:46.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613926+0000", 2024-09-17T21:21:46.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092678324000000006 2024-09-17T21:21:46.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:46.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613974+0000", 2024-09-17T21:21:46.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7754999999999999e-05 2024-09-17T21:21:46.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:46.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614167+0000", 2024-09-17T21:21:46.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193508 2024-09-17T21:21:46.486 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.486 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:46.486 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.486 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15711 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 998400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:46.486 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:46.486 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.935591817000002, 2024-09-17T21:21:46.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093268000000000004, 2024-09-17T21:21:46.486 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:46.486 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:46.486 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:46.486 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:46.487 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:46.487 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:46.487 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15711 2024-09-17T21:21:46.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.487 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:46.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:46.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:46.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:46.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:46.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:46.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521268+0000", 2024-09-17T21:21:46.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-17T21:21:46.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:46.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521273+0000", 2024-09-17T21:21:46.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:21:46.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:46.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521290+0000", 2024-09-17T21:21:46.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7269999999999999e-05 2024-09-17T21:21:46.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:46.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521304+0000", 2024-09-17T21:21:46.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3626e-05 2024-09-17T21:21:46.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614220+0000", 2024-09-17T21:21:46.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092916367 2024-09-17T21:21:46.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:46.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614322+0000", 2024-09-17T21:21:46.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102212 2024-09-17T21:21:46.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:46.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614535+0000", 2024-09-17T21:21:46.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021266199999999999 2024-09-17T21:21:46.490 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.490 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:46.490 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.491 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15712 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1002496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:46.491 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:46.491 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.935536322000004, 2024-09-17T21:21:46.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093568707000000001, 2024-09-17T21:21:46.491 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:46.491 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:46.491 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:46.491 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:46.491 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:46.491 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:46.492 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15712 2024-09-17T21:21:46.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.492 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:46.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:46.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:46.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:46.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:46.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:46.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521324+0000", 2024-09-17T21:21:46.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.17e-06 2024-09-17T21:21:46.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:46.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521334+0000", 2024-09-17T21:21:46.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9369999999999999e-06 2024-09-17T21:21:46.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:46.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521356+0000", 2024-09-17T21:21:46.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2045999999999999e-05 2024-09-17T21:21:46.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:46.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521370+0000", 2024-09-17T21:21:46.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4270000000000001e-05 2024-09-17T21:21:46.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614583+0000", 2024-09-17T21:21:46.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093213220999999999 2024-09-17T21:21:46.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:46.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614680+0000", 2024-09-17T21:21:46.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6900999999999999e-05 2024-09-17T21:21:46.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:46.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614891+0000", 2024-09-17T21:21:46.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021116200000000001 2024-09-17T21:21:46.495 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.495 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:46.495 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.495 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15713 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1006592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:46.496 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:46.496 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.935469620000006, 2024-09-17T21:21:46.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093834641999999996, 2024-09-17T21:21:46.496 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:46.496 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:46.496 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:46.496 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:46.496 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:46.496 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:46.496 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15713 2024-09-17T21:21:46.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.496 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:46.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:46.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:46.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:46.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:46.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:46.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521390+0000", 2024-09-17T21:21:46.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:21:46.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:46.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521396+0000", 2024-09-17T21:21:46.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:21:46.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:46.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521417+0000", 2024-09-17T21:21:46.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1236000000000001e-05 2024-09-17T21:21:46.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:46.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521431+0000", 2024-09-17T21:21:46.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3873000000000001e-05 2024-09-17T21:21:46.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614938+0000", 2024-09-17T21:21:46.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093506522999999994 2024-09-17T21:21:46.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:46.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615025+0000", 2024-09-17T21:21:46.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7635999999999999e-05 2024-09-17T21:21:46.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:46.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615224+0000", 2024-09-17T21:21:46.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019844700000000001 2024-09-17T21:21:46.500 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.500 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:46.500 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.500 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15714 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:46.500 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:46.501 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.935406364000002, 2024-09-17T21:21:46.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094071925000000001, 2024-09-17T21:21:46.501 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:46.501 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:46.501 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:46.501 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:46.501 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:46.501 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:46.501 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15714 2024-09-17T21:21:46.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.501 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:46.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:46.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:46.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:46.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:46.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:46.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521454+0000", 2024-09-17T21:21:46.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.097e-06 2024-09-17T21:21:46.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:46.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521459+0000", 2024-09-17T21:21:46.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2660000000000001e-06 2024-09-17T21:21:46.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:46.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521481+0000", 2024-09-17T21:21:46.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1878999999999998e-05 2024-09-17T21:21:46.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:46.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521498+0000", 2024-09-17T21:21:46.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6807e-05 2024-09-17T21:21:46.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615271+0000", 2024-09-17T21:21:46.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093773286999999997 2024-09-17T21:21:46.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:46.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615366+0000", 2024-09-17T21:21:46.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5322000000000007e-05 2024-09-17T21:21:46.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:46.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615524+0000", 2024-09-17T21:21:46.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015826699999999999 2024-09-17T21:21:46.505 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.505 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:46.505 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.505 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15715 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1014784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:46.505 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:46.505 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.935342028999997, 2024-09-17T21:21:46.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094243840999999995, 2024-09-17T21:21:46.505 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:46.505 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:46.505 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:46.505 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:46.506 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:46.506 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:46.506 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15715 2024-09-17T21:21:46.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.506 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:46.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:46.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:46.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:46.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:46.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:46.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521518+0000", 2024-09-17T21:21:46.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4300000000000001e-06 2024-09-17T21:21:46.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:46.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521524+0000", 2024-09-17T21:21:46.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5860000000000004e-06 2024-09-17T21:21:46.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:46.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521541+0000", 2024-09-17T21:21:46.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7574000000000001e-05 2024-09-17T21:21:46.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:46.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521555+0000", 2024-09-17T21:21:46.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3540000000000001e-05 2024-09-17T21:21:46.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615560+0000", 2024-09-17T21:21:46.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.09400472 2024-09-17T21:21:46.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:46.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615614+0000", 2024-09-17T21:21:46.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4693e-05 2024-09-17T21:21:46.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:46.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615761+0000", 2024-09-17T21:21:46.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000146298 2024-09-17T21:21:46.509 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.509 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:46.509 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.509 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15716 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1018880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:46.510 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:46.510 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.935285683999993, 2024-09-17T21:21:46.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094400676000000003, 2024-09-17T21:21:46.510 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:46.510 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:46.510 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:46.510 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:46.510 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:46.510 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:46.510 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15716 2024-09-17T21:21:46.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.510 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:46.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:46.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:46.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:46.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:46.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:46.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521574+0000", 2024-09-17T21:21:46.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1939999999999999e-06 2024-09-17T21:21:46.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:46.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521579+0000", 2024-09-17T21:21:46.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:21:46.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:46.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521597+0000", 2024-09-17T21:21:46.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8074e-05 2024-09-17T21:21:46.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:46.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521613+0000", 2024-09-17T21:21:46.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5316e-05 2024-09-17T21:21:46.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615780+0000", 2024-09-17T21:21:46.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094166874999999997 2024-09-17T21:21:46.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:46.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615836+0000", 2024-09-17T21:21:46.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6926999999999999e-05 2024-09-17T21:21:46.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:46.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615974+0000", 2024-09-17T21:21:46.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013741400000000001 2024-09-17T21:21:46.514 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.514 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:46.514 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.514 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15717 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1022976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:46.514 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:46.514 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.935226987999997, 2024-09-17T21:21:46.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094586723999999997, 2024-09-17T21:21:46.514 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:46.515 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:46.515 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:46.515 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:46.515 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:46.515 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:46.515 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15717 2024-09-17T21:21:46.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.515 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:46.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:46.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:46.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:46.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:46.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:46.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521633+0000", 2024-09-17T21:21:46.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:21:46.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:46.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521638+0000", 2024-09-17T21:21:46.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:21:46.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:46.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521659+0000", 2024-09-17T21:21:46.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-05 2024-09-17T21:21:46.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:46.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521675+0000", 2024-09-17T21:21:46.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5688999999999998e-05 2024-09-17T21:21:46.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616008+0000", 2024-09-17T21:21:46.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094333314000000001 2024-09-17T21:21:46.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:46.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616066+0000", 2024-09-17T21:21:46.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7797000000000002e-05 2024-09-17T21:21:46.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:46.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616219+0000", 2024-09-17T21:21:46.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015281100000000001 2024-09-17T21:21:46.518 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.518 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:46.518 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.519 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15718 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:46.519 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:46.519 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.935126913000005, 2024-09-17T21:21:46.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094703674000000002, 2024-09-17T21:21:46.519 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:46.519 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:46.519 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:46.519 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:46.519 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:46.519 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:46.519 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15718 2024-09-17T21:21:46.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.520 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:46.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:46.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:46.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:46.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:46.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:46.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521734+0000", 2024-09-17T21:21:46.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.61e-06 2024-09-17T21:21:46.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:46.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521740+0000", 2024-09-17T21:21:46.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:21:46.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:46.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521754+0000", 2024-09-17T21:21:46.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3793e-05 2024-09-17T21:21:46.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:46.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521763+0000", 2024-09-17T21:21:46.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5129999999999996e-06 2024-09-17T21:21:46.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616256+0000", 2024-09-17T21:21:46.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094492919999999994 2024-09-17T21:21:46.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:46.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616312+0000", 2024-09-17T21:21:46.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6737000000000002e-05 2024-09-17T21:21:46.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:46.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616436+0000", 2024-09-17T21:21:46.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012335800000000001 2024-09-17T21:21:46.523 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.523 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:46.523 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.523 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15719 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1031168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:46.523 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:46.524 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.935073508000002, 2024-09-17T21:21:46.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094888963000000007, 2024-09-17T21:21:46.524 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:46.524 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:46.524 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:46.524 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:46.524 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:46.524 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:46.524 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15719 2024-09-17T21:21:46.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.524 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:46.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:46.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:46.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:46.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:46.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:46.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521786+0000", 2024-09-17T21:21:46.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.074e-06 2024-09-17T21:21:46.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:46.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521791+0000", 2024-09-17T21:21:46.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3030000000000001e-06 2024-09-17T21:21:46.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:46.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521804+0000", 2024-09-17T21:21:46.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3027000000000001e-05 2024-09-17T21:21:46.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:46.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521811+0000", 2024-09-17T21:21:46.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6090000000000003e-06 2024-09-17T21:21:46.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616466+0000", 2024-09-17T21:21:46.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094654294 2024-09-17T21:21:46.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:46.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616523+0000", 2024-09-17T21:21:46.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6953999999999997e-05 2024-09-17T21:21:46.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:46.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616674+0000", 2024-09-17T21:21:46.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015170200000000001 2024-09-17T21:21:46.528 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.528 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:46.528 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.528 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15720 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1035264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:46.528 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:46.528 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.935036667999995, 2024-09-17T21:21:46.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095067061999999994, 2024-09-17T21:21:46.529 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:46.529 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:46.529 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:46.529 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:46.529 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:46.529 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:46.529 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15720 2024-09-17T21:21:46.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.529 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:46.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:46.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:46.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:46.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:46.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:46.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521823+0000", 2024-09-17T21:21:46.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:21:46.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:46.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521826+0000", 2024-09-17T21:21:46.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2100000000000002e-06 2024-09-17T21:21:46.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:46.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521835+0000", 2024-09-17T21:21:46.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1630000000000002e-06 2024-09-17T21:21:46.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:46.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521842+0000", 2024-09-17T21:21:46.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:21:46.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616711+0000", 2024-09-17T21:21:46.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094868859 2024-09-17T21:21:46.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:46.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616764+0000", 2024-09-17T21:21:46.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2778999999999997e-05 2024-09-17T21:21:46.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:46.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616889+0000", 2024-09-17T21:21:46.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012533100000000001 2024-09-17T21:21:46.533 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.533 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:46.533 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.533 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15721 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1039360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:46.533 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:46.533 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.934989559000002, 2024-09-17T21:21:46.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095277219999999996, 2024-09-17T21:21:46.533 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:46.533 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:46.533 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:46.533 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:46.534 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:46.534 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:46.534 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15721 2024-09-17T21:21:46.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.534 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:46.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:46.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:46.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:46.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:46.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:46.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521871+0000", 2024-09-17T21:21:46.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.443e-06 2024-09-17T21:21:46.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:46.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521876+0000", 2024-09-17T21:21:46.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0300000000000001e-06 2024-09-17T21:21:46.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:46.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521886+0000", 2024-09-17T21:21:46.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186e-05 2024-09-17T21:21:46.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:46.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521893+0000", 2024-09-17T21:21:46.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.08e-06 2024-09-17T21:21:46.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616918+0000", 2024-09-17T21:21:46.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095025223000000006 2024-09-17T21:21:46.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:46.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616977+0000", 2024-09-17T21:21:46.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8403999999999999e-05 2024-09-17T21:21:46.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:46.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617147+0000", 2024-09-17T21:21:46.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016985399999999999 2024-09-17T21:21:46.537 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.537 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:46.537 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.538 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15722 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:46.538 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:46.538 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.934955110000004, 2024-09-17T21:21:46.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095459783000000006, 2024-09-17T21:21:46.538 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:46.538 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:46.538 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:46.538 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:46.538 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:46.538 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:46.538 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15722 2024-09-17T21:21:46.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.538 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:46.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:46.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:46.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:46.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:46.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:46.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:46.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7700000000000004e-07 2024-09-17T21:21:46.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:46.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521907+0000", 2024-09-17T21:21:46.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.813e-06 2024-09-17T21:21:46.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:46.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521922+0000", 2024-09-17T21:21:46.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4533e-05 2024-09-17T21:21:46.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:46.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521934+0000", 2024-09-17T21:21:46.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2469999999999999e-05 2024-09-17T21:21:46.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617187+0000", 2024-09-17T21:21:46.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095252311000000006 2024-09-17T21:21:46.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:46.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617240+0000", 2024-09-17T21:21:46.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3740999999999999e-05 2024-09-17T21:21:46.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:46.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617364+0000", 2024-09-17T21:21:46.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-17T21:21:46.542 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.542 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:46.542 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.542 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15723 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:46.542 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:46.542 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.934911244000006, 2024-09-17T21:21:46.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095653563999999996, 2024-09-17T21:21:46.543 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:46.543 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:46.543 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:46.543 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:46.543 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:46.543 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:46.543 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15723 2024-09-17T21:21:46.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.543 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:46.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:46.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:46.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:46.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:46.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:46.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521949+0000", 2024-09-17T21:21:46.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:21:46.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:46.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521953+0000", 2024-09-17T21:21:46.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5870000000000003e-06 2024-09-17T21:21:46.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:46.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521970+0000", 2024-09-17T21:21:46.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7282000000000002e-05 2024-09-17T21:21:46.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:46.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521984+0000", 2024-09-17T21:21:46.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:21:46.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617383+0000", 2024-09-17T21:21:46.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095399063000000006 2024-09-17T21:21:46.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:46.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617437+0000", 2024-09-17T21:21:46.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3931999999999998e-05 2024-09-17T21:21:46.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:46.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617601+0000", 2024-09-17T21:21:46.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016407300000000001 2024-09-17T21:21:46.547 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.547 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:46.547 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.547 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5629.0:2384 3.1c 3:38c1263e:::rbd_data.1152d37183c3.000000000000000b:head [read 11780096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-17T21:21:46.547 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:46.547 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 71.032080414000006, 2024-09-17T21:21:46.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0246185320000001, 2024-09-17T21:21:46.547 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:46.547 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:46.547 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:46.547 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:46.547 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5629", 2024-09-17T21:21:46.548 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3798711329", 2024-09-17T21:21:46.548 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2384 2024-09-17T21:21:46.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.548 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:46.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:46.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:46.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:46.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:46.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:46.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424780+0000", 2024-09-17T21:21:46.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.13e-06 2024-09-17T21:21:46.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:46.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424785+0000", 2024-09-17T21:21:46.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4739999999999996e-06 2024-09-17T21:21:46.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:46.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424798+0000", 2024-09-17T21:21:46.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2704000000000001e-05 2024-09-17T21:21:46.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:46.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424806+0000", 2024-09-17T21:21:46.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9899999999999997e-06 2024-09-17T21:21:46.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424868+0000", 2024-09-17T21:21:46.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999997e-05 2024-09-17T21:21:46.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:46.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424875+0000", 2024-09-17T21:21:46.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0870000000000001e-06 2024-09-17T21:21:46.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428897+0000", 2024-09-17T21:21:46.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040226080000000004 2024-09-17T21:21:46.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:46.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428913+0000", 2024-09-17T21:21:46.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:21:46.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.448963+0000", 2024-09-17T21:21:46.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0200498659999999 2024-09-17T21:21:46.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:46.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449158+0000", 2024-09-17T21:21:46.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019581000000000001 2024-09-17T21:21:46.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:46.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449397+0000", 2024-09-17T21:21:46.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023871 2024-09-17T21:21:46.553 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.553 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:46.553 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.553 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:26043 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_remove_image in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-17T21:21:46.553 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:21:46.553 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.0575492940000002, 2024-09-17T21:21:46.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.019978351, 2024-09-17T21:21:46.553 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:46.553 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:46.554 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:46.554 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:46.554 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:46.554 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:46.554 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 26043 2024-09-17T21:21:46.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.554 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:46.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:46.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:21:46.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:46.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:21:46.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:46.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:46.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399314+0000", 2024-09-17T21:21:46.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.963e-06 2024-09-17T21:21:46.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:46.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399330+0000", 2024-09-17T21:21:46.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6684000000000002e-05 2024-09-17T21:21:46.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:46.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399372+0000", 2024-09-17T21:21:46.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1819000000000001e-05 2024-09-17T21:21:46.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:46.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399396+0000", 2024-09-17T21:21:46.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.406e-05 2024-09-17T21:21:46.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399571+0000", 2024-09-17T21:21:46.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017444800000000001 2024-09-17T21:21:46.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.557 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:46.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399595+0000", 2024-09-17T21:21:46.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4709999999999999e-05 2024-09-17T21:21:46.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.557 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:46.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.412735+0000", 2024-09-17T21:21:46.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0131394869999999 2024-09-17T21:21:46.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.557 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:46.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.413090+0000", 2024-09-17T21:21:46.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000355006 2024-09-17T21:21:46.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:21:46.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.413659+0000", 2024-09-17T21:21:46.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056922699999999995 2024-09-17T21:21:46.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:46.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.418940+0000", 2024-09-17T21:21:46.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052814109999999997 2024-09-17T21:21:46.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:46.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.418992+0000", 2024-09-17T21:21:46.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1646e-05 2024-09-17T21:21:46.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:46.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.419168+0000", 2024-09-17T21:21:46.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017573400000000001 2024-09-17T21:21:46.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:46.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:46.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:46.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.419288+0000", 2024-09-17T21:21:46.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012015600000000001 2024-09-17T21:21:46.559 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.559 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:46.559 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.560 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:46.560 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:46.560 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:46.561 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:21:46.562 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:46.557+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:46.662 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:46.657+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:46.734 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:46.734 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:46.734 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:46.734 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:46.748 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:21:46.791 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:46.789+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:46.863 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:46.859+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:46.912 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:46.912 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:46.912 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:46.912 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:46.912 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:46.926 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:21:46.971 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:46.969+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:47.050 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:47.050 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:47.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:47.051 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:47.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.051 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:21:47.051 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:47.051 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 205.441762764, 2024-09-17T21:21:47.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:21:47.051 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:47.051 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:47.051 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:47.051 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:47.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:47.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:47.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:47.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:47.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:47.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:21:47.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:21:47.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:47.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:21:47.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:21:47.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:47.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:21:47.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:21:47.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:47.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:21:47.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:21:47.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:47.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:21:47.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:21:47.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:47.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:21:47.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:21:47.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:47.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:21:47.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:21:47.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:47.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:21:47.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:21:47.055 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.055 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:47.055 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.055 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:21:47.056 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:47.056 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 205.43181612800001, 2024-09-17T21:21:47.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:21:47.056 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:47.056 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:47.056 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:47.056 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:47.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:47.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:47.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:47.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:47.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:47.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:21:47.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:21:47.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:47.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:21:47.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:21:47.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:47.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:21:47.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:21:47.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:47.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:21:47.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:21:47.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:47.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:21:47.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:21:47.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:47.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:21:47.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:21:47.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:47.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:21:47.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:21:47.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:47.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:21:47.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:21:47.059 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.060 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:47.060 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.060 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:21:47.060 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:47.060 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 204.18866528800001, 2024-09-17T21:21:47.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:21:47.060 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:47.060 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:47.060 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:47.060 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:47.060 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:21:47.060 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:21:47.061 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:21:47.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.061 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:47.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:47.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:47.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:47.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:47.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:47.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:21:47.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:21:47.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:47.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:21:47.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:21:47.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:47.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:21:47.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:21:47.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:47.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:21:47.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:21:47.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:47.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:21:47.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:21:47.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:47.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:21:47.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:21:47.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:47.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:21:47.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:21:47.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:47.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:21:47.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:21:47.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:47.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:21:47.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:21:47.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:47.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:21:47.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:21:47.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:47.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:21:47.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:21:47.065 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.065 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:47.066 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.066 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:21:47.066 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:47.066 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 204.179019802, 2024-09-17T21:21:47.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:21:47.066 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:47.066 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:47.066 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:47.066 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:47.066 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:21:47.066 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:21:47.066 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:21:47.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.067 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:47.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:47.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:47.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:47.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:47.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:47.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:21:47.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:21:47.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:47.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:21:47.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:21:47.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:47.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:21:47.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:21:47.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:47.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:21:47.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:21:47.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:47.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:21:47.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:21:47.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:47.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:21:47.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:21:47.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:47.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:21:47.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:21:47.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:47.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:21:47.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:21:47.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:47.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:21:47.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:21:47.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:47.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:21:47.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:21:47.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:47.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:21:47.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:21:47.071 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.071 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:47.071 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.071 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:21:47.072 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:47.072 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 190.81008376899999, 2024-09-17T21:21:47.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:21:47.072 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:47.072 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:47.072 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:47.072 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:47.072 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:21:47.072 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:21:47.072 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:21:47.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.072 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:47.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:47.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:47.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:47.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:47.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:47.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:21:47.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:21:47.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:47.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:21:47.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:21:47.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:47.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:21:47.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:21:47.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:47.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:21:47.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:21:47.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:47.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:21:47.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:21:47.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:47.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:21:47.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:21:47.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:47.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:21:47.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:21:47.076 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.076 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:47.076 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.076 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:47.076 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:47.076 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 172.60839920699999, 2024-09-17T21:21:47.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:21:47.076 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:47.076 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:47.077 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:47.077 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:47.077 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:47.077 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:47.077 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:21:47.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.077 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:47.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:47.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:47.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:47.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:47.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:47.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:21:47.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:21:47.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:47.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:21:47.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:21:47.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:47.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:21:47.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:21:47.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:47.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:21:47.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:21:47.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:47.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:21:47.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:21:47.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:47.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:21:47.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:21:47.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:47.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:21:47.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:21:47.080 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.080 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:47.081 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.081 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:47.081 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:47.081 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 172.60790463000001, 2024-09-17T21:21:47.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:21:47.081 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:47.081 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:47.081 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:47.081 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:47.081 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:47.081 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:47.082 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:21:47.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.082 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:47.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:47.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:47.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:47.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:47.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:47.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:21:47.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:21:47.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:47.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:21:47.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:21:47.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:47.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:21:47.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:21:47.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:47.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:21:47.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:21:47.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:47.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:21:47.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:21:47.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:47.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:21:47.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:21:47.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:47.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:21:47.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:21:47.085 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.085 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:47.085 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.085 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:47.086 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:47.086 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 172.607849249, 2024-09-17T21:21:47.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:21:47.086 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:47.086 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:47.086 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:47.086 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:47.086 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:47.086 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:47.086 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:21:47.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.086 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:47.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:47.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:47.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:47.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:47.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:47.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:21:47.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:21:47.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:47.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:21:47.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:21:47.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:47.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:21:47.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:21:47.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:47.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:21:47.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:21:47.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:47.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:21:47.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:21:47.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:47.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:21:47.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:21:47.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:47.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:21:47.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:21:47.090 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.090 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:47.090 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.090 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:47.090 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:47.090 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 172.60779297100001, 2024-09-17T21:21:47.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:21:47.090 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:47.090 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:47.090 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:47.091 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:47.091 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:47.091 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:47.091 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:21:47.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.091 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:47.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:47.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:47.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:47.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:47.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:47.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:21:47.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:21:47.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:47.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:21:47.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:21:47.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:47.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:21:47.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:21:47.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:47.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:21:47.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:21:47.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:47.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:21:47.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:21:47.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:47.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:21:47.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:21:47.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:47.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:21:47.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:21:47.094 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.094 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:47.094 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.095 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:47.095 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:47.095 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 172.60684333200001, 2024-09-17T21:21:47.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:21:47.095 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:47.095 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:47.095 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:47.095 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:47.095 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:47.095 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:47.095 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:21:47.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.096 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:47.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:47.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:47.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:47.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:47.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:47.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:21:47.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:21:47.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:47.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:21:47.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:21:47.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:47.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:21:47.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:21:47.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:47.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:21:47.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:21:47.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:47.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:21:47.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:21:47.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:47.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:21:47.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:21:47.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:47.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:21:47.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:21:47.099 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.099 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:47.099 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.100 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:47.100 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:47.100 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 172.606764712, 2024-09-17T21:21:47.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:21:47.100 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:47.100 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:47.100 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:47.100 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:47.100 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:47.100 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:47.100 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:21:47.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.101 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:47.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:47.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:47.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:47.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:47.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:47.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:21:47.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:21:47.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:47.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:21:47.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:21:47.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:47.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:21:47.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:21:47.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:47.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:21:47.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:21:47.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:47.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:21:47.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:21:47.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:47.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:21:47.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:21:47.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:47.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:21:47.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:21:47.104 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.104 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:47.104 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.104 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:47.105 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:47.105 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 172.60665177300001, 2024-09-17T21:21:47.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:21:47.105 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:47.105 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:47.105 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:47.105 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:47.105 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:47.105 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:47.105 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:21:47.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.105 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:47.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:47.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:47.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:47.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:47.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:47.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:21:47.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:21:47.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:47.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:21:47.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:21:47.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:47.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:21:47.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:21:47.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:47.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:21:47.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:21:47.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:47.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:21:47.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:21:47.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:47.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:21:47.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:21:47.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:47.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:21:47.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:21:47.109 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.109 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:47.109 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.109 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:47.109 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:47.109 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 172.605990719, 2024-09-17T21:21:47.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:21:47.109 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:47.109 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:21:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:47.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:47.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:47.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:21:47.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:21:47.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:47.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:21:47.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:21:47.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:47.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:21:47.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:21:47.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:47.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:21:47.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:21:47.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:47.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:21:47.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:21:47.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:47.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:21:47.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:21:47.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:47.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:21:47.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:21:47.113 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.114 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:47.114 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.114 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:47.114 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:47.114 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 172.60592590799999, 2024-09-17T21:21:47.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:21:47.114 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:47.114 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:47.114 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:47.114 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:47.114 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:47.115 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:47.115 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:21:47.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.115 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:47.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:47.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:47.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:47.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:47.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:47.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:21:47.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:21:47.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:47.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:21:47.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:21:47.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:47.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:21:47.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:21:47.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:47.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:21:47.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:21:47.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:47.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:21:47.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:21:47.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:47.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:21:47.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:21:47.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:47.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:21:47.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:21:47.118 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.118 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:47.118 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.119 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:47.119 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:47.119 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 172.60586493100001, 2024-09-17T21:21:47.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:21:47.119 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:47.119 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:47.119 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:47.119 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:47.119 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:47.119 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:47.119 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:21:47.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.120 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:47.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:47.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:47.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:47.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:47.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:47.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:21:47.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:21:47.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:47.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:21:47.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:21:47.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:47.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:21:47.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:21:47.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:47.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:21:47.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:21:47.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:47.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:21:47.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:21:47.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:47.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:21:47.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:21:47.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:47.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:21:47.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:21:47.123 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.123 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:47.123 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.123 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:21:47.123 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:47.124 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 117.708293746, 2024-09-17T21:21:47.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:21:47.124 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:47.124 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:47.124 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:47.124 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:47.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:47.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:47.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:47.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:47.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:47.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:21:47.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:21:47.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:47.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:21:47.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:21:47.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:47.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:21:47.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:21:47.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:47.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:21:47.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:21:47.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:47.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:21:47.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:21:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:21:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:21:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:21:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:21:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:47.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:21:47.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:21:47.128 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.128 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:47.128 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.128 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:21:47.128 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:47.128 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 117.499635352, 2024-09-17T21:21:47.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:21:47.128 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:47.128 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:47.128 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:47.129 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:47.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:47.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:47.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:47.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:47.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:47.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:21:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:21:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:21:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:21:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:21:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:21:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:21:47.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:21:47.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:47.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:21:47.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:21:47.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:47.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:21:47.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:21:47.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:47.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:21:47.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:21:47.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:47.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:21:47.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:21:47.132 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.132 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:47.132 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.132 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:9547 3.2 e147/132 3:42b518d9:::rbd_data.1152f056b73f.0000000000000001:head v 147'513, mlcod=142'511)", 2024-09-17T21:21:47.133 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:47.133 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 44.574662230000001, 2024-09-17T21:21:47.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.031059566, 2024-09-17T21:21:47.133 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:47.133 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:47.133 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:47.133 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:47.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:47.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:47.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:47.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:47.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:47.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471293+0000", 2024-09-17T21:21:47.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2029999999999999e-06 2024-09-17T21:21:47.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:47.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480606+0000", 2024-09-17T21:21:47.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093123839999999999 2024-09-17T21:21:47.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:47.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480653+0000", 2024-09-17T21:21:47.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7852000000000002e-05 2024-09-17T21:21:47.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:47.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480673+0000", 2024-09-17T21:21:47.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0013e-05 2024-09-17T21:21:47.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:47.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480862+0000", 2024-09-17T21:21:47.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018833599999999999 2024-09-17T21:21:47.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:47.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480906+0000", 2024-09-17T21:21:47.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4162000000000002e-05 2024-09-17T21:21:47.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:47.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502259+0000", 2024-09-17T21:21:47.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021353154999999999 2024-09-17T21:21:47.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:47.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502351+0000", 2024-09-17T21:21:47.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1460999999999997e-05 2024-09-17T21:21:47.137 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.137 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:47.137 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.137 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:15234 3.1 e147/134 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head v 147'917, mlcod=145'914)", 2024-09-17T21:21:47.137 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:47.137 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 43.942001611999999, 2024-09-17T21:21:47.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045303695999999997, 2024-09-17T21:21:47.137 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:47.137 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:47.137 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:47.137 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:47.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:47.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:47.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:47.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:47.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:47.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103954+0000", 2024-09-17T21:21:47.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2259999999999999e-06 2024-09-17T21:21:47.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:47.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119105+0000", 2024-09-17T21:21:47.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015150871 2024-09-17T21:21:47.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:47.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119141+0000", 2024-09-17T21:21:47.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6239000000000003e-05 2024-09-17T21:21:47.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:47.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119159+0000", 2024-09-17T21:21:47.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7946999999999999e-05 2024-09-17T21:21:47.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:47.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119344+0000", 2024-09-17T21:21:47.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018524599999999999 2024-09-17T21:21:47.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:47.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119399+0000", 2024-09-17T21:21:47.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4922e-05 2024-09-17T21:21:47.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:47.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149165+0000", 2024-09-17T21:21:47.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029765697000000001 2024-09-17T21:21:47.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:47.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149255+0000", 2024-09-17T21:21:47.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0548000000000001e-05 2024-09-17T21:21:47.141 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.141 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:47.141 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.142 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:19859 3.8 e147/133 3:12f72409:::rbd_data.1152f056b73f.0000000000000003:head v 147'482, mlcod=145'479)", 2024-09-17T21:21:47.142 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:47.142 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 43.533110545, 2024-09-17T21:21:47.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033476895999999999, 2024-09-17T21:21:47.142 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:47.142 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:47.142 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:47.142 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:47.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:47.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:47.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:47.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:47.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:47.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:47.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512845+0000", 2024-09-17T21:21:47.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.886e-06 2024-09-17T21:21:47.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:47.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524104+0000", 2024-09-17T21:21:47.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011259668 2024-09-17T21:21:47.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:47.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524148+0000", 2024-09-17T21:21:47.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3386000000000001e-05 2024-09-17T21:21:47.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:47.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524172+0000", 2024-09-17T21:21:47.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4762000000000001e-05 2024-09-17T21:21:47.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:47.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524379+0000", 2024-09-17T21:21:47.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020671599999999999 2024-09-17T21:21:47.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:47.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524421+0000", 2024-09-17T21:21:47.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2182999999999999e-05 2024-09-17T21:21:47.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:47.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546216+0000", 2024-09-17T21:21:47.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021795101000000001 2024-09-17T21:21:47.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:47.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:47.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:47.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546320+0000", 2024-09-17T21:21:47.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103194 2024-09-17T21:21:47.146 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.146 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:47.146 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.146 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:47.146 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:47.146 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:47.148 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:21:47.148 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:47.143+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:47.249 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:47.247+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:47.278 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:47.278 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:47.292 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:21:47.381 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:47.379+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:47.440 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:21:47.463 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:47.463 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:47.463 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:47.459+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:47.478 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:21:47.552 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:47.549+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:47.641 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:47.641 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:47.653 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:47.651+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:47.657 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:21:47.754 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:47.749+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:47.816 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:47.816 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:47.830 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:21:47.877 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:47.875+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:47.953 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:47.953 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:47.954 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:47.951+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:47.967 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:21:48.055 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:48.053+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:48.126 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:48.127 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:48.141 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:21:48.187 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:48.185+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:48.257 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:48.253+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:48.272 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:48.273 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:48.273 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:48.273 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:48.287 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:21:48.379 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:48.377+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:48.414 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-17T21:21:48.458 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:48.453+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:48.464 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:48.464 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:48.464 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:48.464 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:48.465 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:48.480 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:21:48.559 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:48.555+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:48.657 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:48.657 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:48.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:48.657 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:48.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:21:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 231.64030354400001, 2024-09-17T21:21:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:21:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:21:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:21:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:21:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:21:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:21:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:21:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:21:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:21:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:21:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:21:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:21:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:21:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:21:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:21:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:21:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:21:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:21:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:21:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:21:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:21:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:21:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:21:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:21:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:21:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:21:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:21:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:21:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:21:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:21:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:21:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 212.06013926399999, 2024-09-17T21:21:48.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:21:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:21:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:21:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:21:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:21:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:21:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:21:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:21:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:21:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:21:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:21:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:21:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:21:48.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:21:48.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:48.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:21:48.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:21:48.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:21:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:21:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:21:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:21:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:21:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:21:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:21:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:21:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:21:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:21:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:48.670 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 212.06004195200001, 2024-09-17T21:21:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:21:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:21:48.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:48.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:21:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:21:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:21:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:21:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:21:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:21:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:48.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:21:48.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:21:48.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:21:48.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:21:48.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:48.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:21:48.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:21:48.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:21:48.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:21:48.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:48.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:21:48.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:21:48.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:21:48.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:21:48.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:21:48.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:21:48.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:21:48.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:21:48.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:21:48.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:48.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:21:48.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:21:48.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:48.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:21:48.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:21:48.677 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.677 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:48.677 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.678 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:48.678 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:48.678 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 212.05795515099999, 2024-09-17T21:21:48.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:21:48.678 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:48.678 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:48.678 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:48.678 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:48.678 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:21:48.678 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:21:48.678 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:21:48.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.678 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:48.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:48.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:48.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:48.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:48.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:48.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:21:48.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:21:48.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:48.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:21:48.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:21:48.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:48.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:21:48.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:21:48.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:48.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:21:48.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:21:48.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:21:48.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:21:48.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:21:48.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:21:48.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:21:48.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:21:48.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:21:48.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:48.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:21:48.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:21:48.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:21:48.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:21:48.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:48.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:21:48.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:21:48.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:48.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:21:48.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:21:48.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:48.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:21:48.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:21:48.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:48.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:21:48.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:21:48.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:48.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:21:48.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:21:48.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:48.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:21:48.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:21:48.685 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.685 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:48.685 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.685 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:48.686 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:48.686 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 179.37405637000001, 2024-09-17T21:21:48.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:21:48.686 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:48.686 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:48.686 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:48.686 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:48.686 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:48.686 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:48.686 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:21:48.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.687 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:48.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:48.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:48.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:48.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:48.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:48.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:21:48.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:21:48.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:48.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:21:48.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:21:48.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:48.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:21:48.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:21:48.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:48.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:21:48.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:21:48.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:21:48.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:21:48.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:48.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:21:48.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:21:48.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:21:48.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:21:48.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:48.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:21:48.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:21:48.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:21:48.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:21:48.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:48.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:21:48.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:21:48.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:48.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:21:48.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:21:48.692 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.692 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:48.692 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.692 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:48.692 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:48.693 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 179.36496465900001, 2024-09-17T21:21:48.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:21:48.693 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:48.693 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:48.693 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:48.693 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:48.693 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:48.693 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:48.693 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:21:48.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.693 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:48.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:48.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:48.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:48.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:48.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:48.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:21:48.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:21:48.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:48.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:21:48.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:21:48.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:48.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:21:48.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:21:48.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:48.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:21:48.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:21:48.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:21:48.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:21:48.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:48.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:21:48.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:21:48.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:21:48.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:21:48.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:48.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:21:48.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:21:48.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:48.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:21:48.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:21:48.697 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.698 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:48.698 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.698 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:48.698 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:48.698 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 179.36479260799999, 2024-09-17T21:21:48.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:21:48.698 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:48.698 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:48.698 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:48.698 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:48.698 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:48.699 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:48.699 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:21:48.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.699 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:48.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:48.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:48.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:48.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:48.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:48.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:21:48.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:21:48.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:48.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:21:48.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:21:48.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:48.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:21:48.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:21:48.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:48.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:21:48.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:21:48.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:21:48.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:21:48.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:48.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:21:48.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:21:48.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:21:48.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:21:48.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:48.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:21:48.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:21:48.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:48.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:21:48.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:21:48.703 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.703 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:48.704 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.704 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:48.704 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:48.704 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 179.189859861, 2024-09-17T21:21:48.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:21:48.704 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:48.704 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:48.704 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:48.704 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:48.704 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:48.704 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:48.705 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:21:48.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.705 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:48.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:48.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:48.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:48.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:48.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:48.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:21:48.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:21:48.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:48.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:21:48.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:21:48.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:48.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:21:48.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:21:48.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:48.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:21:48.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:21:48.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:21:48.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:21:48.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:48.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:21:48.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:21:48.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:21:48.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:21:48.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:48.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:21:48.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:21:48.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:48.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:21:48.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:21:48.709 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.709 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:48.709 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.709 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:48.710 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:48.710 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 179.18977530399999, 2024-09-17T21:21:48.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:21:48.710 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:48.710 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:48.710 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:48.710 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:48.710 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:48.710 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:48.710 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:21:48.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.710 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:48.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:48.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:48.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:48.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:48.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:48.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:48.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:21:48.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:48.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:21:48.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:21:48.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:48.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:21:48.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:21:48.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:48.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:21:48.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:21:48.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:21:48.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:21:48.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:48.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:21:48.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:21:48.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:21:48.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:21:48.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:48.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:21:48.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:21:48.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:48.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:21:48.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:21:48.714 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.715 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:48.715 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.715 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:48.715 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:48.715 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 179.18970966699999, 2024-09-17T21:21:48.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:21:48.715 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:48.715 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:48.715 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:48.715 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:48.715 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:48.716 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:48.716 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:21:48.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.716 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:48.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:48.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:48.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:48.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:48.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:48.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:21:48.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:21:48.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:48.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:21:48.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:21:48.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:48.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:21:48.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:21:48.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:48.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:21:48.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:21:48.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:21:48.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:21:48.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:48.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:21:48.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:21:48.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:21:48.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:21:48.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:48.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:21:48.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:21:48.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:48.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:21:48.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:21:48.720 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.720 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:48.720 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.720 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:48.720 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:48.720 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 127.81613457, 2024-09-17T21:21:48.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:21:48.720 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:48.721 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:48.721 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:48.721 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:48.721 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:21:48.721 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:48.721 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:21:48.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.721 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:48.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:48.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:48.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:48.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:48.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:48.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:21:48.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:21:48.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:48.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:21:48.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:21:48.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:48.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:21:48.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:21:48.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:48.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:21:48.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:21:48.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:21:48.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:21:48.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:48.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:21:48.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:21:48.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:48.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:21:48.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:21:48.724 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.724 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:48.725 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.725 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:48.725 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:48.725 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 127.816015058, 2024-09-17T21:21:48.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:21:48.725 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:48.725 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:48.725 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:48.725 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:48.725 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:21:48.725 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:48.725 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:21:48.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.726 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:48.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:48.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:48.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:48.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:48.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:48.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:21:48.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:21:48.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:48.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:21:48.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:21:48.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:48.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:21:48.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:21:48.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:48.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:21:48.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:21:48.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:21:48.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:21:48.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:48.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:21:48.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:21:48.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:48.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:21:48.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:21:48.729 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.729 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:48.729 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.729 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:996 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:48.729 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:48.730 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.222901687000004, 2024-09-17T21:21:48.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084493736999999999, 2024-09-17T21:21:48.730 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:48.730 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:48.730 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:48.730 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:48.730 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:48.730 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:48.730 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 996 2024-09-17T21:21:48.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.730 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:48.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:48.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:48.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:48.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:48.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:48.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431168+0000", 2024-09-17T21:21:48.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4e-07 2024-09-17T21:21:48.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:48.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431172+0000", 2024-09-17T21:21:48.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.608e-06 2024-09-17T21:21:48.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:48.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431180+0000", 2024-09-17T21:21:48.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5650000000000002e-06 2024-09-17T21:21:48.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:48.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431188+0000", 2024-09-17T21:21:48.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:21:48.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515354+0000", 2024-09-17T21:21:48.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084165977000000003 2024-09-17T21:21:48.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:48.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515451+0000", 2024-09-17T21:21:48.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6669000000000006e-05 2024-09-17T21:21:48.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:48.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515661+0000", 2024-09-17T21:21:48.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021021599999999999 2024-09-17T21:21:48.734 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.734 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:48.734 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.734 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:997 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 583680~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:48.734 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:48.734 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.222870291999996, 2024-09-17T21:21:48.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084772419000000002, 2024-09-17T21:21:48.734 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:48.734 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:48.734 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:48.734 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:48.735 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:48.735 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:48.735 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 997 2024-09-17T21:21:48.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.735 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:48.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:48.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:48.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:48.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:48.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:48.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:48.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0400000000000005e-07 2024-09-17T21:21:48.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:48.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431202+0000", 2024-09-17T21:21:48.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:21:48.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:48.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431208+0000", 2024-09-17T21:21:48.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4740000000000002e-06 2024-09-17T21:21:48.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:48.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431214+0000", 2024-09-17T21:21:48.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6550000000000001e-06 2024-09-17T21:21:48.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515741+0000", 2024-09-17T21:21:48.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084526904 2024-09-17T21:21:48.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:48.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515826+0000", 2024-09-17T21:21:48.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4643999999999998e-05 2024-09-17T21:21:48.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:48.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515971+0000", 2024-09-17T21:21:48.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145444 2024-09-17T21:21:48.738 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.738 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:48.738 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.739 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:998 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 587776~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:48.739 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:48.739 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.22284732, 2024-09-17T21:21:48.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085108565999999997, 2024-09-17T21:21:48.739 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:48.739 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:48.739 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:48.739 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:48.739 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:48.739 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:48.739 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 998 2024-09-17T21:21:48.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.739 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:48.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:48.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:48.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:48.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:48.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:48.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:48.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:21:48.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:48.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431224+0000", 2024-09-17T21:21:48.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:21:48.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:48.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431231+0000", 2024-09-17T21:21:48.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000002e-06 2024-09-17T21:21:48.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:48.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431236+0000", 2024-09-17T21:21:48.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.648e-06 2024-09-17T21:21:48.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516024+0000", 2024-09-17T21:21:48.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084787584999999999 2024-09-17T21:21:48.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:48.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516122+0000", 2024-09-17T21:21:48.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8076000000000001e-05 2024-09-17T21:21:48.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:48.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516330+0000", 2024-09-17T21:21:48.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020808 2024-09-17T21:21:48.743 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.743 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:48.743 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.743 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1000 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 591872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:48.743 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:48.743 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.222825205000007, 2024-09-17T21:21:48.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085389065, 2024-09-17T21:21:48.743 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:48.743 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:48.744 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:48.744 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:48.744 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:48.744 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:48.744 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1000 2024-09-17T21:21:48.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.744 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:48.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:48.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:48.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:48.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:48.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:48.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:48.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:21:48.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:48.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431246+0000", 2024-09-17T21:21:48.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1749999999999999e-06 2024-09-17T21:21:48.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:48.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431253+0000", 2024-09-17T21:21:48.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2790000000000004e-06 2024-09-17T21:21:48.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:48.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431258+0000", 2024-09-17T21:21:48.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3299999999999998e-06 2024-09-17T21:21:48.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516384+0000", 2024-09-17T21:21:48.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085125549999999994 2024-09-17T21:21:48.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:48.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516460+0000", 2024-09-17T21:21:48.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6359999999999994e-05 2024-09-17T21:21:48.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:48.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516633+0000", 2024-09-17T21:21:48.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017285900000000001 2024-09-17T21:21:48.747 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.747 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:48.747 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.748 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1001 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 595968~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:48.748 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:48.748 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.222801798999996, 2024-09-17T21:21:48.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085614715999999993, 2024-09-17T21:21:48.748 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:48.748 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:48.748 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:48.748 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:48.748 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:48.748 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:48.748 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1001 2024-09-17T21:21:48.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.749 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:48.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:48.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:48.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:48.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:48.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:48.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438066+0000", 2024-09-17T21:21:48.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067989490000000003 2024-09-17T21:21:48.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:48.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438073+0000", 2024-09-17T21:21:48.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8419999999999999e-06 2024-09-17T21:21:48.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:48.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438090+0000", 2024-09-17T21:21:48.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6657e-05 2024-09-17T21:21:48.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:48.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438099+0000", 2024-09-17T21:21:48.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0529999999999996e-06 2024-09-17T21:21:48.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516678+0000", 2024-09-17T21:21:48.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078579422999999995 2024-09-17T21:21:48.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:48.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516739+0000", 2024-09-17T21:21:48.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1315000000000005e-05 2024-09-17T21:21:48.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:48.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516882+0000", 2024-09-17T21:21:48.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014247700000000001 2024-09-17T21:21:48.752 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.752 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:48.752 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.752 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:19998 3.5 3:a4885450:::rbd_object_map.1152d139f9f9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-17T21:21:48.752 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:48.752 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 69.251534554000003, 2024-09-17T21:21:48.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90715489299999996, 2024-09-17T21:21:48.753 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:48.753 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:48.753 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:48.753 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:48.753 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:48.753 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:48.753 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19998 2024-09-17T21:21:48.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.753 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:48.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:48.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:48.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:48.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:48.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:48.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402537+0000", 2024-09-17T21:21:48.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5600000000000001e-06 2024-09-17T21:21:48.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:48.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402549+0000", 2024-09-17T21:21:48.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1929999999999999e-05 2024-09-17T21:21:48.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:48.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402579+0000", 2024-09-17T21:21:48.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0093000000000001e-05 2024-09-17T21:21:48.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:48.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402595+0000", 2024-09-17T21:21:48.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5855999999999999e-05 2024-09-17T21:21:48.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402731+0000", 2024-09-17T21:21:48.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013610799999999999 2024-09-17T21:21:48.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:48.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402750+0000", 2024-09-17T21:21:48.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8737e-05 2024-09-17T21:21:48.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.305977+0000", 2024-09-17T21:21:48.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90322768399999998 2024-09-17T21:21:48.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:48.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306236+0000", 2024-09-17T21:21:48.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000258111 2024-09-17T21:21:48.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:48.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306557+0000", 2024-09-17T21:21:48.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032159799999999998 2024-09-17T21:21:48.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:48.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.308286+0000", 2024-09-17T21:21:48.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017292150000000001 2024-09-17T21:21:48.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:48.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309530+0000", 2024-09-17T21:21:48.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00124409 2024-09-17T21:21:48.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:48.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309630+0000", 2024-09-17T21:21:48.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9689e-05 2024-09-17T21:21:48.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:48.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309689+0000", 2024-09-17T21:21:48.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9222000000000003e-05 2024-09-17T21:21:48.759 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.759 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:48.759 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.759 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:25277 3.1f 3:ff5d7cd5:::rbd_header.11524f65d4ff:head [watch ping cookie 139860845478480 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-17T21:21:48.759 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:21:48.759 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.068578485, 2024-09-17T21:21:48.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.92185720000000004, 2024-09-17T21:21:48.759 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:48.759 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:48.760 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:48.760 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:48.760 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:48.760 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:48.760 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 25277 2024-09-17T21:21:48.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.760 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:48.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:48.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:21:48.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:48.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:21:48.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:48.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585493+0000", 2024-09-17T21:21:48.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:21:48.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:48.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585505+0000", 2024-09-17T21:21:48.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2466000000000001e-05 2024-09-17T21:21:48.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:48.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585537+0000", 2024-09-17T21:21:48.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1183000000000003e-05 2024-09-17T21:21:48.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:48.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585553+0000", 2024-09-17T21:21:48.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.662e-05 2024-09-17T21:21:48.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585696+0000", 2024-09-17T21:21:48.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014293800000000001 2024-09-17T21:21:48.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:48.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585718+0000", 2024-09-17T21:21:48.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1860000000000001e-05 2024-09-17T21:21:48.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.501188+0000", 2024-09-17T21:21:48.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.91547051300000004 2024-09-17T21:21:48.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:21:48.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.501260+0000", 2024-09-17T21:21:48.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1571999999999998e-05 2024-09-17T21:21:48.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507002+0000", 2024-09-17T21:21:48.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057414850000000002 2024-09-17T21:21:48.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:48.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507143+0000", 2024-09-17T21:21:48.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014140299999999999 2024-09-17T21:21:48.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:48.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507348+0000", 2024-09-17T21:21:48.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000204773 2024-09-17T21:21:48.765 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.765 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:48.765 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.765 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:26042 3.f 3:f24134c1:::rbd_id.image_client.0-clone34:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e186)", 2024-09-17T21:21:48.765 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:21:48.765 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.1678901169999998, 2024-09-17T21:21:48.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.91185481000000002, 2024-09-17T21:21:48.765 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:48.765 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:48.766 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:48.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:48.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:48.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:48.766 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 26042 2024-09-17T21:21:48.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.766 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:48.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:48.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:21:48.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:48.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:21:48.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:48.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:48.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486180+0000", 2024-09-17T21:21:48.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5990000000000001e-06 2024-09-17T21:21:48.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:48.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486187+0000", 2024-09-17T21:21:48.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7859999999999999e-06 2024-09-17T21:21:48.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:48.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486206+0000", 2024-09-17T21:21:48.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8641e-05 2024-09-17T21:21:48.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:48.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486219+0000", 2024-09-17T21:21:48.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.308e-05 2024-09-17T21:21:48.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486283+0000", 2024-09-17T21:21:48.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4084000000000005e-05 2024-09-17T21:21:48.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:48.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486290+0000", 2024-09-17T21:21:48.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.584e-06 2024-09-17T21:21:48.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:48.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392182+0000", 2024-09-17T21:21:48.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90589206499999997 2024-09-17T21:21:48.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:48.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392518+0000", 2024-09-17T21:21:48.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033596 2024-09-17T21:21:48.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:21:48.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392802+0000", 2024-09-17T21:21:48.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028394999999999999 2024-09-17T21:21:48.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:48.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397705+0000", 2024-09-17T21:21:48.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0049031769999999999 2024-09-17T21:21:48.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:48.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397770+0000", 2024-09-17T21:21:48.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4765999999999996e-05 2024-09-17T21:21:48.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:48.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397927+0000", 2024-09-17T21:21:48.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015699900000000001 2024-09-17T21:21:48.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:48.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:48.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:48.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.398034+0000", 2024-09-17T21:21:48.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000107119 2024-09-17T21:21:48.772 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.772 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:48.772 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.772 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:48.772 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:48.772 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:48.773 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:21:48.774 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:48.769+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:48.874 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:48.869+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:48.945 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:48.945 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:48.945 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:48.945 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:48.959 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:21:48.975 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:48.971+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:49.076 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:49.071+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:49.125 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:49.126 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:49.126 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:49.126 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:49.126 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:49.140 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:21:49.183 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:49.181+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:49.264 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:49.264 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:49.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:49.264 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:49.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.264 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:18003 3.1d 3:b9765580:::rbd_header.11527d2b15f6:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T21:21:49.264 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:49.264 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 96.901792861999994, 2024-09-17T21:21:49.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94165346900000002, 2024-09-17T21:21:49.264 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.265 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.265 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.265 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.265 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:49.265 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:49.265 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18003 2024-09-17T21:21:49.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.265 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:49.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:49.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357880+0000", 2024-09-17T21:21:49.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:21:49.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357891+0000", 2024-09-17T21:21:49.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0417999999999999e-05 2024-09-17T21:21:49.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357913+0000", 2024-09-17T21:21:49.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2215e-05 2024-09-17T21:21:49.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357922+0000", 2024-09-17T21:21:49.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4779999999999997e-06 2024-09-17T21:21:49.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358023+0000", 2024-09-17T21:21:49.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101025 2024-09-17T21:21:49.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:49.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358040+0000", 2024-09-17T21:21:49.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6291999999999999e-05 2024-09-17T21:21:49.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298203+0000", 2024-09-17T21:21:49.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94016297500000001 2024-09-17T21:21:49.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298626+0000", 2024-09-17T21:21:49.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000422853 2024-09-17T21:21:49.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.299532+0000", 2024-09-17T21:21:49.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00090657300000000004 2024-09-17T21:21:49.269 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.269 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.269 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.269 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15707 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 982016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:49.269 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:49.269 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 86.738627614999999, 2024-09-17T21:21:49.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092305228000000003, 2024-09-17T21:21:49.269 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.270 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.270 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.270 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.270 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:49.270 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:49.270 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15707 2024-09-17T21:21:49.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.270 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:49.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:49.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521045+0000", 2024-09-17T21:21:49.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4899999999999999e-06 2024-09-17T21:21:49.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521066+0000", 2024-09-17T21:21:49.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0236000000000001e-05 2024-09-17T21:21:49.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521085+0000", 2024-09-17T21:21:49.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8819999999999999e-05 2024-09-17T21:21:49.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521100+0000", 2024-09-17T21:21:49.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5146e-05 2024-09-17T21:21:49.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612978+0000", 2024-09-17T21:21:49.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091878847 2024-09-17T21:21:49.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613079+0000", 2024-09-17T21:21:49.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100503 2024-09-17T21:21:49.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613349+0000", 2024-09-17T21:21:49.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000270186 2024-09-17T21:21:49.273 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.273 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.273 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.273 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15708 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 986112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:49.274 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:49.274 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 86.73854944, 2024-09-17T21:21:49.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092547217000000001, 2024-09-17T21:21:49.274 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.274 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.274 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.274 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.274 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:49.274 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:49.274 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15708 2024-09-17T21:21:49.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.274 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:49.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:49.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521123+0000", 2024-09-17T21:21:49.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.353e-06 2024-09-17T21:21:49.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521129+0000", 2024-09-17T21:21:49.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:21:49.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521145+0000", 2024-09-17T21:21:49.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.611e-05 2024-09-17T21:21:49.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521154+0000", 2024-09-17T21:21:49.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4500000000000004e-06 2024-09-17T21:21:49.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613397+0000", 2024-09-17T21:21:49.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092243275 2024-09-17T21:21:49.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613503+0000", 2024-09-17T21:21:49.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106013 2024-09-17T21:21:49.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613669+0000", 2024-09-17T21:21:49.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016642299999999999 2024-09-17T21:21:49.277 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.277 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.277 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.278 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15709 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 990208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:49.278 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:49.278 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 86.738506423999993, 2024-09-17T21:21:49.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092728678999999994, 2024-09-17T21:21:49.278 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.278 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.278 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:49.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:49.278 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15709 2024-09-17T21:21:49.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.278 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:49.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:49.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521166+0000", 2024-09-17T21:21:49.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7000000000000004e-07 2024-09-17T21:21:49.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521169+0000", 2024-09-17T21:21:49.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2660000000000002e-06 2024-09-17T21:21:49.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521181+0000", 2024-09-17T21:21:49.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.208e-05 2024-09-17T21:21:49.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521194+0000", 2024-09-17T21:21:49.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2707e-05 2024-09-17T21:21:49.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613695+0000", 2024-09-17T21:21:49.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092500709 2024-09-17T21:21:49.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613750+0000", 2024-09-17T21:21:49.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.503e-05 2024-09-17T21:21:49.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613894+0000", 2024-09-17T21:21:49.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144117 2024-09-17T21:21:49.282 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.282 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.282 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.282 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15710 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 994304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:49.282 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:49.282 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 86.738461857999994, 2024-09-17T21:21:49.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092957729000000003, 2024-09-17T21:21:49.282 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.282 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.282 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:49.283 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:49.283 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15710 2024-09-17T21:21:49.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.283 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:49.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:49.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:49.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8000000000000005e-07 2024-09-17T21:21:49.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521216+0000", 2024-09-17T21:21:49.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3859999999999998e-06 2024-09-17T21:21:49.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521234+0000", 2024-09-17T21:21:49.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:21:49.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521248+0000", 2024-09-17T21:21:49.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:21:49.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613926+0000", 2024-09-17T21:21:49.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092678324000000006 2024-09-17T21:21:49.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613974+0000", 2024-09-17T21:21:49.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7754999999999999e-05 2024-09-17T21:21:49.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614167+0000", 2024-09-17T21:21:49.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193508 2024-09-17T21:21:49.286 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.286 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.286 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.286 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15711 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 998400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:49.286 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:49.286 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 86.738404475999999, 2024-09-17T21:21:49.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093268000000000004, 2024-09-17T21:21:49.287 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.287 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.287 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.287 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.287 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:49.287 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:49.287 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15711 2024-09-17T21:21:49.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.287 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:49.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:49.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521268+0000", 2024-09-17T21:21:49.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-17T21:21:49.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521273+0000", 2024-09-17T21:21:49.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:21:49.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521290+0000", 2024-09-17T21:21:49.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7269999999999999e-05 2024-09-17T21:21:49.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521304+0000", 2024-09-17T21:21:49.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3626e-05 2024-09-17T21:21:49.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614220+0000", 2024-09-17T21:21:49.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092916367 2024-09-17T21:21:49.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614322+0000", 2024-09-17T21:21:49.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102212 2024-09-17T21:21:49.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614535+0000", 2024-09-17T21:21:49.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021266199999999999 2024-09-17T21:21:49.290 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.291 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.291 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.291 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15712 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1002496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:49.291 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:49.291 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 86.738348981000001, 2024-09-17T21:21:49.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093568707000000001, 2024-09-17T21:21:49.291 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.291 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.291 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.291 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.291 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:49.291 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:49.291 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15712 2024-09-17T21:21:49.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.292 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:49.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:49.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521324+0000", 2024-09-17T21:21:49.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.17e-06 2024-09-17T21:21:49.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521334+0000", 2024-09-17T21:21:49.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9369999999999999e-06 2024-09-17T21:21:49.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521356+0000", 2024-09-17T21:21:49.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2045999999999999e-05 2024-09-17T21:21:49.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521370+0000", 2024-09-17T21:21:49.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4270000000000001e-05 2024-09-17T21:21:49.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614583+0000", 2024-09-17T21:21:49.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093213220999999999 2024-09-17T21:21:49.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614680+0000", 2024-09-17T21:21:49.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6900999999999999e-05 2024-09-17T21:21:49.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614891+0000", 2024-09-17T21:21:49.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021116200000000001 2024-09-17T21:21:49.295 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.295 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.295 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.295 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15713 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1006592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:49.295 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:49.295 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 86.738282279000003, 2024-09-17T21:21:49.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093834641999999996, 2024-09-17T21:21:49.295 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.295 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.295 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.296 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.296 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:49.296 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:49.296 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15713 2024-09-17T21:21:49.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.296 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:49.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:49.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521390+0000", 2024-09-17T21:21:49.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:21:49.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521396+0000", 2024-09-17T21:21:49.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:21:49.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521417+0000", 2024-09-17T21:21:49.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1236000000000001e-05 2024-09-17T21:21:49.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521431+0000", 2024-09-17T21:21:49.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3873000000000001e-05 2024-09-17T21:21:49.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614938+0000", 2024-09-17T21:21:49.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093506522999999994 2024-09-17T21:21:49.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615025+0000", 2024-09-17T21:21:49.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7635999999999999e-05 2024-09-17T21:21:49.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615224+0000", 2024-09-17T21:21:49.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019844700000000001 2024-09-17T21:21:49.299 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.299 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.299 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.300 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15714 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:49.300 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:49.300 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 86.738219022999999, 2024-09-17T21:21:49.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094071925000000001, 2024-09-17T21:21:49.300 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.300 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.300 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.300 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.300 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:49.300 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:49.300 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15714 2024-09-17T21:21:49.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.300 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:49.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:49.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521454+0000", 2024-09-17T21:21:49.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.097e-06 2024-09-17T21:21:49.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521459+0000", 2024-09-17T21:21:49.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2660000000000001e-06 2024-09-17T21:21:49.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521481+0000", 2024-09-17T21:21:49.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1878999999999998e-05 2024-09-17T21:21:49.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521498+0000", 2024-09-17T21:21:49.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6807e-05 2024-09-17T21:21:49.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615271+0000", 2024-09-17T21:21:49.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093773286999999997 2024-09-17T21:21:49.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615366+0000", 2024-09-17T21:21:49.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5322000000000007e-05 2024-09-17T21:21:49.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615524+0000", 2024-09-17T21:21:49.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015826699999999999 2024-09-17T21:21:49.304 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.304 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.304 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.304 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15715 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1014784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:49.304 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:49.304 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 86.738154687999995, 2024-09-17T21:21:49.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094243840999999995, 2024-09-17T21:21:49.304 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.304 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.304 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.304 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.305 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:49.305 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:49.305 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15715 2024-09-17T21:21:49.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.305 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:49.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:49.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521518+0000", 2024-09-17T21:21:49.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4300000000000001e-06 2024-09-17T21:21:49.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521524+0000", 2024-09-17T21:21:49.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5860000000000004e-06 2024-09-17T21:21:49.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521541+0000", 2024-09-17T21:21:49.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7574000000000001e-05 2024-09-17T21:21:49.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521555+0000", 2024-09-17T21:21:49.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3540000000000001e-05 2024-09-17T21:21:49.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615560+0000", 2024-09-17T21:21:49.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.09400472 2024-09-17T21:21:49.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615614+0000", 2024-09-17T21:21:49.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4693e-05 2024-09-17T21:21:49.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615761+0000", 2024-09-17T21:21:49.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000146298 2024-09-17T21:21:49.308 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.308 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.309 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.309 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15716 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1018880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:49.309 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:49.309 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 86.738098343000004, 2024-09-17T21:21:49.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094400676000000003, 2024-09-17T21:21:49.309 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.309 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.309 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.309 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.309 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:49.309 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:49.310 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15716 2024-09-17T21:21:49.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.310 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:49.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:49.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521574+0000", 2024-09-17T21:21:49.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1939999999999999e-06 2024-09-17T21:21:49.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521579+0000", 2024-09-17T21:21:49.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:21:49.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521597+0000", 2024-09-17T21:21:49.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8074e-05 2024-09-17T21:21:49.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521613+0000", 2024-09-17T21:21:49.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5316e-05 2024-09-17T21:21:49.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615780+0000", 2024-09-17T21:21:49.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094166874999999997 2024-09-17T21:21:49.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615836+0000", 2024-09-17T21:21:49.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6926999999999999e-05 2024-09-17T21:21:49.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615974+0000", 2024-09-17T21:21:49.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013741400000000001 2024-09-17T21:21:49.313 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.313 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.313 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.313 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15717 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1022976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:49.313 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:49.314 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 86.738039646999994, 2024-09-17T21:21:49.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094586723999999997, 2024-09-17T21:21:49.314 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.314 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.314 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.314 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.314 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:49.314 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:49.314 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15717 2024-09-17T21:21:49.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.314 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:49.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:49.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521633+0000", 2024-09-17T21:21:49.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:21:49.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521638+0000", 2024-09-17T21:21:49.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:21:49.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521659+0000", 2024-09-17T21:21:49.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-05 2024-09-17T21:21:49.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521675+0000", 2024-09-17T21:21:49.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5688999999999998e-05 2024-09-17T21:21:49.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616008+0000", 2024-09-17T21:21:49.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094333314000000001 2024-09-17T21:21:49.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616066+0000", 2024-09-17T21:21:49.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7797000000000002e-05 2024-09-17T21:21:49.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616219+0000", 2024-09-17T21:21:49.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015281100000000001 2024-09-17T21:21:49.318 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.318 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.318 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.318 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15718 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:49.318 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:49.318 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 86.737939572000002, 2024-09-17T21:21:49.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094703674000000002, 2024-09-17T21:21:49.319 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.319 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.319 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.319 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.319 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:49.319 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:49.319 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15718 2024-09-17T21:21:49.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.319 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:49.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:49.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521734+0000", 2024-09-17T21:21:49.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.61e-06 2024-09-17T21:21:49.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521740+0000", 2024-09-17T21:21:49.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:21:49.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521754+0000", 2024-09-17T21:21:49.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3793e-05 2024-09-17T21:21:49.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521763+0000", 2024-09-17T21:21:49.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5129999999999996e-06 2024-09-17T21:21:49.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616256+0000", 2024-09-17T21:21:49.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094492919999999994 2024-09-17T21:21:49.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616312+0000", 2024-09-17T21:21:49.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6737000000000002e-05 2024-09-17T21:21:49.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616436+0000", 2024-09-17T21:21:49.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012335800000000001 2024-09-17T21:21:49.323 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.323 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.323 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.323 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15719 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1031168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:49.323 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:49.323 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 86.737886166999999, 2024-09-17T21:21:49.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094888963000000007, 2024-09-17T21:21:49.323 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.323 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.323 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.323 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.323 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:49.324 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:49.324 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15719 2024-09-17T21:21:49.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.324 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:49.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:49.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521786+0000", 2024-09-17T21:21:49.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.074e-06 2024-09-17T21:21:49.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521791+0000", 2024-09-17T21:21:49.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3030000000000001e-06 2024-09-17T21:21:49.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521804+0000", 2024-09-17T21:21:49.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3027000000000001e-05 2024-09-17T21:21:49.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521811+0000", 2024-09-17T21:21:49.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6090000000000003e-06 2024-09-17T21:21:49.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616466+0000", 2024-09-17T21:21:49.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094654294 2024-09-17T21:21:49.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616523+0000", 2024-09-17T21:21:49.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6953999999999997e-05 2024-09-17T21:21:49.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616674+0000", 2024-09-17T21:21:49.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015170200000000001 2024-09-17T21:21:49.327 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.327 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.327 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.328 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15720 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1035264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:49.328 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:49.328 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 86.737849327000006, 2024-09-17T21:21:49.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095067061999999994, 2024-09-17T21:21:49.328 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.328 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.328 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.328 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.328 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:49.328 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:49.328 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15720 2024-09-17T21:21:49.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.329 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:49.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:49.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521823+0000", 2024-09-17T21:21:49.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:21:49.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521826+0000", 2024-09-17T21:21:49.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2100000000000002e-06 2024-09-17T21:21:49.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521835+0000", 2024-09-17T21:21:49.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1630000000000002e-06 2024-09-17T21:21:49.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521842+0000", 2024-09-17T21:21:49.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:21:49.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616711+0000", 2024-09-17T21:21:49.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094868859 2024-09-17T21:21:49.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616764+0000", 2024-09-17T21:21:49.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2778999999999997e-05 2024-09-17T21:21:49.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616889+0000", 2024-09-17T21:21:49.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012533100000000001 2024-09-17T21:21:49.332 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.332 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.332 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.332 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15721 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1039360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:49.332 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:49.332 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 86.737802217999999, 2024-09-17T21:21:49.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095277219999999996, 2024-09-17T21:21:49.332 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.332 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.332 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.333 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.333 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:49.333 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:49.333 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15721 2024-09-17T21:21:49.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.333 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:49.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:49.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521871+0000", 2024-09-17T21:21:49.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.443e-06 2024-09-17T21:21:49.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521876+0000", 2024-09-17T21:21:49.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0300000000000001e-06 2024-09-17T21:21:49.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521886+0000", 2024-09-17T21:21:49.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186e-05 2024-09-17T21:21:49.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521893+0000", 2024-09-17T21:21:49.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.08e-06 2024-09-17T21:21:49.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616918+0000", 2024-09-17T21:21:49.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095025223000000006 2024-09-17T21:21:49.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616977+0000", 2024-09-17T21:21:49.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8403999999999999e-05 2024-09-17T21:21:49.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617147+0000", 2024-09-17T21:21:49.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016985399999999999 2024-09-17T21:21:49.336 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.336 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.336 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.336 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15722 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:49.336 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:49.337 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 86.737767769000001, 2024-09-17T21:21:49.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095459783000000006, 2024-09-17T21:21:49.337 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.337 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.337 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.337 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.337 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:49.337 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:49.337 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15722 2024-09-17T21:21:49.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.337 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:49.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:49.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:49.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7700000000000004e-07 2024-09-17T21:21:49.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521907+0000", 2024-09-17T21:21:49.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.813e-06 2024-09-17T21:21:49.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521922+0000", 2024-09-17T21:21:49.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4533e-05 2024-09-17T21:21:49.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521934+0000", 2024-09-17T21:21:49.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2469999999999999e-05 2024-09-17T21:21:49.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617187+0000", 2024-09-17T21:21:49.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095252311000000006 2024-09-17T21:21:49.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617240+0000", 2024-09-17T21:21:49.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3740999999999999e-05 2024-09-17T21:21:49.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617364+0000", 2024-09-17T21:21:49.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-17T21:21:49.340 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.341 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.341 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.341 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15723 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:49.341 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:49.341 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 86.737723903000003, 2024-09-17T21:21:49.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095653563999999996, 2024-09-17T21:21:49.341 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.341 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.341 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.341 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.341 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:49.341 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:49.342 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15723 2024-09-17T21:21:49.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.342 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:49.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:49.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521949+0000", 2024-09-17T21:21:49.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:21:49.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521953+0000", 2024-09-17T21:21:49.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5870000000000003e-06 2024-09-17T21:21:49.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521970+0000", 2024-09-17T21:21:49.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7282000000000002e-05 2024-09-17T21:21:49.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521984+0000", 2024-09-17T21:21:49.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:21:49.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617383+0000", 2024-09-17T21:21:49.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095399063000000006 2024-09-17T21:21:49.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617437+0000", 2024-09-17T21:21:49.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3931999999999998e-05 2024-09-17T21:21:49.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617601+0000", 2024-09-17T21:21:49.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016407300000000001 2024-09-17T21:21:49.345 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.345 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.345 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.345 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5629.0:2384 3.1c 3:38c1263e:::rbd_data.1152d37183c3.000000000000000b:head [read 11780096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-17T21:21:49.346 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:49.346 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 73.834893073000003, 2024-09-17T21:21:49.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0246185320000001, 2024-09-17T21:21:49.346 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.346 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.346 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.346 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.346 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5629", 2024-09-17T21:21:49.346 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3798711329", 2024-09-17T21:21:49.346 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2384 2024-09-17T21:21:49.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.346 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:49.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:49.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424780+0000", 2024-09-17T21:21:49.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.13e-06 2024-09-17T21:21:49.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424785+0000", 2024-09-17T21:21:49.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4739999999999996e-06 2024-09-17T21:21:49.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424798+0000", 2024-09-17T21:21:49.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2704000000000001e-05 2024-09-17T21:21:49.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424806+0000", 2024-09-17T21:21:49.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9899999999999997e-06 2024-09-17T21:21:49.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424868+0000", 2024-09-17T21:21:49.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999997e-05 2024-09-17T21:21:49.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:49.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424875+0000", 2024-09-17T21:21:49.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0870000000000001e-06 2024-09-17T21:21:49.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428897+0000", 2024-09-17T21:21:49.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040226080000000004 2024-09-17T21:21:49.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:49.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428913+0000", 2024-09-17T21:21:49.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:21:49.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.448963+0000", 2024-09-17T21:21:49.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0200498659999999 2024-09-17T21:21:49.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449158+0000", 2024-09-17T21:21:49.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019581000000000001 2024-09-17T21:21:49.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449397+0000", 2024-09-17T21:21:49.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023871 2024-09-17T21:21:49.351 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.351 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.351 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.352 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:26043 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_remove_image in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-17T21:21:49.352 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:21:49.352 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.860361953, 2024-09-17T21:21:49.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.019978351, 2024-09-17T21:21:49.352 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.352 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.352 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:49.352 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.352 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:49.352 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:49.352 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 26043 2024-09-17T21:21:49.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.352 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:21:49.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:21:49.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399314+0000", 2024-09-17T21:21:49.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.963e-06 2024-09-17T21:21:49.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399330+0000", 2024-09-17T21:21:49.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6684000000000002e-05 2024-09-17T21:21:49.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399372+0000", 2024-09-17T21:21:49.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1819000000000001e-05 2024-09-17T21:21:49.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399396+0000", 2024-09-17T21:21:49.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.406e-05 2024-09-17T21:21:49.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399571+0000", 2024-09-17T21:21:49.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017444800000000001 2024-09-17T21:21:49.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:49.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399595+0000", 2024-09-17T21:21:49.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4709999999999999e-05 2024-09-17T21:21:49.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.412735+0000", 2024-09-17T21:21:49.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0131394869999999 2024-09-17T21:21:49.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.413090+0000", 2024-09-17T21:21:49.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000355006 2024-09-17T21:21:49.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:21:49.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.413659+0000", 2024-09-17T21:21:49.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056922699999999995 2024-09-17T21:21:49.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:49.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.418940+0000", 2024-09-17T21:21:49.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052814109999999997 2024-09-17T21:21:49.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:49.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.418992+0000", 2024-09-17T21:21:49.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1646e-05 2024-09-17T21:21:49.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:49.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.419168+0000", 2024-09-17T21:21:49.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017573400000000001 2024-09-17T21:21:49.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.419288+0000", 2024-09-17T21:21:49.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012015600000000001 2024-09-17T21:21:49.358 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.358 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.358 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.358 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.358 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.358 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:49.359 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:21:49.360 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:49.357+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:49.460 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:49.457+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:49.539 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:49.539 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:49.539 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:49.539 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:49.554 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:21:49.595 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:49.593+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:49.662 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:49.657+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:49.727 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:49.727 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:49.727 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:49.727 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:49.727 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:49.741 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:21:49.783 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:49.781+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:49.863 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:49.861+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:49.909 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:49.909 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:49.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:49.909 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:49.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.909 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:21:49.909 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:49.910 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 208.30057226700001, 2024-09-17T21:21:49.910 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:21:49.910 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.910 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.910 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:49.910 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.910 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.910 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:49.910 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.910 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.910 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:49.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.911 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:21:49.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:21:49.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.911 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:21:49.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:21:49.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:21:49.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:21:49.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:21:49.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:21:49.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:21:49.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:21:49.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.913 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:21:49.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:21:49.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.913 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:49.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:21:49.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:21:49.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.913 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:21:49.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:21:49.914 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.914 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.914 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.914 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.914 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:21:49.914 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:49.914 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 208.29062563100001, 2024-09-17T21:21:49.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:21:49.914 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.914 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.914 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:49.914 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.915 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.915 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:49.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.915 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.915 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:49.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.915 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.915 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:21:49.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:21:49.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:21:49.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:21:49.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:21:49.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:21:49.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:21:49.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:21:49.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.917 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:21:49.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:21:49.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.917 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:21:49.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:21:49.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.917 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:49.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:21:49.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:21:49.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.918 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.918 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:21:49.918 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:21:49.918 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.918 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.918 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.918 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:21:49.918 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:49.918 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 207.04747479100001, 2024-09-17T21:21:49.918 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:21:49.919 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.919 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.919 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:49.919 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.919 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:21:49.919 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:21:49.919 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:21:49.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.919 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.919 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:49.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:49.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:21:49.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:21:49.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:21:49.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:21:49.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:21:49.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:21:49.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:21:49.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:21:49.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:21:49.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:21:49.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:21:49.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:21:49.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:49.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:21:49.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:21:49.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.923 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:49.923 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:21:49.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:21:49.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.923 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:49.923 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:21:49.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:21:49.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.923 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:49.923 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:21:49.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:21:49.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:21:49.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:21:49.924 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.924 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.924 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.924 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:21:49.924 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:49.924 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 207.037829305, 2024-09-17T21:21:49.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:21:49.925 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.925 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.925 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:49.925 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.925 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:21:49.925 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:21:49.925 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:21:49.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.925 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.925 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:49.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:49.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:21:49.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:21:49.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:21:49.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:21:49.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:21:49.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:21:49.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:21:49.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:21:49.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:21:49.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:21:49.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:21:49.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:21:49.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:49.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:21:49.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:21:49.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:49.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:21:49.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:21:49.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:49.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:21:49.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:21:49.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:49.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:21:49.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:21:49.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.930 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:21:49.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:21:49.930 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.930 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.930 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.930 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:21:49.930 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:49.930 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 193.66889327199999, 2024-09-17T21:21:49.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:21:49.930 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.930 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.930 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.931 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.931 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:21:49.931 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:21:49.931 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:21:49.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.931 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:49.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:49.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:21:49.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:21:49.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:21:49.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:21:49.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:21:49.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:21:49.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.933 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:21:49.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:21:49.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.933 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:21:49.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:21:49.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:21:49.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:21:49.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:21:49.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:21:49.934 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.934 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.934 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.934 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:49.934 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:49.935 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 175.46720870999999, 2024-09-17T21:21:49.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:21:49.935 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.935 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.935 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.935 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.935 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:49.935 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:49.935 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:21:49.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.935 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:49.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:49.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:21:49.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:21:49.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.937 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:21:49.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:21:49.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.937 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.937 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:21:49.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:21:49.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.937 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.937 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:21:49.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:21:49.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:21:49.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:21:49.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:21:49.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:21:49.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:21:49.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:21:49.939 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.939 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.939 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.939 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.939 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:49.939 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:49.939 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 175.46671413300001, 2024-09-17T21:21:49.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:21:49.939 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.939 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.939 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.940 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.940 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:49.940 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:49.940 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:21:49.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.940 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.940 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.940 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:49.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.940 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.940 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:49.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.941 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:21:49.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:21:49.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.941 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:21:49.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:21:49.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:21:49.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:21:49.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:21:49.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:21:49.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:21:49.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:21:49.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:21:49.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:21:49.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:21:49.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:21:49.943 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.943 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.943 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.944 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:49.944 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:49.944 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 175.466658752, 2024-09-17T21:21:49.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:21:49.944 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.944 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.944 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.944 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.944 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:49.944 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:49.944 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:21:49.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.945 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.945 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.945 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.945 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:49.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.945 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.945 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.945 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:49.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.945 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.945 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:21:49.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:21:49.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:21:49.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:21:49.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:21:49.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:21:49.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:21:49.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:21:49.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:21:49.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:21:49.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:21:49.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:21:49.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:21:49.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:21:49.948 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.948 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.948 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.948 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:49.948 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:49.948 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 175.46660247400001, 2024-09-17T21:21:49.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:21:49.949 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.949 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.949 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.949 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.949 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:49.949 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:49.949 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:21:49.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.949 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:49.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.950 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.950 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.950 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:49.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.950 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.950 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.950 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:21:49.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:21:49.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.950 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.950 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.950 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:21:49.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:21:49.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:21:49.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:21:49.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:21:49.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:21:49.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:21:49.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:21:49.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:21:49.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:21:49.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:21:49.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:21:49.952 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.952 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.953 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.953 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:49.953 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:49.953 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 175.46565283499999, 2024-09-17T21:21:49.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:21:49.953 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.953 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.953 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.953 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.953 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:49.953 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:49.953 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:21:49.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.954 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.954 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.954 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:49.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.954 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.954 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:49.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.955 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.955 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:21:49.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:21:49.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.955 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.955 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:21:49.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:21:49.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.955 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.955 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:21:49.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:21:49.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.956 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.956 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:21:49.956 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:21:49.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.956 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.956 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:21:49.956 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:21:49.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.956 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.956 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:21:49.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:21:49.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:21:49.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:21:49.957 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.957 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.957 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.957 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:49.957 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:49.957 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 175.465574215, 2024-09-17T21:21:49.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:21:49.958 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.958 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.958 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.958 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.958 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:49.958 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:49.958 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:21:49.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.958 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:49.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.959 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:49.959 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.959 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:21:49.959 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:21:49.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.959 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:21:49.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:21:49.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.960 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.960 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:21:49.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:21:49.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.960 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.960 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:21:49.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:21:49.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.960 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:21:49.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:21:49.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:21:49.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:21:49.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:21:49.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:21:49.961 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.961 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.962 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.962 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:49.962 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:49.962 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 175.46546127600001, 2024-09-17T21:21:49.962 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:21:49.962 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.962 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.962 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.962 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.962 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:49.962 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:49.962 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:21:49.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.963 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.963 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:49.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.963 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:49.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.963 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:21:49.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:21:49.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:21:49.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:21:49.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:21:49.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:21:49.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:21:49.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:21:49.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.965 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.965 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:21:49.965 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:21:49.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.965 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.965 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:21:49.965 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:21:49.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.965 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:21:49.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:21:49.966 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.966 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.966 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.966 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:49.966 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:49.966 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 175.46480022200001, 2024-09-17T21:21:49.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:21:49.966 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.966 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.966 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.967 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.967 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:49.967 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:49.967 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:21:49.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.967 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.967 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.967 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:49.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.967 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.967 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:49.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:21:49.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:21:49.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:21:49.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:21:49.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:21:49.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:21:49.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.969 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.969 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:21:49.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:21:49.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.969 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.969 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:21:49.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:21:49.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.969 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.969 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:21:49.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:21:49.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.970 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.970 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:21:49.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:21:49.970 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.970 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.970 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.970 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:49.970 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:49.971 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 175.46473541099999, 2024-09-17T21:21:49.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:21:49.971 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.971 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.971 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.971 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.971 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:49.971 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:49.971 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:21:49.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.971 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:49.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:49.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:21:49.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:21:49.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:21:49.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:21:49.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:21:49.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:21:49.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:21:49.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:21:49.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:21:49.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:21:49.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:21:49.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:21:49.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:21:49.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:21:49.974 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.974 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.975 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.975 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:49.975 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:49.975 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 175.46467443399999, 2024-09-17T21:21:49.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:21:49.975 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.975 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.975 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:49.975 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:49.975 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:49.975 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:49.975 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:21:49.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.976 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.976 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:49.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.976 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:49.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.976 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:21:49.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:21:49.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:21:49.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:21:49.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:21:49.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:21:49.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:21:49.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:21:49.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:21:49.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:21:49.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:21:49.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:21:49.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:21:49.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:21:49.979 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.979 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.979 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.979 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:21:49.979 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:49.979 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 120.567103249, 2024-09-17T21:21:49.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:21:49.979 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.979 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.980 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:49.980 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:49.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:49.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:21:49.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:21:49.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:21:49.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:21:49.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:21:49.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:21:49.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:21:49.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:21:49.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:21:49.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:21:49.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:21:49.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:21:49.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:49.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:21:49.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:21:49.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:21:49.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:21:49.983 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.983 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.983 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.984 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:21:49.984 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:49.984 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 120.358444855, 2024-09-17T21:21:49.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:21:49.984 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.984 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.984 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:49.984 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:49.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:49.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:21:49.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:21:49.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:21:49.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:21:49.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.986 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:21:49.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:21:49.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.986 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:21:49.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:21:49.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:21:49.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:21:49.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:21:49.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:21:49.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:49.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:21:49.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:21:49.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:21:49.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:21:49.988 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.988 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.988 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.988 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:9547 3.2 e147/132 3:42b518d9:::rbd_data.1152f056b73f.0000000000000001:head v 147'513, mlcod=142'511)", 2024-09-17T21:21:49.988 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:49.988 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 47.433471732999998, 2024-09-17T21:21:49.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.031059566, 2024-09-17T21:21:49.988 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.988 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.989 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:49.989 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:49.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:49.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471293+0000", 2024-09-17T21:21:49.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2029999999999999e-06 2024-09-17T21:21:49.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480606+0000", 2024-09-17T21:21:49.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093123839999999999 2024-09-17T21:21:49.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480653+0000", 2024-09-17T21:21:49.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7852000000000002e-05 2024-09-17T21:21:49.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480673+0000", 2024-09-17T21:21:49.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0013e-05 2024-09-17T21:21:49.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480862+0000", 2024-09-17T21:21:49.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018833599999999999 2024-09-17T21:21:49.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480906+0000", 2024-09-17T21:21:49.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4162000000000002e-05 2024-09-17T21:21:49.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.992 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:49.992 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502259+0000", 2024-09-17T21:21:49.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021353154999999999 2024-09-17T21:21:49.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.992 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.992 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502351+0000", 2024-09-17T21:21:49.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1460999999999997e-05 2024-09-17T21:21:49.992 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.992 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.992 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.993 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:15234 3.1 e147/134 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head v 147'917, mlcod=145'914)", 2024-09-17T21:21:49.993 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:49.993 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.800811115000002, 2024-09-17T21:21:49.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045303695999999997, 2024-09-17T21:21:49.993 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.993 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.993 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:49.993 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:49.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:49.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103954+0000", 2024-09-17T21:21:49.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2259999999999999e-06 2024-09-17T21:21:49.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119105+0000", 2024-09-17T21:21:49.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015150871 2024-09-17T21:21:49.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119141+0000", 2024-09-17T21:21:49.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6239000000000003e-05 2024-09-17T21:21:49.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:49.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119159+0000", 2024-09-17T21:21:49.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7946999999999999e-05 2024-09-17T21:21:49.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:49.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119344+0000", 2024-09-17T21:21:49.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018524599999999999 2024-09-17T21:21:49.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:49.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119399+0000", 2024-09-17T21:21:49.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4922e-05 2024-09-17T21:21:49.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:49.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149165+0000", 2024-09-17T21:21:49.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029765697000000001 2024-09-17T21:21:49.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:49.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149255+0000", 2024-09-17T21:21:49.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0548000000000001e-05 2024-09-17T21:21:49.997 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.997 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:49.997 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:49.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.997 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:19859 3.8 e147/133 3:12f72409:::rbd_data.1152f056b73f.0000000000000003:head v 147'482, mlcod=145'479)", 2024-09-17T21:21:49.997 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:49.997 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.391920048000003, 2024-09-17T21:21:49.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033476895999999999, 2024-09-17T21:21:49.998 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:49.998 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:49.998 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:49.998 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:49.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:49.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:49.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:49.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:49.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:49.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:49.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512845+0000", 2024-09-17T21:21:49.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.886e-06 2024-09-17T21:21:49.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:49.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524104+0000", 2024-09-17T21:21:49.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011259668 2024-09-17T21:21:49.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:49.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:49.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:49.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524148+0000", 2024-09-17T21:21:49.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3386000000000001e-05 2024-09-17T21:21:50.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:50.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:50.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:50.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524172+0000", 2024-09-17T21:21:50.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4762000000000001e-05 2024-09-17T21:21:50.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:50.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:50.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:50.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524379+0000", 2024-09-17T21:21:50.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020671599999999999 2024-09-17T21:21:50.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:50.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:50.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:50.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524421+0000", 2024-09-17T21:21:50.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2182999999999999e-05 2024-09-17T21:21:50.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:50.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:50.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:50.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546216+0000", 2024-09-17T21:21:50.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021795101000000001 2024-09-17T21:21:50.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:50.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:50.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:50.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546320+0000", 2024-09-17T21:21:50.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103194 2024-09-17T21:21:50.001 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:50.001 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:50.002 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:50.002 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:50.002 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:50.002 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:50.003 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:21:50.004 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:49.999+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:50.104 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:50.101+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:50.135 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:50.135 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:50.149 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:21:50.205 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:50.201+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:50.272 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:50.272 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:50.287 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:21:50.306 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:50.301+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:50.407 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:50.405+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:50.415 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:50.416 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:50.430 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:21:50.444 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:21:50.508 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:50.505+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:50.552 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:50.552 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:50.567 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:21:50.613 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T21:21:50.613 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:50.611+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:50.637 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:50.637 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:50.658 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:50.658 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:50.681 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:50.686 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:21:50.686 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:21:50.692 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:50.692 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:50.706 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:21:50.709 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:50.705+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:50.811 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:50.807+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:50.890 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:50.890 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:50.904 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:21:50.951 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:50.949+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:51.012 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:51.007+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:51.036 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:51.036 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:51.052 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:21:51.143 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:51.141+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:51.214 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:51.209+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:51.221 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:51.221 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:51.236 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:21:51.323 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:51.321+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:51.400 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:51.401 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:51.415 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:21:51.416 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:51.413+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:51.436 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-17T21:21:51.457 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 0] out_osds: [2, 5] dead_osds: [2] live_osds: [0, 1, 3, 4, 5] 2024-09-17T21:21:51.457 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:21:51.457 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-17T21:21:51.457 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-17T21:21:51.516 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:51.513+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:51.587 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:51.587 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:51.602 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:21:51.633 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-17T21:21:51.647 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:51.645+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:51.717 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:51.713+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:51.768 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:51.769 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:51.783 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:21:51.794 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-17T21:21:51.827 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:51.825+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:51.919 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:51.915+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:51.949 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:51.950 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:51.956 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-17T21:21:51.963 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:21:52.040 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:52.039+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:52.082 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:52.082 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:52.096 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:21:52.121 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:52.119+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:52.220 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:52.220 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:52.221 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:52.217+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:52.234 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:21:52.322 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:52.319+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:52.411 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:52.411 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:52.422 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:52.417+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:52.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:21:52.523 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:52.519+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:52.566 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:52.567 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:52.581 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:21:52.628 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:52.627+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:52.687 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:21:52.725 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:52.721+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:52.752 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:52.752 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:52.770 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:21:52.825 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:52.821+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:52.830 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:52.830 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:52.847 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:52.848 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:52.848 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T21:21:52.868 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:52.868 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:52.888 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:52.889 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:52.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:52.889 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:52.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.889 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:21:52.889 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:52.889 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 156.37640484600001, 2024-09-17T21:21:52.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:21:52.889 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:52.889 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:52.889 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:52.889 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:52.889 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:21:52.890 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:21:52.890 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:21:52.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.890 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:52.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:52.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:52.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:52.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:52.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:52.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:21:52.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:21:52.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:52.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:21:52.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:21:52.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:52.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:21:52.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:21:52.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:52.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:21:52.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:21:52.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:21:52.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:21:52.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:52.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:21:52.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:21:52.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.893 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.893 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:21:52.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:21:52.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.893 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:52.893 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:21:52.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:21:52.893 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.893 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:52.894 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.894 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2597 3.1f 3:fb0e9615:::rbd_data.1152a8ae2019.000000000000000c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:52.894 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:52.894 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 132.04796841800001, 2024-09-17T21:21:52.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097064994000000002, 2024-09-17T21:21:52.894 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:52.894 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:52.894 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:52.894 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:52.894 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:52.894 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:52.894 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2597 2024-09-17T21:21:52.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.895 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:52.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:52.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:52.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:52.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:52.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:52.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838628+0000", 2024-09-17T21:21:52.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.187e-06 2024-09-17T21:21:52.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:52.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838654+0000", 2024-09-17T21:21:52.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:21:52.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:52.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838696+0000", 2024-09-17T21:21:52.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1226000000000001e-05 2024-09-17T21:21:52.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:52.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838715+0000", 2024-09-17T21:21:52.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9646000000000001e-05 2024-09-17T21:21:52.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838937+0000", 2024-09-17T21:21:52.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022163 2024-09-17T21:21:52.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:52.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.839284+0000", 2024-09-17T21:21:52.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034753799999999999 2024-09-17T21:21:52.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:52.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935691+0000", 2024-09-17T21:21:52.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.096406360999999996 2024-09-17T21:21:52.898 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.898 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:52.898 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.898 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2641 3.12 3:488a729f:::rbd_data.1152a8ae2019.0000000000000022:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:52.898 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:52.898 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 132.044511645, 2024-09-17T21:21:52.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098280953000000004, 2024-09-17T21:21:52.898 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:52.898 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:52.899 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:52.899 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:52.899 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:52.899 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:52.899 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2641 2024-09-17T21:21:52.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.899 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:52.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:52.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:52.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:52.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:52.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:52.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842083+0000", 2024-09-17T21:21:52.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5300000000000002e-07 2024-09-17T21:21:52.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:52.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842088+0000", 2024-09-17T21:21:52.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:21:52.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:52.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842105+0000", 2024-09-17T21:21:52.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6773e-05 2024-09-17T21:21:52.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:52.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842119+0000", 2024-09-17T21:21:52.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:21:52.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842320+0000", 2024-09-17T21:21:52.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020052700000000001 2024-09-17T21:21:52.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:52.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842845+0000", 2024-09-17T21:21:52.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052508600000000002 2024-09-17T21:21:52.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:52.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940363+0000", 2024-09-17T21:21:52.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097518795000000005 2024-09-17T21:21:52.902 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.902 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:52.902 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.903 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2674 3.12 3:4c87e3ac:::rbd_data.1152a8ae2019.0000000000000033:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:52.903 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:52.903 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 132.04414900800001, 2024-09-17T21:21:52.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098025125000000005, 2024-09-17T21:21:52.903 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:52.903 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:52.903 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:52.903 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:52.903 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:52.903 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:52.903 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2674 2024-09-17T21:21:52.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.903 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:52.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:52.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:52.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:52.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:52.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:52.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842447+0000", 2024-09-17T21:21:52.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3999999999999999e-06 2024-09-17T21:21:52.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:52.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842453+0000", 2024-09-17T21:21:52.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:21:52.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:52.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842472+0000", 2024-09-17T21:21:52.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9006000000000001e-05 2024-09-17T21:21:52.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:52.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842487+0000", 2024-09-17T21:21:52.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4866e-05 2024-09-17T21:21:52.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843594+0000", 2024-09-17T21:21:52.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011073249999999999 2024-09-17T21:21:52.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:52.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843668+0000", 2024-09-17T21:21:52.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3188000000000005e-05 2024-09-17T21:21:52.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:52.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843893+0000", 2024-09-17T21:21:52.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022503399999999999 2024-09-17T21:21:52.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:52.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850439+0000", 2024-09-17T21:21:52.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065467880000000004 2024-09-17T21:21:52.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:52.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940409+0000", 2024-09-17T21:21:52.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089969511000000002 2024-09-17T21:21:52.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:52.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940444+0000", 2024-09-17T21:21:52.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5345999999999997e-05 2024-09-17T21:21:52.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:52.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940470+0000", 2024-09-17T21:21:52.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5964000000000001e-05 2024-09-17T21:21:52.908 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.908 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:52.908 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.908 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2707 3.1f 3:f805e65e:::rbd_data.1152a8ae2019.0000000000000049:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:52.908 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:52.908 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 132.03910234700001, 2024-09-17T21:21:52.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088327300999999997, 2024-09-17T21:21:52.909 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:52.909 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:52.909 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:52.909 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:52.909 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:52.909 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:52.909 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2707 2024-09-17T21:21:52.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.909 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:52.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:52.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:52.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:52.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:52.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:52.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847494+0000", 2024-09-17T21:21:52.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:21:52.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:52.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847504+0000", 2024-09-17T21:21:52.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:21:52.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:52.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847526+0000", 2024-09-17T21:21:52.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1976000000000001e-05 2024-09-17T21:21:52.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:52.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847541+0000", 2024-09-17T21:21:52.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5277e-05 2024-09-17T21:21:52.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856142+0000", 2024-09-17T21:21:52.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086006710000000007 2024-09-17T21:21:52.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:52.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856435+0000", 2024-09-17T21:21:52.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029301499999999999 2024-09-17T21:21:52.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:52.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935819+0000", 2024-09-17T21:21:52.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079384448999999996 2024-09-17T21:21:52.912 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.913 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:52.913 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.913 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2730 3.12 3:48b3422a:::rbd_data.1152a8ae2019.0000000000000057:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:52.913 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:52.913 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 132.02990944800001, 2024-09-17T21:21:52.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083879975999999995, 2024-09-17T21:21:52.913 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:52.913 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:52.913 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:52.913 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:52.913 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:52.913 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:52.914 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2730 2024-09-17T21:21:52.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.914 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:52.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:52.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:52.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:52.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:52.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:52.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856687+0000", 2024-09-17T21:21:52.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:21:52.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:52.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856697+0000", 2024-09-17T21:21:52.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0207000000000001e-05 2024-09-17T21:21:52.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:52.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856808+0000", 2024-09-17T21:21:52.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011071200000000001 2024-09-17T21:21:52.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:52.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856828+0000", 2024-09-17T21:21:52.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0219000000000001e-05 2024-09-17T21:21:52.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856907+0000", 2024-09-17T21:21:52.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9661999999999997e-05 2024-09-17T21:21:52.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:52.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.857019+0000", 2024-09-17T21:21:52.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011148499999999999 2024-09-17T21:21:52.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:52.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940565+0000", 2024-09-17T21:21:52.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083545791999999994 2024-09-17T21:21:52.917 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.917 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:52.917 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.917 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2743 3.12 3:4d036e62:::rbd_data.1152a8ae2019.000000000000005f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:52.917 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:52.917 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 132.02596394700001, 2024-09-17T21:21:52.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080059962999999998, 2024-09-17T21:21:52.918 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:52.918 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:52.918 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:52.918 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:52.918 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:52.918 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:52.918 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2743 2024-09-17T21:21:52.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.918 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:52.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:52.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:52.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:52.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:52.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:52.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860631+0000", 2024-09-17T21:21:52.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:21:52.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:52.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860636+0000", 2024-09-17T21:21:52.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5279999999999997e-06 2024-09-17T21:21:52.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:52.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860649+0000", 2024-09-17T21:21:52.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-17T21:21:52.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:52.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860655+0000", 2024-09-17T21:21:52.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:21:52.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860731+0000", 2024-09-17T21:21:52.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5920000000000005e-05 2024-09-17T21:21:52.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:52.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.861070+0000", 2024-09-17T21:21:52.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338535 2024-09-17T21:21:52.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:52.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940690+0000", 2024-09-17T21:21:52.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079620503999999995 2024-09-17T21:21:52.921 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.922 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:52.922 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.922 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3378 3.12 3:4832a51b:::rbd_data.1152a8ae2019.000000000000006c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:52.922 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:52.922 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 131.997600097, 2024-09-17T21:21:52.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051812516000000003, 2024-09-17T21:21:52.922 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:52.922 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:52.922 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:52.922 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:52.922 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:52.922 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:52.922 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3378 2024-09-17T21:21:52.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.923 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:52.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:52.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:52.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:52.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:52.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:52.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888995+0000", 2024-09-17T21:21:52.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:21:52.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:52.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889006+0000", 2024-09-17T21:21:52.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1012999999999999e-05 2024-09-17T21:21:52.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:52.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889032+0000", 2024-09-17T21:21:52.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6441e-05 2024-09-17T21:21:52.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:52.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889040+0000", 2024-09-17T21:21:52.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0639999999999994e-06 2024-09-17T21:21:52.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889278+0000", 2024-09-17T21:21:52.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023785899999999999 2024-09-17T21:21:52.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:52.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889408+0000", 2024-09-17T21:21:52.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129219 2024-09-17T21:21:52.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:52.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889723+0000", 2024-09-17T21:21:52.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000315264 2024-09-17T21:21:52.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:52.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.891972+0000", 2024-09-17T21:21:52.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002249633 2024-09-17T21:21:52.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:52.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940733+0000", 2024-09-17T21:21:52.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048760626000000001 2024-09-17T21:21:52.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:52.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940785+0000", 2024-09-17T21:21:52.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1906000000000001e-05 2024-09-17T21:21:52.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:52.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940807+0000", 2024-09-17T21:21:52.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1554999999999999e-05 2024-09-17T21:21:52.927 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.927 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:52.927 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.928 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3379 3.12 3:4e83fd71:::rbd_data.1152a8ae2019.000000000000006d:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:52.928 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:52.928 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 131.99751588999999, 2024-09-17T21:21:52.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051804144000000003, 2024-09-17T21:21:52.928 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:52.928 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:52.928 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:52.928 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:52.928 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:52.928 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:52.928 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3379 2024-09-17T21:21:52.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.929 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:52.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:52.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:52.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:52.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:52.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:52.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889079+0000", 2024-09-17T21:21:52.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1800000000000004e-07 2024-09-17T21:21:52.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:52.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889083+0000", 2024-09-17T21:21:52.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:21:52.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:52.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889094+0000", 2024-09-17T21:21:52.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1562999999999999e-05 2024-09-17T21:21:52.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:52.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889102+0000", 2024-09-17T21:21:52.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.401e-06 2024-09-17T21:21:52.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:21:52.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014279259999999999 2024-09-17T21:21:52.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:52.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890609+0000", 2024-09-17T21:21:52.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9566000000000004e-05 2024-09-17T21:21:52.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:52.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890789+0000", 2024-09-17T21:21:52.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017949700000000001 2024-09-17T21:21:52.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:52.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.893929+0000", 2024-09-17T21:21:52.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003140538 2024-09-17T21:21:52.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:52.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940834+0000", 2024-09-17T21:21:52.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046904811999999997 2024-09-17T21:21:52.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:52.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940860+0000", 2024-09-17T21:21:52.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5341999999999999e-05 2024-09-17T21:21:52.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:52.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940882+0000", 2024-09-17T21:21:52.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2878999999999999e-05 2024-09-17T21:21:52.933 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.933 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:52.933 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.933 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:2644 3.1e 3:79615585:::rbd_data.115266d0b9f2.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2475520~14848,write 3106304~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:21:52.933 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:52.933 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.374887487, 2024-09-17T21:21:52.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050733484000000002, 2024-09-17T21:21:52.934 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:52.934 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:52.934 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:52.934 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:52.934 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:21:52.934 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:21:52.934 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2644 2024-09-17T21:21:52.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.934 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:52.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:52.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:52.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:52.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:52.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:52.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511708+0000", 2024-09-17T21:21:52.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-17T21:21:52.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:52.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522196+0000", 2024-09-17T21:21:52.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010487712999999999 2024-09-17T21:21:52.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:52.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522223+0000", 2024-09-17T21:21:52.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7064999999999999e-05 2024-09-17T21:21:52.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:52.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522236+0000", 2024-09-17T21:21:52.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2931e-05 2024-09-17T21:21:52.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522348+0000", 2024-09-17T21:21:52.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111906 2024-09-17T21:21:52.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:52.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522554+0000", 2024-09-17T21:21:52.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020632699999999999 2024-09-17T21:21:52.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:21:52.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.523185+0000", 2024-09-17T21:21:52.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063081400000000005 2024-09-17T21:21:52.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:52.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.547011+0000", 2024-09-17T21:21:52.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023825570000000001 2024-09-17T21:21:52.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:52.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562268+0000", 2024-09-17T21:21:52.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015257514 2024-09-17T21:21:52.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:52.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562367+0000", 2024-09-17T21:21:52.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8678000000000006e-05 2024-09-17T21:21:52.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:52.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562440+0000", 2024-09-17T21:21:52.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3337000000000006e-05 2024-09-17T21:21:52.939 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.939 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:52.939 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.939 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:231 3.1 3:856fed78:::rbd_data.115220dd393e.000000000000000b:head [read 0~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:52.939 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:52.939 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 94.932132218000007, 2024-09-17T21:21:52.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34254251699999999, 2024-09-17T21:21:52.939 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:52.939 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:52.939 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:52.939 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:52.939 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:52.940 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:52.940 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 231 2024-09-17T21:21:52.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.940 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:52.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:52.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:52.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:52.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:52.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:52.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954464+0000", 2024-09-17T21:21:52.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:21:52.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:52.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954475+0000", 2024-09-17T21:21:52.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1146e-05 2024-09-17T21:21:52.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:52.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954506+0000", 2024-09-17T21:21:52.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0292999999999999e-05 2024-09-17T21:21:52.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:52.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954524+0000", 2024-09-17T21:21:52.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8430000000000001e-05 2024-09-17T21:21:52.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954661+0000", 2024-09-17T21:21:52.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136403 2024-09-17T21:21:52.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:52.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954680+0000", 2024-09-17T21:21:52.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9743000000000001e-05 2024-09-17T21:21:52.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.296896+0000", 2024-09-17T21:21:52.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34221550499999998 2024-09-17T21:21:52.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:52.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297004+0000", 2024-09-17T21:21:52.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010865399999999999 2024-09-17T21:21:52.943 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.943 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:52.943 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.943 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:56 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:52.943 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:52.943 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 94.776498520999994, 2024-09-17T21:21:52.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18718773899999999, 2024-09-17T21:21:52.944 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:52.944 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:52.944 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:52.944 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:52.944 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:21:52.944 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:21:52.944 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 56 2024-09-17T21:21:52.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.944 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:52.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:52.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:52.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:52.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:52.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:52.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110098+0000", 2024-09-17T21:21:52.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:21:52.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:52.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110111+0000", 2024-09-17T21:21:52.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2289999999999999e-05 2024-09-17T21:21:52.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:52.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110143+0000", 2024-09-17T21:21:52.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1979e-05 2024-09-17T21:21:52.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:52.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110161+0000", 2024-09-17T21:21:52.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7839e-05 2024-09-17T21:21:52.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110319+0000", 2024-09-17T21:21:52.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015835400000000001 2024-09-17T21:21:52.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:52.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110353+0000", 2024-09-17T21:21:52.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3849000000000001e-05 2024-09-17T21:21:52.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297041+0000", 2024-09-17T21:21:52.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18668842299999999 2024-09-17T21:21:52.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:52.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297169+0000", 2024-09-17T21:21:52.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012786199999999999 2024-09-17T21:21:52.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:52.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297283+0000", 2024-09-17T21:21:52.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011430300000000001 2024-09-17T21:21:52.948 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.948 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:52.948 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.948 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:68 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:52.948 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:52.948 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 94.767957589999995, 2024-09-17T21:21:52.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17885582999999999, 2024-09-17T21:21:52.948 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:52.948 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:52.949 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:52.949 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:52.949 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:21:52.949 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:21:52.949 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 68 2024-09-17T21:21:52.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.949 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:52.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:52.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:52.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:52.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:52.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:52.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118639+0000", 2024-09-17T21:21:52.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.306e-06 2024-09-17T21:21:52.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:52.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118651+0000", 2024-09-17T21:21:52.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2567e-05 2024-09-17T21:21:52.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:52.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118682+0000", 2024-09-17T21:21:52.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0502e-05 2024-09-17T21:21:52.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:52.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118707+0000", 2024-09-17T21:21:52.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5463e-05 2024-09-17T21:21:52.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118889+0000", 2024-09-17T21:21:52.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018150999999999999 2024-09-17T21:21:52.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:52.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118910+0000", 2024-09-17T21:21:52.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:21:52.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297313+0000", 2024-09-17T21:21:52.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17840366699999999 2024-09-17T21:21:52.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:52.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297389+0000", 2024-09-17T21:21:52.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-17T21:21:52.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:52.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297492+0000", 2024-09-17T21:21:52.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103019 2024-09-17T21:21:52.953 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.953 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:52.953 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.953 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5890.0:15234 3.1 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8451072~29696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-17T21:21:52.953 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:52.953 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 49.793308777999997, 2024-09-17T21:21:52.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.057026052000000001, 2024-09-17T21:21:52.953 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:52.953 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:52.953 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:52.953 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:52.953 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5890", 2024-09-17T21:21:52.953 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3064344947", 2024-09-17T21:21:52.954 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 15234 2024-09-17T21:21:52.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.954 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:52.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:52.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:52.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:52.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:52.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:52.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093289+0000", 2024-09-17T21:21:52.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4939999999999999e-06 2024-09-17T21:21:52.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:52.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103074+0000", 2024-09-17T21:21:52.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00978564 2024-09-17T21:21:52.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:52.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103112+0000", 2024-09-17T21:21:52.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7246999999999998e-05 2024-09-17T21:21:52.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:52.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103128+0000", 2024-09-17T21:21:52.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5812999999999999e-05 2024-09-17T21:21:52.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103207+0000", 2024-09-17T21:21:52.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9785999999999998e-05 2024-09-17T21:21:52.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:52.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103315+0000", 2024-09-17T21:21:52.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107396 2024-09-17T21:21:52.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:52.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103957+0000", 2024-09-17T21:21:52.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000642524 2024-09-17T21:21:52.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:52.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.123856+0000", 2024-09-17T21:21:52.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019898376999999998 2024-09-17T21:21:52.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:52.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150190+0000", 2024-09-17T21:21:52.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026334682000000002 2024-09-17T21:21:52.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:52.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150255+0000", 2024-09-17T21:21:52.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4770999999999994e-05 2024-09-17T21:21:52.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:52.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150311+0000", 2024-09-17T21:21:52.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6322e-05 2024-09-17T21:21:52.958 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.958 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:52.958 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.959 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4239 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 424960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:52.959 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:21:52.959 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.132911120999999, 2024-09-17T21:21:52.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050123167000000003, 2024-09-17T21:21:52.959 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:52.959 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:52.959 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:52.959 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:52.959 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:52.959 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:52.959 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4239 2024-09-17T21:21:52.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.959 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:52.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:52.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:21:52.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:52.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:21:52.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:52.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753684+0000", 2024-09-17T21:21:52.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.37e-07 2024-09-17T21:21:52.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:52.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753686+0000", 2024-09-17T21:21:52.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.322e-06 2024-09-17T21:21:52.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:52.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753693+0000", 2024-09-17T21:21:52.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0890000000000002e-06 2024-09-17T21:21:52.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:52.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753699+0000", 2024-09-17T21:21:52.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:21:52.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803390+0000", 2024-09-17T21:21:52.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049690506000000002 2024-09-17T21:21:52.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:52.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803569+0000", 2024-09-17T21:21:52.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179784 2024-09-17T21:21:52.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:52.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803806+0000", 2024-09-17T21:21:52.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023683499999999999 2024-09-17T21:21:52.963 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.963 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:52.963 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.963 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4240 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 413696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:52.963 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:52.963 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.132887392999997, 2024-09-17T21:21:52.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050331353000000002, 2024-09-17T21:21:52.963 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:52.963 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:52.964 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:52.964 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:52.964 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:52.964 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:52.964 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4240 2024-09-17T21:21:52.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.964 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:52.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:52.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:52.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:52.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:52.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:52.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:52.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5700000000000002e-07 2024-09-17T21:21:52.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:52.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753710+0000", 2024-09-17T21:21:52.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.385e-06 2024-09-17T21:21:52.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:52.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753717+0000", 2024-09-17T21:21:52.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.832e-06 2024-09-17T21:21:52.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:52.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753722+0000", 2024-09-17T21:21:52.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8050000000000003e-06 2024-09-17T21:21:52.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803880+0000", 2024-09-17T21:21:52.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050157641000000003 2024-09-17T21:21:52.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803945+0000", 2024-09-17T21:21:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5357000000000005e-05 2024-09-17T21:21:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804038+0000", 2024-09-17T21:21:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2775999999999994e-05 2024-09-17T21:21:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4241 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 415744~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:52.968 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.132864277000003, 2024-09-17T21:21:52.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050578790999999998, 2024-09-17T21:21:52.968 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:52.968 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:52.968 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:52.968 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:52.968 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:52.968 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:52.968 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4241 2024-09-17T21:21:52.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.968 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:52.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:52.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:52.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:52.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:52.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:52.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:52.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5700000000000002e-07 2024-09-17T21:21:52.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:52.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753733+0000", 2024-09-17T21:21:52.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2570000000000002e-06 2024-09-17T21:21:52.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:52.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753740+0000", 2024-09-17T21:21:52.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.849e-06 2024-09-17T21:21:52.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:52.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753745+0000", 2024-09-17T21:21:52.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7450000000000001e-06 2024-09-17T21:21:52.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804115+0000", 2024-09-17T21:21:52.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050370038999999998 2024-09-17T21:21:52.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:52.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804223+0000", 2024-09-17T21:21:52.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010786 2024-09-17T21:21:52.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:52.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804309+0000", 2024-09-17T21:21:52.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5483999999999997e-05 2024-09-17T21:21:52.971 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.971 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:52.971 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.972 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4242 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 414720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:52.972 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:52.972 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.132841388999999, 2024-09-17T21:21:52.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050748148999999999, 2024-09-17T21:21:52.972 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:52.972 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:52.972 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:52.972 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:52.972 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:52.972 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:52.972 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4242 2024-09-17T21:21:52.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.973 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:52.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:52.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:52.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:52.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:52.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:52.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:52.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6599999999999996e-07 2024-09-17T21:21:52.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:52.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753756+0000", 2024-09-17T21:21:52.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.368e-06 2024-09-17T21:21:52.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:52.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753763+0000", 2024-09-17T21:21:52.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9349999999999997e-06 2024-09-17T21:21:52.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:52.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753768+0000", 2024-09-17T21:21:52.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7620000000000001e-06 2024-09-17T21:21:52.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804343+0000", 2024-09-17T21:21:52.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050575050000000003 2024-09-17T21:21:52.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:52.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804397+0000", 2024-09-17T21:21:52.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4070000000000002e-05 2024-09-17T21:21:52.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:52.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804501+0000", 2024-09-17T21:21:52.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103398 2024-09-17T21:21:52.976 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.976 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:52.976 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.976 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25642 3.1 3.6a5ad301 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e177)", 2024-09-17T21:21:52.976 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:34.316616+0000", 2024-09-17T21:21:52.976 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.569977806000001, 2024-09-17T21:21:52.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99065658300000003, 2024-09-17T21:21:52.977 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:52.977 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:52.977 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:52.977 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:52.977 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:21:52.977 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:52.977 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25642 2024-09-17T21:21:52.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.977 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:52.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:52.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316616+0000", 2024-09-17T21:21:52.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:52.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316616+0000", 2024-09-17T21:21:52.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:52.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316618+0000", 2024-09-17T21:21:52.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9309999999999998e-06 2024-09-17T21:21:52.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:52.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316624+0000", 2024-09-17T21:21:52.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2789999999999998e-06 2024-09-17T21:21:52.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:52.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316634+0000", 2024-09-17T21:21:52.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0774000000000001e-05 2024-09-17T21:21:52.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:52.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316644+0000", 2024-09-17T21:21:52.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0043e-05 2024-09-17T21:21:52.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316767+0000", 2024-09-17T21:21:52.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012289199999999999 2024-09-17T21:21:52.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:52.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316777+0000", 2024-09-17T21:21:52.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6199999999999994e-06 2024-09-17T21:21:52.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.307241+0000", 2024-09-17T21:21:52.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99046395099999995 2024-09-17T21:21:52.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:52.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.307273+0000", 2024-09-17T21:21:52.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2092999999999999e-05 2024-09-17T21:21:52.981 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.981 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:52.981 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.981 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25905 3.6 3:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-17T21:21:52.981 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:39.359572+0000", 2024-09-17T21:21:52.981 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 13.527022523999999, 2024-09-17T21:21:52.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96065702100000006, 2024-09-17T21:21:52.981 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:52.981 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:52.981 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:52.981 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:52.981 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:21:52.982 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:52.982 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25905 2024-09-17T21:21:52.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.982 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:52.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:52.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359572+0000", 2024-09-17T21:21:52.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:52.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359572+0000", 2024-09-17T21:21:52.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:52.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:52.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359573+0000", 2024-09-17T21:21:52.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5370000000000001e-06 2024-09-17T21:21:52.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:52.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359579+0000", 2024-09-17T21:21:52.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8039999999999998e-06 2024-09-17T21:21:52.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:52.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359603+0000", 2024-09-17T21:21:52.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4533999999999999e-05 2024-09-17T21:21:52.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:52.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359613+0000", 2024-09-17T21:21:52.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2820000000000002e-06 2024-09-17T21:21:52.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359686+0000", 2024-09-17T21:21:52.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3301999999999997e-05 2024-09-17T21:21:52.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:52.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359693+0000", 2024-09-17T21:21:52.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3679999999999996e-06 2024-09-17T21:21:52.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:52.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:40.319999+0000", 2024-09-17T21:21:52.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96030524500000003 2024-09-17T21:21:52.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:52.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:52.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:52.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:40.320229+0000", 2024-09-17T21:21:52.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022994899999999999 2024-09-17T21:21:52.985 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.985 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:52.985 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.985 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:52.985 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:52.986 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:52.986 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:52.987 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:52.988 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:21:52.988 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:52.989 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:21:52.989 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:52.989 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:21:52.990 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:52.985+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:53.088 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:53.085+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:53.155 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:53.155 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:53.156 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:21:53.156 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:53.170 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:21:53.217 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:53.215+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:53.290 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:53.285+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:53.292 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:53.292 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:53.292 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:21:53.292 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:21:53.292 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:53.306 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:21:53.393 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:53.391+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:53.470 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:53.470 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:53.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:53.471 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:53.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.471 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:21:53.471 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:53.471 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 193.65697122500001, 2024-09-17T21:21:53.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:21:53.471 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:53.471 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:53.471 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:53.471 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:53.471 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:21:53.471 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:21:53.471 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:21:53.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.471 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:53.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.472 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:53.472 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:53.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.472 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:53.472 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:53.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.472 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:53.472 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:21:53.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:21:53.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.473 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:53.473 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:21:53.473 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:21:53.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.473 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:53.473 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:21:53.473 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:21:53.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.473 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:53.473 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:21:53.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:21:53.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:21:53.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:21:53.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:53.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:21:53.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:21:53.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:21:53.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:21:53.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:53.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:21:53.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:21:53.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:53.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:21:53.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:21:53.475 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.476 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:53.476 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.476 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:21:53.476 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:53.476 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 180.10186943900001, 2024-09-17T21:21:53.476 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:21:53.476 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:53.476 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:53.476 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:53.476 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:53.476 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:21:53.477 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:53.477 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:21:53.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.477 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:53.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.477 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:53.477 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:53.477 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.477 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:53.477 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:53.477 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:53.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:21:53.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:21:53.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:53.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:21:53.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:21:53.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:53.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:21:53.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:21:53.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:53.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:21:53.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:21:53.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:21:53.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:21:53.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:53.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:21:53.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:21:53.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:21:53.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:21:53.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:53.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:21:53.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:21:53.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:53.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:21:53.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:21:53.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.481 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:53.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:21:53.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:21:53.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.481 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:53.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:21:53.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:21:53.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.481 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:53.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:21:53.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:21:53.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.482 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:53.482 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:21:53.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:21:53.482 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.482 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:53.482 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.482 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:793 3.9 3:95f63599:::rbd_data.1152eb799094.0000000000000007:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:53.482 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:53.482 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.034813283999995, 2024-09-17T21:21:53.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88245766400000003, 2024-09-17T21:21:53.482 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:53.482 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:53.483 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:53.483 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:53.483 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:53.483 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:53.483 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 793 2024-09-17T21:21:53.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.483 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:53.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:53.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:53.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:53.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:53.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:53.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431941+0000", 2024-09-17T21:21:53.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.864e-06 2024-09-17T21:21:53.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:53.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431953+0000", 2024-09-17T21:21:53.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1486e-05 2024-09-17T21:21:53.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:53.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431984+0000", 2024-09-17T21:21:53.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1075999999999999e-05 2024-09-17T21:21:53.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:53.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431999+0000", 2024-09-17T21:21:53.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.503e-05 2024-09-17T21:21:53.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432095+0000", 2024-09-17T21:21:53.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5698000000000004e-05 2024-09-17T21:21:53.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:53.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432109+0000", 2024-09-17T21:21:53.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4188e-05 2024-09-17T21:21:53.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.309642+0000", 2024-09-17T21:21:53.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87753353000000001 2024-09-17T21:21:53.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:53.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310104+0000", 2024-09-17T21:21:53.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046103600000000001 2024-09-17T21:21:53.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:53.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310630+0000", 2024-09-17T21:21:53.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052682800000000002 2024-09-17T21:21:53.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:53.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313100+0000", 2024-09-17T21:21:53.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024693369999999998 2024-09-17T21:21:53.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:53.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313935+0000", 2024-09-17T21:21:53.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083540100000000005 2024-09-17T21:21:53.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:53.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314086+0000", 2024-09-17T21:21:53.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015053399999999999 2024-09-17T21:21:53.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:53.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314396+0000", 2024-09-17T21:21:53.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031065599999999997 2024-09-17T21:21:53.488 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.489 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:53.489 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.489 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:800 3.19 3:9dc17f12:::rbd_data.1152eb799094.000000000000000e:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:53.489 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:53.489 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.034566038999998, 2024-09-17T21:21:53.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88154663799999999, 2024-09-17T21:21:53.489 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:53.489 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:53.489 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:53.489 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:53.489 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:53.489 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:53.490 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 800 2024-09-17T21:21:53.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.490 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:53.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:53.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:53.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:53.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:53.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:53.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432187+0000", 2024-09-17T21:21:53.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4100000000000001e-06 2024-09-17T21:21:53.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:53.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432194+0000", 2024-09-17T21:21:53.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0389999999999999e-06 2024-09-17T21:21:53.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:53.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432216+0000", 2024-09-17T21:21:53.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1614e-05 2024-09-17T21:21:53.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:53.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432232+0000", 2024-09-17T21:21:53.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5739e-05 2024-09-17T21:21:53.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432405+0000", 2024-09-17T21:21:53.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017355300000000001 2024-09-17T21:21:53.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:53.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432432+0000", 2024-09-17T21:21:53.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6979e-05 2024-09-17T21:21:53.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.307767+0000", 2024-09-17T21:21:53.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87533472899999998 2024-09-17T21:21:53.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:53.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308192+0000", 2024-09-17T21:21:53.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042485999999999999 2024-09-17T21:21:53.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:53.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308727+0000", 2024-09-17T21:21:53.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00053550400000000001 2024-09-17T21:21:53.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:53.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310347+0000", 2024-09-17T21:21:53.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016198670000000001 2024-09-17T21:21:53.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:53.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313424+0000", 2024-09-17T21:21:53.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030764640000000001 2024-09-17T21:21:53.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:53.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313510+0000", 2024-09-17T21:21:53.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5938000000000003e-05 2024-09-17T21:21:53.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.495 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:53.495 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313733+0000", 2024-09-17T21:21:53.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022294200000000001 2024-09-17T21:21:53.495 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.495 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:53.495 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.495 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:805 3.19 3:9cab2ed6:::rbd_data.1152eb799094.0000000000000013:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:53.495 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:53.495 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.034362020000003, 2024-09-17T21:21:53.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88274355699999996, 2024-09-17T21:21:53.495 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:53.495 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:53.496 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:53.496 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:53.496 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:53.496 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:53.496 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 805 2024-09-17T21:21:53.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.496 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:53.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.496 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:53.496 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:53.496 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.496 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:53.496 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:53.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.497 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:53.497 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432391+0000", 2024-09-17T21:21:53.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3e-06 2024-09-17T21:21:53.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.497 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:53.497 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432398+0000", 2024-09-17T21:21:53.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4489999999999996e-06 2024-09-17T21:21:53.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:53.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432415+0000", 2024-09-17T21:21:53.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7750000000000001e-05 2024-09-17T21:21:53.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:53.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432430+0000", 2024-09-17T21:21:53.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.449e-05 2024-09-17T21:21:53.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432635+0000", 2024-09-17T21:21:53.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205031 2024-09-17T21:21:53.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:53.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432657+0000", 2024-09-17T21:21:53.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2064000000000001e-05 2024-09-17T21:21:53.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310444+0000", 2024-09-17T21:21:53.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87778699199999999 2024-09-17T21:21:53.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:53.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310849+0000", 2024-09-17T21:21:53.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00040497800000000001 2024-09-17T21:21:53.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:53.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.311396+0000", 2024-09-17T21:21:53.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00054729099999999999 2024-09-17T21:21:53.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:53.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313847+0000", 2024-09-17T21:21:53.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024504940000000001 2024-09-17T21:21:53.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:53.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314879+0000", 2024-09-17T21:21:53.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001031817 2024-09-17T21:21:53.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:53.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314964+0000", 2024-09-17T21:21:53.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5685000000000004e-05 2024-09-17T21:21:53.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:53.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.315133+0000", 2024-09-17T21:21:53.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000169216 2024-09-17T21:21:53.501 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.501 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:53.501 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.501 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12089 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 60416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.697136987999997, 2024-09-17T21:21:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088293233999999998, 2024-09-17T21:21:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12089 2024-09-17T21:21:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769616+0000", 2024-09-17T21:21:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:21:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:53.504 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769620+0000", 2024-09-17T21:21:53.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5140000000000002e-06 2024-09-17T21:21:53.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:53.504 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769633+0000", 2024-09-17T21:21:53.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2849e-05 2024-09-17T21:21:53.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:53.504 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769644+0000", 2024-09-17T21:21:53.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1471000000000001e-05 2024-09-17T21:21:53.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857564+0000", 2024-09-17T21:21:53.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087919445999999998 2024-09-17T21:21:53.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.505 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:53.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857716+0000", 2024-09-17T21:21:53.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000152065 2024-09-17T21:21:53.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.505 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:53.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857908+0000", 2024-09-17T21:21:53.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019252599999999999 2024-09-17T21:21:53.505 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.505 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:53.505 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.506 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12090 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 64512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:53.506 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:53.506 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.697090603000007, 2024-09-17T21:21:53.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088582672000000001, 2024-09-17T21:21:53.506 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:53.506 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:53.506 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:53.506 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:53.506 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:53.506 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:53.506 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12090 2024-09-17T21:21:53.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.506 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:53.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.507 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:53.507 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:53.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.507 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:53.507 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:53.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.507 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:53.507 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769662+0000", 2024-09-17T21:21:53.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:21:53.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.508 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:53.508 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769683+0000", 2024-09-17T21:21:53.508 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0979e-05 2024-09-17T21:21:53.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.508 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:53.508 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769699+0000", 2024-09-17T21:21:53.508 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5446000000000001e-05 2024-09-17T21:21:53.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.508 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:53.508 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769708+0000", 2024-09-17T21:21:53.508 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1679999999999993e-06 2024-09-17T21:21:53.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857976+0000", 2024-09-17T21:21:53.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088267795999999996 2024-09-17T21:21:53.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:53.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858095+0000", 2024-09-17T21:21:53.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119147 2024-09-17T21:21:53.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:53.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858244+0000", 2024-09-17T21:21:53.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014917000000000001 2024-09-17T21:21:53.509 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.510 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:53.510 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.510 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12091 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 68608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:53.510 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:53.510 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.697019218999998, 2024-09-17T21:21:53.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088832752000000001, 2024-09-17T21:21:53.510 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:53.510 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:53.510 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:53.510 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:53.510 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:53.510 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:53.510 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12091 2024-09-17T21:21:53.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.511 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:53.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:53.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:53.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:53.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:53.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:53.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769734+0000", 2024-09-17T21:21:53.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:21:53.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.512 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:53.512 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769739+0000", 2024-09-17T21:21:53.512 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8080000000000004e-06 2024-09-17T21:21:53.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.512 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:53.512 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769764+0000", 2024-09-17T21:21:53.512 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4839999999999999e-05 2024-09-17T21:21:53.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.512 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:53.512 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769777+0000", 2024-09-17T21:21:53.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3037000000000001e-05 2024-09-17T21:21:53.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.513 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.513 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858295+0000", 2024-09-17T21:21:53.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088518475999999999 2024-09-17T21:21:53.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.513 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:53.513 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858419+0000", 2024-09-17T21:21:53.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012378700000000001 2024-09-17T21:21:53.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.513 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:53.514 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858565+0000", 2024-09-17T21:21:53.514 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014630099999999999 2024-09-17T21:21:53.514 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.514 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:53.514 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.514 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12092 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:53.514 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:53.514 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.696936894999993, 2024-09-17T21:21:53.514 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089191555000000006, 2024-09-17T21:21:53.514 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:53.514 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:53.515 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:53.515 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:53.515 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:53.515 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:53.515 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12092 2024-09-17T21:21:53.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.515 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:53.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.515 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:53.515 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:53.515 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.515 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:53.516 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:53.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.516 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:53.516 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769816+0000", 2024-09-17T21:21:53.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7199999999999997e-07 2024-09-17T21:21:53.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.516 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:53.516 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769822+0000", 2024-09-17T21:21:53.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:21:53.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:53.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769838+0000", 2024-09-17T21:21:53.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6674e-05 2024-09-17T21:21:53.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:53.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769851+0000", 2024-09-17T21:21:53.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2527999999999999e-05 2024-09-17T21:21:53.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858660+0000", 2024-09-17T21:21:53.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088808556999999996 2024-09-17T21:21:53.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.518 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:53.518 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858817+0000", 2024-09-17T21:21:53.518 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015730400000000001 2024-09-17T21:21:53.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.518 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:53.518 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859007+0000", 2024-09-17T21:21:53.518 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018974899999999999 2024-09-17T21:21:53.518 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.518 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:53.518 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.519 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12093 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:53.519 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:53.519 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.696828791000002, 2024-09-17T21:21:53.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089429710999999995, 2024-09-17T21:21:53.519 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:53.519 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:53.519 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:53.519 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:53.519 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:53.519 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:53.519 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12093 2024-09-17T21:21:53.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.520 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:53.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.520 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:53.520 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:53.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.520 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:53.520 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:53.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.520 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:53.520 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769924+0000", 2024-09-17T21:21:53.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.342e-06 2024-09-17T21:21:53.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:53.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769929+0000", 2024-09-17T21:21:53.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0350000000000001e-06 2024-09-17T21:21:53.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:53.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769946+0000", 2024-09-17T21:21:53.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6793999999999999e-05 2024-09-17T21:21:53.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:53.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769959+0000", 2024-09-17T21:21:53.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3062e-05 2024-09-17T21:21:53.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.522 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.522 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.522 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859075+0000", 2024-09-17T21:21:53.522 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089115696999999994 2024-09-17T21:21:53.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.522 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.522 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:53.522 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859202+0000", 2024-09-17T21:21:53.522 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012657 2024-09-17T21:21:53.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.522 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.522 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:53.522 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859353+0000", 2024-09-17T21:21:53.522 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151211 2024-09-17T21:21:53.523 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.523 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:53.523 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.523 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.523 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12094 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:53.523 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:53.523 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.696774614999995, 2024-09-17T21:21:53.523 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089695001999999996, 2024-09-17T21:21:53.523 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:53.523 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:53.523 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:53.523 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:53.523 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:53.523 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:53.523 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12094 2024-09-17T21:21:53.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.524 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:53.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:53.524 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:53.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:53.524 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:53.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:53.525 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769978+0000", 2024-09-17T21:21:53.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.142e-06 2024-09-17T21:21:53.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.525 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.525 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:53.525 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769983+0000", 2024-09-17T21:21:53.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.386e-06 2024-09-17T21:21:53.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.525 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.525 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:53.525 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769997+0000", 2024-09-17T21:21:53.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.416e-05 2024-09-17T21:21:53.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.525 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:53.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770012+0000", 2024-09-17T21:21:53.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5384999999999999e-05 2024-09-17T21:21:53.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859405+0000", 2024-09-17T21:21:53.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089392376999999995 2024-09-17T21:21:53.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:53.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859525+0000", 2024-09-17T21:21:53.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120555 2024-09-17T21:21:53.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:53.527 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859672+0000", 2024-09-17T21:21:53.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000146997 2024-09-17T21:21:53.527 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.527 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:53.527 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.527 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12095 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:53.527 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:53.527 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.696722128000005, 2024-09-17T21:21:53.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090073476, 2024-09-17T21:21:53.527 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:53.527 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:53.528 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:53.528 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:53.528 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:53.528 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:53.528 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12095 2024-09-17T21:21:53.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.528 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:53.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.528 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:53.528 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:53.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.528 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:53.529 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:53.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.529 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.529 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:53.529 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770031+0000", 2024-09-17T21:21:53.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1699999999999997e-07 2024-09-17T21:21:53.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.529 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.529 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:53.529 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770036+0000", 2024-09-17T21:21:53.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.023e-06 2024-09-17T21:21:53.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.529 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.530 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:53.530 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770051+0000", 2024-09-17T21:21:53.530 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4953999999999999e-05 2024-09-17T21:21:53.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.530 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:53.530 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770063+0000", 2024-09-17T21:21:53.530 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:21:53.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.530 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.530 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859756+0000", 2024-09-17T21:21:53.530 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089692980000000005 2024-09-17T21:21:53.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:53.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859891+0000", 2024-09-17T21:21:53.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013459499999999999 2024-09-17T21:21:53.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:53.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860103+0000", 2024-09-17T21:21:53.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021274400000000001 2024-09-17T21:21:53.531 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.531 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:53.531 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.532 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12096 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:53.532 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:53.532 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.696672442999997, 2024-09-17T21:21:53.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090358274000000002, 2024-09-17T21:21:53.532 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:53.532 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:53.532 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:53.532 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:53.532 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:53.532 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:53.532 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12096 2024-09-17T21:21:53.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.532 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:53.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:53.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:53.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:53.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:53.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:53.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770080+0000", 2024-09-17T21:21:53.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9400000000000004e-07 2024-09-17T21:21:53.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:53.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770085+0000", 2024-09-17T21:21:53.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2169999999999996e-06 2024-09-17T21:21:53.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:53.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770100+0000", 2024-09-17T21:21:53.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5077e-05 2024-09-17T21:21:53.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:53.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770112+0000", 2024-09-17T21:21:53.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1951e-05 2024-09-17T21:21:53.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.535 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.535 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860160+0000", 2024-09-17T21:21:53.535 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090048413999999993 2024-09-17T21:21:53.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.535 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:53.535 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860285+0000", 2024-09-17T21:21:53.535 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012466699999999999 2024-09-17T21:21:53.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.535 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:53.535 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860438+0000", 2024-09-17T21:21:53.535 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015305399999999999 2024-09-17T21:21:53.535 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.536 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:53.536 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.536 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12097 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:53.536 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:53.536 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.696624696000001, 2024-09-17T21:21:53.536 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090702375000000002, 2024-09-17T21:21:53.536 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:53.536 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:53.536 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:53.536 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:53.536 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:53.536 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:53.536 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12097 2024-09-17T21:21:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770128+0000", 2024-09-17T21:21:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.04e-06 2024-09-17T21:21:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770133+0000", 2024-09-17T21:21:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8999999999999997e-06 2024-09-17T21:21:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770152+0000", 2024-09-17T21:21:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8890999999999999e-05 2024-09-17T21:21:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770164+0000", 2024-09-17T21:21:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1769e-05 2024-09-17T21:21:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860511+0000", 2024-09-17T21:21:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090347527999999996 2024-09-17T21:21:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860652+0000", 2024-09-17T21:21:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140894 2024-09-17T21:21:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860830+0000", 2024-09-17T21:21:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017735299999999999 2024-09-17T21:21:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12098 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.696571227999996, 2024-09-17T21:21:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090997262999999995, 2024-09-17T21:21:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12098 2024-09-17T21:21:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775465+0000", 2024-09-17T21:21:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0052843350000000002 2024-09-17T21:21:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775476+0000", 2024-09-17T21:21:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0797e-05 2024-09-17T21:21:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775506+0000", 2024-09-17T21:21:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9709999999999998e-05 2024-09-17T21:21:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775530+0000", 2024-09-17T21:21:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4528999999999998e-05 2024-09-17T21:21:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860894+0000", 2024-09-17T21:21:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.085364195000000004 2024-09-17T21:21:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861019+0000", 2024-09-17T21:21:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012467399999999999 2024-09-17T21:21:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861178+0000", 2024-09-17T21:21:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159023 2024-09-17T21:21:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12104 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.690870118999996, 2024-09-17T21:21:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087969628999999994, 2024-09-17T21:21:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12104 2024-09-17T21:21:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775883+0000", 2024-09-17T21:21:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3399999999999998e-07 2024-09-17T21:21:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775886+0000", 2024-09-17T21:21:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2119999999999999e-06 2024-09-17T21:21:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775895+0000", 2024-09-17T21:21:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8710000000000003e-06 2024-09-17T21:21:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775901+0000", 2024-09-17T21:21:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:21:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863544+0000", 2024-09-17T21:21:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087642642000000007 2024-09-17T21:21:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863694+0000", 2024-09-17T21:21:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014952300000000001 2024-09-17T21:21:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863851+0000", 2024-09-17T21:21:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000157804 2024-09-17T21:21:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12105 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.690841225, 2024-09-17T21:21:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088330400000000003, 2024-09-17T21:21:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12105 2024-09-17T21:21:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9999999999999997e-07 2024-09-17T21:21:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775914+0000", 2024-09-17T21:21:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4710000000000001e-06 2024-09-17T21:21:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775921+0000", 2024-09-17T21:21:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:21:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775928+0000", 2024-09-17T21:21:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:21:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863923+0000", 2024-09-17T21:21:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087995126000000007 2024-09-17T21:21:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864059+0000", 2024-09-17T21:21:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135797 2024-09-17T21:21:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864241+0000", 2024-09-17T21:21:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018252599999999999 2024-09-17T21:21:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12106 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.690752047000004, 2024-09-17T21:21:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088586394999999998, 2024-09-17T21:21:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12106 2024-09-17T21:21:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776001+0000", 2024-09-17T21:21:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4300000000000001e-07 2024-09-17T21:21:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776004+0000", 2024-09-17T21:21:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8049999999999998e-06 2024-09-17T21:21:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776018+0000", 2024-09-17T21:21:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4613999999999999e-05 2024-09-17T21:21:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:53.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776025+0000", 2024-09-17T21:21:53.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:21:53.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864279+0000", 2024-09-17T21:21:53.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088253570000000003 2024-09-17T21:21:53.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:53.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864433+0000", 2024-09-17T21:21:53.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015469400000000001 2024-09-17T21:21:53.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864586+0000", 2024-09-17T21:21:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000153003 2024-09-17T21:21:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25033 3.6 3.b74753e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e168)", 2024-09-17T21:21:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:21:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.957308782999998, 2024-09-17T21:21:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93781952099999999, 2024-09-17T21:21:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:21:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25033 2024-09-17T21:21:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:21:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:21:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509444+0000", 2024-09-17T21:21:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0020000000000001e-06 2024-09-17T21:21:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509450+0000", 2024-09-17T21:21:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4249999999999997e-06 2024-09-17T21:21:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509460+0000", 2024-09-17T21:21:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0183999999999999e-05 2024-09-17T21:21:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509466+0000", 2024-09-17T21:21:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0039999999999996e-06 2024-09-17T21:21:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509540+0000", 2024-09-17T21:21:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3847e-05 2024-09-17T21:21:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509551+0000", 2024-09-17T21:21:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1731e-05 2024-09-17T21:21:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.447243+0000", 2024-09-17T21:21:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93769166000000004 2024-09-17T21:21:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.447263+0000", 2024-09-17T21:21:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9667999999999999e-05 2024-09-17T21:21:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25033 3.6 3:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e168)", 2024-09-17T21:21:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:21:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 27.955783186000001, 2024-09-17T21:21:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.937309383, 2024-09-17T21:21:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:21:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25033 2024-09-17T21:21:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:21:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:21:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510970+0000", 2024-09-17T21:21:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.564e-06 2024-09-17T21:21:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510978+0000", 2024-09-17T21:21:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.362e-06 2024-09-17T21:21:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510998+0000", 2024-09-17T21:21:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0336999999999999e-05 2024-09-17T21:21:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511010+0000", 2024-09-17T21:21:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1786e-05 2024-09-17T21:21:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511080+0000", 2024-09-17T21:21:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0686999999999996e-05 2024-09-17T21:21:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511087+0000", 2024-09-17T21:21:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4899999999999997e-06 2024-09-17T21:21:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.448050+0000", 2024-09-17T21:21:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93696351700000002 2024-09-17T21:21:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.448278+0000", 2024-09-17T21:21:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022764000000000001 2024-09-17T21:21:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:53.566 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:53.567 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:21:53.568 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:21:53.569 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:53.565+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:53.668 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:53.665+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:53.705 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:53.705 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:53.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:53.705 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:27341 3.c 3:34d69eb9:::rbd_header.115262b8e217:head [call rbd.get_access_timestamp in=23b,call rbd.get_modify_timestamp in=23b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e196)", 2024-09-17T21:21:53.705 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:53.703567+0000", 2024-09-17T21:21:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.00040053300000000001, 2024-09-17T21:21:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00041958000000000002, 2024-09-17T21:21:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 27341 2024-09-17T21:21:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:53.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:53.703567+0000", 2024-09-17T21:21:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:53.703567+0000", 2024-09-17T21:21:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:53.703569+0000", 2024-09-17T21:21:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3939999999999999e-06 2024-09-17T21:21:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:53.703577+0000", 2024-09-17T21:21:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7929999999999998e-06 2024-09-17T21:21:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:53.703595+0000", 2024-09-17T21:21:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8946000000000001e-05 2024-09-17T21:21:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:53.703612+0000", 2024-09-17T21:21:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.645e-05 2024-09-17T21:21:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:53.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:53.703755+0000", 2024-09-17T21:21:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142779 2024-09-17T21:21:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:53.703887+0000", 2024-09-17T21:21:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013278399999999999 2024-09-17T21:21:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-09-17T21:21:53.709 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 1 2024-09-17T21:21:53.709 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:53.721 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:21:53.769 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:53.767+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:53.870 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:53.867+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:53.891 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:53.891 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:53.891 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:53.891 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:53.891 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:53.905 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:21:53.971 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:53.969+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:54.028 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:54.028 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:54.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:54.028 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:54.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.028 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:21:54.029 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:54.029 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 237.01148151699999, 2024-09-17T21:21:54.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:21:54.029 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.029 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.029 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:54.029 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.029 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:21:54.029 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:21:54.029 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:21:54.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.029 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:54.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:54.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:21:54.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:21:54.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:21:54.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:21:54.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:21:54.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:21:54.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:21:54.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:21:54.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:21:54.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:21:54.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:54.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:21:54.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:21:54.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:21:54.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:21:54.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:21:54.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:21:54.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:21:54.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:21:54.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:21:54.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:54.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:21:54.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:21:54.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:54.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:21:54.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:21:54.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:54.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:21:54.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:21:54.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:21:54.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:21:54.035 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.035 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.035 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.035 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:54.035 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:54.035 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 217.431317237, 2024-09-17T21:21:54.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:21:54.035 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.036 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.036 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.036 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.036 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:21:54.036 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:21:54.036 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:21:54.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.036 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:54.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:54.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:21:54.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:21:54.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:21:54.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:21:54.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:21:54.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:21:54.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:21:54.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:21:54.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:21:54.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:21:54.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:54.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:21:54.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:21:54.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:21:54.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:21:54.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:54.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:21:54.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:21:54.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:21:54.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:21:54.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:21:54.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:21:54.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:21:54.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:21:54.040 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.041 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.041 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.041 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:54.041 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:54.041 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 217.43121992499999, 2024-09-17T21:21:54.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:21:54.041 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.041 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.041 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.041 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.041 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:54.041 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:54.042 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:21:54.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.042 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:54.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:54.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:21:54.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:21:54.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:21:54.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:21:54.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:21:54.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:21:54.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:21:54.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:21:54.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:21:54.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:21:54.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:54.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:21:54.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:21:54.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:21:54.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:21:54.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:54.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:21:54.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:21:54.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:21:54.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:21:54.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:21:54.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:21:54.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:21:54.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:21:54.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:21:54.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:21:54.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:21:54.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:21:54.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:21:54.047 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.047 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.047 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.047 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:54.047 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:54.047 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 217.429133124, 2024-09-17T21:21:54.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:21:54.047 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.047 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.048 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:54.048 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.048 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:21:54.048 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:21:54.048 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:21:54.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.048 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:54.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:54.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:21:54.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:21:54.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:21:54.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:21:54.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:21:54.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:21:54.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:21:54.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:21:54.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:21:54.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:21:54.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:21:54.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:21:54.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:21:54.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:21:54.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:21:54.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:54.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:21:54.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:21:54.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:21:54.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:21:54.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:21:54.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:21:54.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:54.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:21:54.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:21:54.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:54.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:21:54.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:21:54.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:54.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:21:54.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:21:54.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:54.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:21:54.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:21:54.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:21:54.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:21:54.054 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.054 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.054 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.054 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:54.054 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:54.054 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 184.74523434299999, 2024-09-17T21:21:54.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:21:54.054 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.055 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.055 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.055 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.055 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:54.055 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:54.055 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:21:54.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.055 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:54.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:54.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:21:54.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:21:54.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:21:54.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:21:54.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:21:54.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:21:54.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:21:54.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:21:54.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:21:54.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:21:54.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:54.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:21:54.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:21:54.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:21:54.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:21:54.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:54.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:21:54.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:21:54.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:21:54.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:21:54.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:21:54.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:21:54.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:21:54.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:21:54.060 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.060 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.060 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.060 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:54.060 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:54.060 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 184.736142632, 2024-09-17T21:21:54.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:21:54.060 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.060 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.061 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.061 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.061 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:54.061 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:54.061 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:21:54.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.061 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:54.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:54.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:21:54.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:21:54.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:21:54.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:21:54.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:21:54.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:21:54.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:21:54.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:21:54.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:21:54.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:21:54.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:54.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:21:54.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:21:54.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:21:54.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:21:54.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:21:54.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:21:54.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:21:54.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:21:54.065 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.065 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.065 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.065 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:54.065 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:54.065 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 184.735970581, 2024-09-17T21:21:54.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:21:54.065 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.066 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.066 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.066 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.066 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:54.066 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:54.066 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:21:54.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.066 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:54.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:54.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:21:54.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:21:54.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:21:54.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:21:54.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:21:54.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:21:54.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:21:54.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:21:54.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:21:54.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:21:54.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:54.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:21:54.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:21:54.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:21:54.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:21:54.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:21:54.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:21:54.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:21:54.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:21:54.070 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.070 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.070 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.070 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:54.070 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:54.070 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 184.56103783399999, 2024-09-17T21:21:54.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:21:54.070 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.070 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.071 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:54.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:54.071 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:21:54.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.071 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:54.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:54.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:21:54.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:21:54.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:21:54.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:21:54.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:21:54.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:21:54.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:21:54.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:21:54.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:21:54.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:21:54.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:54.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:21:54.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:21:54.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:21:54.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:21:54.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:21:54.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:21:54.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:21:54.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:21:54.075 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.075 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.075 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.075 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:54.075 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:54.075 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 184.56095327700001, 2024-09-17T21:21:54.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:21:54.075 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.075 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.075 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.075 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.076 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:54.076 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:54.076 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:21:54.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.076 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:54.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:54.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:54.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:21:54.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:21:54.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:21:54.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:21:54.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:21:54.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:21:54.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:21:54.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:21:54.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:21:54.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:54.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:21:54.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:21:54.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:21:54.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:21:54.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:21:54.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:21:54.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:21:54.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:21:54.079 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.079 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.080 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.080 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:54.080 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:54.080 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 184.56088764, 2024-09-17T21:21:54.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:21:54.080 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.080 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.080 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.080 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.080 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:54.080 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:54.080 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:21:54.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.081 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:54.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:54.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:21:54.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:21:54.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:21:54.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:21:54.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:21:54.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:21:54.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:21:54.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:21:54.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:21:54.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:21:54.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:54.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:21:54.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:21:54.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:21:54.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:21:54.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:21:54.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:21:54.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:21:54.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:21:54.085 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.085 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.085 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.085 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:54.085 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:54.085 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 133.18731254299999, 2024-09-17T21:21:54.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:21:54.085 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.085 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.085 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.085 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.086 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:21:54.086 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:54.086 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:21:54.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.086 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:54.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:54.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:21:54.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:21:54.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:21:54.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:21:54.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:21:54.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:21:54.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:21:54.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:21:54.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:21:54.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:21:54.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:21:54.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:21:54.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:21:54.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:21:54.089 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.089 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.089 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.089 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:54.089 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:54.090 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 133.18719303099999, 2024-09-17T21:21:54.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:21:54.090 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.090 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.090 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.090 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.090 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:21:54.090 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:54.090 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:21:54.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.090 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:54.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:54.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:21:54.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:21:54.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:21:54.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:21:54.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:21:54.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:21:54.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:21:54.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:21:54.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:21:54.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:21:54.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:21:54.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:21:54.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:21:54.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:21:54.093 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.093 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.093 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.094 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:996 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:54.094 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:54.094 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 88.594079660000006, 2024-09-17T21:21:54.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084493736999999999, 2024-09-17T21:21:54.094 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.094 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.094 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.094 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.094 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:54.094 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:54.094 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 996 2024-09-17T21:21:54.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.095 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:54.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:54.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431168+0000", 2024-09-17T21:21:54.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4e-07 2024-09-17T21:21:54.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431172+0000", 2024-09-17T21:21:54.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.608e-06 2024-09-17T21:21:54.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431180+0000", 2024-09-17T21:21:54.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5650000000000002e-06 2024-09-17T21:21:54.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431188+0000", 2024-09-17T21:21:54.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:21:54.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515354+0000", 2024-09-17T21:21:54.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084165977000000003 2024-09-17T21:21:54.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515451+0000", 2024-09-17T21:21:54.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6669000000000006e-05 2024-09-17T21:21:54.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515661+0000", 2024-09-17T21:21:54.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021021599999999999 2024-09-17T21:21:54.098 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.098 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.098 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.098 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:997 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 583680~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:54.098 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:54.098 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 88.594048264999998, 2024-09-17T21:21:54.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084772419000000002, 2024-09-17T21:21:54.098 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.098 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.099 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.099 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.099 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:54.099 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:54.099 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 997 2024-09-17T21:21:54.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.099 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:54.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:54.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:54.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0400000000000005e-07 2024-09-17T21:21:54.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431202+0000", 2024-09-17T21:21:54.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:21:54.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431208+0000", 2024-09-17T21:21:54.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4740000000000002e-06 2024-09-17T21:21:54.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431214+0000", 2024-09-17T21:21:54.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6550000000000001e-06 2024-09-17T21:21:54.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515741+0000", 2024-09-17T21:21:54.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084526904 2024-09-17T21:21:54.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515826+0000", 2024-09-17T21:21:54.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4643999999999998e-05 2024-09-17T21:21:54.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515971+0000", 2024-09-17T21:21:54.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145444 2024-09-17T21:21:54.102 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.102 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.102 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.102 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:998 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 587776~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:54.103 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:54.103 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 88.594025293000001, 2024-09-17T21:21:54.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085108565999999997, 2024-09-17T21:21:54.103 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.103 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.103 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.103 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.103 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:54.103 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:54.103 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 998 2024-09-17T21:21:54.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.103 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:21:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431224+0000", 2024-09-17T21:21:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:21:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431231+0000", 2024-09-17T21:21:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000002e-06 2024-09-17T21:21:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431236+0000", 2024-09-17T21:21:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.648e-06 2024-09-17T21:21:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516024+0000", 2024-09-17T21:21:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084787584999999999 2024-09-17T21:21:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516122+0000", 2024-09-17T21:21:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8076000000000001e-05 2024-09-17T21:21:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516330+0000", 2024-09-17T21:21:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020808 2024-09-17T21:21:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1000 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 591872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 88.594003177999994, 2024-09-17T21:21:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085389065, 2024-09-17T21:21:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.107 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1000 2024-09-17T21:21:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:21:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431246+0000", 2024-09-17T21:21:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1749999999999999e-06 2024-09-17T21:21:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431253+0000", 2024-09-17T21:21:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2790000000000004e-06 2024-09-17T21:21:54.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431258+0000", 2024-09-17T21:21:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3299999999999998e-06 2024-09-17T21:21:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516384+0000", 2024-09-17T21:21:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085125549999999994 2024-09-17T21:21:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516460+0000", 2024-09-17T21:21:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6359999999999994e-05 2024-09-17T21:21:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516633+0000", 2024-09-17T21:21:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017285900000000001 2024-09-17T21:21:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1001 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 595968~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 88.593979771999997, 2024-09-17T21:21:54.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085614715999999993, 2024-09-17T21:21:54.112 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.112 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.112 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.112 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.112 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:54.112 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:54.112 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1001 2024-09-17T21:21:54.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.112 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:54.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438066+0000", 2024-09-17T21:21:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067989490000000003 2024-09-17T21:21:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438073+0000", 2024-09-17T21:21:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8419999999999999e-06 2024-09-17T21:21:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438090+0000", 2024-09-17T21:21:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6657e-05 2024-09-17T21:21:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438099+0000", 2024-09-17T21:21:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0529999999999996e-06 2024-09-17T21:21:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516678+0000", 2024-09-17T21:21:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078579422999999995 2024-09-17T21:21:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516739+0000", 2024-09-17T21:21:54.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1315000000000005e-05 2024-09-17T21:21:54.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516882+0000", 2024-09-17T21:21:54.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014247700000000001 2024-09-17T21:21:54.115 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.115 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.115 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.115 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:19998 3.5 3:a4885450:::rbd_object_map.1152d139f9f9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-17T21:21:54.115 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:54.116 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.622712527000004, 2024-09-17T21:21:54.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90715489299999996, 2024-09-17T21:21:54.116 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.116 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.116 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:54.116 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.116 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:54.116 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:54.116 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19998 2024-09-17T21:21:54.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.116 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:54.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:54.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402537+0000", 2024-09-17T21:21:54.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5600000000000001e-06 2024-09-17T21:21:54.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402549+0000", 2024-09-17T21:21:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1929999999999999e-05 2024-09-17T21:21:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402579+0000", 2024-09-17T21:21:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0093000000000001e-05 2024-09-17T21:21:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402595+0000", 2024-09-17T21:21:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5855999999999999e-05 2024-09-17T21:21:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402731+0000", 2024-09-17T21:21:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013610799999999999 2024-09-17T21:21:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402750+0000", 2024-09-17T21:21:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8737e-05 2024-09-17T21:21:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.305977+0000", 2024-09-17T21:21:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90322768399999998 2024-09-17T21:21:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306236+0000", 2024-09-17T21:21:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000258111 2024-09-17T21:21:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306557+0000", 2024-09-17T21:21:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032159799999999998 2024-09-17T21:21:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.308286+0000", 2024-09-17T21:21:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017292150000000001 2024-09-17T21:21:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309530+0000", 2024-09-17T21:21:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00124409 2024-09-17T21:21:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309630+0000", 2024-09-17T21:21:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9689e-05 2024-09-17T21:21:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309689+0000", 2024-09-17T21:21:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9222000000000003e-05 2024-09-17T21:21:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:25277 3.1f 3:ff5d7cd5:::rbd_header.11524f65d4ff:head [watch ping cookie 139860845478480 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-17T21:21:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:21:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.439756458000002, 2024-09-17T21:21:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.92185720000000004, 2024-09-17T21:21:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.123 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.123 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.123 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.123 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:54.123 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:54.123 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 25277 2024-09-17T21:21:54.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.123 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:21:54.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:21:54.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585493+0000", 2024-09-17T21:21:54.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:21:54.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585505+0000", 2024-09-17T21:21:54.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2466000000000001e-05 2024-09-17T21:21:54.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585537+0000", 2024-09-17T21:21:54.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1183000000000003e-05 2024-09-17T21:21:54.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585553+0000", 2024-09-17T21:21:54.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.662e-05 2024-09-17T21:21:54.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585696+0000", 2024-09-17T21:21:54.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014293800000000001 2024-09-17T21:21:54.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:54.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585718+0000", 2024-09-17T21:21:54.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1860000000000001e-05 2024-09-17T21:21:54.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.501188+0000", 2024-09-17T21:21:54.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.91547051300000004 2024-09-17T21:21:54.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:21:54.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.501260+0000", 2024-09-17T21:21:54.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1571999999999998e-05 2024-09-17T21:21:54.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507002+0000", 2024-09-17T21:21:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057414850000000002 2024-09-17T21:21:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507143+0000", 2024-09-17T21:21:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014140299999999999 2024-09-17T21:21:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507348+0000", 2024-09-17T21:21:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000204773 2024-09-17T21:21:54.127 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:26042 3.f 3:f24134c1:::rbd_id.image_client.0-clone34:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e186)", 2024-09-17T21:21:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:21:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.539068090000001, 2024-09-17T21:21:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.91185481000000002, 2024-09-17T21:21:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:54.128 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 26042 2024-09-17T21:21:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:21:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:21:54.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486180+0000", 2024-09-17T21:21:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5990000000000001e-06 2024-09-17T21:21:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486187+0000", 2024-09-17T21:21:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7859999999999999e-06 2024-09-17T21:21:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486206+0000", 2024-09-17T21:21:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8641e-05 2024-09-17T21:21:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486219+0000", 2024-09-17T21:21:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.308e-05 2024-09-17T21:21:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486283+0000", 2024-09-17T21:21:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4084000000000005e-05 2024-09-17T21:21:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486290+0000", 2024-09-17T21:21:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.584e-06 2024-09-17T21:21:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392182+0000", 2024-09-17T21:21:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90589206499999997 2024-09-17T21:21:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392518+0000", 2024-09-17T21:21:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033596 2024-09-17T21:21:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:21:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392802+0000", 2024-09-17T21:21:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028394999999999999 2024-09-17T21:21:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397705+0000", 2024-09-17T21:21:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0049031769999999999 2024-09-17T21:21:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397770+0000", 2024-09-17T21:21:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4765999999999996e-05 2024-09-17T21:21:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397927+0000", 2024-09-17T21:21:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015699900000000001 2024-09-17T21:21:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.398034+0000", 2024-09-17T21:21:54.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000107119 2024-09-17T21:21:54.137 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.137 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.137 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.137 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.137 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.137 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:54.139 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:21:54.139 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:54.135+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:54.239 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:54.237+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:54.309 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:54.309 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:54.309 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:54.309 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:54.323 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:21:54.339 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:54.335+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:54.440 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:54.437+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-17T21:21:54.504 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:54.504 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:54.504 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:54.504 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:54.504 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:54.520 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:21:54.563 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:54.561+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:54.641 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:54.637+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:54.689 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:54.689 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:54.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:54.689 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:54.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.689 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:18003 3.1d 3:b9765580:::rbd_header.11527d2b15f6:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T21:21:54.689 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:54.689 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 102.326372775, 2024-09-17T21:21:54.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94165346900000002, 2024-09-17T21:21:54.690 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.690 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.690 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.690 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.690 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:54.690 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:54.690 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18003 2024-09-17T21:21:54.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.690 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:54.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:21:54.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357880+0000", 2024-09-17T21:21:54.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:21:54.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357891+0000", 2024-09-17T21:21:54.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0417999999999999e-05 2024-09-17T21:21:54.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357913+0000", 2024-09-17T21:21:54.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2215e-05 2024-09-17T21:21:54.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357922+0000", 2024-09-17T21:21:54.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4779999999999997e-06 2024-09-17T21:21:54.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358023+0000", 2024-09-17T21:21:54.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101025 2024-09-17T21:21:54.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:54.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358040+0000", 2024-09-17T21:21:54.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6291999999999999e-05 2024-09-17T21:21:54.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298203+0000", 2024-09-17T21:21:54.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94016297500000001 2024-09-17T21:21:54.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298626+0000", 2024-09-17T21:21:54.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000422853 2024-09-17T21:21:54.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.299532+0000", 2024-09-17T21:21:54.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00090657300000000004 2024-09-17T21:21:54.694 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.694 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.694 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.694 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15707 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 982016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:54.694 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:54.695 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 92.163207528000001, 2024-09-17T21:21:54.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092305228000000003, 2024-09-17T21:21:54.695 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.695 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.695 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.695 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.695 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:54.695 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:54.695 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15707 2024-09-17T21:21:54.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.695 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:21:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521045+0000", 2024-09-17T21:21:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4899999999999999e-06 2024-09-17T21:21:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521066+0000", 2024-09-17T21:21:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0236000000000001e-05 2024-09-17T21:21:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521085+0000", 2024-09-17T21:21:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8819999999999999e-05 2024-09-17T21:21:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521100+0000", 2024-09-17T21:21:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5146e-05 2024-09-17T21:21:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612978+0000", 2024-09-17T21:21:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091878847 2024-09-17T21:21:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613079+0000", 2024-09-17T21:21:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100503 2024-09-17T21:21:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613349+0000", 2024-09-17T21:21:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000270186 2024-09-17T21:21:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.698 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15708 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 986112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 92.163129353000002, 2024-09-17T21:21:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092547217000000001, 2024-09-17T21:21:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:54.699 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15708 2024-09-17T21:21:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:21:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521123+0000", 2024-09-17T21:21:54.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.353e-06 2024-09-17T21:21:54.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521129+0000", 2024-09-17T21:21:54.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:21:54.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521145+0000", 2024-09-17T21:21:54.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.611e-05 2024-09-17T21:21:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521154+0000", 2024-09-17T21:21:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4500000000000004e-06 2024-09-17T21:21:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613397+0000", 2024-09-17T21:21:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092243275 2024-09-17T21:21:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613503+0000", 2024-09-17T21:21:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106013 2024-09-17T21:21:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613669+0000", 2024-09-17T21:21:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016642299999999999 2024-09-17T21:21:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15709 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 990208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 92.163086336999996, 2024-09-17T21:21:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092728678999999994, 2024-09-17T21:21:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15709 2024-09-17T21:21:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:21:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521166+0000", 2024-09-17T21:21:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7000000000000004e-07 2024-09-17T21:21:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521169+0000", 2024-09-17T21:21:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2660000000000002e-06 2024-09-17T21:21:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521181+0000", 2024-09-17T21:21:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.208e-05 2024-09-17T21:21:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521194+0000", 2024-09-17T21:21:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2707e-05 2024-09-17T21:21:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613695+0000", 2024-09-17T21:21:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092500709 2024-09-17T21:21:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613750+0000", 2024-09-17T21:21:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.503e-05 2024-09-17T21:21:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613894+0000", 2024-09-17T21:21:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144117 2024-09-17T21:21:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15710 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 994304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 92.163041770999996, 2024-09-17T21:21:54.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092957729000000003, 2024-09-17T21:21:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15710 2024-09-17T21:21:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:54.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:21:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8000000000000005e-07 2024-09-17T21:21:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521216+0000", 2024-09-17T21:21:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3859999999999998e-06 2024-09-17T21:21:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521234+0000", 2024-09-17T21:21:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:21:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521248+0000", 2024-09-17T21:21:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:21:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613926+0000", 2024-09-17T21:21:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092678324000000006 2024-09-17T21:21:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613974+0000", 2024-09-17T21:21:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7754999999999999e-05 2024-09-17T21:21:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614167+0000", 2024-09-17T21:21:54.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193508 2024-09-17T21:21:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15711 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 998400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 92.162984389000002, 2024-09-17T21:21:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093268000000000004, 2024-09-17T21:21:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.713 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15711 2024-09-17T21:21:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:21:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521268+0000", 2024-09-17T21:21:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-17T21:21:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521273+0000", 2024-09-17T21:21:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:21:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521290+0000", 2024-09-17T21:21:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7269999999999999e-05 2024-09-17T21:21:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521304+0000", 2024-09-17T21:21:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3626e-05 2024-09-17T21:21:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614220+0000", 2024-09-17T21:21:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092916367 2024-09-17T21:21:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614322+0000", 2024-09-17T21:21:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102212 2024-09-17T21:21:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614535+0000", 2024-09-17T21:21:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021266199999999999 2024-09-17T21:21:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15712 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1002496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 92.162928894000004, 2024-09-17T21:21:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093568707000000001, 2024-09-17T21:21:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.717 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15712 2024-09-17T21:21:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:21:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521324+0000", 2024-09-17T21:21:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.17e-06 2024-09-17T21:21:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521334+0000", 2024-09-17T21:21:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9369999999999999e-06 2024-09-17T21:21:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521356+0000", 2024-09-17T21:21:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2045999999999999e-05 2024-09-17T21:21:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521370+0000", 2024-09-17T21:21:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4270000000000001e-05 2024-09-17T21:21:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614583+0000", 2024-09-17T21:21:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093213220999999999 2024-09-17T21:21:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614680+0000", 2024-09-17T21:21:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6900999999999999e-05 2024-09-17T21:21:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614891+0000", 2024-09-17T21:21:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021116200000000001 2024-09-17T21:21:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15713 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1006592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:54.721 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 92.162862192000006, 2024-09-17T21:21:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093834641999999996, 2024-09-17T21:21:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15713 2024-09-17T21:21:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:21:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521390+0000", 2024-09-17T21:21:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:21:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521396+0000", 2024-09-17T21:21:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:21:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521417+0000", 2024-09-17T21:21:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1236000000000001e-05 2024-09-17T21:21:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521431+0000", 2024-09-17T21:21:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3873000000000001e-05 2024-09-17T21:21:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614938+0000", 2024-09-17T21:21:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093506522999999994 2024-09-17T21:21:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615025+0000", 2024-09-17T21:21:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7635999999999999e-05 2024-09-17T21:21:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615224+0000", 2024-09-17T21:21:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019844700000000001 2024-09-17T21:21:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.725 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15714 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 92.162798936000002, 2024-09-17T21:21:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094071925000000001, 2024-09-17T21:21:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15714 2024-09-17T21:21:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:21:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521454+0000", 2024-09-17T21:21:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.097e-06 2024-09-17T21:21:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521459+0000", 2024-09-17T21:21:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2660000000000001e-06 2024-09-17T21:21:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521481+0000", 2024-09-17T21:21:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1878999999999998e-05 2024-09-17T21:21:54.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521498+0000", 2024-09-17T21:21:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6807e-05 2024-09-17T21:21:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615271+0000", 2024-09-17T21:21:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093773286999999997 2024-09-17T21:21:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615366+0000", 2024-09-17T21:21:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5322000000000007e-05 2024-09-17T21:21:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615524+0000", 2024-09-17T21:21:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015826699999999999 2024-09-17T21:21:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15715 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1014784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 92.162734600999997, 2024-09-17T21:21:54.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094243840999999995, 2024-09-17T21:21:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15715 2024-09-17T21:21:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:21:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521518+0000", 2024-09-17T21:21:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4300000000000001e-06 2024-09-17T21:21:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521524+0000", 2024-09-17T21:21:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5860000000000004e-06 2024-09-17T21:21:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521541+0000", 2024-09-17T21:21:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7574000000000001e-05 2024-09-17T21:21:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521555+0000", 2024-09-17T21:21:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3540000000000001e-05 2024-09-17T21:21:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615560+0000", 2024-09-17T21:21:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.09400472 2024-09-17T21:21:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615614+0000", 2024-09-17T21:21:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4693e-05 2024-09-17T21:21:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615761+0000", 2024-09-17T21:21:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000146298 2024-09-17T21:21:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.734 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15716 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1018880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 92.162678256000007, 2024-09-17T21:21:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094400676000000003, 2024-09-17T21:21:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:54.735 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15716 2024-09-17T21:21:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:21:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521574+0000", 2024-09-17T21:21:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1939999999999999e-06 2024-09-17T21:21:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521579+0000", 2024-09-17T21:21:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:21:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521597+0000", 2024-09-17T21:21:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8074e-05 2024-09-17T21:21:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521613+0000", 2024-09-17T21:21:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5316e-05 2024-09-17T21:21:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615780+0000", 2024-09-17T21:21:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094166874999999997 2024-09-17T21:21:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615836+0000", 2024-09-17T21:21:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6926999999999999e-05 2024-09-17T21:21:54.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615974+0000", 2024-09-17T21:21:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013741400000000001 2024-09-17T21:21:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15717 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1022976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 92.162619559999996, 2024-09-17T21:21:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094586723999999997, 2024-09-17T21:21:54.739 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15717 2024-09-17T21:21:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:21:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521633+0000", 2024-09-17T21:21:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:21:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521638+0000", 2024-09-17T21:21:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:21:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521659+0000", 2024-09-17T21:21:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-05 2024-09-17T21:21:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521675+0000", 2024-09-17T21:21:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5688999999999998e-05 2024-09-17T21:21:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616008+0000", 2024-09-17T21:21:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094333314000000001 2024-09-17T21:21:54.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616066+0000", 2024-09-17T21:21:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7797000000000002e-05 2024-09-17T21:21:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616219+0000", 2024-09-17T21:21:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015281100000000001 2024-09-17T21:21:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15718 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 92.162519485000004, 2024-09-17T21:21:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094703674000000002, 2024-09-17T21:21:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15718 2024-09-17T21:21:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:21:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521734+0000", 2024-09-17T21:21:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.61e-06 2024-09-17T21:21:54.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521740+0000", 2024-09-17T21:21:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:21:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521754+0000", 2024-09-17T21:21:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3793e-05 2024-09-17T21:21:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521763+0000", 2024-09-17T21:21:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5129999999999996e-06 2024-09-17T21:21:54.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616256+0000", 2024-09-17T21:21:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094492919999999994 2024-09-17T21:21:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616312+0000", 2024-09-17T21:21:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6737000000000002e-05 2024-09-17T21:21:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616436+0000", 2024-09-17T21:21:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012335800000000001 2024-09-17T21:21:54.747 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15719 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1031168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 92.162466080000002, 2024-09-17T21:21:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094888963000000007, 2024-09-17T21:21:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:54.748 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15719 2024-09-17T21:21:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:21:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521786+0000", 2024-09-17T21:21:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.074e-06 2024-09-17T21:21:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521791+0000", 2024-09-17T21:21:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3030000000000001e-06 2024-09-17T21:21:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521804+0000", 2024-09-17T21:21:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3027000000000001e-05 2024-09-17T21:21:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521811+0000", 2024-09-17T21:21:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6090000000000003e-06 2024-09-17T21:21:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616466+0000", 2024-09-17T21:21:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094654294 2024-09-17T21:21:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616523+0000", 2024-09-17T21:21:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6953999999999997e-05 2024-09-17T21:21:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616674+0000", 2024-09-17T21:21:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015170200000000001 2024-09-17T21:21:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15720 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1035264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 92.162429239999994, 2024-09-17T21:21:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095067061999999994, 2024-09-17T21:21:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15720 2024-09-17T21:21:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:21:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521823+0000", 2024-09-17T21:21:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:21:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521826+0000", 2024-09-17T21:21:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2100000000000002e-06 2024-09-17T21:21:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521835+0000", 2024-09-17T21:21:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1630000000000002e-06 2024-09-17T21:21:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521842+0000", 2024-09-17T21:21:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:21:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616711+0000", 2024-09-17T21:21:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094868859 2024-09-17T21:21:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616764+0000", 2024-09-17T21:21:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2778999999999997e-05 2024-09-17T21:21:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616889+0000", 2024-09-17T21:21:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012533100000000001 2024-09-17T21:21:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15721 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1039360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 92.162382131000001, 2024-09-17T21:21:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095277219999999996, 2024-09-17T21:21:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15721 2024-09-17T21:21:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:21:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521871+0000", 2024-09-17T21:21:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.443e-06 2024-09-17T21:21:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521876+0000", 2024-09-17T21:21:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0300000000000001e-06 2024-09-17T21:21:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521886+0000", 2024-09-17T21:21:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186e-05 2024-09-17T21:21:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521893+0000", 2024-09-17T21:21:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.08e-06 2024-09-17T21:21:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616918+0000", 2024-09-17T21:21:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095025223000000006 2024-09-17T21:21:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616977+0000", 2024-09-17T21:21:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8403999999999999e-05 2024-09-17T21:21:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617147+0000", 2024-09-17T21:21:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016985399999999999 2024-09-17T21:21:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15722 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 92.162347682000004, 2024-09-17T21:21:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095459783000000006, 2024-09-17T21:21:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15722 2024-09-17T21:21:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:21:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7700000000000004e-07 2024-09-17T21:21:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521907+0000", 2024-09-17T21:21:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.813e-06 2024-09-17T21:21:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521922+0000", 2024-09-17T21:21:54.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4533e-05 2024-09-17T21:21:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521934+0000", 2024-09-17T21:21:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2469999999999999e-05 2024-09-17T21:21:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617187+0000", 2024-09-17T21:21:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095252311000000006 2024-09-17T21:21:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617240+0000", 2024-09-17T21:21:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3740999999999999e-05 2024-09-17T21:21:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617364+0000", 2024-09-17T21:21:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-17T21:21:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15723 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 92.162303816000005, 2024-09-17T21:21:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095653563999999996, 2024-09-17T21:21:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:21:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15723 2024-09-17T21:21:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:21:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521949+0000", 2024-09-17T21:21:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:21:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521953+0000", 2024-09-17T21:21:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5870000000000003e-06 2024-09-17T21:21:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521970+0000", 2024-09-17T21:21:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7282000000000002e-05 2024-09-17T21:21:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521984+0000", 2024-09-17T21:21:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:21:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617383+0000", 2024-09-17T21:21:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095399063000000006 2024-09-17T21:21:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617437+0000", 2024-09-17T21:21:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3931999999999998e-05 2024-09-17T21:21:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617601+0000", 2024-09-17T21:21:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016407300000000001 2024-09-17T21:21:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5629.0:2384 3.1c 3:38c1263e:::rbd_data.1152d37183c3.000000000000000b:head [read 11780096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-17T21:21:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 79.259472986000006, 2024-09-17T21:21:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0246185320000001, 2024-09-17T21:21:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5629", 2024-09-17T21:21:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3798711329", 2024-09-17T21:21:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2384 2024-09-17T21:21:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:21:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424780+0000", 2024-09-17T21:21:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.13e-06 2024-09-17T21:21:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424785+0000", 2024-09-17T21:21:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4739999999999996e-06 2024-09-17T21:21:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424798+0000", 2024-09-17T21:21:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2704000000000001e-05 2024-09-17T21:21:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424806+0000", 2024-09-17T21:21:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9899999999999997e-06 2024-09-17T21:21:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424868+0000", 2024-09-17T21:21:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999997e-05 2024-09-17T21:21:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424875+0000", 2024-09-17T21:21:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0870000000000001e-06 2024-09-17T21:21:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428897+0000", 2024-09-17T21:21:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040226080000000004 2024-09-17T21:21:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428913+0000", 2024-09-17T21:21:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:21:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.448963+0000", 2024-09-17T21:21:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0200498659999999 2024-09-17T21:21:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449158+0000", 2024-09-17T21:21:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019581000000000001 2024-09-17T21:21:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449397+0000", 2024-09-17T21:21:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023871 2024-09-17T21:21:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:26043 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_remove_image in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-17T21:21:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:21:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.284941866, 2024-09-17T21:21:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.019978351, 2024-09-17T21:21:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 26043 2024-09-17T21:21:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:21:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:21:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399314+0000", 2024-09-17T21:21:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.963e-06 2024-09-17T21:21:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399330+0000", 2024-09-17T21:21:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6684000000000002e-05 2024-09-17T21:21:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399372+0000", 2024-09-17T21:21:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1819000000000001e-05 2024-09-17T21:21:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399396+0000", 2024-09-17T21:21:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.406e-05 2024-09-17T21:21:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399571+0000", 2024-09-17T21:21:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017444800000000001 2024-09-17T21:21:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399595+0000", 2024-09-17T21:21:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4709999999999999e-05 2024-09-17T21:21:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.412735+0000", 2024-09-17T21:21:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0131394869999999 2024-09-17T21:21:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.413090+0000", 2024-09-17T21:21:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000355006 2024-09-17T21:21:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:21:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.413659+0000", 2024-09-17T21:21:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056922699999999995 2024-09-17T21:21:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.418940+0000", 2024-09-17T21:21:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052814109999999997 2024-09-17T21:21:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.418992+0000", 2024-09-17T21:21:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1646e-05 2024-09-17T21:21:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.419168+0000", 2024-09-17T21:21:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017573400000000001 2024-09-17T21:21:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.419288+0000", 2024-09-17T21:21:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012015600000000001 2024-09-17T21:21:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:54.781 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:54.783 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:21:54.783 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:54.777+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:54.884 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:54.879+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:54.954 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:54.954 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:54.954 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:54.954 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:54.968 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:21:54.990 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:21:55.015 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:55.013+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:55.084 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:55.084 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:55.084 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:55.085 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:55.085 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:55.086 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:55.081+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:55.098 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:21:55.142 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:55.143 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:55.159 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:55.159 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:55.159 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T21:21:55.178 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:55.178 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:55.187 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:55.185+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:55.196 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:55.197 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:55.216 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:55.216 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:55.221 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:21:55.221 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:21:55.263 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:55.263 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:55.278 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:21:55.325 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:55.323+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:55.388 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:55.387+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:55.404 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:55.404 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:55.418 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:21:55.504 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:55.503+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:55.589 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:55.587+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:55.604 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:55.604 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:55.620 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:21:55.690 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:55.685+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:55.791 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:55.789+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:55.796 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:55.796 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:55.811 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:21:55.892 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:55.889+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:55.942 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:55.942 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:55.957 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:21:56.005 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:56.003+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:56.079 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:56.080 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:56.093 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:56.091+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:56.095 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:21:56.194 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:56.191+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:56.258 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:56.258 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:56.272 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:21:56.319 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:56.317+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:56.396 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:56.393+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:56.405 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:56.405 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:56.420 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:21:56.507 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:56.505+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:56.511 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:21:56.598 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:56.593+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:56.604 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:56.604 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:56.620 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:21:56.707 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:56.705+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:56.785 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:56.785 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:56.799 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:21:56.800 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:56.797+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:56.901 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:56.899+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:56.970 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:56.970 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:56.985 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:21:57.001 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:56.999+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:57.103 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:57.099+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:57.106 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:57.107 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:57.121 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:21:57.159 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 0] out_osds: [2, 5] dead_osds: [2] live_osds: [0, 1, 3, 4, 5] 2024-09-17T21:21:57.159 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:21:57.160 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-17T21:21:57.160 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-17T21:21:57.205 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:57.203+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:57.222 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:21:57.236 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:57.237 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:57.251 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:21:57.303 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-17T21:21:57.303 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:57.299+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:57.357 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-17T21:21:57.372 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:57.373 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:57.374 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:57.374 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:57.389 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:21:57.390 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:57.391 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:57.391 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T21:21:57.409 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:57.410 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:57.430 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:57.431 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:57.439 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:57.437+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:57.453 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:57.454 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:57.457 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:21:57.457 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:21:57.505 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:57.503+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:57.514 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:57.514 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:57.514 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:57.514 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:57.514 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:57.530 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:21:57.537 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-17T21:21:57.615 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:57.613+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:57.697 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:57.697 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:57.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:57.697 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:57.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.697 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:21:57.697 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:57.697 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 216.08849602800001, 2024-09-17T21:21:57.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:21:57.698 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:57.698 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:57.698 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:57.698 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:57.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:57.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:57.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:57.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:21:57.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:57.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:21:57.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:21:57.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:57.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:21:57.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:21:57.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:57.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:21:57.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:21:57.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:57.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:21:57.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:21:57.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:57.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:21:57.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:21:57.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:57.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:21:57.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:21:57.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:57.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:21:57.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:21:57.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:57.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:21:57.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:21:57.701 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.701 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:57.701 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.702 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:21:57.702 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:57.702 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 216.07854939200001, 2024-09-17T21:21:57.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:21:57.702 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:57.702 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:57.702 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:57.702 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:57.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:57.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:57.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:57.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:21:57.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:57.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:21:57.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:21:57.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:57.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:21:57.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:21:57.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:57.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:21:57.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:21:57.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:57.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:21:57.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:21:57.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:57.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:21:57.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:21:57.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:57.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:21:57.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:21:57.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:57.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:21:57.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:21:57.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:57.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:21:57.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:21:57.706 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.706 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:57.706 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.706 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:21:57.706 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:57.706 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 214.83539855199999, 2024-09-17T21:21:57.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:21:57.706 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:57.706 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:57.706 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:57.706 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:57.706 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:21:57.706 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:21:57.707 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:21:57.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.707 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:57.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:57.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:57.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:57.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:21:57.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:57.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:21:57.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:21:57.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:57.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:21:57.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:21:57.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:57.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:21:57.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:21:57.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:57.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:21:57.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:21:57.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:57.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:21:57.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:21:57.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:57.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:21:57.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:21:57.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:57.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:21:57.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:21:57.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:57.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:21:57.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:21:57.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:57.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:21:57.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:21:57.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:57.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:21:57.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:21:57.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:57.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:21:57.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:21:57.711 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.711 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:57.711 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.712 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:21:57.712 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:57.712 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 214.825753066, 2024-09-17T21:21:57.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:21:57.712 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:57.712 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:57.712 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:57.712 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:57.712 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:21:57.712 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:21:57.712 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:21:57.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.713 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:57.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:57.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:57.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:57.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:21:57.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:57.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:21:57.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:21:57.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:57.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:21:57.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:21:57.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:57.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:21:57.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:21:57.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:57.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:21:57.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:21:57.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:57.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:21:57.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:21:57.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:57.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:21:57.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:21:57.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:57.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:21:57.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:21:57.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:57.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:21:57.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:21:57.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:57.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:21:57.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:21:57.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:57.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:21:57.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:21:57.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:57.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:21:57.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:21:57.717 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.717 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:57.717 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.718 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:21:57.718 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:57.718 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 201.45681703299999, 2024-09-17T21:21:57.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:21:57.718 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:57.718 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:57.718 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:57.718 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:57.718 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:21:57.718 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:21:57.718 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:21:57.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.718 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:57.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:57.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:57.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:57.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:21:57.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:57.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:21:57.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:21:57.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:57.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:21:57.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:21:57.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:57.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:21:57.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:21:57.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:57.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:21:57.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:21:57.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:57.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:21:57.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:21:57.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:57.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:21:57.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:21:57.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:57.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:21:57.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:21:57.722 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.722 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:57.722 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.722 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:57.722 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:57.722 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 183.255132471, 2024-09-17T21:21:57.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:21:57.722 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:57.722 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:57.722 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:57.723 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:57.723 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:57.723 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:57.723 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:21:57.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.723 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:57.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:57.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:57.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:57.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:21:57.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:57.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:21:57.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:21:57.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:57.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:21:57.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:21:57.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:57.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:21:57.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:21:57.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:57.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:21:57.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:21:57.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:57.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:21:57.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:21:57.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:57.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:21:57.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:21:57.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:57.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:21:57.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:21:57.726 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.726 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:57.726 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.727 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:57.727 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:57.727 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 183.25463789400001, 2024-09-17T21:21:57.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:21:57.727 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:57.727 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:57.727 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:57.727 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:57.727 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:57.727 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:57.727 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:21:57.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.727 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:57.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:57.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:57.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:57.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:21:57.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:57.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:21:57.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:21:57.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:57.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:21:57.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:21:57.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:57.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:21:57.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:21:57.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:57.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:21:57.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:21:57.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:57.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:21:57.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:21:57.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:57.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:21:57.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:21:57.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:57.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:21:57.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:21:57.731 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.731 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:57.731 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.731 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:57.731 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:57.731 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 183.254582513, 2024-09-17T21:21:57.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:21:57.731 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:57.731 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:57.731 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:57.732 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:57.732 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:57.732 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:57.732 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:21:57.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.732 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:57.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:57.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:57.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:57.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:21:57.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:57.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:21:57.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:21:57.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:57.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:21:57.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:21:57.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:57.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:21:57.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:21:57.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:57.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:21:57.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:21:57.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:57.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:21:57.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:21:57.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:57.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:21:57.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:21:57.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:57.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:21:57.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:21:57.736 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.736 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:57.736 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.736 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:57.736 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:57.736 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 183.25452623499999, 2024-09-17T21:21:57.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:21:57.736 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:57.736 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:57.736 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:57.737 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:57.737 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:57.737 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:57.737 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:21:57.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.737 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:57.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:57.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:57.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:57.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:21:57.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:57.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:21:57.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:21:57.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:57.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:21:57.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:21:57.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:57.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:21:57.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:21:57.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:57.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:21:57.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:21:57.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:57.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:21:57.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:21:57.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:57.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:21:57.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:21:57.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:57.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:21:57.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:21:57.741 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.741 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:57.741 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.741 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:57.741 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:57.741 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 183.25357659599999, 2024-09-17T21:21:57.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:21:57.741 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:57.741 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:57.742 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:57.742 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:57.742 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:57.742 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:57.742 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:21:57.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.742 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:57.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:57.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:57.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:57.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:21:57.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:57.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:21:57.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:21:57.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:57.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:21:57.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:21:57.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:57.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:21:57.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:21:57.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:57.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:21:57.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:21:57.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:57.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:21:57.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:21:57.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:57.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:21:57.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:21:57.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:57.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:21:57.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:21:57.745 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.746 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:57.746 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.746 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:57.746 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:57.746 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 183.25349797600001, 2024-09-17T21:21:57.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:21:57.746 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:57.746 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:57.746 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:57.746 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:57.746 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:57.746 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:57.746 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:21:57.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.747 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:57.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:57.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:57.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:57.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:21:57.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:57.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:21:57.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:21:57.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:57.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:21:57.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:21:57.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:57.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:21:57.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:21:57.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:57.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:21:57.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:21:57.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:57.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:21:57.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:21:57.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:57.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:21:57.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:21:57.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:57.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:21:57.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:21:57.750 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.750 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:57.750 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.750 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:57.750 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:57.750 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 183.25338503699999, 2024-09-17T21:21:57.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:21:57.751 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:57.751 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:57.751 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:57.751 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:57.751 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:57.751 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:57.751 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:21:57.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.751 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:57.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:57.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:57.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:57.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:21:57.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:57.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:21:57.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:21:57.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:57.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:21:57.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:21:57.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:57.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:21:57.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:21:57.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:57.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:21:57.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:21:57.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:57.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:21:57.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:21:57.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:57.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:21:57.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:21:57.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:57.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:21:57.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:21:57.754 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.755 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:57.755 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.755 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:57.755 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:57.755 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 183.25272398300001, 2024-09-17T21:21:57.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:21:57.755 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:57.755 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:57.755 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:57.755 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:57.755 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:57.756 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:57.756 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:21:57.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.756 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:57.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:57.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:57.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:57.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:21:57.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:57.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:21:57.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:21:57.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:57.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:21:57.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:21:57.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:57.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:21:57.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:21:57.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:57.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:21:57.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:21:57.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:57.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:21:57.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:21:57.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:57.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:21:57.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:21:57.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:57.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:21:57.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:21:57.759 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.759 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:57.759 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.760 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:57.760 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:57.760 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 183.25265917199999, 2024-09-17T21:21:57.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:21:57.760 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:57.760 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:57.760 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:57.760 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:57.760 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:57.760 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:57.760 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:21:57.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.760 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:57.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:57.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:57.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:57.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:21:57.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:57.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:21:57.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:21:57.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:57.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:21:57.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:21:57.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:57.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:21:57.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:21:57.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:57.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:21:57.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:21:57.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:57.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:21:57.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:21:57.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:57.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:21:57.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:21:57.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:57.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:21:57.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:21:57.764 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.764 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:57.764 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.764 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:21:57.764 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:57.764 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 183.25259819499999, 2024-09-17T21:21:57.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:21:57.764 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:57.764 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:57.765 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:57.765 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:57.765 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:21:57.765 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:57.765 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:21:57.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.765 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:57.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:57.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:57.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:57.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:21:57.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:57.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:21:57.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:21:57.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:57.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:21:57.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:21:57.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:57.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:21:57.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:21:57.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:57.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:21:57.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:21:57.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:57.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:21:57.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:21:57.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:57.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:21:57.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:21:57.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:57.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:21:57.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:21:57.768 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.768 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:57.768 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.769 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:21:57.769 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:57.769 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 128.35502700999999, 2024-09-17T21:21:57.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:21:57.769 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:57.769 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:57.769 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:57.769 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:57.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:57.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:57.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:57.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:21:57.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:57.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:21:57.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:21:57.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:57.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:21:57.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:21:57.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:57.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:21:57.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:21:57.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:57.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:21:57.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:21:57.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:57.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:21:57.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:21:57.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:57.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:21:57.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:21:57.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:57.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:21:57.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:21:57.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:57.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:21:57.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:21:57.773 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.773 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:57.773 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.773 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:21:57.773 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:57.773 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 128.14636861599999, 2024-09-17T21:21:57.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:21:57.773 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:21:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:21:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:21:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:21:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:21:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:21:57.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:21:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:21:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:21:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:21:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:21:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:21:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:21:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:21:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:21:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:21:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:21:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.777 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:9547 3.2 e147/132 3:42b518d9:::rbd_data.1152f056b73f.0000000000000001:head v 147'513, mlcod=142'511)", 2024-09-17T21:21:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.221395493999999, 2024-09-17T21:21:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.031059566, 2024-09-17T21:21:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:57.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:21:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471293+0000", 2024-09-17T21:21:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2029999999999999e-06 2024-09-17T21:21:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480606+0000", 2024-09-17T21:21:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093123839999999999 2024-09-17T21:21:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480653+0000", 2024-09-17T21:21:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7852000000000002e-05 2024-09-17T21:21:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480673+0000", 2024-09-17T21:21:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0013e-05 2024-09-17T21:21:57.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480862+0000", 2024-09-17T21:21:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018833599999999999 2024-09-17T21:21:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480906+0000", 2024-09-17T21:21:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4162000000000002e-05 2024-09-17T21:21:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502259+0000", 2024-09-17T21:21:57.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021353154999999999 2024-09-17T21:21:57.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:57.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502351+0000", 2024-09-17T21:21:57.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1460999999999997e-05 2024-09-17T21:21:57.782 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.782 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:57.782 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.782 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:15234 3.1 e147/134 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head v 147'917, mlcod=145'914)", 2024-09-17T21:21:57.782 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:57.782 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.588734875999997, 2024-09-17T21:21:57.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045303695999999997, 2024-09-17T21:21:57.783 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:57.783 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:57.783 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:57.783 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:57.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:57.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:57.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:57.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:21:57.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103954+0000", 2024-09-17T21:21:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2259999999999999e-06 2024-09-17T21:21:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119105+0000", 2024-09-17T21:21:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015150871 2024-09-17T21:21:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:57.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119141+0000", 2024-09-17T21:21:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6239000000000003e-05 2024-09-17T21:21:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119159+0000", 2024-09-17T21:21:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7946999999999999e-05 2024-09-17T21:21:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119344+0000", 2024-09-17T21:21:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018524599999999999 2024-09-17T21:21:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:57.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119399+0000", 2024-09-17T21:21:57.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4922e-05 2024-09-17T21:21:57.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:57.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149165+0000", 2024-09-17T21:21:57.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029765697000000001 2024-09-17T21:21:57.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:57.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149255+0000", 2024-09-17T21:21:57.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0548000000000001e-05 2024-09-17T21:21:57.786 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.787 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:57.787 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.787 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:19859 3.8 e147/133 3:12f72409:::rbd_data.1152f056b73f.0000000000000003:head v 147'482, mlcod=145'479)", 2024-09-17T21:21:57.787 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:57.787 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.179843808999998, 2024-09-17T21:21:57.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033476895999999999, 2024-09-17T21:21:57.787 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:57.787 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:57.787 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:57.787 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:57.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:57.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:57.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:57.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:21:57.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:57.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:57.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512845+0000", 2024-09-17T21:21:57.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.886e-06 2024-09-17T21:21:57.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:57.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524104+0000", 2024-09-17T21:21:57.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011259668 2024-09-17T21:21:57.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:57.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524148+0000", 2024-09-17T21:21:57.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3386000000000001e-05 2024-09-17T21:21:57.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:57.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524172+0000", 2024-09-17T21:21:57.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4762000000000001e-05 2024-09-17T21:21:57.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:57.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524379+0000", 2024-09-17T21:21:57.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020671599999999999 2024-09-17T21:21:57.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:57.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524421+0000", 2024-09-17T21:21:57.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2182999999999999e-05 2024-09-17T21:21:57.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:57.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546216+0000", 2024-09-17T21:21:57.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021795101000000001 2024-09-17T21:21:57.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:57.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:57.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:57.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546320+0000", 2024-09-17T21:21:57.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103194 2024-09-17T21:21:57.791 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.791 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:57.791 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.791 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:57.791 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:57.791 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:57.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-17T21:21:57.794 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:57.791+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:57.794 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:21:57.893 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:57.891+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:57.959 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:57.959 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:57.959 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:21:57.959 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:57.974 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:21:57.994 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:57.989+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:58.095 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:58.093+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:58.167 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:58.167 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:58.167 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:21:58.167 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:21:58.167 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:58.181 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:21:58.196 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:58.193+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:58.297 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:58.293+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:58.309 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:58.310 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:58.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:58.310 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:58.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.310 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:21:58.310 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:58.310 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 161.79584463, 2024-09-17T21:21:58.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:21:58.310 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.310 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.310 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:58.310 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.310 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:21:58.310 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:21:58.311 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:21:58.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.311 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:58.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:21:58.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:21:58.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:21:58.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:21:58.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:21:58.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:21:58.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:21:58.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:21:58.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:21:58.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:21:58.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:21:58.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:58.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:21:58.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:21:58.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:21:58.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:21:58.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:21:58.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:21:58.314 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.314 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.314 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.315 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2597 3.1f 3:fb0e9615:::rbd_data.1152a8ae2019.000000000000000c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:58.315 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:58.315 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 137.467408202, 2024-09-17T21:21:58.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097064994000000002, 2024-09-17T21:21:58.315 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.315 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.315 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.315 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.315 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:58.315 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:58.315 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2597 2024-09-17T21:21:58.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.315 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:58.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:21:58.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838628+0000", 2024-09-17T21:21:58.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.187e-06 2024-09-17T21:21:58.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838654+0000", 2024-09-17T21:21:58.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:21:58.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838696+0000", 2024-09-17T21:21:58.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1226000000000001e-05 2024-09-17T21:21:58.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838715+0000", 2024-09-17T21:21:58.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9646000000000001e-05 2024-09-17T21:21:58.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838937+0000", 2024-09-17T21:21:58.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022163 2024-09-17T21:21:58.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.839284+0000", 2024-09-17T21:21:58.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034753799999999999 2024-09-17T21:21:58.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935691+0000", 2024-09-17T21:21:58.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.096406360999999996 2024-09-17T21:21:58.319 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.319 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.319 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.319 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2641 3.12 3:488a729f:::rbd_data.1152a8ae2019.0000000000000022:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:58.319 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:58.319 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 137.46395142899999, 2024-09-17T21:21:58.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098280953000000004, 2024-09-17T21:21:58.319 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.319 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.319 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.319 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.319 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:58.320 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:58.320 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2641 2024-09-17T21:21:58.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.320 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:58.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:21:58.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842083+0000", 2024-09-17T21:21:58.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5300000000000002e-07 2024-09-17T21:21:58.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842088+0000", 2024-09-17T21:21:58.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:21:58.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842105+0000", 2024-09-17T21:21:58.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6773e-05 2024-09-17T21:21:58.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842119+0000", 2024-09-17T21:21:58.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:21:58.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842320+0000", 2024-09-17T21:21:58.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020052700000000001 2024-09-17T21:21:58.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842845+0000", 2024-09-17T21:21:58.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052508600000000002 2024-09-17T21:21:58.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940363+0000", 2024-09-17T21:21:58.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097518795000000005 2024-09-17T21:21:58.323 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.323 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.323 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.323 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2674 3.12 3:4c87e3ac:::rbd_data.1152a8ae2019.0000000000000033:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:58.324 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:58.324 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 137.463588792, 2024-09-17T21:21:58.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098025125000000005, 2024-09-17T21:21:58.324 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.324 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.324 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:58.324 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.324 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:58.324 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:58.324 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2674 2024-09-17T21:21:58.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.324 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:58.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:21:58.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842447+0000", 2024-09-17T21:21:58.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3999999999999999e-06 2024-09-17T21:21:58.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842453+0000", 2024-09-17T21:21:58.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:21:58.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842472+0000", 2024-09-17T21:21:58.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9006000000000001e-05 2024-09-17T21:21:58.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842487+0000", 2024-09-17T21:21:58.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4866e-05 2024-09-17T21:21:58.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843594+0000", 2024-09-17T21:21:58.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011073249999999999 2024-09-17T21:21:58.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843668+0000", 2024-09-17T21:21:58.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3188000000000005e-05 2024-09-17T21:21:58.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:58.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843893+0000", 2024-09-17T21:21:58.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022503399999999999 2024-09-17T21:21:58.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:58.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850439+0000", 2024-09-17T21:21:58.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065467880000000004 2024-09-17T21:21:58.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:58.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940409+0000", 2024-09-17T21:21:58.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089969511000000002 2024-09-17T21:21:58.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:58.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940444+0000", 2024-09-17T21:21:58.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5345999999999997e-05 2024-09-17T21:21:58.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940470+0000", 2024-09-17T21:21:58.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5964000000000001e-05 2024-09-17T21:21:58.329 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.329 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.329 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.329 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2707 3.1f 3:f805e65e:::rbd_data.1152a8ae2019.0000000000000049:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:58.329 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:58.330 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 137.458542131, 2024-09-17T21:21:58.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088327300999999997, 2024-09-17T21:21:58.330 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.330 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.330 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.330 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.330 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:58.330 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:58.330 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2707 2024-09-17T21:21:58.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.330 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:58.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:21:58.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847494+0000", 2024-09-17T21:21:58.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:21:58.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847504+0000", 2024-09-17T21:21:58.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:21:58.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847526+0000", 2024-09-17T21:21:58.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1976000000000001e-05 2024-09-17T21:21:58.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847541+0000", 2024-09-17T21:21:58.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5277e-05 2024-09-17T21:21:58.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856142+0000", 2024-09-17T21:21:58.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086006710000000007 2024-09-17T21:21:58.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856435+0000", 2024-09-17T21:21:58.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029301499999999999 2024-09-17T21:21:58.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935819+0000", 2024-09-17T21:21:58.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079384448999999996 2024-09-17T21:21:58.333 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.334 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.334 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.334 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2730 3.12 3:48b3422a:::rbd_data.1152a8ae2019.0000000000000057:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:58.334 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:58.334 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 137.449349232, 2024-09-17T21:21:58.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083879975999999995, 2024-09-17T21:21:58.334 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.334 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.335 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.335 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.335 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:58.335 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:58.335 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2730 2024-09-17T21:21:58.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.335 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:58.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:21:58.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856687+0000", 2024-09-17T21:21:58.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:21:58.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856697+0000", 2024-09-17T21:21:58.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0207000000000001e-05 2024-09-17T21:21:58.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856808+0000", 2024-09-17T21:21:58.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011071200000000001 2024-09-17T21:21:58.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856828+0000", 2024-09-17T21:21:58.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0219000000000001e-05 2024-09-17T21:21:58.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856907+0000", 2024-09-17T21:21:58.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9661999999999997e-05 2024-09-17T21:21:58.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.857019+0000", 2024-09-17T21:21:58.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011148499999999999 2024-09-17T21:21:58.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940565+0000", 2024-09-17T21:21:58.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083545791999999994 2024-09-17T21:21:58.338 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.339 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.339 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.339 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2743 3.12 3:4d036e62:::rbd_data.1152a8ae2019.000000000000005f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:58.339 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:58.339 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 137.445403731, 2024-09-17T21:21:58.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080059962999999998, 2024-09-17T21:21:58.339 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.339 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.339 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.339 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.339 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:58.339 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:58.340 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2743 2024-09-17T21:21:58.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.340 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:58.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:21:58.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860631+0000", 2024-09-17T21:21:58.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:21:58.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860636+0000", 2024-09-17T21:21:58.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5279999999999997e-06 2024-09-17T21:21:58.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860649+0000", 2024-09-17T21:21:58.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-17T21:21:58.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860655+0000", 2024-09-17T21:21:58.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:21:58.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860731+0000", 2024-09-17T21:21:58.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5920000000000005e-05 2024-09-17T21:21:58.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.861070+0000", 2024-09-17T21:21:58.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338535 2024-09-17T21:21:58.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940690+0000", 2024-09-17T21:21:58.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079620503999999995 2024-09-17T21:21:58.343 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.343 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.343 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.343 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3378 3.12 3:4832a51b:::rbd_data.1152a8ae2019.000000000000006c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:58.343 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:58.343 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 137.41703988099999, 2024-09-17T21:21:58.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051812516000000003, 2024-09-17T21:21:58.344 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.344 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.344 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:58.344 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.344 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:58.344 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:58.344 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3378 2024-09-17T21:21:58.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.344 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:58.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:21:58.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888995+0000", 2024-09-17T21:21:58.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:21:58.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889006+0000", 2024-09-17T21:21:58.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1012999999999999e-05 2024-09-17T21:21:58.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889032+0000", 2024-09-17T21:21:58.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6441e-05 2024-09-17T21:21:58.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889040+0000", 2024-09-17T21:21:58.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0639999999999994e-06 2024-09-17T21:21:58.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889278+0000", 2024-09-17T21:21:58.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023785899999999999 2024-09-17T21:21:58.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889408+0000", 2024-09-17T21:21:58.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129219 2024-09-17T21:21:58.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:58.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889723+0000", 2024-09-17T21:21:58.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000315264 2024-09-17T21:21:58.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:58.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.891972+0000", 2024-09-17T21:21:58.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002249633 2024-09-17T21:21:58.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:58.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940733+0000", 2024-09-17T21:21:58.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048760626000000001 2024-09-17T21:21:58.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:58.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940785+0000", 2024-09-17T21:21:58.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1906000000000001e-05 2024-09-17T21:21:58.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940807+0000", 2024-09-17T21:21:58.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1554999999999999e-05 2024-09-17T21:21:58.349 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.349 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.349 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.349 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3379 3.12 3:4e83fd71:::rbd_data.1152a8ae2019.000000000000006d:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:58.349 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:58.349 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 137.41695567400001, 2024-09-17T21:21:58.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051804144000000003, 2024-09-17T21:21:58.350 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.350 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.350 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:58.350 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.350 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:21:58.350 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:58.350 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3379 2024-09-17T21:21:58.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.350 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:58.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:21:58.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889079+0000", 2024-09-17T21:21:58.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1800000000000004e-07 2024-09-17T21:21:58.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889083+0000", 2024-09-17T21:21:58.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:21:58.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889094+0000", 2024-09-17T21:21:58.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1562999999999999e-05 2024-09-17T21:21:58.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889102+0000", 2024-09-17T21:21:58.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.401e-06 2024-09-17T21:21:58.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:21:58.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014279259999999999 2024-09-17T21:21:58.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890609+0000", 2024-09-17T21:21:58.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9566000000000004e-05 2024-09-17T21:21:58.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:58.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890789+0000", 2024-09-17T21:21:58.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017949700000000001 2024-09-17T21:21:58.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:58.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.893929+0000", 2024-09-17T21:21:58.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003140538 2024-09-17T21:21:58.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:58.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940834+0000", 2024-09-17T21:21:58.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046904811999999997 2024-09-17T21:21:58.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:58.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940860+0000", 2024-09-17T21:21:58.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5341999999999999e-05 2024-09-17T21:21:58.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940882+0000", 2024-09-17T21:21:58.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2878999999999999e-05 2024-09-17T21:21:58.355 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.355 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.355 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.355 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:2644 3.1e 3:79615585:::rbd_data.115266d0b9f2.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2475520~14848,write 3106304~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:21:58.355 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:58.355 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 128.79432727099999, 2024-09-17T21:21:58.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050733484000000002, 2024-09-17T21:21:58.355 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.355 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.356 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:58.356 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.356 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:21:58.356 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:21:58.356 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2644 2024-09-17T21:21:58.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.356 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:58.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:21:58.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511708+0000", 2024-09-17T21:21:58.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-17T21:21:58.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522196+0000", 2024-09-17T21:21:58.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010487712999999999 2024-09-17T21:21:58.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522223+0000", 2024-09-17T21:21:58.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7064999999999999e-05 2024-09-17T21:21:58.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522236+0000", 2024-09-17T21:21:58.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2931e-05 2024-09-17T21:21:58.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522348+0000", 2024-09-17T21:21:58.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111906 2024-09-17T21:21:58.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522554+0000", 2024-09-17T21:21:58.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020632699999999999 2024-09-17T21:21:58.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:21:58.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.523185+0000", 2024-09-17T21:21:58.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063081400000000005 2024-09-17T21:21:58.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:58.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.547011+0000", 2024-09-17T21:21:58.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023825570000000001 2024-09-17T21:21:58.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:58.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562268+0000", 2024-09-17T21:21:58.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015257514 2024-09-17T21:21:58.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:58.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562367+0000", 2024-09-17T21:21:58.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8678000000000006e-05 2024-09-17T21:21:58.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562440+0000", 2024-09-17T21:21:58.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3337000000000006e-05 2024-09-17T21:21:58.360 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.360 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.361 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.361 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:231 3.1 3:856fed78:::rbd_data.115220dd393e.000000000000000b:head [read 0~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:58.361 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:58.361 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 100.351572002, 2024-09-17T21:21:58.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34254251699999999, 2024-09-17T21:21:58.361 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.361 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.361 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:58.361 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.361 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:58.361 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:58.361 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 231 2024-09-17T21:21:58.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.362 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:58.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:21:58.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954464+0000", 2024-09-17T21:21:58.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:21:58.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954475+0000", 2024-09-17T21:21:58.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1146e-05 2024-09-17T21:21:58.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954506+0000", 2024-09-17T21:21:58.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0292999999999999e-05 2024-09-17T21:21:58.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954524+0000", 2024-09-17T21:21:58.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8430000000000001e-05 2024-09-17T21:21:58.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954661+0000", 2024-09-17T21:21:58.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136403 2024-09-17T21:21:58.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:58.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954680+0000", 2024-09-17T21:21:58.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9743000000000001e-05 2024-09-17T21:21:58.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.296896+0000", 2024-09-17T21:21:58.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34221550499999998 2024-09-17T21:21:58.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297004+0000", 2024-09-17T21:21:58.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010865399999999999 2024-09-17T21:21:58.365 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.365 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.365 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.366 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:56 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:58.366 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:58.366 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 100.195938305, 2024-09-17T21:21:58.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18718773899999999, 2024-09-17T21:21:58.366 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.366 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.366 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.366 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.366 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:21:58.366 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:21:58.366 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 56 2024-09-17T21:21:58.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.366 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:58.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:21:58.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110098+0000", 2024-09-17T21:21:58.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:21:58.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110111+0000", 2024-09-17T21:21:58.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2289999999999999e-05 2024-09-17T21:21:58.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110143+0000", 2024-09-17T21:21:58.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1979e-05 2024-09-17T21:21:58.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110161+0000", 2024-09-17T21:21:58.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7839e-05 2024-09-17T21:21:58.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110319+0000", 2024-09-17T21:21:58.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015835400000000001 2024-09-17T21:21:58.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:58.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110353+0000", 2024-09-17T21:21:58.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3849000000000001e-05 2024-09-17T21:21:58.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297041+0000", 2024-09-17T21:21:58.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18668842299999999 2024-09-17T21:21:58.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297169+0000", 2024-09-17T21:21:58.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012786199999999999 2024-09-17T21:21:58.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297283+0000", 2024-09-17T21:21:58.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011430300000000001 2024-09-17T21:21:58.370 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.370 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.370 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.371 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:68 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:21:58.371 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:58.371 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 100.187397374, 2024-09-17T21:21:58.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17885582999999999, 2024-09-17T21:21:58.371 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.371 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.371 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.371 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.371 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:21:58.371 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:21:58.371 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 68 2024-09-17T21:21:58.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.371 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:58.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:21:58.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118639+0000", 2024-09-17T21:21:58.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.306e-06 2024-09-17T21:21:58.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118651+0000", 2024-09-17T21:21:58.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2567e-05 2024-09-17T21:21:58.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118682+0000", 2024-09-17T21:21:58.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0502e-05 2024-09-17T21:21:58.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118707+0000", 2024-09-17T21:21:58.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5463e-05 2024-09-17T21:21:58.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118889+0000", 2024-09-17T21:21:58.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018150999999999999 2024-09-17T21:21:58.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:58.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118910+0000", 2024-09-17T21:21:58.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:21:58.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297313+0000", 2024-09-17T21:21:58.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17840366699999999 2024-09-17T21:21:58.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297389+0000", 2024-09-17T21:21:58.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-17T21:21:58.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297492+0000", 2024-09-17T21:21:58.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103019 2024-09-17T21:21:58.375 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.375 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.375 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.375 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5890.0:15234 3.1 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8451072~29696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-17T21:21:58.375 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:58.375 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 55.212748562000002, 2024-09-17T21:21:58.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.057026052000000001, 2024-09-17T21:21:58.376 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.376 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.376 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:58.376 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.376 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5890", 2024-09-17T21:21:58.376 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3064344947", 2024-09-17T21:21:58.376 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 15234 2024-09-17T21:21:58.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.376 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:58.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:21:58.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093289+0000", 2024-09-17T21:21:58.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4939999999999999e-06 2024-09-17T21:21:58.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103074+0000", 2024-09-17T21:21:58.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00978564 2024-09-17T21:21:58.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103112+0000", 2024-09-17T21:21:58.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7246999999999998e-05 2024-09-17T21:21:58.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103128+0000", 2024-09-17T21:21:58.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5812999999999999e-05 2024-09-17T21:21:58.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103207+0000", 2024-09-17T21:21:58.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9785999999999998e-05 2024-09-17T21:21:58.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103315+0000", 2024-09-17T21:21:58.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107396 2024-09-17T21:21:58.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:58.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103957+0000", 2024-09-17T21:21:58.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000642524 2024-09-17T21:21:58.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:58.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.123856+0000", 2024-09-17T21:21:58.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019898376999999998 2024-09-17T21:21:58.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:58.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150190+0000", 2024-09-17T21:21:58.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026334682000000002 2024-09-17T21:21:58.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:58.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150255+0000", 2024-09-17T21:21:58.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4770999999999994e-05 2024-09-17T21:21:58.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150311+0000", 2024-09-17T21:21:58.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6322e-05 2024-09-17T21:21:58.381 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.381 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.381 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.381 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4239 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 424960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:58.381 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:21:58.381 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 45.552350904999997, 2024-09-17T21:21:58.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050123167000000003, 2024-09-17T21:21:58.381 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.381 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.381 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.381 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.381 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:58.381 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:58.382 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4239 2024-09-17T21:21:58.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.382 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:21:58.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:21:58.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753684+0000", 2024-09-17T21:21:58.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.37e-07 2024-09-17T21:21:58.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753686+0000", 2024-09-17T21:21:58.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.322e-06 2024-09-17T21:21:58.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753693+0000", 2024-09-17T21:21:58.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0890000000000002e-06 2024-09-17T21:21:58.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753699+0000", 2024-09-17T21:21:58.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:21:58.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803390+0000", 2024-09-17T21:21:58.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049690506000000002 2024-09-17T21:21:58.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803569+0000", 2024-09-17T21:21:58.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179784 2024-09-17T21:21:58.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803806+0000", 2024-09-17T21:21:58.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023683499999999999 2024-09-17T21:21:58.385 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.385 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.385 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.385 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4240 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 413696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:58.385 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:58.385 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 45.552327177000002, 2024-09-17T21:21:58.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050331353000000002, 2024-09-17T21:21:58.385 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.386 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.386 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.386 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.386 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:58.386 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:58.386 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4240 2024-09-17T21:21:58.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.386 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:58.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:58.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:21:58.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5700000000000002e-07 2024-09-17T21:21:58.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753710+0000", 2024-09-17T21:21:58.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.385e-06 2024-09-17T21:21:58.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753717+0000", 2024-09-17T21:21:58.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.832e-06 2024-09-17T21:21:58.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753722+0000", 2024-09-17T21:21:58.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8050000000000003e-06 2024-09-17T21:21:58.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803880+0000", 2024-09-17T21:21:58.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050157641000000003 2024-09-17T21:21:58.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803945+0000", 2024-09-17T21:21:58.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5357000000000005e-05 2024-09-17T21:21:58.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804038+0000", 2024-09-17T21:21:58.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2775999999999994e-05 2024-09-17T21:21:58.389 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.389 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.389 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.390 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4241 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 415744~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:58.390 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:58.390 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 45.552304061000001, 2024-09-17T21:21:58.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050578790999999998, 2024-09-17T21:21:58.390 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.390 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.390 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.390 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.390 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:58.390 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:58.390 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4241 2024-09-17T21:21:58.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.391 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:58.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:58.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:21:58.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5700000000000002e-07 2024-09-17T21:21:58.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753733+0000", 2024-09-17T21:21:58.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2570000000000002e-06 2024-09-17T21:21:58.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753740+0000", 2024-09-17T21:21:58.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.849e-06 2024-09-17T21:21:58.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753745+0000", 2024-09-17T21:21:58.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7450000000000001e-06 2024-09-17T21:21:58.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804115+0000", 2024-09-17T21:21:58.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050370038999999998 2024-09-17T21:21:58.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804223+0000", 2024-09-17T21:21:58.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010786 2024-09-17T21:21:58.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804309+0000", 2024-09-17T21:21:58.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5483999999999997e-05 2024-09-17T21:21:58.394 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.394 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.394 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.394 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4242 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 414720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:21:58.394 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:58.394 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 45.552281172999997, 2024-09-17T21:21:58.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050748148999999999, 2024-09-17T21:21:58.394 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.394 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.395 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.395 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.395 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:21:58.395 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:21:58.395 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4242 2024-09-17T21:21:58.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.395 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:58.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:58.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:21:58.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6599999999999996e-07 2024-09-17T21:21:58.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753756+0000", 2024-09-17T21:21:58.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.368e-06 2024-09-17T21:21:58.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753763+0000", 2024-09-17T21:21:58.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9349999999999997e-06 2024-09-17T21:21:58.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753768+0000", 2024-09-17T21:21:58.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7620000000000001e-06 2024-09-17T21:21:58.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804343+0000", 2024-09-17T21:21:58.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050575050000000003 2024-09-17T21:21:58.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804397+0000", 2024-09-17T21:21:58.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4070000000000002e-05 2024-09-17T21:21:58.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804501+0000", 2024-09-17T21:21:58.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103398 2024-09-17T21:21:58.398 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.398 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.398 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.399 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25642 3.1 3.6a5ad301 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e177)", 2024-09-17T21:21:58.399 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:34.316616+0000", 2024-09-17T21:21:58.399 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 23.989417589999999, 2024-09-17T21:21:58.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99065658300000003, 2024-09-17T21:21:58.399 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.399 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.399 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:58.399 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.399 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:21:58.399 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:58.399 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25642 2024-09-17T21:21:58.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.399 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316616+0000", 2024-09-17T21:21:58.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316616+0000", 2024-09-17T21:21:58.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316618+0000", 2024-09-17T21:21:58.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9309999999999998e-06 2024-09-17T21:21:58.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316624+0000", 2024-09-17T21:21:58.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2789999999999998e-06 2024-09-17T21:21:58.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316634+0000", 2024-09-17T21:21:58.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0774000000000001e-05 2024-09-17T21:21:58.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316644+0000", 2024-09-17T21:21:58.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0043e-05 2024-09-17T21:21:58.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316767+0000", 2024-09-17T21:21:58.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012289199999999999 2024-09-17T21:21:58.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:58.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316777+0000", 2024-09-17T21:21:58.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6199999999999994e-06 2024-09-17T21:21:58.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.307241+0000", 2024-09-17T21:21:58.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99046395099999995 2024-09-17T21:21:58.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.307273+0000", 2024-09-17T21:21:58.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2092999999999999e-05 2024-09-17T21:21:58.403 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.403 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.403 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.403 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25905 3.6 3:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-17T21:21:58.404 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:39.359572+0000", 2024-09-17T21:21:58.404 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 18.946462308000001, 2024-09-17T21:21:58.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96065702100000006, 2024-09-17T21:21:58.404 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.404 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.404 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:58.404 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.404 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:21:58.404 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:58.404 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25905 2024-09-17T21:21:58.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.404 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359572+0000", 2024-09-17T21:21:58.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359572+0000", 2024-09-17T21:21:58.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359573+0000", 2024-09-17T21:21:58.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5370000000000001e-06 2024-09-17T21:21:58.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359579+0000", 2024-09-17T21:21:58.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8039999999999998e-06 2024-09-17T21:21:58.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359603+0000", 2024-09-17T21:21:58.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4533999999999999e-05 2024-09-17T21:21:58.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359613+0000", 2024-09-17T21:21:58.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2820000000000002e-06 2024-09-17T21:21:58.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359686+0000", 2024-09-17T21:21:58.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3301999999999997e-05 2024-09-17T21:21:58.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:58.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359693+0000", 2024-09-17T21:21:58.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3679999999999996e-06 2024-09-17T21:21:58.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:40.319999+0000", 2024-09-17T21:21:58.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96030524500000003 2024-09-17T21:21:58.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:40.320229+0000", 2024-09-17T21:21:58.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022994899999999999 2024-09-17T21:21:58.408 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.408 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.408 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.408 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.408 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.409 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:58.410 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:21:58.410 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:58.405+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:58.511 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:58.509+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:58.550 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:58.551 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:58.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.551 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:28196 3.4 3:2067d2f2:::rbd_header.1152a8ae2019:head [watch unwatch cookie 139860912946224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e201)", 2024-09-17T21:21:58.551 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:58.547727+0000", 2024-09-17T21:21:58.551 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0021559370000000001, 2024-09-17T21:21:58.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021744939999999999, 2024-09-17T21:21:58.551 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.551 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.551 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:21:58.551 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.551 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:21:58.551 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:58.552 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 28196 2024-09-17T21:21:58.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.552 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:58.547727+0000", 2024-09-17T21:21:58.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:58.547727+0000", 2024-09-17T21:21:58.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:58.547728+0000", 2024-09-17T21:21:58.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.057e-06 2024-09-17T21:21:58.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:58.547732+0000", 2024-09-17T21:21:58.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9330000000000003e-06 2024-09-17T21:21:58.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:58.547744+0000", 2024-09-17T21:21:58.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2437e-05 2024-09-17T21:21:58.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:58.547753+0000", 2024-09-17T21:21:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2730000000000002e-06 2024-09-17T21:21:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:58.548067+0000", 2024-09-17T21:21:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031444200000000001 2024-09-17T21:21:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:58.548311+0000", 2024-09-17T21:21:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024341200000000001 2024-09-17T21:21:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:58.548720+0000", 2024-09-17T21:21:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00040910600000000001 2024-09-17T21:21:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:28197 3.4 3.4f4be604 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e201)", 2024-09-17T21:21:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:58.548391+0000", 2024-09-17T21:21:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0014923919999999999, 2024-09-17T21:21:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015523119999999999, 2024-09-17T21:21:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T21:21:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:21:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 28197 2024-09-17T21:21:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:58.548391+0000", 2024-09-17T21:21:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:58.548391+0000", 2024-09-17T21:21:58.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:58.548393+0000", 2024-09-17T21:21:58.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3829999999999999e-06 2024-09-17T21:21:58.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:58.548407+0000", 2024-09-17T21:21:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4113e-05 2024-09-17T21:21:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:58.548434+0000", 2024-09-17T21:21:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7248999999999999e-05 2024-09-17T21:21:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:58.548459+0000", 2024-09-17T21:21:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4545999999999998e-05 2024-09-17T21:21:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.559 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.559 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T21:21:58.559 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 2 2024-09-17T21:21:58.559 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:58.570 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:21:58.611 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:58.609+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:58.699 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:21:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:21:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:21:58.700 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:58.712 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:58.707+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:58.715 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:21:58.813 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:58.811+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:58.887 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:21:58.887 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:21:58.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:21:58.887 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:21:58.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.887 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:21:58.887 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:58.887 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 199.07330003499999, 2024-09-17T21:21:58.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:21:58.887 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.887 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.887 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.887 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.888 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:21:58.888 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:21:58.888 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:21:58.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.888 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:58.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:21:58.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:21:58.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:21:58.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:21:58.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:21:58.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:21:58.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:21:58.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:21:58.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:21:58.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:21:58.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:21:58.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:58.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:21:58.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:21:58.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:21:58.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:21:58.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:21:58.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:21:58.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:21:58.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:21:58.892 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.892 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.892 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.892 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:21:58.892 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:58.892 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 185.51819824899999, 2024-09-17T21:21:58.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:21:58.892 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.892 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.892 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:58.892 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.892 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:21:58.893 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:21:58.893 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:21:58.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.893 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.893 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.893 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:58.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.893 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.893 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:21:58.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:21:58.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:21:58.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:21:58.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:21:58.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:21:58.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:21:58.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:21:58.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:21:58.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:21:58.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:21:58.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:58.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:21:58.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:21:58.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:21:58.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:21:58.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:21:58.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:21:58.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:58.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:21:58.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:21:58.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:58.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:21:58.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:21:58.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:58.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:21:58.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:21:58.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:58.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:21:58.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:21:58.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:21:58.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:21:58.898 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.898 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.898 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.898 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:793 3.9 3:95f63599:::rbd_data.1152eb799094.0000000000000007:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:58.898 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:58.899 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 102.45114209400001, 2024-09-17T21:21:58.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88245766400000003, 2024-09-17T21:21:58.899 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.899 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.899 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:58.899 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.899 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:58.899 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:58.899 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 793 2024-09-17T21:21:58.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.899 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:58.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:21:58.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431941+0000", 2024-09-17T21:21:58.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.864e-06 2024-09-17T21:21:58.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431953+0000", 2024-09-17T21:21:58.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1486e-05 2024-09-17T21:21:58.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431984+0000", 2024-09-17T21:21:58.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1075999999999999e-05 2024-09-17T21:21:58.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431999+0000", 2024-09-17T21:21:58.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.503e-05 2024-09-17T21:21:58.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432095+0000", 2024-09-17T21:21:58.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5698000000000004e-05 2024-09-17T21:21:58.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:58.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432109+0000", 2024-09-17T21:21:58.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4188e-05 2024-09-17T21:21:58.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.309642+0000", 2024-09-17T21:21:58.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87753353000000001 2024-09-17T21:21:58.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310104+0000", 2024-09-17T21:21:58.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046103600000000001 2024-09-17T21:21:58.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:58.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310630+0000", 2024-09-17T21:21:58.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052682800000000002 2024-09-17T21:21:58.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:58.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313100+0000", 2024-09-17T21:21:58.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024693369999999998 2024-09-17T21:21:58.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:58.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313935+0000", 2024-09-17T21:21:58.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083540100000000005 2024-09-17T21:21:58.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:58.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314086+0000", 2024-09-17T21:21:58.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015053399999999999 2024-09-17T21:21:58.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314396+0000", 2024-09-17T21:21:58.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031065599999999997 2024-09-17T21:21:58.904 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.905 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.905 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.905 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:800 3.19 3:9dc17f12:::rbd_data.1152eb799094.000000000000000e:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:58.905 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:58.905 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 102.45089484899999, 2024-09-17T21:21:58.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88154663799999999, 2024-09-17T21:21:58.905 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.905 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.905 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:58.905 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.905 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:58.905 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:58.906 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 800 2024-09-17T21:21:58.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.906 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:58.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:21:58.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432187+0000", 2024-09-17T21:21:58.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4100000000000001e-06 2024-09-17T21:21:58.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432194+0000", 2024-09-17T21:21:58.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0389999999999999e-06 2024-09-17T21:21:58.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432216+0000", 2024-09-17T21:21:58.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1614e-05 2024-09-17T21:21:58.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432232+0000", 2024-09-17T21:21:58.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5739e-05 2024-09-17T21:21:58.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432405+0000", 2024-09-17T21:21:58.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017355300000000001 2024-09-17T21:21:58.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:58.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432432+0000", 2024-09-17T21:21:58.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6979e-05 2024-09-17T21:21:58.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.307767+0000", 2024-09-17T21:21:58.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87533472899999998 2024-09-17T21:21:58.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308192+0000", 2024-09-17T21:21:58.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042485999999999999 2024-09-17T21:21:58.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:58.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308727+0000", 2024-09-17T21:21:58.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00053550400000000001 2024-09-17T21:21:58.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:58.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310347+0000", 2024-09-17T21:21:58.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016198670000000001 2024-09-17T21:21:58.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:58.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313424+0000", 2024-09-17T21:21:58.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030764640000000001 2024-09-17T21:21:58.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:58.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313510+0000", 2024-09-17T21:21:58.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5938000000000003e-05 2024-09-17T21:21:58.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313733+0000", 2024-09-17T21:21:58.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022294200000000001 2024-09-17T21:21:58.911 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.911 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.911 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.911 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:805 3.19 3:9cab2ed6:::rbd_data.1152eb799094.0000000000000013:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:21:58.911 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:58.911 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 102.45069083, 2024-09-17T21:21:58.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88274355699999996, 2024-09-17T21:21:58.911 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.911 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.911 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:58.912 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.912 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:21:58.912 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:21:58.912 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 805 2024-09-17T21:21:58.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.912 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:58.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:21:58.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432391+0000", 2024-09-17T21:21:58.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3e-06 2024-09-17T21:21:58.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432398+0000", 2024-09-17T21:21:58.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4489999999999996e-06 2024-09-17T21:21:58.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432415+0000", 2024-09-17T21:21:58.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7750000000000001e-05 2024-09-17T21:21:58.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432430+0000", 2024-09-17T21:21:58.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.449e-05 2024-09-17T21:21:58.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432635+0000", 2024-09-17T21:21:58.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205031 2024-09-17T21:21:58.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:58.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432657+0000", 2024-09-17T21:21:58.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2064000000000001e-05 2024-09-17T21:21:58.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310444+0000", 2024-09-17T21:21:58.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87778699199999999 2024-09-17T21:21:58.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310849+0000", 2024-09-17T21:21:58.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00040497800000000001 2024-09-17T21:21:58.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:21:58.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.311396+0000", 2024-09-17T21:21:58.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00054729099999999999 2024-09-17T21:21:58.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:58.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313847+0000", 2024-09-17T21:21:58.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024504940000000001 2024-09-17T21:21:58.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:21:58.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314879+0000", 2024-09-17T21:21:58.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001031817 2024-09-17T21:21:58.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:21:58.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314964+0000", 2024-09-17T21:21:58.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5685000000000004e-05 2024-09-17T21:21:58.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.315133+0000", 2024-09-17T21:21:58.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000169216 2024-09-17T21:21:58.917 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.917 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.917 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.917 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12089 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 60416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:58.918 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:58.918 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.113465797999993, 2024-09-17T21:21:58.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088293233999999998, 2024-09-17T21:21:58.918 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.918 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.918 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.918 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.918 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:58.918 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:58.918 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12089 2024-09-17T21:21:58.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.918 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:58.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:21:58.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769616+0000", 2024-09-17T21:21:58.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:21:58.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769620+0000", 2024-09-17T21:21:58.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5140000000000002e-06 2024-09-17T21:21:58.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769633+0000", 2024-09-17T21:21:58.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2849e-05 2024-09-17T21:21:58.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769644+0000", 2024-09-17T21:21:58.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1471000000000001e-05 2024-09-17T21:21:58.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857564+0000", 2024-09-17T21:21:58.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087919445999999998 2024-09-17T21:21:58.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857716+0000", 2024-09-17T21:21:58.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000152065 2024-09-17T21:21:58.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857908+0000", 2024-09-17T21:21:58.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019252599999999999 2024-09-17T21:21:58.922 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.922 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.922 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.922 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12090 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 64512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:58.922 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:58.922 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.113419413000003, 2024-09-17T21:21:58.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088582672000000001, 2024-09-17T21:21:58.922 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.923 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.923 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.923 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.923 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:58.923 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:58.923 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12090 2024-09-17T21:21:58.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.923 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:58.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:21:58.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769662+0000", 2024-09-17T21:21:58.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:21:58.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769683+0000", 2024-09-17T21:21:58.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0979e-05 2024-09-17T21:21:58.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769699+0000", 2024-09-17T21:21:58.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5446000000000001e-05 2024-09-17T21:21:58.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769708+0000", 2024-09-17T21:21:58.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1679999999999993e-06 2024-09-17T21:21:58.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857976+0000", 2024-09-17T21:21:58.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088267795999999996 2024-09-17T21:21:58.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858095+0000", 2024-09-17T21:21:58.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119147 2024-09-17T21:21:58.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858244+0000", 2024-09-17T21:21:58.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014917000000000001 2024-09-17T21:21:58.926 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.926 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.926 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.927 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12091 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 68608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:58.927 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:58.927 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.113348028999994, 2024-09-17T21:21:58.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088832752000000001, 2024-09-17T21:21:58.927 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.927 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.927 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.927 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.927 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:58.927 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:58.927 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12091 2024-09-17T21:21:58.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.927 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:58.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:21:58.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769734+0000", 2024-09-17T21:21:58.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:21:58.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769739+0000", 2024-09-17T21:21:58.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8080000000000004e-06 2024-09-17T21:21:58.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769764+0000", 2024-09-17T21:21:58.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4839999999999999e-05 2024-09-17T21:21:58.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769777+0000", 2024-09-17T21:21:58.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3037000000000001e-05 2024-09-17T21:21:58.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858295+0000", 2024-09-17T21:21:58.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088518475999999999 2024-09-17T21:21:58.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858419+0000", 2024-09-17T21:21:58.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012378700000000001 2024-09-17T21:21:58.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858565+0000", 2024-09-17T21:21:58.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014630099999999999 2024-09-17T21:21:58.931 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.931 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.931 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.931 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12092 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:58.931 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:58.931 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.113265705000003, 2024-09-17T21:21:58.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089191555000000006, 2024-09-17T21:21:58.931 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.932 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.932 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.932 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.932 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:58.932 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:58.932 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12092 2024-09-17T21:21:58.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.932 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:58.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:21:58.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769816+0000", 2024-09-17T21:21:58.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7199999999999997e-07 2024-09-17T21:21:58.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769822+0000", 2024-09-17T21:21:58.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:21:58.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769838+0000", 2024-09-17T21:21:58.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6674e-05 2024-09-17T21:21:58.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769851+0000", 2024-09-17T21:21:58.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2527999999999999e-05 2024-09-17T21:21:58.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858660+0000", 2024-09-17T21:21:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088808556999999996 2024-09-17T21:21:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858817+0000", 2024-09-17T21:21:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015730400000000001 2024-09-17T21:21:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859007+0000", 2024-09-17T21:21:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018974899999999999 2024-09-17T21:21:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12093 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.113157600999997, 2024-09-17T21:21:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089429710999999995, 2024-09-17T21:21:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12093 2024-09-17T21:21:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:21:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769924+0000", 2024-09-17T21:21:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.342e-06 2024-09-17T21:21:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769929+0000", 2024-09-17T21:21:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0350000000000001e-06 2024-09-17T21:21:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769946+0000", 2024-09-17T21:21:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6793999999999999e-05 2024-09-17T21:21:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769959+0000", 2024-09-17T21:21:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3062e-05 2024-09-17T21:21:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859075+0000", 2024-09-17T21:21:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089115696999999994 2024-09-17T21:21:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859202+0000", 2024-09-17T21:21:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012657 2024-09-17T21:21:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859353+0000", 2024-09-17T21:21:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151211 2024-09-17T21:21:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12094 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.113103425000006, 2024-09-17T21:21:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089695001999999996, 2024-09-17T21:21:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12094 2024-09-17T21:21:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:21:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769978+0000", 2024-09-17T21:21:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.142e-06 2024-09-17T21:21:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769983+0000", 2024-09-17T21:21:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.386e-06 2024-09-17T21:21:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769997+0000", 2024-09-17T21:21:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.416e-05 2024-09-17T21:21:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770012+0000", 2024-09-17T21:21:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5384999999999999e-05 2024-09-17T21:21:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859405+0000", 2024-09-17T21:21:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089392376999999995 2024-09-17T21:21:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859525+0000", 2024-09-17T21:21:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120555 2024-09-17T21:21:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859672+0000", 2024-09-17T21:21:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000146997 2024-09-17T21:21:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12095 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.113050938000001, 2024-09-17T21:21:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090073476, 2024-09-17T21:21:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12095 2024-09-17T21:21:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:21:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770031+0000", 2024-09-17T21:21:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1699999999999997e-07 2024-09-17T21:21:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770036+0000", 2024-09-17T21:21:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.023e-06 2024-09-17T21:21:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770051+0000", 2024-09-17T21:21:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4953999999999999e-05 2024-09-17T21:21:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770063+0000", 2024-09-17T21:21:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:21:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859756+0000", 2024-09-17T21:21:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089692980000000005 2024-09-17T21:21:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859891+0000", 2024-09-17T21:21:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013459499999999999 2024-09-17T21:21:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860103+0000", 2024-09-17T21:21:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021274400000000001 2024-09-17T21:21:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12096 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.113001252999993, 2024-09-17T21:21:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090358274000000002, 2024-09-17T21:21:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12096 2024-09-17T21:21:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:21:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770080+0000", 2024-09-17T21:21:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9400000000000004e-07 2024-09-17T21:21:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770085+0000", 2024-09-17T21:21:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2169999999999996e-06 2024-09-17T21:21:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770100+0000", 2024-09-17T21:21:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5077e-05 2024-09-17T21:21:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770112+0000", 2024-09-17T21:21:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1951e-05 2024-09-17T21:21:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860160+0000", 2024-09-17T21:21:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090048413999999993 2024-09-17T21:21:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860285+0000", 2024-09-17T21:21:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012466699999999999 2024-09-17T21:21:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860438+0000", 2024-09-17T21:21:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015305399999999999 2024-09-17T21:21:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12097 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.112953505999997, 2024-09-17T21:21:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090702375000000002, 2024-09-17T21:21:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12097 2024-09-17T21:21:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:21:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770128+0000", 2024-09-17T21:21:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.04e-06 2024-09-17T21:21:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770133+0000", 2024-09-17T21:21:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8999999999999997e-06 2024-09-17T21:21:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770152+0000", 2024-09-17T21:21:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8890999999999999e-05 2024-09-17T21:21:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770164+0000", 2024-09-17T21:21:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1769e-05 2024-09-17T21:21:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860511+0000", 2024-09-17T21:21:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090347527999999996 2024-09-17T21:21:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860652+0000", 2024-09-17T21:21:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140894 2024-09-17T21:21:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860830+0000", 2024-09-17T21:21:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017735299999999999 2024-09-17T21:21:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12098 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.112900038000006, 2024-09-17T21:21:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090997262999999995, 2024-09-17T21:21:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12098 2024-09-17T21:21:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:21:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775465+0000", 2024-09-17T21:21:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0052843350000000002 2024-09-17T21:21:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775476+0000", 2024-09-17T21:21:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0797e-05 2024-09-17T21:21:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775506+0000", 2024-09-17T21:21:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9709999999999998e-05 2024-09-17T21:21:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775530+0000", 2024-09-17T21:21:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4528999999999998e-05 2024-09-17T21:21:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860894+0000", 2024-09-17T21:21:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.085364195000000004 2024-09-17T21:21:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861019+0000", 2024-09-17T21:21:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012467399999999999 2024-09-17T21:21:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861178+0000", 2024-09-17T21:21:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159023 2024-09-17T21:21:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12104 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.107198929000006, 2024-09-17T21:21:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087969628999999994, 2024-09-17T21:21:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12104 2024-09-17T21:21:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:21:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775883+0000", 2024-09-17T21:21:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3399999999999998e-07 2024-09-17T21:21:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775886+0000", 2024-09-17T21:21:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2119999999999999e-06 2024-09-17T21:21:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775895+0000", 2024-09-17T21:21:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8710000000000003e-06 2024-09-17T21:21:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775901+0000", 2024-09-17T21:21:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:21:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863544+0000", 2024-09-17T21:21:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087642642000000007 2024-09-17T21:21:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863694+0000", 2024-09-17T21:21:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014952300000000001 2024-09-17T21:21:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863851+0000", 2024-09-17T21:21:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000157804 2024-09-17T21:21:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12105 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.107170034999996, 2024-09-17T21:21:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088330400000000003, 2024-09-17T21:21:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12105 2024-09-17T21:21:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:21:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9999999999999997e-07 2024-09-17T21:21:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775914+0000", 2024-09-17T21:21:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4710000000000001e-06 2024-09-17T21:21:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775921+0000", 2024-09-17T21:21:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:21:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775928+0000", 2024-09-17T21:21:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:21:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863923+0000", 2024-09-17T21:21:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087995126000000007 2024-09-17T21:21:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864059+0000", 2024-09-17T21:21:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135797 2024-09-17T21:21:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864241+0000", 2024-09-17T21:21:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018252599999999999 2024-09-17T21:21:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12106 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:21:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 97.107080857, 2024-09-17T21:21:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088586394999999998, 2024-09-17T21:21:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:21:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:21:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:21:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12106 2024-09-17T21:21:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:21:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776001+0000", 2024-09-17T21:21:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4300000000000001e-07 2024-09-17T21:21:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776004+0000", 2024-09-17T21:21:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8049999999999998e-06 2024-09-17T21:21:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776018+0000", 2024-09-17T21:21:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4613999999999999e-05 2024-09-17T21:21:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776025+0000", 2024-09-17T21:21:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:21:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864279+0000", 2024-09-17T21:21:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088253570000000003 2024-09-17T21:21:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:21:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864433+0000", 2024-09-17T21:21:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015469400000000001 2024-09-17T21:21:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864586+0000", 2024-09-17T21:21:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000153003 2024-09-17T21:21:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25033 3.6 3.b74753e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e168)", 2024-09-17T21:21:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:21:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 33.373637592999998, 2024-09-17T21:21:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93781952099999999, 2024-09-17T21:21:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:21:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25033 2024-09-17T21:21:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:21:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:21:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509444+0000", 2024-09-17T21:21:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0020000000000001e-06 2024-09-17T21:21:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509450+0000", 2024-09-17T21:21:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4249999999999997e-06 2024-09-17T21:21:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509460+0000", 2024-09-17T21:21:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0183999999999999e-05 2024-09-17T21:21:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509466+0000", 2024-09-17T21:21:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0039999999999996e-06 2024-09-17T21:21:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509540+0000", 2024-09-17T21:21:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3847e-05 2024-09-17T21:21:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509551+0000", 2024-09-17T21:21:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1731e-05 2024-09-17T21:21:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.447243+0000", 2024-09-17T21:21:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93769166000000004 2024-09-17T21:21:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.447263+0000", 2024-09-17T21:21:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9667999999999999e-05 2024-09-17T21:21:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25033 3.6 3:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e168)", 2024-09-17T21:21:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:21:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 33.372111996000001, 2024-09-17T21:21:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.937309383, 2024-09-17T21:21:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:21:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:21:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:21:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:21:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:21:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25033 2024-09-17T21:21:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:21:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:21:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:21:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:21:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:21:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:21:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:21:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510970+0000", 2024-09-17T21:21:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.564e-06 2024-09-17T21:21:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:21:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510978+0000", 2024-09-17T21:21:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.362e-06 2024-09-17T21:21:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:21:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510998+0000", 2024-09-17T21:21:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0336999999999999e-05 2024-09-17T21:21:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:21:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511010+0000", 2024-09-17T21:21:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1786e-05 2024-09-17T21:21:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511080+0000", 2024-09-17T21:21:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0686999999999996e-05 2024-09-17T21:21:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:21:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511087+0000", 2024-09-17T21:21:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4899999999999997e-06 2024-09-17T21:21:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:21:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.448050+0000", 2024-09-17T21:21:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93696351700000002 2024-09-17T21:21:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:21:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:21:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:21:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.448278+0000", 2024-09-17T21:21:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022764000000000001 2024-09-17T21:21:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:21:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:21:58.983 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:21:58.985 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:21:58.985 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:58.983+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:59.085 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:21:59.081+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:21:59.115 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:59.115 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:59.115 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:21:59.115 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:59.129 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:21:59.187 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:59.183+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:59.287 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:59.283+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:21:59.293 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:59.293 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:21:59.293 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:21:59.293 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:21:59.293 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:59.307 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:21:59.339 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:21:59.388 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:21:59.383+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:21:59.429 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:59.429 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:21:59.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:21:59.429 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:59.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.429 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:21:59.429 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:59.429 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 242.410727232, 2024-09-17T21:21:59.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:21:59.430 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:59.430 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:59.430 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:59.430 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:59.430 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:21:59.430 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:21:59.430 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:21:59.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.430 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:59.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:59.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:59.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:59.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:21:59.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:59.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:21:59.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:21:59.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:59.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:21:59.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:21:59.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:59.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:21:59.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:21:59.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:59.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:21:59.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:21:59.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:21:59.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:21:59.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:59.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:21:59.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:21:59.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:21:59.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:21:59.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:59.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:21:59.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:21:59.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:21:59.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:21:59.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:21:59.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:59.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:21:59.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:21:59.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:59.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:21:59.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:21:59.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:59.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:21:59.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:21:59.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:59.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:21:59.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:21:59.436 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.436 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:59.436 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.436 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:59.436 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:59.436 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 222.83056295200001, 2024-09-17T21:21:59.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:21:59.437 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:59.437 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:59.437 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:59.437 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:59.437 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:21:59.437 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:21:59.437 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:21:59.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.437 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:59.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:59.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:59.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:59.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:21:59.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:59.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:21:59.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:21:59.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:59.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:21:59.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:21:59.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:59.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:21:59.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:21:59.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:59.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:21:59.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:21:59.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:21:59.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:21:59.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:59.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:21:59.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:21:59.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:21:59.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:21:59.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:59.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:21:59.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:21:59.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:21:59.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:21:59.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:59.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:21:59.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:21:59.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:59.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:21:59.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:21:59.442 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.442 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:59.442 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.443 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:59.443 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:59.443 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 222.83046564, 2024-09-17T21:21:59.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:21:59.443 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:59.443 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:59.443 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:59.443 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:59.443 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:59.443 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:59.443 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:21:59.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.444 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:59.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:59.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:59.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:59.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:21:59.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:59.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:21:59.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:21:59.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:59.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:21:59.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:21:59.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:59.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:21:59.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:21:59.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:59.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:21:59.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:21:59.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:21:59.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:21:59.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:59.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:21:59.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:21:59.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:21:59.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:21:59.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:59.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:21:59.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:21:59.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:21:59.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:21:59.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:21:59.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:21:59.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:21:59.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:21:59.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:21:59.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:59.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:21:59.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:21:59.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:59.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:21:59.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:21:59.449 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.449 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:59.449 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.449 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:21:59.449 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:59.450 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 222.82837883900001, 2024-09-17T21:21:59.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:21:59.450 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:59.450 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:59.450 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:59.450 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:59.450 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:21:59.450 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:21:59.450 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:21:59.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.450 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:59.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:59.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:59.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:59.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:21:59.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:59.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:21:59.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:21:59.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:59.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:21:59.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:21:59.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:59.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:21:59.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:21:59.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:59.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:21:59.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:21:59.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:21:59.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:21:59.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:21:59.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:21:59.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:21:59.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:21:59.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:21:59.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:59.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:21:59.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:21:59.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:21:59.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:21:59.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:59.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:21:59.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:21:59.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:21:59.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:21:59.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:21:59.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:59.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:21:59.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:21:59.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:59.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:21:59.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:21:59.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:59.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:21:59.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:21:59.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:59.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:21:59.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:21:59.456 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.456 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:59.457 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.457 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:59.457 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:59.457 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 190.144480058, 2024-09-17T21:21:59.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:21:59.457 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:59.457 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:59.457 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:59.457 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:59.457 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:59.457 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:59.458 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:21:59.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.458 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:59.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:59.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:59.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:59.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:21:59.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:59.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:21:59.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:21:59.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:59.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:21:59.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:21:59.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:59.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:21:59.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:21:59.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:59.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:21:59.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:21:59.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:21:59.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:21:59.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:21:59.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:21:59.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:21:59.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:21:59.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:21:59.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:59.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:21:59.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:21:59.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:21:59.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:21:59.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:59.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:21:59.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:21:59.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:59.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:21:59.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:21:59.462 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.462 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:59.462 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.463 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:59.463 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:59.463 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 190.135388347, 2024-09-17T21:21:59.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:21:59.463 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:59.463 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:59.463 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:59.463 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:59.463 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:59.463 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:59.463 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:21:59.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.464 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:59.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:59.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:59.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:59.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:21:59.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:59.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:21:59.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:21:59.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:59.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:21:59.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:21:59.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:59.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:21:59.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:21:59.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:59.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:21:59.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:21:59.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:21:59.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:21:59.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:59.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:21:59.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:21:59.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:21:59.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:21:59.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:59.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:21:59.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:21:59.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:59.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:21:59.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:21:59.467 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.467 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:59.467 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.468 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:59.468 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:59.468 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 190.13521629600001, 2024-09-17T21:21:59.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:21:59.468 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:59.468 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:59.468 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:59.468 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:59.468 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:21:59.468 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:21:59.469 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:21:59.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.469 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:59.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:59.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:59.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:59.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:21:59.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:59.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:21:59.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:21:59.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:59.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:21:59.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:21:59.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:59.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:21:59.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:21:59.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:59.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:21:59.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:21:59.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:21:59.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:21:59.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:59.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:21:59.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:21:59.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:21:59.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:21:59.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:59.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:21:59.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:21:59.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:59.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:21:59.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:21:59.473 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.473 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:59.473 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.473 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:59.473 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:59.473 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 189.960283549, 2024-09-17T21:21:59.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:21:59.474 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:59.474 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:59.474 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:59.474 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:59.474 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:59.474 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:59.474 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:21:59.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.474 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:59.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:59.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:59.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:59.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:21:59.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:59.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:21:59.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:21:59.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:59.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:21:59.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:21:59.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:59.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:21:59.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:21:59.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:59.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:21:59.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:21:59.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:21:59.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:21:59.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:59.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:21:59.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:21:59.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:21:59.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:21:59.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:59.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:21:59.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:21:59.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:59.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:21:59.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:21:59.478 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.478 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:59.479 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.479 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:59.479 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:59.479 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 189.96019899199999, 2024-09-17T21:21:59.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:21:59.479 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:59.479 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:59.479 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:59.479 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:59.479 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:59.479 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:59.479 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:21:59.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.480 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:59.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:59.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:59.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:59.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:59.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:59.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:21:59.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:21:59.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:59.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:21:59.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:21:59.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:59.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:21:59.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:21:59.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:59.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:21:59.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:21:59.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:21:59.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:21:59.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:59.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:21:59.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:21:59.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:21:59.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:21:59.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:59.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:21:59.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:21:59.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:59.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:21:59.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:21:59.483 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.484 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:59.484 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.484 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:21:59.484 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:59.484 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 189.96013335500001, 2024-09-17T21:21:59.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:21:59.484 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:59.484 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:59.484 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:59.484 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:59.484 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:59.485 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:59.485 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:21:59.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.485 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:59.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:59.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:59.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:59.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:21:59.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:59.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:21:59.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:21:59.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:59.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:21:59.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:21:59.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:59.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:21:59.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:21:59.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:59.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:21:59.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:21:59.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:21:59.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:21:59.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:59.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:21:59.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:21:59.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:21:59.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:21:59.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:59.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:21:59.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:21:59.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:59.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:21:59.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:21:59.489 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.489 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:59.489 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.489 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:59.489 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:59.489 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 138.586558258, 2024-09-17T21:21:59.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:21:59.489 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:59.489 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:59.489 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:59.489 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:59.489 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:21:59.489 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:59.490 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:21:59.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.490 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:59.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:59.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:59.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:59.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:21:59.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:59.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:21:59.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:21:59.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:59.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:21:59.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:21:59.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:59.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:21:59.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:21:59.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:59.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:21:59.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:21:59.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:21:59.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:21:59.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:59.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:21:59.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:21:59.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:59.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:21:59.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:21:59.493 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.493 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:59.493 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.493 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:21:59.493 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:59.493 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 138.586438746, 2024-09-17T21:21:59.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:21:59.493 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:59.493 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:59.493 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:59.494 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:59.494 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:21:59.494 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:21:59.494 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:21:59.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.494 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:59.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:59.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:59.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:59.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:21:59.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:59.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:21:59.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:21:59.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:59.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:21:59.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:21:59.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:59.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:21:59.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:21:59.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:59.496 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:21:59.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:21:59.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.496 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:21:59.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:21:59.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:59.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:21:59.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:21:59.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:59.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:21:59.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:21:59.497 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.497 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:59.497 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.497 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:996 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:59.497 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:59.498 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 93.993325374999998, 2024-09-17T21:21:59.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084493736999999999, 2024-09-17T21:21:59.498 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:59.498 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:59.498 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:59.498 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:59.498 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:59.498 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:59.498 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 996 2024-09-17T21:21:59.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.498 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:59.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:59.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:59.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:59.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:21:59.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:59.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431168+0000", 2024-09-17T21:21:59.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4e-07 2024-09-17T21:21:59.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:59.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431172+0000", 2024-09-17T21:21:59.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.608e-06 2024-09-17T21:21:59.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:59.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431180+0000", 2024-09-17T21:21:59.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5650000000000002e-06 2024-09-17T21:21:59.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:59.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431188+0000", 2024-09-17T21:21:59.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:21:59.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515354+0000", 2024-09-17T21:21:59.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084165977000000003 2024-09-17T21:21:59.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:59.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515451+0000", 2024-09-17T21:21:59.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6669000000000006e-05 2024-09-17T21:21:59.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:59.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515661+0000", 2024-09-17T21:21:59.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021021599999999999 2024-09-17T21:21:59.501 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.501 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:59.501 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.502 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:997 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 583680~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:59.502 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:59.502 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 93.993293980000004, 2024-09-17T21:21:59.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084772419000000002, 2024-09-17T21:21:59.502 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:59.502 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:59.502 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:59.502 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:59.502 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:59.502 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:59.502 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 997 2024-09-17T21:21:59.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.502 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:59.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:59.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:59.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:59.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:59.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:59.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:21:59.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0400000000000005e-07 2024-09-17T21:21:59.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:59.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431202+0000", 2024-09-17T21:21:59.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:21:59.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:59.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431208+0000", 2024-09-17T21:21:59.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4740000000000002e-06 2024-09-17T21:21:59.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:59.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431214+0000", 2024-09-17T21:21:59.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6550000000000001e-06 2024-09-17T21:21:59.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515741+0000", 2024-09-17T21:21:59.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084526904 2024-09-17T21:21:59.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:59.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515826+0000", 2024-09-17T21:21:59.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4643999999999998e-05 2024-09-17T21:21:59.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:59.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515971+0000", 2024-09-17T21:21:59.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145444 2024-09-17T21:21:59.506 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.506 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:59.506 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.506 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:998 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 587776~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:59.506 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:59.506 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 93.993271007999994, 2024-09-17T21:21:59.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085108565999999997, 2024-09-17T21:21:59.506 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:59.506 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:59.506 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:59.506 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:59.506 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:59.507 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:59.507 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 998 2024-09-17T21:21:59.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.507 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:59.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:59.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:59.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:59.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:59.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:59.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:21:59.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:21:59.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:59.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431224+0000", 2024-09-17T21:21:59.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:21:59.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:59.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431231+0000", 2024-09-17T21:21:59.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000002e-06 2024-09-17T21:21:59.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:59.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431236+0000", 2024-09-17T21:21:59.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.648e-06 2024-09-17T21:21:59.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516024+0000", 2024-09-17T21:21:59.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084787584999999999 2024-09-17T21:21:59.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:59.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516122+0000", 2024-09-17T21:21:59.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8076000000000001e-05 2024-09-17T21:21:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516330+0000", 2024-09-17T21:21:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020808 2024-09-17T21:21:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1000 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 591872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 93.993248893000001, 2024-09-17T21:21:59.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085389065, 2024-09-17T21:21:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1000 2024-09-17T21:21:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:21:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:21:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:59.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431246+0000", 2024-09-17T21:21:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1749999999999999e-06 2024-09-17T21:21:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431253+0000", 2024-09-17T21:21:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2790000000000004e-06 2024-09-17T21:21:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431258+0000", 2024-09-17T21:21:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3299999999999998e-06 2024-09-17T21:21:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516384+0000", 2024-09-17T21:21:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085125549999999994 2024-09-17T21:21:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516460+0000", 2024-09-17T21:21:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6359999999999994e-05 2024-09-17T21:21:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516633+0000", 2024-09-17T21:21:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017285900000000001 2024-09-17T21:21:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:59.514 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1001 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 595968~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:21:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 93.993225487000004, 2024-09-17T21:21:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085614715999999993, 2024-09-17T21:21:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:21:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:21:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1001 2024-09-17T21:21:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:21:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:59.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438066+0000", 2024-09-17T21:21:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067989490000000003 2024-09-17T21:21:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438073+0000", 2024-09-17T21:21:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8419999999999999e-06 2024-09-17T21:21:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438090+0000", 2024-09-17T21:21:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6657e-05 2024-09-17T21:21:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438099+0000", 2024-09-17T21:21:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0529999999999996e-06 2024-09-17T21:21:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516678+0000", 2024-09-17T21:21:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078579422999999995 2024-09-17T21:21:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516739+0000", 2024-09-17T21:21:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1315000000000005e-05 2024-09-17T21:21:59.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516882+0000", 2024-09-17T21:21:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014247700000000001 2024-09-17T21:21:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:19998 3.5 3:a4885450:::rbd_object_map.1152d139f9f9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-17T21:21:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.021958241999997, 2024-09-17T21:21:59.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90715489299999996, 2024-09-17T21:21:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19998 2024-09-17T21:21:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:21:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402537+0000", 2024-09-17T21:21:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5600000000000001e-06 2024-09-17T21:21:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402549+0000", 2024-09-17T21:21:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1929999999999999e-05 2024-09-17T21:21:59.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402579+0000", 2024-09-17T21:21:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0093000000000001e-05 2024-09-17T21:21:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402595+0000", 2024-09-17T21:21:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5855999999999999e-05 2024-09-17T21:21:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402731+0000", 2024-09-17T21:21:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013610799999999999 2024-09-17T21:21:59.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402750+0000", 2024-09-17T21:21:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8737e-05 2024-09-17T21:21:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.305977+0000", 2024-09-17T21:21:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90322768399999998 2024-09-17T21:21:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306236+0000", 2024-09-17T21:21:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000258111 2024-09-17T21:21:59.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:21:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306557+0000", 2024-09-17T21:21:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032159799999999998 2024-09-17T21:21:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.308286+0000", 2024-09-17T21:21:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017292150000000001 2024-09-17T21:21:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:59.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309530+0000", 2024-09-17T21:21:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00124409 2024-09-17T21:21:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309630+0000", 2024-09-17T21:21:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9689e-05 2024-09-17T21:21:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309689+0000", 2024-09-17T21:21:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9222000000000003e-05 2024-09-17T21:21:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:59.525 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:25277 3.1f 3:ff5d7cd5:::rbd_header.11524f65d4ff:head [watch ping cookie 139860845478480 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-17T21:21:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:21:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 29.839002173000001, 2024-09-17T21:21:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.92185720000000004, 2024-09-17T21:21:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:21:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 25277 2024-09-17T21:21:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:21:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:21:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585493+0000", 2024-09-17T21:21:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:21:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585505+0000", 2024-09-17T21:21:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2466000000000001e-05 2024-09-17T21:21:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585537+0000", 2024-09-17T21:21:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1183000000000003e-05 2024-09-17T21:21:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585553+0000", 2024-09-17T21:21:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.662e-05 2024-09-17T21:21:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585696+0000", 2024-09-17T21:21:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014293800000000001 2024-09-17T21:21:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585718+0000", 2024-09-17T21:21:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1860000000000001e-05 2024-09-17T21:21:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.501188+0000", 2024-09-17T21:21:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.91547051300000004 2024-09-17T21:21:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:21:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.501260+0000", 2024-09-17T21:21:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1571999999999998e-05 2024-09-17T21:21:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507002+0000", 2024-09-17T21:21:59.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057414850000000002 2024-09-17T21:21:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507143+0000", 2024-09-17T21:21:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014140299999999999 2024-09-17T21:21:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507348+0000", 2024-09-17T21:21:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000204773 2024-09-17T21:21:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:26042 3.f 3:f24134c1:::rbd_id.image_client.0-clone34:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e186)", 2024-09-17T21:21:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:21:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.938313805, 2024-09-17T21:21:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.91185481000000002, 2024-09-17T21:21:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:21:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 26042 2024-09-17T21:21:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.532 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:21:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:21:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486180+0000", 2024-09-17T21:21:59.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5990000000000001e-06 2024-09-17T21:21:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486187+0000", 2024-09-17T21:21:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7859999999999999e-06 2024-09-17T21:21:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486206+0000", 2024-09-17T21:21:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8641e-05 2024-09-17T21:21:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486219+0000", 2024-09-17T21:21:59.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.308e-05 2024-09-17T21:21:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486283+0000", 2024-09-17T21:21:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4084000000000005e-05 2024-09-17T21:21:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:21:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486290+0000", 2024-09-17T21:21:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.584e-06 2024-09-17T21:21:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392182+0000", 2024-09-17T21:21:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90589206499999997 2024-09-17T21:21:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392518+0000", 2024-09-17T21:21:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033596 2024-09-17T21:21:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:21:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392802+0000", 2024-09-17T21:21:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028394999999999999 2024-09-17T21:21:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:21:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397705+0000", 2024-09-17T21:21:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0049031769999999999 2024-09-17T21:21:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:21:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397770+0000", 2024-09-17T21:21:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4765999999999996e-05 2024-09-17T21:21:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:21:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397927+0000", 2024-09-17T21:21:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015699900000000001 2024-09-17T21:21:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:21:59.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.398034+0000", 2024-09-17T21:21:59.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000107119 2024-09-17T21:21:59.538 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.538 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:59.538 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.538 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.538 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:59.538 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:59.538 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:21:59.540 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:21:59.541 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:59.537+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:59.641 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:21:59.637+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:21:59.687 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:21:59.687 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:21:59.706 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:21:59.706 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:21:59.706 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T21:21:59.721 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:21:59.721 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:21:59.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.721 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:28516 3.14 3:2e5a6d0f:::rbd_header.1152c6ce51b2:head [watch watch cookie 139860846403648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e202)", 2024-09-17T21:21:59.721 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:59.718888+0000", 2024-09-17T21:21:59.721 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.00080126500000000001, 2024-09-17T21:21:59.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00084629099999999999, 2024-09-17T21:21:59.722 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:21:59.722 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:21:59.722 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:21:59.722 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:21:59.722 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:21:59.722 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:21:59.722 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 28516 2024-09-17T21:21:59.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.722 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:21:59.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:21:59.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:59.718888+0000", 2024-09-17T21:21:59.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:21:59.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:59.718888+0000", 2024-09-17T21:21:59.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:21:59.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:21:59.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:59.718890+0000", 2024-09-17T21:21:59.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4559999999999999e-06 2024-09-17T21:21:59.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:21:59.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:59.718896+0000", 2024-09-17T21:21:59.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4819999999999999e-06 2024-09-17T21:21:59.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:21:59.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:59.718918+0000", 2024-09-17T21:21:59.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1290000000000001e-05 2024-09-17T21:21:59.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:21:59.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:59.718928+0000", 2024-09-17T21:21:59.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0674e-05 2024-09-17T21:21:59.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:21:59.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:59.719003+0000", 2024-09-17T21:21:59.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4779999999999999e-05 2024-09-17T21:21:59.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:21:59.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:59.719082+0000", 2024-09-17T21:21:59.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8996000000000001e-05 2024-09-17T21:21:59.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:21:59.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:21:59.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:21:59.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:59.719240+0000", 2024-09-17T21:21:59.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015749700000000001 2024-09-17T21:21:59.725 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.725 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:21:59.726 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.726 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:21:59.726 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-09-17T21:21:59.726 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 1 2024-09-17T21:21:59.726 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:21:59.726 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:21:59.726 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:21:59.736 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:21:59.746 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:21:59.746 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:21:59.767 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:21:59.767 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:21:59.771 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:21:59.771 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:21:59.779 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:59.777+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:59.842 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:21:59.837+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:21:59.899 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:21:59.899 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:21:59.914 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:21:59.943 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:21:59.939+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:00.044 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:00.043+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:00.080 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:00.080 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:00.094 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:22:00.145 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:00.143+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:00.219 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:00.219 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:00.233 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:22:00.279 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:00.277+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:00.347 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:00.343+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:00.357 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:00.357 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:00.371 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:22:00.448 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:00.447+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:00.525 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-17T21:22:00.549 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:00.549 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:00.549 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:00.545+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:00.565 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:22:00.651 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:00.649+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:00.724 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:00.725 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:00.741 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:22:00.788 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:00.787+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:00.853 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:00.853 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:00.853 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:00.849+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:00.865 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:22:00.957 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:00.955+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:01.029 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:01.029 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:01.043 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:22:01.055 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:01.053+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:01.156 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:01.151+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:01.205 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:01.205 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:01.220 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:22:01.257 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:01.253+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:01.358 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:01.353+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:01.386 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:01.386 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:01.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:22:01.459 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:01.455+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:01.518 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:01.518 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:01.535 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:22:01.576 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:01.575+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:01.660 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:01.655+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:01.683 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:01.683 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:01.699 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:22:01.760 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:01.757+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:01.773 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:22:01.819 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:01.820 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:01.834 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:22:01.861 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:01.859+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:01.926 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:22:01.926 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:22:01.942 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:22:01.943 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:22:01.943 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T21:22:01.962 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:01.959+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:01.963 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:22:01.963 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:22:01.981 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:22:01.981 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:22:01.998 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:01.998 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:22:01.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:22:01.998 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:22:01.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:01.998 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:22:01.999 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:22:01.999 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 244.98198221000001, 2024-09-17T21:22:01.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:22:01.999 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:01.999 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:01.999 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:01.999 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:01.999 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:22:01.999 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:22:01.999 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:22:01.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:01.999 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:01.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:01.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:01.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:22:02.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:22:02.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:22:02.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:22:02.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:22:02.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:22:02.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:22:02.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:22:02.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:22:02.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:22:02.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:22:02.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:22:02.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:02.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:22:02.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:22:02.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:22:02.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:22:02.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:22:02.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:22:02.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:22:02.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:22:02.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:22:02.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:02.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:22:02.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:22:02.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:02.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:22:02.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:22:02.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:02.004 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:22:02.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:22:02.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.004 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:22:02.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:22:02.004 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.004 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.004 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.005 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:22:02.005 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:22:02.005 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 225.40181792999999, 2024-09-17T21:22:02.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:22:02.005 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.005 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.005 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.005 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.005 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:22:02.005 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:22:02.005 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:22:02.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.005 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:22:02.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:22:02.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:22:02.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:22:02.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:22:02.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:22:02.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:22:02.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:22:02.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:22:02.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:22:02.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:22:02.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:22:02.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:22:02.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:22:02.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:22:02.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:22:02.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:22:02.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:02.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:22:02.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:22:02.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:22:02.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:22:02.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:22:02.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:22:02.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:22:02.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:22:02.010 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.010 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.010 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.010 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:22:02.010 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:22:02.010 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 225.40172061800001, 2024-09-17T21:22:02.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:22:02.010 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.010 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.011 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.011 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.011 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:02.011 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:02.011 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:22:02.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.011 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:22:02.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:22:02.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:22:02.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:22:02.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:22:02.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:22:02.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:22:02.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:22:02.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:22:02.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:22:02.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:22:02.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:22:02.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:22:02.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:22:02.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:22:02.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:22:02.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:22:02.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:02.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:22:02.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:22:02.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:22:02.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:22:02.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:22:02.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:22:02.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:22:02.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:22:02.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:22:02.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:22:02.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:22:02.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:22:02.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:22:02.016 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.016 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.016 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.016 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:22:02.016 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:22:02.017 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 225.39963381699999, 2024-09-17T21:22:02.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:22:02.017 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.017 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.017 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:02.017 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.017 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:22:02.017 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:22:02.017 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:22:02.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.017 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:22:02.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:22:02.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:22:02.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:22:02.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:22:02.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:22:02.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:22:02.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:22:02.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:22:02.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:22:02.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:22:02.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:22:02.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:22:02.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:22:02.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:22:02.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:22:02.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:22:02.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:02.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:22:02.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:22:02.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:22:02.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:22:02.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:22:02.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:22:02.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:22:02.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:22:02.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:22:02.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:02.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:22:02.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:22:02.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:02.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:22:02.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:22:02.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:02.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:22:02.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:22:02.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:22:02.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:22:02.023 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.023 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.023 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.023 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:02.023 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:22:02.023 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 192.71573503600001, 2024-09-17T21:22:02.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:22:02.023 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.023 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.024 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.024 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.024 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:22:02.024 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:22:02.024 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:22:02.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.024 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:22:02.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:22:02.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:22:02.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:22:02.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:22:02.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:22:02.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:22:02.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:22:02.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:22:02.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:22:02.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:22:02.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:22:02.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:22:02.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:22:02.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:22:02.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:22:02.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:22:02.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:02.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:22:02.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:22:02.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:22:02.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:22:02.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:22:02.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:22:02.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:22:02.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:22:02.029 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.029 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.029 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.029 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:02.029 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:22:02.029 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 192.70664332499999, 2024-09-17T21:22:02.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:22:02.029 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.029 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.029 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.029 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.029 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:22:02.030 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:22:02.030 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:22:02.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.030 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:22:02.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:22:02.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:22:02.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:22:02.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:22:02.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:22:02.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:22:02.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:22:02.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:22:02.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:22:02.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:22:02.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:22:02.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:02.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:22:02.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:22:02.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:22:02.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:22:02.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:22:02.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:22:02.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:22:02.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:22:02.034 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.034 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.034 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.034 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:02.034 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:22:02.034 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 192.70647127399999, 2024-09-17T21:22:02.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:22:02.035 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.035 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.035 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.035 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.035 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:22:02.035 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:22:02.035 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:22:02.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.035 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:22:02.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:22:02.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:22:02.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:22:02.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:22:02.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:22:02.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:22:02.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:22:02.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:22:02.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:22:02.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:22:02.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:22:02.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:02.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:22:02.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:22:02.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:22:02.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:22:02.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:22:02.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:22:02.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:22:02.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:22:02.039 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.039 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.039 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.040 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:02.040 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:22:02.040 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 192.53153852700001, 2024-09-17T21:22:02.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:22:02.040 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.040 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.040 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.040 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.040 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:02.040 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:02.040 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:22:02.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.040 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:22:02.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:22:02.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:22:02.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:22:02.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:22:02.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:22:02.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:22:02.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:22:02.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:22:02.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:22:02.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:22:02.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:22:02.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:02.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:22:02.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:22:02.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:22:02.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:22:02.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:22:02.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:22:02.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:22:02.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:22:02.044 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.044 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.044 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.044 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:02.044 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:22:02.044 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 192.53145397, 2024-09-17T21:22:02.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:22:02.045 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.045 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.045 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.045 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.045 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:02.045 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:02.045 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:22:02.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.045 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:22:02.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:22:02.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:22:02.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:22:02.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:22:02.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:22:02.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:22:02.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:22:02.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:22:02.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:22:02.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:22:02.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:22:02.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:02.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:22:02.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:22:02.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:22:02.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:22:02.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:22:02.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:22:02.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:22:02.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:22:02.049 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.049 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.049 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.049 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:02.049 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:22:02.049 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 192.531388333, 2024-09-17T21:22:02.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:22:02.050 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.050 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.050 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.050 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.050 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:02.050 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:02.050 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:22:02.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.050 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:22:02.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.051 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:22:02.052 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:22:02.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:22:02.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:22:02.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:22:02.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:22:02.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:22:02.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:22:02.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:22:02.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:22:02.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:22:02.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:22:02.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:22:02.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:02.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:22:02.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:22:02.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:22:02.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:22:02.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:22:02.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:22:02.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:22:02.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:22:02.056 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.056 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.056 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.056 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:02.056 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:22:02.056 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 141.15781323600001, 2024-09-17T21:22:02.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:22:02.056 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.056 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.056 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.057 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.057 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:22:02.057 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:02.057 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:22:02.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.057 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:22:02.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:22:02.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:22:02.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:22:02.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:22:02.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:22:02.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:22:02.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:22:02.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:22:02.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:22:02.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:22:02.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:22:02.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:22:02.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:22:02.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:22:02.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:22:02.060 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.060 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.060 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.060 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:02.060 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:22:02.060 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 141.15769372400001, 2024-09-17T21:22:02.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:22:02.060 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.061 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.061 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.061 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.061 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:22:02.061 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:02.061 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:22:02.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.061 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:22:02.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:22:02.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:22:02.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:22:02.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:22:02.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:22:02.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:22:02.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:22:02.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:22:02.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:22:02.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:22:02.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:22:02.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:22:02.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:22:02.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:22:02.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:22:02.064 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.064 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.064 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.064 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:996 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:02.065 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:22:02.065 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 96.564580352999997, 2024-09-17T21:22:02.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084493736999999999, 2024-09-17T21:22:02.065 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.065 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.065 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.065 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.065 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:02.065 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:02.065 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 996 2024-09-17T21:22:02.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.065 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:22:02.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:22:02.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431168+0000", 2024-09-17T21:22:02.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4e-07 2024-09-17T21:22:02.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431172+0000", 2024-09-17T21:22:02.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.608e-06 2024-09-17T21:22:02.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431180+0000", 2024-09-17T21:22:02.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5650000000000002e-06 2024-09-17T21:22:02.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431188+0000", 2024-09-17T21:22:02.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:22:02.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515354+0000", 2024-09-17T21:22:02.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084165977000000003 2024-09-17T21:22:02.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515451+0000", 2024-09-17T21:22:02.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6669000000000006e-05 2024-09-17T21:22:02.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515661+0000", 2024-09-17T21:22:02.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021021599999999999 2024-09-17T21:22:02.068 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.068 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.068 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.069 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:997 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 583680~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:02.069 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:22:02.069 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 96.564548958000003, 2024-09-17T21:22:02.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084772419000000002, 2024-09-17T21:22:02.069 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.069 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.069 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.069 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.069 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:02.069 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:02.069 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 997 2024-09-17T21:22:02.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.069 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:22:02.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:22:02.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:22:02.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0400000000000005e-07 2024-09-17T21:22:02.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431202+0000", 2024-09-17T21:22:02.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:22:02.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431208+0000", 2024-09-17T21:22:02.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4740000000000002e-06 2024-09-17T21:22:02.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431214+0000", 2024-09-17T21:22:02.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6550000000000001e-06 2024-09-17T21:22:02.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515741+0000", 2024-09-17T21:22:02.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084526904 2024-09-17T21:22:02.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515826+0000", 2024-09-17T21:22:02.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4643999999999998e-05 2024-09-17T21:22:02.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515971+0000", 2024-09-17T21:22:02.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145444 2024-09-17T21:22:02.072 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.072 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.072 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.073 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:998 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 587776~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:02.073 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:22:02.073 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 96.564525986000007, 2024-09-17T21:22:02.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085108565999999997, 2024-09-17T21:22:02.073 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.073 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.073 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.073 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.073 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:02.073 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:02.073 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 998 2024-09-17T21:22:02.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.073 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:22:02.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:22:02.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:22:02.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:22:02.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431224+0000", 2024-09-17T21:22:02.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:22:02.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431231+0000", 2024-09-17T21:22:02.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000002e-06 2024-09-17T21:22:02.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431236+0000", 2024-09-17T21:22:02.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.648e-06 2024-09-17T21:22:02.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516024+0000", 2024-09-17T21:22:02.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084787584999999999 2024-09-17T21:22:02.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516122+0000", 2024-09-17T21:22:02.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8076000000000001e-05 2024-09-17T21:22:02.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516330+0000", 2024-09-17T21:22:02.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020808 2024-09-17T21:22:02.076 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.076 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.076 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.077 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1000 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 591872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:02.077 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:22:02.077 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 96.564503870999999, 2024-09-17T21:22:02.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085389065, 2024-09-17T21:22:02.077 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.077 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.077 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.077 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.077 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:02.077 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:02.077 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1000 2024-09-17T21:22:02.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.078 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:22:02.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:22:02.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:22:02.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:22:02.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431246+0000", 2024-09-17T21:22:02.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1749999999999999e-06 2024-09-17T21:22:02.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431253+0000", 2024-09-17T21:22:02.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2790000000000004e-06 2024-09-17T21:22:02.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431258+0000", 2024-09-17T21:22:02.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3299999999999998e-06 2024-09-17T21:22:02.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516384+0000", 2024-09-17T21:22:02.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085125549999999994 2024-09-17T21:22:02.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516460+0000", 2024-09-17T21:22:02.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6359999999999994e-05 2024-09-17T21:22:02.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516633+0000", 2024-09-17T21:22:02.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017285900000000001 2024-09-17T21:22:02.081 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.081 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.081 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.081 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1001 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 595968~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:02.081 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:22:02.081 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 96.564480465000003, 2024-09-17T21:22:02.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085614715999999993, 2024-09-17T21:22:02.081 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.081 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.081 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.081 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.082 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:02.082 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:02.082 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1001 2024-09-17T21:22:02.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.082 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:22:02.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:22:02.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438066+0000", 2024-09-17T21:22:02.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067989490000000003 2024-09-17T21:22:02.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438073+0000", 2024-09-17T21:22:02.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8419999999999999e-06 2024-09-17T21:22:02.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438090+0000", 2024-09-17T21:22:02.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6657e-05 2024-09-17T21:22:02.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438099+0000", 2024-09-17T21:22:02.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0529999999999996e-06 2024-09-17T21:22:02.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516678+0000", 2024-09-17T21:22:02.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078579422999999995 2024-09-17T21:22:02.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516739+0000", 2024-09-17T21:22:02.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1315000000000005e-05 2024-09-17T21:22:02.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516882+0000", 2024-09-17T21:22:02.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014247700000000001 2024-09-17T21:22:02.085 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.085 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.085 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.085 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:19998 3.5 3:a4885450:::rbd_object_map.1152d139f9f9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-17T21:22:02.085 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:22:02.085 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 82.593213219999996, 2024-09-17T21:22:02.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90715489299999996, 2024-09-17T21:22:02.086 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.086 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.086 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:02.086 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.086 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:02.086 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:02.086 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19998 2024-09-17T21:22:02.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.086 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:22:02.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:22:02.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402537+0000", 2024-09-17T21:22:02.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5600000000000001e-06 2024-09-17T21:22:02.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402549+0000", 2024-09-17T21:22:02.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1929999999999999e-05 2024-09-17T21:22:02.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402579+0000", 2024-09-17T21:22:02.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0093000000000001e-05 2024-09-17T21:22:02.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402595+0000", 2024-09-17T21:22:02.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5855999999999999e-05 2024-09-17T21:22:02.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402731+0000", 2024-09-17T21:22:02.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013610799999999999 2024-09-17T21:22:02.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:02.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402750+0000", 2024-09-17T21:22:02.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8737e-05 2024-09-17T21:22:02.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.305977+0000", 2024-09-17T21:22:02.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90322768399999998 2024-09-17T21:22:02.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306236+0000", 2024-09-17T21:22:02.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000258111 2024-09-17T21:22:02.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:22:02.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306557+0000", 2024-09-17T21:22:02.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032159799999999998 2024-09-17T21:22:02.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:02.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.308286+0000", 2024-09-17T21:22:02.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017292150000000001 2024-09-17T21:22:02.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:02.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309530+0000", 2024-09-17T21:22:02.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00124409 2024-09-17T21:22:02.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:02.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309630+0000", 2024-09-17T21:22:02.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9689e-05 2024-09-17T21:22:02.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309689+0000", 2024-09-17T21:22:02.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9222000000000003e-05 2024-09-17T21:22:02.091 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.091 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.092 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.092 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:25277 3.1f 3:ff5d7cd5:::rbd_header.11524f65d4ff:head [watch ping cookie 139860845478480 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-17T21:22:02.092 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:22:02.092 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.410257151000003, 2024-09-17T21:22:02.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.92185720000000004, 2024-09-17T21:22:02.092 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.092 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.092 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.092 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.092 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:02.092 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:02.093 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 25277 2024-09-17T21:22:02.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.093 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:22:02.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:22:02.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585493+0000", 2024-09-17T21:22:02.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:22:02.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585505+0000", 2024-09-17T21:22:02.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2466000000000001e-05 2024-09-17T21:22:02.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585537+0000", 2024-09-17T21:22:02.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1183000000000003e-05 2024-09-17T21:22:02.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585553+0000", 2024-09-17T21:22:02.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.662e-05 2024-09-17T21:22:02.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585696+0000", 2024-09-17T21:22:02.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014293800000000001 2024-09-17T21:22:02.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:02.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585718+0000", 2024-09-17T21:22:02.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1860000000000001e-05 2024-09-17T21:22:02.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.501188+0000", 2024-09-17T21:22:02.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.91547051300000004 2024-09-17T21:22:02.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:22:02.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.501260+0000", 2024-09-17T21:22:02.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1571999999999998e-05 2024-09-17T21:22:02.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507002+0000", 2024-09-17T21:22:02.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057414850000000002 2024-09-17T21:22:02.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507143+0000", 2024-09-17T21:22:02.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014140299999999999 2024-09-17T21:22:02.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507348+0000", 2024-09-17T21:22:02.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000204773 2024-09-17T21:22:02.097 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.097 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.097 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.098 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:26042 3.f 3:f24134c1:::rbd_id.image_client.0-clone34:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e186)", 2024-09-17T21:22:02.098 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:22:02.098 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.509568782999999, 2024-09-17T21:22:02.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.91185481000000002, 2024-09-17T21:22:02.098 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.098 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.098 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:02.098 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.098 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:02.098 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:02.098 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 26042 2024-09-17T21:22:02.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.098 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:22:02.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:22:02.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486180+0000", 2024-09-17T21:22:02.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5990000000000001e-06 2024-09-17T21:22:02.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486187+0000", 2024-09-17T21:22:02.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7859999999999999e-06 2024-09-17T21:22:02.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486206+0000", 2024-09-17T21:22:02.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8641e-05 2024-09-17T21:22:02.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486219+0000", 2024-09-17T21:22:02.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.308e-05 2024-09-17T21:22:02.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486283+0000", 2024-09-17T21:22:02.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4084000000000005e-05 2024-09-17T21:22:02.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:02.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486290+0000", 2024-09-17T21:22:02.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.584e-06 2024-09-17T21:22:02.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392182+0000", 2024-09-17T21:22:02.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90589206499999997 2024-09-17T21:22:02.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392518+0000", 2024-09-17T21:22:02.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033596 2024-09-17T21:22:02.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:22:02.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392802+0000", 2024-09-17T21:22:02.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028394999999999999 2024-09-17T21:22:02.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:02.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397705+0000", 2024-09-17T21:22:02.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0049031769999999999 2024-09-17T21:22:02.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:02.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397770+0000", 2024-09-17T21:22:02.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4765999999999996e-05 2024-09-17T21:22:02.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:02.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397927+0000", 2024-09-17T21:22:02.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015699900000000001 2024-09-17T21:22:02.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.398034+0000", 2024-09-17T21:22:02.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000107119 2024-09-17T21:22:02.104 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.104 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.104 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.104 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.104 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.104 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:02.104 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:22:02.105 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:22:02.105 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:22:02.149 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:02.147+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:02.207 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:02.205+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:02.233 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:02.233 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:02.233 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:22:02.233 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:02.248 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:22:02.309 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:02.307+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:02.409 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:02.405+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:02.420 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:02.420 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:02.420 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:22:02.420 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:22:02.420 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:02.435 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:22:02.510 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:02.507+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:02.556 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:22:02.610 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:02.605+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:02.614 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:02.614 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:22:02.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:22:02.615 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:22:02.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.615 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:18003 3.1d 3:b9765580:::rbd_header.11527d2b15f6:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T21:22:02.615 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:22:02.615 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 110.251917554, 2024-09-17T21:22:02.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94165346900000002, 2024-09-17T21:22:02.615 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.615 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.615 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.615 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.615 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:02.615 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:02.615 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18003 2024-09-17T21:22:02.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.616 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:22:02.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:22:02.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357880+0000", 2024-09-17T21:22:02.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:22:02.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357891+0000", 2024-09-17T21:22:02.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0417999999999999e-05 2024-09-17T21:22:02.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357913+0000", 2024-09-17T21:22:02.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2215e-05 2024-09-17T21:22:02.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357922+0000", 2024-09-17T21:22:02.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4779999999999997e-06 2024-09-17T21:22:02.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358023+0000", 2024-09-17T21:22:02.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101025 2024-09-17T21:22:02.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:02.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358040+0000", 2024-09-17T21:22:02.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6291999999999999e-05 2024-09-17T21:22:02.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298203+0000", 2024-09-17T21:22:02.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94016297500000001 2024-09-17T21:22:02.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298626+0000", 2024-09-17T21:22:02.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000422853 2024-09-17T21:22:02.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.299532+0000", 2024-09-17T21:22:02.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00090657300000000004 2024-09-17T21:22:02.619 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.620 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.620 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.620 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15707 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 982016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:02.620 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:22:02.620 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 100.08875230699999, 2024-09-17T21:22:02.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092305228000000003, 2024-09-17T21:22:02.620 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.620 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.620 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.620 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.620 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:02.621 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:02.621 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15707 2024-09-17T21:22:02.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.621 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:22:02.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:22:02.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521045+0000", 2024-09-17T21:22:02.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4899999999999999e-06 2024-09-17T21:22:02.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521066+0000", 2024-09-17T21:22:02.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0236000000000001e-05 2024-09-17T21:22:02.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521085+0000", 2024-09-17T21:22:02.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8819999999999999e-05 2024-09-17T21:22:02.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521100+0000", 2024-09-17T21:22:02.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5146e-05 2024-09-17T21:22:02.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612978+0000", 2024-09-17T21:22:02.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091878847 2024-09-17T21:22:02.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613079+0000", 2024-09-17T21:22:02.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100503 2024-09-17T21:22:02.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613349+0000", 2024-09-17T21:22:02.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000270186 2024-09-17T21:22:02.624 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.624 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.624 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.624 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15708 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 986112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:02.624 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:22:02.624 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 100.08867413199999, 2024-09-17T21:22:02.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092547217000000001, 2024-09-17T21:22:02.625 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.625 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.625 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.625 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.625 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:02.625 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:02.625 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15708 2024-09-17T21:22:02.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.625 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:22:02.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:22:02.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521123+0000", 2024-09-17T21:22:02.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.353e-06 2024-09-17T21:22:02.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521129+0000", 2024-09-17T21:22:02.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:22:02.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521145+0000", 2024-09-17T21:22:02.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.611e-05 2024-09-17T21:22:02.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521154+0000", 2024-09-17T21:22:02.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4500000000000004e-06 2024-09-17T21:22:02.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613397+0000", 2024-09-17T21:22:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092243275 2024-09-17T21:22:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613503+0000", 2024-09-17T21:22:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106013 2024-09-17T21:22:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613669+0000", 2024-09-17T21:22:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016642299999999999 2024-09-17T21:22:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15709 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 990208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:22:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 100.088631116, 2024-09-17T21:22:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092728678999999994, 2024-09-17T21:22:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15709 2024-09-17T21:22:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:22:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:22:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521166+0000", 2024-09-17T21:22:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7000000000000004e-07 2024-09-17T21:22:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521169+0000", 2024-09-17T21:22:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2660000000000002e-06 2024-09-17T21:22:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521181+0000", 2024-09-17T21:22:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.208e-05 2024-09-17T21:22:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521194+0000", 2024-09-17T21:22:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2707e-05 2024-09-17T21:22:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613695+0000", 2024-09-17T21:22:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092500709 2024-09-17T21:22:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613750+0000", 2024-09-17T21:22:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.503e-05 2024-09-17T21:22:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613894+0000", 2024-09-17T21:22:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144117 2024-09-17T21:22:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15710 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 994304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 100.08858655, 2024-09-17T21:22:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092957729000000003, 2024-09-17T21:22:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15710 2024-09-17T21:22:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8000000000000005e-07 2024-09-17T21:22:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521216+0000", 2024-09-17T21:22:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3859999999999998e-06 2024-09-17T21:22:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521234+0000", 2024-09-17T21:22:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:22:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521248+0000", 2024-09-17T21:22:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:22:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613926+0000", 2024-09-17T21:22:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092678324000000006 2024-09-17T21:22:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613974+0000", 2024-09-17T21:22:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7754999999999999e-05 2024-09-17T21:22:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614167+0000", 2024-09-17T21:22:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193508 2024-09-17T21:22:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15711 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 998400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:22:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 100.08852916799999, 2024-09-17T21:22:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093268000000000004, 2024-09-17T21:22:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15711 2024-09-17T21:22:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:22:02.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:22:02.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521268+0000", 2024-09-17T21:22:02.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-17T21:22:02.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521273+0000", 2024-09-17T21:22:02.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:22:02.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521290+0000", 2024-09-17T21:22:02.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7269999999999999e-05 2024-09-17T21:22:02.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521304+0000", 2024-09-17T21:22:02.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3626e-05 2024-09-17T21:22:02.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614220+0000", 2024-09-17T21:22:02.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092916367 2024-09-17T21:22:02.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614322+0000", 2024-09-17T21:22:02.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102212 2024-09-17T21:22:02.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614535+0000", 2024-09-17T21:22:02.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021266199999999999 2024-09-17T21:22:02.641 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.642 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.642 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.642 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15712 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1002496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:02.642 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:22:02.642 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 100.088473673, 2024-09-17T21:22:02.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093568707000000001, 2024-09-17T21:22:02.642 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.642 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.642 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.642 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.642 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:02.642 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:02.643 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15712 2024-09-17T21:22:02.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.643 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:22:02.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:22:02.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521324+0000", 2024-09-17T21:22:02.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.17e-06 2024-09-17T21:22:02.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521334+0000", 2024-09-17T21:22:02.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9369999999999999e-06 2024-09-17T21:22:02.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521356+0000", 2024-09-17T21:22:02.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2045999999999999e-05 2024-09-17T21:22:02.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521370+0000", 2024-09-17T21:22:02.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4270000000000001e-05 2024-09-17T21:22:02.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614583+0000", 2024-09-17T21:22:02.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093213220999999999 2024-09-17T21:22:02.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614680+0000", 2024-09-17T21:22:02.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6900999999999999e-05 2024-09-17T21:22:02.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614891+0000", 2024-09-17T21:22:02.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021116200000000001 2024-09-17T21:22:02.646 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.646 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.646 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.646 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15713 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1006592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:02.646 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:22:02.646 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 100.088406971, 2024-09-17T21:22:02.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093834641999999996, 2024-09-17T21:22:02.646 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.646 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.646 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.647 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.647 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:02.647 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:02.647 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15713 2024-09-17T21:22:02.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.647 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:22:02.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:22:02.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521390+0000", 2024-09-17T21:22:02.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:22:02.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521396+0000", 2024-09-17T21:22:02.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:22:02.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521417+0000", 2024-09-17T21:22:02.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1236000000000001e-05 2024-09-17T21:22:02.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521431+0000", 2024-09-17T21:22:02.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3873000000000001e-05 2024-09-17T21:22:02.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614938+0000", 2024-09-17T21:22:02.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093506522999999994 2024-09-17T21:22:02.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615025+0000", 2024-09-17T21:22:02.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7635999999999999e-05 2024-09-17T21:22:02.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615224+0000", 2024-09-17T21:22:02.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019844700000000001 2024-09-17T21:22:02.650 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.650 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.650 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.651 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15714 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:02.651 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:22:02.651 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 100.08834371499999, 2024-09-17T21:22:02.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094071925000000001, 2024-09-17T21:22:02.651 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.651 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.651 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.651 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.651 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:02.651 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:02.651 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15714 2024-09-17T21:22:02.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.651 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:22:02.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:22:02.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521454+0000", 2024-09-17T21:22:02.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.097e-06 2024-09-17T21:22:02.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521459+0000", 2024-09-17T21:22:02.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2660000000000001e-06 2024-09-17T21:22:02.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521481+0000", 2024-09-17T21:22:02.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1878999999999998e-05 2024-09-17T21:22:02.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521498+0000", 2024-09-17T21:22:02.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6807e-05 2024-09-17T21:22:02.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615271+0000", 2024-09-17T21:22:02.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093773286999999997 2024-09-17T21:22:02.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615366+0000", 2024-09-17T21:22:02.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5322000000000007e-05 2024-09-17T21:22:02.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615524+0000", 2024-09-17T21:22:02.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015826699999999999 2024-09-17T21:22:02.655 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.655 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.655 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.655 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15715 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1014784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:02.655 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:22:02.655 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 100.08827938, 2024-09-17T21:22:02.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094243840999999995, 2024-09-17T21:22:02.655 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.655 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.655 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.656 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.656 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:02.656 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:02.656 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15715 2024-09-17T21:22:02.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.656 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:22:02.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:22:02.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521518+0000", 2024-09-17T21:22:02.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4300000000000001e-06 2024-09-17T21:22:02.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521524+0000", 2024-09-17T21:22:02.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5860000000000004e-06 2024-09-17T21:22:02.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521541+0000", 2024-09-17T21:22:02.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7574000000000001e-05 2024-09-17T21:22:02.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521555+0000", 2024-09-17T21:22:02.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3540000000000001e-05 2024-09-17T21:22:02.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615560+0000", 2024-09-17T21:22:02.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.09400472 2024-09-17T21:22:02.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615614+0000", 2024-09-17T21:22:02.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4693e-05 2024-09-17T21:22:02.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615761+0000", 2024-09-17T21:22:02.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000146298 2024-09-17T21:22:02.659 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.659 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.659 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.660 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15716 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1018880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:02.660 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:22:02.660 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 100.088223035, 2024-09-17T21:22:02.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094400676000000003, 2024-09-17T21:22:02.660 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.660 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.660 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:02.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:02.660 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15716 2024-09-17T21:22:02.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.660 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:22:02.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:22:02.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521574+0000", 2024-09-17T21:22:02.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1939999999999999e-06 2024-09-17T21:22:02.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521579+0000", 2024-09-17T21:22:02.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:22:02.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521597+0000", 2024-09-17T21:22:02.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8074e-05 2024-09-17T21:22:02.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521613+0000", 2024-09-17T21:22:02.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5316e-05 2024-09-17T21:22:02.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615780+0000", 2024-09-17T21:22:02.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094166874999999997 2024-09-17T21:22:02.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615836+0000", 2024-09-17T21:22:02.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6926999999999999e-05 2024-09-17T21:22:02.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615974+0000", 2024-09-17T21:22:02.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013741400000000001 2024-09-17T21:22:02.664 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.664 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.664 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.664 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15717 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1022976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:02.664 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:22:02.664 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 100.088164339, 2024-09-17T21:22:02.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094586723999999997, 2024-09-17T21:22:02.664 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.665 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.665 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.665 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.665 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:02.665 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:02.665 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15717 2024-09-17T21:22:02.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.665 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:22:02.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:22:02.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521633+0000", 2024-09-17T21:22:02.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:22:02.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521638+0000", 2024-09-17T21:22:02.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:22:02.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521659+0000", 2024-09-17T21:22:02.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-05 2024-09-17T21:22:02.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521675+0000", 2024-09-17T21:22:02.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5688999999999998e-05 2024-09-17T21:22:02.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616008+0000", 2024-09-17T21:22:02.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094333314000000001 2024-09-17T21:22:02.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616066+0000", 2024-09-17T21:22:02.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7797000000000002e-05 2024-09-17T21:22:02.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616219+0000", 2024-09-17T21:22:02.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015281100000000001 2024-09-17T21:22:02.668 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.668 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15718 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 100.088064264, 2024-09-17T21:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094703674000000002, 2024-09-17T21:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15718 2024-09-17T21:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.670 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:22:02.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:22:02.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521734+0000", 2024-09-17T21:22:02.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.61e-06 2024-09-17T21:22:02.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521740+0000", 2024-09-17T21:22:02.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:22:02.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521754+0000", 2024-09-17T21:22:02.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3793e-05 2024-09-17T21:22:02.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521763+0000", 2024-09-17T21:22:02.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5129999999999996e-06 2024-09-17T21:22:02.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616256+0000", 2024-09-17T21:22:02.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094492919999999994 2024-09-17T21:22:02.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616312+0000", 2024-09-17T21:22:02.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6737000000000002e-05 2024-09-17T21:22:02.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616436+0000", 2024-09-17T21:22:02.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012335800000000001 2024-09-17T21:22:02.673 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.673 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.673 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.673 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15719 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1031168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:02.673 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:22:02.673 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 100.08801085899999, 2024-09-17T21:22:02.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094888963000000007, 2024-09-17T21:22:02.673 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.674 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.674 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.674 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.674 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:02.674 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:02.674 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15719 2024-09-17T21:22:02.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.674 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:22:02.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:22:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521786+0000", 2024-09-17T21:22:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.074e-06 2024-09-17T21:22:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521791+0000", 2024-09-17T21:22:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3030000000000001e-06 2024-09-17T21:22:02.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521804+0000", 2024-09-17T21:22:02.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3027000000000001e-05 2024-09-17T21:22:02.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521811+0000", 2024-09-17T21:22:02.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6090000000000003e-06 2024-09-17T21:22:02.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616466+0000", 2024-09-17T21:22:02.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094654294 2024-09-17T21:22:02.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616523+0000", 2024-09-17T21:22:02.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6953999999999997e-05 2024-09-17T21:22:02.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616674+0000", 2024-09-17T21:22:02.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015170200000000001 2024-09-17T21:22:02.677 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.677 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.677 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.678 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15720 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1035264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:02.678 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:22:02.678 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 100.087974019, 2024-09-17T21:22:02.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095067061999999994, 2024-09-17T21:22:02.678 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.678 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.678 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.678 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.678 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:02.678 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:02.678 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15720 2024-09-17T21:22:02.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.679 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:22:02.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:22:02.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521823+0000", 2024-09-17T21:22:02.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:22:02.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521826+0000", 2024-09-17T21:22:02.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2100000000000002e-06 2024-09-17T21:22:02.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521835+0000", 2024-09-17T21:22:02.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1630000000000002e-06 2024-09-17T21:22:02.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521842+0000", 2024-09-17T21:22:02.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:22:02.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616711+0000", 2024-09-17T21:22:02.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094868859 2024-09-17T21:22:02.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616764+0000", 2024-09-17T21:22:02.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2778999999999997e-05 2024-09-17T21:22:02.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616889+0000", 2024-09-17T21:22:02.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012533100000000001 2024-09-17T21:22:02.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.682 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.682 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15721 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1039360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:02.682 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:22:02.682 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 100.08792690999999, 2024-09-17T21:22:02.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095277219999999996, 2024-09-17T21:22:02.682 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.683 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.683 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:02.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:02.683 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15721 2024-09-17T21:22:02.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.683 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:22:02.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:22:02.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521871+0000", 2024-09-17T21:22:02.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.443e-06 2024-09-17T21:22:02.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521876+0000", 2024-09-17T21:22:02.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0300000000000001e-06 2024-09-17T21:22:02.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521886+0000", 2024-09-17T21:22:02.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186e-05 2024-09-17T21:22:02.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521893+0000", 2024-09-17T21:22:02.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.08e-06 2024-09-17T21:22:02.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616918+0000", 2024-09-17T21:22:02.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095025223000000006 2024-09-17T21:22:02.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616977+0000", 2024-09-17T21:22:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8403999999999999e-05 2024-09-17T21:22:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617147+0000", 2024-09-17T21:22:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016985399999999999 2024-09-17T21:22:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.687 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15722 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:02.687 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:02.687 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 100.087892461, 2024-09-17T21:22:02.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095459783000000006, 2024-09-17T21:22:02.687 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.687 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.687 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.687 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.687 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:02.687 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:02.687 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15722 2024-09-17T21:22:02.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.687 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:02.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:02.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:02.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7700000000000004e-07 2024-09-17T21:22:02.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521907+0000", 2024-09-17T21:22:02.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.813e-06 2024-09-17T21:22:02.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521922+0000", 2024-09-17T21:22:02.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4533e-05 2024-09-17T21:22:02.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521934+0000", 2024-09-17T21:22:02.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2469999999999999e-05 2024-09-17T21:22:02.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617187+0000", 2024-09-17T21:22:02.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095252311000000006 2024-09-17T21:22:02.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617240+0000", 2024-09-17T21:22:02.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3740999999999999e-05 2024-09-17T21:22:02.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617364+0000", 2024-09-17T21:22:02.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-17T21:22:02.691 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.691 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.691 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.691 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15723 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:02.691 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:22:02.691 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 100.087848595, 2024-09-17T21:22:02.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095653563999999996, 2024-09-17T21:22:02.691 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.691 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.691 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.691 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:02.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:02.692 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15723 2024-09-17T21:22:02.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.692 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:22:02.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:22:02.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521949+0000", 2024-09-17T21:22:02.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:22:02.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521953+0000", 2024-09-17T21:22:02.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5870000000000003e-06 2024-09-17T21:22:02.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521970+0000", 2024-09-17T21:22:02.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7282000000000002e-05 2024-09-17T21:22:02.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521984+0000", 2024-09-17T21:22:02.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:22:02.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617383+0000", 2024-09-17T21:22:02.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095399063000000006 2024-09-17T21:22:02.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617437+0000", 2024-09-17T21:22:02.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3931999999999998e-05 2024-09-17T21:22:02.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617601+0000", 2024-09-17T21:22:02.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016407300000000001 2024-09-17T21:22:02.695 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.695 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.695 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.695 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5629.0:2384 3.1c 3:38c1263e:::rbd_data.1152d37183c3.000000000000000b:head [read 11780096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-17T21:22:02.695 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:22:02.696 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 87.185017764999998, 2024-09-17T21:22:02.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0246185320000001, 2024-09-17T21:22:02.696 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.696 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.696 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:02.696 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.696 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5629", 2024-09-17T21:22:02.696 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3798711329", 2024-09-17T21:22:02.696 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2384 2024-09-17T21:22:02.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.696 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:22:02.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:22:02.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424780+0000", 2024-09-17T21:22:02.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.13e-06 2024-09-17T21:22:02.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424785+0000", 2024-09-17T21:22:02.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4739999999999996e-06 2024-09-17T21:22:02.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424798+0000", 2024-09-17T21:22:02.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2704000000000001e-05 2024-09-17T21:22:02.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424806+0000", 2024-09-17T21:22:02.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9899999999999997e-06 2024-09-17T21:22:02.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424868+0000", 2024-09-17T21:22:02.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999997e-05 2024-09-17T21:22:02.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:22:02.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424875+0000", 2024-09-17T21:22:02.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0870000000000001e-06 2024-09-17T21:22:02.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428897+0000", 2024-09-17T21:22:02.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040226080000000004 2024-09-17T21:22:02.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:02.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428913+0000", 2024-09-17T21:22:02.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:22:02.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.448963+0000", 2024-09-17T21:22:02.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0200498659999999 2024-09-17T21:22:02.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449158+0000", 2024-09-17T21:22:02.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019581000000000001 2024-09-17T21:22:02.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449397+0000", 2024-09-17T21:22:02.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023871 2024-09-17T21:22:02.701 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.701 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.701 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.701 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:26043 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_remove_image in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-17T21:22:02.701 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:22:02.702 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.210486645, 2024-09-17T21:22:02.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.019978351, 2024-09-17T21:22:02.702 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:02.702 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:02.702 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:02.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:02.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:02.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:02.702 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 26043 2024-09-17T21:22:02.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.702 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:02.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:02.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:22:02.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:02.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:22:02.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:02.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:02.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399314+0000", 2024-09-17T21:22:02.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.963e-06 2024-09-17T21:22:02.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:02.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399330+0000", 2024-09-17T21:22:02.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6684000000000002e-05 2024-09-17T21:22:02.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:02.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399372+0000", 2024-09-17T21:22:02.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1819000000000001e-05 2024-09-17T21:22:02.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:02.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399396+0000", 2024-09-17T21:22:02.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.406e-05 2024-09-17T21:22:02.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399571+0000", 2024-09-17T21:22:02.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017444800000000001 2024-09-17T21:22:02.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:02.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399595+0000", 2024-09-17T21:22:02.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4709999999999999e-05 2024-09-17T21:22:02.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:02.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.412735+0000", 2024-09-17T21:22:02.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0131394869999999 2024-09-17T21:22:02.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:02.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.413090+0000", 2024-09-17T21:22:02.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000355006 2024-09-17T21:22:02.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:22:02.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.413659+0000", 2024-09-17T21:22:02.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056922699999999995 2024-09-17T21:22:02.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:02.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.418940+0000", 2024-09-17T21:22:02.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052814109999999997 2024-09-17T21:22:02.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:02.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.418992+0000", 2024-09-17T21:22:02.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1646e-05 2024-09-17T21:22:02.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:02.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.419168+0000", 2024-09-17T21:22:02.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017573400000000001 2024-09-17T21:22:02.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:02.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:02.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:02.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.419288+0000", 2024-09-17T21:22:02.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012015600000000001 2024-09-17T21:22:02.707 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.708 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.708 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.708 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:02.708 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:02.708 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:02.709 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:22:02.711 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:02.707+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:02.812 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:02.809+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:02.877 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:02.877 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:02.878 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:22:02.878 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:02.892 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:22:02.913 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:02.911+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:03.010 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 0] out_osds: [2, 5] dead_osds: [2] live_osds: [0, 1, 3, 4, 5] 2024-09-17T21:22:03.010 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:22:03.010 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-17T21:22:03.010 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T21:22:03.010 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T21:22:03.011 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-17T21:22:03.014 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:03.011+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:03.060 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:03.060 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:03.060 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:22:03.060 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:22:03.060 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:03.075 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:22:03.123 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:03.121+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:03.129 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:03.129 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "" 2024-09-17T21:22:03.129 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:03.198 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:03.198 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:22:03.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:22:03.198 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:22:03.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.199 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:22:03.199 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:22:03.199 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 221.590791602, 2024-09-17T21:22:03.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:22:03.199 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:03.199 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:03.199 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:03.199 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:03.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:03.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:22:03.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:03.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:22:03.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:03.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:22:03.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:22:03.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:03.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:22:03.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:22:03.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:03.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:22:03.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:22:03.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:22:03.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:22:03.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:03.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:22:03.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:22:03.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:03.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:22:03.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:22:03.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:03.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:22:03.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:22:03.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:03.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:22:03.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:22:03.203 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.203 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:03.203 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.203 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:22:03.203 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:22:03.203 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 221.580844966, 2024-09-17T21:22:03.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:22:03.203 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:03.203 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:03.203 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:03.203 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:03.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:03.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:22:03.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:03.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:22:03.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:03.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:22:03.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:22:03.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:03.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:22:03.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:22:03.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:03.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:22:03.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:22:03.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:22:03.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:22:03.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:03.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:22:03.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:22:03.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:03.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:22:03.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:22:03.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:03.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:22:03.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:22:03.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:03.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:22:03.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:22:03.207 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.207 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:03.207 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.207 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:22:03.207 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:22:03.208 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 220.337694126, 2024-09-17T21:22:03.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:22:03.208 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:03.208 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:03.208 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:03.208 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:03.208 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:22:03.208 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:22:03.208 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:22:03.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.208 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:03.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:03.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:22:03.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:03.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:22:03.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:03.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:22:03.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:22:03.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:03.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:22:03.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:22:03.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:03.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:22:03.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:22:03.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:22:03.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:22:03.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:03.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:22:03.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:22:03.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:03.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:22:03.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:22:03.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:22:03.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:22:03.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:22:03.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:03.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:22:03.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:22:03.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:03.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:22:03.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:22:03.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:03.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:22:03.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:22:03.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:03.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:22:03.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:22:03.213 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.213 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:03.213 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.213 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:22:03.213 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:22:03.213 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 220.32804863999999, 2024-09-17T21:22:03.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:22:03.213 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:03.213 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:03.214 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:03.214 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:03.214 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:22:03.214 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:22:03.214 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:22:03.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.214 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:03.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:03.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:22:03.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:03.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:22:03.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:03.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:22:03.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:22:03.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:03.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:22:03.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:22:03.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:03.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:22:03.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:22:03.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:22:03.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:22:03.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:03.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:22:03.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:22:03.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:03.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:22:03.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:22:03.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:22:03.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:22:03.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:22:03.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:03.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:22:03.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:22:03.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:03.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:22:03.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:22:03.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:03.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:22:03.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:22:03.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:03.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:22:03.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:22:03.219 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.219 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:03.219 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.219 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:22:03.219 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:22:03.220 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 206.95911260700001, 2024-09-17T21:22:03.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:22:03.220 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:03.220 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:03.220 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:03.220 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:03.220 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:22:03.220 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:22:03.220 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:22:03.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.220 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:03.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:03.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:22:03.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:03.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:22:03.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:03.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:22:03.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:22:03.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:03.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:22:03.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:22:03.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:03.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:22:03.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:22:03.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:22:03.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:22:03.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:03.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:22:03.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:22:03.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:03.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:22:03.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:22:03.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:03.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:22:03.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:22:03.224 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.224 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:03.224 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.224 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:03.224 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:22:03.224 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 188.75742804500001, 2024-09-17T21:22:03.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:22:03.224 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:03.224 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:03.224 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:03.224 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:03.224 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:03.224 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:03.225 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:22:03.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.225 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:03.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:03.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:22:03.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:03.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:22:03.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:03.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:22:03.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:22:03.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:03.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:22:03.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:22:03.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:03.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:22:03.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:22:03.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:22:03.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:22:03.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:03.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:22:03.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:22:03.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:03.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:22:03.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:22:03.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:03.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:22:03.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:22:03.228 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.228 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:03.228 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.229 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:03.229 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:22:03.229 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 188.756933468, 2024-09-17T21:22:03.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:22:03.229 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:03.229 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:03.229 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:03.229 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:03.229 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:03.229 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:03.229 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:22:03.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.229 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:03.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:03.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:22:03.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:03.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:22:03.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:03.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:22:03.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:22:03.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:03.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:22:03.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:22:03.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:03.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:22:03.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:22:03.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:22:03.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:22:03.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:03.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:22:03.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:22:03.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:03.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:22:03.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:22:03.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:03.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:22:03.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:22:03.232 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.233 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:03.233 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.233 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:03.233 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:22:03.233 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 188.75687808699999, 2024-09-17T21:22:03.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:22:03.233 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:03.233 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:03.233 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:03.233 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:03.233 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:03.234 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:03.234 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:22:03.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.234 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:03.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:03.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:22:03.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:03.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:22:03.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:03.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:22:03.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:22:03.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:03.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:22:03.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:22:03.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:03.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:22:03.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:22:03.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:22:03.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:22:03.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:03.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:22:03.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:22:03.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:03.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:22:03.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:22:03.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:03.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:22:03.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:22:03.237 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.237 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:03.237 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.237 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:03.237 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:22:03.238 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 188.756821809, 2024-09-17T21:22:03.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:22:03.238 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:03.238 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:03.238 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:03.238 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:03.238 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:03.238 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:03.238 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:22:03.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.238 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:03.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:03.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:22:03.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:03.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:22:03.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:03.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:22:03.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:22:03.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:03.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:22:03.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:22:03.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:03.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:22:03.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:22:03.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:22:03.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:22:03.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:03.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:22:03.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:22:03.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:03.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:22:03.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:22:03.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:03.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:22:03.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:22:03.242 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.242 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:03.242 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.242 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:03.242 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:22:03.242 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 188.75587217, 2024-09-17T21:22:03.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:22:03.242 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:03.242 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:03.242 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:03.242 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:03.243 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:03.243 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:03.243 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:22:03.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.243 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:03.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:03.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:22:03.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:03.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:22:03.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:03.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:22:03.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:22:03.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:03.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:22:03.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:22:03.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:03.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:22:03.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:22:03.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:22:03.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:22:03.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:03.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:22:03.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:22:03.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:03.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:22:03.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:22:03.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:03.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:22:03.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:22:03.246 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.246 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:03.246 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.246 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:03.246 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:22:03.246 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 188.75579354999999, 2024-09-17T21:22:03.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:22:03.247 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:03.247 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:03.247 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:03.247 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:03.247 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:03.247 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:03.247 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:22:03.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.247 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:03.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:03.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:22:03.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:03.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:22:03.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:03.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:22:03.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:22:03.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:03.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:22:03.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:22:03.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:03.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:22:03.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:22:03.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:22:03.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:22:03.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:03.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:22:03.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:22:03.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:03.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:22:03.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:22:03.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:03.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:22:03.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:22:03.250 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.250 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:03.250 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.251 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:03.251 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:22:03.251 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 188.755680611, 2024-09-17T21:22:03.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:22:03.251 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:03.251 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:03.251 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:03.251 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:03.251 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:03.251 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:03.251 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:22:03.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.251 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:03.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:03.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:22:03.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:03.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:22:03.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:03.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:22:03.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:22:03.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:03.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:22:03.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:22:03.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:03.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:22:03.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:22:03.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:22:03.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:22:03.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:03.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:22:03.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:22:03.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:03.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:22:03.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:22:03.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:03.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:22:03.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:22:03.255 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.255 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:03.255 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.255 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:03.255 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:22:03.255 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 188.755019557, 2024-09-17T21:22:03.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:22:03.255 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:03.256 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:03.256 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:03.256 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:03.256 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:03.256 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:03.256 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:22:03.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.256 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:03.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:03.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:22:03.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:03.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:22:03.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:03.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:22:03.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:22:03.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:03.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:22:03.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:22:03.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:03.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:22:03.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:22:03.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:22:03.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:22:03.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:03.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:22:03.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:22:03.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:03.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:22:03.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:22:03.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:03.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:22:03.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:22:03.259 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.259 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:22:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 188.75495474600001, 2024-09-17T21:22:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:22:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:22:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:22:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:22:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:22:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:22:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:22:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:22:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:22:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:22:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:22:03.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:22:03.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:03.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:22:03.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:22:03.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:03.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:22:03.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:22:03.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:22:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:22:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:22:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 188.75489376900001, 2024-09-17T21:22:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:22:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:22:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:22:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:22:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:22:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:22:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:22:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:22:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:22:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:22:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:22:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:22:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:22:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:22:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:22:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:22:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:22:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:22:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:22:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:22:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 133.857322584, 2024-09-17T21:22:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:22:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:22:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:22:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:22:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:22:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:22:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:22:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:22:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:22:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:22:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:22:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:22:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:22:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:22:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:22:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:22:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:22:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:22:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:22:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:22:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:22:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 133.64866419000001, 2024-09-17T21:22:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:22:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:22:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:22:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:22:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:22:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:22:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:22:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:22:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:22:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:22:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:22:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:22:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:22:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:22:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:22:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:22:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:22:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:22:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:22:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:9547 3.2 e147/132 3:42b518d9:::rbd_data.1152f056b73f.0000000000000001:head v 147'513, mlcod=142'511)", 2024-09-17T21:22:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:22:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 60.723691068000001, 2024-09-17T21:22:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.031059566, 2024-09-17T21:22:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:22:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:22:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471293+0000", 2024-09-17T21:22:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2029999999999999e-06 2024-09-17T21:22:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480606+0000", 2024-09-17T21:22:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093123839999999999 2024-09-17T21:22:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480653+0000", 2024-09-17T21:22:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7852000000000002e-05 2024-09-17T21:22:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480673+0000", 2024-09-17T21:22:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0013e-05 2024-09-17T21:22:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480862+0000", 2024-09-17T21:22:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018833599999999999 2024-09-17T21:22:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480906+0000", 2024-09-17T21:22:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4162000000000002e-05 2024-09-17T21:22:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502259+0000", 2024-09-17T21:22:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021353154999999999 2024-09-17T21:22:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502351+0000", 2024-09-17T21:22:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1460999999999997e-05 2024-09-17T21:22:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:15234 3.1 e147/134 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head v 147'917, mlcod=145'914)", 2024-09-17T21:22:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:22:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 60.091030449999998, 2024-09-17T21:22:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045303695999999997, 2024-09-17T21:22:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:22:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:22:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103954+0000", 2024-09-17T21:22:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2259999999999999e-06 2024-09-17T21:22:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119105+0000", 2024-09-17T21:22:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015150871 2024-09-17T21:22:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119141+0000", 2024-09-17T21:22:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6239000000000003e-05 2024-09-17T21:22:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119159+0000", 2024-09-17T21:22:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7946999999999999e-05 2024-09-17T21:22:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119344+0000", 2024-09-17T21:22:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018524599999999999 2024-09-17T21:22:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119399+0000", 2024-09-17T21:22:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4922e-05 2024-09-17T21:22:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149165+0000", 2024-09-17T21:22:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029765697000000001 2024-09-17T21:22:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149255+0000", 2024-09-17T21:22:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0548000000000001e-05 2024-09-17T21:22:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:19859 3.8 e147/133 3:12f72409:::rbd_data.1152f056b73f.0000000000000003:head v 147'482, mlcod=145'479)", 2024-09-17T21:22:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:22:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.682139382999999, 2024-09-17T21:22:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033476895999999999, 2024-09-17T21:22:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:22:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:22:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512845+0000", 2024-09-17T21:22:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.886e-06 2024-09-17T21:22:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524104+0000", 2024-09-17T21:22:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011259668 2024-09-17T21:22:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524148+0000", 2024-09-17T21:22:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3386000000000001e-05 2024-09-17T21:22:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524172+0000", 2024-09-17T21:22:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4762000000000001e-05 2024-09-17T21:22:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524379+0000", 2024-09-17T21:22:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020671599999999999 2024-09-17T21:22:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524421+0000", 2024-09-17T21:22:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2182999999999999e-05 2024-09-17T21:22:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546216+0000", 2024-09-17T21:22:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021795101000000001 2024-09-17T21:22:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546320+0000", 2024-09-17T21:22:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103194 2024-09-17T21:22:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.291 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:03.291 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.291 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:03.291 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:03.291 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:03.292 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:22:03.293 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:03.287+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:03.393 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:03.391+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:03.425 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:03.425 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:22:03.425 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:22:03.425 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:03.441 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:22:03.495 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:03.493+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-17T21:22:03.596 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:03.593+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:03.613 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:22:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:22:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:22:03.613 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:03.628 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:22:03.697 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:03.693+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:03.797 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:03.793+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:03.801 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:03.801 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:22:03.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:22:03.801 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:22:03.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:22:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:22:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 167.287286107, 2024-09-17T21:22:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:22:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:22:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:22:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:22:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:22:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:22:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:22:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:22:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:22:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:22:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:22:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:22:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:22:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:22:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:22:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:22:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:22:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:22:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:22:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:22:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:22:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:22:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2597 3.1f 3:fb0e9615:::rbd_data.1152a8ae2019.000000000000000c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:22:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.958849679, 2024-09-17T21:22:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097064994000000002, 2024-09-17T21:22:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2597 2024-09-17T21:22:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:22:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:22:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838628+0000", 2024-09-17T21:22:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.187e-06 2024-09-17T21:22:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838654+0000", 2024-09-17T21:22:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:22:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838696+0000", 2024-09-17T21:22:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1226000000000001e-05 2024-09-17T21:22:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838715+0000", 2024-09-17T21:22:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9646000000000001e-05 2024-09-17T21:22:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838937+0000", 2024-09-17T21:22:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022163 2024-09-17T21:22:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.839284+0000", 2024-09-17T21:22:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034753799999999999 2024-09-17T21:22:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935691+0000", 2024-09-17T21:22:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.096406360999999996 2024-09-17T21:22:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2641 3.12 3:488a729f:::rbd_data.1152a8ae2019.0000000000000022:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:22:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.95539290599999, 2024-09-17T21:22:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098280953000000004, 2024-09-17T21:22:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2641 2024-09-17T21:22:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:22:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:22:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842083+0000", 2024-09-17T21:22:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5300000000000002e-07 2024-09-17T21:22:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842088+0000", 2024-09-17T21:22:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:22:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842105+0000", 2024-09-17T21:22:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6773e-05 2024-09-17T21:22:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842119+0000", 2024-09-17T21:22:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:22:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842320+0000", 2024-09-17T21:22:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020052700000000001 2024-09-17T21:22:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842845+0000", 2024-09-17T21:22:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052508600000000002 2024-09-17T21:22:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940363+0000", 2024-09-17T21:22:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097518795000000005 2024-09-17T21:22:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2674 3.12 3:4c87e3ac:::rbd_data.1152a8ae2019.0000000000000033:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:22:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.95503026899999, 2024-09-17T21:22:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098025125000000005, 2024-09-17T21:22:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2674 2024-09-17T21:22:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:22:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:22:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842447+0000", 2024-09-17T21:22:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3999999999999999e-06 2024-09-17T21:22:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842453+0000", 2024-09-17T21:22:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:22:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842472+0000", 2024-09-17T21:22:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9006000000000001e-05 2024-09-17T21:22:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842487+0000", 2024-09-17T21:22:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4866e-05 2024-09-17T21:22:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843594+0000", 2024-09-17T21:22:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011073249999999999 2024-09-17T21:22:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843668+0000", 2024-09-17T21:22:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3188000000000005e-05 2024-09-17T21:22:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843893+0000", 2024-09-17T21:22:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022503399999999999 2024-09-17T21:22:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850439+0000", 2024-09-17T21:22:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065467880000000004 2024-09-17T21:22:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940409+0000", 2024-09-17T21:22:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089969511000000002 2024-09-17T21:22:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940444+0000", 2024-09-17T21:22:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5345999999999997e-05 2024-09-17T21:22:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940470+0000", 2024-09-17T21:22:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5964000000000001e-05 2024-09-17T21:22:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2707 3.1f 3:f805e65e:::rbd_data.1152a8ae2019.0000000000000049:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:22:03.822 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.949983608, 2024-09-17T21:22:03.822 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088327300999999997, 2024-09-17T21:22:03.822 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:03.822 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:03.822 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:03.822 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:03.822 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:03.822 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:03.822 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2707 2024-09-17T21:22:03.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.822 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:03.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.822 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:03.822 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:22:03.822 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:03.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:22:03.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:03.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847494+0000", 2024-09-17T21:22:03.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:22:03.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847504+0000", 2024-09-17T21:22:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:22:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847526+0000", 2024-09-17T21:22:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1976000000000001e-05 2024-09-17T21:22:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847541+0000", 2024-09-17T21:22:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5277e-05 2024-09-17T21:22:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856142+0000", 2024-09-17T21:22:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086006710000000007 2024-09-17T21:22:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856435+0000", 2024-09-17T21:22:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029301499999999999 2024-09-17T21:22:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935819+0000", 2024-09-17T21:22:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079384448999999996 2024-09-17T21:22:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2730 3.12 3:48b3422a:::rbd_data.1152a8ae2019.0000000000000057:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:22:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.940790709, 2024-09-17T21:22:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083879975999999995, 2024-09-17T21:22:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2730 2024-09-17T21:22:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:22:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:22:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856687+0000", 2024-09-17T21:22:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:22:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856697+0000", 2024-09-17T21:22:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0207000000000001e-05 2024-09-17T21:22:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856808+0000", 2024-09-17T21:22:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011071200000000001 2024-09-17T21:22:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856828+0000", 2024-09-17T21:22:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0219000000000001e-05 2024-09-17T21:22:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856907+0000", 2024-09-17T21:22:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9661999999999997e-05 2024-09-17T21:22:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.857019+0000", 2024-09-17T21:22:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011148499999999999 2024-09-17T21:22:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940565+0000", 2024-09-17T21:22:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083545791999999994 2024-09-17T21:22:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2743 3.12 3:4d036e62:::rbd_data.1152a8ae2019.000000000000005f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:22:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.93684520799999, 2024-09-17T21:22:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080059962999999998, 2024-09-17T21:22:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2743 2024-09-17T21:22:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:22:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:22:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860631+0000", 2024-09-17T21:22:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:22:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860636+0000", 2024-09-17T21:22:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5279999999999997e-06 2024-09-17T21:22:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:03.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860649+0000", 2024-09-17T21:22:03.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-17T21:22:03.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860655+0000", 2024-09-17T21:22:03.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:22:03.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860731+0000", 2024-09-17T21:22:03.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5920000000000005e-05 2024-09-17T21:22:03.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.835 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:03.835 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.861070+0000", 2024-09-17T21:22:03.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338535 2024-09-17T21:22:03.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.835 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:03.835 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940690+0000", 2024-09-17T21:22:03.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079620503999999995 2024-09-17T21:22:03.835 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.835 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:03.835 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.836 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3378 3.12 3:4832a51b:::rbd_data.1152a8ae2019.000000000000006c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:03.836 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:22:03.836 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.90848135799999, 2024-09-17T21:22:03.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051812516000000003, 2024-09-17T21:22:03.836 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:03.836 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:03.836 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:03.836 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:03.836 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:03.836 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:03.836 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3378 2024-09-17T21:22:03.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.837 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:03.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:03.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:22:03.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:03.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:22:03.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:03.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888995+0000", 2024-09-17T21:22:03.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:22:03.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:03.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889006+0000", 2024-09-17T21:22:03.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1012999999999999e-05 2024-09-17T21:22:03.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:03.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889032+0000", 2024-09-17T21:22:03.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6441e-05 2024-09-17T21:22:03.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889040+0000", 2024-09-17T21:22:03.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0639999999999994e-06 2024-09-17T21:22:03.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889278+0000", 2024-09-17T21:22:03.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023785899999999999 2024-09-17T21:22:03.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889408+0000", 2024-09-17T21:22:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129219 2024-09-17T21:22:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889723+0000", 2024-09-17T21:22:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000315264 2024-09-17T21:22:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.891972+0000", 2024-09-17T21:22:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002249633 2024-09-17T21:22:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940733+0000", 2024-09-17T21:22:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048760626000000001 2024-09-17T21:22:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940785+0000", 2024-09-17T21:22:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1906000000000001e-05 2024-09-17T21:22:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940807+0000", 2024-09-17T21:22:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1554999999999999e-05 2024-09-17T21:22:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3379 3.12 3:4e83fd71:::rbd_data.1152a8ae2019.000000000000006d:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:22:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.908397151, 2024-09-17T21:22:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051804144000000003, 2024-09-17T21:22:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3379 2024-09-17T21:22:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:22:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:22:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889079+0000", 2024-09-17T21:22:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1800000000000004e-07 2024-09-17T21:22:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889083+0000", 2024-09-17T21:22:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:22:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889094+0000", 2024-09-17T21:22:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1562999999999999e-05 2024-09-17T21:22:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889102+0000", 2024-09-17T21:22:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.401e-06 2024-09-17T21:22:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:22:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014279259999999999 2024-09-17T21:22:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890609+0000", 2024-09-17T21:22:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9566000000000004e-05 2024-09-17T21:22:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890789+0000", 2024-09-17T21:22:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017949700000000001 2024-09-17T21:22:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.893929+0000", 2024-09-17T21:22:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003140538 2024-09-17T21:22:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940834+0000", 2024-09-17T21:22:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046904811999999997 2024-09-17T21:22:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940860+0000", 2024-09-17T21:22:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5341999999999999e-05 2024-09-17T21:22:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940882+0000", 2024-09-17T21:22:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2878999999999999e-05 2024-09-17T21:22:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:2644 3.1e 3:79615585:::rbd_data.115266d0b9f2.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2475520~14848,write 3106304~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:22:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:22:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 134.28576874800001, 2024-09-17T21:22:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050733484000000002, 2024-09-17T21:22:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:22:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:22:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2644 2024-09-17T21:22:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:22:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:22:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511708+0000", 2024-09-17T21:22:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-17T21:22:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522196+0000", 2024-09-17T21:22:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010487712999999999 2024-09-17T21:22:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522223+0000", 2024-09-17T21:22:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7064999999999999e-05 2024-09-17T21:22:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522236+0000", 2024-09-17T21:22:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2931e-05 2024-09-17T21:22:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522348+0000", 2024-09-17T21:22:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111906 2024-09-17T21:22:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522554+0000", 2024-09-17T21:22:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020632699999999999 2024-09-17T21:22:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:22:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.523185+0000", 2024-09-17T21:22:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063081400000000005 2024-09-17T21:22:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.547011+0000", 2024-09-17T21:22:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023825570000000001 2024-09-17T21:22:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562268+0000", 2024-09-17T21:22:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015257514 2024-09-17T21:22:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562367+0000", 2024-09-17T21:22:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8678000000000006e-05 2024-09-17T21:22:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562440+0000", 2024-09-17T21:22:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3337000000000006e-05 2024-09-17T21:22:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:231 3.1 3:856fed78:::rbd_data.115220dd393e.000000000000000b:head [read 0~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:22:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:22:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 105.84301347900001, 2024-09-17T21:22:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34254251699999999, 2024-09-17T21:22:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 231 2024-09-17T21:22:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:22:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:22:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954464+0000", 2024-09-17T21:22:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:22:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954475+0000", 2024-09-17T21:22:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1146e-05 2024-09-17T21:22:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954506+0000", 2024-09-17T21:22:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0292999999999999e-05 2024-09-17T21:22:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954524+0000", 2024-09-17T21:22:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8430000000000001e-05 2024-09-17T21:22:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954661+0000", 2024-09-17T21:22:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136403 2024-09-17T21:22:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954680+0000", 2024-09-17T21:22:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9743000000000001e-05 2024-09-17T21:22:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.296896+0000", 2024-09-17T21:22:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34221550499999998 2024-09-17T21:22:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297004+0000", 2024-09-17T21:22:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010865399999999999 2024-09-17T21:22:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:56 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:22:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:22:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 105.68737978199999, 2024-09-17T21:22:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18718773899999999, 2024-09-17T21:22:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:22:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:22:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 56 2024-09-17T21:22:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:22:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:22:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110098+0000", 2024-09-17T21:22:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:22:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110111+0000", 2024-09-17T21:22:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2289999999999999e-05 2024-09-17T21:22:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110143+0000", 2024-09-17T21:22:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1979e-05 2024-09-17T21:22:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110161+0000", 2024-09-17T21:22:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7839e-05 2024-09-17T21:22:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110319+0000", 2024-09-17T21:22:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015835400000000001 2024-09-17T21:22:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110353+0000", 2024-09-17T21:22:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3849000000000001e-05 2024-09-17T21:22:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297041+0000", 2024-09-17T21:22:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18668842299999999 2024-09-17T21:22:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297169+0000", 2024-09-17T21:22:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012786199999999999 2024-09-17T21:22:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297283+0000", 2024-09-17T21:22:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011430300000000001 2024-09-17T21:22:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:68 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:22:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:22:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 105.67883885099999, 2024-09-17T21:22:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17885582999999999, 2024-09-17T21:22:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:22:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:22:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 68 2024-09-17T21:22:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:22:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:22:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118639+0000", 2024-09-17T21:22:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.306e-06 2024-09-17T21:22:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118651+0000", 2024-09-17T21:22:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2567e-05 2024-09-17T21:22:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118682+0000", 2024-09-17T21:22:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0502e-05 2024-09-17T21:22:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118707+0000", 2024-09-17T21:22:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5463e-05 2024-09-17T21:22:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118889+0000", 2024-09-17T21:22:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018150999999999999 2024-09-17T21:22:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118910+0000", 2024-09-17T21:22:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:22:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297313+0000", 2024-09-17T21:22:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17840366699999999 2024-09-17T21:22:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297389+0000", 2024-09-17T21:22:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-17T21:22:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297492+0000", 2024-09-17T21:22:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103019 2024-09-17T21:22:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5890.0:15234 3.1 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8451072~29696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-17T21:22:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:22:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 60.704190038999997, 2024-09-17T21:22:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.057026052000000001, 2024-09-17T21:22:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5890", 2024-09-17T21:22:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3064344947", 2024-09-17T21:22:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 15234 2024-09-17T21:22:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:22:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:22:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093289+0000", 2024-09-17T21:22:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4939999999999999e-06 2024-09-17T21:22:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103074+0000", 2024-09-17T21:22:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00978564 2024-09-17T21:22:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103112+0000", 2024-09-17T21:22:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7246999999999998e-05 2024-09-17T21:22:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103128+0000", 2024-09-17T21:22:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5812999999999999e-05 2024-09-17T21:22:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103207+0000", 2024-09-17T21:22:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9785999999999998e-05 2024-09-17T21:22:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103315+0000", 2024-09-17T21:22:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107396 2024-09-17T21:22:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:22:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103957+0000", 2024-09-17T21:22:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000642524 2024-09-17T21:22:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.123856+0000", 2024-09-17T21:22:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019898376999999998 2024-09-17T21:22:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150190+0000", 2024-09-17T21:22:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026334682000000002 2024-09-17T21:22:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150255+0000", 2024-09-17T21:22:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4770999999999994e-05 2024-09-17T21:22:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150311+0000", 2024-09-17T21:22:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6322e-05 2024-09-17T21:22:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4239 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 424960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:22:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:22:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.043792381999999, 2024-09-17T21:22:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050123167000000003, 2024-09-17T21:22:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:22:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:22:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4239 2024-09-17T21:22:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:22:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:22:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753684+0000", 2024-09-17T21:22:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.37e-07 2024-09-17T21:22:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753686+0000", 2024-09-17T21:22:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.322e-06 2024-09-17T21:22:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753693+0000", 2024-09-17T21:22:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0890000000000002e-06 2024-09-17T21:22:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753699+0000", 2024-09-17T21:22:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:22:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803390+0000", 2024-09-17T21:22:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049690506000000002 2024-09-17T21:22:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803569+0000", 2024-09-17T21:22:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179784 2024-09-17T21:22:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803806+0000", 2024-09-17T21:22:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023683499999999999 2024-09-17T21:22:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4240 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 413696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:22:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:22:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.043768653999997, 2024-09-17T21:22:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050331353000000002, 2024-09-17T21:22:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:22:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:22:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4240 2024-09-17T21:22:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:22:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:22:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:22:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5700000000000002e-07 2024-09-17T21:22:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753710+0000", 2024-09-17T21:22:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.385e-06 2024-09-17T21:22:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753717+0000", 2024-09-17T21:22:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.832e-06 2024-09-17T21:22:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753722+0000", 2024-09-17T21:22:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8050000000000003e-06 2024-09-17T21:22:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803880+0000", 2024-09-17T21:22:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050157641000000003 2024-09-17T21:22:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803945+0000", 2024-09-17T21:22:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5357000000000005e-05 2024-09-17T21:22:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804038+0000", 2024-09-17T21:22:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2775999999999994e-05 2024-09-17T21:22:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4241 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 415744~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:22:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:22:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.043745538000003, 2024-09-17T21:22:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050578790999999998, 2024-09-17T21:22:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:22:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:22:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4241 2024-09-17T21:22:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:22:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:22:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:22:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5700000000000002e-07 2024-09-17T21:22:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753733+0000", 2024-09-17T21:22:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2570000000000002e-06 2024-09-17T21:22:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753740+0000", 2024-09-17T21:22:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.849e-06 2024-09-17T21:22:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753745+0000", 2024-09-17T21:22:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7450000000000001e-06 2024-09-17T21:22:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804115+0000", 2024-09-17T21:22:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050370038999999998 2024-09-17T21:22:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804223+0000", 2024-09-17T21:22:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010786 2024-09-17T21:22:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804309+0000", 2024-09-17T21:22:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5483999999999997e-05 2024-09-17T21:22:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4242 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 414720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:22:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:22:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.043722649999999, 2024-09-17T21:22:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050748148999999999, 2024-09-17T21:22:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:22:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:22:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4242 2024-09-17T21:22:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:22:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:22:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:22:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6599999999999996e-07 2024-09-17T21:22:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753756+0000", 2024-09-17T21:22:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.368e-06 2024-09-17T21:22:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753763+0000", 2024-09-17T21:22:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9349999999999997e-06 2024-09-17T21:22:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753768+0000", 2024-09-17T21:22:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7620000000000001e-06 2024-09-17T21:22:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804343+0000", 2024-09-17T21:22:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050575050000000003 2024-09-17T21:22:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804397+0000", 2024-09-17T21:22:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4070000000000002e-05 2024-09-17T21:22:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804501+0000", 2024-09-17T21:22:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103398 2024-09-17T21:22:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25642 3.1 3.6a5ad301 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e177)", 2024-09-17T21:22:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:34.316616+0000", 2024-09-17T21:22:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 29.480859067000001, 2024-09-17T21:22:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99065658300000003, 2024-09-17T21:22:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:22:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25642 2024-09-17T21:22:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316616+0000", 2024-09-17T21:22:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316616+0000", 2024-09-17T21:22:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316618+0000", 2024-09-17T21:22:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9309999999999998e-06 2024-09-17T21:22:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316624+0000", 2024-09-17T21:22:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2789999999999998e-06 2024-09-17T21:22:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316634+0000", 2024-09-17T21:22:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0774000000000001e-05 2024-09-17T21:22:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316644+0000", 2024-09-17T21:22:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0043e-05 2024-09-17T21:22:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316767+0000", 2024-09-17T21:22:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012289199999999999 2024-09-17T21:22:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316777+0000", 2024-09-17T21:22:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6199999999999994e-06 2024-09-17T21:22:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.307241+0000", 2024-09-17T21:22:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99046395099999995 2024-09-17T21:22:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.307273+0000", 2024-09-17T21:22:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2092999999999999e-05 2024-09-17T21:22:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25905 3.6 3:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-17T21:22:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:39.359572+0000", 2024-09-17T21:22:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 24.437903785, 2024-09-17T21:22:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96065702100000006, 2024-09-17T21:22:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:22:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25905 2024-09-17T21:22:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359572+0000", 2024-09-17T21:22:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359572+0000", 2024-09-17T21:22:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359573+0000", 2024-09-17T21:22:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5370000000000001e-06 2024-09-17T21:22:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359579+0000", 2024-09-17T21:22:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8039999999999998e-06 2024-09-17T21:22:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359603+0000", 2024-09-17T21:22:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4533999999999999e-05 2024-09-17T21:22:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359613+0000", 2024-09-17T21:22:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2820000000000002e-06 2024-09-17T21:22:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359686+0000", 2024-09-17T21:22:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3301999999999997e-05 2024-09-17T21:22:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359693+0000", 2024-09-17T21:22:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3679999999999996e-06 2024-09-17T21:22:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:40.319999+0000", 2024-09-17T21:22:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96030524500000003 2024-09-17T21:22:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:40.320229+0000", 2024-09-17T21:22:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022994899999999999 2024-09-17T21:22:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:03.902 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:03.903 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:22:03.904 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:03.901+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:04.004 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:04.001+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:04.031 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:04.031 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:22:04.031 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:22:04.031 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:04.045 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:22:04.105 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:22:04.106 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:04.101+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:04.168 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:04.168 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:22:04.168 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:22:04.168 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:22:04.168 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:04.183 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:22:04.228 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:04.227+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:04.248 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:22:04.249 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:22:04.266 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:22:04.266 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:22:04.266 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T21:22:04.287 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:22:04.287 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:22:04.307 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:04.305+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:04.308 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:22:04.308 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:22:04.313 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:04.313 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:04.327 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:22:04.330 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:22:04.331 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:22:04.333 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:22:04.333 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:22:04.408 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:04.405+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:04.456 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:04.457 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:04.471 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:22:04.515 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:04.513+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:04.609 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:04.605+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:04.648 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:04.648 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:04.663 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:22:04.711 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:04.706+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:04.812 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:04.806+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:04.846 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:04.846 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:04.861 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:22:04.913 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:04.909+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:04.983 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:04.983 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:04.998 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:22:05.047 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:05.045+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:05.115 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:05.113+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:05.169 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:05.169 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:05.183 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:22:05.227 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:05.225+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:05.316 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:05.313+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:05.349 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:05.350 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:05.364 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:22:05.451 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:05.449+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:05.519 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:05.517+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:05.532 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:05.532 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:05.546 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:22:05.596 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:22:05.641 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:05.638+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:05.721 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:05.717+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:05.728 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:05.728 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:05.744 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:22:05.835 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:05.833+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:05.911 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:05.912 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:05.922 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:05.917+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:05.925 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:22:06.022 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:06.018+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:06.047 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:06.048 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:06.061 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:22:06.123 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:06.121+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:06.225 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:06.222+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:06.229 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:06.230 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:06.244 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:22:06.325 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:06.321+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:06.334 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:22:06.408 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:06.408 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:06.424 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:22:06.426 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:06.421+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:06.472 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:22:06.472 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:22:06.491 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:22:06.491 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:22:06.492 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T21:22:06.512 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:22:06.513 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:22:06.527 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:06.522+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:06.533 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:22:06.533 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:22:06.547 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:06.547 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:22:06.547 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:22:06.547 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:06.554 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:22:06.554 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:22:06.558 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:22:06.558 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:22:06.563 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:22:06.585 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-17T21:22:06.656 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:06.654+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:06.709 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:22:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:22:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:22:06.709 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:06.724 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:22:06.729 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:06.725+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:06.830 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:06.825+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:06.891 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:06.891 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:22:06.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:22:06.892 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:22:06.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.892 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:22:06.892 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:22:06.892 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 207.07914270699999, 2024-09-17T21:22:06.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:22:06.892 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:06.892 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:06.892 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:06.892 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:06.892 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:22:06.893 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:22:06.893 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:22:06.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.893 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:06.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.893 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:06.893 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:22:06.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.893 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:06.893 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:22:06.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:06.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:22:06.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:22:06.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:06.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:22:06.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:22:06.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:06.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:22:06.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:22:06.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:06.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:22:06.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:22:06.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:22:06.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:22:06.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:06.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:22:06.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:22:06.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:22:06.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:22:06.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:06.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:22:06.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:22:06.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:06.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:22:06.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:22:06.897 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.897 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:06.897 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.897 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:22:06.897 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:22:06.897 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 193.52404092099999, 2024-09-17T21:22:06.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:22:06.898 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:06.898 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:06.898 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:06.898 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:06.898 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:22:06.898 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:06.898 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:22:06.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.898 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:06.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:06.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:22:06.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:06.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:22:06.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:06.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:22:06.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:22:06.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:06.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:22:06.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:22:06.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:06.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:22:06.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:22:06.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:06.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:22:06.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:22:06.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:22:06.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:22:06.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:06.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:22:06.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:22:06.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:22:06.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:22:06.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:06.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:22:06.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:22:06.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:22:06.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:22:06.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:22:06.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:06.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:22:06.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:22:06.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:06.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:22:06.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:22:06.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:06.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:22:06.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:22:06.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:06.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:22:06.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:22:06.903 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.903 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:06.903 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.904 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:793 3.9 3:95f63599:::rbd_data.1152eb799094.0000000000000007:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:22:06.904 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:22:06.904 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.45698476600001, 2024-09-17T21:22:06.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88245766400000003, 2024-09-17T21:22:06.904 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:06.904 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:06.904 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:06.904 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:06.904 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:22:06.904 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:22:06.904 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 793 2024-09-17T21:22:06.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.904 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:06.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:06.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:22:06.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:06.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:22:06.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:06.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431941+0000", 2024-09-17T21:22:06.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.864e-06 2024-09-17T21:22:06.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:06.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431953+0000", 2024-09-17T21:22:06.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1486e-05 2024-09-17T21:22:06.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:06.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431984+0000", 2024-09-17T21:22:06.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1075999999999999e-05 2024-09-17T21:22:06.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:06.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431999+0000", 2024-09-17T21:22:06.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.503e-05 2024-09-17T21:22:06.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432095+0000", 2024-09-17T21:22:06.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5698000000000004e-05 2024-09-17T21:22:06.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:06.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432109+0000", 2024-09-17T21:22:06.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4188e-05 2024-09-17T21:22:06.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.309642+0000", 2024-09-17T21:22:06.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87753353000000001 2024-09-17T21:22:06.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:06.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310104+0000", 2024-09-17T21:22:06.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046103600000000001 2024-09-17T21:22:06.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:06.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310630+0000", 2024-09-17T21:22:06.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052682800000000002 2024-09-17T21:22:06.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:06.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313100+0000", 2024-09-17T21:22:06.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024693369999999998 2024-09-17T21:22:06.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:06.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313935+0000", 2024-09-17T21:22:06.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083540100000000005 2024-09-17T21:22:06.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:06.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314086+0000", 2024-09-17T21:22:06.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015053399999999999 2024-09-17T21:22:06.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:06.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314396+0000", 2024-09-17T21:22:06.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031065599999999997 2024-09-17T21:22:06.910 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.910 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:06.910 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.910 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:800 3.19 3:9dc17f12:::rbd_data.1152eb799094.000000000000000e:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:22:06.911 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:22:06.911 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.45673752099999, 2024-09-17T21:22:06.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88154663799999999, 2024-09-17T21:22:06.911 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:06.911 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:06.911 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:06.911 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:06.911 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:22:06.911 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:22:06.911 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 800 2024-09-17T21:22:06.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.911 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:06.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:06.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:22:06.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:06.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:22:06.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:06.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432187+0000", 2024-09-17T21:22:06.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4100000000000001e-06 2024-09-17T21:22:06.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:06.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432194+0000", 2024-09-17T21:22:06.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0389999999999999e-06 2024-09-17T21:22:06.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:06.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432216+0000", 2024-09-17T21:22:06.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1614e-05 2024-09-17T21:22:06.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:06.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432232+0000", 2024-09-17T21:22:06.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5739e-05 2024-09-17T21:22:06.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432405+0000", 2024-09-17T21:22:06.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017355300000000001 2024-09-17T21:22:06.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:06.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432432+0000", 2024-09-17T21:22:06.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6979e-05 2024-09-17T21:22:06.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.307767+0000", 2024-09-17T21:22:06.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87533472899999998 2024-09-17T21:22:06.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:06.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308192+0000", 2024-09-17T21:22:06.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042485999999999999 2024-09-17T21:22:06.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:06.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308727+0000", 2024-09-17T21:22:06.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00053550400000000001 2024-09-17T21:22:06.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:06.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310347+0000", 2024-09-17T21:22:06.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016198670000000001 2024-09-17T21:22:06.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:06.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313424+0000", 2024-09-17T21:22:06.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030764640000000001 2024-09-17T21:22:06.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:06.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313510+0000", 2024-09-17T21:22:06.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5938000000000003e-05 2024-09-17T21:22:06.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:06.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313733+0000", 2024-09-17T21:22:06.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022294200000000001 2024-09-17T21:22:06.917 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.917 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:06.917 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.917 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:805 3.19 3:9cab2ed6:::rbd_data.1152eb799094.0000000000000013:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:22:06.917 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:22:06.918 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.456533502, 2024-09-17T21:22:06.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88274355699999996, 2024-09-17T21:22:06.918 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:06.918 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:06.918 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:06.918 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:06.918 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:22:06.918 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:22:06.918 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 805 2024-09-17T21:22:06.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.918 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:06.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:06.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:22:06.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:06.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:22:06.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:06.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432391+0000", 2024-09-17T21:22:06.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3e-06 2024-09-17T21:22:06.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:06.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432398+0000", 2024-09-17T21:22:06.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4489999999999996e-06 2024-09-17T21:22:06.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:06.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432415+0000", 2024-09-17T21:22:06.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7750000000000001e-05 2024-09-17T21:22:06.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:06.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432430+0000", 2024-09-17T21:22:06.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.449e-05 2024-09-17T21:22:06.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432635+0000", 2024-09-17T21:22:06.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205031 2024-09-17T21:22:06.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:06.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432657+0000", 2024-09-17T21:22:06.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2064000000000001e-05 2024-09-17T21:22:06.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310444+0000", 2024-09-17T21:22:06.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87778699199999999 2024-09-17T21:22:06.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:06.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310849+0000", 2024-09-17T21:22:06.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00040497800000000001 2024-09-17T21:22:06.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:06.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.311396+0000", 2024-09-17T21:22:06.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00054729099999999999 2024-09-17T21:22:06.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:06.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313847+0000", 2024-09-17T21:22:06.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024504940000000001 2024-09-17T21:22:06.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:06.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314879+0000", 2024-09-17T21:22:06.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001031817 2024-09-17T21:22:06.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:06.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314964+0000", 2024-09-17T21:22:06.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5685000000000004e-05 2024-09-17T21:22:06.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:06.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.315133+0000", 2024-09-17T21:22:06.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000169216 2024-09-17T21:22:06.923 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.924 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:06.924 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.924 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12089 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 60416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:06.924 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:22:06.924 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 105.11930846999999, 2024-09-17T21:22:06.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088293233999999998, 2024-09-17T21:22:06.924 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:06.924 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:06.924 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:06.924 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:06.924 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:06.924 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:06.924 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12089 2024-09-17T21:22:06.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.925 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:06.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:06.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:22:06.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:06.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:22:06.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:06.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769616+0000", 2024-09-17T21:22:06.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:22:06.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:06.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769620+0000", 2024-09-17T21:22:06.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5140000000000002e-06 2024-09-17T21:22:06.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:06.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769633+0000", 2024-09-17T21:22:06.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2849e-05 2024-09-17T21:22:06.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:06.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769644+0000", 2024-09-17T21:22:06.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1471000000000001e-05 2024-09-17T21:22:06.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857564+0000", 2024-09-17T21:22:06.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087919445999999998 2024-09-17T21:22:06.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:06.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857716+0000", 2024-09-17T21:22:06.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000152065 2024-09-17T21:22:06.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:06.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857908+0000", 2024-09-17T21:22:06.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019252599999999999 2024-09-17T21:22:06.928 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.928 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:06.928 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.928 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12090 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 64512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:06.928 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:22:06.928 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 105.119262085, 2024-09-17T21:22:06.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088582672000000001, 2024-09-17T21:22:06.929 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:06.929 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:06.929 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:06.929 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:06.929 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:06.929 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:06.929 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12090 2024-09-17T21:22:06.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.929 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:06.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:06.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:22:06.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:06.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:22:06.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:06.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769662+0000", 2024-09-17T21:22:06.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:22:06.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:06.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769683+0000", 2024-09-17T21:22:06.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0979e-05 2024-09-17T21:22:06.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:06.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769699+0000", 2024-09-17T21:22:06.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5446000000000001e-05 2024-09-17T21:22:06.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:06.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769708+0000", 2024-09-17T21:22:06.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1679999999999993e-06 2024-09-17T21:22:06.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857976+0000", 2024-09-17T21:22:06.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088267795999999996 2024-09-17T21:22:06.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:06.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858095+0000", 2024-09-17T21:22:06.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119147 2024-09-17T21:22:06.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:06.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858244+0000", 2024-09-17T21:22:06.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014917000000000001 2024-09-17T21:22:06.932 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.932 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:06.932 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.933 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12091 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 68608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:06.933 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:22:06.933 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 105.11919070099999, 2024-09-17T21:22:06.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088832752000000001, 2024-09-17T21:22:06.933 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:06.933 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:06.933 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:06.933 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:06.933 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:06.933 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:06.933 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12091 2024-09-17T21:22:06.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.934 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:06.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:06.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:22:06.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:06.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:22:06.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:06.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769734+0000", 2024-09-17T21:22:06.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:22:06.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:06.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769739+0000", 2024-09-17T21:22:06.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8080000000000004e-06 2024-09-17T21:22:06.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:06.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769764+0000", 2024-09-17T21:22:06.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4839999999999999e-05 2024-09-17T21:22:06.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:06.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769777+0000", 2024-09-17T21:22:06.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3037000000000001e-05 2024-09-17T21:22:06.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858295+0000", 2024-09-17T21:22:06.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088518475999999999 2024-09-17T21:22:06.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:06.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858419+0000", 2024-09-17T21:22:06.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012378700000000001 2024-09-17T21:22:06.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:06.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858565+0000", 2024-09-17T21:22:06.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014630099999999999 2024-09-17T21:22:06.937 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.937 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:06.937 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.937 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12092 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:06.937 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:22:06.937 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 105.119108377, 2024-09-17T21:22:06.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089191555000000006, 2024-09-17T21:22:06.937 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:06.937 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:06.937 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:06.938 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:06.938 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:06.938 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:06.938 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12092 2024-09-17T21:22:06.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.938 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:06.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:06.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:22:06.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:06.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:22:06.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:06.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769816+0000", 2024-09-17T21:22:06.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7199999999999997e-07 2024-09-17T21:22:06.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:06.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769822+0000", 2024-09-17T21:22:06.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:22:06.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:06.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769838+0000", 2024-09-17T21:22:06.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6674e-05 2024-09-17T21:22:06.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:06.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769851+0000", 2024-09-17T21:22:06.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2527999999999999e-05 2024-09-17T21:22:06.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858660+0000", 2024-09-17T21:22:06.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088808556999999996 2024-09-17T21:22:06.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:06.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858817+0000", 2024-09-17T21:22:06.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015730400000000001 2024-09-17T21:22:06.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:06.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859007+0000", 2024-09-17T21:22:06.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018974899999999999 2024-09-17T21:22:06.941 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.941 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:06.941 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.941 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12093 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:06.941 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:22:06.942 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 105.119000273, 2024-09-17T21:22:06.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089429710999999995, 2024-09-17T21:22:06.942 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:06.942 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:06.942 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:06.942 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:06.942 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:06.942 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:06.942 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12093 2024-09-17T21:22:06.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.942 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:06.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:06.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:22:06.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:06.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:22:06.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:06.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769924+0000", 2024-09-17T21:22:06.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.342e-06 2024-09-17T21:22:06.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:06.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769929+0000", 2024-09-17T21:22:06.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0350000000000001e-06 2024-09-17T21:22:06.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:06.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769946+0000", 2024-09-17T21:22:06.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6793999999999999e-05 2024-09-17T21:22:06.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:06.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769959+0000", 2024-09-17T21:22:06.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3062e-05 2024-09-17T21:22:06.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859075+0000", 2024-09-17T21:22:06.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089115696999999994 2024-09-17T21:22:06.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:06.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859202+0000", 2024-09-17T21:22:06.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012657 2024-09-17T21:22:06.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:06.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859353+0000", 2024-09-17T21:22:06.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151211 2024-09-17T21:22:06.946 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.946 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:06.946 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.946 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12094 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:06.946 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:22:06.946 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 105.11894609700001, 2024-09-17T21:22:06.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089695001999999996, 2024-09-17T21:22:06.946 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:06.947 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:06.947 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:06.947 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:06.947 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:06.947 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:06.947 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12094 2024-09-17T21:22:06.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.947 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:06.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:06.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:22:06.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:06.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:22:06.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:06.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769978+0000", 2024-09-17T21:22:06.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.142e-06 2024-09-17T21:22:06.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:06.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769983+0000", 2024-09-17T21:22:06.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.386e-06 2024-09-17T21:22:06.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:06.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769997+0000", 2024-09-17T21:22:06.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.416e-05 2024-09-17T21:22:06.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:06.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770012+0000", 2024-09-17T21:22:06.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5384999999999999e-05 2024-09-17T21:22:06.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859405+0000", 2024-09-17T21:22:06.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089392376999999995 2024-09-17T21:22:06.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:06.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859525+0000", 2024-09-17T21:22:06.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120555 2024-09-17T21:22:06.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:06.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859672+0000", 2024-09-17T21:22:06.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000146997 2024-09-17T21:22:06.951 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.951 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:06.951 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.951 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12095 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:06.951 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:22:06.951 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 105.11889361, 2024-09-17T21:22:06.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090073476, 2024-09-17T21:22:06.951 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:06.951 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:06.951 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:06.951 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:06.952 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:06.952 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:06.952 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12095 2024-09-17T21:22:06.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.952 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:06.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:06.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:22:06.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:06.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:22:06.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:06.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770031+0000", 2024-09-17T21:22:06.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1699999999999997e-07 2024-09-17T21:22:06.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:06.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770036+0000", 2024-09-17T21:22:06.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.023e-06 2024-09-17T21:22:06.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:06.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770051+0000", 2024-09-17T21:22:06.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4953999999999999e-05 2024-09-17T21:22:06.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:06.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770063+0000", 2024-09-17T21:22:06.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:22:06.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859756+0000", 2024-09-17T21:22:06.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089692980000000005 2024-09-17T21:22:06.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:06.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859891+0000", 2024-09-17T21:22:06.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013459499999999999 2024-09-17T21:22:06.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:06.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860103+0000", 2024-09-17T21:22:06.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021274400000000001 2024-09-17T21:22:06.955 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.955 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:06.955 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.955 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12096 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:06.956 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:22:06.956 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 105.11884392499999, 2024-09-17T21:22:06.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090358274000000002, 2024-09-17T21:22:06.956 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:06.956 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:06.956 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:06.956 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:06.956 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:06.956 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:06.956 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12096 2024-09-17T21:22:06.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.956 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:06.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:06.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:22:06.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:06.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:22:06.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:06.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770080+0000", 2024-09-17T21:22:06.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9400000000000004e-07 2024-09-17T21:22:06.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:06.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770085+0000", 2024-09-17T21:22:06.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2169999999999996e-06 2024-09-17T21:22:06.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:06.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770100+0000", 2024-09-17T21:22:06.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5077e-05 2024-09-17T21:22:06.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:06.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770112+0000", 2024-09-17T21:22:06.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1951e-05 2024-09-17T21:22:06.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860160+0000", 2024-09-17T21:22:06.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090048413999999993 2024-09-17T21:22:06.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:06.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860285+0000", 2024-09-17T21:22:06.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012466699999999999 2024-09-17T21:22:06.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:06.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860438+0000", 2024-09-17T21:22:06.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015305399999999999 2024-09-17T21:22:06.960 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.960 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:06.960 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.960 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12097 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:06.960 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:22:06.960 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 105.118796178, 2024-09-17T21:22:06.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090702375000000002, 2024-09-17T21:22:06.960 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:06.960 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:06.961 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:06.961 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:06.961 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:06.961 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:06.961 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12097 2024-09-17T21:22:06.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.961 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:06.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:06.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:22:06.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:06.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:22:06.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:06.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770128+0000", 2024-09-17T21:22:06.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.04e-06 2024-09-17T21:22:06.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:06.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770133+0000", 2024-09-17T21:22:06.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8999999999999997e-06 2024-09-17T21:22:06.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:06.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770152+0000", 2024-09-17T21:22:06.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8890999999999999e-05 2024-09-17T21:22:06.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:06.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770164+0000", 2024-09-17T21:22:06.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1769e-05 2024-09-17T21:22:06.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860511+0000", 2024-09-17T21:22:06.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090347527999999996 2024-09-17T21:22:06.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:06.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860652+0000", 2024-09-17T21:22:06.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140894 2024-09-17T21:22:06.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:06.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860830+0000", 2024-09-17T21:22:06.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017735299999999999 2024-09-17T21:22:06.964 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.964 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:06.964 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.964 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12098 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:06.964 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:22:06.964 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 105.11874271000001, 2024-09-17T21:22:06.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090997262999999995, 2024-09-17T21:22:06.965 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:06.965 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:06.965 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:06.965 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:06.965 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:06.965 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:06.965 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12098 2024-09-17T21:22:06.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.965 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:06.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:06.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:22:06.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:06.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:22:06.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:06.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775465+0000", 2024-09-17T21:22:06.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0052843350000000002 2024-09-17T21:22:06.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:06.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775476+0000", 2024-09-17T21:22:06.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0797e-05 2024-09-17T21:22:06.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:06.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775506+0000", 2024-09-17T21:22:06.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9709999999999998e-05 2024-09-17T21:22:06.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:06.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775530+0000", 2024-09-17T21:22:06.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4528999999999998e-05 2024-09-17T21:22:06.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860894+0000", 2024-09-17T21:22:06.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.085364195000000004 2024-09-17T21:22:06.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:06.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861019+0000", 2024-09-17T21:22:06.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012467399999999999 2024-09-17T21:22:06.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:06.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861178+0000", 2024-09-17T21:22:06.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159023 2024-09-17T21:22:06.968 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.969 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:06.969 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.969 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12104 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:06.969 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:22:06.969 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 105.11304160100001, 2024-09-17T21:22:06.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087969628999999994, 2024-09-17T21:22:06.969 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:06.969 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:06.969 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:06.969 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:06.969 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:06.969 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:06.969 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12104 2024-09-17T21:22:06.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.970 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:06.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:06.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:22:06.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:06.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:22:06.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:06.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775883+0000", 2024-09-17T21:22:06.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3399999999999998e-07 2024-09-17T21:22:06.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:06.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775886+0000", 2024-09-17T21:22:06.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2119999999999999e-06 2024-09-17T21:22:06.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:06.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775895+0000", 2024-09-17T21:22:06.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8710000000000003e-06 2024-09-17T21:22:06.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:06.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775901+0000", 2024-09-17T21:22:06.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:22:06.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863544+0000", 2024-09-17T21:22:06.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087642642000000007 2024-09-17T21:22:06.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:06.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863694+0000", 2024-09-17T21:22:06.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014952300000000001 2024-09-17T21:22:06.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:06.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863851+0000", 2024-09-17T21:22:06.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000157804 2024-09-17T21:22:06.973 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.973 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:06.973 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.973 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12105 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:06.973 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:22:06.973 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 105.113012707, 2024-09-17T21:22:06.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088330400000000003, 2024-09-17T21:22:06.974 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:06.974 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:06.974 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:06.974 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:06.974 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:06.974 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:06.974 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12105 2024-09-17T21:22:06.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.974 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:06.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:06.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:22:06.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:06.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:22:06.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:06.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:22:06.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9999999999999997e-07 2024-09-17T21:22:06.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:06.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775914+0000", 2024-09-17T21:22:06.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4710000000000001e-06 2024-09-17T21:22:06.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:06.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775921+0000", 2024-09-17T21:22:06.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:22:06.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:06.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775928+0000", 2024-09-17T21:22:06.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:22:06.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863923+0000", 2024-09-17T21:22:06.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087995126000000007 2024-09-17T21:22:06.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:06.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864059+0000", 2024-09-17T21:22:06.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135797 2024-09-17T21:22:06.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:06.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864241+0000", 2024-09-17T21:22:06.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018252599999999999 2024-09-17T21:22:06.977 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.977 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:06.977 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.978 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12106 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:06.978 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:22:06.978 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 105.112923529, 2024-09-17T21:22:06.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088586394999999998, 2024-09-17T21:22:06.978 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:06.978 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:06.978 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:06.978 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:06.978 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:06.978 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:06.978 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12106 2024-09-17T21:22:06.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.979 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:06.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:06.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:22:06.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:06.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:22:06.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:06.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776001+0000", 2024-09-17T21:22:06.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4300000000000001e-07 2024-09-17T21:22:06.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:06.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776004+0000", 2024-09-17T21:22:06.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8049999999999998e-06 2024-09-17T21:22:06.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:06.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776018+0000", 2024-09-17T21:22:06.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4613999999999999e-05 2024-09-17T21:22:06.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:06.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776025+0000", 2024-09-17T21:22:06.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:22:06.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864279+0000", 2024-09-17T21:22:06.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088253570000000003 2024-09-17T21:22:06.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:06.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864433+0000", 2024-09-17T21:22:06.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015469400000000001 2024-09-17T21:22:06.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:06.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864586+0000", 2024-09-17T21:22:06.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000153003 2024-09-17T21:22:06.982 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.982 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:06.982 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.982 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25033 3.6 3.b74753e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e168)", 2024-09-17T21:22:06.982 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:22:06.982 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.379480264999998, 2024-09-17T21:22:06.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93781952099999999, 2024-09-17T21:22:06.983 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:06.983 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:06.983 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:06.983 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:06.983 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:22:06.983 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:06.983 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25033 2024-09-17T21:22:06.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.983 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:06.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:06.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:22:06.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:06.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:22:06.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:06.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509444+0000", 2024-09-17T21:22:06.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0020000000000001e-06 2024-09-17T21:22:06.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:06.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509450+0000", 2024-09-17T21:22:06.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4249999999999997e-06 2024-09-17T21:22:06.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:06.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509460+0000", 2024-09-17T21:22:06.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0183999999999999e-05 2024-09-17T21:22:06.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:06.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509466+0000", 2024-09-17T21:22:06.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0039999999999996e-06 2024-09-17T21:22:06.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509540+0000", 2024-09-17T21:22:06.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3847e-05 2024-09-17T21:22:06.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:06.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509551+0000", 2024-09-17T21:22:06.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1731e-05 2024-09-17T21:22:06.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.447243+0000", 2024-09-17T21:22:06.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93769166000000004 2024-09-17T21:22:06.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:06.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.447263+0000", 2024-09-17T21:22:06.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9667999999999999e-05 2024-09-17T21:22:06.987 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.987 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:06.987 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.987 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25033 3.6 3:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e168)", 2024-09-17T21:22:06.987 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:22:06.987 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 41.377954668000001, 2024-09-17T21:22:06.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.937309383, 2024-09-17T21:22:06.987 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:06.987 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:06.988 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:06.988 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:06.988 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:22:06.988 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:06.988 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25033 2024-09-17T21:22:06.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.988 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:06.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:06.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:22:06.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:06.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:22:06.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:06.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:06.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510970+0000", 2024-09-17T21:22:06.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.564e-06 2024-09-17T21:22:06.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:06.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510978+0000", 2024-09-17T21:22:06.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.362e-06 2024-09-17T21:22:06.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:06.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510998+0000", 2024-09-17T21:22:06.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0336999999999999e-05 2024-09-17T21:22:06.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:06.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511010+0000", 2024-09-17T21:22:06.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1786e-05 2024-09-17T21:22:06.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511080+0000", 2024-09-17T21:22:06.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0686999999999996e-05 2024-09-17T21:22:06.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:06.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511087+0000", 2024-09-17T21:22:06.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4899999999999997e-06 2024-09-17T21:22:06.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:06.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.448050+0000", 2024-09-17T21:22:06.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93696351700000002 2024-09-17T21:22:06.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:06.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:06.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:06.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.448278+0000", 2024-09-17T21:22:06.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022764000000000001 2024-09-17T21:22:06.991 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.991 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:06.992 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.992 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:06.992 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:06.992 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:06.993 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:22:06.994 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:06.990+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:07.095 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:07.090+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:07.125 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:07.126 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:07.126 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:22:07.126 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:07.140 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:22:07.196 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:07.193+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:07.264 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:07.264 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:07.264 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:22:07.264 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:22:07.264 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:07.279 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:22:07.323 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:07.321+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:07.398 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:07.393+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:07.445 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:07.445 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:22:07.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:22:07.445 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:22:07.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.445 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:22:07.445 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:22:07.445 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 250.42675655599999, 2024-09-17T21:22:07.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:22:07.445 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:07.446 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:07.446 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:07.446 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:07.446 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:22:07.446 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:22:07.446 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:22:07.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.446 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:07.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:07.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:22:07.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:07.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:22:07.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:07.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:22:07.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:22:07.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:07.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:22:07.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:22:07.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:07.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:22:07.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:22:07.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:07.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:22:07.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:22:07.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:22:07.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:22:07.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:07.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:22:07.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:22:07.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:22:07.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:22:07.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:07.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:22:07.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:22:07.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:22:07.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:22:07.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:22:07.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:07.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:22:07.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:22:07.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:07.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:22:07.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:22:07.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:07.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:22:07.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:22:07.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:07.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:22:07.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:22:07.452 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.452 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:07.452 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.452 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:22:07.452 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:22:07.453 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 230.846592276, 2024-09-17T21:22:07.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:22:07.453 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:07.453 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:07.453 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:07.453 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:07.453 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:22:07.453 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:22:07.453 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:22:07.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.453 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:07.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:07.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:22:07.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:07.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:22:07.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:07.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:22:07.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:22:07.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:07.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:22:07.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:22:07.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:07.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:22:07.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:22:07.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:07.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:22:07.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:22:07.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:22:07.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:22:07.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:22:07.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:22:07.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:22:07.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:22:07.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:22:07.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:07.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:22:07.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:22:07.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:22:07.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:22:07.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:07.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:22:07.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:22:07.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:07.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:22:07.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:22:07.458 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.458 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:07.458 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.458 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:22:07.458 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:22:07.458 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 230.84649496399999, 2024-09-17T21:22:07.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:22:07.459 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:07.459 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:07.459 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:07.459 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:07.459 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:07.459 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:07.459 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:22:07.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.459 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:07.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:07.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:22:07.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:07.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:22:07.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:07.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:22:07.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:22:07.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:07.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:22:07.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:22:07.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:07.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:22:07.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:22:07.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:07.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:22:07.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:22:07.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:22:07.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:22:07.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:22:07.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:22:07.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:22:07.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:22:07.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:22:07.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:07.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:22:07.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:22:07.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:22:07.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:22:07.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:22:07.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:22:07.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:22:07.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:22:07.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:22:07.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:07.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:22:07.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:22:07.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:07.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:22:07.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:22:07.464 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.464 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:07.465 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.465 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:22:07.465 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:22:07.465 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 230.844408163, 2024-09-17T21:22:07.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:22:07.465 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:07.465 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:07.465 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:07.465 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:07.465 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:22:07.465 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:22:07.466 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:22:07.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.466 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:07.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:07.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:22:07.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:07.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:22:07.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:07.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:22:07.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:22:07.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:07.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:22:07.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:22:07.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:07.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:22:07.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:22:07.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:07.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:22:07.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:22:07.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:22:07.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:22:07.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:22:07.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:22:07.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:22:07.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:22:07.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:22:07.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:07.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:22:07.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:22:07.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:22:07.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:22:07.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:07.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:22:07.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:22:07.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:22:07.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:22:07.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:22:07.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:07.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:22:07.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:22:07.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:07.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:22:07.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:22:07.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:07.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:22:07.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:22:07.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:07.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:22:07.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:22:07.472 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.472 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:07.472 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.472 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:07.472 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:22:07.472 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 198.16050938199999, 2024-09-17T21:22:07.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:22:07.473 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:07.473 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:07.473 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:07.473 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:07.473 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:22:07.473 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:22:07.473 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:22:07.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.473 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:07.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:07.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:22:07.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:07.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:22:07.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:07.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:22:07.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:22:07.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:07.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:22:07.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:22:07.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:07.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:22:07.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:22:07.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:07.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:22:07.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:22:07.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:22:07.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:22:07.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:22:07.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:22:07.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:22:07.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:22:07.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:22:07.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:07.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:22:07.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:22:07.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:22:07.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:22:07.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:07.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:22:07.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:22:07.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:07.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:22:07.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:22:07.478 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.478 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:07.478 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.478 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:07.478 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:22:07.479 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 198.15141767099999, 2024-09-17T21:22:07.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:22:07.479 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:07.479 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:07.479 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:07.479 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:07.479 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:22:07.479 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:22:07.479 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:22:07.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.479 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:07.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:07.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:22:07.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:07.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:22:07.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:07.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:22:07.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:22:07.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:07.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:22:07.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:22:07.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:07.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:22:07.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:22:07.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:07.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:22:07.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:22:07.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:22:07.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:22:07.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:07.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:22:07.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:22:07.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:22:07.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:22:07.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:07.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:22:07.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:22:07.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:07.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:22:07.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:22:07.483 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.483 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:07.484 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.484 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:07.484 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:22:07.484 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 198.15124562, 2024-09-17T21:22:07.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:22:07.484 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:07.484 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:07.484 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:07.484 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:07.484 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:22:07.485 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:22:07.485 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:22:07.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.485 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:07.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:07.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:22:07.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:07.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:22:07.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:07.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:22:07.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:22:07.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:07.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:22:07.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:22:07.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:07.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:22:07.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:22:07.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:07.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:22:07.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:22:07.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:22:07.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:22:07.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:07.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:22:07.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:22:07.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:22:07.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:22:07.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:07.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:22:07.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:22:07.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:07.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:22:07.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:22:07.489 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.489 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:07.489 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.490 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:07.490 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:22:07.490 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 197.97631287300001, 2024-09-17T21:22:07.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:22:07.490 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:07.490 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:07.490 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:07.490 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:07.490 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:07.490 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:07.490 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:22:07.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.490 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:07.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:07.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:22:07.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:07.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:22:07.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:07.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:22:07.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:22:07.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:07.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:22:07.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:22:07.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:07.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:22:07.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:22:07.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:07.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:22:07.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:22:07.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:22:07.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:22:07.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:07.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:22:07.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:22:07.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:22:07.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:22:07.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:07.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:22:07.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:22:07.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:07.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:22:07.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:22:07.494 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.494 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:07.494 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.495 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:07.495 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:22:07.495 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 197.976228316, 2024-09-17T21:22:07.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:22:07.495 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:07.495 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:07.495 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:07.495 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:07.495 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:07.495 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:07.495 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:22:07.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.495 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:07.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:07.496 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:22:07.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:07.496 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:22:07.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:07.496 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:22:07.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:22:07.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:07.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:22:07.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:22:07.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:07.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:22:07.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:22:07.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:07.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:22:07.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:22:07.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:22:07.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:22:07.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:07.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:22:07.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:22:07.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:22:07.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:22:07.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:07.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:22:07.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:22:07.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:07.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:22:07.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:22:07.499 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.499 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:07.499 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.499 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:07.499 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:22:07.499 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 197.976162679, 2024-09-17T21:22:07.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:22:07.500 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:07.500 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:07.500 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:07.500 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:07.500 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:07.500 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:07.500 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:22:07.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.500 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:07.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:07.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:22:07.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:07.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:22:07.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:07.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:22:07.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:22:07.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:07.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:22:07.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:22:07.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:07.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:22:07.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:22:07.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:07.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:22:07.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:22:07.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:22:07.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:22:07.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:07.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:22:07.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:22:07.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:22:07.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:22:07.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:07.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:22:07.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:22:07.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:07.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:22:07.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:22:07.504 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.504 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:07.504 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.504 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:07.504 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:22:07.504 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 146.60258758200001, 2024-09-17T21:22:07.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:22:07.505 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:07.505 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:07.505 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:07.505 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:07.505 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:22:07.505 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:07.505 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:22:07.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.505 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:07.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:07.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:22:07.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:07.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:22:07.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:07.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:22:07.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:22:07.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:07.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:22:07.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:22:07.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:07.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:22:07.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:22:07.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:07.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:22:07.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:22:07.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:22:07.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:22:07.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:07.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:22:07.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:22:07.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:07.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:22:07.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:22:07.508 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.508 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:07.508 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.509 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:07.509 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:22:07.509 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 146.60246806999999, 2024-09-17T21:22:07.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:22:07.509 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:07.509 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:07.509 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:07.509 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:07.509 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:22:07.509 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:07.509 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:22:07.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.509 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:07.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:07.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:22:07.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:07.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:22:07.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:07.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:22:07.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:22:07.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:07.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:22:07.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:22:07.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:07.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:22:07.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:22:07.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:07.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:22:07.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:22:07.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:22:07.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:22:07.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:07.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:22:07.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:22:07.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:07.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:22:07.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:22:07.512 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.512 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:07.513 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.513 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:996 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:07.513 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:22:07.513 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 102.009354699, 2024-09-17T21:22:07.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084493736999999999, 2024-09-17T21:22:07.513 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:07.513 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:07.513 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:07.513 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:07.513 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:07.514 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:07.514 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 996 2024-09-17T21:22:07.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.514 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:07.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:07.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:22:07.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:07.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:22:07.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:07.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431168+0000", 2024-09-17T21:22:07.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4e-07 2024-09-17T21:22:07.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:07.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431172+0000", 2024-09-17T21:22:07.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.608e-06 2024-09-17T21:22:07.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:07.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431180+0000", 2024-09-17T21:22:07.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5650000000000002e-06 2024-09-17T21:22:07.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:07.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431188+0000", 2024-09-17T21:22:07.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:22:07.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515354+0000", 2024-09-17T21:22:07.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084165977000000003 2024-09-17T21:22:07.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:07.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515451+0000", 2024-09-17T21:22:07.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6669000000000006e-05 2024-09-17T21:22:07.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:07.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515661+0000", 2024-09-17T21:22:07.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021021599999999999 2024-09-17T21:22:07.517 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.517 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:07.517 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.517 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:997 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 583680~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:07.517 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:22:07.517 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 102.00932330400001, 2024-09-17T21:22:07.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084772419000000002, 2024-09-17T21:22:07.518 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:07.518 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:07.518 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:07.518 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:07.518 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:07.518 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:07.518 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 997 2024-09-17T21:22:07.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.518 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:07.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:07.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:22:07.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:07.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:22:07.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:07.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:22:07.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0400000000000005e-07 2024-09-17T21:22:07.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:07.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431202+0000", 2024-09-17T21:22:07.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:22:07.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:07.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431208+0000", 2024-09-17T21:22:07.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4740000000000002e-06 2024-09-17T21:22:07.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:07.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431214+0000", 2024-09-17T21:22:07.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6550000000000001e-06 2024-09-17T21:22:07.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515741+0000", 2024-09-17T21:22:07.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084526904 2024-09-17T21:22:07.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:07.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515826+0000", 2024-09-17T21:22:07.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4643999999999998e-05 2024-09-17T21:22:07.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:07.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515971+0000", 2024-09-17T21:22:07.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145444 2024-09-17T21:22:07.521 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.521 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:07.521 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.521 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:998 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 587776~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:07.522 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:22:07.522 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 102.009300332, 2024-09-17T21:22:07.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085108565999999997, 2024-09-17T21:22:07.522 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:07.522 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:07.522 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:07.522 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:07.522 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:07.522 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:07.522 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 998 2024-09-17T21:22:07.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.522 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:07.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:07.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:22:07.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:07.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:22:07.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:07.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:22:07.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:22:07.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:07.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431224+0000", 2024-09-17T21:22:07.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:22:07.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:07.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431231+0000", 2024-09-17T21:22:07.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000002e-06 2024-09-17T21:22:07.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:07.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431236+0000", 2024-09-17T21:22:07.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.648e-06 2024-09-17T21:22:07.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516024+0000", 2024-09-17T21:22:07.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084787584999999999 2024-09-17T21:22:07.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:07.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516122+0000", 2024-09-17T21:22:07.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8076000000000001e-05 2024-09-17T21:22:07.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:07.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516330+0000", 2024-09-17T21:22:07.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020808 2024-09-17T21:22:07.525 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.525 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:07.525 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.526 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1000 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 591872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:07.526 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:22:07.526 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 102.009278217, 2024-09-17T21:22:07.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085389065, 2024-09-17T21:22:07.526 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:07.526 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:07.526 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:07.526 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:07.526 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:07.526 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:07.527 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1000 2024-09-17T21:22:07.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.527 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:07.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:07.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:22:07.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:07.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:22:07.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:07.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:22:07.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:22:07.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:07.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431246+0000", 2024-09-17T21:22:07.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1749999999999999e-06 2024-09-17T21:22:07.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:07.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431253+0000", 2024-09-17T21:22:07.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2790000000000004e-06 2024-09-17T21:22:07.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:07.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431258+0000", 2024-09-17T21:22:07.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3299999999999998e-06 2024-09-17T21:22:07.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516384+0000", 2024-09-17T21:22:07.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085125549999999994 2024-09-17T21:22:07.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:07.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516460+0000", 2024-09-17T21:22:07.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6359999999999994e-05 2024-09-17T21:22:07.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:07.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516633+0000", 2024-09-17T21:22:07.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017285900000000001 2024-09-17T21:22:07.530 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.530 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:07.530 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.531 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1001 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 595968~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:07.531 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:22:07.531 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 102.00925481100001, 2024-09-17T21:22:07.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085614715999999993, 2024-09-17T21:22:07.531 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:07.531 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:07.531 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:07.531 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:07.531 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:07.531 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:07.531 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1001 2024-09-17T21:22:07.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.531 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:07.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:07.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:22:07.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:07.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:22:07.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:07.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438066+0000", 2024-09-17T21:22:07.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067989490000000003 2024-09-17T21:22:07.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:07.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438073+0000", 2024-09-17T21:22:07.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8419999999999999e-06 2024-09-17T21:22:07.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:07.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438090+0000", 2024-09-17T21:22:07.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6657e-05 2024-09-17T21:22:07.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:07.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438099+0000", 2024-09-17T21:22:07.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0529999999999996e-06 2024-09-17T21:22:07.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516678+0000", 2024-09-17T21:22:07.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078579422999999995 2024-09-17T21:22:07.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:07.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516739+0000", 2024-09-17T21:22:07.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1315000000000005e-05 2024-09-17T21:22:07.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:07.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516882+0000", 2024-09-17T21:22:07.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014247700000000001 2024-09-17T21:22:07.534 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.534 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:07.534 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.535 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:19998 3.5 3:a4885450:::rbd_object_map.1152d139f9f9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-17T21:22:07.535 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:22:07.535 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 88.037987565999998, 2024-09-17T21:22:07.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90715489299999996, 2024-09-17T21:22:07.535 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:07.535 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:07.535 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:07.535 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:07.535 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:07.535 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:07.535 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19998 2024-09-17T21:22:07.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.536 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:07.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:07.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:22:07.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:07.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:22:07.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:07.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402537+0000", 2024-09-17T21:22:07.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5600000000000001e-06 2024-09-17T21:22:07.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:07.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402549+0000", 2024-09-17T21:22:07.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1929999999999999e-05 2024-09-17T21:22:07.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:07.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402579+0000", 2024-09-17T21:22:07.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0093000000000001e-05 2024-09-17T21:22:07.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:07.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402595+0000", 2024-09-17T21:22:07.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5855999999999999e-05 2024-09-17T21:22:07.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402731+0000", 2024-09-17T21:22:07.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013610799999999999 2024-09-17T21:22:07.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:07.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402750+0000", 2024-09-17T21:22:07.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8737e-05 2024-09-17T21:22:07.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.305977+0000", 2024-09-17T21:22:07.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90322768399999998 2024-09-17T21:22:07.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:07.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306236+0000", 2024-09-17T21:22:07.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000258111 2024-09-17T21:22:07.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:22:07.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306557+0000", 2024-09-17T21:22:07.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032159799999999998 2024-09-17T21:22:07.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:07.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.308286+0000", 2024-09-17T21:22:07.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017292150000000001 2024-09-17T21:22:07.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:07.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309530+0000", 2024-09-17T21:22:07.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00124409 2024-09-17T21:22:07.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:07.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309630+0000", 2024-09-17T21:22:07.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9689e-05 2024-09-17T21:22:07.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:07.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309689+0000", 2024-09-17T21:22:07.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9222000000000003e-05 2024-09-17T21:22:07.541 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.541 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:07.541 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.541 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:25277 3.1f 3:ff5d7cd5:::rbd_header.11524f65d4ff:head [watch ping cookie 139860845478480 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-17T21:22:07.541 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:22:07.541 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 37.855031496999999, 2024-09-17T21:22:07.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.92185720000000004, 2024-09-17T21:22:07.541 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:07.541 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:07.542 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:07.542 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:07.542 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:07.542 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:07.542 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 25277 2024-09-17T21:22:07.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.542 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:07.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:07.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:22:07.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:07.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:22:07.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:07.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585493+0000", 2024-09-17T21:22:07.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:22:07.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:07.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585505+0000", 2024-09-17T21:22:07.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2466000000000001e-05 2024-09-17T21:22:07.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:07.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585537+0000", 2024-09-17T21:22:07.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1183000000000003e-05 2024-09-17T21:22:07.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:07.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585553+0000", 2024-09-17T21:22:07.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.662e-05 2024-09-17T21:22:07.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585696+0000", 2024-09-17T21:22:07.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014293800000000001 2024-09-17T21:22:07.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:07.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585718+0000", 2024-09-17T21:22:07.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1860000000000001e-05 2024-09-17T21:22:07.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.501188+0000", 2024-09-17T21:22:07.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.91547051300000004 2024-09-17T21:22:07.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:22:07.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.501260+0000", 2024-09-17T21:22:07.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1571999999999998e-05 2024-09-17T21:22:07.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507002+0000", 2024-09-17T21:22:07.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057414850000000002 2024-09-17T21:22:07.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:07.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507143+0000", 2024-09-17T21:22:07.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014140299999999999 2024-09-17T21:22:07.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:07.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507348+0000", 2024-09-17T21:22:07.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000204773 2024-09-17T21:22:07.547 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.547 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:07.547 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.548 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:26042 3.f 3:f24134c1:::rbd_id.image_client.0-clone34:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e186)", 2024-09-17T21:22:07.548 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:22:07.548 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.954343129000002, 2024-09-17T21:22:07.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.91185481000000002, 2024-09-17T21:22:07.548 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:07.548 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:07.548 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:07.548 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:07.548 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:07.548 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:07.548 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 26042 2024-09-17T21:22:07.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.549 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:07.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:07.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:22:07.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:07.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:22:07.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:07.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:07.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486180+0000", 2024-09-17T21:22:07.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5990000000000001e-06 2024-09-17T21:22:07.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:07.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486187+0000", 2024-09-17T21:22:07.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7859999999999999e-06 2024-09-17T21:22:07.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:07.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486206+0000", 2024-09-17T21:22:07.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8641e-05 2024-09-17T21:22:07.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:07.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486219+0000", 2024-09-17T21:22:07.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.308e-05 2024-09-17T21:22:07.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486283+0000", 2024-09-17T21:22:07.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4084000000000005e-05 2024-09-17T21:22:07.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:07.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486290+0000", 2024-09-17T21:22:07.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.584e-06 2024-09-17T21:22:07.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:07.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392182+0000", 2024-09-17T21:22:07.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90589206499999997 2024-09-17T21:22:07.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:07.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392518+0000", 2024-09-17T21:22:07.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033596 2024-09-17T21:22:07.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.553 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:22:07.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392802+0000", 2024-09-17T21:22:07.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028394999999999999 2024-09-17T21:22:07.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.553 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:07.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397705+0000", 2024-09-17T21:22:07.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0049031769999999999 2024-09-17T21:22:07.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:07.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397770+0000", 2024-09-17T21:22:07.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4765999999999996e-05 2024-09-17T21:22:07.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:07.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397927+0000", 2024-09-17T21:22:07.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015699900000000001 2024-09-17T21:22:07.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:07.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:07.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:07.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.398034+0000", 2024-09-17T21:22:07.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000107119 2024-09-17T21:22:07.554 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.554 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:07.554 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.555 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:07.555 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:07.555 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:07.556 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:22:07.557 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:07.553+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:07.657 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:07.653+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:07.731 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:22:07.731 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:07.745 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:22:07.787 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:07.785+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:07.859 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:07.854+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:07.909 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:07.909 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:07.910 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:22:07.910 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:22:07.910 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:07.923 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:22:07.959 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:07.958+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:08.042 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:08.042 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:22:08.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:22:08.042 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:22:08.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.042 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:18003 3.1d 3:b9765580:::rbd_header.11527d2b15f6:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T21:22:08.042 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:22:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 115.681705507, 2024-09-17T21:22:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94165346900000002, 2024-09-17T21:22:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18003 2024-09-17T21:22:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:22:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:22:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357880+0000", 2024-09-17T21:22:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:22:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357891+0000", 2024-09-17T21:22:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0417999999999999e-05 2024-09-17T21:22:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357913+0000", 2024-09-17T21:22:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2215e-05 2024-09-17T21:22:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357922+0000", 2024-09-17T21:22:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4779999999999997e-06 2024-09-17T21:22:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358023+0000", 2024-09-17T21:22:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101025 2024-09-17T21:22:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358040+0000", 2024-09-17T21:22:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6291999999999999e-05 2024-09-17T21:22:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298203+0000", 2024-09-17T21:22:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94016297500000001 2024-09-17T21:22:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298626+0000", 2024-09-17T21:22:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000422853 2024-09-17T21:22:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.299532+0000", 2024-09-17T21:22:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00090657300000000004 2024-09-17T21:22:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15707 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 982016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:22:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 105.51854025999999, 2024-09-17T21:22:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092305228000000003, 2024-09-17T21:22:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15707 2024-09-17T21:22:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:22:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:22:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521045+0000", 2024-09-17T21:22:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4899999999999999e-06 2024-09-17T21:22:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521066+0000", 2024-09-17T21:22:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0236000000000001e-05 2024-09-17T21:22:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521085+0000", 2024-09-17T21:22:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8819999999999999e-05 2024-09-17T21:22:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521100+0000", 2024-09-17T21:22:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5146e-05 2024-09-17T21:22:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612978+0000", 2024-09-17T21:22:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091878847 2024-09-17T21:22:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613079+0000", 2024-09-17T21:22:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100503 2024-09-17T21:22:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613349+0000", 2024-09-17T21:22:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000270186 2024-09-17T21:22:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15708 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 986112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:22:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 105.518462085, 2024-09-17T21:22:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092547217000000001, 2024-09-17T21:22:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15708 2024-09-17T21:22:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:22:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:22:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521123+0000", 2024-09-17T21:22:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.353e-06 2024-09-17T21:22:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521129+0000", 2024-09-17T21:22:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:22:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521145+0000", 2024-09-17T21:22:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.611e-05 2024-09-17T21:22:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521154+0000", 2024-09-17T21:22:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4500000000000004e-06 2024-09-17T21:22:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613397+0000", 2024-09-17T21:22:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092243275 2024-09-17T21:22:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613503+0000", 2024-09-17T21:22:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106013 2024-09-17T21:22:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613669+0000", 2024-09-17T21:22:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016642299999999999 2024-09-17T21:22:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15709 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 990208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:22:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 105.518419069, 2024-09-17T21:22:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092728678999999994, 2024-09-17T21:22:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15709 2024-09-17T21:22:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:22:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:22:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521166+0000", 2024-09-17T21:22:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7000000000000004e-07 2024-09-17T21:22:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521169+0000", 2024-09-17T21:22:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2660000000000002e-06 2024-09-17T21:22:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521181+0000", 2024-09-17T21:22:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.208e-05 2024-09-17T21:22:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521194+0000", 2024-09-17T21:22:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2707e-05 2024-09-17T21:22:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613695+0000", 2024-09-17T21:22:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092500709 2024-09-17T21:22:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613750+0000", 2024-09-17T21:22:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.503e-05 2024-09-17T21:22:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613894+0000", 2024-09-17T21:22:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144117 2024-09-17T21:22:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15710 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 994304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 105.518374503, 2024-09-17T21:22:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092957729000000003, 2024-09-17T21:22:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15710 2024-09-17T21:22:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8000000000000005e-07 2024-09-17T21:22:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521216+0000", 2024-09-17T21:22:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3859999999999998e-06 2024-09-17T21:22:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521234+0000", 2024-09-17T21:22:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:22:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521248+0000", 2024-09-17T21:22:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:22:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613926+0000", 2024-09-17T21:22:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092678324000000006 2024-09-17T21:22:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613974+0000", 2024-09-17T21:22:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7754999999999999e-05 2024-09-17T21:22:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614167+0000", 2024-09-17T21:22:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193508 2024-09-17T21:22:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15711 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 998400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:22:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 105.518317121, 2024-09-17T21:22:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093268000000000004, 2024-09-17T21:22:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15711 2024-09-17T21:22:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:22:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:22:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521268+0000", 2024-09-17T21:22:08.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-17T21:22:08.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521273+0000", 2024-09-17T21:22:08.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:22:08.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521290+0000", 2024-09-17T21:22:08.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7269999999999999e-05 2024-09-17T21:22:08.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521304+0000", 2024-09-17T21:22:08.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3626e-05 2024-09-17T21:22:08.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614220+0000", 2024-09-17T21:22:08.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092916367 2024-09-17T21:22:08.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614322+0000", 2024-09-17T21:22:08.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102212 2024-09-17T21:22:08.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614535+0000", 2024-09-17T21:22:08.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021266199999999999 2024-09-17T21:22:08.070 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.070 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.070 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.070 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15712 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1002496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:08.071 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:22:08.071 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 105.518261626, 2024-09-17T21:22:08.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093568707000000001, 2024-09-17T21:22:08.071 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.071 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.071 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:08.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:08.071 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15712 2024-09-17T21:22:08.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.071 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:22:08.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:22:08.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521324+0000", 2024-09-17T21:22:08.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.17e-06 2024-09-17T21:22:08.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521334+0000", 2024-09-17T21:22:08.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9369999999999999e-06 2024-09-17T21:22:08.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521356+0000", 2024-09-17T21:22:08.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2045999999999999e-05 2024-09-17T21:22:08.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521370+0000", 2024-09-17T21:22:08.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4270000000000001e-05 2024-09-17T21:22:08.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614583+0000", 2024-09-17T21:22:08.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093213220999999999 2024-09-17T21:22:08.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614680+0000", 2024-09-17T21:22:08.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6900999999999999e-05 2024-09-17T21:22:08.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614891+0000", 2024-09-17T21:22:08.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021116200000000001 2024-09-17T21:22:08.075 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.075 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.075 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.075 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15713 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1006592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:08.075 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:22:08.075 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 105.518194924, 2024-09-17T21:22:08.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093834641999999996, 2024-09-17T21:22:08.075 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.075 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.076 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.076 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.076 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:08.076 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:08.076 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15713 2024-09-17T21:22:08.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.076 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:22:08.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:22:08.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521390+0000", 2024-09-17T21:22:08.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:22:08.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521396+0000", 2024-09-17T21:22:08.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:22:08.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521417+0000", 2024-09-17T21:22:08.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1236000000000001e-05 2024-09-17T21:22:08.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521431+0000", 2024-09-17T21:22:08.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3873000000000001e-05 2024-09-17T21:22:08.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614938+0000", 2024-09-17T21:22:08.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093506522999999994 2024-09-17T21:22:08.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615025+0000", 2024-09-17T21:22:08.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7635999999999999e-05 2024-09-17T21:22:08.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615224+0000", 2024-09-17T21:22:08.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019844700000000001 2024-09-17T21:22:08.079 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.079 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.079 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.080 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15714 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:08.080 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:22:08.080 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 105.518131668, 2024-09-17T21:22:08.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094071925000000001, 2024-09-17T21:22:08.080 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.080 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.080 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.080 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.080 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:08.080 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:08.080 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15714 2024-09-17T21:22:08.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.080 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:22:08.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:22:08.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521454+0000", 2024-09-17T21:22:08.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.097e-06 2024-09-17T21:22:08.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521459+0000", 2024-09-17T21:22:08.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2660000000000001e-06 2024-09-17T21:22:08.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521481+0000", 2024-09-17T21:22:08.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1878999999999998e-05 2024-09-17T21:22:08.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521498+0000", 2024-09-17T21:22:08.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6807e-05 2024-09-17T21:22:08.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615271+0000", 2024-09-17T21:22:08.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093773286999999997 2024-09-17T21:22:08.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615366+0000", 2024-09-17T21:22:08.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5322000000000007e-05 2024-09-17T21:22:08.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615524+0000", 2024-09-17T21:22:08.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015826699999999999 2024-09-17T21:22:08.083 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.084 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.084 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.084 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15715 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1014784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:08.084 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:22:08.084 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 105.518067333, 2024-09-17T21:22:08.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094243840999999995, 2024-09-17T21:22:08.084 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.084 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.084 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.084 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.084 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:08.084 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:08.085 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15715 2024-09-17T21:22:08.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.085 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:22:08.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:22:08.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521518+0000", 2024-09-17T21:22:08.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4300000000000001e-06 2024-09-17T21:22:08.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521524+0000", 2024-09-17T21:22:08.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5860000000000004e-06 2024-09-17T21:22:08.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521541+0000", 2024-09-17T21:22:08.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7574000000000001e-05 2024-09-17T21:22:08.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521555+0000", 2024-09-17T21:22:08.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3540000000000001e-05 2024-09-17T21:22:08.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615560+0000", 2024-09-17T21:22:08.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.09400472 2024-09-17T21:22:08.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615614+0000", 2024-09-17T21:22:08.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4693e-05 2024-09-17T21:22:08.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615761+0000", 2024-09-17T21:22:08.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000146298 2024-09-17T21:22:08.088 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.088 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.088 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.088 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15716 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1018880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:08.088 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:22:08.089 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 105.518010988, 2024-09-17T21:22:08.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094400676000000003, 2024-09-17T21:22:08.089 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.089 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.089 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.089 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.089 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:08.089 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:08.089 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15716 2024-09-17T21:22:08.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.089 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:22:08.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:22:08.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521574+0000", 2024-09-17T21:22:08.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1939999999999999e-06 2024-09-17T21:22:08.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521579+0000", 2024-09-17T21:22:08.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:22:08.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521597+0000", 2024-09-17T21:22:08.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8074e-05 2024-09-17T21:22:08.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521613+0000", 2024-09-17T21:22:08.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5316e-05 2024-09-17T21:22:08.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615780+0000", 2024-09-17T21:22:08.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094166874999999997 2024-09-17T21:22:08.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615836+0000", 2024-09-17T21:22:08.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6926999999999999e-05 2024-09-17T21:22:08.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615974+0000", 2024-09-17T21:22:08.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013741400000000001 2024-09-17T21:22:08.093 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.093 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.093 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.093 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15717 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1022976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:08.093 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:22:08.093 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 105.517952292, 2024-09-17T21:22:08.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094586723999999997, 2024-09-17T21:22:08.093 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.094 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.094 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.094 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.094 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:08.094 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:08.094 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15717 2024-09-17T21:22:08.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.094 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:22:08.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:22:08.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521633+0000", 2024-09-17T21:22:08.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:22:08.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521638+0000", 2024-09-17T21:22:08.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:22:08.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521659+0000", 2024-09-17T21:22:08.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-05 2024-09-17T21:22:08.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521675+0000", 2024-09-17T21:22:08.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5688999999999998e-05 2024-09-17T21:22:08.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616008+0000", 2024-09-17T21:22:08.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094333314000000001 2024-09-17T21:22:08.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616066+0000", 2024-09-17T21:22:08.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7797000000000002e-05 2024-09-17T21:22:08.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616219+0000", 2024-09-17T21:22:08.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015281100000000001 2024-09-17T21:22:08.097 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.097 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.098 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.098 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15718 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:08.098 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:22:08.098 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 105.517852217, 2024-09-17T21:22:08.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094703674000000002, 2024-09-17T21:22:08.098 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.098 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.098 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.098 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.098 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:08.098 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:08.098 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15718 2024-09-17T21:22:08.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.099 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:22:08.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:22:08.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521734+0000", 2024-09-17T21:22:08.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.61e-06 2024-09-17T21:22:08.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521740+0000", 2024-09-17T21:22:08.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:22:08.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521754+0000", 2024-09-17T21:22:08.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3793e-05 2024-09-17T21:22:08.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521763+0000", 2024-09-17T21:22:08.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5129999999999996e-06 2024-09-17T21:22:08.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616256+0000", 2024-09-17T21:22:08.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094492919999999994 2024-09-17T21:22:08.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616312+0000", 2024-09-17T21:22:08.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6737000000000002e-05 2024-09-17T21:22:08.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616436+0000", 2024-09-17T21:22:08.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012335800000000001 2024-09-17T21:22:08.102 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.102 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.102 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.102 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15719 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1031168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:08.102 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:22:08.102 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 105.517798812, 2024-09-17T21:22:08.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094888963000000007, 2024-09-17T21:22:08.102 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.103 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.103 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.103 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.103 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:08.103 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:08.103 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15719 2024-09-17T21:22:08.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.103 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:22:08.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:22:08.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521786+0000", 2024-09-17T21:22:08.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.074e-06 2024-09-17T21:22:08.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521791+0000", 2024-09-17T21:22:08.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3030000000000001e-06 2024-09-17T21:22:08.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521804+0000", 2024-09-17T21:22:08.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3027000000000001e-05 2024-09-17T21:22:08.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521811+0000", 2024-09-17T21:22:08.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6090000000000003e-06 2024-09-17T21:22:08.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616466+0000", 2024-09-17T21:22:08.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094654294 2024-09-17T21:22:08.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616523+0000", 2024-09-17T21:22:08.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6953999999999997e-05 2024-09-17T21:22:08.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616674+0000", 2024-09-17T21:22:08.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015170200000000001 2024-09-17T21:22:08.106 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.106 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.106 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.107 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15720 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1035264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:08.107 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:22:08.107 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 105.517761972, 2024-09-17T21:22:08.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095067061999999994, 2024-09-17T21:22:08.107 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.107 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.107 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.107 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.107 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:08.107 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:08.107 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15720 2024-09-17T21:22:08.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.107 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:22:08.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:22:08.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521823+0000", 2024-09-17T21:22:08.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:22:08.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521826+0000", 2024-09-17T21:22:08.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2100000000000002e-06 2024-09-17T21:22:08.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521835+0000", 2024-09-17T21:22:08.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1630000000000002e-06 2024-09-17T21:22:08.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521842+0000", 2024-09-17T21:22:08.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:22:08.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616711+0000", 2024-09-17T21:22:08.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094868859 2024-09-17T21:22:08.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616764+0000", 2024-09-17T21:22:08.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2778999999999997e-05 2024-09-17T21:22:08.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616889+0000", 2024-09-17T21:22:08.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012533100000000001 2024-09-17T21:22:08.111 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.111 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.111 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.111 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15721 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1039360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:08.111 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:22:08.111 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 105.51771486299999, 2024-09-17T21:22:08.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095277219999999996, 2024-09-17T21:22:08.111 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.111 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.111 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.111 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.112 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:08.112 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:08.112 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15721 2024-09-17T21:22:08.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.112 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:22:08.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:22:08.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521871+0000", 2024-09-17T21:22:08.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.443e-06 2024-09-17T21:22:08.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521876+0000", 2024-09-17T21:22:08.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0300000000000001e-06 2024-09-17T21:22:08.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521886+0000", 2024-09-17T21:22:08.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186e-05 2024-09-17T21:22:08.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521893+0000", 2024-09-17T21:22:08.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.08e-06 2024-09-17T21:22:08.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616918+0000", 2024-09-17T21:22:08.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095025223000000006 2024-09-17T21:22:08.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616977+0000", 2024-09-17T21:22:08.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8403999999999999e-05 2024-09-17T21:22:08.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617147+0000", 2024-09-17T21:22:08.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016985399999999999 2024-09-17T21:22:08.115 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.115 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.115 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.115 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15722 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:08.116 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:08.116 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 105.517680414, 2024-09-17T21:22:08.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095459783000000006, 2024-09-17T21:22:08.116 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.116 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.116 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.116 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.116 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:08.116 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:08.116 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15722 2024-09-17T21:22:08.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.116 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:08.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:08.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:08.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7700000000000004e-07 2024-09-17T21:22:08.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521907+0000", 2024-09-17T21:22:08.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.813e-06 2024-09-17T21:22:08.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521922+0000", 2024-09-17T21:22:08.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4533e-05 2024-09-17T21:22:08.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521934+0000", 2024-09-17T21:22:08.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2469999999999999e-05 2024-09-17T21:22:08.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617187+0000", 2024-09-17T21:22:08.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095252311000000006 2024-09-17T21:22:08.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617240+0000", 2024-09-17T21:22:08.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3740999999999999e-05 2024-09-17T21:22:08.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617364+0000", 2024-09-17T21:22:08.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-17T21:22:08.119 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.120 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.120 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.120 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15723 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:08.120 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:22:08.120 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 105.517636548, 2024-09-17T21:22:08.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095653563999999996, 2024-09-17T21:22:08.120 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.120 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.120 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.120 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.120 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:08.120 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:08.121 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15723 2024-09-17T21:22:08.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.121 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:22:08.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:22:08.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521949+0000", 2024-09-17T21:22:08.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:22:08.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521953+0000", 2024-09-17T21:22:08.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5870000000000003e-06 2024-09-17T21:22:08.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521970+0000", 2024-09-17T21:22:08.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7282000000000002e-05 2024-09-17T21:22:08.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521984+0000", 2024-09-17T21:22:08.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:22:08.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617383+0000", 2024-09-17T21:22:08.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095399063000000006 2024-09-17T21:22:08.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617437+0000", 2024-09-17T21:22:08.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3931999999999998e-05 2024-09-17T21:22:08.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617601+0000", 2024-09-17T21:22:08.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016407300000000001 2024-09-17T21:22:08.124 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.124 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.124 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.124 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5629.0:2384 3.1c 3:38c1263e:::rbd_data.1152d37183c3.000000000000000b:head [read 11780096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-17T21:22:08.124 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:22:08.124 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 92.614805718, 2024-09-17T21:22:08.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0246185320000001, 2024-09-17T21:22:08.125 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.125 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.125 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.125 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.125 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5629", 2024-09-17T21:22:08.125 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3798711329", 2024-09-17T21:22:08.125 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2384 2024-09-17T21:22:08.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.125 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:22:08.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:22:08.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424780+0000", 2024-09-17T21:22:08.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.13e-06 2024-09-17T21:22:08.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424785+0000", 2024-09-17T21:22:08.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4739999999999996e-06 2024-09-17T21:22:08.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424798+0000", 2024-09-17T21:22:08.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2704000000000001e-05 2024-09-17T21:22:08.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424806+0000", 2024-09-17T21:22:08.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9899999999999997e-06 2024-09-17T21:22:08.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424868+0000", 2024-09-17T21:22:08.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999997e-05 2024-09-17T21:22:08.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:22:08.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424875+0000", 2024-09-17T21:22:08.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0870000000000001e-06 2024-09-17T21:22:08.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428897+0000", 2024-09-17T21:22:08.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040226080000000004 2024-09-17T21:22:08.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:08.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428913+0000", 2024-09-17T21:22:08.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:22:08.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.448963+0000", 2024-09-17T21:22:08.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0200498659999999 2024-09-17T21:22:08.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449158+0000", 2024-09-17T21:22:08.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019581000000000001 2024-09-17T21:22:08.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449397+0000", 2024-09-17T21:22:08.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023871 2024-09-17T21:22:08.130 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.130 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.130 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.130 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:26043 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_remove_image in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-17T21:22:08.130 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:22:08.130 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.640274598000001, 2024-09-17T21:22:08.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.019978351, 2024-09-17T21:22:08.131 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.131 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.131 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:08.131 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.131 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:08.131 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:08.131 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 26043 2024-09-17T21:22:08.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.131 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:22:08.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:22:08.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399314+0000", 2024-09-17T21:22:08.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.963e-06 2024-09-17T21:22:08.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399330+0000", 2024-09-17T21:22:08.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6684000000000002e-05 2024-09-17T21:22:08.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399372+0000", 2024-09-17T21:22:08.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1819000000000001e-05 2024-09-17T21:22:08.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399396+0000", 2024-09-17T21:22:08.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.406e-05 2024-09-17T21:22:08.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399571+0000", 2024-09-17T21:22:08.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017444800000000001 2024-09-17T21:22:08.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:08.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399595+0000", 2024-09-17T21:22:08.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4709999999999999e-05 2024-09-17T21:22:08.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.412735+0000", 2024-09-17T21:22:08.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0131394869999999 2024-09-17T21:22:08.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.413090+0000", 2024-09-17T21:22:08.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000355006 2024-09-17T21:22:08.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:22:08.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.413659+0000", 2024-09-17T21:22:08.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056922699999999995 2024-09-17T21:22:08.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:08.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.418940+0000", 2024-09-17T21:22:08.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052814109999999997 2024-09-17T21:22:08.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:08.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.418992+0000", 2024-09-17T21:22:08.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1646e-05 2024-09-17T21:22:08.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:08.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.419168+0000", 2024-09-17T21:22:08.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017573400000000001 2024-09-17T21:22:08.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.419288+0000", 2024-09-17T21:22:08.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012015600000000001 2024-09-17T21:22:08.137 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.137 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.137 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.137 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.137 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.137 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:08.139 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:22:08.139 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:08.133+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:08.144 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 0] out_osds: [2, 5] dead_osds: [2] live_osds: [0, 1, 3, 4, 5] 2024-09-17T21:22:08.144 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:22:08.144 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T21:22:08.144 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 0.606214 2024-09-17T21:22:08.145 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 0.6062138262084126 2024-09-17T21:22:08.239 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:08.234+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:08.308 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:08.308 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:08.308 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:22:08.308 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:08.322 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:22:08.367 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:08.365+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:08.440 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:08.437+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:08.446 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:08.446 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:08.446 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:22:08.446 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:22:08.446 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:08.460 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:22:08.547 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:08.545+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:08.559 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:22:08.592 INFO:teuthology.orchestra.run.smithi083.stderr:set osd.3 primary-affinity to 0.606214 (8397282) 2024-09-17T21:22:08.612 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:22:08.635 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:08.635 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:22:08.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:22:08.635 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:22:08.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.635 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:22:08.635 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:22:08.635 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 227.02813257, 2024-09-17T21:22:08.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:22:08.635 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.635 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.635 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:08.636 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:22:08.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:22:08.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:22:08.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:22:08.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:22:08.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:22:08.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:22:08.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:22:08.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:22:08.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:22:08.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:22:08.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:22:08.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:22:08.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:22:08.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:08.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:22:08.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:22:08.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:22:08.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:22:08.643 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.643 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.643 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.643 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:22:08.643 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:22:08.643 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 227.018185934, 2024-09-17T21:22:08.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:22:08.643 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.643 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.643 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:08.643 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:22:08.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:22:08.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:22:08.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:22:08.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:22:08.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:22:08.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:22:08.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:22:08.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:22:08.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:22:08.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:22:08.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:22:08.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:22:08.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:22:08.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:08.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:22:08.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:22:08.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:22:08.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:22:08.647 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.647 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.647 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.647 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:22:08.647 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:22:08.647 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 225.775035094, 2024-09-17T21:22:08.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:22:08.648 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.648 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.648 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:08.648 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.648 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:22:08.648 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:22:08.648 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:22:08.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.648 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:22:08.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:22:08.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:22:08.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:22:08.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:22:08.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:22:08.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:22:08.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:22:08.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:22:08.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:22:08.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:22:08.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:22:08.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:22:08.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:22:08.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:22:08.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:22:08.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:22:08.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:08.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:22:08.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:22:08.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:08.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:22:08.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:22:08.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:08.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:22:08.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:22:08.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:22:08.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:22:08.653 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.653 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.653 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.653 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:22:08.653 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:22:08.653 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 225.76538960799999, 2024-09-17T21:22:08.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:22:08.653 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.653 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.653 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:08.654 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.654 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:22:08.654 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:22:08.654 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:22:08.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.654 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:22:08.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:22:08.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:22:08.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:22:08.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:22:08.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:22:08.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:22:08.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:22:08.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:22:08.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:22:08.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:22:08.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:22:08.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:22:08.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:22:08.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:22:08.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:22:08.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:22:08.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:08.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:22:08.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:22:08.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:08.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:22:08.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:22:08.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:08.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:22:08.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:22:08.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:22:08.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:22:08.658 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.658 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.658 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.659 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:22:08.659 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:22:08.659 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 212.39645357500001, 2024-09-17T21:22:08.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:22:08.659 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.659 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.659 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.659 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.659 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:22:08.659 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:22:08.659 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:22:08.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.659 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:22:08.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:22:08.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:22:08.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:22:08.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:22:08.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:22:08.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:22:08.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:22:08.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:22:08.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:22:08.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:22:08.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:22:08.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:22:08.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:22:08.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:22:08.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:22:08.663 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.663 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.663 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.663 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:08.663 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:22:08.663 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 194.19476901300001, 2024-09-17T21:22:08.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:22:08.663 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.663 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.663 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.663 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.664 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:08.664 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:08.664 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:22:08.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.664 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:22:08.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:22:08.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:22:08.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:22:08.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:22:08.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:22:08.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:22:08.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:22:08.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:22:08.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:22:08.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:22:08.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:22:08.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:22:08.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:22:08.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:22:08.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:22:08.667 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.667 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.667 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.667 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:08.667 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:22:08.667 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 194.194274436, 2024-09-17T21:22:08.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:22:08.667 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.667 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.668 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.668 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.668 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:08.668 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:08.668 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:22:08.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.668 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:22:08.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:22:08.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:22:08.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:22:08.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:22:08.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:22:08.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:22:08.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:22:08.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:22:08.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:22:08.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:22:08.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:22:08.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:22:08.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:22:08.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:22:08.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:22:08.671 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.671 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.671 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.672 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:08.672 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:22:08.672 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 194.19421905499999, 2024-09-17T21:22:08.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:22:08.672 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.672 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.672 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.672 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.672 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:08.672 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:08.672 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:22:08.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.672 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:22:08.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:22:08.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:22:08.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:22:08.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:22:08.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:22:08.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:22:08.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:22:08.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:22:08.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:22:08.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:22:08.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:22:08.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:22:08.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:22:08.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:22:08.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:22:08.676 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.676 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.676 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.676 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:08.676 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:22:08.676 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 194.194162777, 2024-09-17T21:22:08.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:22:08.676 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.676 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.676 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.676 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.676 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:08.677 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:08.677 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:22:08.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.677 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:22:08.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:22:08.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:22:08.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:22:08.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:22:08.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:22:08.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:22:08.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:22:08.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:22:08.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:22:08.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:22:08.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:22:08.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:22:08.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:22:08.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:22:08.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:22:08.680 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.680 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.680 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.680 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:08.680 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:22:08.680 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 194.193213138, 2024-09-17T21:22:08.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:22:08.681 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.681 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.681 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.681 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.681 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:08.681 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:08.681 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:22:08.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.681 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:22:08.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:22:08.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:22:08.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:22:08.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:22:08.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:22:08.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:22:08.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:22:08.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:22:08.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:22:08.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:22:08.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:22:08.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:22:08.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:22:08.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:22:08.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:22:08.684 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.685 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.685 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.685 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:08.685 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:22:08.685 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 194.19313451799999, 2024-09-17T21:22:08.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:22:08.685 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.685 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.685 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.685 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.685 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:08.685 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:08.685 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:22:08.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.686 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:22:08.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:22:08.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:22:08.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:22:08.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:22:08.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:22:08.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:22:08.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:22:08.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:22:08.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:22:08.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:22:08.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:22:08.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:22:08.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:22:08.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:22:08.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:22:08.689 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.689 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.689 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.689 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:08.689 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:22:08.689 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 194.193021579, 2024-09-17T21:22:08.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:22:08.689 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.689 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.689 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.689 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.689 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:08.690 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:08.690 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:22:08.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.690 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:22:08.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:22:08.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:22:08.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:22:08.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:22:08.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:22:08.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:22:08.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:22:08.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:22:08.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:22:08.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:22:08.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:22:08.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:22:08.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:22:08.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:22:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:22:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:22:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 194.192360525, 2024-09-17T21:22:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:22:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.694 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.694 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.694 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:08.694 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:08.694 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:22:08.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.694 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:22:08.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:22:08.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:22:08.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:22:08.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:22:08.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:22:08.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:22:08.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:22:08.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:22:08.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:22:08.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:22:08.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:22:08.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:22:08.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:22:08.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:22:08.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:22:08.697 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.697 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.697 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.697 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:08.697 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:22:08.697 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 194.19229571400001, 2024-09-17T21:22:08.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:22:08.697 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.698 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.698 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.698 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.698 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:08.698 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:08.698 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:22:08.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.698 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:22:08.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:22:08.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:22:08.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:22:08.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:22:08.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:22:08.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:22:08.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:22:08.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:22:08.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:22:08.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:22:08.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:22:08.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:22:08.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:22:08.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:22:08.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:22:08.701 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.701 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.701 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.701 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:08.701 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:22:08.702 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 194.19223473700001, 2024-09-17T21:22:08.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:22:08.702 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.702 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.702 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:08.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:08.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:08.702 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:08.702 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:22:08.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.702 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:22:08.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:22:08.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:22:08.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:22:08.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:22:08.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:22:08.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:22:08.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:22:08.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:22:08.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:22:08.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:22:08.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:22:08.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:22:08.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:22:08.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:22:08.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:22:08.705 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.705 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.705 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.705 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:22:08.706 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:22:08.706 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 139.294663552, 2024-09-17T21:22:08.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:22:08.706 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.706 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.706 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:08.706 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:22:08.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:22:08.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:22:08.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:22:08.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:22:08.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:22:08.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:22:08.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:22:08.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:22:08.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:22:08.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:22:08.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:22:08.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:22:08.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:22:08.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:08.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:22:08.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:22:08.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:22:08.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:22:08.709 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.709 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.710 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.710 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:22:08.710 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:22:08.710 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 139.08600515800001, 2024-09-17T21:22:08.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:22:08.710 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.710 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.710 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:08.710 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:22:08.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:22:08.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:22:08.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:22:08.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:22:08.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:22:08.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:22:08.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:22:08.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:22:08.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:22:08.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:22:08.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:22:08.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:22:08.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:22:08.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:08.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:22:08.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:22:08.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:22:08.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:22:08.714 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.714 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.714 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.714 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:9547 3.2 e147/132 3:42b518d9:::rbd_data.1152f056b73f.0000000000000001:head v 147'513, mlcod=142'511)", 2024-09-17T21:22:08.715 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:22:08.715 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 66.161032035999995, 2024-09-17T21:22:08.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.031059566, 2024-09-17T21:22:08.715 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.715 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.715 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:08.715 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:22:08.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:22:08.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471293+0000", 2024-09-17T21:22:08.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2029999999999999e-06 2024-09-17T21:22:08.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480606+0000", 2024-09-17T21:22:08.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093123839999999999 2024-09-17T21:22:08.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480653+0000", 2024-09-17T21:22:08.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7852000000000002e-05 2024-09-17T21:22:08.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480673+0000", 2024-09-17T21:22:08.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0013e-05 2024-09-17T21:22:08.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480862+0000", 2024-09-17T21:22:08.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018833599999999999 2024-09-17T21:22:08.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480906+0000", 2024-09-17T21:22:08.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4162000000000002e-05 2024-09-17T21:22:08.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:08.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502259+0000", 2024-09-17T21:22:08.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021353154999999999 2024-09-17T21:22:08.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502351+0000", 2024-09-17T21:22:08.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1460999999999997e-05 2024-09-17T21:22:08.718 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.719 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.719 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.719 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:15234 3.1 e147/134 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head v 147'917, mlcod=145'914)", 2024-09-17T21:22:08.719 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:22:08.719 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.528371418000006, 2024-09-17T21:22:08.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045303695999999997, 2024-09-17T21:22:08.719 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.719 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.719 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:08.719 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:22:08.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:22:08.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103954+0000", 2024-09-17T21:22:08.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2259999999999999e-06 2024-09-17T21:22:08.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119105+0000", 2024-09-17T21:22:08.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015150871 2024-09-17T21:22:08.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119141+0000", 2024-09-17T21:22:08.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6239000000000003e-05 2024-09-17T21:22:08.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119159+0000", 2024-09-17T21:22:08.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7946999999999999e-05 2024-09-17T21:22:08.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119344+0000", 2024-09-17T21:22:08.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018524599999999999 2024-09-17T21:22:08.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119399+0000", 2024-09-17T21:22:08.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4922e-05 2024-09-17T21:22:08.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:08.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149165+0000", 2024-09-17T21:22:08.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029765697000000001 2024-09-17T21:22:08.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149255+0000", 2024-09-17T21:22:08.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0548000000000001e-05 2024-09-17T21:22:08.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.723 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.723 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:19859 3.8 e147/133 3:12f72409:::rbd_data.1152f056b73f.0000000000000003:head v 147'482, mlcod=145'479)", 2024-09-17T21:22:08.723 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:22:08.723 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.119480351000007, 2024-09-17T21:22:08.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033476895999999999, 2024-09-17T21:22:08.723 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:08.723 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:08.724 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:08.724 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:08.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:08.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:22:08.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:08.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:22:08.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:08.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:08.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512845+0000", 2024-09-17T21:22:08.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.886e-06 2024-09-17T21:22:08.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:08.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524104+0000", 2024-09-17T21:22:08.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011259668 2024-09-17T21:22:08.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:08.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524148+0000", 2024-09-17T21:22:08.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3386000000000001e-05 2024-09-17T21:22:08.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:08.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524172+0000", 2024-09-17T21:22:08.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4762000000000001e-05 2024-09-17T21:22:08.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:08.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524379+0000", 2024-09-17T21:22:08.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020671599999999999 2024-09-17T21:22:08.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:08.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524421+0000", 2024-09-17T21:22:08.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2182999999999999e-05 2024-09-17T21:22:08.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:08.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546216+0000", 2024-09-17T21:22:08.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021795101000000001 2024-09-17T21:22:08.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:08.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:08.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:08.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546320+0000", 2024-09-17T21:22:08.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103194 2024-09-17T21:22:08.727 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.727 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.727 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.727 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:08.727 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:08.727 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:08.728 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:22:08.729 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:22:08.729 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:08.726+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:08.730 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:22:08.734 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:22:08.734 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:22:08.735 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T21:22:08.757 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:22:08.757 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:22:08.777 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:22:08.777 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:22:08.800 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:22:08.801 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:22:08.801 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:22:08.802 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:22:08.829 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:08.825+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:08.849 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:08.849 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:08.864 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:22:08.930 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:08.925+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:08.977 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:08.977 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:08.991 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:22:09.030 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:09.029+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:09.111 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:09.111 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:09.125 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:22:09.132 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:09.129+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:09.232 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:09.232 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:09.233 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:09.229+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:09.246 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:22:09.333 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:09.329+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:09.366 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:09.366 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:09.380 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:22:09.434 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:09.429+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:09.501 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:09.501 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:09.515 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:22:09.563 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:09.561+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:09.600 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-17T21:22:09.636 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:09.634+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:09.659 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:09.659 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:09.674 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:22:09.763 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:09.761+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:09.838 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:09.833+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:09.846 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:09.846 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:09.860 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:22:09.939 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:09.934+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:10.029 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:10.029 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:10.039 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:10.034+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:10.043 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:22:10.141 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:10.137+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:10.164 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:10.164 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:10.178 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:22:10.242 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:10.238+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:10.343 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:10.338+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:10.346 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:10.346 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:10.360 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:22:10.443 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:10.438+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:10.482 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:10.482 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:10.496 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:22:10.543 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:10.541+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:10.618 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:10.618 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:10.640 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:22:10.644 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:10.638+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:10.745 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:10.741+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:10.803 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:22:10.811 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:10.811 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:10.826 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:22:10.871 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:10.869+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:10.946 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:10.942+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:10.950 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:22:10.951 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:22:10.967 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:22:10.967 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:22:10.967 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T21:22:10.987 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:22:10.987 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:22:10.992 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:10.992 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:11.007 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:22:11.007 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:22:11.008 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:22:11.025 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:22:11.026 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:22:11.029 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:22:11.029 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:22:11.047 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:11.045+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:11.124 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:11.124 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:22:11.125 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:22:11.125 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:11.138 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:22:11.189 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:11.186+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:11.248 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:11.242+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:11.256 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:11.256 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:22:11.256 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:22:11.256 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:22:11.256 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:11.270 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:22:11.349 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:11.345+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:11.436 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:11.436 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:22:11.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:22:11.436 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:22:11.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.436 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:22:11.436 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:22:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 174.921976288, 2024-09-17T21:22:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:22:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:22:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:22:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:22:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:22:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:22:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:22:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:22:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:22:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:22:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:22:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:22:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:22:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:22:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:22:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:22:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:22:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:22:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:22:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:22:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:22:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:22:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2597 3.1f 3:fb0e9615:::rbd_data.1152a8ae2019.000000000000000c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:22:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 150.59353985999999, 2024-09-17T21:22:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097064994000000002, 2024-09-17T21:22:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2597 2024-09-17T21:22:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:22:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:22:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.443 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:11.443 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838628+0000", 2024-09-17T21:22:11.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.187e-06 2024-09-17T21:22:11.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.443 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:11.443 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838654+0000", 2024-09-17T21:22:11.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:22:11.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.443 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:11.443 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838696+0000", 2024-09-17T21:22:11.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1226000000000001e-05 2024-09-17T21:22:11.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:11.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838715+0000", 2024-09-17T21:22:11.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9646000000000001e-05 2024-09-17T21:22:11.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838937+0000", 2024-09-17T21:22:11.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022163 2024-09-17T21:22:11.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:11.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.839284+0000", 2024-09-17T21:22:11.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034753799999999999 2024-09-17T21:22:11.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:11.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935691+0000", 2024-09-17T21:22:11.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.096406360999999996 2024-09-17T21:22:11.445 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.445 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:11.445 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.445 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2641 3.12 3:488a729f:::rbd_data.1152a8ae2019.0000000000000022:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:11.445 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:22:11.446 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 150.59008308700001, 2024-09-17T21:22:11.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098280953000000004, 2024-09-17T21:22:11.446 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:11.446 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:11.446 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:11.446 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:11.446 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:11.446 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:11.446 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2641 2024-09-17T21:22:11.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.446 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:11.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:11.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:22:11.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:11.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:22:11.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:11.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842083+0000", 2024-09-17T21:22:11.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5300000000000002e-07 2024-09-17T21:22:11.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:11.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842088+0000", 2024-09-17T21:22:11.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:22:11.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:11.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842105+0000", 2024-09-17T21:22:11.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6773e-05 2024-09-17T21:22:11.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:11.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842119+0000", 2024-09-17T21:22:11.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:22:11.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842320+0000", 2024-09-17T21:22:11.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020052700000000001 2024-09-17T21:22:11.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:11.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842845+0000", 2024-09-17T21:22:11.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052508600000000002 2024-09-17T21:22:11.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:11.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940363+0000", 2024-09-17T21:22:11.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097518795000000005 2024-09-17T21:22:11.450 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.450 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:11.450 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.450 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2674 3.12 3:4c87e3ac:::rbd_data.1152a8ae2019.0000000000000033:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:11.450 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:22:11.450 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 150.58972044999999, 2024-09-17T21:22:11.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098025125000000005, 2024-09-17T21:22:11.450 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:11.450 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:11.450 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:11.450 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:11.451 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:11.451 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:11.451 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2674 2024-09-17T21:22:11.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.451 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:11.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:11.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:22:11.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:11.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:22:11.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:11.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842447+0000", 2024-09-17T21:22:11.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3999999999999999e-06 2024-09-17T21:22:11.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:11.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842453+0000", 2024-09-17T21:22:11.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:22:11.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:11.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842472+0000", 2024-09-17T21:22:11.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9006000000000001e-05 2024-09-17T21:22:11.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:11.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842487+0000", 2024-09-17T21:22:11.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4866e-05 2024-09-17T21:22:11.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843594+0000", 2024-09-17T21:22:11.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011073249999999999 2024-09-17T21:22:11.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:11.454 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843668+0000", 2024-09-17T21:22:11.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3188000000000005e-05 2024-09-17T21:22:11.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.454 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:11.454 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843893+0000", 2024-09-17T21:22:11.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022503399999999999 2024-09-17T21:22:11.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.454 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:11.454 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850439+0000", 2024-09-17T21:22:11.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065467880000000004 2024-09-17T21:22:11.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.455 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:11.455 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940409+0000", 2024-09-17T21:22:11.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089969511000000002 2024-09-17T21:22:11.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.455 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:11.455 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940444+0000", 2024-09-17T21:22:11.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5345999999999997e-05 2024-09-17T21:22:11.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.455 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:11.455 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940470+0000", 2024-09-17T21:22:11.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5964000000000001e-05 2024-09-17T21:22:11.455 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.455 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:11.456 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.456 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2707 3.1f 3:f805e65e:::rbd_data.1152a8ae2019.0000000000000049:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:11.456 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:22:11.456 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 150.58467378899999, 2024-09-17T21:22:11.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088327300999999997, 2024-09-17T21:22:11.456 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:11.456 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:11.456 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:11.456 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:11.456 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:11.456 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:11.456 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2707 2024-09-17T21:22:11.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.457 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:11.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:11.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:22:11.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:11.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:22:11.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:11.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847494+0000", 2024-09-17T21:22:11.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:22:11.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:11.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847504+0000", 2024-09-17T21:22:11.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:22:11.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:11.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847526+0000", 2024-09-17T21:22:11.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1976000000000001e-05 2024-09-17T21:22:11.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:11.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847541+0000", 2024-09-17T21:22:11.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5277e-05 2024-09-17T21:22:11.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856142+0000", 2024-09-17T21:22:11.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086006710000000007 2024-09-17T21:22:11.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:11.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856435+0000", 2024-09-17T21:22:11.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029301499999999999 2024-09-17T21:22:11.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.460 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:11.460 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935819+0000", 2024-09-17T21:22:11.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079384448999999996 2024-09-17T21:22:11.460 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.460 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:11.460 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.460 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2730 3.12 3:48b3422a:::rbd_data.1152a8ae2019.0000000000000057:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:11.460 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:22:11.460 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 150.57548088999999, 2024-09-17T21:22:11.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083879975999999995, 2024-09-17T21:22:11.461 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:11.461 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:11.461 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:11.461 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:11.461 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:11.461 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:11.461 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2730 2024-09-17T21:22:11.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.461 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:11.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:11.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:22:11.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:11.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:22:11.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:11.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856687+0000", 2024-09-17T21:22:11.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:22:11.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:11.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856697+0000", 2024-09-17T21:22:11.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0207000000000001e-05 2024-09-17T21:22:11.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:11.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856808+0000", 2024-09-17T21:22:11.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011071200000000001 2024-09-17T21:22:11.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:11.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856828+0000", 2024-09-17T21:22:11.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0219000000000001e-05 2024-09-17T21:22:11.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856907+0000", 2024-09-17T21:22:11.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9661999999999997e-05 2024-09-17T21:22:11.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:11.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.857019+0000", 2024-09-17T21:22:11.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011148499999999999 2024-09-17T21:22:11.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:11.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940565+0000", 2024-09-17T21:22:11.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083545791999999994 2024-09-17T21:22:11.465 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.465 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:11.465 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.465 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2743 3.12 3:4d036e62:::rbd_data.1152a8ae2019.000000000000005f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:11.465 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:22:11.465 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 150.57153538899999, 2024-09-17T21:22:11.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080059962999999998, 2024-09-17T21:22:11.465 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:11.466 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:11.466 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:11.466 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:11.466 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:11.466 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:11.466 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2743 2024-09-17T21:22:11.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.466 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:11.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.466 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:11.467 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:22:11.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.467 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:11.467 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:22:11.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.467 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:11.467 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860631+0000", 2024-09-17T21:22:11.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:22:11.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.467 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:11.468 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860636+0000", 2024-09-17T21:22:11.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5279999999999997e-06 2024-09-17T21:22:11.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.468 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:11.468 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860649+0000", 2024-09-17T21:22:11.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-17T21:22:11.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.468 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:11.468 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860655+0000", 2024-09-17T21:22:11.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:22:11.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860731+0000", 2024-09-17T21:22:11.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5920000000000005e-05 2024-09-17T21:22:11.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:11.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.861070+0000", 2024-09-17T21:22:11.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338535 2024-09-17T21:22:11.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:11.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940690+0000", 2024-09-17T21:22:11.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079620503999999995 2024-09-17T21:22:11.470 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.470 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:11.470 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.470 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3378 3.12 3:4832a51b:::rbd_data.1152a8ae2019.000000000000006c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:11.470 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:22:11.470 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 150.54317153900001, 2024-09-17T21:22:11.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051812516000000003, 2024-09-17T21:22:11.470 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:11.470 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:11.470 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:11.470 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:11.470 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:11.470 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:11.471 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3378 2024-09-17T21:22:11.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.471 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:11.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:11.471 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:22:11.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:11.471 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:22:11.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:11.472 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888995+0000", 2024-09-17T21:22:11.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:22:11.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.472 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:11.472 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889006+0000", 2024-09-17T21:22:11.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1012999999999999e-05 2024-09-17T21:22:11.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.472 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:11.472 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889032+0000", 2024-09-17T21:22:11.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6441e-05 2024-09-17T21:22:11.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.473 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:11.473 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889040+0000", 2024-09-17T21:22:11.473 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0639999999999994e-06 2024-09-17T21:22:11.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.473 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.473 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889278+0000", 2024-09-17T21:22:11.473 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023785899999999999 2024-09-17T21:22:11.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:11.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889408+0000", 2024-09-17T21:22:11.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129219 2024-09-17T21:22:11.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:11.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889723+0000", 2024-09-17T21:22:11.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000315264 2024-09-17T21:22:11.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:11.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.891972+0000", 2024-09-17T21:22:11.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002249633 2024-09-17T21:22:11.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:11.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940733+0000", 2024-09-17T21:22:11.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048760626000000001 2024-09-17T21:22:11.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:11.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940785+0000", 2024-09-17T21:22:11.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1906000000000001e-05 2024-09-17T21:22:11.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:11.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940807+0000", 2024-09-17T21:22:11.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1554999999999999e-05 2024-09-17T21:22:11.475 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.476 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:11.476 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.476 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3379 3.12 3:4e83fd71:::rbd_data.1152a8ae2019.000000000000006d:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:11.476 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:22:11.476 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 150.543087332, 2024-09-17T21:22:11.476 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051804144000000003, 2024-09-17T21:22:11.476 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:11.476 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:11.476 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:11.476 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:11.476 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:11.476 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:11.477 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3379 2024-09-17T21:22:11.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.477 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:11.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.477 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:11.477 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:22:11.477 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.477 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:11.477 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:22:11.477 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:11.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889079+0000", 2024-09-17T21:22:11.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1800000000000004e-07 2024-09-17T21:22:11.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:11.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889083+0000", 2024-09-17T21:22:11.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:22:11.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:11.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889094+0000", 2024-09-17T21:22:11.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1562999999999999e-05 2024-09-17T21:22:11.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:11.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889102+0000", 2024-09-17T21:22:11.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.401e-06 2024-09-17T21:22:11.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:22:11.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014279259999999999 2024-09-17T21:22:11.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:11.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890609+0000", 2024-09-17T21:22:11.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9566000000000004e-05 2024-09-17T21:22:11.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:11.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890789+0000", 2024-09-17T21:22:11.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017949700000000001 2024-09-17T21:22:11.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:11.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.893929+0000", 2024-09-17T21:22:11.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003140538 2024-09-17T21:22:11.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:11.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940834+0000", 2024-09-17T21:22:11.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046904811999999997 2024-09-17T21:22:11.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.481 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:11.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940860+0000", 2024-09-17T21:22:11.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5341999999999999e-05 2024-09-17T21:22:11.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.481 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:11.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940882+0000", 2024-09-17T21:22:11.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2878999999999999e-05 2024-09-17T21:22:11.481 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.481 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:11.481 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.482 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:2644 3.1e 3:79615585:::rbd_data.115266d0b9f2.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2475520~14848,write 3106304~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:22:11.482 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:22:11.482 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 141.92045892900001, 2024-09-17T21:22:11.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050733484000000002, 2024-09-17T21:22:11.482 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:11.482 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:11.482 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:11.482 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:11.482 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:22:11.482 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:22:11.482 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2644 2024-09-17T21:22:11.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.482 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:11.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:11.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:22:11.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:11.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:22:11.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:11.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511708+0000", 2024-09-17T21:22:11.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-17T21:22:11.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:11.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522196+0000", 2024-09-17T21:22:11.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010487712999999999 2024-09-17T21:22:11.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:11.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522223+0000", 2024-09-17T21:22:11.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7064999999999999e-05 2024-09-17T21:22:11.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:11.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522236+0000", 2024-09-17T21:22:11.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2931e-05 2024-09-17T21:22:11.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522348+0000", 2024-09-17T21:22:11.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111906 2024-09-17T21:22:11.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:11.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522554+0000", 2024-09-17T21:22:11.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020632699999999999 2024-09-17T21:22:11.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:22:11.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.523185+0000", 2024-09-17T21:22:11.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063081400000000005 2024-09-17T21:22:11.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:11.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.547011+0000", 2024-09-17T21:22:11.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023825570000000001 2024-09-17T21:22:11.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:11.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562268+0000", 2024-09-17T21:22:11.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015257514 2024-09-17T21:22:11.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:11.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562367+0000", 2024-09-17T21:22:11.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8678000000000006e-05 2024-09-17T21:22:11.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:11.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562440+0000", 2024-09-17T21:22:11.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3337000000000006e-05 2024-09-17T21:22:11.487 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.487 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:11.487 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.487 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:231 3.1 3:856fed78:::rbd_data.115220dd393e.000000000000000b:head [read 0~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:22:11.487 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:22:11.487 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 113.47770366, 2024-09-17T21:22:11.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34254251699999999, 2024-09-17T21:22:11.487 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:11.487 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:11.487 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:11.488 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:11.488 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:11.488 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:11.488 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 231 2024-09-17T21:22:11.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.488 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:11.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:11.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:22:11.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:11.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:22:11.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:11.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954464+0000", 2024-09-17T21:22:11.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:22:11.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:11.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954475+0000", 2024-09-17T21:22:11.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1146e-05 2024-09-17T21:22:11.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:11.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954506+0000", 2024-09-17T21:22:11.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0292999999999999e-05 2024-09-17T21:22:11.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:11.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954524+0000", 2024-09-17T21:22:11.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8430000000000001e-05 2024-09-17T21:22:11.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954661+0000", 2024-09-17T21:22:11.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136403 2024-09-17T21:22:11.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:11.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954680+0000", 2024-09-17T21:22:11.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9743000000000001e-05 2024-09-17T21:22:11.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.296896+0000", 2024-09-17T21:22:11.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34221550499999998 2024-09-17T21:22:11.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:11.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297004+0000", 2024-09-17T21:22:11.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010865399999999999 2024-09-17T21:22:11.491 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.491 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:11.491 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.491 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:56 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:22:11.492 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:22:11.492 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 113.322069963, 2024-09-17T21:22:11.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18718773899999999, 2024-09-17T21:22:11.492 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:11.492 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:11.492 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:11.492 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:11.492 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:22:11.492 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:22:11.492 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 56 2024-09-17T21:22:11.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.492 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:11.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:11.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:22:11.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:11.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:22:11.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:11.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110098+0000", 2024-09-17T21:22:11.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:22:11.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:11.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110111+0000", 2024-09-17T21:22:11.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2289999999999999e-05 2024-09-17T21:22:11.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:11.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110143+0000", 2024-09-17T21:22:11.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1979e-05 2024-09-17T21:22:11.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:11.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110161+0000", 2024-09-17T21:22:11.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7839e-05 2024-09-17T21:22:11.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.495 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.495 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110319+0000", 2024-09-17T21:22:11.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015835400000000001 2024-09-17T21:22:11.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.495 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:11.495 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110353+0000", 2024-09-17T21:22:11.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3849000000000001e-05 2024-09-17T21:22:11.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.495 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.495 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297041+0000", 2024-09-17T21:22:11.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18668842299999999 2024-09-17T21:22:11.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.496 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:11.496 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297169+0000", 2024-09-17T21:22:11.496 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012786199999999999 2024-09-17T21:22:11.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.496 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:11.496 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297283+0000", 2024-09-17T21:22:11.496 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011430300000000001 2024-09-17T21:22:11.496 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.496 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:11.496 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.496 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:68 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:22:11.496 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:22:11.497 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 113.31352903200001, 2024-09-17T21:22:11.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17885582999999999, 2024-09-17T21:22:11.497 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:11.497 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:11.497 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:11.497 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:11.497 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:22:11.497 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:22:11.497 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 68 2024-09-17T21:22:11.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.497 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:11.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:11.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:22:11.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:11.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:22:11.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:11.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118639+0000", 2024-09-17T21:22:11.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.306e-06 2024-09-17T21:22:11.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:11.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118651+0000", 2024-09-17T21:22:11.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2567e-05 2024-09-17T21:22:11.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:11.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118682+0000", 2024-09-17T21:22:11.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0502e-05 2024-09-17T21:22:11.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:11.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118707+0000", 2024-09-17T21:22:11.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5463e-05 2024-09-17T21:22:11.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118889+0000", 2024-09-17T21:22:11.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018150999999999999 2024-09-17T21:22:11.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:11.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118910+0000", 2024-09-17T21:22:11.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:22:11.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297313+0000", 2024-09-17T21:22:11.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17840366699999999 2024-09-17T21:22:11.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:11.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297389+0000", 2024-09-17T21:22:11.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-17T21:22:11.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:11.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297492+0000", 2024-09-17T21:22:11.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103019 2024-09-17T21:22:11.501 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.501 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:11.501 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.502 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5890.0:15234 3.1 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8451072~29696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-17T21:22:11.502 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:22:11.502 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 68.338880219999993, 2024-09-17T21:22:11.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.057026052000000001, 2024-09-17T21:22:11.502 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:11.502 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:11.502 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:11.502 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:11.502 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5890", 2024-09-17T21:22:11.502 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3064344947", 2024-09-17T21:22:11.502 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 15234 2024-09-17T21:22:11.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.502 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:11.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:11.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:22:11.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:11.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:22:11.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:11.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093289+0000", 2024-09-17T21:22:11.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4939999999999999e-06 2024-09-17T21:22:11.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:11.504 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103074+0000", 2024-09-17T21:22:11.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00978564 2024-09-17T21:22:11.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:11.504 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103112+0000", 2024-09-17T21:22:11.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7246999999999998e-05 2024-09-17T21:22:11.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:11.504 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103128+0000", 2024-09-17T21:22:11.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5812999999999999e-05 2024-09-17T21:22:11.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.505 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103207+0000", 2024-09-17T21:22:11.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9785999999999998e-05 2024-09-17T21:22:11.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.505 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:11.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103315+0000", 2024-09-17T21:22:11.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107396 2024-09-17T21:22:11.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.505 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:22:11.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103957+0000", 2024-09-17T21:22:11.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000642524 2024-09-17T21:22:11.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:11.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.123856+0000", 2024-09-17T21:22:11.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019898376999999998 2024-09-17T21:22:11.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:11.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150190+0000", 2024-09-17T21:22:11.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026334682000000002 2024-09-17T21:22:11.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:11.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150255+0000", 2024-09-17T21:22:11.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4770999999999994e-05 2024-09-17T21:22:11.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.507 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:11.507 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150311+0000", 2024-09-17T21:22:11.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6322e-05 2024-09-17T21:22:11.507 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.507 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:11.507 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.507 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4239 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 424960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:22:11.507 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:22:11.507 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 58.678482563000003, 2024-09-17T21:22:11.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050123167000000003, 2024-09-17T21:22:11.508 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:11.508 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:11.508 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:11.508 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:11.508 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:22:11.508 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:22:11.508 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4239 2024-09-17T21:22:11.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.508 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:11.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.508 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:11.508 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:22:11.508 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:11.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:22:11.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:11.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753684+0000", 2024-09-17T21:22:11.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.37e-07 2024-09-17T21:22:11.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:11.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753686+0000", 2024-09-17T21:22:11.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.322e-06 2024-09-17T21:22:11.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:11.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753693+0000", 2024-09-17T21:22:11.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0890000000000002e-06 2024-09-17T21:22:11.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:11.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753699+0000", 2024-09-17T21:22:11.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:22:11.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803390+0000", 2024-09-17T21:22:11.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049690506000000002 2024-09-17T21:22:11.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:11.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803569+0000", 2024-09-17T21:22:11.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179784 2024-09-17T21:22:11.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:11.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803806+0000", 2024-09-17T21:22:11.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023683499999999999 2024-09-17T21:22:11.511 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.511 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:11.511 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.511 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4240 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 413696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:22:11.512 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:22:11.512 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 58.678458835000001, 2024-09-17T21:22:11.512 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050331353000000002, 2024-09-17T21:22:11.512 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:11.512 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:11.512 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:11.512 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:11.512 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:22:11.512 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:22:11.512 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4240 2024-09-17T21:22:11.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.512 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:11.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.512 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:11.512 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:22:11.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.513 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:11.513 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:22:11.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.513 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:11.513 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:22:11.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5700000000000002e-07 2024-09-17T21:22:11.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.513 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:11.514 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753710+0000", 2024-09-17T21:22:11.514 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.385e-06 2024-09-17T21:22:11.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.514 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:11.514 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753717+0000", 2024-09-17T21:22:11.514 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.832e-06 2024-09-17T21:22:11.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.514 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:11.514 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753722+0000", 2024-09-17T21:22:11.514 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8050000000000003e-06 2024-09-17T21:22:11.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.514 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.514 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803880+0000", 2024-09-17T21:22:11.515 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050157641000000003 2024-09-17T21:22:11.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.515 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:11.515 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803945+0000", 2024-09-17T21:22:11.515 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5357000000000005e-05 2024-09-17T21:22:11.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.515 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:11.515 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804038+0000", 2024-09-17T21:22:11.515 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2775999999999994e-05 2024-09-17T21:22:11.515 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.515 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:11.515 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.516 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4241 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 415744~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:22:11.516 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:22:11.516 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 58.678435718999999, 2024-09-17T21:22:11.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050578790999999998, 2024-09-17T21:22:11.516 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:11.516 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:11.516 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:11.516 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:11.516 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:22:11.516 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:22:11.516 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4241 2024-09-17T21:22:11.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.516 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:11.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:11.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:22:11.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:11.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:22:11.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:11.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:22:11.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5700000000000002e-07 2024-09-17T21:22:11.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.518 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:11.518 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753733+0000", 2024-09-17T21:22:11.518 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2570000000000002e-06 2024-09-17T21:22:11.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.518 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:11.518 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753740+0000", 2024-09-17T21:22:11.518 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.849e-06 2024-09-17T21:22:11.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.518 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:11.518 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753745+0000", 2024-09-17T21:22:11.518 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7450000000000001e-06 2024-09-17T21:22:11.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.519 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.519 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804115+0000", 2024-09-17T21:22:11.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050370038999999998 2024-09-17T21:22:11.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.519 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:11.519 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804223+0000", 2024-09-17T21:22:11.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010786 2024-09-17T21:22:11.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.519 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:11.519 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804309+0000", 2024-09-17T21:22:11.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5483999999999997e-05 2024-09-17T21:22:11.519 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.519 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:11.520 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.520 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4242 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 414720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:22:11.520 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:22:11.520 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 58.678412831000003, 2024-09-17T21:22:11.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050748148999999999, 2024-09-17T21:22:11.520 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:11.520 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:11.520 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:11.520 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:11.520 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:22:11.520 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:22:11.520 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4242 2024-09-17T21:22:11.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.521 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:11.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:11.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:22:11.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:11.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:22:11.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:11.522 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:22:11.522 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6599999999999996e-07 2024-09-17T21:22:11.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.522 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.522 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:11.522 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753756+0000", 2024-09-17T21:22:11.522 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.368e-06 2024-09-17T21:22:11.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.522 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.522 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:11.522 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753763+0000", 2024-09-17T21:22:11.523 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9349999999999997e-06 2024-09-17T21:22:11.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.523 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.523 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:11.523 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753768+0000", 2024-09-17T21:22:11.523 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7620000000000001e-06 2024-09-17T21:22:11.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.523 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.523 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.523 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804343+0000", 2024-09-17T21:22:11.523 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050575050000000003 2024-09-17T21:22:11.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.523 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:11.524 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804397+0000", 2024-09-17T21:22:11.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4070000000000002e-05 2024-09-17T21:22:11.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:11.524 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804501+0000", 2024-09-17T21:22:11.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103398 2024-09-17T21:22:11.524 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.524 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:11.524 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.524 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25642 3.1 3.6a5ad301 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e177)", 2024-09-17T21:22:11.525 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:34.316616+0000", 2024-09-17T21:22:11.525 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.115549248000001, 2024-09-17T21:22:11.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99065658300000003, 2024-09-17T21:22:11.525 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:11.525 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:11.525 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:11.525 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:11.525 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:22:11.525 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:11.525 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25642 2024-09-17T21:22:11.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.525 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:11.525 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.525 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:11.525 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316616+0000", 2024-09-17T21:22:11.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:11.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316616+0000", 2024-09-17T21:22:11.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:11.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316618+0000", 2024-09-17T21:22:11.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9309999999999998e-06 2024-09-17T21:22:11.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:11.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316624+0000", 2024-09-17T21:22:11.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2789999999999998e-06 2024-09-17T21:22:11.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:11.527 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316634+0000", 2024-09-17T21:22:11.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0774000000000001e-05 2024-09-17T21:22:11.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:11.527 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316644+0000", 2024-09-17T21:22:11.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0043e-05 2024-09-17T21:22:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316767+0000", 2024-09-17T21:22:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012289199999999999 2024-09-17T21:22:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316777+0000", 2024-09-17T21:22:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6199999999999994e-06 2024-09-17T21:22:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.528 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.307241+0000", 2024-09-17T21:22:11.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99046395099999995 2024-09-17T21:22:11.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.529 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.529 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:11.530 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.307273+0000", 2024-09-17T21:22:11.530 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2092999999999999e-05 2024-09-17T21:22:11.530 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.530 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:11.530 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.530 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25905 3.6 3:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-17T21:22:11.530 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:39.359572+0000", 2024-09-17T21:22:11.530 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 32.072593965999999, 2024-09-17T21:22:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96065702100000006, 2024-09-17T21:22:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:22:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25905 2024-09-17T21:22:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359572+0000", 2024-09-17T21:22:11.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359572+0000", 2024-09-17T21:22:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359573+0000", 2024-09-17T21:22:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5370000000000001e-06 2024-09-17T21:22:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359579+0000", 2024-09-17T21:22:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8039999999999998e-06 2024-09-17T21:22:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359603+0000", 2024-09-17T21:22:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4533999999999999e-05 2024-09-17T21:22:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359613+0000", 2024-09-17T21:22:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2820000000000002e-06 2024-09-17T21:22:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359686+0000", 2024-09-17T21:22:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3301999999999997e-05 2024-09-17T21:22:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359693+0000", 2024-09-17T21:22:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3679999999999996e-06 2024-09-17T21:22:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:40.319999+0000", 2024-09-17T21:22:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96030524500000003 2024-09-17T21:22:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:40.320229+0000", 2024-09-17T21:22:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022994899999999999 2024-09-17T21:22:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:11.535 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:11.536 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:22:11.537 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:11.534+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:11.629 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:22:11.637 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:11.633+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:11.715 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:11.715 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:22:11.715 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:22:11.715 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:11.731 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:22:11.737 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:11.733+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:11.838 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:11.834+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:11.859 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:11.860 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:22:11.860 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:22:11.860 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:22:11.860 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:11.874 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:22:11.961 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:11.958+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:12.033 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:12.034 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:22:12.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:22:12.034 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:22:12.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.034 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:22:12.034 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:22:12.034 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 212.22032276900001, 2024-09-17T21:22:12.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:22:12.034 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:12.034 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:12.034 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:12.034 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:12.034 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:22:12.035 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:22:12.035 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:22:12.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.035 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:12.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:12.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:22:12.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:12.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:22:12.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:12.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:22:12.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:22:12.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:12.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:22:12.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:22:12.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:12.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:22:12.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:22:12.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:22:12.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:22:12.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:22:12.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:22:12.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:12.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:22:12.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:22:12.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.038 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.038 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:22:12.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:22:12.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.038 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:12.038 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:22:12.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:22:12.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.038 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:12.038 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:22:12.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:22:12.038 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.039 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:12.039 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.039 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:22:12.039 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:22:12.039 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 198.66522098300001, 2024-09-17T21:22:12.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:22:12.039 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:12.039 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:12.039 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:12.039 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:12.039 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:22:12.039 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:12.039 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:22:12.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.040 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:12.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:12.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:22:12.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:12.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:22:12.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:12.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:22:12.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:22:12.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:12.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:22:12.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:22:12.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:12.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:22:12.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:22:12.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:22:12.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:22:12.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:22:12.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:22:12.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:12.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:22:12.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:22:12.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:22:12.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:22:12.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:12.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:22:12.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:22:12.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:22:12.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:22:12.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:22:12.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:12.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:22:12.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:22:12.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:12.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:22:12.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:22:12.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:12.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:22:12.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:22:12.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:12.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:22:12.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:22:12.045 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.045 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:12.045 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.045 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:793 3.9 3:95f63599:::rbd_data.1152eb799094.0000000000000007:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:22:12.045 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:22:12.045 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 115.59816482799999, 2024-09-17T21:22:12.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88245766400000003, 2024-09-17T21:22:12.045 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:12.045 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:12.045 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:12.045 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:12.045 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:22:12.046 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:22:12.046 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 793 2024-09-17T21:22:12.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.046 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:12.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:12.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:22:12.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:12.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:22:12.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:12.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431941+0000", 2024-09-17T21:22:12.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.864e-06 2024-09-17T21:22:12.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:12.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431953+0000", 2024-09-17T21:22:12.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1486e-05 2024-09-17T21:22:12.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:12.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431984+0000", 2024-09-17T21:22:12.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1075999999999999e-05 2024-09-17T21:22:12.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431999+0000", 2024-09-17T21:22:12.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.503e-05 2024-09-17T21:22:12.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432095+0000", 2024-09-17T21:22:12.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5698000000000004e-05 2024-09-17T21:22:12.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:12.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432109+0000", 2024-09-17T21:22:12.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4188e-05 2024-09-17T21:22:12.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.049 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.049 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.309642+0000", 2024-09-17T21:22:12.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87753353000000001 2024-09-17T21:22:12.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.049 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:12.049 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310104+0000", 2024-09-17T21:22:12.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046103600000000001 2024-09-17T21:22:12.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.049 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:12.049 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310630+0000", 2024-09-17T21:22:12.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052682800000000002 2024-09-17T21:22:12.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.050 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:12.050 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313100+0000", 2024-09-17T21:22:12.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024693369999999998 2024-09-17T21:22:12.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.050 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:12.050 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313935+0000", 2024-09-17T21:22:12.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083540100000000005 2024-09-17T21:22:12.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:12.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314086+0000", 2024-09-17T21:22:12.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015053399999999999 2024-09-17T21:22:12.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:12.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314396+0000", 2024-09-17T21:22:12.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031065599999999997 2024-09-17T21:22:12.051 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.051 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:12.051 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.051 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:800 3.19 3:9dc17f12:::rbd_data.1152eb799094.000000000000000e:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:22:12.051 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:22:12.052 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 115.597917583, 2024-09-17T21:22:12.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88154663799999999, 2024-09-17T21:22:12.052 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:12.052 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:12.052 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:12.052 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:12.052 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:22:12.052 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:22:12.052 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 800 2024-09-17T21:22:12.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.052 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:12.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:12.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:22:12.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:12.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:22:12.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:12.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432187+0000", 2024-09-17T21:22:12.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4100000000000001e-06 2024-09-17T21:22:12.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:12.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432194+0000", 2024-09-17T21:22:12.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0389999999999999e-06 2024-09-17T21:22:12.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:12.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432216+0000", 2024-09-17T21:22:12.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1614e-05 2024-09-17T21:22:12.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432232+0000", 2024-09-17T21:22:12.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5739e-05 2024-09-17T21:22:12.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432405+0000", 2024-09-17T21:22:12.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017355300000000001 2024-09-17T21:22:12.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.055 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:12.055 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432432+0000", 2024-09-17T21:22:12.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6979e-05 2024-09-17T21:22:12.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.055 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.055 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.307767+0000", 2024-09-17T21:22:12.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87533472899999998 2024-09-17T21:22:12.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.055 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:12.055 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308192+0000", 2024-09-17T21:22:12.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042485999999999999 2024-09-17T21:22:12.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:12.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308727+0000", 2024-09-17T21:22:12.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00053550400000000001 2024-09-17T21:22:12.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:12.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310347+0000", 2024-09-17T21:22:12.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016198670000000001 2024-09-17T21:22:12.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:12.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313424+0000", 2024-09-17T21:22:12.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030764640000000001 2024-09-17T21:22:12.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.057 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:12.057 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313510+0000", 2024-09-17T21:22:12.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5938000000000003e-05 2024-09-17T21:22:12.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.057 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:12.057 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313733+0000", 2024-09-17T21:22:12.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022294200000000001 2024-09-17T21:22:12.057 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.057 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:12.057 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.057 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:805 3.19 3:9cab2ed6:::rbd_data.1152eb799094.0000000000000013:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:22:12.057 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:22:12.058 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 115.597713564, 2024-09-17T21:22:12.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88274355699999996, 2024-09-17T21:22:12.058 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:12.058 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:12.058 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:12.058 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:12.058 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:22:12.058 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:22:12.058 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 805 2024-09-17T21:22:12.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.058 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:12.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:12.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:22:12.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:12.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:22:12.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:12.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432391+0000", 2024-09-17T21:22:12.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3e-06 2024-09-17T21:22:12.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:12.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432398+0000", 2024-09-17T21:22:12.060 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4489999999999996e-06 2024-09-17T21:22:12.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.060 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.060 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:12.060 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432415+0000", 2024-09-17T21:22:12.060 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7750000000000001e-05 2024-09-17T21:22:12.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.060 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.060 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.060 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432430+0000", 2024-09-17T21:22:12.060 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.449e-05 2024-09-17T21:22:12.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.060 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.060 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.060 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432635+0000", 2024-09-17T21:22:12.061 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205031 2024-09-17T21:22:12.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.061 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:12.061 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432657+0000", 2024-09-17T21:22:12.061 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2064000000000001e-05 2024-09-17T21:22:12.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.061 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.061 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310444+0000", 2024-09-17T21:22:12.061 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87778699199999999 2024-09-17T21:22:12.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.062 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:12.062 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310849+0000", 2024-09-17T21:22:12.062 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00040497800000000001 2024-09-17T21:22:12.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.062 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.062 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:12.062 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.311396+0000", 2024-09-17T21:22:12.062 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00054729099999999999 2024-09-17T21:22:12.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.062 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.062 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:12.062 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313847+0000", 2024-09-17T21:22:12.062 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024504940000000001 2024-09-17T21:22:12.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:12.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314879+0000", 2024-09-17T21:22:12.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001031817 2024-09-17T21:22:12.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:12.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314964+0000", 2024-09-17T21:22:12.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5685000000000004e-05 2024-09-17T21:22:12.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:12.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.315133+0000", 2024-09-17T21:22:12.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000169216 2024-09-17T21:22:12.064 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.064 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:12.064 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.064 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12089 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 60416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:12.064 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:22:12.064 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.260488532, 2024-09-17T21:22:12.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088293233999999998, 2024-09-17T21:22:12.064 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:12.064 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:12.064 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:12.065 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:12.065 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:12.065 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:12.065 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12089 2024-09-17T21:22:12.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.065 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:12.065 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.066 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:12.066 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:22:12.066 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.066 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:12.066 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:22:12.066 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:12.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769616+0000", 2024-09-17T21:22:12.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:22:12.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.068 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:12.068 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769620+0000", 2024-09-17T21:22:12.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5140000000000002e-06 2024-09-17T21:22:12.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.069 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:12.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769633+0000", 2024-09-17T21:22:12.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2849e-05 2024-09-17T21:22:12.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.069 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.069 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769644+0000", 2024-09-17T21:22:12.070 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1471000000000001e-05 2024-09-17T21:22:12.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.070 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.070 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.070 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857564+0000", 2024-09-17T21:22:12.070 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087919445999999998 2024-09-17T21:22:12.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.070 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:12.071 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857716+0000", 2024-09-17T21:22:12.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000152065 2024-09-17T21:22:12.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:12.071 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857908+0000", 2024-09-17T21:22:12.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019252599999999999 2024-09-17T21:22:12.072 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.072 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:12.072 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.073 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12090 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 64512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:12.073 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:22:12.073 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.26044214700001, 2024-09-17T21:22:12.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088582672000000001, 2024-09-17T21:22:12.073 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:12.073 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:12.074 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:12.074 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:12.074 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:12.074 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:12.074 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12090 2024-09-17T21:22:12.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.074 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:12.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.075 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:12.075 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:22:12.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.075 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:12.076 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:22:12.076 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.076 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.076 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:12.076 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769662+0000", 2024-09-17T21:22:12.076 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:22:12.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:12.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769683+0000", 2024-09-17T21:22:12.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0979e-05 2024-09-17T21:22:12.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:12.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769699+0000", 2024-09-17T21:22:12.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5446000000000001e-05 2024-09-17T21:22:12.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769708+0000", 2024-09-17T21:22:12.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1679999999999993e-06 2024-09-17T21:22:12.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.079 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.079 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857976+0000", 2024-09-17T21:22:12.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088267795999999996 2024-09-17T21:22:12.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:12.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858095+0000", 2024-09-17T21:22:12.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119147 2024-09-17T21:22:12.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:12.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858244+0000", 2024-09-17T21:22:12.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014917000000000001 2024-09-17T21:22:12.080 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.081 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:12.081 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.081 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12091 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 68608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:12.081 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:22:12.081 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.260370763, 2024-09-17T21:22:12.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088832752000000001, 2024-09-17T21:22:12.081 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:12.081 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:12.081 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:12.081 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:12.082 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:12.082 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:12.082 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12091 2024-09-17T21:22:12.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.082 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:12.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.082 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:12.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:22:12.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.082 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:12.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:22:12.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.083 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:12.083 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769734+0000", 2024-09-17T21:22:12.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:22:12.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.083 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:12.083 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769739+0000", 2024-09-17T21:22:12.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8080000000000004e-06 2024-09-17T21:22:12.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:12.084 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769764+0000", 2024-09-17T21:22:12.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4839999999999999e-05 2024-09-17T21:22:12.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.084 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769777+0000", 2024-09-17T21:22:12.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3037000000000001e-05 2024-09-17T21:22:12.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858295+0000", 2024-09-17T21:22:12.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088518475999999999 2024-09-17T21:22:12.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:12.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858419+0000", 2024-09-17T21:22:12.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012378700000000001 2024-09-17T21:22:12.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:12.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858565+0000", 2024-09-17T21:22:12.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014630099999999999 2024-09-17T21:22:12.086 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.086 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:12.086 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.086 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12092 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:12.086 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:22:12.086 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.26028843899999, 2024-09-17T21:22:12.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089191555000000006, 2024-09-17T21:22:12.086 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:12.086 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:12.086 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:12.087 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:12.087 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:12.087 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:12.087 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12092 2024-09-17T21:22:12.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.087 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:12.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.087 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:12.087 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:22:12.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.087 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:12.088 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:22:12.088 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.088 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:12.088 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769816+0000", 2024-09-17T21:22:12.088 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7199999999999997e-07 2024-09-17T21:22:12.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.088 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:12.088 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769822+0000", 2024-09-17T21:22:12.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:22:12.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:12.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769838+0000", 2024-09-17T21:22:12.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6674e-05 2024-09-17T21:22:12.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769851+0000", 2024-09-17T21:22:12.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2527999999999999e-05 2024-09-17T21:22:12.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858660+0000", 2024-09-17T21:22:12.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088808556999999996 2024-09-17T21:22:12.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:12.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858817+0000", 2024-09-17T21:22:12.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015730400000000001 2024-09-17T21:22:12.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:12.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859007+0000", 2024-09-17T21:22:12.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018974899999999999 2024-09-17T21:22:12.090 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.091 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:12.091 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.091 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12093 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:12.091 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:22:12.091 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.260180335, 2024-09-17T21:22:12.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089429710999999995, 2024-09-17T21:22:12.091 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:12.091 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:12.091 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:12.091 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:12.091 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:12.091 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:12.092 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12093 2024-09-17T21:22:12.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.092 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:12.092 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.092 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:12.092 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:22:12.092 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.092 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.092 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:12.092 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:22:12.092 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.093 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:12.093 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769924+0000", 2024-09-17T21:22:12.093 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.342e-06 2024-09-17T21:22:12.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.093 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:12.093 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769929+0000", 2024-09-17T21:22:12.093 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0350000000000001e-06 2024-09-17T21:22:12.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.093 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:12.093 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769946+0000", 2024-09-17T21:22:12.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6793999999999999e-05 2024-09-17T21:22:12.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769959+0000", 2024-09-17T21:22:12.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3062e-05 2024-09-17T21:22:12.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859075+0000", 2024-09-17T21:22:12.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089115696999999994 2024-09-17T21:22:12.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:12.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859202+0000", 2024-09-17T21:22:12.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012657 2024-09-17T21:22:12.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:12.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859353+0000", 2024-09-17T21:22:12.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151211 2024-09-17T21:22:12.095 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.095 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:12.095 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.095 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12094 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:12.095 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:22:12.095 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.260126159, 2024-09-17T21:22:12.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089695001999999996, 2024-09-17T21:22:12.096 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:12.096 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:12.096 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:12.096 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:12.096 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:12.096 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:12.096 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12094 2024-09-17T21:22:12.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.096 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:12.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.096 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:12.096 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:22:12.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:12.097 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:22:12.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:12.097 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769978+0000", 2024-09-17T21:22:12.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.142e-06 2024-09-17T21:22:12.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:12.097 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769983+0000", 2024-09-17T21:22:12.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.386e-06 2024-09-17T21:22:12.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:12.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769997+0000", 2024-09-17T21:22:12.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.416e-05 2024-09-17T21:22:12.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770012+0000", 2024-09-17T21:22:12.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5384999999999999e-05 2024-09-17T21:22:12.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859405+0000", 2024-09-17T21:22:12.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089392376999999995 2024-09-17T21:22:12.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:12.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859525+0000", 2024-09-17T21:22:12.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120555 2024-09-17T21:22:12.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:12.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859672+0000", 2024-09-17T21:22:12.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000146997 2024-09-17T21:22:12.099 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.099 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:12.099 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.100 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12095 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:12.100 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:22:12.100 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.260073672, 2024-09-17T21:22:12.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090073476, 2024-09-17T21:22:12.100 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:12.100 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:12.100 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:12.100 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:12.100 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:12.100 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:12.100 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12095 2024-09-17T21:22:12.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.101 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:12.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.101 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:12.101 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:22:12.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.101 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:12.101 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:22:12.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.102 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:12.102 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770031+0000", 2024-09-17T21:22:12.102 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1699999999999997e-07 2024-09-17T21:22:12.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.102 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:12.102 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770036+0000", 2024-09-17T21:22:12.102 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.023e-06 2024-09-17T21:22:12.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.102 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:12.102 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770051+0000", 2024-09-17T21:22:12.102 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4953999999999999e-05 2024-09-17T21:22:12.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770063+0000", 2024-09-17T21:22:12.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:22:12.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859756+0000", 2024-09-17T21:22:12.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089692980000000005 2024-09-17T21:22:12.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:12.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859891+0000", 2024-09-17T21:22:12.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013459499999999999 2024-09-17T21:22:12.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:12.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860103+0000", 2024-09-17T21:22:12.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021274400000000001 2024-09-17T21:22:12.104 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.104 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:12.104 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.104 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12096 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:12.104 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:22:12.104 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.260023987, 2024-09-17T21:22:12.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090358274000000002, 2024-09-17T21:22:12.105 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:12.105 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:12.105 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:12.105 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:12.105 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:12.105 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:12.105 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12096 2024-09-17T21:22:12.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.105 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:12.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:12.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:22:12.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.106 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:12.106 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:22:12.106 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.106 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:12.106 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770080+0000", 2024-09-17T21:22:12.106 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9400000000000004e-07 2024-09-17T21:22:12.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.106 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:12.106 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770085+0000", 2024-09-17T21:22:12.106 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2169999999999996e-06 2024-09-17T21:22:12.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:12.107 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770100+0000", 2024-09-17T21:22:12.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5077e-05 2024-09-17T21:22:12.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.107 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770112+0000", 2024-09-17T21:22:12.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1951e-05 2024-09-17T21:22:12.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.107 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860160+0000", 2024-09-17T21:22:12.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090048413999999993 2024-09-17T21:22:12.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:12.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860285+0000", 2024-09-17T21:22:12.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012466699999999999 2024-09-17T21:22:12.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:12.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860438+0000", 2024-09-17T21:22:12.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015305399999999999 2024-09-17T21:22:12.108 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.108 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:12.108 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.108 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12097 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:12.109 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:22:12.109 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.25997624, 2024-09-17T21:22:12.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090702375000000002, 2024-09-17T21:22:12.109 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:12.109 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:12.109 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:12.109 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:12.109 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:12.109 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:12.109 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12097 2024-09-17T21:22:12.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.109 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:12.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:12.110 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:22:12.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.110 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:12.110 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:22:12.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.110 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:12.110 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770128+0000", 2024-09-17T21:22:12.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.04e-06 2024-09-17T21:22:12.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.110 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:12.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770133+0000", 2024-09-17T21:22:12.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8999999999999997e-06 2024-09-17T21:22:12.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:12.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770152+0000", 2024-09-17T21:22:12.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8890999999999999e-05 2024-09-17T21:22:12.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770164+0000", 2024-09-17T21:22:12.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1769e-05 2024-09-17T21:22:12.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860511+0000", 2024-09-17T21:22:12.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090347527999999996 2024-09-17T21:22:12.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:12.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860652+0000", 2024-09-17T21:22:12.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140894 2024-09-17T21:22:12.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:12.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860830+0000", 2024-09-17T21:22:12.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017735299999999999 2024-09-17T21:22:12.113 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.113 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:12.113 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.113 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12098 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:12.113 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:22:12.113 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.259922772, 2024-09-17T21:22:12.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090997262999999995, 2024-09-17T21:22:12.114 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:12.114 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:12.114 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:12.114 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:12.114 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:12.114 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:12.114 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12098 2024-09-17T21:22:12.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.114 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:12.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:12.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:22:12.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:12.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:22:12.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:12.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775465+0000", 2024-09-17T21:22:12.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0052843350000000002 2024-09-17T21:22:12.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:12.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775476+0000", 2024-09-17T21:22:12.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0797e-05 2024-09-17T21:22:12.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:12.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775506+0000", 2024-09-17T21:22:12.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9709999999999998e-05 2024-09-17T21:22:12.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775530+0000", 2024-09-17T21:22:12.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4528999999999998e-05 2024-09-17T21:22:12.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860894+0000", 2024-09-17T21:22:12.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.085364195000000004 2024-09-17T21:22:12.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:12.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861019+0000", 2024-09-17T21:22:12.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012467399999999999 2024-09-17T21:22:12.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:12.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861178+0000", 2024-09-17T21:22:12.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159023 2024-09-17T21:22:12.118 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.118 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:12.118 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.118 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12104 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:12.118 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:22:12.118 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.254221663, 2024-09-17T21:22:12.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087969628999999994, 2024-09-17T21:22:12.119 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:12.119 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:12.119 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:12.119 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:12.119 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:12.119 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:12.119 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12104 2024-09-17T21:22:12.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.119 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:12.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.119 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:12.119 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:22:12.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:12.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775882+0000", 2024-09-17T21:22:12.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:12.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775883+0000", 2024-09-17T21:22:12.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3399999999999998e-07 2024-09-17T21:22:12.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:12.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775886+0000", 2024-09-17T21:22:12.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2119999999999999e-06 2024-09-17T21:22:12.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:12.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775895+0000", 2024-09-17T21:22:12.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8710000000000003e-06 2024-09-17T21:22:12.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775901+0000", 2024-09-17T21:22:12.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:22:12.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863544+0000", 2024-09-17T21:22:12.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087642642000000007 2024-09-17T21:22:12.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:12.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863694+0000", 2024-09-17T21:22:12.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014952300000000001 2024-09-17T21:22:12.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:12.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863851+0000", 2024-09-17T21:22:12.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000157804 2024-09-17T21:22:12.123 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.123 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:12.123 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.123 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12105 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:12.123 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:22:12.123 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.254192769, 2024-09-17T21:22:12.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088330400000000003, 2024-09-17T21:22:12.123 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:12.123 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:12.123 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:12.123 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:12.123 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:12.124 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:12.124 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12105 2024-09-17T21:22:12.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.124 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:12.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.124 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:12.124 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:22:12.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.124 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:12.124 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:22:12.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:12.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:22:12.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9999999999999997e-07 2024-09-17T21:22:12.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:12.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775914+0000", 2024-09-17T21:22:12.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4710000000000001e-06 2024-09-17T21:22:12.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:12.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775921+0000", 2024-09-17T21:22:12.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:22:12.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775928+0000", 2024-09-17T21:22:12.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:22:12.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863923+0000", 2024-09-17T21:22:12.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087995126000000007 2024-09-17T21:22:12.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:12.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864059+0000", 2024-09-17T21:22:12.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135797 2024-09-17T21:22:12.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:12.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864241+0000", 2024-09-17T21:22:12.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018252599999999999 2024-09-17T21:22:12.127 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.127 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:12.127 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.128 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12106 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:12.128 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:22:12.128 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 110.254103591, 2024-09-17T21:22:12.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088586394999999998, 2024-09-17T21:22:12.128 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:12.128 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:12.128 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:12.128 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:12.128 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:12.128 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:12.128 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12106 2024-09-17T21:22:12.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.128 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:12.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.129 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:12.129 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:22:12.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.129 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:12.129 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:22:12.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.129 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:12.129 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776001+0000", 2024-09-17T21:22:12.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4300000000000001e-07 2024-09-17T21:22:12.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:12.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776004+0000", 2024-09-17T21:22:12.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8049999999999998e-06 2024-09-17T21:22:12.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:12.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776018+0000", 2024-09-17T21:22:12.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4613999999999999e-05 2024-09-17T21:22:12.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776025+0000", 2024-09-17T21:22:12.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:22:12.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864279+0000", 2024-09-17T21:22:12.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088253570000000003 2024-09-17T21:22:12.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:12.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864433+0000", 2024-09-17T21:22:12.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015469400000000001 2024-09-17T21:22:12.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:12.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864586+0000", 2024-09-17T21:22:12.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000153003 2024-09-17T21:22:12.132 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.132 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:12.132 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.132 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25033 3.6 3.b74753e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e168)", 2024-09-17T21:22:12.132 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:22:12.132 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 46.520660327000002, 2024-09-17T21:22:12.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93781952099999999, 2024-09-17T21:22:12.132 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:12.132 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:12.133 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:12.133 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:12.133 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:22:12.133 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:12.133 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25033 2024-09-17T21:22:12.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.133 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:12.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.133 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:12.133 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:22:12.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:12.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:22:12.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:12.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509444+0000", 2024-09-17T21:22:12.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0020000000000001e-06 2024-09-17T21:22:12.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:12.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509450+0000", 2024-09-17T21:22:12.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4249999999999997e-06 2024-09-17T21:22:12.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:12.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509460+0000", 2024-09-17T21:22:12.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0183999999999999e-05 2024-09-17T21:22:12.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509466+0000", 2024-09-17T21:22:12.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0039999999999996e-06 2024-09-17T21:22:12.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509540+0000", 2024-09-17T21:22:12.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3847e-05 2024-09-17T21:22:12.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:12.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509551+0000", 2024-09-17T21:22:12.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1731e-05 2024-09-17T21:22:12.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.447243+0000", 2024-09-17T21:22:12.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93769166000000004 2024-09-17T21:22:12.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:12.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.447263+0000", 2024-09-17T21:22:12.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9667999999999999e-05 2024-09-17T21:22:12.137 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.137 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:12.137 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.137 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25033 3.6 3:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e168)", 2024-09-17T21:22:12.137 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:22:12.137 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 46.519134729999998, 2024-09-17T21:22:12.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.937309383, 2024-09-17T21:22:12.137 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:12.137 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:12.137 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:12.137 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:12.138 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:22:12.138 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:12.138 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25033 2024-09-17T21:22:12.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.138 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:12.138 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.138 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:12.138 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:22:12.138 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.138 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.138 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:12.138 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:22:12.138 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:12.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:12.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510970+0000", 2024-09-17T21:22:12.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.564e-06 2024-09-17T21:22:12.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:12.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510978+0000", 2024-09-17T21:22:12.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.362e-06 2024-09-17T21:22:12.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:12.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510998+0000", 2024-09-17T21:22:12.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0336999999999999e-05 2024-09-17T21:22:12.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.140 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511010+0000", 2024-09-17T21:22:12.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1786e-05 2024-09-17T21:22:12.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.140 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511080+0000", 2024-09-17T21:22:12.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0686999999999996e-05 2024-09-17T21:22:12.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:12.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511087+0000", 2024-09-17T21:22:12.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4899999999999997e-06 2024-09-17T21:22:12.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:12.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.448050+0000", 2024-09-17T21:22:12.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93696351700000002 2024-09-17T21:22:12.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:12.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:12.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:12.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.448278+0000", 2024-09-17T21:22:12.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022764000000000001 2024-09-17T21:22:12.142 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.142 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:12.142 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.142 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:12.142 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:12.142 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:12.143 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:22:12.144 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:12.141+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:12.245 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:12.242+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:12.276 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:12.276 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:12.276 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:22:12.276 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:12.290 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:22:12.309 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-17T21:22:12.345 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:12.341+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:12.412 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:12.412 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:12.413 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:22:12.413 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:22:12.413 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:12.430 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:22:12.471 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:12.469+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:12.547 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:12.545+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:12.596 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:12.596 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:22:12.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:22:12.596 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:22:12.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.597 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:22:12.597 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:22:12.597 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 255.57943003899999, 2024-09-17T21:22:12.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:22:12.597 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:12.597 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:12.597 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:12.597 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:12.597 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:22:12.597 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:22:12.597 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:22:12.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.598 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:12.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:12.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:22:12.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:12.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:22:12.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:12.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:22:12.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:22:12.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:12.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:22:12.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:22:12.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:12.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:22:12.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:22:12.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:22:12.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:22:12.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:22:12.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:22:12.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:12.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:22:12.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:22:12.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:22:12.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:22:12.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:12.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:22:12.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:22:12.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:22:12.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:22:12.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:22:12.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:12.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:22:12.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:22:12.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:12.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:22:12.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:22:12.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:12.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:22:12.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:22:12.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:12.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:22:12.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:22:12.603 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.603 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:12.603 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.604 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:22:12.604 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:22:12.604 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 235.999265759, 2024-09-17T21:22:12.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:22:12.604 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:12.604 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:12.604 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:12.604 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:12.604 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:22:12.604 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:22:12.604 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:22:12.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.605 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:12.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:12.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:22:12.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:12.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:22:12.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:12.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:22:12.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:22:12.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:12.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:22:12.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:22:12.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:12.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:22:12.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:22:12.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:22:12.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:22:12.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:22:12.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:22:12.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:22:12.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:22:12.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:22:12.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:22:12.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:22:12.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:12.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:22:12.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:22:12.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:22:12.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:22:12.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:12.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:22:12.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:22:12.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:12.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:22:12.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:22:12.609 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.610 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:12.610 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.610 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:22:12.610 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:22:12.610 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 235.99916844699999, 2024-09-17T21:22:12.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:22:12.610 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:12.610 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:12.610 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:12.610 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:12.610 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:12.611 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:12.611 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:22:12.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.611 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:12.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:12.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:22:12.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:12.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:22:12.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:12.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:22:12.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:22:12.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:12.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:22:12.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:22:12.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:12.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:22:12.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:22:12.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:22:12.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:22:12.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:22:12.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:22:12.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.614 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:22:12.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:22:12.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:22:12.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.614 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:22:12.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:22:12.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.614 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:12.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:22:12.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:22:12.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:22:12.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:22:12.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:22:12.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:22:12.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:22:12.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:22:12.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:22:12.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:12.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:22:12.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:22:12.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:12.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:22:12.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:22:12.616 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.616 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:22:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:22:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 235.997081646, 2024-09-17T21:22:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:22:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:22:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:22:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:22:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:22:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:22:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:22:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:22:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:22:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:22:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:22:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:22:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:22:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:22:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:22:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:22:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:22:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:22:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:22:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:22:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:22:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:22:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:22:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:22:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:22:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:22:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:22:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:22:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:22:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:22:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:22:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:22:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:22:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:22:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:22:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:22:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:22:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:22:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:22:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 203.31318286499999, 2024-09-17T21:22:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:22:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:22:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:22:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:22:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:22:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:22:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:22:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:22:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:22:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:22:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:22:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:22:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:22:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:22:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:22:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:22:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:22:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:22:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:22:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:22:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:22:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:22:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:22:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:22:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:22:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:22:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:22:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:22:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:22:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:22:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 203.30409115399999, 2024-09-17T21:22:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:22:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:22:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:22:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:22:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:22:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:22:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:22:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:22:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:22:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:22:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:22:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:22:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:22:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:22:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:22:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:22:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:22:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:22:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:22:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:22:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:22:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:22:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:22:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:22:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:22:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 203.303919103, 2024-09-17T21:22:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:22:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:22:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:22:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:22:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:22:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:22:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:22:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:22:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:22:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:22:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:22:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:22:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:22:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:22:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:22:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:22:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:22:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:22:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:22:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:22:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:22:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:22:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:22:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:22:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:22:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 203.12898635600001, 2024-09-17T21:22:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:22:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:22:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:22:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:22:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:22:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:22:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:22:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:22:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:22:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:22:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:22:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:22:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:22:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:22:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:22:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:22:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:22:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:22:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:22:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:22:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:22:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:22:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:22:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 203.128901799, 2024-09-17T21:22:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:22:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:22:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:22:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:22:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:22:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:22:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:22:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:22:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:22:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:22:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:22:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:22:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:22:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:22:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:22:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:22:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:22:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:22:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:22:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:22:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:22:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:22:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:22:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 203.128836162, 2024-09-17T21:22:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:22:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:22:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:22:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:22:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:22:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:22:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:22:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:22:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:22:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:22:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:22:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:22:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:22:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:22:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:22:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:22:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:22:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:22:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:22:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:22:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:22:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:22:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:22:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 151.75526106500001, 2024-09-17T21:22:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:22:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:22:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:22:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:22:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:22:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:22:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:22:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:22:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:22:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:22:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:22:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:22:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:22:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:22:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:22:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:22:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:22:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:22:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:22:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:22:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 151.75514155299999, 2024-09-17T21:22:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:22:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:22:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:22:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:22:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:22:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:22:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:22:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:22:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:22:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:22:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:22:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:22:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:22:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:22:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:22:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:22:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:22:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:22:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:22:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:996 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:22:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 107.162028182, 2024-09-17T21:22:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084493736999999999, 2024-09-17T21:22:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 996 2024-09-17T21:22:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:22:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:22:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431168+0000", 2024-09-17T21:22:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4e-07 2024-09-17T21:22:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431172+0000", 2024-09-17T21:22:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.608e-06 2024-09-17T21:22:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431180+0000", 2024-09-17T21:22:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5650000000000002e-06 2024-09-17T21:22:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431188+0000", 2024-09-17T21:22:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:22:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515354+0000", 2024-09-17T21:22:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084165977000000003 2024-09-17T21:22:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515451+0000", 2024-09-17T21:22:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6669000000000006e-05 2024-09-17T21:22:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515661+0000", 2024-09-17T21:22:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021021599999999999 2024-09-17T21:22:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:997 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 583680~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:22:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 107.16199678700001, 2024-09-17T21:22:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084772419000000002, 2024-09-17T21:22:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 997 2024-09-17T21:22:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:22:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:22:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:22:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0400000000000005e-07 2024-09-17T21:22:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431202+0000", 2024-09-17T21:22:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:22:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431208+0000", 2024-09-17T21:22:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4740000000000002e-06 2024-09-17T21:22:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431214+0000", 2024-09-17T21:22:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6550000000000001e-06 2024-09-17T21:22:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515741+0000", 2024-09-17T21:22:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084526904 2024-09-17T21:22:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515826+0000", 2024-09-17T21:22:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4643999999999998e-05 2024-09-17T21:22:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515971+0000", 2024-09-17T21:22:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145444 2024-09-17T21:22:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:998 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 587776~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:22:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 107.161973815, 2024-09-17T21:22:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085108565999999997, 2024-09-17T21:22:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 998 2024-09-17T21:22:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:22:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:22:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:22:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:22:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431224+0000", 2024-09-17T21:22:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:22:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431231+0000", 2024-09-17T21:22:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000002e-06 2024-09-17T21:22:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431236+0000", 2024-09-17T21:22:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.648e-06 2024-09-17T21:22:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516024+0000", 2024-09-17T21:22:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084787584999999999 2024-09-17T21:22:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516122+0000", 2024-09-17T21:22:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8076000000000001e-05 2024-09-17T21:22:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516330+0000", 2024-09-17T21:22:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020808 2024-09-17T21:22:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1000 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 591872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:22:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 107.1619517, 2024-09-17T21:22:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085389065, 2024-09-17T21:22:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1000 2024-09-17T21:22:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:22:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:22:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:22:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:22:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431246+0000", 2024-09-17T21:22:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1749999999999999e-06 2024-09-17T21:22:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431253+0000", 2024-09-17T21:22:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2790000000000004e-06 2024-09-17T21:22:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431258+0000", 2024-09-17T21:22:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3299999999999998e-06 2024-09-17T21:22:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516384+0000", 2024-09-17T21:22:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085125549999999994 2024-09-17T21:22:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516460+0000", 2024-09-17T21:22:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6359999999999994e-05 2024-09-17T21:22:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516633+0000", 2024-09-17T21:22:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017285900000000001 2024-09-17T21:22:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1001 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 595968~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:22:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 107.16192829400001, 2024-09-17T21:22:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085614715999999993, 2024-09-17T21:22:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1001 2024-09-17T21:22:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:22:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:22:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438066+0000", 2024-09-17T21:22:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067989490000000003 2024-09-17T21:22:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438073+0000", 2024-09-17T21:22:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8419999999999999e-06 2024-09-17T21:22:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438090+0000", 2024-09-17T21:22:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6657e-05 2024-09-17T21:22:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438099+0000", 2024-09-17T21:22:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0529999999999996e-06 2024-09-17T21:22:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516678+0000", 2024-09-17T21:22:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078579422999999995 2024-09-17T21:22:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516739+0000", 2024-09-17T21:22:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1315000000000005e-05 2024-09-17T21:22:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516882+0000", 2024-09-17T21:22:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014247700000000001 2024-09-17T21:22:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:19998 3.5 3:a4885450:::rbd_object_map.1152d139f9f9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-17T21:22:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:22:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 93.190661048999999, 2024-09-17T21:22:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90715489299999996, 2024-09-17T21:22:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19998 2024-09-17T21:22:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:22:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:22:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402537+0000", 2024-09-17T21:22:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5600000000000001e-06 2024-09-17T21:22:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402549+0000", 2024-09-17T21:22:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1929999999999999e-05 2024-09-17T21:22:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402579+0000", 2024-09-17T21:22:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0093000000000001e-05 2024-09-17T21:22:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402595+0000", 2024-09-17T21:22:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5855999999999999e-05 2024-09-17T21:22:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402731+0000", 2024-09-17T21:22:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013610799999999999 2024-09-17T21:22:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402750+0000", 2024-09-17T21:22:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8737e-05 2024-09-17T21:22:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.305977+0000", 2024-09-17T21:22:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90322768399999998 2024-09-17T21:22:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306236+0000", 2024-09-17T21:22:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000258111 2024-09-17T21:22:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:22:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306557+0000", 2024-09-17T21:22:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032159799999999998 2024-09-17T21:22:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.308286+0000", 2024-09-17T21:22:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017292150000000001 2024-09-17T21:22:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309530+0000", 2024-09-17T21:22:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00124409 2024-09-17T21:22:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309630+0000", 2024-09-17T21:22:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9689e-05 2024-09-17T21:22:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309689+0000", 2024-09-17T21:22:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9222000000000003e-05 2024-09-17T21:22:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:25277 3.1f 3:ff5d7cd5:::rbd_header.11524f65d4ff:head [watch ping cookie 139860845478480 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-17T21:22:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:22:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 43.00770498, 2024-09-17T21:22:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.92185720000000004, 2024-09-17T21:22:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 25277 2024-09-17T21:22:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:22:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:22:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585493+0000", 2024-09-17T21:22:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:22:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585505+0000", 2024-09-17T21:22:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2466000000000001e-05 2024-09-17T21:22:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:12.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585537+0000", 2024-09-17T21:22:12.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1183000000000003e-05 2024-09-17T21:22:12.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585553+0000", 2024-09-17T21:22:12.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.662e-05 2024-09-17T21:22:12.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585696+0000", 2024-09-17T21:22:12.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014293800000000001 2024-09-17T21:22:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585718+0000", 2024-09-17T21:22:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1860000000000001e-05 2024-09-17T21:22:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.501188+0000", 2024-09-17T21:22:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.91547051300000004 2024-09-17T21:22:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:22:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.501260+0000", 2024-09-17T21:22:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1571999999999998e-05 2024-09-17T21:22:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507002+0000", 2024-09-17T21:22:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057414850000000002 2024-09-17T21:22:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507143+0000", 2024-09-17T21:22:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014140299999999999 2024-09-17T21:22:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507348+0000", 2024-09-17T21:22:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000204773 2024-09-17T21:22:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:26042 3.f 3:f24134c1:::rbd_id.image_client.0-clone34:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e186)", 2024-09-17T21:22:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:22:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 29.107016611999999, 2024-09-17T21:22:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.91185481000000002, 2024-09-17T21:22:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 26042 2024-09-17T21:22:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:22:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:22:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486180+0000", 2024-09-17T21:22:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5990000000000001e-06 2024-09-17T21:22:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486187+0000", 2024-09-17T21:22:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7859999999999999e-06 2024-09-17T21:22:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486206+0000", 2024-09-17T21:22:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8641e-05 2024-09-17T21:22:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486219+0000", 2024-09-17T21:22:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.308e-05 2024-09-17T21:22:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486283+0000", 2024-09-17T21:22:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4084000000000005e-05 2024-09-17T21:22:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486290+0000", 2024-09-17T21:22:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.584e-06 2024-09-17T21:22:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392182+0000", 2024-09-17T21:22:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90589206499999997 2024-09-17T21:22:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392518+0000", 2024-09-17T21:22:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033596 2024-09-17T21:22:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:22:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392802+0000", 2024-09-17T21:22:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028394999999999999 2024-09-17T21:22:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397705+0000", 2024-09-17T21:22:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0049031769999999999 2024-09-17T21:22:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397770+0000", 2024-09-17T21:22:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4765999999999996e-05 2024-09-17T21:22:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397927+0000", 2024-09-17T21:22:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015699900000000001 2024-09-17T21:22:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.398034+0000", 2024-09-17T21:22:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000107119 2024-09-17T21:22:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.707 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:12.707 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.707 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:12.707 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:12.707 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:12.708 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:22:12.709 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:12.702+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:12.809 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:12.806+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:12.880 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:12.881 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:12.881 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:22:12.881 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:12.894 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:22:12.939 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:12.937+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:13.010 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:13.010 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:13.010 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:22:13.010 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:22:13.011 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:13.011 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:13.005+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:13.024 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:22:13.030 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:22:13.111 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:13.109+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:13.182 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:22:13.182 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:22:13.192 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:13.192 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:22:13.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:22:13.192 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:22:13.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.192 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:18003 3.1d 3:b9765580:::rbd_header.11527d2b15f6:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T21:22:13.192 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:22:13.192 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 120.83033518000001, 2024-09-17T21:22:13.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94165346900000002, 2024-09-17T21:22:13.193 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:13.193 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:13.193 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:13.193 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:13.193 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:13.193 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:13.193 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18003 2024-09-17T21:22:13.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.193 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:13.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:13.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:22:13.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:13.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:22:13.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:13.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357880+0000", 2024-09-17T21:22:13.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:22:13.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:13.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357891+0000", 2024-09-17T21:22:13.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0417999999999999e-05 2024-09-17T21:22:13.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:13.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357913+0000", 2024-09-17T21:22:13.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2215e-05 2024-09-17T21:22:13.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:13.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357922+0000", 2024-09-17T21:22:13.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4779999999999997e-06 2024-09-17T21:22:13.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358023+0000", 2024-09-17T21:22:13.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101025 2024-09-17T21:22:13.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:13.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358040+0000", 2024-09-17T21:22:13.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6291999999999999e-05 2024-09-17T21:22:13.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298203+0000", 2024-09-17T21:22:13.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94016297500000001 2024-09-17T21:22:13.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:13.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298626+0000", 2024-09-17T21:22:13.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000422853 2024-09-17T21:22:13.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:13.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.299532+0000", 2024-09-17T21:22:13.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00090657300000000004 2024-09-17T21:22:13.197 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.197 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:13.197 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.197 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15707 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 982016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:13.197 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:22:13.197 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 110.667169933, 2024-09-17T21:22:13.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092305228000000003, 2024-09-17T21:22:13.197 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:13.197 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:13.198 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:13.198 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:13.198 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:13.198 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:13.198 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15707 2024-09-17T21:22:13.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.198 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:13.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:13.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:22:13.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:13.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:22:13.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:13.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521045+0000", 2024-09-17T21:22:13.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4899999999999999e-06 2024-09-17T21:22:13.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:13.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521066+0000", 2024-09-17T21:22:13.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0236000000000001e-05 2024-09-17T21:22:13.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:13.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521085+0000", 2024-09-17T21:22:13.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8819999999999999e-05 2024-09-17T21:22:13.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:13.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521100+0000", 2024-09-17T21:22:13.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5146e-05 2024-09-17T21:22:13.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612978+0000", 2024-09-17T21:22:13.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091878847 2024-09-17T21:22:13.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:13.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613079+0000", 2024-09-17T21:22:13.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100503 2024-09-17T21:22:13.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:13.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613349+0000", 2024-09-17T21:22:13.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000270186 2024-09-17T21:22:13.201 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.201 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:13.201 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.202 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15708 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 986112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:13.202 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:22:13.202 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 110.667091758, 2024-09-17T21:22:13.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092547217000000001, 2024-09-17T21:22:13.202 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:13.202 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:13.202 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:13.202 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:13.202 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:13.202 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:13.202 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15708 2024-09-17T21:22:13.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.202 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:13.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:13.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:22:13.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:13.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:22:13.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:13.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521123+0000", 2024-09-17T21:22:13.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.353e-06 2024-09-17T21:22:13.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:13.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521129+0000", 2024-09-17T21:22:13.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:22:13.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:13.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521145+0000", 2024-09-17T21:22:13.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.611e-05 2024-09-17T21:22:13.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:13.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521154+0000", 2024-09-17T21:22:13.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4500000000000004e-06 2024-09-17T21:22:13.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613397+0000", 2024-09-17T21:22:13.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092243275 2024-09-17T21:22:13.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:13.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613503+0000", 2024-09-17T21:22:13.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106013 2024-09-17T21:22:13.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:13.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613669+0000", 2024-09-17T21:22:13.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016642299999999999 2024-09-17T21:22:13.206 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.206 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:13.206 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.206 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15709 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 990208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:13.206 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:22:13.206 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 110.66704874200001, 2024-09-17T21:22:13.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092728678999999994, 2024-09-17T21:22:13.206 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:13.206 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:13.206 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:13.207 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:13.207 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:13.207 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:13.207 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15709 2024-09-17T21:22:13.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.207 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:13.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:13.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:22:13.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:13.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:22:13.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:13.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521166+0000", 2024-09-17T21:22:13.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7000000000000004e-07 2024-09-17T21:22:13.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:13.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521169+0000", 2024-09-17T21:22:13.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2660000000000002e-06 2024-09-17T21:22:13.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:13.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521181+0000", 2024-09-17T21:22:13.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.208e-05 2024-09-17T21:22:13.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:13.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521194+0000", 2024-09-17T21:22:13.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2707e-05 2024-09-17T21:22:13.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613695+0000", 2024-09-17T21:22:13.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092500709 2024-09-17T21:22:13.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:13.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613750+0000", 2024-09-17T21:22:13.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.503e-05 2024-09-17T21:22:13.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:13.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613894+0000", 2024-09-17T21:22:13.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144117 2024-09-17T21:22:13.210 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.210 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:13.210 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.210 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15710 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 994304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:13.210 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:13.210 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 110.66700417600001, 2024-09-17T21:22:13.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092957729000000003, 2024-09-17T21:22:13.211 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:13.211 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:13.211 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:13.211 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:13.211 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:13.211 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:13.211 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15710 2024-09-17T21:22:13.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.211 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:13.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:13.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:13.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:13.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:13.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:13.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:13.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8000000000000005e-07 2024-09-17T21:22:13.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:13.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521216+0000", 2024-09-17T21:22:13.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3859999999999998e-06 2024-09-17T21:22:13.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:13.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521234+0000", 2024-09-17T21:22:13.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:22:13.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:13.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521248+0000", 2024-09-17T21:22:13.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:22:13.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613926+0000", 2024-09-17T21:22:13.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092678324000000006 2024-09-17T21:22:13.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:13.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613974+0000", 2024-09-17T21:22:13.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7754999999999999e-05 2024-09-17T21:22:13.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:13.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614167+0000", 2024-09-17T21:22:13.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193508 2024-09-17T21:22:13.214 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.214 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:13.215 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.215 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15711 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 998400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:13.215 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:22:13.215 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 110.666946794, 2024-09-17T21:22:13.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093268000000000004, 2024-09-17T21:22:13.215 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:13.215 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:13.215 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:13.215 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:13.215 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:13.215 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:13.215 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15711 2024-09-17T21:22:13.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.216 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:13.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:13.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:22:13.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:13.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:22:13.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:13.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521268+0000", 2024-09-17T21:22:13.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-17T21:22:13.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:13.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521273+0000", 2024-09-17T21:22:13.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:22:13.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:13.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521290+0000", 2024-09-17T21:22:13.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7269999999999999e-05 2024-09-17T21:22:13.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:13.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521304+0000", 2024-09-17T21:22:13.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3626e-05 2024-09-17T21:22:13.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614220+0000", 2024-09-17T21:22:13.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092916367 2024-09-17T21:22:13.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:13.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614322+0000", 2024-09-17T21:22:13.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102212 2024-09-17T21:22:13.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:13.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614535+0000", 2024-09-17T21:22:13.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021266199999999999 2024-09-17T21:22:13.219 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.219 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:13.219 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.219 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15712 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1002496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:13.219 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:22:13.219 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 110.666891299, 2024-09-17T21:22:13.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093568707000000001, 2024-09-17T21:22:13.219 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:13.220 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:13.220 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:13.220 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:13.220 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:13.220 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:13.220 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15712 2024-09-17T21:22:13.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.220 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:13.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:13.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:22:13.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:13.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:22:13.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:13.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521324+0000", 2024-09-17T21:22:13.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.17e-06 2024-09-17T21:22:13.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:13.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521334+0000", 2024-09-17T21:22:13.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9369999999999999e-06 2024-09-17T21:22:13.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:13.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521356+0000", 2024-09-17T21:22:13.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2045999999999999e-05 2024-09-17T21:22:13.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:13.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521370+0000", 2024-09-17T21:22:13.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4270000000000001e-05 2024-09-17T21:22:13.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614583+0000", 2024-09-17T21:22:13.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093213220999999999 2024-09-17T21:22:13.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:13.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614680+0000", 2024-09-17T21:22:13.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6900999999999999e-05 2024-09-17T21:22:13.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:13.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614891+0000", 2024-09-17T21:22:13.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021116200000000001 2024-09-17T21:22:13.223 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.223 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:13.223 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.224 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15713 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1006592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:13.224 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:22:13.224 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 110.666824597, 2024-09-17T21:22:13.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093834641999999996, 2024-09-17T21:22:13.224 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:13.224 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:13.224 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:13.224 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:13.224 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:13.224 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:13.224 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15713 2024-09-17T21:22:13.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.224 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:13.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:13.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:22:13.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:13.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:22:13.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:13.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521390+0000", 2024-09-17T21:22:13.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:22:13.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:13.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521396+0000", 2024-09-17T21:22:13.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:22:13.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:13.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521417+0000", 2024-09-17T21:22:13.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1236000000000001e-05 2024-09-17T21:22:13.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:13.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521431+0000", 2024-09-17T21:22:13.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3873000000000001e-05 2024-09-17T21:22:13.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614938+0000", 2024-09-17T21:22:13.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093506522999999994 2024-09-17T21:22:13.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:13.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615025+0000", 2024-09-17T21:22:13.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7635999999999999e-05 2024-09-17T21:22:13.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:13.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615224+0000", 2024-09-17T21:22:13.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019844700000000001 2024-09-17T21:22:13.228 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.228 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:13.228 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.228 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15714 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:13.228 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:22:13.228 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 110.666761341, 2024-09-17T21:22:13.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094071925000000001, 2024-09-17T21:22:13.228 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:13.228 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:13.228 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:13.229 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:13.229 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:13.229 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:13.229 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15714 2024-09-17T21:22:13.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.229 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:13.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:13.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:22:13.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:13.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:22:13.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:13.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521454+0000", 2024-09-17T21:22:13.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.097e-06 2024-09-17T21:22:13.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:13.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521459+0000", 2024-09-17T21:22:13.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2660000000000001e-06 2024-09-17T21:22:13.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:13.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521481+0000", 2024-09-17T21:22:13.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1878999999999998e-05 2024-09-17T21:22:13.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:13.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521498+0000", 2024-09-17T21:22:13.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6807e-05 2024-09-17T21:22:13.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615271+0000", 2024-09-17T21:22:13.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093773286999999997 2024-09-17T21:22:13.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:13.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615366+0000", 2024-09-17T21:22:13.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5322000000000007e-05 2024-09-17T21:22:13.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:13.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615524+0000", 2024-09-17T21:22:13.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015826699999999999 2024-09-17T21:22:13.232 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.232 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:13.232 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.233 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15715 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1014784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:13.233 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:22:13.233 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 110.66669700600001, 2024-09-17T21:22:13.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094243840999999995, 2024-09-17T21:22:13.233 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:13.233 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:13.233 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:13.233 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:13.233 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:13.233 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:13.233 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15715 2024-09-17T21:22:13.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.233 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:13.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:13.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:22:13.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:13.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:22:13.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:13.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521518+0000", 2024-09-17T21:22:13.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4300000000000001e-06 2024-09-17T21:22:13.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:13.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521524+0000", 2024-09-17T21:22:13.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5860000000000004e-06 2024-09-17T21:22:13.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:13.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521541+0000", 2024-09-17T21:22:13.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7574000000000001e-05 2024-09-17T21:22:13.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:13.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521555+0000", 2024-09-17T21:22:13.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3540000000000001e-05 2024-09-17T21:22:13.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615560+0000", 2024-09-17T21:22:13.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.09400472 2024-09-17T21:22:13.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:13.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615614+0000", 2024-09-17T21:22:13.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4693e-05 2024-09-17T21:22:13.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:13.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615761+0000", 2024-09-17T21:22:13.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000146298 2024-09-17T21:22:13.237 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.237 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:13.237 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.237 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15716 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1018880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:13.237 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:22:13.237 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 110.666640661, 2024-09-17T21:22:13.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094400676000000003, 2024-09-17T21:22:13.237 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:13.237 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:13.237 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:13.237 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:13.237 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:13.237 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:13.238 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15716 2024-09-17T21:22:13.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.238 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:13.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:13.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:22:13.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:13.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:22:13.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521574+0000", 2024-09-17T21:22:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1939999999999999e-06 2024-09-17T21:22:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521579+0000", 2024-09-17T21:22:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:22:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521597+0000", 2024-09-17T21:22:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8074e-05 2024-09-17T21:22:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521613+0000", 2024-09-17T21:22:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5316e-05 2024-09-17T21:22:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615780+0000", 2024-09-17T21:22:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094166874999999997 2024-09-17T21:22:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615836+0000", 2024-09-17T21:22:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6926999999999999e-05 2024-09-17T21:22:13.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:13.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615974+0000", 2024-09-17T21:22:13.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013741400000000001 2024-09-17T21:22:13.241 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.241 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:13.241 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.241 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15717 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1022976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:13.241 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:22:13.241 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 110.66658196500001, 2024-09-17T21:22:13.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094586723999999997, 2024-09-17T21:22:13.241 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:13.242 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:13.242 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:13.242 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:13.242 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:13.242 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:13.242 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15717 2024-09-17T21:22:13.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.242 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:13.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:13.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:22:13.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:13.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:22:13.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:13.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521633+0000", 2024-09-17T21:22:13.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:22:13.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:13.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521638+0000", 2024-09-17T21:22:13.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:22:13.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:13.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521659+0000", 2024-09-17T21:22:13.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-05 2024-09-17T21:22:13.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:13.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521675+0000", 2024-09-17T21:22:13.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5688999999999998e-05 2024-09-17T21:22:13.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616008+0000", 2024-09-17T21:22:13.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094333314000000001 2024-09-17T21:22:13.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:13.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616066+0000", 2024-09-17T21:22:13.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7797000000000002e-05 2024-09-17T21:22:13.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:13.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616219+0000", 2024-09-17T21:22:13.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015281100000000001 2024-09-17T21:22:13.245 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.245 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:13.245 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.246 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15718 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:13.246 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:22:13.246 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 110.66648189, 2024-09-17T21:22:13.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094703674000000002, 2024-09-17T21:22:13.246 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:13.246 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:13.246 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:13.246 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:13.246 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:13.246 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:13.246 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15718 2024-09-17T21:22:13.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.246 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:13.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:13.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:22:13.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:13.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:22:13.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:13.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521734+0000", 2024-09-17T21:22:13.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.61e-06 2024-09-17T21:22:13.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:13.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521740+0000", 2024-09-17T21:22:13.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:22:13.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:13.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521754+0000", 2024-09-17T21:22:13.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3793e-05 2024-09-17T21:22:13.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:13.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521763+0000", 2024-09-17T21:22:13.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5129999999999996e-06 2024-09-17T21:22:13.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616256+0000", 2024-09-17T21:22:13.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094492919999999994 2024-09-17T21:22:13.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:13.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616312+0000", 2024-09-17T21:22:13.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6737000000000002e-05 2024-09-17T21:22:13.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:13.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616436+0000", 2024-09-17T21:22:13.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012335800000000001 2024-09-17T21:22:13.250 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.250 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:13.250 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.250 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15719 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1031168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:13.250 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:22:13.250 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 110.666428485, 2024-09-17T21:22:13.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094888963000000007, 2024-09-17T21:22:13.250 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:13.250 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:13.250 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:13.250 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:13.250 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:13.250 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:13.251 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15719 2024-09-17T21:22:13.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.251 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:13.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:13.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:22:13.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:13.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:22:13.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:13.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521786+0000", 2024-09-17T21:22:13.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.074e-06 2024-09-17T21:22:13.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:13.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521791+0000", 2024-09-17T21:22:13.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3030000000000001e-06 2024-09-17T21:22:13.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:13.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521804+0000", 2024-09-17T21:22:13.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3027000000000001e-05 2024-09-17T21:22:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521811+0000", 2024-09-17T21:22:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6090000000000003e-06 2024-09-17T21:22:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616466+0000", 2024-09-17T21:22:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094654294 2024-09-17T21:22:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616523+0000", 2024-09-17T21:22:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6953999999999997e-05 2024-09-17T21:22:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616674+0000", 2024-09-17T21:22:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015170200000000001 2024-09-17T21:22:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15720 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1035264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:22:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 110.666391645, 2024-09-17T21:22:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095067061999999994, 2024-09-17T21:22:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15720 2024-09-17T21:22:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:13.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:22:13.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:13.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:22:13.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:13.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521823+0000", 2024-09-17T21:22:13.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:22:13.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:13.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521826+0000", 2024-09-17T21:22:13.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2100000000000002e-06 2024-09-17T21:22:13.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:13.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521835+0000", 2024-09-17T21:22:13.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1630000000000002e-06 2024-09-17T21:22:13.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521842+0000", 2024-09-17T21:22:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:22:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616711+0000", 2024-09-17T21:22:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094868859 2024-09-17T21:22:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616764+0000", 2024-09-17T21:22:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2778999999999997e-05 2024-09-17T21:22:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616889+0000", 2024-09-17T21:22:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012533100000000001 2024-09-17T21:22:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15721 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1039360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:22:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 110.666344536, 2024-09-17T21:22:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095277219999999996, 2024-09-17T21:22:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15721 2024-09-17T21:22:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:22:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:22:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521871+0000", 2024-09-17T21:22:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.443e-06 2024-09-17T21:22:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521876+0000", 2024-09-17T21:22:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0300000000000001e-06 2024-09-17T21:22:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521886+0000", 2024-09-17T21:22:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186e-05 2024-09-17T21:22:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521893+0000", 2024-09-17T21:22:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.08e-06 2024-09-17T21:22:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616918+0000", 2024-09-17T21:22:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095025223000000006 2024-09-17T21:22:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616977+0000", 2024-09-17T21:22:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8403999999999999e-05 2024-09-17T21:22:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617147+0000", 2024-09-17T21:22:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016985399999999999 2024-09-17T21:22:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15722 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 110.666310087, 2024-09-17T21:22:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095459783000000006, 2024-09-17T21:22:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15722 2024-09-17T21:22:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7700000000000004e-07 2024-09-17T21:22:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521907+0000", 2024-09-17T21:22:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.813e-06 2024-09-17T21:22:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521922+0000", 2024-09-17T21:22:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4533e-05 2024-09-17T21:22:13.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:13.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521934+0000", 2024-09-17T21:22:13.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2469999999999999e-05 2024-09-17T21:22:13.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617187+0000", 2024-09-17T21:22:13.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095252311000000006 2024-09-17T21:22:13.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617240+0000", 2024-09-17T21:22:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3740999999999999e-05 2024-09-17T21:22:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617364+0000", 2024-09-17T21:22:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-17T21:22:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15723 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:22:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 110.666266221, 2024-09-17T21:22:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095653563999999996, 2024-09-17T21:22:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15723 2024-09-17T21:22:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:22:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:22:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521949+0000", 2024-09-17T21:22:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:22:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:13.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521953+0000", 2024-09-17T21:22:13.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5870000000000003e-06 2024-09-17T21:22:13.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:13.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521970+0000", 2024-09-17T21:22:13.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7282000000000002e-05 2024-09-17T21:22:13.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:13.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521984+0000", 2024-09-17T21:22:13.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:22:13.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617383+0000", 2024-09-17T21:22:13.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095399063000000006 2024-09-17T21:22:13.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:13.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617437+0000", 2024-09-17T21:22:13.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3931999999999998e-05 2024-09-17T21:22:13.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:13.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617601+0000", 2024-09-17T21:22:13.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016407300000000001 2024-09-17T21:22:13.273 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.273 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:13.273 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.273 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5629.0:2384 3.1c 3:38c1263e:::rbd_data.1152d37183c3.000000000000000b:head [read 11780096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-17T21:22:13.273 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:22:13.273 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 97.763435391000002, 2024-09-17T21:22:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0246185320000001, 2024-09-17T21:22:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5629", 2024-09-17T21:22:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3798711329", 2024-09-17T21:22:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2384 2024-09-17T21:22:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:22:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:22:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424780+0000", 2024-09-17T21:22:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.13e-06 2024-09-17T21:22:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424785+0000", 2024-09-17T21:22:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4739999999999996e-06 2024-09-17T21:22:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424798+0000", 2024-09-17T21:22:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2704000000000001e-05 2024-09-17T21:22:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424806+0000", 2024-09-17T21:22:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9899999999999997e-06 2024-09-17T21:22:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424868+0000", 2024-09-17T21:22:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999997e-05 2024-09-17T21:22:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:22:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424875+0000", 2024-09-17T21:22:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0870000000000001e-06 2024-09-17T21:22:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428897+0000", 2024-09-17T21:22:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040226080000000004 2024-09-17T21:22:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428913+0000", 2024-09-17T21:22:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:22:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.448963+0000", 2024-09-17T21:22:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0200498659999999 2024-09-17T21:22:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449158+0000", 2024-09-17T21:22:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019581000000000001 2024-09-17T21:22:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:13.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449397+0000", 2024-09-17T21:22:13.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023871 2024-09-17T21:22:13.279 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.279 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:13.279 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.279 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:26043 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_remove_image in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-17T21:22:13.279 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:22:13.279 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.788904271, 2024-09-17T21:22:13.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.019978351, 2024-09-17T21:22:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 26043 2024-09-17T21:22:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:22:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:22:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399314+0000", 2024-09-17T21:22:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.963e-06 2024-09-17T21:22:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399330+0000", 2024-09-17T21:22:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6684000000000002e-05 2024-09-17T21:22:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399372+0000", 2024-09-17T21:22:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1819000000000001e-05 2024-09-17T21:22:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399396+0000", 2024-09-17T21:22:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.406e-05 2024-09-17T21:22:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399571+0000", 2024-09-17T21:22:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017444800000000001 2024-09-17T21:22:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399595+0000", 2024-09-17T21:22:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4709999999999999e-05 2024-09-17T21:22:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.412735+0000", 2024-09-17T21:22:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0131394869999999 2024-09-17T21:22:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:13.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.413090+0000", 2024-09-17T21:22:13.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000355006 2024-09-17T21:22:13.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:22:13.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.413659+0000", 2024-09-17T21:22:13.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056922699999999995 2024-09-17T21:22:13.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:13.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.418940+0000", 2024-09-17T21:22:13.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052814109999999997 2024-09-17T21:22:13.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.418992+0000", 2024-09-17T21:22:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1646e-05 2024-09-17T21:22:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.419168+0000", 2024-09-17T21:22:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017573400000000001 2024-09-17T21:22:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.419288+0000", 2024-09-17T21:22:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012015600000000001 2024-09-17T21:22:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.286 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:13.286 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.286 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:13.286 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:13.286 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:13.287 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:22:13.288 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:22:13.288 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T21:22:13.288 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:22:13.289 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:22:13.289 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:22:13.289 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:22:13.290 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:22:13.290 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:22:13.290 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:22:13.291 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:22:13.291 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:22:13.291 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:13.282+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:13.388 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:13.385+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:13.461 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:13.461 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:13.476 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:22:13.519 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:13.517+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:13.590 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:13.585+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:13.620 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 0] out_osds: [2, 5] dead_osds: [2] live_osds: [0, 1, 3, 4, 5] 2024-09-17T21:22:13.621 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:22:13.621 INFO:tasks.thrashosds.thrasher:Reweighting osd 4 to 0.4261032784313644 2024-09-17T21:22:13.621 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 0.4261032784313644 2024-09-17T21:22:13.644 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:13.645 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:13.663 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:22:13.711 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:13.709+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:13.791 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:13.786+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:13.839 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:13.839 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:13.856 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:22:13.892 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:13.889+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:13.979 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:13.979 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:13.993 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:22:13.994 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:13.989+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:14.094 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:14.090+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:14.117 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:14.117 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:14.133 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:22:14.195 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:14.193+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:14.296 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:14.293+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:14.296 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:14.296 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:14.311 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:22:14.341 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:22:14.397 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:14.393+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:14.435 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:14.435 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:14.450 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:22:14.498 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:14.493+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:14.583 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:14.583 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:14.598 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:22:14.599 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:14.594+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:14.643 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.4 to 0.426103 (6d15) 2024-09-17T21:22:14.700 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:14.697+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:14.731 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:14.731 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:14.745 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:22:14.800 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:14.797+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:14.875 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:14.876 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:14.890 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:22:14.901 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:14.898+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:15.003 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:15.001+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:15.054 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:15.054 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:15.069 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:22:15.111 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:15.109+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:15.205 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:15.201+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:15.232 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:15.232 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:15.246 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:22:15.292 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:22:15.339 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:15.337+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:15.406 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:15.401+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:15.412 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:15.412 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:15.427 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:22:15.445 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:22:15.445 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:22:15.461 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:22:15.461 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:22:15.461 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T21:22:15.479 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:22:15.479 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:22:15.497 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:22:15.497 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:22:15.515 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:15.513+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:15.516 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:22:15.516 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:22:15.519 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:22:15.520 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:22:15.594 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:15.595 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:15.595 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:22:15.595 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:22:15.595 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:15.609 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:22:15.609 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:15.605+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:15.633 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-17T21:22:15.709 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:15.705+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:15.779 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:15.779 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:22:15.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:22:15.779 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:22:15.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.780 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:18003 3.1d 3:b9765580:::rbd_header.11527d2b15f6:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T21:22:15.780 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:22:15.780 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.41734952500001, 2024-09-17T21:22:15.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94165346900000002, 2024-09-17T21:22:15.780 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:15.780 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:15.780 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:15.780 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:15.780 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:15.780 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:15.780 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18003 2024-09-17T21:22:15.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.781 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:15.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:15.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:22:15.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:15.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:22:15.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:15.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357880+0000", 2024-09-17T21:22:15.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:22:15.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:15.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357891+0000", 2024-09-17T21:22:15.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0417999999999999e-05 2024-09-17T21:22:15.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:15.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357913+0000", 2024-09-17T21:22:15.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2215e-05 2024-09-17T21:22:15.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:15.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357922+0000", 2024-09-17T21:22:15.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4779999999999997e-06 2024-09-17T21:22:15.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358023+0000", 2024-09-17T21:22:15.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101025 2024-09-17T21:22:15.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:15.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358040+0000", 2024-09-17T21:22:15.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6291999999999999e-05 2024-09-17T21:22:15.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298203+0000", 2024-09-17T21:22:15.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94016297500000001 2024-09-17T21:22:15.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:15.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298626+0000", 2024-09-17T21:22:15.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000422853 2024-09-17T21:22:15.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:15.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.299532+0000", 2024-09-17T21:22:15.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00090657300000000004 2024-09-17T21:22:15.785 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.785 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:15.785 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.785 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15707 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 982016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:15.785 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:22:15.785 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 113.254184278, 2024-09-17T21:22:15.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092305228000000003, 2024-09-17T21:22:15.785 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:15.785 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:15.785 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:15.786 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:15.786 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:15.786 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:15.786 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15707 2024-09-17T21:22:15.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.786 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:15.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:15.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:22:15.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:15.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:22:15.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:15.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521045+0000", 2024-09-17T21:22:15.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4899999999999999e-06 2024-09-17T21:22:15.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:15.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521066+0000", 2024-09-17T21:22:15.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0236000000000001e-05 2024-09-17T21:22:15.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:15.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521085+0000", 2024-09-17T21:22:15.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8819999999999999e-05 2024-09-17T21:22:15.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:15.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521100+0000", 2024-09-17T21:22:15.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5146e-05 2024-09-17T21:22:15.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612978+0000", 2024-09-17T21:22:15.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091878847 2024-09-17T21:22:15.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:15.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613079+0000", 2024-09-17T21:22:15.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100503 2024-09-17T21:22:15.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:15.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613349+0000", 2024-09-17T21:22:15.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000270186 2024-09-17T21:22:15.789 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.789 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:15.789 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.790 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15708 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 986112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:15.790 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:22:15.790 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 113.254106103, 2024-09-17T21:22:15.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092547217000000001, 2024-09-17T21:22:15.790 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:15.790 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:15.790 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:15.790 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:15.790 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:15.790 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:15.790 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15708 2024-09-17T21:22:15.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.790 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:15.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:15.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:22:15.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:15.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:22:15.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:15.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521123+0000", 2024-09-17T21:22:15.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.353e-06 2024-09-17T21:22:15.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:15.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521129+0000", 2024-09-17T21:22:15.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:22:15.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:15.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521145+0000", 2024-09-17T21:22:15.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.611e-05 2024-09-17T21:22:15.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:15.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521154+0000", 2024-09-17T21:22:15.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4500000000000004e-06 2024-09-17T21:22:15.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613397+0000", 2024-09-17T21:22:15.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092243275 2024-09-17T21:22:15.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:15.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613503+0000", 2024-09-17T21:22:15.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106013 2024-09-17T21:22:15.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:15.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613669+0000", 2024-09-17T21:22:15.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016642299999999999 2024-09-17T21:22:15.794 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.794 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:15.794 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.794 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15709 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 990208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:15.794 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:22:15.794 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 113.25406308700001, 2024-09-17T21:22:15.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092728678999999994, 2024-09-17T21:22:15.794 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:15.795 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:15.795 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:15.795 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:15.795 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:15.795 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:15.795 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15709 2024-09-17T21:22:15.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.795 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:15.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:15.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:22:15.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:15.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:22:15.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:15.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521166+0000", 2024-09-17T21:22:15.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7000000000000004e-07 2024-09-17T21:22:15.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:15.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521169+0000", 2024-09-17T21:22:15.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2660000000000002e-06 2024-09-17T21:22:15.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:15.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521181+0000", 2024-09-17T21:22:15.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.208e-05 2024-09-17T21:22:15.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:15.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521194+0000", 2024-09-17T21:22:15.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2707e-05 2024-09-17T21:22:15.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613695+0000", 2024-09-17T21:22:15.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092500709 2024-09-17T21:22:15.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:15.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613750+0000", 2024-09-17T21:22:15.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.503e-05 2024-09-17T21:22:15.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:15.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613894+0000", 2024-09-17T21:22:15.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144117 2024-09-17T21:22:15.798 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.798 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:15.798 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.799 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15710 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 994304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:15.799 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:15.799 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 113.25401852100001, 2024-09-17T21:22:15.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092957729000000003, 2024-09-17T21:22:15.799 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:15.799 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:15.799 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:15.799 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:15.799 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:15.799 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:15.799 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15710 2024-09-17T21:22:15.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.800 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:15.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:15.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:15.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:15.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:15.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:15.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:15.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8000000000000005e-07 2024-09-17T21:22:15.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:15.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521216+0000", 2024-09-17T21:22:15.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3859999999999998e-06 2024-09-17T21:22:15.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:15.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521234+0000", 2024-09-17T21:22:15.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:22:15.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:15.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521248+0000", 2024-09-17T21:22:15.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:22:15.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613926+0000", 2024-09-17T21:22:15.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092678324000000006 2024-09-17T21:22:15.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:15.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613974+0000", 2024-09-17T21:22:15.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7754999999999999e-05 2024-09-17T21:22:15.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:15.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614167+0000", 2024-09-17T21:22:15.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193508 2024-09-17T21:22:15.803 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.803 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:15.803 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.803 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15711 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 998400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:15.803 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:22:15.803 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 113.253961139, 2024-09-17T21:22:15.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093268000000000004, 2024-09-17T21:22:15.803 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:15.803 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:15.803 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:15.803 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:15.803 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:15.804 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:15.804 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15711 2024-09-17T21:22:15.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.804 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:15.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:15.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:22:15.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:15.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:22:15.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:15.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521268+0000", 2024-09-17T21:22:15.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-17T21:22:15.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:15.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521273+0000", 2024-09-17T21:22:15.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:22:15.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:15.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521290+0000", 2024-09-17T21:22:15.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7269999999999999e-05 2024-09-17T21:22:15.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:15.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521304+0000", 2024-09-17T21:22:15.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3626e-05 2024-09-17T21:22:15.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614220+0000", 2024-09-17T21:22:15.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092916367 2024-09-17T21:22:15.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:15.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614322+0000", 2024-09-17T21:22:15.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102212 2024-09-17T21:22:15.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:15.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614535+0000", 2024-09-17T21:22:15.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021266199999999999 2024-09-17T21:22:15.807 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.807 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:15.807 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.807 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15712 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1002496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:15.807 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:22:15.807 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 113.253905644, 2024-09-17T21:22:15.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093568707000000001, 2024-09-17T21:22:15.807 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:15.807 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:15.807 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:15.808 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:15.808 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:15.808 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:15.808 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15712 2024-09-17T21:22:15.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.808 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:15.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:15.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:22:15.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:15.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:22:15.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:15.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521324+0000", 2024-09-17T21:22:15.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.17e-06 2024-09-17T21:22:15.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:15.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521334+0000", 2024-09-17T21:22:15.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9369999999999999e-06 2024-09-17T21:22:15.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:15.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521356+0000", 2024-09-17T21:22:15.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2045999999999999e-05 2024-09-17T21:22:15.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:15.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521370+0000", 2024-09-17T21:22:15.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4270000000000001e-05 2024-09-17T21:22:15.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614583+0000", 2024-09-17T21:22:15.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093213220999999999 2024-09-17T21:22:15.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.811 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:15.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614680+0000", 2024-09-17T21:22:15.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6900999999999999e-05 2024-09-17T21:22:15.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.811 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:15.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614891+0000", 2024-09-17T21:22:15.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021116200000000001 2024-09-17T21:22:15.811 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.811 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:15.811 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.811 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15713 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1006592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:15.812 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:22:15.812 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 113.253838942, 2024-09-17T21:22:15.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093834641999999996, 2024-09-17T21:22:15.812 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:15.812 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:15.812 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:15.812 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:15.812 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:15.812 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:15.812 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15713 2024-09-17T21:22:15.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.812 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:15.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:15.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:22:15.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:15.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:22:15.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:15.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521390+0000", 2024-09-17T21:22:15.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:22:15.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:15.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521396+0000", 2024-09-17T21:22:15.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:22:15.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:15.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521417+0000", 2024-09-17T21:22:15.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1236000000000001e-05 2024-09-17T21:22:15.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:15.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521431+0000", 2024-09-17T21:22:15.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3873000000000001e-05 2024-09-17T21:22:15.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614938+0000", 2024-09-17T21:22:15.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093506522999999994 2024-09-17T21:22:15.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:15.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615025+0000", 2024-09-17T21:22:15.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7635999999999999e-05 2024-09-17T21:22:15.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:15.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615224+0000", 2024-09-17T21:22:15.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019844700000000001 2024-09-17T21:22:15.815 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.815 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:15.816 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.816 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15714 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:15.816 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:22:15.816 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 113.253775686, 2024-09-17T21:22:15.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094071925000000001, 2024-09-17T21:22:15.816 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:15.816 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:15.816 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:15.816 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:15.816 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:15.816 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:15.816 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15714 2024-09-17T21:22:15.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.817 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:15.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:15.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:22:15.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:15.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:22:15.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:15.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521454+0000", 2024-09-17T21:22:15.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.097e-06 2024-09-17T21:22:15.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:15.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521459+0000", 2024-09-17T21:22:15.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2660000000000001e-06 2024-09-17T21:22:15.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:15.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521481+0000", 2024-09-17T21:22:15.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1878999999999998e-05 2024-09-17T21:22:15.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:15.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521498+0000", 2024-09-17T21:22:15.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6807e-05 2024-09-17T21:22:15.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615271+0000", 2024-09-17T21:22:15.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093773286999999997 2024-09-17T21:22:15.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:15.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615366+0000", 2024-09-17T21:22:15.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5322000000000007e-05 2024-09-17T21:22:15.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:15.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615524+0000", 2024-09-17T21:22:15.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015826699999999999 2024-09-17T21:22:15.819 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.820 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:15.820 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.820 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15715 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1014784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:15.820 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:22:15.820 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 113.25371135100001, 2024-09-17T21:22:15.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094243840999999995, 2024-09-17T21:22:15.820 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:15.820 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:15.820 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:15.820 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:15.820 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:15.820 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:15.820 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15715 2024-09-17T21:22:15.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.821 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:15.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:15.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:22:15.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:15.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:22:15.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:15.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521518+0000", 2024-09-17T21:22:15.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4300000000000001e-06 2024-09-17T21:22:15.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:15.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521524+0000", 2024-09-17T21:22:15.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5860000000000004e-06 2024-09-17T21:22:15.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:15.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521541+0000", 2024-09-17T21:22:15.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7574000000000001e-05 2024-09-17T21:22:15.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:15.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521555+0000", 2024-09-17T21:22:15.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3540000000000001e-05 2024-09-17T21:22:15.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615560+0000", 2024-09-17T21:22:15.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.09400472 2024-09-17T21:22:15.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:15.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615614+0000", 2024-09-17T21:22:15.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4693e-05 2024-09-17T21:22:15.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:15.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615761+0000", 2024-09-17T21:22:15.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000146298 2024-09-17T21:22:15.824 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.824 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:15.824 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.824 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15716 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1018880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:15.824 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:22:15.824 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 113.253655006, 2024-09-17T21:22:15.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094400676000000003, 2024-09-17T21:22:15.824 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:15.825 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:15.825 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:15.825 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:15.825 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:15.825 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:15.825 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15716 2024-09-17T21:22:15.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.825 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:15.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:15.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:22:15.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:15.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:22:15.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:15.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521574+0000", 2024-09-17T21:22:15.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1939999999999999e-06 2024-09-17T21:22:15.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:15.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521579+0000", 2024-09-17T21:22:15.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:22:15.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:15.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521597+0000", 2024-09-17T21:22:15.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8074e-05 2024-09-17T21:22:15.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:15.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521613+0000", 2024-09-17T21:22:15.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5316e-05 2024-09-17T21:22:15.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615780+0000", 2024-09-17T21:22:15.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094166874999999997 2024-09-17T21:22:15.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:15.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615836+0000", 2024-09-17T21:22:15.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6926999999999999e-05 2024-09-17T21:22:15.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:15.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615974+0000", 2024-09-17T21:22:15.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013741400000000001 2024-09-17T21:22:15.828 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.828 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:15.828 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.829 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15717 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1022976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:15.829 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:22:15.829 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 113.25359631000001, 2024-09-17T21:22:15.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094586723999999997, 2024-09-17T21:22:15.829 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:15.829 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:15.829 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:15.829 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:15.829 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:15.829 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:15.829 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15717 2024-09-17T21:22:15.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.830 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:15.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:15.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:22:15.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:15.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:22:15.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:15.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521633+0000", 2024-09-17T21:22:15.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:22:15.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:15.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521638+0000", 2024-09-17T21:22:15.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:22:15.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:15.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521659+0000", 2024-09-17T21:22:15.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-05 2024-09-17T21:22:15.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:15.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521675+0000", 2024-09-17T21:22:15.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5688999999999998e-05 2024-09-17T21:22:15.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616008+0000", 2024-09-17T21:22:15.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094333314000000001 2024-09-17T21:22:15.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:15.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616066+0000", 2024-09-17T21:22:15.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7797000000000002e-05 2024-09-17T21:22:15.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.833 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:15.833 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616219+0000", 2024-09-17T21:22:15.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015281100000000001 2024-09-17T21:22:15.833 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.833 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:15.833 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.833 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15718 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:15.833 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:22:15.833 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 113.253496235, 2024-09-17T21:22:15.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094703674000000002, 2024-09-17T21:22:15.833 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:15.833 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:15.834 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:15.834 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:15.834 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:15.834 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:15.834 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15718 2024-09-17T21:22:15.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.834 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:15.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:15.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:22:15.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:15.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:22:15.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:15.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521734+0000", 2024-09-17T21:22:15.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.61e-06 2024-09-17T21:22:15.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:15.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521740+0000", 2024-09-17T21:22:15.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:22:15.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:15.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521754+0000", 2024-09-17T21:22:15.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3793e-05 2024-09-17T21:22:15.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:15.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521763+0000", 2024-09-17T21:22:15.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5129999999999996e-06 2024-09-17T21:22:15.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616256+0000", 2024-09-17T21:22:15.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094492919999999994 2024-09-17T21:22:15.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:15.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616312+0000", 2024-09-17T21:22:15.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6737000000000002e-05 2024-09-17T21:22:15.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:15.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616436+0000", 2024-09-17T21:22:15.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012335800000000001 2024-09-17T21:22:15.837 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.837 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:15.837 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.837 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15719 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1031168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:15.837 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:22:15.837 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 113.25344283, 2024-09-17T21:22:15.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094888963000000007, 2024-09-17T21:22:15.838 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:15.838 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:15.838 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:15.838 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:15.838 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:15.838 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:15.838 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15719 2024-09-17T21:22:15.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.838 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:15.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:15.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:22:15.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:15.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:22:15.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:15.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521786+0000", 2024-09-17T21:22:15.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.074e-06 2024-09-17T21:22:15.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:15.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521791+0000", 2024-09-17T21:22:15.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3030000000000001e-06 2024-09-17T21:22:15.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:15.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521804+0000", 2024-09-17T21:22:15.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3027000000000001e-05 2024-09-17T21:22:15.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:15.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521811+0000", 2024-09-17T21:22:15.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6090000000000003e-06 2024-09-17T21:22:15.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616466+0000", 2024-09-17T21:22:15.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094654294 2024-09-17T21:22:15.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:15.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616523+0000", 2024-09-17T21:22:15.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6953999999999997e-05 2024-09-17T21:22:15.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:15.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616674+0000", 2024-09-17T21:22:15.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015170200000000001 2024-09-17T21:22:15.841 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.841 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:15.841 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.842 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15720 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1035264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:15.842 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:22:15.842 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 113.25340599, 2024-09-17T21:22:15.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095067061999999994, 2024-09-17T21:22:15.842 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:15.842 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:15.842 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:15.842 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:15.842 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:15.842 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:15.842 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15720 2024-09-17T21:22:15.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.842 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:15.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:15.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:22:15.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:15.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:22:15.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:15.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521823+0000", 2024-09-17T21:22:15.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:22:15.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:15.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521826+0000", 2024-09-17T21:22:15.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2100000000000002e-06 2024-09-17T21:22:15.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:15.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521835+0000", 2024-09-17T21:22:15.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1630000000000002e-06 2024-09-17T21:22:15.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:15.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521842+0000", 2024-09-17T21:22:15.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:22:15.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616711+0000", 2024-09-17T21:22:15.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094868859 2024-09-17T21:22:15.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:15.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616764+0000", 2024-09-17T21:22:15.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2778999999999997e-05 2024-09-17T21:22:15.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:15.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616889+0000", 2024-09-17T21:22:15.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012533100000000001 2024-09-17T21:22:15.845 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.845 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:15.846 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.846 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15721 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1039360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:15.846 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:22:15.846 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 113.253358881, 2024-09-17T21:22:15.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095277219999999996, 2024-09-17T21:22:15.846 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:15.846 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:15.846 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:15.846 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:15.846 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:15.846 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:15.846 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15721 2024-09-17T21:22:15.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.846 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:15.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:15.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:22:15.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:15.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:22:15.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:15.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521871+0000", 2024-09-17T21:22:15.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.443e-06 2024-09-17T21:22:15.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:15.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521876+0000", 2024-09-17T21:22:15.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0300000000000001e-06 2024-09-17T21:22:15.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:15.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521886+0000", 2024-09-17T21:22:15.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186e-05 2024-09-17T21:22:15.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:15.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521893+0000", 2024-09-17T21:22:15.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.08e-06 2024-09-17T21:22:15.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616918+0000", 2024-09-17T21:22:15.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095025223000000006 2024-09-17T21:22:15.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:15.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616977+0000", 2024-09-17T21:22:15.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8403999999999999e-05 2024-09-17T21:22:15.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:15.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617147+0000", 2024-09-17T21:22:15.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016985399999999999 2024-09-17T21:22:15.849 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.849 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:15.850 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.850 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15722 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:15.850 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:15.850 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 113.253324432, 2024-09-17T21:22:15.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095459783000000006, 2024-09-17T21:22:15.850 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:15.850 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:15.850 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:15.850 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:15.850 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:15.850 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:15.850 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15722 2024-09-17T21:22:15.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.851 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:15.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:15.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:15.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:15.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:15.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:15.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:15.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7700000000000004e-07 2024-09-17T21:22:15.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:15.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521907+0000", 2024-09-17T21:22:15.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.813e-06 2024-09-17T21:22:15.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:15.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521922+0000", 2024-09-17T21:22:15.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4533e-05 2024-09-17T21:22:15.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:15.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521934+0000", 2024-09-17T21:22:15.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2469999999999999e-05 2024-09-17T21:22:15.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617187+0000", 2024-09-17T21:22:15.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095252311000000006 2024-09-17T21:22:15.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:15.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617240+0000", 2024-09-17T21:22:15.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3740999999999999e-05 2024-09-17T21:22:15.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:15.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617364+0000", 2024-09-17T21:22:15.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-17T21:22:15.854 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.854 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:15.854 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.854 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15723 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:15.854 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:22:15.854 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 113.253280566, 2024-09-17T21:22:15.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095653563999999996, 2024-09-17T21:22:15.854 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:15.855 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:15.855 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:15.855 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:15.855 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:15.855 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:15.855 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15723 2024-09-17T21:22:15.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.855 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:15.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:15.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:22:15.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:15.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:22:15.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:15.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521949+0000", 2024-09-17T21:22:15.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:22:15.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:15.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521953+0000", 2024-09-17T21:22:15.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5870000000000003e-06 2024-09-17T21:22:15.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:15.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521970+0000", 2024-09-17T21:22:15.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7282000000000002e-05 2024-09-17T21:22:15.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:15.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521984+0000", 2024-09-17T21:22:15.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:22:15.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617383+0000", 2024-09-17T21:22:15.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095399063000000006 2024-09-17T21:22:15.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:15.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617437+0000", 2024-09-17T21:22:15.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3931999999999998e-05 2024-09-17T21:22:15.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:15.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617601+0000", 2024-09-17T21:22:15.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016407300000000001 2024-09-17T21:22:15.858 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.858 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:15.858 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.858 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5629.0:2384 3.1c 3:38c1263e:::rbd_data.1152d37183c3.000000000000000b:head [read 11780096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-17T21:22:15.858 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:22:15.858 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 100.350449736, 2024-09-17T21:22:15.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0246185320000001, 2024-09-17T21:22:15.859 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:15.859 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:15.859 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:15.859 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:15.859 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5629", 2024-09-17T21:22:15.859 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3798711329", 2024-09-17T21:22:15.859 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2384 2024-09-17T21:22:15.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.859 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:15.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:15.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:22:15.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:15.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:22:15.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:15.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424780+0000", 2024-09-17T21:22:15.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.13e-06 2024-09-17T21:22:15.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:15.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424785+0000", 2024-09-17T21:22:15.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4739999999999996e-06 2024-09-17T21:22:15.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:15.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424798+0000", 2024-09-17T21:22:15.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2704000000000001e-05 2024-09-17T21:22:15.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:15.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424806+0000", 2024-09-17T21:22:15.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9899999999999997e-06 2024-09-17T21:22:15.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424868+0000", 2024-09-17T21:22:15.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999997e-05 2024-09-17T21:22:15.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:22:15.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424875+0000", 2024-09-17T21:22:15.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0870000000000001e-06 2024-09-17T21:22:15.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428897+0000", 2024-09-17T21:22:15.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040226080000000004 2024-09-17T21:22:15.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:15.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428913+0000", 2024-09-17T21:22:15.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:22:15.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.448963+0000", 2024-09-17T21:22:15.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0200498659999999 2024-09-17T21:22:15.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:15.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449158+0000", 2024-09-17T21:22:15.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019581000000000001 2024-09-17T21:22:15.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:15.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449397+0000", 2024-09-17T21:22:15.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023871 2024-09-17T21:22:15.863 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.864 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:15.864 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.864 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:26043 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_remove_image in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-17T21:22:15.864 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:22:15.864 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.375918616, 2024-09-17T21:22:15.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.019978351, 2024-09-17T21:22:15.864 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:15.864 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:15.864 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:15.864 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:15.864 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:15.864 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:15.864 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 26043 2024-09-17T21:22:15.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.865 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:15.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:15.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:22:15.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:15.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:22:15.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:15.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:15.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399314+0000", 2024-09-17T21:22:15.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.963e-06 2024-09-17T21:22:15.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:15.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399330+0000", 2024-09-17T21:22:15.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6684000000000002e-05 2024-09-17T21:22:15.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:15.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399372+0000", 2024-09-17T21:22:15.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1819000000000001e-05 2024-09-17T21:22:15.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:15.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399396+0000", 2024-09-17T21:22:15.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.406e-05 2024-09-17T21:22:15.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399571+0000", 2024-09-17T21:22:15.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017444800000000001 2024-09-17T21:22:15.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:15.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399595+0000", 2024-09-17T21:22:15.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4709999999999999e-05 2024-09-17T21:22:15.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:15.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.412735+0000", 2024-09-17T21:22:15.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0131394869999999 2024-09-17T21:22:15.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:15.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.413090+0000", 2024-09-17T21:22:15.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000355006 2024-09-17T21:22:15.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:22:15.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.413659+0000", 2024-09-17T21:22:15.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056922699999999995 2024-09-17T21:22:15.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.869 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:15.869 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.418940+0000", 2024-09-17T21:22:15.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052814109999999997 2024-09-17T21:22:15.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.869 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:15.869 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.418992+0000", 2024-09-17T21:22:15.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1646e-05 2024-09-17T21:22:15.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.869 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:15.869 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.419168+0000", 2024-09-17T21:22:15.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017573400000000001 2024-09-17T21:22:15.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:15.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:15.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:15.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.419288+0000", 2024-09-17T21:22:15.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012015600000000001 2024-09-17T21:22:15.870 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.870 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:15.870 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.870 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:15.870 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:15.870 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:15.871 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:22:15.872 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:15.870+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:15.972 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:15.969+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:16.041 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:16.041 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:16.041 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:22:16.041 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:16.055 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:22:16.073 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:16.070+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:16.173 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:16.169+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:16.222 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:16.222 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:16.223 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:22:16.223 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:22:16.223 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:16.237 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:22:16.274 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:16.270+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:16.375 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:16.373+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:16.400 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:16.400 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:22:16.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:22:16.401 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:22:16.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.401 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:22:16.401 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:22:16.401 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 234.79251179100001, 2024-09-17T21:22:16.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:22:16.401 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:16.401 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:16.401 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:16.401 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:16.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:16.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:22:16.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:16.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:22:16.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:16.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:22:16.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:22:16.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:16.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:22:16.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:22:16.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:16.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:22:16.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:22:16.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:16.404 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:22:16.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:22:16.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.404 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:16.404 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:22:16.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:22:16.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.404 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:16.404 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:22:16.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:22:16.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:16.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:22:16.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:22:16.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:16.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:22:16.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:22:16.405 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.405 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:16.405 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.406 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:22:16.406 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:22:16.406 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 234.78256515499999, 2024-09-17T21:22:16.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:22:16.406 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:16.407 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:16.407 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:16.407 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:16.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.408 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:16.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:22:16.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.408 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:16.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:22:16.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.408 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:16.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:22:16.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:22:16.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:16.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:22:16.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:22:16.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:16.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:22:16.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:22:16.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:16.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:22:16.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:22:16.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:16.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:22:16.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:22:16.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:16.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:22:16.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:22:16.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:16.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:22:16.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:22:16.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:16.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:22:16.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:22:16.411 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.411 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:16.411 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.411 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:22:16.411 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:22:16.412 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 233.53941431499999, 2024-09-17T21:22:16.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:22:16.412 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:16.412 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:16.412 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:16.412 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:16.412 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:22:16.412 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:22:16.412 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:22:16.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.412 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:16.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.412 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:16.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:22:16.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:16.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:22:16.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:16.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:22:16.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:22:16.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:16.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:22:16.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:22:16.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:16.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:22:16.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:22:16.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:16.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:22:16.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:22:16.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:16.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:22:16.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:22:16.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:16.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:22:16.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:22:16.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:22:16.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:22:16.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:22:16.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:16.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:22:16.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:22:16.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:16.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:22:16.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:22:16.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:16.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:22:16.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:22:16.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:16.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:22:16.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:22:16.417 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.417 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:16.417 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.417 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:22:16.417 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:22:16.418 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 233.52976882900001, 2024-09-17T21:22:16.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:22:16.418 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:16.418 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:16.418 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:16.418 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:16.418 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:22:16.418 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:22:16.418 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:22:16.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.418 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:16.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:16.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:22:16.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:16.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:22:16.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:16.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:22:16.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:22:16.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:16.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:22:16.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:22:16.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:16.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:22:16.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:22:16.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:16.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:22:16.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:22:16.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:16.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:22:16.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:22:16.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:16.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:22:16.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:22:16.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:22:16.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:22:16.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:22:16.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.422 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:16.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:22:16.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:22:16.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.422 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:16.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:22:16.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:22:16.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.422 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:16.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:22:16.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:22:16.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:16.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:22:16.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:22:16.423 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.423 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:16.423 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.423 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:22:16.423 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:22:16.423 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 220.16083279599999, 2024-09-17T21:22:16.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:22:16.423 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:16.424 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:16.424 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:16.424 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:16.424 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:22:16.424 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:22:16.424 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:22:16.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.424 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:16.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:16.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:22:16.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:16.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:22:16.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:16.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:22:16.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:22:16.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:16.426 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:22:16.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:22:16.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.426 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:16.426 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:22:16.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:22:16.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.426 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:16.426 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:22:16.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:22:16.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:16.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:22:16.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:22:16.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:16.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:22:16.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:22:16.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:16.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:22:16.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:22:16.428 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.428 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:16.428 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.428 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:16.428 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:22:16.428 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 201.959148234, 2024-09-17T21:22:16.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:22:16.428 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:16.428 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:16.429 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:16.429 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:16.429 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:16.429 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:16.429 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:22:16.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.429 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:16.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:16.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:22:16.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:16.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:22:16.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:16.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:22:16.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:22:16.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:16.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:22:16.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:22:16.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:16.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:22:16.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:22:16.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:16.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:22:16.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:22:16.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:16.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:22:16.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:22:16.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:16.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:22:16.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:22:16.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:16.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:22:16.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:22:16.433 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.433 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:16.433 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.433 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:16.433 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:22:16.433 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 201.95865365700001, 2024-09-17T21:22:16.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:22:16.433 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:16.433 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:16.433 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:16.434 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:16.434 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:16.434 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:16.434 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:22:16.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.434 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:16.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:16.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:22:16.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:16.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:22:16.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:16.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:22:16.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:22:16.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:16.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:22:16.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:22:16.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:16.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:22:16.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:22:16.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:16.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:22:16.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:22:16.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:16.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:22:16.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:22:16.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:16.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:22:16.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:22:16.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:16.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:22:16.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:22:16.437 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.437 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:16.437 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.438 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:16.438 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:22:16.438 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 201.958598276, 2024-09-17T21:22:16.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:22:16.438 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:16.438 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:16.438 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:16.438 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:16.438 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:16.438 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:16.438 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:22:16.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.438 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:16.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:16.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:22:16.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:16.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:22:16.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:16.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:22:16.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:22:16.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:16.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:22:16.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:22:16.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:16.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:22:16.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:22:16.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:16.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:22:16.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:22:16.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:16.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:22:16.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:22:16.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:16.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:22:16.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:22:16.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:16.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:22:16.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:22:16.442 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.442 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:16.442 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.442 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:16.442 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:22:16.442 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 201.95854199799999, 2024-09-17T21:22:16.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:22:16.442 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:16.442 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:16.442 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:16.442 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:16.443 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:16.443 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:16.443 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:22:16.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.443 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:16.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:16.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:22:16.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:16.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:22:16.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:16.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:22:16.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:22:16.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:16.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:22:16.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:22:16.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:16.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:22:16.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:22:16.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:16.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:22:16.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:22:16.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:16.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:22:16.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:22:16.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:16.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:22:16.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:22:16.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:16.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:22:16.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:22:16.446 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.446 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:16.446 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.446 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:16.446 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:22:16.446 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 201.95759235899999, 2024-09-17T21:22:16.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:22:16.447 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:16.447 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:16.447 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:16.447 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:16.447 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:16.447 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:16.447 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:22:16.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.447 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:16.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:16.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:22:16.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:16.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:22:16.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:16.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:22:16.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:22:16.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:16.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:22:16.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:22:16.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:16.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:22:16.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:22:16.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:16.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:22:16.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:22:16.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:16.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:22:16.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:22:16.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:16.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:22:16.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:22:16.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:16.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:22:16.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:22:16.450 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.450 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:16.450 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.451 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:16.451 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:22:16.451 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 201.95751373900001, 2024-09-17T21:22:16.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:22:16.451 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:16.451 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:16.451 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:16.451 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:16.451 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:16.451 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:16.451 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:22:16.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.451 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:16.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:16.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:22:16.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:16.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:22:16.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:16.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:22:16.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:22:16.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:16.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:22:16.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:22:16.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:16.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:22:16.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:22:16.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:16.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:22:16.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:22:16.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:16.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:22:16.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:22:16.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:16.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:22:16.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:22:16.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:16.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:22:16.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:22:16.455 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.455 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:16.455 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.455 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:16.455 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:22:16.455 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 201.95740079999999, 2024-09-17T21:22:16.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:22:16.455 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:16.455 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:16.455 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:16.455 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:16.456 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:16.456 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:16.456 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:22:16.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.456 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:16.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:16.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:22:16.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:16.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:22:16.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:16.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:22:16.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:22:16.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:16.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:22:16.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:22:16.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:16.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:22:16.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:22:16.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:16.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:22:16.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:22:16.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:16.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:22:16.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:22:16.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:16.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:22:16.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:22:16.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:16.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:22:16.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:22:16.459 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.459 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:16.459 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.459 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:16.459 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:22:16.459 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 201.95673974600001, 2024-09-17T21:22:16.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:22:16.460 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:16.460 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:16.460 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:16.460 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:16.460 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:16.460 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:16.460 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:22:16.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.460 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:16.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:16.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:22:16.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:16.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:22:16.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:16.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:22:16.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:22:16.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:16.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:22:16.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:22:16.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:16.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:22:16.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:22:16.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:16.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:22:16.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:22:16.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:16.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:22:16.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:22:16.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:16.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:22:16.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:22:16.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:16.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:22:16.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:22:16.463 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.463 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:16.463 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.464 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:16.464 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:22:16.464 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 201.956674935, 2024-09-17T21:22:16.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:22:16.464 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:16.464 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:16.464 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:16.464 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:16.464 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:16.464 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:16.464 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:22:16.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.464 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:16.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:16.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:22:16.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:16.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:22:16.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:16.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:22:16.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:22:16.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:16.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:22:16.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:22:16.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:16.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:22:16.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:22:16.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:16.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:22:16.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:22:16.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:16.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:22:16.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:22:16.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:16.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:22:16.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:22:16.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:16.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:22:16.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:22:16.467 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.467 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:16.467 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.468 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:16.468 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:22:16.468 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 201.95661395799999, 2024-09-17T21:22:16.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:22:16.468 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:16.468 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:16.468 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:16.468 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:16.468 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:16.468 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:16.468 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:22:16.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.469 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:16.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:16.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:22:16.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:16.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:22:16.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:16.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:22:16.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:22:16.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:16.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:22:16.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:22:16.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:16.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:22:16.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:22:16.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:16.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:22:16.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:22:16.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:16.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:22:16.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:22:16.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:16.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:22:16.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:22:16.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:16.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:22:16.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:22:16.472 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.472 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:16.472 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.472 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:22:16.472 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:22:16.472 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 147.05904277299999, 2024-09-17T21:22:16.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:22:16.473 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:16.473 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:16.473 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:16.473 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:16.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:16.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:22:16.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:16.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:22:16.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:16.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:22:16.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:22:16.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:16.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:22:16.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:22:16.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:16.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:22:16.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:22:16.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:16.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:22:16.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:22:16.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:16.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:22:16.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:22:16.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:16.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:22:16.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:22:16.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:16.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:22:16.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:22:16.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:16.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:22:16.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:22:16.476 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.476 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:16.477 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.477 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:22:16.477 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:22:16.477 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 146.85038437899999, 2024-09-17T21:22:16.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:22:16.477 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:16.477 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:16.477 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:16.477 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:16.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:16.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:22:16.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:16.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:22:16.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:16.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:22:16.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:22:16.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:16.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:22:16.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:22:16.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:16.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:22:16.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:22:16.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:16.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:22:16.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:22:16.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:16.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:22:16.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:22:16.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:16.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:22:16.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:22:16.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:16.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:22:16.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:22:16.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:16.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:22:16.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:22:16.481 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.481 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:16.481 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.481 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:9547 3.2 e147/132 3:42b518d9:::rbd_data.1152f056b73f.0000000000000001:head v 147'513, mlcod=142'511)", 2024-09-17T21:22:16.481 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:22:16.481 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 73.925411256999993, 2024-09-17T21:22:16.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.031059566, 2024-09-17T21:22:16.481 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:16.482 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:16.482 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:16.482 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:16.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:16.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:22:16.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:16.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:22:16.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:16.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471293+0000", 2024-09-17T21:22:16.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2029999999999999e-06 2024-09-17T21:22:16.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:16.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480606+0000", 2024-09-17T21:22:16.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093123839999999999 2024-09-17T21:22:16.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:16.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480653+0000", 2024-09-17T21:22:16.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7852000000000002e-05 2024-09-17T21:22:16.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:16.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480673+0000", 2024-09-17T21:22:16.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0013e-05 2024-09-17T21:22:16.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:16.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480862+0000", 2024-09-17T21:22:16.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018833599999999999 2024-09-17T21:22:16.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:16.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480906+0000", 2024-09-17T21:22:16.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4162000000000002e-05 2024-09-17T21:22:16.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:16.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502259+0000", 2024-09-17T21:22:16.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021353154999999999 2024-09-17T21:22:16.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:16.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502351+0000", 2024-09-17T21:22:16.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1460999999999997e-05 2024-09-17T21:22:16.486 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.486 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:16.486 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.486 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:15234 3.1 e147/134 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head v 147'917, mlcod=145'914)", 2024-09-17T21:22:16.486 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:22:16.486 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 73.292750639000005, 2024-09-17T21:22:16.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045303695999999997, 2024-09-17T21:22:16.486 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:16.486 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:16.486 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:16.486 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:16.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:16.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:22:16.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:16.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:22:16.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:16.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103954+0000", 2024-09-17T21:22:16.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2259999999999999e-06 2024-09-17T21:22:16.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:16.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119105+0000", 2024-09-17T21:22:16.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015150871 2024-09-17T21:22:16.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:16.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119141+0000", 2024-09-17T21:22:16.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6239000000000003e-05 2024-09-17T21:22:16.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:16.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119159+0000", 2024-09-17T21:22:16.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7946999999999999e-05 2024-09-17T21:22:16.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:16.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119344+0000", 2024-09-17T21:22:16.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018524599999999999 2024-09-17T21:22:16.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:16.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119399+0000", 2024-09-17T21:22:16.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4922e-05 2024-09-17T21:22:16.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:16.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149165+0000", 2024-09-17T21:22:16.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029765697000000001 2024-09-17T21:22:16.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:16.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149255+0000", 2024-09-17T21:22:16.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0548000000000001e-05 2024-09-17T21:22:16.490 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.490 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:16.490 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.491 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:19859 3.8 e147/133 3:12f72409:::rbd_data.1152f056b73f.0000000000000003:head v 147'482, mlcod=145'479)", 2024-09-17T21:22:16.491 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:22:16.491 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 72.883859572000006, 2024-09-17T21:22:16.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033476895999999999, 2024-09-17T21:22:16.491 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:16.491 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:16.491 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:16.491 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:16.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:16.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:22:16.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:16.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:22:16.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:16.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:16.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512845+0000", 2024-09-17T21:22:16.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.886e-06 2024-09-17T21:22:16.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:16.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524104+0000", 2024-09-17T21:22:16.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011259668 2024-09-17T21:22:16.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:16.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524148+0000", 2024-09-17T21:22:16.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3386000000000001e-05 2024-09-17T21:22:16.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:16.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524172+0000", 2024-09-17T21:22:16.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4762000000000001e-05 2024-09-17T21:22:16.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:16.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524379+0000", 2024-09-17T21:22:16.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020671599999999999 2024-09-17T21:22:16.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:16.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524421+0000", 2024-09-17T21:22:16.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2182999999999999e-05 2024-09-17T21:22:16.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:16.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546216+0000", 2024-09-17T21:22:16.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021795101000000001 2024-09-17T21:22:16.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:16.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:16.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:16.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546320+0000", 2024-09-17T21:22:16.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103194 2024-09-17T21:22:16.495 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.495 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:16.495 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.495 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:16.495 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:16.495 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:16.497 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:16.493+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:16.497 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:22:16.597 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:16.593+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:16.628 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:22:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:22:16.628 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:16.646 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:22:16.698 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:16.693+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:16.795 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:16.795 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:22:16.795 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:22:16.795 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:22:16.795 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:16.798 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:16.794+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:16.813 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:22:16.899 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:16.897+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:16.995 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:16.995 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:22:16.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:22:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:22:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:22:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:22:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 180.48150094299999, 2024-09-17T21:22:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:22:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:22:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:22:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:22:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:22:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:22:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:22:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:22:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:22:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:22:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:22:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:22:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:22:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:22:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:22:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:22:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:22:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:22:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:22:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:22:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:22:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:22:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2597 3.1f 3:fb0e9615:::rbd_data.1152a8ae2019.000000000000000c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:22:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 156.15306451500001, 2024-09-17T21:22:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097064994000000002, 2024-09-17T21:22:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2597 2024-09-17T21:22:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:22:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:22:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838628+0000", 2024-09-17T21:22:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.187e-06 2024-09-17T21:22:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838654+0000", 2024-09-17T21:22:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:22:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838696+0000", 2024-09-17T21:22:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1226000000000001e-05 2024-09-17T21:22:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838715+0000", 2024-09-17T21:22:17.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9646000000000001e-05 2024-09-17T21:22:17.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838937+0000", 2024-09-17T21:22:17.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022163 2024-09-17T21:22:17.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.839284+0000", 2024-09-17T21:22:17.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034753799999999999 2024-09-17T21:22:17.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935691+0000", 2024-09-17T21:22:17.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.096406360999999996 2024-09-17T21:22:17.005 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.005 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.005 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.005 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2641 3.12 3:488a729f:::rbd_data.1152a8ae2019.0000000000000022:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:17.005 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:22:17.005 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 156.149607742, 2024-09-17T21:22:17.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098280953000000004, 2024-09-17T21:22:17.006 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.006 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.006 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.006 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.006 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:17.006 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:17.006 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2641 2024-09-17T21:22:17.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.006 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:22:17.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:22:17.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842083+0000", 2024-09-17T21:22:17.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5300000000000002e-07 2024-09-17T21:22:17.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.008 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842088+0000", 2024-09-17T21:22:17.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:22:17.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.008 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.008 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842105+0000", 2024-09-17T21:22:17.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6773e-05 2024-09-17T21:22:17.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.008 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.008 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842119+0000", 2024-09-17T21:22:17.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:22:17.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842320+0000", 2024-09-17T21:22:17.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020052700000000001 2024-09-17T21:22:17.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842845+0000", 2024-09-17T21:22:17.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052508600000000002 2024-09-17T21:22:17.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940363+0000", 2024-09-17T21:22:17.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097518795000000005 2024-09-17T21:22:17.009 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.010 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.010 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.010 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2674 3.12 3:4c87e3ac:::rbd_data.1152a8ae2019.0000000000000033:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:17.010 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:22:17.010 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 156.14924510500001, 2024-09-17T21:22:17.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098025125000000005, 2024-09-17T21:22:17.010 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.010 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.010 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:17.010 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.011 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:17.011 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:17.011 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2674 2024-09-17T21:22:17.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.011 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.011 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.011 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:22:17.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.011 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.012 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:22:17.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.012 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.012 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842447+0000", 2024-09-17T21:22:17.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3999999999999999e-06 2024-09-17T21:22:17.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.012 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.012 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842453+0000", 2024-09-17T21:22:17.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:22:17.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842472+0000", 2024-09-17T21:22:17.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9006000000000001e-05 2024-09-17T21:22:17.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842487+0000", 2024-09-17T21:22:17.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4866e-05 2024-09-17T21:22:17.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843594+0000", 2024-09-17T21:22:17.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011073249999999999 2024-09-17T21:22:17.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843668+0000", 2024-09-17T21:22:17.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3188000000000005e-05 2024-09-17T21:22:17.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:17.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843893+0000", 2024-09-17T21:22:17.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022503399999999999 2024-09-17T21:22:17.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:17.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850439+0000", 2024-09-17T21:22:17.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065467880000000004 2024-09-17T21:22:17.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:17.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940409+0000", 2024-09-17T21:22:17.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089969511000000002 2024-09-17T21:22:17.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:17.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940444+0000", 2024-09-17T21:22:17.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5345999999999997e-05 2024-09-17T21:22:17.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.016 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.016 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940470+0000", 2024-09-17T21:22:17.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5964000000000001e-05 2024-09-17T21:22:17.016 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.016 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.016 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.016 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2707 3.1f 3:f805e65e:::rbd_data.1152a8ae2019.0000000000000049:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:17.016 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:22:17.016 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 156.14419844400001, 2024-09-17T21:22:17.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088327300999999997, 2024-09-17T21:22:17.017 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.017 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.017 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.017 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.017 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:17.017 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:17.017 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2707 2024-09-17T21:22:17.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.017 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.017 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.017 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:22:17.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:22:17.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847494+0000", 2024-09-17T21:22:17.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:22:17.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847504+0000", 2024-09-17T21:22:17.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:22:17.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847526+0000", 2024-09-17T21:22:17.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1976000000000001e-05 2024-09-17T21:22:17.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847541+0000", 2024-09-17T21:22:17.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5277e-05 2024-09-17T21:22:17.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856142+0000", 2024-09-17T21:22:17.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086006710000000007 2024-09-17T21:22:17.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856435+0000", 2024-09-17T21:22:17.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029301499999999999 2024-09-17T21:22:17.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935819+0000", 2024-09-17T21:22:17.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079384448999999996 2024-09-17T21:22:17.020 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.021 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.021 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.021 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2730 3.12 3:48b3422a:::rbd_data.1152a8ae2019.0000000000000057:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:17.021 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:22:17.021 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 156.13500554500001, 2024-09-17T21:22:17.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083879975999999995, 2024-09-17T21:22:17.021 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.021 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.021 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.021 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.021 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:17.022 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:17.022 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2730 2024-09-17T21:22:17.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.022 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.022 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.022 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:22:17.022 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:22:17.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856687+0000", 2024-09-17T21:22:17.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:22:17.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856697+0000", 2024-09-17T21:22:17.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0207000000000001e-05 2024-09-17T21:22:17.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856808+0000", 2024-09-17T21:22:17.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011071200000000001 2024-09-17T21:22:17.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856828+0000", 2024-09-17T21:22:17.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0219000000000001e-05 2024-09-17T21:22:17.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.026 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.026 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856907+0000", 2024-09-17T21:22:17.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9661999999999997e-05 2024-09-17T21:22:17.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.026 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.026 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.857019+0000", 2024-09-17T21:22:17.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011148499999999999 2024-09-17T21:22:17.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.026 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.026 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940565+0000", 2024-09-17T21:22:17.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083545791999999994 2024-09-17T21:22:17.027 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.027 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.027 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.027 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2743 3.12 3:4d036e62:::rbd_data.1152a8ae2019.000000000000005f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:17.027 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:22:17.027 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 156.13106004400001, 2024-09-17T21:22:17.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080059962999999998, 2024-09-17T21:22:17.027 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.027 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.027 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.027 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.027 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:17.027 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:17.028 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2743 2024-09-17T21:22:17.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.028 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:22:17.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:22:17.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860631+0000", 2024-09-17T21:22:17.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:22:17.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860636+0000", 2024-09-17T21:22:17.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5279999999999997e-06 2024-09-17T21:22:17.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860649+0000", 2024-09-17T21:22:17.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-17T21:22:17.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.030 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860655+0000", 2024-09-17T21:22:17.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:22:17.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.030 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860731+0000", 2024-09-17T21:22:17.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5920000000000005e-05 2024-09-17T21:22:17.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.861070+0000", 2024-09-17T21:22:17.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338535 2024-09-17T21:22:17.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940690+0000", 2024-09-17T21:22:17.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079620503999999995 2024-09-17T21:22:17.031 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.031 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.031 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.031 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3378 3.12 3:4832a51b:::rbd_data.1152a8ae2019.000000000000006c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:17.032 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:22:17.032 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 156.102696194, 2024-09-17T21:22:17.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051812516000000003, 2024-09-17T21:22:17.032 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.032 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.032 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:17.032 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.032 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:17.032 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:17.032 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3378 2024-09-17T21:22:17.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.032 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:22:17.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:22:17.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888995+0000", 2024-09-17T21:22:17.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:22:17.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.034 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889006+0000", 2024-09-17T21:22:17.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1012999999999999e-05 2024-09-17T21:22:17.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.034 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889032+0000", 2024-09-17T21:22:17.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6441e-05 2024-09-17T21:22:17.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.034 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889040+0000", 2024-09-17T21:22:17.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0639999999999994e-06 2024-09-17T21:22:17.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889278+0000", 2024-09-17T21:22:17.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023785899999999999 2024-09-17T21:22:17.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889408+0000", 2024-09-17T21:22:17.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129219 2024-09-17T21:22:17.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:17.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889723+0000", 2024-09-17T21:22:17.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000315264 2024-09-17T21:22:17.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:17.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.891972+0000", 2024-09-17T21:22:17.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002249633 2024-09-17T21:22:17.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:17.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940733+0000", 2024-09-17T21:22:17.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048760626000000001 2024-09-17T21:22:17.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:17.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940785+0000", 2024-09-17T21:22:17.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1906000000000001e-05 2024-09-17T21:22:17.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940807+0000", 2024-09-17T21:22:17.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1554999999999999e-05 2024-09-17T21:22:17.037 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.037 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.037 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.038 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3379 3.12 3:4e83fd71:::rbd_data.1152a8ae2019.000000000000006d:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:17.038 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:22:17.038 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 156.10261198699999, 2024-09-17T21:22:17.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051804144000000003, 2024-09-17T21:22:17.038 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.038 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.038 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:17.038 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.038 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:17.038 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:17.038 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3379 2024-09-17T21:22:17.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.038 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.039 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:22:17.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.039 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:22:17.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.039 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889079+0000", 2024-09-17T21:22:17.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1800000000000004e-07 2024-09-17T21:22:17.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889083+0000", 2024-09-17T21:22:17.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:22:17.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889094+0000", 2024-09-17T21:22:17.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1562999999999999e-05 2024-09-17T21:22:17.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889102+0000", 2024-09-17T21:22:17.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.401e-06 2024-09-17T21:22:17.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:22:17.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014279259999999999 2024-09-17T21:22:17.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890609+0000", 2024-09-17T21:22:17.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9566000000000004e-05 2024-09-17T21:22:17.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:17.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890789+0000", 2024-09-17T21:22:17.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017949700000000001 2024-09-17T21:22:17.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:17.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.893929+0000", 2024-09-17T21:22:17.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003140538 2024-09-17T21:22:17.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:17.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940834+0000", 2024-09-17T21:22:17.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046904811999999997 2024-09-17T21:22:17.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:17.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940860+0000", 2024-09-17T21:22:17.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5341999999999999e-05 2024-09-17T21:22:17.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940882+0000", 2024-09-17T21:22:17.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2878999999999999e-05 2024-09-17T21:22:17.043 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.043 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.044 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.044 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:2644 3.1e 3:79615585:::rbd_data.115266d0b9f2.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2475520~14848,write 3106304~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:22:17.044 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:22:17.044 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 147.479983584, 2024-09-17T21:22:17.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050733484000000002, 2024-09-17T21:22:17.044 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.044 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.044 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:17.044 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.044 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:22:17.044 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:22:17.044 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2644 2024-09-17T21:22:17.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.045 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:22:17.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:22:17.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511708+0000", 2024-09-17T21:22:17.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-17T21:22:17.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522196+0000", 2024-09-17T21:22:17.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010487712999999999 2024-09-17T21:22:17.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522223+0000", 2024-09-17T21:22:17.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7064999999999999e-05 2024-09-17T21:22:17.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522236+0000", 2024-09-17T21:22:17.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2931e-05 2024-09-17T21:22:17.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522348+0000", 2024-09-17T21:22:17.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111906 2024-09-17T21:22:17.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522554+0000", 2024-09-17T21:22:17.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020632699999999999 2024-09-17T21:22:17.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:22:17.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.523185+0000", 2024-09-17T21:22:17.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063081400000000005 2024-09-17T21:22:17.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:17.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.547011+0000", 2024-09-17T21:22:17.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023825570000000001 2024-09-17T21:22:17.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:17.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562268+0000", 2024-09-17T21:22:17.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015257514 2024-09-17T21:22:17.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.049 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:17.049 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562367+0000", 2024-09-17T21:22:17.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8678000000000006e-05 2024-09-17T21:22:17.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.049 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.049 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562440+0000", 2024-09-17T21:22:17.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3337000000000006e-05 2024-09-17T21:22:17.049 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.049 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.049 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.050 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:231 3.1 3:856fed78:::rbd_data.115220dd393e.000000000000000b:head [read 0~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:22:17.050 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:22:17.050 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 119.03722831499999, 2024-09-17T21:22:17.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34254251699999999, 2024-09-17T21:22:17.050 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.050 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.050 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:17.050 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.050 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:17.050 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:17.050 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 231 2024-09-17T21:22:17.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.050 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:22:17.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:22:17.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954464+0000", 2024-09-17T21:22:17.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:22:17.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954475+0000", 2024-09-17T21:22:17.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1146e-05 2024-09-17T21:22:17.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954506+0000", 2024-09-17T21:22:17.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0292999999999999e-05 2024-09-17T21:22:17.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954524+0000", 2024-09-17T21:22:17.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8430000000000001e-05 2024-09-17T21:22:17.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954661+0000", 2024-09-17T21:22:17.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136403 2024-09-17T21:22:17.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:17.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954680+0000", 2024-09-17T21:22:17.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9743000000000001e-05 2024-09-17T21:22:17.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.296896+0000", 2024-09-17T21:22:17.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34221550499999998 2024-09-17T21:22:17.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297004+0000", 2024-09-17T21:22:17.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010865399999999999 2024-09-17T21:22:17.054 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.054 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.054 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.054 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:56 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:22:17.055 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:22:17.055 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 118.88159461799999, 2024-09-17T21:22:17.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18718773899999999, 2024-09-17T21:22:17.055 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.055 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.055 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.055 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.055 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:22:17.055 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:22:17.055 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 56 2024-09-17T21:22:17.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.055 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.055 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:22:17.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:22:17.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110098+0000", 2024-09-17T21:22:17.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:22:17.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.057 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.057 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110111+0000", 2024-09-17T21:22:17.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2289999999999999e-05 2024-09-17T21:22:17.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.057 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.057 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110143+0000", 2024-09-17T21:22:17.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1979e-05 2024-09-17T21:22:17.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.057 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.057 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110161+0000", 2024-09-17T21:22:17.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7839e-05 2024-09-17T21:22:17.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110319+0000", 2024-09-17T21:22:17.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015835400000000001 2024-09-17T21:22:17.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:17.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110353+0000", 2024-09-17T21:22:17.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3849000000000001e-05 2024-09-17T21:22:17.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297041+0000", 2024-09-17T21:22:17.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18668842299999999 2024-09-17T21:22:17.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297169+0000", 2024-09-17T21:22:17.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012786199999999999 2024-09-17T21:22:17.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297283+0000", 2024-09-17T21:22:17.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011430300000000001 2024-09-17T21:22:17.059 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.059 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.059 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.060 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.060 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:68 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:22:17.060 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:22:17.060 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 118.873053687, 2024-09-17T21:22:17.060 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17885582999999999, 2024-09-17T21:22:17.060 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.060 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.060 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.060 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.060 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:22:17.060 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:22:17.060 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 68 2024-09-17T21:22:17.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.060 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.061 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.061 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:22:17.061 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.061 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.061 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:22:17.061 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.061 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.061 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118639+0000", 2024-09-17T21:22:17.061 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.306e-06 2024-09-17T21:22:17.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.062 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.062 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.062 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118651+0000", 2024-09-17T21:22:17.062 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2567e-05 2024-09-17T21:22:17.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.062 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.062 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.062 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118682+0000", 2024-09-17T21:22:17.062 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0502e-05 2024-09-17T21:22:17.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.062 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.062 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118707+0000", 2024-09-17T21:22:17.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5463e-05 2024-09-17T21:22:17.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118889+0000", 2024-09-17T21:22:17.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018150999999999999 2024-09-17T21:22:17.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:17.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118910+0000", 2024-09-17T21:22:17.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:22:17.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.064 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.064 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297313+0000", 2024-09-17T21:22:17.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17840366699999999 2024-09-17T21:22:17.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.064 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.064 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297389+0000", 2024-09-17T21:22:17.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-17T21:22:17.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.064 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.064 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297492+0000", 2024-09-17T21:22:17.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103019 2024-09-17T21:22:17.065 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.065 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.065 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.065 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.065 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5890.0:15234 3.1 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8451072~29696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-17T21:22:17.065 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:22:17.065 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 73.898404874999997, 2024-09-17T21:22:17.065 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.057026052000000001, 2024-09-17T21:22:17.065 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.065 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.065 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:17.065 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.065 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5890", 2024-09-17T21:22:17.066 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3064344947", 2024-09-17T21:22:17.066 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 15234 2024-09-17T21:22:17.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.066 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.066 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.066 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:22:17.066 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.066 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.066 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:22:17.066 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093289+0000", 2024-09-17T21:22:17.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4939999999999999e-06 2024-09-17T21:22:17.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103074+0000", 2024-09-17T21:22:17.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00978564 2024-09-17T21:22:17.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103112+0000", 2024-09-17T21:22:17.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7246999999999998e-05 2024-09-17T21:22:17.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.068 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.068 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103128+0000", 2024-09-17T21:22:17.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5812999999999999e-05 2024-09-17T21:22:17.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.068 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.068 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103207+0000", 2024-09-17T21:22:17.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9785999999999998e-05 2024-09-17T21:22:17.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.068 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103315+0000", 2024-09-17T21:22:17.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107396 2024-09-17T21:22:17.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.069 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.069 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:22:17.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103957+0000", 2024-09-17T21:22:17.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000642524 2024-09-17T21:22:17.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.069 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.069 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:17.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.123856+0000", 2024-09-17T21:22:17.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019898376999999998 2024-09-17T21:22:17.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.070 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.070 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:17.070 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150190+0000", 2024-09-17T21:22:17.070 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026334682000000002 2024-09-17T21:22:17.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.070 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.070 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:17.070 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150255+0000", 2024-09-17T21:22:17.070 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4770999999999994e-05 2024-09-17T21:22:17.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.070 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.070 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.070 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150311+0000", 2024-09-17T21:22:17.070 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6322e-05 2024-09-17T21:22:17.071 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.071 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.071 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.071 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4239 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 424960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:22:17.071 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:22:17.071 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.238007218000007, 2024-09-17T21:22:17.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050123167000000003, 2024-09-17T21:22:17.071 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.071 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.071 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.071 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.072 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:22:17.072 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:22:17.072 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4239 2024-09-17T21:22:17.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.072 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:22:17.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:22:17.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753684+0000", 2024-09-17T21:22:17.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.37e-07 2024-09-17T21:22:17.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753686+0000", 2024-09-17T21:22:17.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.322e-06 2024-09-17T21:22:17.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753693+0000", 2024-09-17T21:22:17.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0890000000000002e-06 2024-09-17T21:22:17.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753699+0000", 2024-09-17T21:22:17.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:22:17.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803390+0000", 2024-09-17T21:22:17.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049690506000000002 2024-09-17T21:22:17.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.075 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.075 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803569+0000", 2024-09-17T21:22:17.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179784 2024-09-17T21:22:17.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.075 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.075 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803806+0000", 2024-09-17T21:22:17.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023683499999999999 2024-09-17T21:22:17.075 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.075 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.075 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.076 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.076 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4240 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 413696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:22:17.076 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:22:17.076 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.237983490000005, 2024-09-17T21:22:17.076 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050331353000000002, 2024-09-17T21:22:17.076 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.076 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.076 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.076 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.076 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:22:17.076 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:22:17.076 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4240 2024-09-17T21:22:17.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.077 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:22:17.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:22:17.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:22:17.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5700000000000002e-07 2024-09-17T21:22:17.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753710+0000", 2024-09-17T21:22:17.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.385e-06 2024-09-17T21:22:17.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753717+0000", 2024-09-17T21:22:17.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.832e-06 2024-09-17T21:22:17.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.079 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.079 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753722+0000", 2024-09-17T21:22:17.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8050000000000003e-06 2024-09-17T21:22:17.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.079 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.079 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803880+0000", 2024-09-17T21:22:17.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050157641000000003 2024-09-17T21:22:17.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.079 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.079 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803945+0000", 2024-09-17T21:22:17.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5357000000000005e-05 2024-09-17T21:22:17.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804038+0000", 2024-09-17T21:22:17.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2775999999999994e-05 2024-09-17T21:22:17.080 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.080 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.080 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.080 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4241 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 415744~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:22:17.080 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:22:17.080 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.237960373999996, 2024-09-17T21:22:17.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050578790999999998, 2024-09-17T21:22:17.081 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.081 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.081 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.081 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.081 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:22:17.081 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:22:17.081 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4241 2024-09-17T21:22:17.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.081 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.081 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.081 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:22:17.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.082 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:22:17.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.082 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:22:17.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5700000000000002e-07 2024-09-17T21:22:17.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.082 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753733+0000", 2024-09-17T21:22:17.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2570000000000002e-06 2024-09-17T21:22:17.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.083 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.083 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753740+0000", 2024-09-17T21:22:17.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.849e-06 2024-09-17T21:22:17.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.083 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.083 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753745+0000", 2024-09-17T21:22:17.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7450000000000001e-06 2024-09-17T21:22:17.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.084 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804115+0000", 2024-09-17T21:22:17.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050370038999999998 2024-09-17T21:22:17.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.084 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804223+0000", 2024-09-17T21:22:17.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010786 2024-09-17T21:22:17.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.084 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804309+0000", 2024-09-17T21:22:17.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5483999999999997e-05 2024-09-17T21:22:17.085 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.085 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.085 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.085 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4242 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 414720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:22:17.085 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:22:17.085 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.237937486000007, 2024-09-17T21:22:17.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050748148999999999, 2024-09-17T21:22:17.085 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.085 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.085 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.085 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.086 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:22:17.086 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:22:17.086 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4242 2024-09-17T21:22:17.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.086 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:22:17.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.087 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.087 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:22:17.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.087 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.087 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:22:17.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6599999999999996e-07 2024-09-17T21:22:17.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.087 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.087 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753756+0000", 2024-09-17T21:22:17.088 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.368e-06 2024-09-17T21:22:17.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.088 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.088 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753763+0000", 2024-09-17T21:22:17.088 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9349999999999997e-06 2024-09-17T21:22:17.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.088 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.088 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753768+0000", 2024-09-17T21:22:17.088 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7620000000000001e-06 2024-09-17T21:22:17.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804343+0000", 2024-09-17T21:22:17.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050575050000000003 2024-09-17T21:22:17.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804397+0000", 2024-09-17T21:22:17.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4070000000000002e-05 2024-09-17T21:22:17.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804501+0000", 2024-09-17T21:22:17.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103398 2024-09-17T21:22:17.089 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.090 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.090 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.090 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25642 3.1 3.6a5ad301 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e177)", 2024-09-17T21:22:17.090 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:34.316616+0000", 2024-09-17T21:22:17.090 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 42.675073902999998, 2024-09-17T21:22:17.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99065658300000003, 2024-09-17T21:22:17.090 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.090 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.090 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:17.090 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.090 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:22:17.091 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:17.091 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25642 2024-09-17T21:22:17.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.091 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.091 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.091 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316616+0000", 2024-09-17T21:22:17.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.091 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.091 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316616+0000", 2024-09-17T21:22:17.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.092 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.092 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.092 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316618+0000", 2024-09-17T21:22:17.092 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9309999999999998e-06 2024-09-17T21:22:17.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.092 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.092 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.092 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316624+0000", 2024-09-17T21:22:17.092 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2789999999999998e-06 2024-09-17T21:22:17.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.092 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.092 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.092 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316634+0000", 2024-09-17T21:22:17.092 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0774000000000001e-05 2024-09-17T21:22:17.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.093 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.093 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316644+0000", 2024-09-17T21:22:17.093 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0043e-05 2024-09-17T21:22:17.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.093 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.093 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316767+0000", 2024-09-17T21:22:17.093 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012289199999999999 2024-09-17T21:22:17.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.093 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:17.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316777+0000", 2024-09-17T21:22:17.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6199999999999994e-06 2024-09-17T21:22:17.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.307241+0000", 2024-09-17T21:22:17.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99046395099999995 2024-09-17T21:22:17.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.307273+0000", 2024-09-17T21:22:17.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2092999999999999e-05 2024-09-17T21:22:17.094 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25905 3.6 3:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-17T21:22:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:39.359572+0000", 2024-09-17T21:22:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.632118621000004, 2024-09-17T21:22:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96065702100000006, 2024-09-17T21:22:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:22:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25905 2024-09-17T21:22:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359572+0000", 2024-09-17T21:22:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359572+0000", 2024-09-17T21:22:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359573+0000", 2024-09-17T21:22:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5370000000000001e-06 2024-09-17T21:22:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359579+0000", 2024-09-17T21:22:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8039999999999998e-06 2024-09-17T21:22:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359603+0000", 2024-09-17T21:22:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4533999999999999e-05 2024-09-17T21:22:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359613+0000", 2024-09-17T21:22:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2820000000000002e-06 2024-09-17T21:22:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359686+0000", 2024-09-17T21:22:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3301999999999997e-05 2024-09-17T21:22:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359693+0000", 2024-09-17T21:22:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3679999999999996e-06 2024-09-17T21:22:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:40.319999+0000", 2024-09-17T21:22:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96030524500000003 2024-09-17T21:22:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:40.320229+0000", 2024-09-17T21:22:17.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022994899999999999 2024-09-17T21:22:17.100 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.100 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.100 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.100 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.100 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.100 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:17.101 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:22:17.102 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:17.097+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:17.203 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:17.201+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:17.239 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:17.239 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:22:17.239 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:22:17.239 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:17.255 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:22:17.304 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:17.301+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:17.394 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:17.394 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:22:17.394 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:22:17.394 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:22:17.394 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:17.405 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:17.401+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:17.409 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:22:17.506 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:17.501+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:17.521 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:22:17.542 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:17.542 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:22:17.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:22:17.542 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:22:17.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.542 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:22:17.543 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:22:17.543 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 217.73020578699999, 2024-09-17T21:22:17.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:22:17.543 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.543 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.543 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.543 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.543 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:22:17.543 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:22:17.543 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:22:17.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.543 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:22:17.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.544 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:22:17.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.544 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:22:17.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:22:17.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.544 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:22:17.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:22:17.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:22:17.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:22:17.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:22:17.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:22:17.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:22:17.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:22:17.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:17.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:22:17.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:22:17.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:22:17.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:22:17.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:22:17.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:22:17.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:22:17.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:22:17.547 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.547 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.547 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.547 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:22:17.547 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:22:17.548 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 204.17510400099999, 2024-09-17T21:22:17.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:22:17.548 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.548 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.548 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:17.548 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.548 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:22:17.548 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:17.548 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:22:17.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.548 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:22:17.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:22:17.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:22:17.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:22:17.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:22:17.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:22:17.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:22:17.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:22:17.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:22:17.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:22:17.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:22:17.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:22:17.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:17.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:22:17.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:22:17.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:22:17.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:22:17.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:22:17.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:22:17.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:22:17.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:22:17.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:22:17.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:17.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:22:17.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:22:17.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:17.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:22:17.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:22:17.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:17.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:22:17.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:22:17.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:22:17.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:22:17.554 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.554 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.554 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.554 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:793 3.9 3:95f63599:::rbd_data.1152eb799094.0000000000000007:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:22:17.554 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:22:17.554 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 121.10804784600001, 2024-09-17T21:22:17.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88245766400000003, 2024-09-17T21:22:17.554 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.554 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.554 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:17.554 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.555 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:22:17.555 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:22:17.555 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 793 2024-09-17T21:22:17.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.555 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:22:17.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:22:17.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431941+0000", 2024-09-17T21:22:17.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.864e-06 2024-09-17T21:22:17.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431953+0000", 2024-09-17T21:22:17.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1486e-05 2024-09-17T21:22:17.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431984+0000", 2024-09-17T21:22:17.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1075999999999999e-05 2024-09-17T21:22:17.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431999+0000", 2024-09-17T21:22:17.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.503e-05 2024-09-17T21:22:17.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432095+0000", 2024-09-17T21:22:17.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5698000000000004e-05 2024-09-17T21:22:17.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:17.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432109+0000", 2024-09-17T21:22:17.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4188e-05 2024-09-17T21:22:17.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.309642+0000", 2024-09-17T21:22:17.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87753353000000001 2024-09-17T21:22:17.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310104+0000", 2024-09-17T21:22:17.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046103600000000001 2024-09-17T21:22:17.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:17.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310630+0000", 2024-09-17T21:22:17.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052682800000000002 2024-09-17T21:22:17.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:17.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313100+0000", 2024-09-17T21:22:17.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024693369999999998 2024-09-17T21:22:17.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:17.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313935+0000", 2024-09-17T21:22:17.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083540100000000005 2024-09-17T21:22:17.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.560 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:17.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314086+0000", 2024-09-17T21:22:17.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015053399999999999 2024-09-17T21:22:17.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.560 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314396+0000", 2024-09-17T21:22:17.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031065599999999997 2024-09-17T21:22:17.560 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.560 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.560 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.560 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:800 3.19 3:9dc17f12:::rbd_data.1152eb799094.000000000000000e:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:22:17.561 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:22:17.561 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 121.10780060099999, 2024-09-17T21:22:17.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88154663799999999, 2024-09-17T21:22:17.561 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.561 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.561 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:17.561 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.561 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:22:17.561 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:22:17.561 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 800 2024-09-17T21:22:17.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.561 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:22:17.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:22:17.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432187+0000", 2024-09-17T21:22:17.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4100000000000001e-06 2024-09-17T21:22:17.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432194+0000", 2024-09-17T21:22:17.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0389999999999999e-06 2024-09-17T21:22:17.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432216+0000", 2024-09-17T21:22:17.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1614e-05 2024-09-17T21:22:17.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432232+0000", 2024-09-17T21:22:17.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5739e-05 2024-09-17T21:22:17.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432405+0000", 2024-09-17T21:22:17.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017355300000000001 2024-09-17T21:22:17.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:17.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432432+0000", 2024-09-17T21:22:17.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6979e-05 2024-09-17T21:22:17.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.307767+0000", 2024-09-17T21:22:17.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87533472899999998 2024-09-17T21:22:17.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.565 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.565 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308192+0000", 2024-09-17T21:22:17.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042485999999999999 2024-09-17T21:22:17.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.565 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:17.565 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308727+0000", 2024-09-17T21:22:17.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00053550400000000001 2024-09-17T21:22:17.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.565 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:17.565 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310347+0000", 2024-09-17T21:22:17.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016198670000000001 2024-09-17T21:22:17.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:17.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313424+0000", 2024-09-17T21:22:17.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030764640000000001 2024-09-17T21:22:17.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:17.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313510+0000", 2024-09-17T21:22:17.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5938000000000003e-05 2024-09-17T21:22:17.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313733+0000", 2024-09-17T21:22:17.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022294200000000001 2024-09-17T21:22:17.566 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.567 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.567 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.567 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:805 3.19 3:9cab2ed6:::rbd_data.1152eb799094.0000000000000013:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:22:17.567 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:22:17.567 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 121.107596582, 2024-09-17T21:22:17.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88274355699999996, 2024-09-17T21:22:17.567 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.567 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.567 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:17.567 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.567 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:22:17.567 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:22:17.567 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 805 2024-09-17T21:22:17.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.568 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.568 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.568 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.568 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:22:17.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.568 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.568 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.568 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:22:17.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.568 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.568 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.568 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432391+0000", 2024-09-17T21:22:17.569 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3e-06 2024-09-17T21:22:17.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.569 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.569 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432398+0000", 2024-09-17T21:22:17.569 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4489999999999996e-06 2024-09-17T21:22:17.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.569 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.569 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432415+0000", 2024-09-17T21:22:17.569 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7750000000000001e-05 2024-09-17T21:22:17.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.569 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.569 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432430+0000", 2024-09-17T21:22:17.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.449e-05 2024-09-17T21:22:17.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.570 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.570 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432635+0000", 2024-09-17T21:22:17.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205031 2024-09-17T21:22:17.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.570 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:17.570 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432657+0000", 2024-09-17T21:22:17.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2064000000000001e-05 2024-09-17T21:22:17.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.570 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.570 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310444+0000", 2024-09-17T21:22:17.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87778699199999999 2024-09-17T21:22:17.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.571 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.571 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.571 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310849+0000", 2024-09-17T21:22:17.571 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00040497800000000001 2024-09-17T21:22:17.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.571 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.571 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:17.571 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.311396+0000", 2024-09-17T21:22:17.571 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00054729099999999999 2024-09-17T21:22:17.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.571 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.571 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:17.571 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313847+0000", 2024-09-17T21:22:17.571 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024504940000000001 2024-09-17T21:22:17.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.572 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:17.572 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314879+0000", 2024-09-17T21:22:17.572 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001031817 2024-09-17T21:22:17.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.572 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:17.572 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314964+0000", 2024-09-17T21:22:17.572 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5685000000000004e-05 2024-09-17T21:22:17.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.572 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.572 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.315133+0000", 2024-09-17T21:22:17.573 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000169216 2024-09-17T21:22:17.573 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.573 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.573 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.573 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.573 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12089 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 60416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:17.573 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:22:17.573 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 115.77037154999999, 2024-09-17T21:22:17.573 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088293233999999998, 2024-09-17T21:22:17.573 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.573 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.573 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.573 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.573 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:17.574 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:17.574 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12089 2024-09-17T21:22:17.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.574 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.574 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.574 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:22:17.574 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.574 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.574 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:22:17.574 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.575 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.575 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.575 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769616+0000", 2024-09-17T21:22:17.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:22:17.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.575 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.575 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.575 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769620+0000", 2024-09-17T21:22:17.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5140000000000002e-06 2024-09-17T21:22:17.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.575 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.575 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.575 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769633+0000", 2024-09-17T21:22:17.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2849e-05 2024-09-17T21:22:17.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.576 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.576 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.576 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769644+0000", 2024-09-17T21:22:17.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1471000000000001e-05 2024-09-17T21:22:17.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.576 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.576 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.576 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857564+0000", 2024-09-17T21:22:17.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087919445999999998 2024-09-17T21:22:17.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.576 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.576 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.576 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857716+0000", 2024-09-17T21:22:17.577 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000152065 2024-09-17T21:22:17.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.577 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.577 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857908+0000", 2024-09-17T21:22:17.577 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019252599999999999 2024-09-17T21:22:17.577 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.577 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.577 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.577 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12090 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 64512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:17.577 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:22:17.577 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 115.770325165, 2024-09-17T21:22:17.577 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088582672000000001, 2024-09-17T21:22:17.578 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.578 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.578 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.578 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.578 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:17.578 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:17.578 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12090 2024-09-17T21:22:17.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.578 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.578 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.578 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.578 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:22:17.578 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.579 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.579 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:22:17.579 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.579 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.579 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769662+0000", 2024-09-17T21:22:17.579 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:22:17.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.579 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.580 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769683+0000", 2024-09-17T21:22:17.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0979e-05 2024-09-17T21:22:17.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.580 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.580 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.580 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769699+0000", 2024-09-17T21:22:17.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5446000000000001e-05 2024-09-17T21:22:17.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.580 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.580 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.580 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769708+0000", 2024-09-17T21:22:17.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1679999999999993e-06 2024-09-17T21:22:17.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.581 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.581 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857976+0000", 2024-09-17T21:22:17.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088267795999999996 2024-09-17T21:22:17.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.581 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.581 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858095+0000", 2024-09-17T21:22:17.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119147 2024-09-17T21:22:17.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.581 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.581 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858244+0000", 2024-09-17T21:22:17.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014917000000000001 2024-09-17T21:22:17.581 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.582 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.582 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.582 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.582 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12091 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 68608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:17.582 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:22:17.582 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 115.77025378099999, 2024-09-17T21:22:17.582 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088832752000000001, 2024-09-17T21:22:17.582 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.582 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.582 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.582 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.582 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:17.582 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:17.583 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12091 2024-09-17T21:22:17.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.583 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.583 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.583 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:22:17.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.583 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.583 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:22:17.583 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.584 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.584 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769734+0000", 2024-09-17T21:22:17.584 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:22:17.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.584 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.584 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.584 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769739+0000", 2024-09-17T21:22:17.584 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8080000000000004e-06 2024-09-17T21:22:17.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.584 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.584 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.584 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769764+0000", 2024-09-17T21:22:17.584 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4839999999999999e-05 2024-09-17T21:22:17.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.584 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.585 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.585 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769777+0000", 2024-09-17T21:22:17.585 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3037000000000001e-05 2024-09-17T21:22:17.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.585 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.585 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.585 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858295+0000", 2024-09-17T21:22:17.585 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088518475999999999 2024-09-17T21:22:17.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.585 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.585 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.585 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858419+0000", 2024-09-17T21:22:17.585 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012378700000000001 2024-09-17T21:22:17.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.586 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.586 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.586 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858565+0000", 2024-09-17T21:22:17.586 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014630099999999999 2024-09-17T21:22:17.586 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.586 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.586 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.586 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.586 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12092 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:17.586 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:22:17.586 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 115.770171457, 2024-09-17T21:22:17.586 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089191555000000006, 2024-09-17T21:22:17.587 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.587 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.587 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.587 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.587 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:17.587 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:17.587 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12092 2024-09-17T21:22:17.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.587 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.587 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.587 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:22:17.587 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.588 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.588 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:22:17.588 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.588 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.588 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769816+0000", 2024-09-17T21:22:17.588 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7199999999999997e-07 2024-09-17T21:22:17.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.589 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.589 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.589 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769822+0000", 2024-09-17T21:22:17.589 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:22:17.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.589 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.589 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.589 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769838+0000", 2024-09-17T21:22:17.589 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6674e-05 2024-09-17T21:22:17.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.589 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.589 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.589 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769851+0000", 2024-09-17T21:22:17.589 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2527999999999999e-05 2024-09-17T21:22:17.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.590 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858660+0000", 2024-09-17T21:22:17.590 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088808556999999996 2024-09-17T21:22:17.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.590 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858817+0000", 2024-09-17T21:22:17.590 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015730400000000001 2024-09-17T21:22:17.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.590 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859007+0000", 2024-09-17T21:22:17.591 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018974899999999999 2024-09-17T21:22:17.591 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.591 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.591 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.591 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12093 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:17.591 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:22:17.591 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 115.770063353, 2024-09-17T21:22:17.591 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089429710999999995, 2024-09-17T21:22:17.591 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.591 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.591 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.591 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.591 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:17.592 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:17.592 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12093 2024-09-17T21:22:17.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.592 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.592 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:22:17.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.592 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:22:17.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.593 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.593 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769924+0000", 2024-09-17T21:22:17.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.342e-06 2024-09-17T21:22:17.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.593 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.593 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769929+0000", 2024-09-17T21:22:17.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0350000000000001e-06 2024-09-17T21:22:17.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.593 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.593 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769946+0000", 2024-09-17T21:22:17.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6793999999999999e-05 2024-09-17T21:22:17.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.594 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.594 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769959+0000", 2024-09-17T21:22:17.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3062e-05 2024-09-17T21:22:17.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.594 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.594 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859075+0000", 2024-09-17T21:22:17.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089115696999999994 2024-09-17T21:22:17.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.594 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.594 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859202+0000", 2024-09-17T21:22:17.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012657 2024-09-17T21:22:17.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.595 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.595 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859353+0000", 2024-09-17T21:22:17.595 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151211 2024-09-17T21:22:17.595 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.595 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.595 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.595 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12094 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:17.595 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:22:17.595 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 115.77000917700001, 2024-09-17T21:22:17.596 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089695001999999996, 2024-09-17T21:22:17.596 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.596 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.596 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.596 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.596 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:17.596 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:17.596 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12094 2024-09-17T21:22:17.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.596 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.596 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.596 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:22:17.596 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:22:17.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769978+0000", 2024-09-17T21:22:17.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.142e-06 2024-09-17T21:22:17.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769983+0000", 2024-09-17T21:22:17.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.386e-06 2024-09-17T21:22:17.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769997+0000", 2024-09-17T21:22:17.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.416e-05 2024-09-17T21:22:17.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770012+0000", 2024-09-17T21:22:17.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5384999999999999e-05 2024-09-17T21:22:17.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859405+0000", 2024-09-17T21:22:17.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089392376999999995 2024-09-17T21:22:17.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859525+0000", 2024-09-17T21:22:17.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120555 2024-09-17T21:22:17.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859672+0000", 2024-09-17T21:22:17.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000146997 2024-09-17T21:22:17.599 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.599 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.600 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.600 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12095 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:17.600 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:22:17.600 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 115.76995669, 2024-09-17T21:22:17.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090073476, 2024-09-17T21:22:17.600 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.600 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.600 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.600 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.600 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:17.600 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:17.600 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12095 2024-09-17T21:22:17.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.601 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.601 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:22:17.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.601 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:22:17.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770031+0000", 2024-09-17T21:22:17.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1699999999999997e-07 2024-09-17T21:22:17.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.602 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770036+0000", 2024-09-17T21:22:17.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.023e-06 2024-09-17T21:22:17.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.602 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.602 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770051+0000", 2024-09-17T21:22:17.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4953999999999999e-05 2024-09-17T21:22:17.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770063+0000", 2024-09-17T21:22:17.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:22:17.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859756+0000", 2024-09-17T21:22:17.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089692980000000005 2024-09-17T21:22:17.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859891+0000", 2024-09-17T21:22:17.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013459499999999999 2024-09-17T21:22:17.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860103+0000", 2024-09-17T21:22:17.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021274400000000001 2024-09-17T21:22:17.604 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.604 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.604 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.604 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12096 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:17.604 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:22:17.604 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 115.76990700499999, 2024-09-17T21:22:17.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090358274000000002, 2024-09-17T21:22:17.604 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.604 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.605 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.605 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.605 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:17.605 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:17.605 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12096 2024-09-17T21:22:17.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.605 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.605 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.605 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:22:17.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.605 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:22:17.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770080+0000", 2024-09-17T21:22:17.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9400000000000004e-07 2024-09-17T21:22:17.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770085+0000", 2024-09-17T21:22:17.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2169999999999996e-06 2024-09-17T21:22:17.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770100+0000", 2024-09-17T21:22:17.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5077e-05 2024-09-17T21:22:17.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770112+0000", 2024-09-17T21:22:17.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1951e-05 2024-09-17T21:22:17.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860160+0000", 2024-09-17T21:22:17.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090048413999999993 2024-09-17T21:22:17.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860285+0000", 2024-09-17T21:22:17.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012466699999999999 2024-09-17T21:22:17.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860438+0000", 2024-09-17T21:22:17.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015305399999999999 2024-09-17T21:22:17.608 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.608 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.608 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.608 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12097 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:17.608 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:22:17.608 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 115.769859258, 2024-09-17T21:22:17.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090702375000000002, 2024-09-17T21:22:17.609 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.609 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.609 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.609 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.609 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:17.609 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:17.609 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12097 2024-09-17T21:22:17.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.609 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.609 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.609 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:22:17.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.609 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:22:17.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770128+0000", 2024-09-17T21:22:17.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.04e-06 2024-09-17T21:22:17.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770133+0000", 2024-09-17T21:22:17.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8999999999999997e-06 2024-09-17T21:22:17.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770152+0000", 2024-09-17T21:22:17.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8890999999999999e-05 2024-09-17T21:22:17.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770164+0000", 2024-09-17T21:22:17.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1769e-05 2024-09-17T21:22:17.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860511+0000", 2024-09-17T21:22:17.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090347527999999996 2024-09-17T21:22:17.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860652+0000", 2024-09-17T21:22:17.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140894 2024-09-17T21:22:17.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860830+0000", 2024-09-17T21:22:17.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017735299999999999 2024-09-17T21:22:17.612 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.612 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.612 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.612 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12098 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:17.612 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:22:17.613 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 115.76980579000001, 2024-09-17T21:22:17.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090997262999999995, 2024-09-17T21:22:17.613 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.613 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.613 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.613 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.613 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:17.613 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:17.613 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12098 2024-09-17T21:22:17.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.613 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.613 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:22:17.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:22:17.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775465+0000", 2024-09-17T21:22:17.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0052843350000000002 2024-09-17T21:22:17.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775476+0000", 2024-09-17T21:22:17.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0797e-05 2024-09-17T21:22:17.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775506+0000", 2024-09-17T21:22:17.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9709999999999998e-05 2024-09-17T21:22:17.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775530+0000", 2024-09-17T21:22:17.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4528999999999998e-05 2024-09-17T21:22:17.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860894+0000", 2024-09-17T21:22:17.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.085364195000000004 2024-09-17T21:22:17.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861019+0000", 2024-09-17T21:22:17.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012467399999999999 2024-09-17T21:22:17.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861178+0000", 2024-09-17T21:22:17.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159023 2024-09-17T21:22:17.616 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.616 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.617 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.617 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12105 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:17.617 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:22:17.617 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 115.764075787, 2024-09-17T21:22:17.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088330400000000003, 2024-09-17T21:22:17.617 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.617 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.617 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.617 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.617 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:17.617 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:17.617 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12105 2024-09-17T21:22:17.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.618 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:22:17.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:22:17.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:22:17.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9999999999999997e-07 2024-09-17T21:22:17.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775914+0000", 2024-09-17T21:22:17.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4710000000000001e-06 2024-09-17T21:22:17.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775921+0000", 2024-09-17T21:22:17.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:22:17.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775928+0000", 2024-09-17T21:22:17.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:22:17.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863923+0000", 2024-09-17T21:22:17.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087995126000000007 2024-09-17T21:22:17.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864059+0000", 2024-09-17T21:22:17.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135797 2024-09-17T21:22:17.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864241+0000", 2024-09-17T21:22:17.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018252599999999999 2024-09-17T21:22:17.621 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.621 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.621 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.621 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12106 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:17.621 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:22:17.621 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 115.763986609, 2024-09-17T21:22:17.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088586394999999998, 2024-09-17T21:22:17.621 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.621 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.622 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.622 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.622 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:17.622 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:17.622 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12106 2024-09-17T21:22:17.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.622 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:22:17.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:22:17.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776001+0000", 2024-09-17T21:22:17.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4300000000000001e-07 2024-09-17T21:22:17.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776004+0000", 2024-09-17T21:22:17.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8049999999999998e-06 2024-09-17T21:22:17.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776018+0000", 2024-09-17T21:22:17.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4613999999999999e-05 2024-09-17T21:22:17.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776025+0000", 2024-09-17T21:22:17.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:22:17.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864279+0000", 2024-09-17T21:22:17.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088253570000000003 2024-09-17T21:22:17.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864433+0000", 2024-09-17T21:22:17.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015469400000000001 2024-09-17T21:22:17.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864586+0000", 2024-09-17T21:22:17.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000153003 2024-09-17T21:22:17.625 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.625 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.625 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.625 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25033 3.6 3.b74753e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e168)", 2024-09-17T21:22:17.625 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:22:17.626 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 52.030543344999998, 2024-09-17T21:22:17.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93781952099999999, 2024-09-17T21:22:17.626 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.626 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.626 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:17.626 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.626 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:22:17.626 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:17.626 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25033 2024-09-17T21:22:17.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.626 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:22:17.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:22:17.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509444+0000", 2024-09-17T21:22:17.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0020000000000001e-06 2024-09-17T21:22:17.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509450+0000", 2024-09-17T21:22:17.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4249999999999997e-06 2024-09-17T21:22:17.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509460+0000", 2024-09-17T21:22:17.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0183999999999999e-05 2024-09-17T21:22:17.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509466+0000", 2024-09-17T21:22:17.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0039999999999996e-06 2024-09-17T21:22:17.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509540+0000", 2024-09-17T21:22:17.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3847e-05 2024-09-17T21:22:17.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:17.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509551+0000", 2024-09-17T21:22:17.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1731e-05 2024-09-17T21:22:17.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.447243+0000", 2024-09-17T21:22:17.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93769166000000004 2024-09-17T21:22:17.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.447263+0000", 2024-09-17T21:22:17.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9667999999999999e-05 2024-09-17T21:22:17.630 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.630 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.630 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.630 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25033 3.6 3:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e168)", 2024-09-17T21:22:17.630 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:22:17.630 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 52.029017748000001, 2024-09-17T21:22:17.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.937309383, 2024-09-17T21:22:17.630 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.630 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.630 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:17.630 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.630 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:22:17.631 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:17.631 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25033 2024-09-17T21:22:17.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.631 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:22:17.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:22:17.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510970+0000", 2024-09-17T21:22:17.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.564e-06 2024-09-17T21:22:17.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510978+0000", 2024-09-17T21:22:17.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.362e-06 2024-09-17T21:22:17.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510998+0000", 2024-09-17T21:22:17.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0336999999999999e-05 2024-09-17T21:22:17.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511010+0000", 2024-09-17T21:22:17.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1786e-05 2024-09-17T21:22:17.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511080+0000", 2024-09-17T21:22:17.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0686999999999996e-05 2024-09-17T21:22:17.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:17.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511087+0000", 2024-09-17T21:22:17.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4899999999999997e-06 2024-09-17T21:22:17.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.448050+0000", 2024-09-17T21:22:17.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93696351700000002 2024-09-17T21:22:17.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.448278+0000", 2024-09-17T21:22:17.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022764000000000001 2024-09-17T21:22:17.635 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.635 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.636 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.636 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:30858 3.1a 3:5b9355dc:::rbd_header.11527824718:head [watch reconnect cookie 139860845522384 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e218)", 2024-09-17T21:22:17.636 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:22:15.639403+0000", 2024-09-17T21:22:17.636 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.900583124, 2024-09-17T21:22:17.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.041145395, 2024-09-17T21:22:17.636 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:17.636 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:17.636 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:17.636 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:17.636 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:22:17.636 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:17.636 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 30858 2024-09-17T21:22:17.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.637 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:17.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:17.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639403+0000", 2024-09-17T21:22:17.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:17.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639403+0000", 2024-09-17T21:22:17.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:17.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:17.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639406+0000", 2024-09-17T21:22:17.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2979999999999999e-06 2024-09-17T21:22:17.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:17.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639412+0000", 2024-09-17T21:22:17.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.494e-06 2024-09-17T21:22:17.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:17.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639423+0000", 2024-09-17T21:22:17.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1277000000000001e-05 2024-09-17T21:22:17.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:17.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639433+0000", 2024-09-17T21:22:17.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7699999999999996e-06 2024-09-17T21:22:17.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639520+0000", 2024-09-17T21:22:17.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7121999999999997e-05 2024-09-17T21:22:17.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:17.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639534+0000", 2024-09-17T21:22:17.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3651e-05 2024-09-17T21:22:17.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:16.675183+0000", 2024-09-17T21:22:17.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0356492959999999 2024-09-17T21:22:17.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for rw locks", 2024-09-17T21:22:17.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:16.675242+0000", 2024-09-17T21:22:17.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8712000000000003e-05 2024-09-17T21:22:17.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:17.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:16.678347+0000", 2024-09-17T21:22:17.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031052940000000002 2024-09-17T21:22:17.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:17.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:16.678578+0000", 2024-09-17T21:22:17.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023119300000000001 2024-09-17T21:22:17.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:17.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:17.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:17.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:16.680549+0000", 2024-09-17T21:22:17.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019702880000000002 2024-09-17T21:22:17.641 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.641 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.641 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.641 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:17.641 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:17.641 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:17.643 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:22:17.643 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:17.637+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:17.660 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:22:17.666 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:22:17.667 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:22:17.685 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:22:17.685 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:22:17.685 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T21:22:17.706 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:22:17.706 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:22:17.725 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:22:17.725 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:22:17.743 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:17.738+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:17.746 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:22:17.746 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:22:17.750 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:22:17.750 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:22:17.771 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:17.771 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:17.785 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:22:17.844 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:17.838+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:17.907 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:17.907 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:17.921 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:22:17.967 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:17.965+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:18.044 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:18.044 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:18.046 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:18.041+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:18.057 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:22:18.148 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:18.146+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:18.221 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:18.221 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:18.236 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:22:18.283 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:18.281+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:18.323 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-17T21:22:18.349 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:18.345+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:18.362 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:18.362 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:18.376 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:22:18.463 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:18.461+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:18.543 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:18.543 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:18.551 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:18.549+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:18.557 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:22:18.653 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:18.650+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:18.721 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:18.722 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:18.737 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:22:18.783 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:18.781+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:18.855 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:18.853+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:18.861 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:18.862 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:18.875 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:22:18.963 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:18.961+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:19.039 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:19.039 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:19.054 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:22:19.099 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:19.097+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:19.157 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:19.153+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:19.187 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:19.187 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:19.201 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:22:19.258 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:19.254+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:19.320 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:19.320 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:19.339 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:22:19.358 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:19.353+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:19.460 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:19.457+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:19.506 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:19.506 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:19.520 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:22:19.560 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:19.557+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:19.643 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:19.643 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:19.657 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:22:19.661 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:19.658+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:19.670 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 0] out_osds: [2, 5] dead_osds: [2] live_osds: [0, 1, 3, 4, 5] 2024-09-17T21:22:19.670 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:22:19.671 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:22:19.751 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:22:19.778 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:19.778 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:19.778 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:19.774+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:19.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:22:19.862 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:19.857+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:19.909 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:22:19.910 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:22:19.926 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:22:19.927 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:22:19.927 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T21:22:19.948 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:22:19.948 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:22:19.955 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:19.955 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:22:19.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:22:19.955 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:22:19.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.955 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:22:19.955 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:22:19.955 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 220.142729737, 2024-09-17T21:22:19.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:22:19.956 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:19.956 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:19.956 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:19.956 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:19.956 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:22:19.956 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:22:19.956 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:22:19.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.956 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:19.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:19.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:22:19.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:19.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:19.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:22:19.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:19.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:19.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:22:19.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:22:19.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:19.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:22:19.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:22:19.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:19.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:22:19.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:22:19.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:19.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:22:19.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:22:19.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:19.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:22:19.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:22:19.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:19.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:22:19.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:22:19.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:19.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:22:19.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:22:19.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:19.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:22:19.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:22:19.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:19.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:22:19.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:22:19.960 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:19.960 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:19.960 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:19.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.961 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:22:19.961 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:22:19.961 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 206.587627951, 2024-09-17T21:22:19.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:22:19.961 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:19.961 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:19.961 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:19.961 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:19.961 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:22:19.961 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:19.961 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:22:19.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.962 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:19.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:19.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:22:19.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:19.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:19.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:22:19.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:19.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:19.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:22:19.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:22:19.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:19.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:22:19.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:22:19.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:19.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:22:19.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:22:19.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:19.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:22:19.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:22:19.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:19.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:22:19.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:22:19.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:19.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:22:19.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:22:19.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:19.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:22:19.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:22:19.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:19.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:22:19.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:22:19.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:22:19.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:22:19.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:22:19.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:19.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:22:19.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:22:19.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:19.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:22:19.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:22:19.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:19.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:22:19.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:22:19.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:19.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:22:19.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:22:19.967 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:19.967 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:19.967 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:19.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.967 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:793 3.9 3:95f63599:::rbd_data.1152eb799094.0000000000000007:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:22:19.968 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:22:19.968 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.520571796, 2024-09-17T21:22:19.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88245766400000003, 2024-09-17T21:22:19.968 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:19.968 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:19.968 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:19.968 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:19.968 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:22:19.968 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:22:19.968 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 793 2024-09-17T21:22:19.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.968 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:19.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:19.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:22:19.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:19.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:19.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:22:19.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:19.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:19.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431941+0000", 2024-09-17T21:22:19.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.864e-06 2024-09-17T21:22:19.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:19.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431953+0000", 2024-09-17T21:22:19.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1486e-05 2024-09-17T21:22:19.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:19.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431984+0000", 2024-09-17T21:22:19.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1075999999999999e-05 2024-09-17T21:22:19.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:19.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431999+0000", 2024-09-17T21:22:19.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.503e-05 2024-09-17T21:22:19.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:19.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432095+0000", 2024-09-17T21:22:19.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5698000000000004e-05 2024-09-17T21:22:19.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:19.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432109+0000", 2024-09-17T21:22:19.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4188e-05 2024-09-17T21:22:19.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:19.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.309642+0000", 2024-09-17T21:22:19.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87753353000000001 2024-09-17T21:22:19.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:19.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310104+0000", 2024-09-17T21:22:19.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046103600000000001 2024-09-17T21:22:19.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:19.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310630+0000", 2024-09-17T21:22:19.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052682800000000002 2024-09-17T21:22:19.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:19.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313100+0000", 2024-09-17T21:22:19.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024693369999999998 2024-09-17T21:22:19.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:19.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313935+0000", 2024-09-17T21:22:19.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083540100000000005 2024-09-17T21:22:19.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:19.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314086+0000", 2024-09-17T21:22:19.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015053399999999999 2024-09-17T21:22:19.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:19.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314396+0000", 2024-09-17T21:22:19.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031065599999999997 2024-09-17T21:22:19.974 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:19.974 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:19.974 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:19.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.974 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:800 3.19 3:9dc17f12:::rbd_data.1152eb799094.000000000000000e:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:22:19.974 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:22:19.974 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.520324551, 2024-09-17T21:22:19.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88154663799999999, 2024-09-17T21:22:19.975 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:19.975 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:19.975 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:19.975 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:19.975 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:22:19.975 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:22:19.975 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 800 2024-09-17T21:22:19.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.975 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:19.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:19.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:22:19.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:19.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:19.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:22:19.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:19.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:19.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432187+0000", 2024-09-17T21:22:19.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4100000000000001e-06 2024-09-17T21:22:19.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:19.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432194+0000", 2024-09-17T21:22:19.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0389999999999999e-06 2024-09-17T21:22:19.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:19.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432216+0000", 2024-09-17T21:22:19.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1614e-05 2024-09-17T21:22:19.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:19.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432232+0000", 2024-09-17T21:22:19.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5739e-05 2024-09-17T21:22:19.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:19.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432405+0000", 2024-09-17T21:22:19.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017355300000000001 2024-09-17T21:22:19.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:19.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432432+0000", 2024-09-17T21:22:19.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6979e-05 2024-09-17T21:22:19.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:19.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.307767+0000", 2024-09-17T21:22:19.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87533472899999998 2024-09-17T21:22:19.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:19.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308192+0000", 2024-09-17T21:22:19.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042485999999999999 2024-09-17T21:22:19.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:19.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308727+0000", 2024-09-17T21:22:19.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00053550400000000001 2024-09-17T21:22:19.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:19.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310347+0000", 2024-09-17T21:22:19.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016198670000000001 2024-09-17T21:22:19.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:19.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313424+0000", 2024-09-17T21:22:19.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030764640000000001 2024-09-17T21:22:19.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:19.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313510+0000", 2024-09-17T21:22:19.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5938000000000003e-05 2024-09-17T21:22:19.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:19.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313733+0000", 2024-09-17T21:22:19.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022294200000000001 2024-09-17T21:22:19.981 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:19.981 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:19.981 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:19.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.981 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:805 3.19 3:9cab2ed6:::rbd_data.1152eb799094.0000000000000013:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:22:19.981 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:22:19.981 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.52012053199999, 2024-09-17T21:22:19.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88274355699999996, 2024-09-17T21:22:19.981 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:19.981 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:19.981 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:19.982 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:19.982 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:22:19.982 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:22:19.982 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 805 2024-09-17T21:22:19.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.982 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:19.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:19.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:22:19.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:19.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:19.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:22:19.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:19.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:19.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432391+0000", 2024-09-17T21:22:19.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3e-06 2024-09-17T21:22:19.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:19.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432398+0000", 2024-09-17T21:22:19.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4489999999999996e-06 2024-09-17T21:22:19.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:19.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432415+0000", 2024-09-17T21:22:19.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7750000000000001e-05 2024-09-17T21:22:19.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:19.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432430+0000", 2024-09-17T21:22:19.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.449e-05 2024-09-17T21:22:19.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:19.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432635+0000", 2024-09-17T21:22:19.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205031 2024-09-17T21:22:19.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:19.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432657+0000", 2024-09-17T21:22:19.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2064000000000001e-05 2024-09-17T21:22:19.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:19.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310444+0000", 2024-09-17T21:22:19.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87778699199999999 2024-09-17T21:22:19.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:19.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310849+0000", 2024-09-17T21:22:19.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00040497800000000001 2024-09-17T21:22:19.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:19.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.311396+0000", 2024-09-17T21:22:19.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00054729099999999999 2024-09-17T21:22:19.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:19.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313847+0000", 2024-09-17T21:22:19.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024504940000000001 2024-09-17T21:22:19.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:19.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314879+0000", 2024-09-17T21:22:19.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001031817 2024-09-17T21:22:19.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:19.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314964+0000", 2024-09-17T21:22:19.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5685000000000004e-05 2024-09-17T21:22:19.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:19.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.315133+0000", 2024-09-17T21:22:19.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000169216 2024-09-17T21:22:19.987 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:19.988 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:19.988 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:19.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.988 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12089 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 60416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:19.988 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:22:19.988 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 118.1828955, 2024-09-17T21:22:19.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088293233999999998, 2024-09-17T21:22:19.988 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:19.988 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:19.988 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:19.988 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:19.988 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:19.988 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:19.989 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12089 2024-09-17T21:22:19.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.989 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:19.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:19.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:22:19.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:19.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:19.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:22:19.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:19.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:19.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769616+0000", 2024-09-17T21:22:19.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:22:19.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:19.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769620+0000", 2024-09-17T21:22:19.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5140000000000002e-06 2024-09-17T21:22:19.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:19.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769633+0000", 2024-09-17T21:22:19.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2849e-05 2024-09-17T21:22:19.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:19.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769644+0000", 2024-09-17T21:22:19.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1471000000000001e-05 2024-09-17T21:22:19.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:19.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857564+0000", 2024-09-17T21:22:19.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087919445999999998 2024-09-17T21:22:19.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:19.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857716+0000", 2024-09-17T21:22:19.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000152065 2024-09-17T21:22:19.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:19.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857908+0000", 2024-09-17T21:22:19.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019252599999999999 2024-09-17T21:22:19.992 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:19.992 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:19.992 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:19.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.992 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12090 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 64512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:19.993 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:22:19.993 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 118.182849115, 2024-09-17T21:22:19.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088582672000000001, 2024-09-17T21:22:19.993 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:19.993 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:19.993 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:19.993 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:19.993 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:19.993 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:19.993 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12090 2024-09-17T21:22:19.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.993 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:19.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:19.994 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:22:19.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:19.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.994 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:19.994 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:22:19.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:19.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.994 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:19.994 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769662+0000", 2024-09-17T21:22:19.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:22:19.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:19.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769683+0000", 2024-09-17T21:22:19.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0979e-05 2024-09-17T21:22:19.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:19.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769699+0000", 2024-09-17T21:22:19.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5446000000000001e-05 2024-09-17T21:22:19.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:19.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769708+0000", 2024-09-17T21:22:19.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1679999999999993e-06 2024-09-17T21:22:19.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:19.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857976+0000", 2024-09-17T21:22:19.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088267795999999996 2024-09-17T21:22:19.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:19.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858095+0000", 2024-09-17T21:22:19.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119147 2024-09-17T21:22:19.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:19.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858244+0000", 2024-09-17T21:22:19.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014917000000000001 2024-09-17T21:22:19.997 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:19.997 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:19.997 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:19.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.997 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12091 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 68608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:19.997 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:22:19.997 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 118.182777731, 2024-09-17T21:22:19.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088832752000000001, 2024-09-17T21:22:19.997 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:19.997 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:19.997 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:19.997 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:19.998 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:19.998 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:19.998 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12091 2024-09-17T21:22:19.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.998 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:19.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:19.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:22:19.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:19.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:19.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:22:19.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:19.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:19.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769734+0000", 2024-09-17T21:22:19.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:22:19.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:19.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769739+0000", 2024-09-17T21:22:19.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8080000000000004e-06 2024-09-17T21:22:19.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:19.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:19.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:19.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769764+0000", 2024-09-17T21:22:20.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4839999999999999e-05 2024-09-17T21:22:20.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769777+0000", 2024-09-17T21:22:20.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3037000000000001e-05 2024-09-17T21:22:20.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:20.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858295+0000", 2024-09-17T21:22:20.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088518475999999999 2024-09-17T21:22:20.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:20.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858419+0000", 2024-09-17T21:22:20.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012378700000000001 2024-09-17T21:22:20.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:20.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858565+0000", 2024-09-17T21:22:20.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014630099999999999 2024-09-17T21:22:20.001 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.001 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:20.001 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.001 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12092 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:20.002 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:22:20.002 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 118.182695407, 2024-09-17T21:22:20.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089191555000000006, 2024-09-17T21:22:20.002 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:20.002 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:20.002 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:20.002 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:20.002 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:20.002 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:20.002 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12092 2024-09-17T21:22:20.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.002 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:20.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:20.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:22:20.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:20.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:22:20.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:20.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769816+0000", 2024-09-17T21:22:20.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7199999999999997e-07 2024-09-17T21:22:20.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:20.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769822+0000", 2024-09-17T21:22:20.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:22:20.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:20.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769838+0000", 2024-09-17T21:22:20.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6674e-05 2024-09-17T21:22:20.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769851+0000", 2024-09-17T21:22:20.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2527999999999999e-05 2024-09-17T21:22:20.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:20.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858660+0000", 2024-09-17T21:22:20.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088808556999999996 2024-09-17T21:22:20.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:20.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858817+0000", 2024-09-17T21:22:20.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015730400000000001 2024-09-17T21:22:20.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:20.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859007+0000", 2024-09-17T21:22:20.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018974899999999999 2024-09-17T21:22:20.006 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.006 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:20.006 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.006 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12093 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:20.006 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:22:20.006 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 118.18258730300001, 2024-09-17T21:22:20.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089429710999999995, 2024-09-17T21:22:20.006 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:20.006 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:20.007 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:20.007 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:20.007 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:20.007 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:20.007 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12093 2024-09-17T21:22:20.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.007 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:20.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:20.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:22:20.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:20.008 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:22:20.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.008 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:20.008 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769924+0000", 2024-09-17T21:22:20.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.342e-06 2024-09-17T21:22:20.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.008 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:20.008 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769929+0000", 2024-09-17T21:22:20.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0350000000000001e-06 2024-09-17T21:22:20.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:20.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769946+0000", 2024-09-17T21:22:20.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6793999999999999e-05 2024-09-17T21:22:20.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769959+0000", 2024-09-17T21:22:20.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3062e-05 2024-09-17T21:22:20.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:20.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859075+0000", 2024-09-17T21:22:20.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089115696999999994 2024-09-17T21:22:20.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:20.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859202+0000", 2024-09-17T21:22:20.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012657 2024-09-17T21:22:20.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:20.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859353+0000", 2024-09-17T21:22:20.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151211 2024-09-17T21:22:20.010 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.010 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:20.010 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.011 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12094 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:20.011 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:22:20.011 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 118.182533127, 2024-09-17T21:22:20.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089695001999999996, 2024-09-17T21:22:20.011 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:20.011 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:20.011 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:20.011 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:20.011 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:20.011 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:20.012 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12094 2024-09-17T21:22:20.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.012 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:20.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.012 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:20.012 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:22:20.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.012 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:20.012 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:22:20.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:20.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769978+0000", 2024-09-17T21:22:20.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.142e-06 2024-09-17T21:22:20.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:20.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769983+0000", 2024-09-17T21:22:20.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.386e-06 2024-09-17T21:22:20.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:20.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769997+0000", 2024-09-17T21:22:20.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.416e-05 2024-09-17T21:22:20.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770012+0000", 2024-09-17T21:22:20.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5384999999999999e-05 2024-09-17T21:22:20.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:20.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859405+0000", 2024-09-17T21:22:20.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089392376999999995 2024-09-17T21:22:20.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:20.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859525+0000", 2024-09-17T21:22:20.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120555 2024-09-17T21:22:20.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:20.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859672+0000", 2024-09-17T21:22:20.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000146997 2024-09-17T21:22:20.015 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.015 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:20.015 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.015 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12095 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:20.016 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:22:20.016 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 118.18248063999999, 2024-09-17T21:22:20.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090073476, 2024-09-17T21:22:20.016 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:20.016 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:20.016 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:20.016 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:20.016 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:20.016 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:20.016 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12095 2024-09-17T21:22:20.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.016 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:20.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.016 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:20.017 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:22:20.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.017 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:20.017 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:22:20.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.017 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:20.017 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770031+0000", 2024-09-17T21:22:20.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1699999999999997e-07 2024-09-17T21:22:20.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:20.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770036+0000", 2024-09-17T21:22:20.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.023e-06 2024-09-17T21:22:20.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:20.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770051+0000", 2024-09-17T21:22:20.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4953999999999999e-05 2024-09-17T21:22:20.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770063+0000", 2024-09-17T21:22:20.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:22:20.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:20.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859756+0000", 2024-09-17T21:22:20.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089692980000000005 2024-09-17T21:22:20.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:20.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859891+0000", 2024-09-17T21:22:20.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013459499999999999 2024-09-17T21:22:20.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:20.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860103+0000", 2024-09-17T21:22:20.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021274400000000001 2024-09-17T21:22:20.020 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.020 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:20.020 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.020 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12096 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:20.020 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:22:20.020 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 118.182430955, 2024-09-17T21:22:20.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090358274000000002, 2024-09-17T21:22:20.020 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:20.020 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:20.020 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:20.021 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:20.021 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:20.021 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:20.021 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12096 2024-09-17T21:22:20.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.021 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:20.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.021 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:20.021 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:22:20.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.021 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:20.021 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:22:20.022 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.022 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:20.022 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770080+0000", 2024-09-17T21:22:20.022 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9400000000000004e-07 2024-09-17T21:22:20.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.022 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:20.022 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770085+0000", 2024-09-17T21:22:20.022 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2169999999999996e-06 2024-09-17T21:22:20.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.023 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:20.023 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770100+0000", 2024-09-17T21:22:20.023 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5077e-05 2024-09-17T21:22:20.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.023 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.023 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770112+0000", 2024-09-17T21:22:20.023 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1951e-05 2024-09-17T21:22:20.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.023 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:20.023 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860160+0000", 2024-09-17T21:22:20.023 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090048413999999993 2024-09-17T21:22:20.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:20.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860285+0000", 2024-09-17T21:22:20.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012466699999999999 2024-09-17T21:22:20.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:20.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860438+0000", 2024-09-17T21:22:20.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015305399999999999 2024-09-17T21:22:20.024 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.024 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:20.024 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.025 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12097 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:20.025 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:22:20.025 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 118.182383208, 2024-09-17T21:22:20.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090702375000000002, 2024-09-17T21:22:20.025 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:20.025 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:20.025 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:20.025 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:20.025 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:20.025 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:20.025 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12097 2024-09-17T21:22:20.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.025 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:20.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.026 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:20.026 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:22:20.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.026 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:20.026 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:22:20.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.026 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:20.027 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770128+0000", 2024-09-17T21:22:20.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.04e-06 2024-09-17T21:22:20.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.027 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:20.027 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770133+0000", 2024-09-17T21:22:20.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8999999999999997e-06 2024-09-17T21:22:20.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.027 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:20.027 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770152+0000", 2024-09-17T21:22:20.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8890999999999999e-05 2024-09-17T21:22:20.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770164+0000", 2024-09-17T21:22:20.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1769e-05 2024-09-17T21:22:20.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:20.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860511+0000", 2024-09-17T21:22:20.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090347527999999996 2024-09-17T21:22:20.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:20.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860652+0000", 2024-09-17T21:22:20.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140894 2024-09-17T21:22:20.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:20.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860830+0000", 2024-09-17T21:22:20.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017735299999999999 2024-09-17T21:22:20.029 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.029 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:20.029 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.029 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12098 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:20.029 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:22:20.029 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 118.18232974, 2024-09-17T21:22:20.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090997262999999995, 2024-09-17T21:22:20.030 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:20.030 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:20.030 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:20.030 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:20.030 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:20.030 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:20.030 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12098 2024-09-17T21:22:20.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.030 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:20.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:20.030 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:22:20.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:20.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:22:20.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:20.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775465+0000", 2024-09-17T21:22:20.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0052843350000000002 2024-09-17T21:22:20.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:20.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775476+0000", 2024-09-17T21:22:20.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0797e-05 2024-09-17T21:22:20.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:20.032 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775506+0000", 2024-09-17T21:22:20.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9709999999999998e-05 2024-09-17T21:22:20.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.032 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775530+0000", 2024-09-17T21:22:20.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4528999999999998e-05 2024-09-17T21:22:20.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:20.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860894+0000", 2024-09-17T21:22:20.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.085364195000000004 2024-09-17T21:22:20.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:20.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861019+0000", 2024-09-17T21:22:20.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012467399999999999 2024-09-17T21:22:20.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:20.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861178+0000", 2024-09-17T21:22:20.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159023 2024-09-17T21:22:20.033 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.033 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:20.034 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.034 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12105 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:20.034 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:22:20.034 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 118.176599737, 2024-09-17T21:22:20.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088330400000000003, 2024-09-17T21:22:20.034 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:20.034 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:20.034 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:20.034 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:20.034 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:20.034 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:20.035 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12105 2024-09-17T21:22:20.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.035 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:20.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:20.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:22:20.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:20.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:22:20.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:20.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:22:20.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9999999999999997e-07 2024-09-17T21:22:20.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:20.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775914+0000", 2024-09-17T21:22:20.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4710000000000001e-06 2024-09-17T21:22:20.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:20.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775921+0000", 2024-09-17T21:22:20.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:22:20.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775928+0000", 2024-09-17T21:22:20.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:22:20.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:20.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863923+0000", 2024-09-17T21:22:20.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087995126000000007 2024-09-17T21:22:20.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:20.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864059+0000", 2024-09-17T21:22:20.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135797 2024-09-17T21:22:20.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.038 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:20.038 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864241+0000", 2024-09-17T21:22:20.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018252599999999999 2024-09-17T21:22:20.038 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.038 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:20.038 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.038 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12106 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:20.038 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:22:20.038 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 118.17651055899999, 2024-09-17T21:22:20.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088586394999999998, 2024-09-17T21:22:20.039 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:20.039 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:20.039 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:20.039 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:20.039 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:20.039 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:20.039 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12106 2024-09-17T21:22:20.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.039 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:20.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.039 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:20.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:22:20.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:20.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:22:20.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:20.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776001+0000", 2024-09-17T21:22:20.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4300000000000001e-07 2024-09-17T21:22:20.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:20.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776004+0000", 2024-09-17T21:22:20.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8049999999999998e-06 2024-09-17T21:22:20.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:20.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776018+0000", 2024-09-17T21:22:20.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4613999999999999e-05 2024-09-17T21:22:20.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776025+0000", 2024-09-17T21:22:20.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:22:20.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:20.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864279+0000", 2024-09-17T21:22:20.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088253570000000003 2024-09-17T21:22:20.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:20.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864433+0000", 2024-09-17T21:22:20.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015469400000000001 2024-09-17T21:22:20.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:20.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864586+0000", 2024-09-17T21:22:20.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000153003 2024-09-17T21:22:20.042 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.042 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:20.043 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.043 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25033 3.6 3.b74753e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e168)", 2024-09-17T21:22:20.043 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:22:20.043 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 54.443067294999999, 2024-09-17T21:22:20.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93781952099999999, 2024-09-17T21:22:20.043 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:20.043 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:20.043 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:20.043 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:20.043 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:22:20.043 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:20.044 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25033 2024-09-17T21:22:20.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.044 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:20.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:20.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:22:20.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:20.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:22:20.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:20.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509444+0000", 2024-09-17T21:22:20.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0020000000000001e-06 2024-09-17T21:22:20.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:20.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509450+0000", 2024-09-17T21:22:20.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4249999999999997e-06 2024-09-17T21:22:20.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:20.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509460+0000", 2024-09-17T21:22:20.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0183999999999999e-05 2024-09-17T21:22:20.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509466+0000", 2024-09-17T21:22:20.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0039999999999996e-06 2024-09-17T21:22:20.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:20.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509540+0000", 2024-09-17T21:22:20.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3847e-05 2024-09-17T21:22:20.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:20.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509551+0000", 2024-09-17T21:22:20.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1731e-05 2024-09-17T21:22:20.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:20.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.447243+0000", 2024-09-17T21:22:20.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93769166000000004 2024-09-17T21:22:20.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:20.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.447263+0000", 2024-09-17T21:22:20.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9667999999999999e-05 2024-09-17T21:22:20.047 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.047 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:20.047 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.048 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25033 3.6 3:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e168)", 2024-09-17T21:22:20.048 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:22:20.048 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 54.441541698000002, 2024-09-17T21:22:20.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.937309383, 2024-09-17T21:22:20.048 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:20.048 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:20.048 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:20.048 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:20.048 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:22:20.048 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:20.048 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25033 2024-09-17T21:22:20.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.048 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:20.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.049 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:20.049 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:22:20.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.049 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:20.049 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:22:20.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.049 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:20.049 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510970+0000", 2024-09-17T21:22:20.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.564e-06 2024-09-17T21:22:20.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.050 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:20.050 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510978+0000", 2024-09-17T21:22:20.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.362e-06 2024-09-17T21:22:20.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.050 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:20.050 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510998+0000", 2024-09-17T21:22:20.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0336999999999999e-05 2024-09-17T21:22:20.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.050 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.050 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511010+0000", 2024-09-17T21:22:20.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1786e-05 2024-09-17T21:22:20.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:20.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511080+0000", 2024-09-17T21:22:20.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0686999999999996e-05 2024-09-17T21:22:20.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:20.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511087+0000", 2024-09-17T21:22:20.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4899999999999997e-06 2024-09-17T21:22:20.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:20.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.448050+0000", 2024-09-17T21:22:20.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93696351700000002 2024-09-17T21:22:20.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:20.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.448278+0000", 2024-09-17T21:22:20.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022764000000000001 2024-09-17T21:22:20.052 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.052 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:20.052 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.052 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:30858 3.1a 3:5b9355dc:::rbd_header.11527824718:head [watch reconnect cookie 139860845522384 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e218)", 2024-09-17T21:22:20.052 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:22:15.639403+0000", 2024-09-17T21:22:20.052 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.3131070740000004, 2024-09-17T21:22:20.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.041145395, 2024-09-17T21:22:20.052 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:20.053 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:20.053 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:20.053 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:20.053 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:22:20.053 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:20.053 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 30858 2024-09-17T21:22:20.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.053 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:20.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:20.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639403+0000", 2024-09-17T21:22:20.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:20.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639403+0000", 2024-09-17T21:22:20.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:20.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:20.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639406+0000", 2024-09-17T21:22:20.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2979999999999999e-06 2024-09-17T21:22:20.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:20.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639412+0000", 2024-09-17T21:22:20.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.494e-06 2024-09-17T21:22:20.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:20.055 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639423+0000", 2024-09-17T21:22:20.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1277000000000001e-05 2024-09-17T21:22:20.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.055 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.055 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639433+0000", 2024-09-17T21:22:20.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7699999999999996e-06 2024-09-17T21:22:20.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.055 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:20.055 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639520+0000", 2024-09-17T21:22:20.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7121999999999997e-05 2024-09-17T21:22:20.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:20.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639534+0000", 2024-09-17T21:22:20.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3651e-05 2024-09-17T21:22:20.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:20.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:16.675183+0000", 2024-09-17T21:22:20.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0356492959999999 2024-09-17T21:22:20.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for rw locks", 2024-09-17T21:22:20.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:16.675242+0000", 2024-09-17T21:22:20.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8712000000000003e-05 2024-09-17T21:22:20.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.057 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:20.057 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:16.678347+0000", 2024-09-17T21:22:20.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031052940000000002 2024-09-17T21:22:20.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.057 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:20.057 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:16.678578+0000", 2024-09-17T21:22:20.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023119300000000001 2024-09-17T21:22:20.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:20.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:20.057 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:20.057 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:16.680549+0000", 2024-09-17T21:22:20.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019702880000000002 2024-09-17T21:22:20.058 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.058 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:20.058 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.058 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:20.058 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:20.058 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:20.059 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:22:20.060 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:22:20.060 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:22:20.060 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:22:20.061 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:22:20.061 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:22:20.062 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:22:20.063 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:22:20.063 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T21:22:20.064 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:20.057+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:20.065 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-17T21:22:19.601995+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":227034181,"num_objects":445,"num_object_clones":32,"num_object_copies":890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":445,"num_whiteouts":325,"num_read":352840,"num_read_kb":1144989,"num_write":19804,"num_write_kb":386951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10056,"num_bytes_recovered":615643092,"num_keys_recovered":1529,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18276,"ondisk_log_size":18276,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":361172,"kb_used_data":107308,"kb_used_omap":6,"kb_used_meta":253753,"kb_avail":377126188,"statfs":{"total":386547056640,"available":386177216512,"internally_reserved":0,"allocated":109883392,"data_stored":162517380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6382,"internal_metadata":259843858},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-96735483,"num_objects":-47,"num_object_clones":-11,"num_object_copies":-94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-47,"num_whiteouts":-11,"num_read":437,"num_read_kb":375,"num_write":92,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":567,"num_bytes_recovered":4828805,"num_keys_recovered":25,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.968417"},"pg_stats":[{"pgid":"3.1f","version":"214'754","reported_seq":4041,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.322590+0000","last_change":"2024-09-17T21:22:17.327739+0000","last_active":"2024-09-17T21:22:19.322590+0000","last_peered":"2024-09-17T21:22:19.322590+0000","last_clean":"2024-09-17T21:22:19.322590+0000","last_became_active":"2024-09-17T21:21:40.315476+0000","last_became_peered":"2024-09-17T21:21:40.315476+0000","last_unstale":"2024-09-17T21:22:19.322590+0000","last_undegraded":"2024-09-17T21:22:19.322590+0000","last_fullsized":"2024-09-17T21:22:19.322590+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:15:27.359719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097189970000000001,"stat_sum":{"num_bytes":2888768,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":10,"num_read":5912,"num_read_kb":16775,"num_write":1023,"num_write_kb":3495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":5143732,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"3.1e","version":"222'817","reported_seq":9369,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.414531+0000","last_change":"2024-09-17T21:22:17.327058+0000","last_active":"2024-09-17T21:22:19.414531+0000","last_peered":"2024-09-17T21:22:19.414531+0000","last_clean":"2024-09-17T21:22:19.414531+0000","last_became_active":"2024-09-17T21:21:41.323281+0000","last_became_peered":"2024-09-17T21:21:41.323281+0000","last_unstale":"2024-09-17T21:22:19.414531+0000","last_undegraded":"2024-09-17T21:22:19.414531+0000","last_fullsized":"2024-09-17T21:22:19.414531+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:10:40.033654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0058371789999999996,"stat_sum":{"num_bytes":4484138,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":9,"num_read":10754,"num_read_kb":52960,"num_write":1050,"num_write_kb":18698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":50670363,"num_keys_recovered":90,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"3.1d","version":"221'825","reported_seq":8618,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.360110+0000","last_change":"2024-09-17T21:22:17.327907+0000","last_active":"2024-09-17T21:22:19.360110+0000","last_peered":"2024-09-17T21:22:19.360110+0000","last_clean":"2024-09-17T21:22:19.360110+0000","last_became_active":"2024-09-17T21:21:45.406217+0000","last_became_peered":"2024-09-17T21:21:45.406217+0000","last_unstale":"2024-09-17T21:22:19.360110+0000","last_undegraded":"2024-09-17T21:22:19.360110+0000","last_fullsized":"2024-09-17T21:22:19.360110+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":825,"log_dups_size":0,"ondisk_log_size":825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:03:41.552726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082244970000000008,"stat_sum":{"num_bytes":5282304,"num_objects":19,"num_object_clones":2,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":13,"num_read":10230,"num_read_kb":27857,"num_write":974,"num_write_kb":10466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":12493301,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"3.1c","version":"221'371","reported_seq":3960,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.359860+0000","last_change":"2024-09-17T21:22:17.330798+0000","last_active":"2024-09-17T21:22:19.359860+0000","last_peered":"2024-09-17T21:22:19.359860+0000","last_clean":"2024-09-17T21:22:19.359860+0000","last_became_active":"2024-09-17T21:21:45.408459+0000","last_became_peered":"2024-09-17T21:21:45.408459+0000","last_unstale":"2024-09-17T21:22:19.359860+0000","last_undegraded":"2024-09-17T21:22:19.359860+0000","last_fullsized":"2024-09-17T21:22:19.359860+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:48:42.200521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045578329999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":2698,"num_read_kb":23710,"num_write":373,"num_write_kb":5937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":9397808,"num_keys_recovered":202,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"3.1b","version":"221'703","reported_seq":8210,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.320088+0000","last_change":"2024-09-17T21:22:17.319551+0000","last_active":"2024-09-17T21:22:19.320088+0000","last_peered":"2024-09-17T21:22:19.320088+0000","last_clean":"2024-09-17T21:22:19.320088+0000","last_became_active":"2024-09-17T21:21:41.329830+0000","last_became_peered":"2024-09-17T21:21:41.329830+0000","last_unstale":"2024-09-17T21:22:19.320088+0000","last_undegraded":"2024-09-17T21:22:19.320088+0000","last_fullsized":"2024-09-17T21:22:19.320088+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:00:39.258992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00528072,"stat_sum":{"num_bytes":2921472,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":12,"num_read":11634,"num_read_kb":21103,"num_write":552,"num_write_kb":4725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":7872032,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"3.1a","version":"221'733","reported_seq":13570,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.404096+0000","last_change":"2024-09-17T21:22:18.335594+0000","last_active":"2024-09-17T21:22:19.404096+0000","last_peered":"2024-09-17T21:22:19.404096+0000","last_clean":"2024-09-17T21:22:19.404096+0000","last_became_active":"2024-09-17T21:22:18.327089+0000","last_became_peered":"2024-09-17T21:22:18.327089+0000","last_unstale":"2024-09-17T21:22:19.404096+0000","last_undegraded":"2024-09-17T21:22:19.404096+0000","last_fullsized":"2024-09-17T21:22:19.404096+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:18:23.406969+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0063102549999999999,"stat_sum":{"num_bytes":5898771,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":7,"num_read":16862,"num_read_kb":53508,"num_write":667,"num_write_kb":14565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":647699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"3.19","version":"216'473","reported_seq":12158,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.318298+0000","last_change":"2024-09-17T21:22:17.318808+0000","last_active":"2024-09-17T21:22:19.318298+0000","last_peered":"2024-09-17T21:22:19.318298+0000","last_clean":"2024-09-17T21:22:19.318298+0000","last_became_active":"2024-09-17T21:20:19.295432+0000","last_became_peered":"2024-09-17T21:20:19.295432+0000","last_unstale":"2024-09-17T21:22:19.318298+0000","last_undegraded":"2024-09-17T21:22:19.318298+0000","last_fullsized":"2024-09-17T21:22:19.318298+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:22:20.365650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016331799999999999,"stat_sum":{"num_bytes":800256,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":8,"num_read":12648,"num_read_kb":27025,"num_write":373,"num_write_kb":5615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":3100207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"3.18","version":"217'304","reported_seq":2975,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.318969+0000","last_change":"2024-09-17T21:22:17.318901+0000","last_active":"2024-09-17T21:22:19.318969+0000","last_peered":"2024-09-17T21:22:19.318969+0000","last_clean":"2024-09-17T21:22:19.318969+0000","last_became_active":"2024-09-17T21:21:39.315626+0000","last_became_peered":"2024-09-17T21:21:39.315626+0000","last_unstale":"2024-09-17T21:22:19.318969+0000","last_undegraded":"2024-09-17T21:22:19.318969+0000","last_fullsized":"2024-09-17T21:22:19.318969+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:02:19.851319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00158796,"stat_sum":{"num_bytes":337968,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":15,"num_read":2361,"num_read_kb":9115,"num_write":290,"num_write_kb":2893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":9226431,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"3.17","version":"214'390","reported_seq":7070,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.360471+0000","last_change":"2024-09-17T21:22:17.330854+0000","last_active":"2024-09-17T21:22:19.360471+0000","last_peered":"2024-09-17T21:22:19.360471+0000","last_clean":"2024-09-17T21:22:19.360471+0000","last_became_active":"2024-09-17T21:21:44.381885+0000","last_became_peered":"2024-09-17T21:21:44.381885+0000","last_unstale":"2024-09-17T21:22:19.360471+0000","last_undegraded":"2024-09-17T21:22:19.360471+0000","last_fullsized":"2024-09-17T21:22:19.360471+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:02:57.207166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080018420000000003,"stat_sum":{"num_bytes":6293504,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":11,"num_read":7054,"num_read_kb":21313,"num_write":474,"num_write_kb":6946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":614,"num_bytes_recovered":31066012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"3.16","version":"222'601","reported_seq":22188,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.341797+0000","last_change":"2024-09-17T21:22:17.325146+0000","last_active":"2024-09-17T21:22:19.341797+0000","last_peered":"2024-09-17T21:22:19.341797+0000","last_clean":"2024-09-17T21:22:19.341797+0000","last_became_active":"2024-09-17T21:21:44.390397+0000","last_became_peered":"2024-09-17T21:21:44.390397+0000","last_unstale":"2024-09-17T21:22:19.341797+0000","last_undegraded":"2024-09-17T21:22:19.341797+0000","last_fullsized":"2024-09-17T21:22:19.341797+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:53:28.383582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0081944659999999992,"stat_sum":{"num_bytes":2584576,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":22417,"num_read_kb":27659,"num_write":857,"num_write_kb":13022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":3432016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"3.15","version":"221'753","reported_seq":7082,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.322369+0000","last_change":"2024-09-17T21:22:17.326382+0000","last_active":"2024-09-17T21:22:19.322369+0000","last_peered":"2024-09-17T21:22:19.322369+0000","last_clean":"2024-09-17T21:22:19.322369+0000","last_became_active":"2024-09-17T21:21:40.321648+0000","last_became_peered":"2024-09-17T21:21:40.321648+0000","last_unstale":"2024-09-17T21:22:19.322369+0000","last_undegraded":"2024-09-17T21:22:19.322369+0000","last_fullsized":"2024-09-17T21:22:19.322369+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:39:42.888310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084582640000000001,"stat_sum":{"num_bytes":1048576,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":8592,"num_read_kb":29445,"num_write":953,"num_write_kb":5554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":9885558,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"3.14","version":"221'446","reported_seq":5293,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.319328+0000","last_change":"2024-09-17T21:22:18.333783+0000","last_active":"2024-09-17T21:22:19.319328+0000","last_peered":"2024-09-17T21:22:19.319328+0000","last_clean":"2024-09-17T21:22:19.319328+0000","last_became_active":"2024-09-17T21:22:18.327047+0000","last_became_peered":"2024-09-17T21:22:18.327047+0000","last_unstale":"2024-09-17T21:22:19.319328+0000","last_undegraded":"2024-09-17T21:22:19.319328+0000","last_fullsized":"2024-09-17T21:22:19.319328+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:03:40.807419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025416829999999999,"stat_sum":{"num_bytes":6291456,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":10,"num_read":5351,"num_read_kb":26972,"num_write":427,"num_write_kb":16010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":6943878,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":423,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.322885+0000","last_change":"2024-09-17T21:18:03.059715+0000","last_active":"2024-09-17T21:22:19.322885+0000","last_peered":"2024-09-17T21:22:19.322885+0000","last_clean":"2024-09-17T21:22:19.322885+0000","last_became_active":"2024-09-17T21:18:01.213047+0000","last_became_peered":"2024-09-17T21:18:01.213047+0000","last_unstale":"2024-09-17T21:22:19.322885+0000","last_undegraded":"2024-09-17T21:22:19.322885+0000","last_fullsized":"2024-09-17T21:22:19.322885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:46:35.222373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"222'583","reported_seq":13607,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.319046+0000","last_change":"2024-09-17T21:22:17.320979+0000","last_active":"2024-09-17T21:22:19.319046+0000","last_peered":"2024-09-17T21:22:19.319046+0000","last_clean":"2024-09-17T21:22:19.319046+0000","last_became_active":"2024-09-17T21:21:40.318488+0000","last_became_peered":"2024-09-17T21:21:40.318488+0000","last_unstale":"2024-09-17T21:22:19.319046+0000","last_undegraded":"2024-09-17T21:22:19.319046+0000","last_fullsized":"2024-09-17T21:22:19.319046+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:23:40.223235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059902760000000001,"stat_sum":{"num_bytes":19694118,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":12,"num_read":14880,"num_read_kb":61122,"num_write":577,"num_write_kb":23555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":49089084,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":423,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.319673+0000","last_change":"2024-09-17T21:18:03.073440+0000","last_active":"2024-09-17T21:22:19.319673+0000","last_peered":"2024-09-17T21:22:19.319673+0000","last_clean":"2024-09-17T21:22:19.319673+0000","last_became_active":"2024-09-17T21:18:01.048806+0000","last_became_peered":"2024-09-17T21:18:01.048806+0000","last_unstale":"2024-09-17T21:22:19.319673+0000","last_undegraded":"2024-09-17T21:22:19.319673+0000","last_fullsized":"2024-09-17T21:22:19.319673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:26.487163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094161400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"217'338","reported_seq":5531,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.319745+0000","last_change":"2024-09-17T21:22:17.323182+0000","last_active":"2024-09-17T21:22:19.319745+0000","last_peered":"2024-09-17T21:22:19.319745+0000","last_clean":"2024-09-17T21:22:19.319745+0000","last_became_active":"2024-09-17T21:21:33.312794+0000","last_became_peered":"2024-09-17T21:21:33.312794+0000","last_unstale":"2024-09-17T21:22:19.319745+0000","last_undegraded":"2024-09-17T21:22:19.319745+0000","last_fullsized":"2024-09-17T21:22:19.319745+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:01:40.838545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001605551,"stat_sum":{"num_bytes":383012,"num_objects":16,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":14,"num_read":4625,"num_read_kb":36256,"num_write":379,"num_write_kb":13872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":5244262,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":448,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.322698+0000","last_change":"2024-09-17T21:21:35.319221+0000","last_active":"2024-09-17T21:22:19.322698+0000","last_peered":"2024-09-17T21:22:19.322698+0000","last_clean":"2024-09-17T21:22:19.322698+0000","last_became_active":"2024-09-17T21:21:35.312507+0000","last_became_peered":"2024-09-17T21:21:35.312507+0000","last_unstale":"2024-09-17T21:22:19.322698+0000","last_undegraded":"2024-09-17T21:22:19.322698+0000","last_fullsized":"2024-09-17T21:22:19.322698+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:53.042567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099219900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"214'696","reported_seq":15823,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.318324+0000","last_change":"2024-09-17T21:22:17.323166+0000","last_active":"2024-09-17T21:22:19.318324+0000","last_peered":"2024-09-17T21:22:19.318324+0000","last_clean":"2024-09-17T21:22:19.318324+0000","last_became_active":"2024-09-17T21:21:33.312729+0000","last_became_peered":"2024-09-17T21:21:33.312729+0000","last_unstale":"2024-09-17T21:22:19.318324+0000","last_undegraded":"2024-09-17T21:22:19.318324+0000","last_fullsized":"2024-09-17T21:22:19.318324+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:44:33.343769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001742611,"stat_sum":{"num_bytes":20897792,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":13,"num_read":13338,"num_read_kb":29382,"num_write":601,"num_write_kb":10802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":21385444,"num_keys_recovered":118,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":448,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.318425+0000","last_change":"2024-09-17T21:21:35.326814+0000","last_active":"2024-09-17T21:22:19.318425+0000","last_peered":"2024-09-17T21:22:19.318425+0000","last_clean":"2024-09-17T21:22:19.318425+0000","last_became_active":"2024-09-17T21:21:35.325777+0000","last_became_peered":"2024-09-17T21:21:35.325777+0000","last_unstale":"2024-09-17T21:22:19.318425+0000","last_undegraded":"2024-09-17T21:22:19.318425+0000","last_fullsized":"2024-09-17T21:22:19.318425+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:47:00.558900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006222,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"219'1052","reported_seq":33502,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.322010+0000","last_change":"2024-09-17T21:22:17.325890+0000","last_active":"2024-09-17T21:22:19.322010+0000","last_peered":"2024-09-17T21:22:19.322010+0000","last_clean":"2024-09-17T21:22:19.322010+0000","last_became_active":"2024-09-17T21:21:45.406273+0000","last_became_peered":"2024-09-17T21:21:45.406273+0000","last_unstale":"2024-09-17T21:22:19.322010+0000","last_undegraded":"2024-09-17T21:22:19.322010+0000","last_fullsized":"2024-09-17T21:22:19.322010+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":1052,"log_dups_size":0,"ondisk_log_size":1052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:39:48.418663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0087063660000000001,"stat_sum":{"num_bytes":15728656,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":11,"num_read":37174,"num_read_kb":61902,"num_write":869,"num_write_kb":21852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":631,"num_bytes_recovered":58900004,"num_keys_recovered":204,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"3.3","version":"222'514","reported_seq":3769,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.364848+0000","last_change":"2024-09-17T21:22:17.325586+0000","last_active":"2024-09-17T21:22:19.364848+0000","last_peered":"2024-09-17T21:22:19.364848+0000","last_clean":"2024-09-17T21:22:19.364848+0000","last_became_active":"2024-09-17T21:21:39.315142+0000","last_became_peered":"2024-09-17T21:21:39.315142+0000","last_unstale":"2024-09-17T21:22:19.364848+0000","last_undegraded":"2024-09-17T21:22:19.364848+0000","last_fullsized":"2024-09-17T21:22:19.364848+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:56:01.148674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054623010000000001,"stat_sum":{"num_bytes":3272704,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":11,"num_read":3023,"num_read_kb":14393,"num_write":480,"num_write_kb":4803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":7199060,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"2.2","version":"21'2","reported_seq":375,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.319322+0000","last_change":"2024-09-17T21:21:41.334062+0000","last_active":"2024-09-17T21:22:19.319322+0000","last_peered":"2024-09-17T21:22:19.319322+0000","last_clean":"2024-09-17T21:22:19.319322+0000","last_became_active":"2024-09-17T21:21:41.326490+0000","last_became_peered":"2024-09-17T21:21:41.326490+0000","last_unstale":"2024-09-17T21:22:19.319322+0000","last_undegraded":"2024-09-17T21:22:19.319322+0000","last_fullsized":"2024-09-17T21:22:19.319322+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:52:09.055100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"214'241","reported_seq":9167,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.319075+0000","last_change":"2024-09-17T21:22:17.319948+0000","last_active":"2024-09-17T21:22:19.319075+0000","last_peered":"2024-09-17T21:22:19.319075+0000","last_clean":"2024-09-17T21:22:19.319075+0000","last_became_active":"2024-09-17T21:21:32.308212+0000","last_became_peered":"2024-09-17T21:21:32.308212+0000","last_unstale":"2024-09-17T21:22:19.319075+0000","last_undegraded":"2024-09-17T21:22:19.319075+0000","last_fullsized":"2024-09-17T21:22:19.319075+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:17:48.312199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024890139999999999,"stat_sum":{"num_bytes":8762384,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":6,"num_read":5337,"num_read_kb":22604,"num_write":212,"num_write_kb":6424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":15742651,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"2.1","version":"0'0","reported_seq":425,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.318574+0000","last_change":"2024-09-17T21:21:35.324570+0000","last_active":"2024-09-17T21:22:19.318574+0000","last_peered":"2024-09-17T21:22:19.318574+0000","last_clean":"2024-09-17T21:22:19.318574+0000","last_became_active":"2024-09-17T21:21:35.323008+0000","last_became_peered":"2024-09-17T21:21:35.323008+0000","last_unstale":"2024-09-17T21:22:19.318574+0000","last_undegraded":"2024-09-17T21:22:19.318574+0000","last_fullsized":"2024-09-17T21:22:19.318574+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:05:09.466151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"214'1022","reported_seq":8350,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.322942+0000","last_change":"2024-09-17T21:22:17.317911+0000","last_active":"2024-09-17T21:22:19.322942+0000","last_peered":"2024-09-17T21:22:19.322942+0000","last_clean":"2024-09-17T21:22:19.322942+0000","last_became_active":"2024-09-17T21:21:44.389522+0000","last_became_peered":"2024-09-17T21:21:44.389522+0000","last_unstale":"2024-09-17T21:22:19.322942+0000","last_undegraded":"2024-09-17T21:22:19.322942+0000","last_fullsized":"2024-09-17T21:22:19.322942+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":1022,"log_dups_size":0,"ondisk_log_size":1022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:38:21.419287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028102719999999999,"stat_sum":{"num_bytes":7517232,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":5,"num_read":9635,"num_read_kb":63078,"num_write":1559,"num_write_kb":18127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":690,"num_bytes_recovered":48903910,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":424,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.322291+0000","last_change":"2024-09-17T21:18:03.059156+0000","last_active":"2024-09-17T21:22:19.322291+0000","last_peered":"2024-09-17T21:22:19.322291+0000","last_clean":"2024-09-17T21:22:19.322291+0000","last_became_active":"2024-09-17T21:18:01.049180+0000","last_became_peered":"2024-09-17T21:18:01.049180+0000","last_unstale":"2024-09-17T21:22:19.322291+0000","last_undegraded":"2024-09-17T21:22:19.322291+0000","last_fullsized":"2024-09-17T21:22:19.322291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:27.069794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077408,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"221'648","reported_seq":14285,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.319156+0000","last_change":"2024-09-17T21:22:17.326202+0000","last_active":"2024-09-17T21:22:19.319156+0000","last_peered":"2024-09-17T21:22:19.319156+0000","last_clean":"2024-09-17T21:22:19.319156+0000","last_became_active":"2024-09-17T21:21:46.418776+0000","last_became_peered":"2024-09-17T21:21:46.418776+0000","last_unstale":"2024-09-17T21:22:19.319156+0000","last_undegraded":"2024-09-17T21:22:19.319156+0000","last_fullsized":"2024-09-17T21:22:19.319156+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":648,"log_dups_size":0,"ondisk_log_size":648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:27:06.148218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010729489999999999,"stat_sum":{"num_bytes":32962576,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":12,"num_read":14738,"num_read_kb":38982,"num_write":741,"num_write_kb":20177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":265,"num_bytes_recovered":27258987,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"2.3","version":"19'1","reported_seq":125,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.318011+0000","last_change":"2024-09-17T21:21:41.350628+0000","last_active":"2024-09-17T21:22:19.318011+0000","last_peered":"2024-09-17T21:22:19.318011+0000","last_clean":"2024-09-17T21:22:19.318011+0000","last_became_active":"2024-09-17T21:21:41.334440+0000","last_became_peered":"2024-09-17T21:21:41.334440+0000","last_unstale":"2024-09-17T21:22:19.318011+0000","last_undegraded":"2024-09-17T21:22:19.318011+0000","last_fullsized":"2024-09-17T21:22:19.318011+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:44:18.810938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":139,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.360175+0000","last_change":"2024-09-17T21:21:17.347819+0000","last_active":"2024-09-17T21:22:19.360175+0000","last_peered":"2024-09-17T21:22:19.360175+0000","last_clean":"2024-09-17T21:22:19.360175+0000","last_became_active":"2024-09-17T21:21:17.337621+0000","last_became_peered":"2024-09-17T21:21:17.337621+0000","last_unstale":"2024-09-17T21:22:19.360175+0000","last_undegraded":"2024-09-17T21:22:19.360175+0000","last_fullsized":"2024-09-17T21:22:19.360175+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_clean_scrub_stamp":"2024-09-17T21:17:55.982339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:24:32.371870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"222'591","reported_seq":13970,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.373100+0000","last_change":"2024-09-17T21:22:19.331668+0000","last_active":"2024-09-17T21:22:19.373100+0000","last_peered":"2024-09-17T21:22:19.373100+0000","last_clean":"2024-09-17T21:22:19.373100+0000","last_became_active":"2024-09-17T21:22:19.324930+0000","last_became_peered":"2024-09-17T21:22:19.324930+0000","last_unstale":"2024-09-17T21:22:19.373100+0000","last_undegraded":"2024-09-17T21:22:19.373100+0000","last_fullsized":"2024-09-17T21:22:19.373100+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:06:26.031781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024336829999999999,"stat_sum":{"num_bytes":4549648,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":8,"num_read":13820,"num_read_kb":68286,"num_write":566,"num_write_kb":29108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":40776135,"num_keys_recovered":165,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"3.9","version":"216'685","reported_seq":11014,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.317975+0000","last_change":"2024-09-17T21:22:17.319978+0000","last_active":"2024-09-17T21:22:19.317975+0000","last_peered":"2024-09-17T21:22:19.317975+0000","last_clean":"2024-09-17T21:22:19.317975+0000","last_became_active":"2024-09-17T21:21:46.419053+0000","last_became_peered":"2024-09-17T21:21:46.419053+0000","last_unstale":"2024-09-17T21:22:19.317975+0000","last_undegraded":"2024-09-17T21:22:19.317975+0000","last_fullsized":"2024-09-17T21:22:19.317975+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:51:07.629364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001763925,"stat_sum":{"num_bytes":6532649,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":7,"num_read":11646,"num_read_kb":43590,"num_write":1073,"num_write_kb":7291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":544,"num_bytes_recovered":30193805,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"3.a","version":"222'421","reported_seq":7966,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.331182+0000","last_change":"2024-09-17T21:22:19.331182+0000","last_active":"2024-09-17T21:22:19.331182+0000","last_peered":"2024-09-17T21:22:19.331182+0000","last_clean":"2024-09-17T21:22:19.331182+0000","last_became_active":"2024-09-17T21:22:19.324649+0000","last_became_peered":"2024-09-17T21:22:19.324649+0000","last_unstale":"2024-09-17T21:22:19.331182+0000","last_undegraded":"2024-09-17T21:22:19.331182+0000","last_fullsized":"2024-09-17T21:22:19.331182+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:43:14.702046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040308100000000001,"stat_sum":{"num_bytes":5216850,"num_objects":16,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":11,"num_read":7180,"num_read_kb":40866,"num_write":364,"num_write_kb":16675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":179794,"num_keys_recovered":2,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"3.b","version":"221'373","reported_seq":4841,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.318129+0000","last_change":"2024-09-17T21:22:17.322445+0000","last_active":"2024-09-17T21:22:19.318129+0000","last_peered":"2024-09-17T21:22:19.318129+0000","last_clean":"2024-09-17T21:22:19.318129+0000","last_became_active":"2024-09-17T21:21:40.319613+0000","last_became_peered":"2024-09-17T21:21:40.319613+0000","last_unstale":"2024-09-17T21:22:19.318129+0000","last_undegraded":"2024-09-17T21:22:19.318129+0000","last_fullsized":"2024-09-17T21:22:19.318129+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:20.417697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096784200000000001,"stat_sum":{"num_bytes":6107648,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":7,"num_read":4694,"num_read_kb":23619,"num_write":319,"num_write_kb":13765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":4886965,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"3.c","version":"217'422","reported_seq":13258,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.321959+0000","last_change":"2024-09-17T21:22:17.322778+0000","last_active":"2024-09-17T21:22:19.321959+0000","last_peered":"2024-09-17T21:22:19.321959+0000","last_clean":"2024-09-17T21:22:19.321959+0000","last_became_active":"2024-09-17T21:21:45.403681+0000","last_became_peered":"2024-09-17T21:21:45.403681+0000","last_unstale":"2024-09-17T21:22:19.321959+0000","last_undegraded":"2024-09-17T21:22:19.321959+0000","last_fullsized":"2024-09-17T21:22:19.321959+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:20.142917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064363930000000003,"stat_sum":{"num_bytes":18737152,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":17,"num_read":15211,"num_read_kb":28919,"num_write":559,"num_write_kb":14313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":5960704,"num_keys_recovered":96,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"3.d","version":"221'526","reported_seq":15180,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.359917+0000","last_change":"2024-09-17T21:22:17.323527+0000","last_active":"2024-09-17T21:22:19.359917+0000","last_peered":"2024-09-17T21:22:19.359917+0000","last_clean":"2024-09-17T21:22:19.359917+0000","last_became_active":"2024-09-17T21:21:35.321558+0000","last_became_peered":"2024-09-17T21:21:35.321558+0000","last_unstale":"2024-09-17T21:22:19.359917+0000","last_undegraded":"2024-09-17T21:22:19.359917+0000","last_fullsized":"2024-09-17T21:22:19.359917+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:07:11.389523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00482724,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15785,"num_read_kb":43220,"num_write":654,"num_write_kb":16401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":28397872,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"3.e","version":"222'598","reported_seq":14109,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.418214+0000","last_change":"2024-09-17T21:22:17.324640+0000","last_active":"2024-09-17T21:22:19.418214+0000","last_peered":"2024-09-17T21:22:19.418214+0000","last_clean":"2024-09-17T21:22:19.418214+0000","last_became_active":"2024-09-17T21:21:44.379111+0000","last_became_peered":"2024-09-17T21:21:44.379111+0000","last_unstale":"2024-09-17T21:22:19.418214+0000","last_undegraded":"2024-09-17T21:22:19.418214+0000","last_fullsized":"2024-09-17T21:22:19.418214+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:16:45.163227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025899500000000002,"stat_sum":{"num_bytes":16759808,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":7,"num_read":15038,"num_read_kb":70182,"num_write":520,"num_write_kb":35851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":91925553,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"3.f","version":"215'460","reported_seq":7057,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.322712+0000","last_change":"2024-09-17T21:22:17.327962+0000","last_active":"2024-09-17T21:22:19.322712+0000","last_peered":"2024-09-17T21:22:19.322712+0000","last_clean":"2024-09-17T21:22:19.322712+0000","last_became_active":"2024-09-17T21:21:44.389560+0000","last_became_peered":"2024-09-17T21:21:44.389560+0000","last_unstale":"2024-09-17T21:22:19.322712+0000","last_undegraded":"2024-09-17T21:22:19.322712+0000","last_fullsized":"2024-09-17T21:22:19.322712+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:08.731222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089207880000000007,"stat_sum":{"num_bytes":1242112,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":13,"num_read":7427,"num_read_kb":28404,"num_write":434,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":4659071,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"3.10","version":"222'271","reported_seq":7340,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.348356+0000","last_change":"2024-09-17T21:22:17.319537+0000","last_active":"2024-09-17T21:22:19.348356+0000","last_peered":"2024-09-17T21:22:19.348356+0000","last_clean":"2024-09-17T21:22:19.348356+0000","last_became_active":"2024-09-17T21:21:44.378962+0000","last_became_peered":"2024-09-17T21:21:44.378962+0000","last_unstale":"2024-09-17T21:22:19.348356+0000","last_undegraded":"2024-09-17T21:22:19.348356+0000","last_fullsized":"2024-09-17T21:22:19.348356+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:51:47.838995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043344780000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7827,"num_read_kb":19883,"num_write":219,"num_write_kb":3298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":6733181,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"3.11","version":"217'398","reported_seq":5696,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.323118+0000","last_change":"2024-09-17T21:22:17.323027+0000","last_active":"2024-09-17T21:22:19.323118+0000","last_peered":"2024-09-17T21:22:19.323118+0000","last_clean":"2024-09-17T21:22:19.323118+0000","last_became_active":"2024-09-17T21:18:11.299946+0000","last_became_peered":"2024-09-17T21:18:11.299946+0000","last_unstale":"2024-09-17T21:22:19.323118+0000","last_undegraded":"2024-09-17T21:22:19.323118+0000","last_fullsized":"2024-09-17T21:22:19.323118+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:04:41.023939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069782109999999998,"stat_sum":{"num_bytes":5532720,"num_objects":16,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":11,"num_read":5166,"num_read_kb":22084,"num_write":378,"num_write_kb":4452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"3.12","version":"222'564","reported_seq":15914,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.419820+0000","last_change":"2024-09-17T21:22:17.317101+0000","last_active":"2024-09-17T21:22:19.419820+0000","last_peered":"2024-09-17T21:22:19.419820+0000","last_clean":"2024-09-17T21:22:19.419820+0000","last_became_active":"2024-09-17T21:22:09.609844+0000","last_became_peered":"2024-09-17T21:22:09.609844+0000","last_unstale":"2024-09-17T21:22:19.419820+0000","last_undegraded":"2024-09-17T21:22:19.419820+0000","last_fullsized":"2024-09-17T21:22:19.419820+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:56:55.519957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029157969999999999,"stat_sum":{"num_bytes":12535296,"num_objects":16,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":11,"num_read":17911,"num_read_kb":38759,"num_write":585,"num_write_kb":6968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":11589682,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"2"}]},{"pgid":"3.13","version":"222'628","reported_seq":12188,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:22:19.337837+0000","last_change":"2024-09-17T21:22:19.339270+0000","last_active":"2024-09-17T21:22:19.337837+0000","last_peered":"2024-09-17T21:22:19.337837+0000","last_clean":"2024-09-17T21:22:19.337837+0000","last_became_active":"2024-09-17T21:22:19.324694+0000","last_became_peered":"2024-09-17T21:22:19.324694+0000","last_unstale":"2024-09-17T21:22:19.337837+0000","last_undegraded":"2024-09-17T21:22:19.337837+0000","last_fullsized":"2024-09-17T21:22:19.337837+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:11:47.244466+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0097881410000000002,"stat_sum":{"num_bytes":1179648,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":12,"num_read":11806,"num_read_kb":51961,"num_write":599,"num_write_kb":5437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":5557098,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"2"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":226443794,"num_objects":441,"num_object_clones":32,"num_object_copies":882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":441,"num_whiteouts":325,"num_read":352774,"num_read_kb":1144934,"num_write":19701,"num_write_kb":385321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10043,"num_bytes_recovered":614462299,"num_keys_recovered":1529,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":145399808,"data_stored":228501735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18201,"ondisk_log_size":18201,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":0,"seq":55834574904,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81108,"kb_used_data":9656,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94290732,"statfs":{"total":96636764160,"available":96553709568,"internally_reserved":0,"allocated":9887744,"data_stored":8263788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":73136580},"hb_peers":[0,1,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574904,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":108016,"kb_used_data":32400,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94263824,"statfs":{"total":96636764160,"available":96526155776,"internally_reserved":0,"allocated":33177600,"data_stored":51678555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,1,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94000,"kb_used_data":40600,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94277840,"statfs":{"total":96636764160,"available":96540508160,"internally_reserved":0,"allocated":41574400,"data_stored":66394897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[0,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574905,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78048,"kb_used_data":24652,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94293792,"statfs":{"total":96636764160,"available":96556843008,"internally_reserved":0,"allocated":25243648,"data_stored":36180140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":51539607602,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22102016,"data_stored":34588419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":38875136,"data_stored":65240392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5705728,"data_stored":10004626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":29405184,"data_stored":49463643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3211264,"data_stored":3163774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":46100480,"data_stored":66040881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:22:20.161 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:20.157+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:20.195 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:20.195 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:20.195 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:22:20.195 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:20.209 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:22:20.262 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:20.258+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:20.352 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:22:20.379 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:20.379 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:20.379 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:22:20.379 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:22:20.379 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:20.393 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:22:20.401 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:20.398+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:20.464 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:20.458+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:20.519 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:20.519 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:22:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:22:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:22:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:22:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:22:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 263.50116349199999, 2024-09-17T21:22:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:22:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:22:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:22:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:22:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:22:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:22:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:22:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:22:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:22:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:22:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:22:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:22:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:22:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:22:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:22:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:22:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:22:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:22:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:22:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:22:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:22:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:22:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:22:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:22:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:22:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:22:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:22:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:22:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:22:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:22:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:22:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:22:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:22:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:22:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:22:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 243.920999212, 2024-09-17T21:22:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:22:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:22:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:22:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:22:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:22:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:22:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:22:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:22:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:22:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:22:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:22:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:22:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:22:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:22:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:22:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:22:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:22:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:22:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:22:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:22:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:22:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:22:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:22:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:22:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:22:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:22:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:22:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:22:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:22:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:22:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:22:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 243.92090189999999, 2024-09-17T21:22:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:22:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:22:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:22:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:22:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:22:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:22:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:22:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:22:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:22:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:22:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:22:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:22:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:22:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:22:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:22:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:22:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:22:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:22:20.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:22:20.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:20.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:22:20.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:22:20.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:22:20.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:22:20.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:22:20.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:22:20.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:22:20.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:22:20.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:22:20.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:22:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:22:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:22:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:22:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:22:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:22:20.540 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 243.918815099, 2024-09-17T21:22:20.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:22:20.540 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:20.540 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:20.540 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:20.540 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:20.540 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:22:20.540 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:22:20.540 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:22:20.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.540 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:20.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:20.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:22:20.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:20.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:22:20.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:20.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:22:20.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:22:20.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:20.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:22:20.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:22:20.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:20.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:22:20.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:22:20.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:22:20.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:22:20.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:22:20.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:22:20.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:22:20.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:22:20.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:22:20.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:22:20.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:22:20.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:20.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:22:20.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:22:20.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:22:20.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:22:20.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:20.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:22:20.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:22:20.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:22:20.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:22:20.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:22:20.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:20.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:22:20.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:22:20.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:20.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:22:20.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:22:20.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:20.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:22:20.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:22:20.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:20.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:22:20.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:22:20.546 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.547 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:20.547 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.547 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:20.547 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:22:20.547 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 211.23491631799999, 2024-09-17T21:22:20.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:22:20.547 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:20.547 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:20.547 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:20.547 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:20.547 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:22:20.548 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:22:20.548 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:22:20.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.548 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:20.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:20.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:22:20.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:20.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:22:20.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:20.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:22:20.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:22:20.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:20.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:22:20.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:22:20.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:20.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:22:20.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:22:20.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:22:20.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:22:20.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:22:20.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:22:20.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:22:20.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:22:20.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:22:20.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:22:20.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:22:20.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:20.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:22:20.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:22:20.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:22:20.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:22:20.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:20.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:22:20.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:22:20.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:20.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:22:20.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:22:20.553 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.553 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:20.553 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.553 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:20.553 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:22:20.553 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 211.22582460699999, 2024-09-17T21:22:20.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:22:20.553 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:20.553 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:20.553 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:20.553 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:20.553 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:22:20.554 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:22:20.554 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:22:20.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.554 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:20.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:20.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:22:20.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:20.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:22:20.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:20.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:22:20.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:22:20.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:20.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:22:20.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:22:20.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:20.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:22:20.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:22:20.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:22:20.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:22:20.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:22:20.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:22:20.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:20.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:22:20.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:22:20.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.557 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:22:20.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:22:20.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.557 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:20.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:22:20.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:22:20.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:20.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:22:20.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:22:20.558 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.558 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:20.558 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.558 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:20.558 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:22:20.558 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 211.225652556, 2024-09-17T21:22:20.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:22:20.558 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:20.558 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:20.559 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:20.559 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:20.559 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:22:20.559 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:22:20.559 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:22:20.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.559 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:20.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:20.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:22:20.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:20.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:22:20.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:20.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:22:20.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:22:20.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:20.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:22:20.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:22:20.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:20.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:22:20.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:22:20.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:22:20.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:22:20.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:22:20.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:22:20.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:20.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:22:20.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:22:20.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.562 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:22:20.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:22:20.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.562 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:20.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:22:20.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:22:20.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:20.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:22:20.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:22:20.563 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.563 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:20.563 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.563 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:20.563 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:22:20.563 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 211.05071980899999, 2024-09-17T21:22:20.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:22:20.563 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:20.563 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:20.563 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:20.564 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:20.564 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:20.564 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:20.564 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:22:20.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.564 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:20.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:20.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:22:20.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:20.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:22:20.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:20.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:22:20.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:22:20.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:20.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:22:20.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:22:20.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:20.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:22:20.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:22:20.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:22:20.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:22:20.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:22:20.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:22:20.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:20.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:22:20.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:22:20.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:22:20.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:22:20.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:20.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:22:20.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:22:20.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:20.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:22:20.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:22:20.568 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.568 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:20.568 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.568 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:20.568 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:22:20.568 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 211.05063525200001, 2024-09-17T21:22:20.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:22:20.568 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:20.568 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:20.568 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:20.569 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:20.569 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:20.569 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:20.569 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:22:20.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.569 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:20.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:20.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:22:20.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:20.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:22:20.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:20.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:22:20.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:22:20.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:20.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:22:20.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:22:20.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:20.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:22:20.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:22:20.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:22:20.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:22:20.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:22:20.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:22:20.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:20.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:22:20.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:22:20.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:22:20.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:22:20.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:20.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:22:20.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:22:20.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:20.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:22:20.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:22:20.573 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.573 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:20.573 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.573 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:20.573 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:22:20.573 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 211.050569615, 2024-09-17T21:22:20.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:22:20.573 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:20.573 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:20.573 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:20.574 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:20.574 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:20.574 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:20.574 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:22:20.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.574 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:20.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:20.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:22:20.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:20.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:22:20.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:20.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:22:20.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:22:20.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:20.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:22:20.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:22:20.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:20.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:22:20.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:22:20.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:22:20.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:22:20.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:22:20.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:22:20.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:20.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:22:20.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:22:20.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:22:20.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:22:20.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:20.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:22:20.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:22:20.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:20.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:22:20.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:22:20.578 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.578 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:20.578 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.578 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:20.578 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:22:20.578 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 159.67699451799999, 2024-09-17T21:22:20.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:22:20.578 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:20.578 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:20.579 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:20.579 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:20.579 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:22:20.579 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:20.579 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:22:20.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.579 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:20.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:20.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:22:20.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:20.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:22:20.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:20.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:22:20.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:22:20.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:20.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:22:20.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:22:20.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:20.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:22:20.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:22:20.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:22:20.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:22:20.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:22:20.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:22:20.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:20.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:22:20.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:22:20.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:20.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:22:20.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:22:20.582 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.582 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:20.582 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.583 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:20.583 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:22:20.583 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 159.67687500599999, 2024-09-17T21:22:20.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:22:20.583 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:20.583 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:20.583 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:20.583 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:20.583 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:22:20.583 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:20.583 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:22:20.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.584 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:20.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:20.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:22:20.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:20.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:22:20.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:20.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:22:20.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:22:20.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:20.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:22:20.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:22:20.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:20.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:22:20.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:22:20.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:22:20.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:22:20.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.586 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:22:20.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:22:20.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.586 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:20.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:22:20.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:22:20.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.586 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:20.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:22:20.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:22:20.587 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.587 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:20.587 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.587 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:996 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:20.587 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:22:20.587 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 115.083761635, 2024-09-17T21:22:20.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084493736999999999, 2024-09-17T21:22:20.587 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:20.587 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:20.588 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:20.588 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:20.588 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:20.588 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:20.588 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 996 2024-09-17T21:22:20.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.588 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:20.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:20.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:22:20.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:20.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:22:20.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:20.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431168+0000", 2024-09-17T21:22:20.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4e-07 2024-09-17T21:22:20.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:20.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431172+0000", 2024-09-17T21:22:20.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.608e-06 2024-09-17T21:22:20.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:20.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431180+0000", 2024-09-17T21:22:20.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5650000000000002e-06 2024-09-17T21:22:20.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431188+0000", 2024-09-17T21:22:20.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:22:20.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515354+0000", 2024-09-17T21:22:20.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084165977000000003 2024-09-17T21:22:20.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:20.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515451+0000", 2024-09-17T21:22:20.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6669000000000006e-05 2024-09-17T21:22:20.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:20.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515661+0000", 2024-09-17T21:22:20.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021021599999999999 2024-09-17T21:22:20.591 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.591 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:20.591 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.592 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:997 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 583680~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:20.592 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:22:20.592 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 115.08373023999999, 2024-09-17T21:22:20.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084772419000000002, 2024-09-17T21:22:20.592 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:20.592 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:20.592 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:20.592 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:20.592 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:20.592 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:20.592 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 997 2024-09-17T21:22:20.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.592 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:20.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:20.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:22:20.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:20.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:22:20.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:20.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:22:20.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0400000000000005e-07 2024-09-17T21:22:20.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:20.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431202+0000", 2024-09-17T21:22:20.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:22:20.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:20.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431208+0000", 2024-09-17T21:22:20.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4740000000000002e-06 2024-09-17T21:22:20.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431214+0000", 2024-09-17T21:22:20.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6550000000000001e-06 2024-09-17T21:22:20.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515741+0000", 2024-09-17T21:22:20.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084526904 2024-09-17T21:22:20.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:20.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515826+0000", 2024-09-17T21:22:20.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4643999999999998e-05 2024-09-17T21:22:20.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:20.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515971+0000", 2024-09-17T21:22:20.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145444 2024-09-17T21:22:20.596 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.596 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:20.596 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.596 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:998 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 587776~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:20.596 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:22:20.596 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 115.083707268, 2024-09-17T21:22:20.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085108565999999997, 2024-09-17T21:22:20.597 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:20.597 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:20.597 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:20.597 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:20.597 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:20.597 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:20.597 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 998 2024-09-17T21:22:20.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.597 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:20.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:20.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:22:20.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:20.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:22:20.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:20.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:22:20.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:22:20.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:20.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431224+0000", 2024-09-17T21:22:20.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:22:20.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:20.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431231+0000", 2024-09-17T21:22:20.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000002e-06 2024-09-17T21:22:20.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431236+0000", 2024-09-17T21:22:20.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.648e-06 2024-09-17T21:22:20.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516024+0000", 2024-09-17T21:22:20.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084787584999999999 2024-09-17T21:22:20.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:20.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516122+0000", 2024-09-17T21:22:20.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8076000000000001e-05 2024-09-17T21:22:20.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:20.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516330+0000", 2024-09-17T21:22:20.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020808 2024-09-17T21:22:20.600 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.600 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:20.601 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.601 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1000 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 591872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:20.601 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:22:20.601 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 115.083685153, 2024-09-17T21:22:20.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085389065, 2024-09-17T21:22:20.601 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:20.601 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:20.601 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:20.601 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:20.601 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:20.601 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:20.602 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1000 2024-09-17T21:22:20.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.602 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:20.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:20.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:22:20.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:20.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:22:20.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:20.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:22:20.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:22:20.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:20.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431246+0000", 2024-09-17T21:22:20.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1749999999999999e-06 2024-09-17T21:22:20.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:20.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431253+0000", 2024-09-17T21:22:20.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2790000000000004e-06 2024-09-17T21:22:20.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431258+0000", 2024-09-17T21:22:20.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3299999999999998e-06 2024-09-17T21:22:20.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516384+0000", 2024-09-17T21:22:20.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085125549999999994 2024-09-17T21:22:20.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:20.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516460+0000", 2024-09-17T21:22:20.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6359999999999994e-05 2024-09-17T21:22:20.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:20.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516633+0000", 2024-09-17T21:22:20.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017285900000000001 2024-09-17T21:22:20.605 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.605 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:20.605 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.605 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1001 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 595968~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:20.605 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:22:20.605 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 115.08366174699999, 2024-09-17T21:22:20.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085614715999999993, 2024-09-17T21:22:20.606 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:20.606 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:20.606 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:20.606 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:20.606 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:20.606 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:20.606 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1001 2024-09-17T21:22:20.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.606 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:20.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:20.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:22:20.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:20.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:22:20.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:20.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438066+0000", 2024-09-17T21:22:20.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067989490000000003 2024-09-17T21:22:20.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:20.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438073+0000", 2024-09-17T21:22:20.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8419999999999999e-06 2024-09-17T21:22:20.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:20.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438090+0000", 2024-09-17T21:22:20.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6657e-05 2024-09-17T21:22:20.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438099+0000", 2024-09-17T21:22:20.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0529999999999996e-06 2024-09-17T21:22:20.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516678+0000", 2024-09-17T21:22:20.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078579422999999995 2024-09-17T21:22:20.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:20.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516739+0000", 2024-09-17T21:22:20.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1315000000000005e-05 2024-09-17T21:22:20.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:20.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516882+0000", 2024-09-17T21:22:20.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014247700000000001 2024-09-17T21:22:20.610 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.610 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:20.610 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.610 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:19998 3.5 3:a4885450:::rbd_object_map.1152d139f9f9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-17T21:22:20.610 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:22:20.610 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.112394502, 2024-09-17T21:22:20.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90715489299999996, 2024-09-17T21:22:20.610 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:20.610 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:20.610 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:20.610 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:20.610 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:20.611 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:20.611 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19998 2024-09-17T21:22:20.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.611 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:20.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:20.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:22:20.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:20.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:22:20.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:20.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402537+0000", 2024-09-17T21:22:20.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5600000000000001e-06 2024-09-17T21:22:20.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:20.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402549+0000", 2024-09-17T21:22:20.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1929999999999999e-05 2024-09-17T21:22:20.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:20.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402579+0000", 2024-09-17T21:22:20.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0093000000000001e-05 2024-09-17T21:22:20.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402595+0000", 2024-09-17T21:22:20.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5855999999999999e-05 2024-09-17T21:22:20.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402731+0000", 2024-09-17T21:22:20.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013610799999999999 2024-09-17T21:22:20.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:20.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402750+0000", 2024-09-17T21:22:20.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8737e-05 2024-09-17T21:22:20.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.614 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.305977+0000", 2024-09-17T21:22:20.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90322768399999998 2024-09-17T21:22:20.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.614 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:20.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306236+0000", 2024-09-17T21:22:20.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000258111 2024-09-17T21:22:20.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:22:20.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306557+0000", 2024-09-17T21:22:20.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032159799999999998 2024-09-17T21:22:20.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:20.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.308286+0000", 2024-09-17T21:22:20.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017292150000000001 2024-09-17T21:22:20.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:20.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309530+0000", 2024-09-17T21:22:20.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00124409 2024-09-17T21:22:20.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:20.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309630+0000", 2024-09-17T21:22:20.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9689e-05 2024-09-17T21:22:20.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:20.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309689+0000", 2024-09-17T21:22:20.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9222000000000003e-05 2024-09-17T21:22:20.616 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.616 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:20.616 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.617 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:25277 3.1f 3:ff5d7cd5:::rbd_header.11524f65d4ff:head [watch ping cookie 139860845478480 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-17T21:22:20.617 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:22:20.617 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 50.929438433000001, 2024-09-17T21:22:20.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.92185720000000004, 2024-09-17T21:22:20.617 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:20.617 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:20.617 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:20.617 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:20.617 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:20.617 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:20.617 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 25277 2024-09-17T21:22:20.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.617 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:22:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:22:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585493+0000", 2024-09-17T21:22:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:22:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:20.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585505+0000", 2024-09-17T21:22:20.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2466000000000001e-05 2024-09-17T21:22:20.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:20.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585537+0000", 2024-09-17T21:22:20.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1183000000000003e-05 2024-09-17T21:22:20.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585553+0000", 2024-09-17T21:22:20.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.662e-05 2024-09-17T21:22:20.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585696+0000", 2024-09-17T21:22:20.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014293800000000001 2024-09-17T21:22:20.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:20.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585718+0000", 2024-09-17T21:22:20.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1860000000000001e-05 2024-09-17T21:22:20.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.501188+0000", 2024-09-17T21:22:20.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.91547051300000004 2024-09-17T21:22:20.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:22:20.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.501260+0000", 2024-09-17T21:22:20.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1571999999999998e-05 2024-09-17T21:22:20.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507002+0000", 2024-09-17T21:22:20.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057414850000000002 2024-09-17T21:22:20.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:20.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507143+0000", 2024-09-17T21:22:20.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014140299999999999 2024-09-17T21:22:20.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:20.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507348+0000", 2024-09-17T21:22:20.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000204773 2024-09-17T21:22:20.622 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.622 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:20.622 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.622 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:26042 3.f 3:f24134c1:::rbd_id.image_client.0-clone34:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e186)", 2024-09-17T21:22:20.623 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:22:20.623 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 37.028750064999997, 2024-09-17T21:22:20.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.91185481000000002, 2024-09-17T21:22:20.623 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:20.623 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:20.623 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:20.623 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:20.623 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:20.623 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:20.623 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 26042 2024-09-17T21:22:20.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.623 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:20.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:20.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:22:20.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:20.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:22:20.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:20.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:20.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486180+0000", 2024-09-17T21:22:20.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5990000000000001e-06 2024-09-17T21:22:20.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:20.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486187+0000", 2024-09-17T21:22:20.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7859999999999999e-06 2024-09-17T21:22:20.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:20.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486206+0000", 2024-09-17T21:22:20.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8641e-05 2024-09-17T21:22:20.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:20.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486219+0000", 2024-09-17T21:22:20.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.308e-05 2024-09-17T21:22:20.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486283+0000", 2024-09-17T21:22:20.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4084000000000005e-05 2024-09-17T21:22:20.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:20.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486290+0000", 2024-09-17T21:22:20.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.584e-06 2024-09-17T21:22:20.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:20.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392182+0000", 2024-09-17T21:22:20.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90589206499999997 2024-09-17T21:22:20.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:20.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392518+0000", 2024-09-17T21:22:20.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033596 2024-09-17T21:22:20.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:22:20.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392802+0000", 2024-09-17T21:22:20.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028394999999999999 2024-09-17T21:22:20.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:20.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397705+0000", 2024-09-17T21:22:20.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0049031769999999999 2024-09-17T21:22:20.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:20.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397770+0000", 2024-09-17T21:22:20.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4765999999999996e-05 2024-09-17T21:22:20.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:20.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397927+0000", 2024-09-17T21:22:20.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015699900000000001 2024-09-17T21:22:20.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:20.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:20.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:20.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.398034+0000", 2024-09-17T21:22:20.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000107119 2024-09-17T21:22:20.629 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.629 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:20.629 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.629 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:20.629 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:20.629 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:20.631 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:22:20.631 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:20.625+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:20.731 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:20.726+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:20.799 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:20.799 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:20.799 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:22:20.799 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:20.813 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:22:20.859 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:20.857+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:20.933 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:20.929+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:20.937 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:20.937 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:20.938 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:22:20.938 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:22:20.938 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:20.951 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:22:21.034 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:21.030+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:21.120 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:21.121 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:22:21.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:22:21.121 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:22:21.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.121 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:18003 3.1d 3:b9765580:::rbd_header.11527d2b15f6:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T21:22:21.121 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:22:21.121 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 128.75841895400001, 2024-09-17T21:22:21.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94165346900000002, 2024-09-17T21:22:21.121 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.121 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.121 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.121 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.121 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:21.122 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:21.122 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18003 2024-09-17T21:22:21.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.122 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:22:21.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:22:21.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357880+0000", 2024-09-17T21:22:21.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:22:21.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357891+0000", 2024-09-17T21:22:21.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0417999999999999e-05 2024-09-17T21:22:21.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357913+0000", 2024-09-17T21:22:21.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2215e-05 2024-09-17T21:22:21.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357922+0000", 2024-09-17T21:22:21.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4779999999999997e-06 2024-09-17T21:22:21.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358023+0000", 2024-09-17T21:22:21.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101025 2024-09-17T21:22:21.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:21.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358040+0000", 2024-09-17T21:22:21.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6291999999999999e-05 2024-09-17T21:22:21.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298203+0000", 2024-09-17T21:22:21.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94016297500000001 2024-09-17T21:22:21.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298626+0000", 2024-09-17T21:22:21.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000422853 2024-09-17T21:22:21.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.299532+0000", 2024-09-17T21:22:21.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00090657300000000004 2024-09-17T21:22:21.126 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.126 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.126 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.127 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15707 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 982016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:21.127 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:22:21.127 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 118.595253707, 2024-09-17T21:22:21.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092305228000000003, 2024-09-17T21:22:21.127 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.127 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.127 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.127 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.127 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:21.127 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:21.127 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15707 2024-09-17T21:22:21.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.128 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:22:21.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:22:21.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521045+0000", 2024-09-17T21:22:21.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4899999999999999e-06 2024-09-17T21:22:21.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521066+0000", 2024-09-17T21:22:21.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0236000000000001e-05 2024-09-17T21:22:21.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521085+0000", 2024-09-17T21:22:21.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8819999999999999e-05 2024-09-17T21:22:21.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521100+0000", 2024-09-17T21:22:21.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5146e-05 2024-09-17T21:22:21.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612978+0000", 2024-09-17T21:22:21.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091878847 2024-09-17T21:22:21.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613079+0000", 2024-09-17T21:22:21.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100503 2024-09-17T21:22:21.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613349+0000", 2024-09-17T21:22:21.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000270186 2024-09-17T21:22:21.131 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.131 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.131 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.132 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15708 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 986112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:21.132 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:22:21.132 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 118.595175532, 2024-09-17T21:22:21.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092547217000000001, 2024-09-17T21:22:21.132 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.132 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.132 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.132 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.132 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:21.132 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:21.133 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15708 2024-09-17T21:22:21.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.133 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:22:21.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:22:21.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521123+0000", 2024-09-17T21:22:21.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.353e-06 2024-09-17T21:22:21.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521129+0000", 2024-09-17T21:22:21.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:22:21.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521145+0000", 2024-09-17T21:22:21.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.611e-05 2024-09-17T21:22:21.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521154+0000", 2024-09-17T21:22:21.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4500000000000004e-06 2024-09-17T21:22:21.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613397+0000", 2024-09-17T21:22:21.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092243275 2024-09-17T21:22:21.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613503+0000", 2024-09-17T21:22:21.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106013 2024-09-17T21:22:21.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613669+0000", 2024-09-17T21:22:21.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016642299999999999 2024-09-17T21:22:21.136 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.136 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.136 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.137 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15709 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 990208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:21.137 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:22:21.137 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 118.59513251600001, 2024-09-17T21:22:21.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092728678999999994, 2024-09-17T21:22:21.137 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.137 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.137 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.137 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.137 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:21.137 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:21.138 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15709 2024-09-17T21:22:21.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.138 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:22:21.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:22:21.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521166+0000", 2024-09-17T21:22:21.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7000000000000004e-07 2024-09-17T21:22:21.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521169+0000", 2024-09-17T21:22:21.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2660000000000002e-06 2024-09-17T21:22:21.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521181+0000", 2024-09-17T21:22:21.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.208e-05 2024-09-17T21:22:21.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521194+0000", 2024-09-17T21:22:21.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2707e-05 2024-09-17T21:22:21.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613695+0000", 2024-09-17T21:22:21.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092500709 2024-09-17T21:22:21.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613750+0000", 2024-09-17T21:22:21.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.503e-05 2024-09-17T21:22:21.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613894+0000", 2024-09-17T21:22:21.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144117 2024-09-17T21:22:21.141 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.141 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.141 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.142 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15710 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 994304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:21.142 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:21.142 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 118.59508795000001, 2024-09-17T21:22:21.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092957729000000003, 2024-09-17T21:22:21.142 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.142 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.142 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.142 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.142 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:21.142 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:21.142 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15710 2024-09-17T21:22:21.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.142 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:21.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:21.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:21.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8000000000000005e-07 2024-09-17T21:22:21.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521216+0000", 2024-09-17T21:22:21.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3859999999999998e-06 2024-09-17T21:22:21.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521234+0000", 2024-09-17T21:22:21.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:22:21.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521248+0000", 2024-09-17T21:22:21.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:22:21.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613926+0000", 2024-09-17T21:22:21.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092678324000000006 2024-09-17T21:22:21.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613974+0000", 2024-09-17T21:22:21.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7754999999999999e-05 2024-09-17T21:22:21.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614167+0000", 2024-09-17T21:22:21.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193508 2024-09-17T21:22:21.146 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.146 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.147 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.147 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15711 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 998400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:21.147 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:22:21.147 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 118.595030568, 2024-09-17T21:22:21.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093268000000000004, 2024-09-17T21:22:21.147 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.147 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.147 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.147 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.147 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:21.147 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:21.147 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15711 2024-09-17T21:22:21.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.148 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:22:21.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:22:21.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521268+0000", 2024-09-17T21:22:21.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-17T21:22:21.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521273+0000", 2024-09-17T21:22:21.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:22:21.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521290+0000", 2024-09-17T21:22:21.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7269999999999999e-05 2024-09-17T21:22:21.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521304+0000", 2024-09-17T21:22:21.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3626e-05 2024-09-17T21:22:21.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614220+0000", 2024-09-17T21:22:21.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092916367 2024-09-17T21:22:21.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614322+0000", 2024-09-17T21:22:21.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102212 2024-09-17T21:22:21.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614535+0000", 2024-09-17T21:22:21.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021266199999999999 2024-09-17T21:22:21.151 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.151 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.151 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.151 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15712 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1002496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:21.151 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:22:21.151 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 118.594975073, 2024-09-17T21:22:21.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093568707000000001, 2024-09-17T21:22:21.152 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.152 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.152 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.152 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.152 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:21.152 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:21.152 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15712 2024-09-17T21:22:21.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.152 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:22:21.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:22:21.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521324+0000", 2024-09-17T21:22:21.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.17e-06 2024-09-17T21:22:21.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521334+0000", 2024-09-17T21:22:21.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9369999999999999e-06 2024-09-17T21:22:21.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521356+0000", 2024-09-17T21:22:21.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2045999999999999e-05 2024-09-17T21:22:21.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521370+0000", 2024-09-17T21:22:21.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4270000000000001e-05 2024-09-17T21:22:21.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614583+0000", 2024-09-17T21:22:21.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093213220999999999 2024-09-17T21:22:21.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614680+0000", 2024-09-17T21:22:21.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6900999999999999e-05 2024-09-17T21:22:21.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614891+0000", 2024-09-17T21:22:21.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021116200000000001 2024-09-17T21:22:21.156 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.156 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.156 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.156 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15713 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1006592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:21.156 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:22:21.156 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 118.594908371, 2024-09-17T21:22:21.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093834641999999996, 2024-09-17T21:22:21.156 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.157 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.157 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.157 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.157 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:21.157 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:21.157 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15713 2024-09-17T21:22:21.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.157 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:22:21.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:22:21.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521390+0000", 2024-09-17T21:22:21.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:22:21.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521396+0000", 2024-09-17T21:22:21.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:22:21.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521417+0000", 2024-09-17T21:22:21.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1236000000000001e-05 2024-09-17T21:22:21.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521431+0000", 2024-09-17T21:22:21.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3873000000000001e-05 2024-09-17T21:22:21.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614938+0000", 2024-09-17T21:22:21.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093506522999999994 2024-09-17T21:22:21.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615025+0000", 2024-09-17T21:22:21.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7635999999999999e-05 2024-09-17T21:22:21.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615224+0000", 2024-09-17T21:22:21.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019844700000000001 2024-09-17T21:22:21.160 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.160 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.160 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.161 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15714 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:21.161 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:22:21.161 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 118.594845115, 2024-09-17T21:22:21.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094071925000000001, 2024-09-17T21:22:21.161 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.161 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.161 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.161 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.161 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:21.161 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:21.161 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15714 2024-09-17T21:22:21.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.162 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:22:21.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:22:21.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521454+0000", 2024-09-17T21:22:21.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.097e-06 2024-09-17T21:22:21.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521459+0000", 2024-09-17T21:22:21.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2660000000000001e-06 2024-09-17T21:22:21.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521481+0000", 2024-09-17T21:22:21.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1878999999999998e-05 2024-09-17T21:22:21.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521498+0000", 2024-09-17T21:22:21.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6807e-05 2024-09-17T21:22:21.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615271+0000", 2024-09-17T21:22:21.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093773286999999997 2024-09-17T21:22:21.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615366+0000", 2024-09-17T21:22:21.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5322000000000007e-05 2024-09-17T21:22:21.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615524+0000", 2024-09-17T21:22:21.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015826699999999999 2024-09-17T21:22:21.165 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.165 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.165 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.165 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15715 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1014784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:21.165 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:22:21.165 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 118.59478077999999, 2024-09-17T21:22:21.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094243840999999995, 2024-09-17T21:22:21.166 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.166 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.166 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.166 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.166 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:21.166 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:21.166 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15715 2024-09-17T21:22:21.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.166 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:22:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:22:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521518+0000", 2024-09-17T21:22:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4300000000000001e-06 2024-09-17T21:22:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521524+0000", 2024-09-17T21:22:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5860000000000004e-06 2024-09-17T21:22:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521541+0000", 2024-09-17T21:22:21.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7574000000000001e-05 2024-09-17T21:22:21.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521555+0000", 2024-09-17T21:22:21.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3540000000000001e-05 2024-09-17T21:22:21.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615560+0000", 2024-09-17T21:22:21.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.09400472 2024-09-17T21:22:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615614+0000", 2024-09-17T21:22:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4693e-05 2024-09-17T21:22:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615761+0000", 2024-09-17T21:22:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000146298 2024-09-17T21:22:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.170 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15716 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1018880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:22:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 118.594724435, 2024-09-17T21:22:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094400676000000003, 2024-09-17T21:22:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:21.171 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15716 2024-09-17T21:22:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:22:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:22:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521574+0000", 2024-09-17T21:22:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1939999999999999e-06 2024-09-17T21:22:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521579+0000", 2024-09-17T21:22:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:22:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521597+0000", 2024-09-17T21:22:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8074e-05 2024-09-17T21:22:21.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521613+0000", 2024-09-17T21:22:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5316e-05 2024-09-17T21:22:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615780+0000", 2024-09-17T21:22:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094166874999999997 2024-09-17T21:22:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615836+0000", 2024-09-17T21:22:21.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6926999999999999e-05 2024-09-17T21:22:21.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615974+0000", 2024-09-17T21:22:21.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013741400000000001 2024-09-17T21:22:21.175 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.175 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.175 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.175 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15717 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1022976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:21.175 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:22:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 118.59466573900001, 2024-09-17T21:22:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094586723999999997, 2024-09-17T21:22:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15717 2024-09-17T21:22:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:22:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:22:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521633+0000", 2024-09-17T21:22:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:22:21.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521638+0000", 2024-09-17T21:22:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:22:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521659+0000", 2024-09-17T21:22:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-05 2024-09-17T21:22:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521675+0000", 2024-09-17T21:22:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5688999999999998e-05 2024-09-17T21:22:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616008+0000", 2024-09-17T21:22:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094333314000000001 2024-09-17T21:22:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616066+0000", 2024-09-17T21:22:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7797000000000002e-05 2024-09-17T21:22:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616219+0000", 2024-09-17T21:22:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015281100000000001 2024-09-17T21:22:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.179 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15718 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:22:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 118.594565664, 2024-09-17T21:22:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094703674000000002, 2024-09-17T21:22:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:21.180 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15718 2024-09-17T21:22:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:22:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:22:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521734+0000", 2024-09-17T21:22:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.61e-06 2024-09-17T21:22:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521740+0000", 2024-09-17T21:22:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:22:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521754+0000", 2024-09-17T21:22:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3793e-05 2024-09-17T21:22:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521763+0000", 2024-09-17T21:22:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5129999999999996e-06 2024-09-17T21:22:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616256+0000", 2024-09-17T21:22:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094492919999999994 2024-09-17T21:22:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616312+0000", 2024-09-17T21:22:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6737000000000002e-05 2024-09-17T21:22:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616436+0000", 2024-09-17T21:22:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012335800000000001 2024-09-17T21:22:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15719 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1031168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:22:21.184 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 118.594512259, 2024-09-17T21:22:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094888963000000007, 2024-09-17T21:22:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15719 2024-09-17T21:22:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.185 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:22:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:22:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521786+0000", 2024-09-17T21:22:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.074e-06 2024-09-17T21:22:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521791+0000", 2024-09-17T21:22:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3030000000000001e-06 2024-09-17T21:22:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521804+0000", 2024-09-17T21:22:21.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3027000000000001e-05 2024-09-17T21:22:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521811+0000", 2024-09-17T21:22:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6090000000000003e-06 2024-09-17T21:22:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616466+0000", 2024-09-17T21:22:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094654294 2024-09-17T21:22:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616523+0000", 2024-09-17T21:22:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6953999999999997e-05 2024-09-17T21:22:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616674+0000", 2024-09-17T21:22:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015170200000000001 2024-09-17T21:22:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15720 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1035264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:22:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 118.59447541900001, 2024-09-17T21:22:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095067061999999994, 2024-09-17T21:22:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15720 2024-09-17T21:22:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.190 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:22:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:22:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521823+0000", 2024-09-17T21:22:21.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:22:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521826+0000", 2024-09-17T21:22:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2100000000000002e-06 2024-09-17T21:22:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521835+0000", 2024-09-17T21:22:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1630000000000002e-06 2024-09-17T21:22:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521842+0000", 2024-09-17T21:22:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:22:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616711+0000", 2024-09-17T21:22:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094868859 2024-09-17T21:22:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616764+0000", 2024-09-17T21:22:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2778999999999997e-05 2024-09-17T21:22:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616889+0000", 2024-09-17T21:22:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012533100000000001 2024-09-17T21:22:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15721 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1039360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:22:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 118.59442831, 2024-09-17T21:22:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095277219999999996, 2024-09-17T21:22:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.194 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15721 2024-09-17T21:22:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:22:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:22:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521871+0000", 2024-09-17T21:22:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.443e-06 2024-09-17T21:22:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521876+0000", 2024-09-17T21:22:21.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0300000000000001e-06 2024-09-17T21:22:21.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521886+0000", 2024-09-17T21:22:21.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186e-05 2024-09-17T21:22:21.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521893+0000", 2024-09-17T21:22:21.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.08e-06 2024-09-17T21:22:21.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616918+0000", 2024-09-17T21:22:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095025223000000006 2024-09-17T21:22:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616977+0000", 2024-09-17T21:22:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8403999999999999e-05 2024-09-17T21:22:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617147+0000", 2024-09-17T21:22:21.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016985399999999999 2024-09-17T21:22:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15722 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 118.594393861, 2024-09-17T21:22:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095459783000000006, 2024-09-17T21:22:21.199 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15722 2024-09-17T21:22:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7700000000000004e-07 2024-09-17T21:22:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521907+0000", 2024-09-17T21:22:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.813e-06 2024-09-17T21:22:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521922+0000", 2024-09-17T21:22:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4533e-05 2024-09-17T21:22:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521934+0000", 2024-09-17T21:22:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2469999999999999e-05 2024-09-17T21:22:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617187+0000", 2024-09-17T21:22:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095252311000000006 2024-09-17T21:22:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617240+0000", 2024-09-17T21:22:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3740999999999999e-05 2024-09-17T21:22:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617364+0000", 2024-09-17T21:22:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-17T21:22:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15723 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:22:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 118.594349995, 2024-09-17T21:22:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095653563999999996, 2024-09-17T21:22:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15723 2024-09-17T21:22:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:22:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:22:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521949+0000", 2024-09-17T21:22:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:22:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521953+0000", 2024-09-17T21:22:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5870000000000003e-06 2024-09-17T21:22:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521970+0000", 2024-09-17T21:22:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7282000000000002e-05 2024-09-17T21:22:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521984+0000", 2024-09-17T21:22:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:22:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617383+0000", 2024-09-17T21:22:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095399063000000006 2024-09-17T21:22:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617437+0000", 2024-09-17T21:22:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3931999999999998e-05 2024-09-17T21:22:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617601+0000", 2024-09-17T21:22:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016407300000000001 2024-09-17T21:22:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5629.0:2384 3.1c 3:38c1263e:::rbd_data.1152d37183c3.000000000000000b:head [read 11780096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-17T21:22:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:22:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 105.691519165, 2024-09-17T21:22:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0246185320000001, 2024-09-17T21:22:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5629", 2024-09-17T21:22:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3798711329", 2024-09-17T21:22:21.209 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2384 2024-09-17T21:22:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:22:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:22:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424780+0000", 2024-09-17T21:22:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.13e-06 2024-09-17T21:22:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424785+0000", 2024-09-17T21:22:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4739999999999996e-06 2024-09-17T21:22:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424798+0000", 2024-09-17T21:22:21.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2704000000000001e-05 2024-09-17T21:22:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424806+0000", 2024-09-17T21:22:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9899999999999997e-06 2024-09-17T21:22:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424868+0000", 2024-09-17T21:22:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999997e-05 2024-09-17T21:22:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:22:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424875+0000", 2024-09-17T21:22:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0870000000000001e-06 2024-09-17T21:22:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428897+0000", 2024-09-17T21:22:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040226080000000004 2024-09-17T21:22:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428913+0000", 2024-09-17T21:22:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:22:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.448963+0000", 2024-09-17T21:22:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0200498659999999 2024-09-17T21:22:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449158+0000", 2024-09-17T21:22:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019581000000000001 2024-09-17T21:22:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449397+0000", 2024-09-17T21:22:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023871 2024-09-17T21:22:21.214 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:26043 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_remove_image in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-17T21:22:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:22:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.716988045000001, 2024-09-17T21:22:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.019978351, 2024-09-17T21:22:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:21.215 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 26043 2024-09-17T21:22:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:22:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:22:21.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399314+0000", 2024-09-17T21:22:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.963e-06 2024-09-17T21:22:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399330+0000", 2024-09-17T21:22:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6684000000000002e-05 2024-09-17T21:22:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399372+0000", 2024-09-17T21:22:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1819000000000001e-05 2024-09-17T21:22:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399396+0000", 2024-09-17T21:22:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.406e-05 2024-09-17T21:22:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399571+0000", 2024-09-17T21:22:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017444800000000001 2024-09-17T21:22:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399595+0000", 2024-09-17T21:22:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4709999999999999e-05 2024-09-17T21:22:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.412735+0000", 2024-09-17T21:22:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0131394869999999 2024-09-17T21:22:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.413090+0000", 2024-09-17T21:22:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000355006 2024-09-17T21:22:21.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:22:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.413659+0000", 2024-09-17T21:22:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056922699999999995 2024-09-17T21:22:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.418940+0000", 2024-09-17T21:22:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052814109999999997 2024-09-17T21:22:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.418992+0000", 2024-09-17T21:22:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1646e-05 2024-09-17T21:22:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.419168+0000", 2024-09-17T21:22:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017573400000000001 2024-09-17T21:22:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.419288+0000", 2024-09-17T21:22:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012015600000000001 2024-09-17T21:22:21.222 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.222 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.222 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.222 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.222 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.222 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:21.223 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:22:21.224 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:21.222+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:21.325 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:21.322+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:21.339 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-17T21:22:21.351 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:21.352 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:21.352 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:22:21.352 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:21.370 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:22:21.425 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:21.422+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:21.496 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:21.497 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:21.497 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:22:21.497 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:22:21.497 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:21.512 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:22:21.526 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:21.522+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:21.626 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:21.622+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:21.646 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:21.646 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:22:21.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:22:21.646 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:22:21.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.646 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:22:21.647 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:22:21.647 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 240.03783807299999, 2024-09-17T21:22:21.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:22:21.647 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.647 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.647 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:21.647 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:22:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:22:21.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:22:21.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:22:21.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:22:21.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:22:21.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:22:21.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:22:21.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:22:21.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:22:21.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:22:21.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:22:21.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:22:21.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:22:21.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:21.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:22:21.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:22:21.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:22:21.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:22:21.654 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.654 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.654 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.655 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:22:21.655 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:22:21.655 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 240.02789143699999, 2024-09-17T21:22:21.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:22:21.655 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.655 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.655 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:21.656 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:22:21.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:22:21.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:22:21.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:22:21.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:22:21.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:22:21.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:22:21.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:22:21.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:22:21.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:22:21.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:22:21.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:22:21.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:22:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:22:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:22:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:22:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:22:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:22:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:22:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:22:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 238.784740597, 2024-09-17T21:22:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:22:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:22:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:22:21.663 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:22:21.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.663 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:22:21.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:22:21.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:22:21.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:22:21.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:22:21.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:22:21.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:22:21.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:22:21.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:22:21.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:22:21.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:22:21.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:22:21.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:22:21.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:22:21.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:22:21.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:22:21.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:22:21.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:21.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:22:21.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:22:21.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:21.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:22:21.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:22:21.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:21.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:22:21.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:22:21.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:22:21.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:22:21.667 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.667 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.667 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.667 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:22:21.667 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:22:21.667 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 238.77509511100001, 2024-09-17T21:22:21.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:22:21.668 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.668 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.668 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:21.668 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.668 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:22:21.668 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:22:21.668 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:22:21.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.668 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:22:21.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:22:21.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:22:21.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:22:21.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:22:21.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:22:21.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:22:21.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:22:21.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:22:21.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:22:21.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:22:21.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:22:21.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:22:21.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:22:21.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:22:21.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:22:21.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:22:21.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:21.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:22:21.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:22:21.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:21.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:22:21.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:22:21.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:21.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:22:21.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:22:21.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:22:21.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:22:21.673 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.673 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.673 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.673 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:22:21.673 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:22:21.674 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 225.406159078, 2024-09-17T21:22:21.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:22:21.674 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.674 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.674 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.674 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.674 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:22:21.674 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:22:21.674 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:22:21.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.674 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:22:21.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:22:21.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:22:21.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:22:21.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:22:21.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:22:21.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:22:21.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:22:21.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:22:21.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:22:21.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:22:21.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:22:21.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:22:21.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:22:21.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:22:21.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:22:21.677 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.678 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.678 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.678 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:21.678 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:22:21.678 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 207.204474516, 2024-09-17T21:22:21.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:22:21.678 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.678 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.678 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.678 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.678 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:21.678 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:21.678 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:22:21.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.679 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:22:21.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:22:21.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:22:21.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:22:21.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:22:21.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:22:21.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:22:21.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:22:21.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:22:21.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:22:21.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:22:21.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:22:21.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:22:21.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:22:21.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:22:21.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:22:21.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.682 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.682 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:21.682 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:22:21.682 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 207.20397993899999, 2024-09-17T21:22:21.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:22:21.683 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.683 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.683 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:21.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:21.683 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:22:21.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.683 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:22:21.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:22:21.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:22:21.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:22:21.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:22:21.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:22:21.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:22:21.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:22:21.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:22:21.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:22:21.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:22:21.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:22:21.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:22:21.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:22:21.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:22:21.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:22:21.686 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.686 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.686 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.687 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:21.687 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:22:21.687 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 207.20392455800001, 2024-09-17T21:22:21.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:22:21.687 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.687 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.687 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.687 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.687 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:21.687 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:21.687 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:22:21.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.688 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:22:21.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:22:21.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:22:21.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:22:21.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:22:21.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:22:21.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:22:21.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:22:21.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:22:21.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:22:21.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:22:21.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:22:21.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:22:21.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:22:21.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:22:21.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:22:21.691 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.691 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.691 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.691 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:21.691 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:22:21.691 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 207.20386827999999, 2024-09-17T21:22:21.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:22:21.691 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.692 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.692 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:21.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:21.692 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:22:21.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.692 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:22:21.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:22:21.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:22:21.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:22:21.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:22:21.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:22:21.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:22:21.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:22:21.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:22:21.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:22:21.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:22:21.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:22:21.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:22:21.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:22:21.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:22:21.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:22:21.695 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.695 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.696 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.696 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:21.696 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:22:21.696 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 207.202918641, 2024-09-17T21:22:21.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:22:21.696 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.696 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.696 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.696 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.696 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:21.697 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:21.697 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:22:21.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.697 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:22:21.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:22:21.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:22:21.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:22:21.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:22:21.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:22:21.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:22:21.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:22:21.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:22:21.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:22:21.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:22:21.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:22:21.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:22:21.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:22:21.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:22:21.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:22:21.700 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.700 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.700 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.701 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:21.701 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:22:21.701 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 207.20284002099999, 2024-09-17T21:22:21.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:22:21.701 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.701 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.701 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.701 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.701 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:21.701 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:21.701 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:22:21.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.701 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:22:21.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:22:21.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:22:21.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:22:21.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:22:21.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:22:21.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:22:21.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:22:21.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:22:21.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:22:21.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:22:21.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:22:21.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:22:21.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:22:21.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:22:21.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:22:21.705 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.705 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.705 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.705 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:21.705 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:22:21.705 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 207.202727082, 2024-09-17T21:22:21.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:22:21.705 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.705 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.705 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.706 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.706 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:21.706 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:21.706 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:22:21.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.706 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:22:21.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:22:21.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:22:21.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:22:21.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:22:21.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:22:21.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:22:21.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:22:21.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:22:21.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:22:21.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:22:21.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:22:21.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:22:21.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:22:21.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:22:21.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:22:21.709 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.709 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.709 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.709 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:21.709 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:22:21.710 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 207.20206602799999, 2024-09-17T21:22:21.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:22:21.710 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.710 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.710 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.710 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.710 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:21.710 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:21.710 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:22:21.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.710 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:22:21.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:22:21.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:22:21.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:22:21.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:22:21.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:22:21.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:22:21.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:22:21.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:22:21.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:22:21.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:22:21.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:22:21.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:22:21.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:22:21.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:22:21.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:22:21.714 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.714 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.714 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.714 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:21.714 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:22:21.714 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 207.202001217, 2024-09-17T21:22:21.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:22:21.714 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.714 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.714 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.714 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.714 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:21.714 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:21.715 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:22:21.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.715 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:22:21.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:22:21.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:22:21.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:22:21.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:22:21.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:22:21.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:22:21.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:22:21.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:22:21.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:22:21.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:22:21.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:22:21.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:22:21.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:22:21.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:22:21.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:22:21.719 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.719 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.719 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.719 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:21.719 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:22:21.719 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 207.20194024, 2024-09-17T21:22:21.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:22:21.719 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.719 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.719 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:21.719 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:21.719 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:21.720 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:21.720 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:22:21.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.720 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:22:21.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:22:21.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:22:21.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:22:21.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:22:21.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:22:21.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:22:21.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:22:21.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:22:21.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:22:21.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:22:21.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:22:21.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:22:21.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:22:21.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:22:21.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:22:21.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.723 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.723 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:22:21.723 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:22:21.723 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 152.304369055, 2024-09-17T21:22:21.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:22:21.724 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.724 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.724 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:21.724 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:22:21.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:22:21.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:22:21.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:22:21.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:22:21.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:22:21.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:22:21.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:22:21.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:22:21.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:22:21.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:22:21.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:22:21.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:22:21.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:22:21.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:21.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:22:21.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:22:21.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:22:21.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:22:21.727 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.727 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.728 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.728 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:22:21.728 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:22:21.728 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 152.095710661, 2024-09-17T21:22:21.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:22:21.728 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.728 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.728 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:21.728 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:22:21.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:22:21.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:22:21.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:22:21.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:22:21.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:22:21.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:22:21.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:22:21.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:22:21.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:22:21.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:22:21.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:22:21.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:22:21.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:22:21.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:21.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:22:21.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:22:21.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:22:21.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:22:21.732 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.732 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.732 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.732 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:9547 3.2 e147/132 3:42b518d9:::rbd_data.1152f056b73f.0000000000000001:head v 147'513, mlcod=142'511)", 2024-09-17T21:22:21.732 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:22:21.732 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 79.170737539000001, 2024-09-17T21:22:21.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.031059566, 2024-09-17T21:22:21.733 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.733 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.733 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:21.733 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:22:21.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:22:21.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471293+0000", 2024-09-17T21:22:21.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2029999999999999e-06 2024-09-17T21:22:21.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480606+0000", 2024-09-17T21:22:21.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093123839999999999 2024-09-17T21:22:21.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480653+0000", 2024-09-17T21:22:21.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7852000000000002e-05 2024-09-17T21:22:21.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480673+0000", 2024-09-17T21:22:21.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0013e-05 2024-09-17T21:22:21.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480862+0000", 2024-09-17T21:22:21.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018833599999999999 2024-09-17T21:22:21.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480906+0000", 2024-09-17T21:22:21.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4162000000000002e-05 2024-09-17T21:22:21.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:21.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502259+0000", 2024-09-17T21:22:21.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021353154999999999 2024-09-17T21:22:21.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502351+0000", 2024-09-17T21:22:21.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1460999999999997e-05 2024-09-17T21:22:21.736 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.736 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.736 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.737 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:15234 3.1 e147/134 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head v 147'917, mlcod=145'914)", 2024-09-17T21:22:21.737 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:22:21.737 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 78.538076920999998, 2024-09-17T21:22:21.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045303695999999997, 2024-09-17T21:22:21.737 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.737 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.737 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:21.737 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:22:21.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:22:21.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103954+0000", 2024-09-17T21:22:21.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2259999999999999e-06 2024-09-17T21:22:21.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119105+0000", 2024-09-17T21:22:21.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015150871 2024-09-17T21:22:21.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119141+0000", 2024-09-17T21:22:21.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6239000000000003e-05 2024-09-17T21:22:21.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119159+0000", 2024-09-17T21:22:21.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7946999999999999e-05 2024-09-17T21:22:21.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119344+0000", 2024-09-17T21:22:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018524599999999999 2024-09-17T21:22:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119399+0000", 2024-09-17T21:22:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4922e-05 2024-09-17T21:22:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149165+0000", 2024-09-17T21:22:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029765697000000001 2024-09-17T21:22:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149255+0000", 2024-09-17T21:22:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0548000000000001e-05 2024-09-17T21:22:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:19859 3.8 e147/133 3:12f72409:::rbd_data.1152f056b73f.0000000000000003:head v 147'482, mlcod=145'479)", 2024-09-17T21:22:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:22:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 78.129185853999999, 2024-09-17T21:22:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033476895999999999, 2024-09-17T21:22:21.742 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:21.742 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:21.742 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:21.742 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:21.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:21.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:22:21.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:21.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:22:21.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:21.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:21.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512845+0000", 2024-09-17T21:22:21.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.886e-06 2024-09-17T21:22:21.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:21.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524104+0000", 2024-09-17T21:22:21.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011259668 2024-09-17T21:22:21.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:21.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524148+0000", 2024-09-17T21:22:21.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3386000000000001e-05 2024-09-17T21:22:21.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:21.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524172+0000", 2024-09-17T21:22:21.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4762000000000001e-05 2024-09-17T21:22:21.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:21.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524379+0000", 2024-09-17T21:22:21.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020671599999999999 2024-09-17T21:22:21.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:21.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524421+0000", 2024-09-17T21:22:21.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2182999999999999e-05 2024-09-17T21:22:21.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:21.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546216+0000", 2024-09-17T21:22:21.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021795101000000001 2024-09-17T21:22:21.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:21.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:21.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:21.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546320+0000", 2024-09-17T21:22:21.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103194 2024-09-17T21:22:21.745 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.745 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.745 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.746 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:21.746 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:21.746 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:21.747 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:21.745+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:21.747 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:22:21.848 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:21.846+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:21.879 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:21.879 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:22:21.879 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:22:21.879 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:21.893 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:22:21.981 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:21.978+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:22.050 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:22.046+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:22.054 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:22.054 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:22:22.054 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:22:22.054 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:22:22.054 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:22.063 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:22:22.069 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:22:22.151 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:22.149+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:22.206 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:22:22.206 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:22:22.223 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:22:22.224 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:22:22.224 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T21:22:22.242 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:22.242 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:22.245 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:22:22.245 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:22:22.251 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:22.246+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:22.256 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:22:22.267 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:22:22.267 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:22:22.288 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:22:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:22:22.292 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:22:22.293 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:22:22.352 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:22.349+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:22.393 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:22.393 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:22.408 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:22:22.453 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:22.449+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:22.537 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:22.537 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:22.553 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:22:22.554 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:22.550+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:22.655 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:22.650+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:22.675 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:22.675 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:22.689 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:22:22.757 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:22.753+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:22.821 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:22.822 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:22.836 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:22:22.879 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:22.877+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:22.958 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:22.953+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:22.998 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:22.998 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:23.014 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:22:23.099 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-09-17T21:22:23.097+0000 7f2b065e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25322) UID: 0 2024-09-17T21:22:23.160 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:23.154+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:23.177 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:23.177 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:23.192 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:22:23.261 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:23.258+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:23.359 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:23.360 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:23.362 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:23.358+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:23.375 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:22:23.379 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:22:23.463 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:23.461+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:23.512 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:23.512 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:23.530 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:22:23.563 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:23.558+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:23.664 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:23.658+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:23.691 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:23.691 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:23.706 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:22:23.765 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:23.760+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:23.822 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:23.822 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:23.836 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:22:23.866 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:23.862+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:23.968 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:23.962+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:24.001 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:24.001 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:24.015 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:22:24.068 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:24.062+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:24.134 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:24.134 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:24.148 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:22:24.193 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T21:22:24.190+0000 7f128c189640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25230) UID: 0 2024-09-17T21:22:24.270 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:24.266+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:24.278 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:24.278 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:24.292 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:22:24.294 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:22:24.365 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-17T21:22:24.373 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:24.368+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:24.469 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-17T21:22:24.469 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:22:24.472 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T21:22:24.470+0000 7f8566f94640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25228) UID: 0 2024-09-17T21:22:24.486 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:22:24.500 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-17T21:22:24.501 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:22:24.518 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-17T21:22:24.518 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:22:24.518 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T21:22:24.541 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-17T21:22:24.541 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:22:24.563 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-17T21:22:24.563 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:22:24.574 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-17T21:22:24.568+0000 7f4babbf4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25318) UID: 0 2024-09-17T21:22:24.585 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-17T21:22:24.585 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:22:24.586 INFO:teuthology.orchestra.run.smithi184.stdout:All operations completed A-OK! 2024-09-17T21:22:24.589 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:22:24.589 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:22:24.619 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:24.620 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:22:24.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:22:24.620 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:22:24.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.620 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4871.0:6102 3.1 3.e0923501 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T21:22:24.620 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:22:24.620 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 188.10717762900001, 2024-09-17T21:22:24.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76714069500000004, 2024-09-17T21:22:24.620 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:24.620 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:24.620 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:24.621 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:24.621 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4871", 2024-09-17T21:22:24.621 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/490652803", 2024-09-17T21:22:24.621 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 6102 2024-09-17T21:22:24.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.621 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:24.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:24.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:22:24.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:24.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510189+0000", 2024-09-17T21:22:24.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:24.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510192+0000", 2024-09-17T21:22:24.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3439999999999999e-06 2024-09-17T21:22:24.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:24.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510198+0000", 2024-09-17T21:22:24.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:22:24.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:24.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510209+0000", 2024-09-17T21:22:24.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0838e-05 2024-09-17T21:22:24.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:24.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510220+0000", 2024-09-17T21:22:24.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0964999999999999e-05 2024-09-17T21:22:24.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510298+0000", 2024-09-17T21:22:24.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7749000000000004e-05 2024-09-17T21:22:24.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:24.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:16.510310+0000", 2024-09-17T21:22:24.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2738999999999999e-05 2024-09-17T21:22:24.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277300+0000", 2024-09-17T21:22:24.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.76698927100000003 2024-09-17T21:22:24.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:24.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:17.277330+0000", 2024-09-17T21:22:24.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0278999999999999e-05 2024-09-17T21:22:24.625 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.625 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:24.625 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.625 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2597 3.1f 3:fb0e9615:::rbd_data.1152a8ae2019.000000000000000c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:24.625 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:22:24.625 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 163.778741201, 2024-09-17T21:22:24.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097064994000000002, 2024-09-17T21:22:24.625 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:24.625 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:24.625 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:24.626 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:24.626 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:24.626 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:24.626 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2597 2024-09-17T21:22:24.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.626 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:24.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:24.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:22:24.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:24.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838626+0000", 2024-09-17T21:22:24.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:24.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838628+0000", 2024-09-17T21:22:24.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.187e-06 2024-09-17T21:22:24.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:24.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838654+0000", 2024-09-17T21:22:24.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6406000000000001e-05 2024-09-17T21:22:24.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:24.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838696+0000", 2024-09-17T21:22:24.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1226000000000001e-05 2024-09-17T21:22:24.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:24.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838715+0000", 2024-09-17T21:22:24.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9646000000000001e-05 2024-09-17T21:22:24.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.838937+0000", 2024-09-17T21:22:24.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022163 2024-09-17T21:22:24.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:24.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.839284+0000", 2024-09-17T21:22:24.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034753799999999999 2024-09-17T21:22:24.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:24.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935691+0000", 2024-09-17T21:22:24.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.096406360999999996 2024-09-17T21:22:24.629 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.629 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:24.630 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.630 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2641 3.12 3:488a729f:::rbd_data.1152a8ae2019.0000000000000022:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:24.630 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:22:24.630 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 163.77528442799999, 2024-09-17T21:22:24.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098280953000000004, 2024-09-17T21:22:24.630 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:24.630 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:24.630 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:24.630 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:24.630 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:24.630 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:24.630 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2641 2024-09-17T21:22:24.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.630 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:24.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:24.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:22:24.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:24.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842082+0000", 2024-09-17T21:22:24.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:24.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842083+0000", 2024-09-17T21:22:24.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5300000000000002e-07 2024-09-17T21:22:24.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:24.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842088+0000", 2024-09-17T21:22:24.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:22:24.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:24.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842105+0000", 2024-09-17T21:22:24.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6773e-05 2024-09-17T21:22:24.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:24.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842119+0000", 2024-09-17T21:22:24.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3913e-05 2024-09-17T21:22:24.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842320+0000", 2024-09-17T21:22:24.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020052700000000001 2024-09-17T21:22:24.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:24.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842845+0000", 2024-09-17T21:22:24.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052508600000000002 2024-09-17T21:22:24.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:24.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940363+0000", 2024-09-17T21:22:24.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097518795000000005 2024-09-17T21:22:24.634 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.634 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:24.634 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.634 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2674 3.12 3:4c87e3ac:::rbd_data.1152a8ae2019.0000000000000033:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:24.634 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:22:24.635 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 163.774921791, 2024-09-17T21:22:24.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098025125000000005, 2024-09-17T21:22:24.635 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:24.635 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:24.635 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:24.635 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:24.635 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:24.635 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:24.635 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2674 2024-09-17T21:22:24.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.635 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:24.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:24.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:22:24.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:24.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842445+0000", 2024-09-17T21:22:24.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:24.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842447+0000", 2024-09-17T21:22:24.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3999999999999999e-06 2024-09-17T21:22:24.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:24.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842453+0000", 2024-09-17T21:22:24.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:22:24.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:24.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842472+0000", 2024-09-17T21:22:24.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9006000000000001e-05 2024-09-17T21:22:24.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:24.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.842487+0000", 2024-09-17T21:22:24.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4866e-05 2024-09-17T21:22:24.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843594+0000", 2024-09-17T21:22:24.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011073249999999999 2024-09-17T21:22:24.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:24.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843668+0000", 2024-09-17T21:22:24.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3188000000000005e-05 2024-09-17T21:22:24.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:24.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.843893+0000", 2024-09-17T21:22:24.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022503399999999999 2024-09-17T21:22:24.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:24.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.850439+0000", 2024-09-17T21:22:24.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065467880000000004 2024-09-17T21:22:24.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:24.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940409+0000", 2024-09-17T21:22:24.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089969511000000002 2024-09-17T21:22:24.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:24.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940444+0000", 2024-09-17T21:22:24.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5345999999999997e-05 2024-09-17T21:22:24.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:24.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940470+0000", 2024-09-17T21:22:24.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5964000000000001e-05 2024-09-17T21:22:24.640 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.640 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:24.640 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.640 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2707 3.1f 3:f805e65e:::rbd_data.1152a8ae2019.0000000000000049:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:24.640 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:22:24.640 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 163.76987513, 2024-09-17T21:22:24.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088327300999999997, 2024-09-17T21:22:24.640 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:24.641 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:24.641 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:24.641 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:24.641 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:24.641 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:24.641 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2707 2024-09-17T21:22:24.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.641 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:24.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:24.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:22:24.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:24.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847492+0000", 2024-09-17T21:22:24.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:24.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847494+0000", 2024-09-17T21:22:24.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1569999999999998e-06 2024-09-17T21:22:24.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:24.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847504+0000", 2024-09-17T21:22:24.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7559999999999994e-06 2024-09-17T21:22:24.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:24.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847526+0000", 2024-09-17T21:22:24.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1976000000000001e-05 2024-09-17T21:22:24.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:24.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.847541+0000", 2024-09-17T21:22:24.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5277e-05 2024-09-17T21:22:24.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856142+0000", 2024-09-17T21:22:24.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086006710000000007 2024-09-17T21:22:24.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:24.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856435+0000", 2024-09-17T21:22:24.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029301499999999999 2024-09-17T21:22:24.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:24.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.935819+0000", 2024-09-17T21:22:24.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079384448999999996 2024-09-17T21:22:24.644 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.644 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:24.644 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.645 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2730 3.12 3:48b3422a:::rbd_data.1152a8ae2019.0000000000000057:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:24.645 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:22:24.645 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 163.760682231, 2024-09-17T21:22:24.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083879975999999995, 2024-09-17T21:22:24.645 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:24.645 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:24.645 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:24.645 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:24.645 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:24.645 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:24.645 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2730 2024-09-17T21:22:24.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.646 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:24.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:24.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:22:24.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:24.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856685+0000", 2024-09-17T21:22:24.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:24.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856687+0000", 2024-09-17T21:22:24.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:22:24.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:24.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856697+0000", 2024-09-17T21:22:24.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0207000000000001e-05 2024-09-17T21:22:24.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:24.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856808+0000", 2024-09-17T21:22:24.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011071200000000001 2024-09-17T21:22:24.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:24.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856828+0000", 2024-09-17T21:22:24.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0219000000000001e-05 2024-09-17T21:22:24.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.648 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.856907+0000", 2024-09-17T21:22:24.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9661999999999997e-05 2024-09-17T21:22:24.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.648 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:24.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.857019+0000", 2024-09-17T21:22:24.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011148499999999999 2024-09-17T21:22:24.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.649 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:24.649 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940565+0000", 2024-09-17T21:22:24.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083545791999999994 2024-09-17T21:22:24.649 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.649 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:24.649 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.649 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:2743 3.12 3:4d036e62:::rbd_data.1152a8ae2019.000000000000005f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:24.649 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:22:24.649 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 163.75673673, 2024-09-17T21:22:24.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080059962999999998, 2024-09-17T21:22:24.650 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:24.650 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:24.650 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:24.650 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:24.650 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:24.650 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:24.650 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2743 2024-09-17T21:22:24.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.650 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:24.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:24.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:22:24.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:24.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860630+0000", 2024-09-17T21:22:24.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:24.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860631+0000", 2024-09-17T21:22:24.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:22:24.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:24.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860636+0000", 2024-09-17T21:22:24.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5279999999999997e-06 2024-09-17T21:22:24.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:24.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860649+0000", 2024-09-17T21:22:24.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-17T21:22:24.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:24.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860655+0000", 2024-09-17T21:22:24.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:22:24.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.653 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.860731+0000", 2024-09-17T21:22:24.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5920000000000005e-05 2024-09-17T21:22:24.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.653 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:24.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.861070+0000", 2024-09-17T21:22:24.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000338535 2024-09-17T21:22:24.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.653 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:24.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940690+0000", 2024-09-17T21:22:24.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.079620503999999995 2024-09-17T21:22:24.653 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.654 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:24.654 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.654 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3378 3.12 3:4832a51b:::rbd_data.1152a8ae2019.000000000000006c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:24.654 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:22:24.654 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 163.72837287999999, 2024-09-17T21:22:24.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051812516000000003, 2024-09-17T21:22:24.654 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:24.654 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:24.654 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:24.654 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:24.654 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:24.654 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:24.655 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3378 2024-09-17T21:22:24.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.655 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:24.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:24.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:22:24.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:24.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888994+0000", 2024-09-17T21:22:24.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:24.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.888995+0000", 2024-09-17T21:22:24.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:22:24.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:24.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889006+0000", 2024-09-17T21:22:24.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1012999999999999e-05 2024-09-17T21:22:24.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:24.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889032+0000", 2024-09-17T21:22:24.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6441e-05 2024-09-17T21:22:24.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:24.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889040+0000", 2024-09-17T21:22:24.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0639999999999994e-06 2024-09-17T21:22:24.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889278+0000", 2024-09-17T21:22:24.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023785899999999999 2024-09-17T21:22:24.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:24.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889408+0000", 2024-09-17T21:22:24.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129219 2024-09-17T21:22:24.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:24.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889723+0000", 2024-09-17T21:22:24.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000315264 2024-09-17T21:22:24.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:24.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.891972+0000", 2024-09-17T21:22:24.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002249633 2024-09-17T21:22:24.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:24.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940733+0000", 2024-09-17T21:22:24.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048760626000000001 2024-09-17T21:22:24.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:24.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940785+0000", 2024-09-17T21:22:24.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1906000000000001e-05 2024-09-17T21:22:24.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:24.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940807+0000", 2024-09-17T21:22:24.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1554999999999999e-05 2024-09-17T21:22:24.659 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.659 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:24.660 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.660 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5472.0:3379 3.12 3:4e83fd71:::rbd_data.1152a8ae2019.000000000000006d:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:24.660 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:22:24.660 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 163.72828867300001, 2024-09-17T21:22:24.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051804144000000003, 2024-09-17T21:22:24.660 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:24.660 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:24.660 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:24.660 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:24.660 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5472", 2024-09-17T21:22:24.660 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:24.661 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3379 2024-09-17T21:22:24.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.661 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:24.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:24.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:22:24.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:24.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889078+0000", 2024-09-17T21:22:24.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:24.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889079+0000", 2024-09-17T21:22:24.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1800000000000004e-07 2024-09-17T21:22:24.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:24.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889083+0000", 2024-09-17T21:22:24.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:22:24.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:24.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889094+0000", 2024-09-17T21:22:24.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1562999999999999e-05 2024-09-17T21:22:24.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:24.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.889102+0000", 2024-09-17T21:22:24.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.401e-06 2024-09-17T21:22:24.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890530+0000", 2024-09-17T21:22:24.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014279259999999999 2024-09-17T21:22:24.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:24.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890609+0000", 2024-09-17T21:22:24.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9566000000000004e-05 2024-09-17T21:22:24.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:24.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.890789+0000", 2024-09-17T21:22:24.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017949700000000001 2024-09-17T21:22:24.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:24.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.893929+0000", 2024-09-17T21:22:24.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003140538 2024-09-17T21:22:24.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:24.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940834+0000", 2024-09-17T21:22:24.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046904811999999997 2024-09-17T21:22:24.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:24.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940860+0000", 2024-09-17T21:22:24.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5341999999999999e-05 2024-09-17T21:22:24.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:24.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:40.940882+0000", 2024-09-17T21:22:24.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2878999999999999e-05 2024-09-17T21:22:24.666 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.666 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:24.666 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.666 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5541.0:2644 3.1e 3:79615585:::rbd_data.115266d0b9f2.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2475520~14848,write 3106304~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:22:24.666 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:22:24.666 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 155.10566026999999, 2024-09-17T21:22:24.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050733484000000002, 2024-09-17T21:22:24.666 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:24.666 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:24.666 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:24.666 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:24.666 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5541", 2024-09-17T21:22:24.667 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1482032604", 2024-09-17T21:22:24.667 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2644 2024-09-17T21:22:24.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.667 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:24.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:24.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:22:24.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:24.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511707+0000", 2024-09-17T21:22:24.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:24.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.511708+0000", 2024-09-17T21:22:24.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-17T21:22:24.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:24.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522196+0000", 2024-09-17T21:22:24.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010487712999999999 2024-09-17T21:22:24.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:24.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522223+0000", 2024-09-17T21:22:24.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7064999999999999e-05 2024-09-17T21:22:24.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:24.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522236+0000", 2024-09-17T21:22:24.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2931e-05 2024-09-17T21:22:24.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522348+0000", 2024-09-17T21:22:24.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111906 2024-09-17T21:22:24.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:24.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.522554+0000", 2024-09-17T21:22:24.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020632699999999999 2024-09-17T21:22:24.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4", 2024-09-17T21:22:24.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.523185+0000", 2024-09-17T21:22:24.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063081400000000005 2024-09-17T21:22:24.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:24.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.547011+0000", 2024-09-17T21:22:24.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023825570000000001 2024-09-17T21:22:24.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:24.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562268+0000", 2024-09-17T21:22:24.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015257514 2024-09-17T21:22:24.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:24.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562367+0000", 2024-09-17T21:22:24.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8678000000000006e-05 2024-09-17T21:22:24.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:24.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:19:49.562440+0000", 2024-09-17T21:22:24.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3337000000000006e-05 2024-09-17T21:22:24.671 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.671 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:24.671 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.672 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:231 3.1 3:856fed78:::rbd_data.115220dd393e.000000000000000b:head [read 0~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:22:24.672 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:22:24.672 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 126.662905001, 2024-09-17T21:22:24.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34254251699999999, 2024-09-17T21:22:24.672 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:24.672 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:24.672 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:24.672 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:24.672 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:24.672 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:24.672 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 231 2024-09-17T21:22:24.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.673 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:24.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:24.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:22:24.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:24.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954462+0000", 2024-09-17T21:22:24.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:24.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954464+0000", 2024-09-17T21:22:24.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:22:24.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:24.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954475+0000", 2024-09-17T21:22:24.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1146e-05 2024-09-17T21:22:24.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:24.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954506+0000", 2024-09-17T21:22:24.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0292999999999999e-05 2024-09-17T21:22:24.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:24.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954524+0000", 2024-09-17T21:22:24.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8430000000000001e-05 2024-09-17T21:22:24.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954661+0000", 2024-09-17T21:22:24.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136403 2024-09-17T21:22:24.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:24.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.954680+0000", 2024-09-17T21:22:24.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9743000000000001e-05 2024-09-17T21:22:24.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.296896+0000", 2024-09-17T21:22:24.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34221550499999998 2024-09-17T21:22:24.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:24.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297004+0000", 2024-09-17T21:22:24.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010865399999999999 2024-09-17T21:22:24.676 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.676 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:24.676 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.676 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:56 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:22:24.677 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:22:24.677 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 126.507271304, 2024-09-17T21:22:24.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18718773899999999, 2024-09-17T21:22:24.677 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:24.677 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:24.677 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:24.677 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:24.677 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:22:24.677 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:22:24.677 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 56 2024-09-17T21:22:24.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.677 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:24.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:24.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:22:24.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:24.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110096+0000", 2024-09-17T21:22:24.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:24.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110098+0000", 2024-09-17T21:22:24.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:22:24.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:24.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110111+0000", 2024-09-17T21:22:24.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2289999999999999e-05 2024-09-17T21:22:24.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:24.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110143+0000", 2024-09-17T21:22:24.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1979e-05 2024-09-17T21:22:24.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:24.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110161+0000", 2024-09-17T21:22:24.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7839e-05 2024-09-17T21:22:24.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110319+0000", 2024-09-17T21:22:24.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015835400000000001 2024-09-17T21:22:24.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:24.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.110353+0000", 2024-09-17T21:22:24.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3849000000000001e-05 2024-09-17T21:22:24.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297041+0000", 2024-09-17T21:22:24.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18668842299999999 2024-09-17T21:22:24.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:24.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297169+0000", 2024-09-17T21:22:24.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012786199999999999 2024-09-17T21:22:24.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:24.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297283+0000", 2024-09-17T21:22:24.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011430300000000001 2024-09-17T21:22:24.681 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.681 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:24.681 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.682 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5536.0:68 3.1 3:819f8879:::rbd_data.1152b85b6af9.0000000000000007:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-17T21:22:24.682 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:22:24.682 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 126.498730373, 2024-09-17T21:22:24.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17885582999999999, 2024-09-17T21:22:24.682 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:24.682 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:24.682 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:24.682 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:24.682 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5536", 2024-09-17T21:22:24.682 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/202535935", 2024-09-17T21:22:24.682 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 68 2024-09-17T21:22:24.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.683 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:24.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:24.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:22:24.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:24.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118637+0000", 2024-09-17T21:22:24.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:24.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118639+0000", 2024-09-17T21:22:24.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.306e-06 2024-09-17T21:22:24.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:24.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118651+0000", 2024-09-17T21:22:24.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2567e-05 2024-09-17T21:22:24.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:24.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118682+0000", 2024-09-17T21:22:24.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0502e-05 2024-09-17T21:22:24.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:24.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118707+0000", 2024-09-17T21:22:24.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5463e-05 2024-09-17T21:22:24.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118889+0000", 2024-09-17T21:22:24.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018150999999999999 2024-09-17T21:22:24.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:24.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.118910+0000", 2024-09-17T21:22:24.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:22:24.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297313+0000", 2024-09-17T21:22:24.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17840366699999999 2024-09-17T21:22:24.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:24.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297389+0000", 2024-09-17T21:22:24.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-17T21:22:24.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:24.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:18.297492+0000", 2024-09-17T21:22:24.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103019 2024-09-17T21:22:24.687 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.687 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:24.687 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.687 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5890.0:15234 3.1 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8451072~29696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-17T21:22:24.687 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:22:24.687 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 81.524081561000003, 2024-09-17T21:22:24.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.057026052000000001, 2024-09-17T21:22:24.687 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:24.687 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:24.687 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:24.688 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:24.688 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5890", 2024-09-17T21:22:24.688 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3064344947", 2024-09-17T21:22:24.688 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 15234 2024-09-17T21:22:24.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.688 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:24.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:24.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:22:24.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:24.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093285+0000", 2024-09-17T21:22:24.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:24.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.093289+0000", 2024-09-17T21:22:24.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4939999999999999e-06 2024-09-17T21:22:24.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:24.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103074+0000", 2024-09-17T21:22:24.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00978564 2024-09-17T21:22:24.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:24.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103112+0000", 2024-09-17T21:22:24.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7246999999999998e-05 2024-09-17T21:22:24.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:24.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103128+0000", 2024-09-17T21:22:24.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5812999999999999e-05 2024-09-17T21:22:24.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103207+0000", 2024-09-17T21:22:24.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9785999999999998e-05 2024-09-17T21:22:24.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:24.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103315+0000", 2024-09-17T21:22:24.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107396 2024-09-17T21:22:24.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:22:24.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.103957+0000", 2024-09-17T21:22:24.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000642524 2024-09-17T21:22:24.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:24.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.123856+0000", 2024-09-17T21:22:24.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019898376999999998 2024-09-17T21:22:24.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:24.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150190+0000", 2024-09-17T21:22:24.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026334682000000002 2024-09-17T21:22:24.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:24.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150255+0000", 2024-09-17T21:22:24.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4770999999999994e-05 2024-09-17T21:22:24.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:24.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:03.150311+0000", 2024-09-17T21:22:24.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6322e-05 2024-09-17T21:22:24.693 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.693 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:24.693 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.693 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4239 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 424960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:22:24.693 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:22:24.693 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 71.863683903999998, 2024-09-17T21:22:24.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050123167000000003, 2024-09-17T21:22:24.693 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:24.693 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:24.693 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:24.693 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:24.693 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:22:24.694 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:22:24.694 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4239 2024-09-17T21:22:24.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.694 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:24.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:24.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:22:24.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:24.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753683+0000", 2024-09-17T21:22:24.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:24.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753684+0000", 2024-09-17T21:22:24.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.37e-07 2024-09-17T21:22:24.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:24.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753686+0000", 2024-09-17T21:22:24.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.322e-06 2024-09-17T21:22:24.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:24.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753693+0000", 2024-09-17T21:22:24.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0890000000000002e-06 2024-09-17T21:22:24.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:24.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753699+0000", 2024-09-17T21:22:24.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:22:24.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803390+0000", 2024-09-17T21:22:24.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049690506000000002 2024-09-17T21:22:24.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:24.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803569+0000", 2024-09-17T21:22:24.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179784 2024-09-17T21:22:24.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:24.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803806+0000", 2024-09-17T21:22:24.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023683499999999999 2024-09-17T21:22:24.697 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.697 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:24.697 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.698 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4240 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 413696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:22:24.698 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:22:24.698 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 71.863660175999996, 2024-09-17T21:22:24.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050331353000000002, 2024-09-17T21:22:24.698 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:24.698 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:24.698 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:24.698 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:24.698 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:22:24.698 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:22:24.698 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4240 2024-09-17T21:22:24.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.698 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:24.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:24.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:22:24.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:24.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:22:24.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:24.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753707+0000", 2024-09-17T21:22:24.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5700000000000002e-07 2024-09-17T21:22:24.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:24.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753710+0000", 2024-09-17T21:22:24.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.385e-06 2024-09-17T21:22:24.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:24.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753717+0000", 2024-09-17T21:22:24.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.832e-06 2024-09-17T21:22:24.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:24.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753722+0000", 2024-09-17T21:22:24.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8050000000000003e-06 2024-09-17T21:22:24.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803880+0000", 2024-09-17T21:22:24.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050157641000000003 2024-09-17T21:22:24.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:24.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.803945+0000", 2024-09-17T21:22:24.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5357000000000005e-05 2024-09-17T21:22:24.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:24.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804038+0000", 2024-09-17T21:22:24.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2775999999999994e-05 2024-09-17T21:22:24.702 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.702 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:24.702 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.702 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4241 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 415744~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:22:24.702 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:22:24.702 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 71.863637060000002, 2024-09-17T21:22:24.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050578790999999998, 2024-09-17T21:22:24.702 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:24.702 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:24.702 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:24.702 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:24.703 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:22:24.703 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:22:24.703 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4241 2024-09-17T21:22:24.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.703 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:24.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:24.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:22:24.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:24.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:22:24.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:24.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753730+0000", 2024-09-17T21:22:24.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5700000000000002e-07 2024-09-17T21:22:24.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:24.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753733+0000", 2024-09-17T21:22:24.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2570000000000002e-06 2024-09-17T21:22:24.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:24.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753740+0000", 2024-09-17T21:22:24.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.849e-06 2024-09-17T21:22:24.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:24.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753745+0000", 2024-09-17T21:22:24.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7450000000000001e-06 2024-09-17T21:22:24.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804115+0000", 2024-09-17T21:22:24.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050370038999999998 2024-09-17T21:22:24.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:24.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804223+0000", 2024-09-17T21:22:24.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010786 2024-09-17T21:22:24.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:24.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804309+0000", 2024-09-17T21:22:24.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5483999999999997e-05 2024-09-17T21:22:24.706 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.706 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:24.706 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.706 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5965.0:4242 3.6 3:65307e4a:::rbd_data.11524f65d4ff.0000000000000001:29 [read 414720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-17T21:22:24.707 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:22:24.707 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 71.863614171999998, 2024-09-17T21:22:24.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050748148999999999, 2024-09-17T21:22:24.707 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:24.707 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:24.707 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:24.707 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:24.707 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5965", 2024-09-17T21:22:24.707 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2650574883", 2024-09-17T21:22:24.707 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 4242 2024-09-17T21:22:24.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.707 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:24.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:24.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:22:24.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:24.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:22:24.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:24.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753753+0000", 2024-09-17T21:22:24.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6599999999999996e-07 2024-09-17T21:22:24.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:24.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753756+0000", 2024-09-17T21:22:24.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.368e-06 2024-09-17T21:22:24.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:24.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753763+0000", 2024-09-17T21:22:24.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9349999999999997e-06 2024-09-17T21:22:24.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:24.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.753768+0000", 2024-09-17T21:22:24.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7620000000000001e-06 2024-09-17T21:22:24.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804343+0000", 2024-09-17T21:22:24.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050575050000000003 2024-09-17T21:22:24.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:24.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804397+0000", 2024-09-17T21:22:24.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4070000000000002e-05 2024-09-17T21:22:24.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:24.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:12.804501+0000", 2024-09-17T21:22:24.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103398 2024-09-17T21:22:24.711 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.711 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:24.711 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.711 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25642 3.1 3.6a5ad301 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e177)", 2024-09-17T21:22:24.711 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:34.316616+0000", 2024-09-17T21:22:24.711 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.300750589000003, 2024-09-17T21:22:24.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99065658300000003, 2024-09-17T21:22:24.711 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:24.711 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:24.711 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:24.711 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:24.711 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:22:24.712 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:24.712 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25642 2024-09-17T21:22:24.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.712 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:24.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:24.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316616+0000", 2024-09-17T21:22:24.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:24.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316616+0000", 2024-09-17T21:22:24.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:24.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316618+0000", 2024-09-17T21:22:24.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9309999999999998e-06 2024-09-17T21:22:24.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:24.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316624+0000", 2024-09-17T21:22:24.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2789999999999998e-06 2024-09-17T21:22:24.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:24.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316634+0000", 2024-09-17T21:22:24.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0774000000000001e-05 2024-09-17T21:22:24.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:24.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316644+0000", 2024-09-17T21:22:24.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0043e-05 2024-09-17T21:22:24.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316767+0000", 2024-09-17T21:22:24.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012289199999999999 2024-09-17T21:22:24.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:24.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:34.316777+0000", 2024-09-17T21:22:24.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6199999999999994e-06 2024-09-17T21:22:24.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.307241+0000", 2024-09-17T21:22:24.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99046395099999995 2024-09-17T21:22:24.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:24.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:35.307273+0000", 2024-09-17T21:22:24.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2092999999999999e-05 2024-09-17T21:22:24.715 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.716 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:24.716 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.716 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25905 3.6 3:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-17T21:22:24.716 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:39.359572+0000", 2024-09-17T21:22:24.716 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 45.257795307000002, 2024-09-17T21:22:24.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96065702100000006, 2024-09-17T21:22:24.716 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:24.716 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:24.716 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:24.716 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:24.716 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:22:24.716 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:24.717 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25905 2024-09-17T21:22:24.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.717 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:24.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:24.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359572+0000", 2024-09-17T21:22:24.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:24.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359572+0000", 2024-09-17T21:22:24.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:24.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:24.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359573+0000", 2024-09-17T21:22:24.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5370000000000001e-06 2024-09-17T21:22:24.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:24.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359579+0000", 2024-09-17T21:22:24.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8039999999999998e-06 2024-09-17T21:22:24.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:24.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359603+0000", 2024-09-17T21:22:24.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4533999999999999e-05 2024-09-17T21:22:24.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:24.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359613+0000", 2024-09-17T21:22:24.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2820000000000002e-06 2024-09-17T21:22:24.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359686+0000", 2024-09-17T21:22:24.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3301999999999997e-05 2024-09-17T21:22:24.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:24.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:39.359693+0000", 2024-09-17T21:22:24.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3679999999999996e-06 2024-09-17T21:22:24.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.720 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:24.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:40.319999+0000", 2024-09-17T21:22:24.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96030524500000003 2024-09-17T21:22:24.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:24.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:24.720 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:24.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:40.320229+0000", 2024-09-17T21:22:24.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022994899999999999 2024-09-17T21:22:24.720 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.720 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:24.721 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.721 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:24.721 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:24.721 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:24.723 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:22:24.723 DEBUG:teuthology.parallel:result is None 2024-09-17T21:22:24.724 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-17T21:22:24.732 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-17T21:22:24.740 INFO:tasks.thrashosds:joining thrashosds 2024-09-17T21:22:24.741 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-09-17T21:22:24.720+0000 7f2f68e20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25319) UID: 0 2024-09-17T21:22:24.892 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:24.893 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:22:24.893 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:22:24.893 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:24.907 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:22:25.029 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:25.029 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:22:25.029 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-17T21:22:25.029 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-17T21:22:25.029 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:25.043 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:22:25.067 INFO:tasks.thrashosds.thrasher:reviving osd 2024-09-17T21:22:25.067 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2024-09-17T21:22:25.068 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-17T21:22:25.068 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-17T21:22:25.089 INFO:tasks.ceph.osd.2:Started 2024-09-17T21:22:25.090 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:22:25.170 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:25.170 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-17T21:22:25.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-17T21:22:25.170 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T21:22:25.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.170 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4615.0:3706 3.b 3:d693fac5:::rbd_data.11527035abe.000000000000000a:head [read 15235072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T21:22:25.170 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:22:25.170 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 225.358074354, 2024-09-17T21:22:25.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0069495850000001, 2024-09-17T21:22:25.171 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:25.171 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:25.171 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:25.171 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:25.171 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4615", 2024-09-17T21:22:25.171 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3701169283", 2024-09-17T21:22:25.171 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 3706 2024-09-17T21:22:25.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.171 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:25.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:25.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:22:25.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:25.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809781+0000", 2024-09-17T21:22:25.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:25.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809782+0000", 2024-09-17T21:22:25.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:22:25.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:25.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809785+0000", 2024-09-17T21:22:25.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4769999999999999e-06 2024-09-17T21:22:25.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:25.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809792+0000", 2024-09-17T21:22:25.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3629999999999996e-06 2024-09-17T21:22:25.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809797+0000", 2024-09-17T21:22:25.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:22:25.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809854+0000", 2024-09-17T21:22:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:22:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:39.809863+0000", 2024-09-17T21:22:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6479999999999998e-06 2024-09-17T21:22:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816427+0000", 2024-09-17T21:22:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.006563487 2024-09-17T21:22:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816578+0000", 2024-09-17T21:22:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151593 2024-09-17T21:22:25.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:25.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:40.816730+0000", 2024-09-17T21:22:25.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151996 2024-09-17T21:22:25.175 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.175 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:25.175 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.175 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4923.0:534 3.c 3:31b0d4fb:::rbd_data.1152b39bd6b2.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 2031616~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-17T21:22:25.175 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:22:25.175 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 211.802972568, 2024-09-17T21:22:25.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98125005499999995, 2024-09-17T21:22:25.176 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:25.176 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:25.176 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:25.176 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:25.176 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4923", 2024-09-17T21:22:25.176 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:25.176 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 534 2024-09-17T21:22:25.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.176 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:25.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.176 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:25.176 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:22:25.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364882+0000", 2024-09-17T21:22:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364884+0000", 2024-09-17T21:22:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9769999999999999e-06 2024-09-17T21:22:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364933+0000", 2024-09-17T21:22:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8721000000000003e-05 2024-09-17T21:22:25.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:25.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364958+0000", 2024-09-17T21:22:25.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4419000000000001e-05 2024-09-17T21:22:25.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.364971+0000", 2024-09-17T21:22:25.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3869000000000001e-05 2024-09-17T21:22:25.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365078+0000", 2024-09-17T21:22:25.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106142 2024-09-17T21:22:25.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:25.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:53.365094+0000", 2024-09-17T21:22:25.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6402999999999999e-05 2024-09-17T21:22:25.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343670+0000", 2024-09-17T21:22:25.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97857634199999999 2024-09-17T21:22:25.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:25.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.343841+0000", 2024-09-17T21:22:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017082600000000001 2024-09-17T21:22:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5", 2024-09-17T21:22:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.344043+0000", 2024-09-17T21:22:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020164400000000001 2024-09-17T21:22:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345463+0000", 2024-09-17T21:22:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014202279999999999 2024-09-17T21:22:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.345977+0000", 2024-09-17T21:22:25.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051379300000000002 2024-09-17T21:22:25.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:25.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346029+0000", 2024-09-17T21:22:25.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2278999999999999e-05 2024-09-17T21:22:25.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:25.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:18:54.346133+0000", 2024-09-17T21:22:25.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010341200000000001 2024-09-17T21:22:25.181 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.181 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:25.181 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.182 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:793 3.9 3:95f63599:::rbd_data.1152eb799094.0000000000000007:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:22:25.182 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:22:25.182 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 128.73591641300001, 2024-09-17T21:22:25.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88245766400000003, 2024-09-17T21:22:25.182 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:25.182 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:25.182 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:25.182 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:25.182 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:22:25.182 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:22:25.182 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 793 2024-09-17T21:22:25.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.182 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:25.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:25.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:22:25.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:25.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431939+0000", 2024-09-17T21:22:25.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:25.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431941+0000", 2024-09-17T21:22:25.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.864e-06 2024-09-17T21:22:25.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:25.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431953+0000", 2024-09-17T21:22:25.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1486e-05 2024-09-17T21:22:25.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:25.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431984+0000", 2024-09-17T21:22:25.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1075999999999999e-05 2024-09-17T21:22:25.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.431999+0000", 2024-09-17T21:22:25.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.503e-05 2024-09-17T21:22:25.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432095+0000", 2024-09-17T21:22:25.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5698000000000004e-05 2024-09-17T21:22:25.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:25.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432109+0000", 2024-09-17T21:22:25.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4188e-05 2024-09-17T21:22:25.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.309642+0000", 2024-09-17T21:22:25.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87753353000000001 2024-09-17T21:22:25.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:25.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310104+0000", 2024-09-17T21:22:25.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046103600000000001 2024-09-17T21:22:25.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:25.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310630+0000", 2024-09-17T21:22:25.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052682800000000002 2024-09-17T21:22:25.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:25.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313100+0000", 2024-09-17T21:22:25.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024693369999999998 2024-09-17T21:22:25.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:25.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313935+0000", 2024-09-17T21:22:25.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083540100000000005 2024-09-17T21:22:25.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:25.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314086+0000", 2024-09-17T21:22:25.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015053399999999999 2024-09-17T21:22:25.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:25.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314396+0000", 2024-09-17T21:22:25.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031065599999999997 2024-09-17T21:22:25.189 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.189 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:25.189 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.190 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:800 3.19 3:9dc17f12:::rbd_data.1152eb799094.000000000000000e:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:22:25.190 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:22:25.190 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 128.73566916799999, 2024-09-17T21:22:25.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88154663799999999, 2024-09-17T21:22:25.190 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:25.190 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:25.190 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:25.190 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:25.190 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:22:25.190 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:22:25.190 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 800 2024-09-17T21:22:25.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.190 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:25.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:25.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:22:25.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:25.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432186+0000", 2024-09-17T21:22:25.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:25.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432187+0000", 2024-09-17T21:22:25.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4100000000000001e-06 2024-09-17T21:22:25.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:25.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432194+0000", 2024-09-17T21:22:25.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0389999999999999e-06 2024-09-17T21:22:25.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:25.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432216+0000", 2024-09-17T21:22:25.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1614e-05 2024-09-17T21:22:25.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432232+0000", 2024-09-17T21:22:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5739e-05 2024-09-17T21:22:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432405+0000", 2024-09-17T21:22:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017355300000000001 2024-09-17T21:22:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432432+0000", 2024-09-17T21:22:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6979e-05 2024-09-17T21:22:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.307767+0000", 2024-09-17T21:22:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87533472899999998 2024-09-17T21:22:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308192+0000", 2024-09-17T21:22:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042485999999999999 2024-09-17T21:22:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.308727+0000", 2024-09-17T21:22:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00053550400000000001 2024-09-17T21:22:25.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310347+0000", 2024-09-17T21:22:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016198670000000001 2024-09-17T21:22:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313424+0000", 2024-09-17T21:22:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030764640000000001 2024-09-17T21:22:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313510+0000", 2024-09-17T21:22:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5938000000000003e-05 2024-09-17T21:22:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313733+0000", 2024-09-17T21:22:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022294200000000001 2024-09-17T21:22:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5883.0:805 3.19 3:9cab2ed6:::rbd_data.1152eb799094.0000000000000013:head [delete] snapc 22=[22] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:22:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:22:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 128.73546514899999, 2024-09-17T21:22:25.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88274355699999996, 2024-09-17T21:22:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5883", 2024-09-17T21:22:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/2494840312", 2024-09-17T21:22:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 805 2024-09-17T21:22:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:22:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432390+0000", 2024-09-17T21:22:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432391+0000", 2024-09-17T21:22:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3e-06 2024-09-17T21:22:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:25.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432398+0000", 2024-09-17T21:22:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4489999999999996e-06 2024-09-17T21:22:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432415+0000", 2024-09-17T21:22:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7750000000000001e-05 2024-09-17T21:22:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432430+0000", 2024-09-17T21:22:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.449e-05 2024-09-17T21:22:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432635+0000", 2024-09-17T21:22:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000205031 2024-09-17T21:22:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:16.432657+0000", 2024-09-17T21:22:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2064000000000001e-05 2024-09-17T21:22:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310444+0000", 2024-09-17T21:22:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87778699199999999 2024-09-17T21:22:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.310849+0000", 2024-09-17T21:22:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00040497800000000001 2024-09-17T21:22:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-17T21:22:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.311396+0000", 2024-09-17T21:22:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00054729099999999999 2024-09-17T21:22:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.313847+0000", 2024-09-17T21:22:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024504940000000001 2024-09-17T21:22:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-17T21:22:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314879+0000", 2024-09-17T21:22:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001031817 2024-09-17T21:22:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-17T21:22:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.314964+0000", 2024-09-17T21:22:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5685000000000004e-05 2024-09-17T21:22:25.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:17.315133+0000", 2024-09-17T21:22:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000169216 2024-09-17T21:22:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12089 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 60416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:22:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.398240117, 2024-09-17T21:22:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088293233999999998, 2024-09-17T21:22:25.203 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12089 2024-09-17T21:22:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:22:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769615+0000", 2024-09-17T21:22:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769616+0000", 2024-09-17T21:22:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3629999999999999e-06 2024-09-17T21:22:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769620+0000", 2024-09-17T21:22:25.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5140000000000002e-06 2024-09-17T21:22:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769633+0000", 2024-09-17T21:22:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2849e-05 2024-09-17T21:22:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769644+0000", 2024-09-17T21:22:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1471000000000001e-05 2024-09-17T21:22:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857564+0000", 2024-09-17T21:22:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087919445999999998 2024-09-17T21:22:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857716+0000", 2024-09-17T21:22:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000152065 2024-09-17T21:22:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857908+0000", 2024-09-17T21:22:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019252599999999999 2024-09-17T21:22:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12090 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 64512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:22:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.398193732, 2024-09-17T21:22:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088582672000000001, 2024-09-17T21:22:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:25.208 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12090 2024-09-17T21:22:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:22:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769661+0000", 2024-09-17T21:22:25.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769662+0000", 2024-09-17T21:22:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:22:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769683+0000", 2024-09-17T21:22:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0979e-05 2024-09-17T21:22:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769699+0000", 2024-09-17T21:22:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5446000000000001e-05 2024-09-17T21:22:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769708+0000", 2024-09-17T21:22:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1679999999999993e-06 2024-09-17T21:22:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.857976+0000", 2024-09-17T21:22:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088267795999999996 2024-09-17T21:22:25.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858095+0000", 2024-09-17T21:22:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119147 2024-09-17T21:22:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858244+0000", 2024-09-17T21:22:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014917000000000001 2024-09-17T21:22:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12091 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 68608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:22:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.398122348, 2024-09-17T21:22:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088832752000000001, 2024-09-17T21:22:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12091 2024-09-17T21:22:25.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:22:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769733+0000", 2024-09-17T21:22:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769734+0000", 2024-09-17T21:22:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:22:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769739+0000", 2024-09-17T21:22:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8080000000000004e-06 2024-09-17T21:22:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769764+0000", 2024-09-17T21:22:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4839999999999999e-05 2024-09-17T21:22:25.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769777+0000", 2024-09-17T21:22:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3037000000000001e-05 2024-09-17T21:22:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858295+0000", 2024-09-17T21:22:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088518475999999999 2024-09-17T21:22:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858419+0000", 2024-09-17T21:22:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012378700000000001 2024-09-17T21:22:25.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858565+0000", 2024-09-17T21:22:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014630099999999999 2024-09-17T21:22:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12092 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:22:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.398040024, 2024-09-17T21:22:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089191555000000006, 2024-09-17T21:22:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12092 2024-09-17T21:22:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:22:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769815+0000", 2024-09-17T21:22:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769816+0000", 2024-09-17T21:22:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7199999999999997e-07 2024-09-17T21:22:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769822+0000", 2024-09-17T21:22:25.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:22:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769838+0000", 2024-09-17T21:22:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6674e-05 2024-09-17T21:22:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769851+0000", 2024-09-17T21:22:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2527999999999999e-05 2024-09-17T21:22:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858660+0000", 2024-09-17T21:22:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088808556999999996 2024-09-17T21:22:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.858817+0000", 2024-09-17T21:22:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015730400000000001 2024-09-17T21:22:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859007+0000", 2024-09-17T21:22:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018974899999999999 2024-09-17T21:22:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12093 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:22:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.39793192, 2024-09-17T21:22:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089429710999999995, 2024-09-17T21:22:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12093 2024-09-17T21:22:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:22:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769923+0000", 2024-09-17T21:22:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769924+0000", 2024-09-17T21:22:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.342e-06 2024-09-17T21:22:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769929+0000", 2024-09-17T21:22:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0350000000000001e-06 2024-09-17T21:22:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769946+0000", 2024-09-17T21:22:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6793999999999999e-05 2024-09-17T21:22:25.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769959+0000", 2024-09-17T21:22:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3062e-05 2024-09-17T21:22:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859075+0000", 2024-09-17T21:22:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089115696999999994 2024-09-17T21:22:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:25.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859202+0000", 2024-09-17T21:22:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012657 2024-09-17T21:22:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859353+0000", 2024-09-17T21:22:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151211 2024-09-17T21:22:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12094 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:22:25.226 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.397877744, 2024-09-17T21:22:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089695001999999996, 2024-09-17T21:22:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12094 2024-09-17T21:22:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:25.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:22:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769977+0000", 2024-09-17T21:22:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769978+0000", 2024-09-17T21:22:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.142e-06 2024-09-17T21:22:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769983+0000", 2024-09-17T21:22:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.386e-06 2024-09-17T21:22:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.769997+0000", 2024-09-17T21:22:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.416e-05 2024-09-17T21:22:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770012+0000", 2024-09-17T21:22:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5384999999999999e-05 2024-09-17T21:22:25.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859405+0000", 2024-09-17T21:22:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089392376999999995 2024-09-17T21:22:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859525+0000", 2024-09-17T21:22:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120555 2024-09-17T21:22:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:25.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859672+0000", 2024-09-17T21:22:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000146997 2024-09-17T21:22:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12095 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:22:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.39782525699999, 2024-09-17T21:22:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090073476, 2024-09-17T21:22:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:25.231 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12095 2024-09-17T21:22:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:22:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770030+0000", 2024-09-17T21:22:25.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770031+0000", 2024-09-17T21:22:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1699999999999997e-07 2024-09-17T21:22:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770036+0000", 2024-09-17T21:22:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.023e-06 2024-09-17T21:22:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770051+0000", 2024-09-17T21:22:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4953999999999999e-05 2024-09-17T21:22:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770063+0000", 2024-09-17T21:22:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2262999999999999e-05 2024-09-17T21:22:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859756+0000", 2024-09-17T21:22:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.089692980000000005 2024-09-17T21:22:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.859891+0000", 2024-09-17T21:22:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013459499999999999 2024-09-17T21:22:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860103+0000", 2024-09-17T21:22:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021274400000000001 2024-09-17T21:22:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12096 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:22:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.397775572, 2024-09-17T21:22:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090358274000000002, 2024-09-17T21:22:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12096 2024-09-17T21:22:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:25.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:22:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770079+0000", 2024-09-17T21:22:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770080+0000", 2024-09-17T21:22:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9400000000000004e-07 2024-09-17T21:22:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770085+0000", 2024-09-17T21:22:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2169999999999996e-06 2024-09-17T21:22:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770100+0000", 2024-09-17T21:22:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5077e-05 2024-09-17T21:22:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770112+0000", 2024-09-17T21:22:25.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1951e-05 2024-09-17T21:22:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860160+0000", 2024-09-17T21:22:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090048413999999993 2024-09-17T21:22:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860285+0000", 2024-09-17T21:22:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012466699999999999 2024-09-17T21:22:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860438+0000", 2024-09-17T21:22:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015305399999999999 2024-09-17T21:22:25.239 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12097 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:22:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.397727825, 2024-09-17T21:22:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090702375000000002, 2024-09-17T21:22:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:25.240 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12097 2024-09-17T21:22:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:22:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770127+0000", 2024-09-17T21:22:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770128+0000", 2024-09-17T21:22:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.04e-06 2024-09-17T21:22:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770133+0000", 2024-09-17T21:22:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8999999999999997e-06 2024-09-17T21:22:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770152+0000", 2024-09-17T21:22:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8890999999999999e-05 2024-09-17T21:22:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770164+0000", 2024-09-17T21:22:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1769e-05 2024-09-17T21:22:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860511+0000", 2024-09-17T21:22:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090347527999999996 2024-09-17T21:22:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860652+0000", 2024-09-17T21:22:25.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000140894 2024-09-17T21:22:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860830+0000", 2024-09-17T21:22:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017735299999999999 2024-09-17T21:22:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12098 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:22:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.397674357, 2024-09-17T21:22:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090997262999999995, 2024-09-17T21:22:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12098 2024-09-17T21:22:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:22:25.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.770181+0000", 2024-09-17T21:22:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775465+0000", 2024-09-17T21:22:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0052843350000000002 2024-09-17T21:22:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:25.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775476+0000", 2024-09-17T21:22:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0797e-05 2024-09-17T21:22:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775506+0000", 2024-09-17T21:22:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9709999999999998e-05 2024-09-17T21:22:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775530+0000", 2024-09-17T21:22:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4528999999999998e-05 2024-09-17T21:22:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.860894+0000", 2024-09-17T21:22:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.085364195000000004 2024-09-17T21:22:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861019+0000", 2024-09-17T21:22:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012467399999999999 2024-09-17T21:22:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.861178+0000", 2024-09-17T21:22:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159023 2024-09-17T21:22:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12105 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:22:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.391944354, 2024-09-17T21:22:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088330400000000003, 2024-09-17T21:22:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:25.249 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12105 2024-09-17T21:22:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:22:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:22:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775911+0000", 2024-09-17T21:22:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9999999999999997e-07 2024-09-17T21:22:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775914+0000", 2024-09-17T21:22:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4710000000000001e-06 2024-09-17T21:22:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775921+0000", 2024-09-17T21:22:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:22:25.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.775928+0000", 2024-09-17T21:22:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:22:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.863923+0000", 2024-09-17T21:22:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087995126000000007 2024-09-17T21:22:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864059+0000", 2024-09-17T21:22:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135797 2024-09-17T21:22:25.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864241+0000", 2024-09-17T21:22:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018252599999999999 2024-09-17T21:22:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.5958.0:12106 3.2 3:43f3a183:::rbd_data.1152eb799094.0000000000000003:22 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:22:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 123.39185517600001, 2024-09-17T21:22:25.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088586394999999998, 2024-09-17T21:22:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.5958", 2024-09-17T21:22:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12106 2024-09-17T21:22:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:22:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776000+0000", 2024-09-17T21:22:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776001+0000", 2024-09-17T21:22:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4300000000000001e-07 2024-09-17T21:22:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776004+0000", 2024-09-17T21:22:25.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8049999999999998e-06 2024-09-17T21:22:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776018+0000", 2024-09-17T21:22:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4613999999999999e-05 2024-09-17T21:22:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.776025+0000", 2024-09-17T21:22:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:22:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864279+0000", 2024-09-17T21:22:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088253570000000003 2024-09-17T21:22:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864433+0000", 2024-09-17T21:22:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015469400000000001 2024-09-17T21:22:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:20:21.864586+0000", 2024-09-17T21:22:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000153003 2024-09-17T21:22:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.257 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25033 3.6 3.b74753e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e168)", 2024-09-17T21:22:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:22:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 59.658411911999998, 2024-09-17T21:22:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93781952099999999, 2024-09-17T21:22:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:22:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:25.258 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25033 2024-09-17T21:22:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:22:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509443+0000", 2024-09-17T21:22:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509444+0000", 2024-09-17T21:22:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0020000000000001e-06 2024-09-17T21:22:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509450+0000", 2024-09-17T21:22:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4249999999999997e-06 2024-09-17T21:22:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509460+0000", 2024-09-17T21:22:25.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0183999999999999e-05 2024-09-17T21:22:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509466+0000", 2024-09-17T21:22:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0039999999999996e-06 2024-09-17T21:22:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509540+0000", 2024-09-17T21:22:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3847e-05 2024-09-17T21:22:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:25.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.509551+0000", 2024-09-17T21:22:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1731e-05 2024-09-17T21:22:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.447243+0000", 2024-09-17T21:22:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93769166000000004 2024-09-17T21:22:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.447263+0000", 2024-09-17T21:22:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9667999999999999e-05 2024-09-17T21:22:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:25.262 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:25033 3.6 3:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e168)", 2024-09-17T21:22:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:22:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 59.656886315000001, 2024-09-17T21:22:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.937309383, 2024-09-17T21:22:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-17T21:22:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:22:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25033 2024-09-17T21:22:25.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:22:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510969+0000", 2024-09-17T21:22:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510970+0000", 2024-09-17T21:22:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.564e-06 2024-09-17T21:22:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510978+0000", 2024-09-17T21:22:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.362e-06 2024-09-17T21:22:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.510998+0000", 2024-09-17T21:22:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0336999999999999e-05 2024-09-17T21:22:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511010+0000", 2024-09-17T21:22:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1786e-05 2024-09-17T21:22:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511080+0000", 2024-09-17T21:22:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0686999999999996e-05 2024-09-17T21:22:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:25.511087+0000", 2024-09-17T21:22:25.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4899999999999997e-06 2024-09-17T21:22:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.448050+0000", 2024-09-17T21:22:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93696351700000002 2024-09-17T21:22:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:21:26.448278+0000", 2024-09-17T21:22:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022764000000000001 2024-09-17T21:22:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.4434.0:30858 3.1a 3:5b9355dc:::rbd_header.11527824718:head [watch reconnect cookie 139860845522384 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e218)", 2024-09-17T21:22:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T21:22:15.639403+0000", 2024-09-17T21:22:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.5284516910000008, 2024-09-17T21:22:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.041145395, 2024-09-17T21:22:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T21:22:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T21:22:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T21:22:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-17T21:22:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.4434", 2024-09-17T21:22:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 30858 2024-09-17T21:22:25.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.269 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T21:22:25.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T21:22:25.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639403+0000", 2024-09-17T21:22:25.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T21:22:25.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639403+0000", 2024-09-17T21:22:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T21:22:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T21:22:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639406+0000", 2024-09-17T21:22:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2979999999999999e-06 2024-09-17T21:22:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T21:22:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639412+0000", 2024-09-17T21:22:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.494e-06 2024-09-17T21:22:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T21:22:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639423+0000", 2024-09-17T21:22:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1277000000000001e-05 2024-09-17T21:22:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639433+0000", 2024-09-17T21:22:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7699999999999996e-06 2024-09-17T21:22:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639520+0000", 2024-09-17T21:22:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7121999999999997e-05 2024-09-17T21:22:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-17T21:22:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:15.639534+0000", 2024-09-17T21:22:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3651e-05 2024-09-17T21:22:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:16.675183+0000", 2024-09-17T21:22:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0356492959999999 2024-09-17T21:22:25.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for rw locks", 2024-09-17T21:22:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:16.675242+0000", 2024-09-17T21:22:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8712000000000003e-05 2024-09-17T21:22:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T21:22:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:16.678347+0000", 2024-09-17T21:22:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031052940000000002 2024-09-17T21:22:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T21:22:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:16.678578+0000", 2024-09-17T21:22:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023119300000000001 2024-09-17T21:22:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T21:22:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T21:22:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-17T21:22:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T21:22:16.680549+0000", 2024-09-17T21:22:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019702880000000002 2024-09-17T21:22:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T21:22:25.274 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T21:22:25.275 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:25.276 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:22:25.277 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T21:22:25.278 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-17T21:22:25.278 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-17T21:22:25.278 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-17T21:22:25.278 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-17T21:22:25.278 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-17T21:22:25.278 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-17T21:22:25.278 INFO:teuthology.orchestra.run.smithi083.stderr:config diff 2024-09-17T21:22:25.278 INFO:teuthology.orchestra.run.smithi083.stderr:config diff get 2024-09-17T21:22:25.278 INFO:teuthology.orchestra.run.smithi083.stderr:config get 2024-09-17T21:22:25.278 INFO:teuthology.orchestra.run.smithi083.stderr:config help [] 2024-09-17T21:22:25.279 INFO:teuthology.orchestra.run.smithi083.stderr:config set ... 2024-09-17T21:22:25.279 INFO:teuthology.orchestra.run.smithi083.stderr:admin_socket: invalid command 2024-09-17T21:22:25.279 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-17T21:22:25.412 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:25.412 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:25.412 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:22:25.413 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:25.426 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:22:25.547 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:25.547 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:25.548 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:22:25.548 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:22:25.548 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:25.561 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:22:25.685 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:25.685 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:22:25.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:22:25.685 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:22:25.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.685 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T21:22:25.685 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:22:25.685 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 268.66644526900001, 2024-09-17T21:22:25.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.18241695199999999, 2024-09-17T21:22:25.685 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:25.686 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:25.686 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:25.686 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:25.686 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4195", 2024-09-17T21:22:25.686 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/669728366", 2024-09-17T21:22:25.686 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1 2024-09-17T21:22:25.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.686 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:25.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:25.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:22:25.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:25.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013765+0000", 2024-09-17T21:22:25.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:25.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013768+0000", 2024-09-17T21:22:25.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6170000000000001e-06 2024-09-17T21:22:25.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:25.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013782+0000", 2024-09-17T21:22:25.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4419999999999999e-05 2024-09-17T21:22:25.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:25.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013808+0000", 2024-09-17T21:22:25.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5058999999999998e-05 2024-09-17T21:22:25.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013824+0000", 2024-09-17T21:22:25.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6456999999999998e-05 2024-09-17T21:22:25.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013973+0000", 2024-09-17T21:22:25.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000148758 2024-09-17T21:22:25.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:25.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.013987+0000", 2024-09-17T21:22:25.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4229999999999999e-05 2024-09-17T21:22:25.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.192956+0000", 2024-09-17T21:22:25.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.17896857899999999 2024-09-17T21:22:25.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:25.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193224+0000", 2024-09-17T21:22:25.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026860499999999999 2024-09-17T21:22:25.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:22:25.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.193728+0000", 2024-09-17T21:22:25.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050416600000000001 2024-09-17T21:22:25.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:25.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195839+0000", 2024-09-17T21:22:25.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021110959999999998 2024-09-17T21:22:25.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:25.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.195878+0000", 2024-09-17T21:22:25.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8303999999999998e-05 2024-09-17T21:22:25.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:25.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196042+0000", 2024-09-17T21:22:25.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163902 2024-09-17T21:22:25.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:25.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:17:57.196182+0000", 2024-09-17T21:22:25.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014075900000000001 2024-09-17T21:22:25.691 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.691 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:25.691 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.692 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:561 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 140256452023056 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:22:25.692 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:22:25.692 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 249.08628098899999, 2024-09-17T21:22:25.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69349519299999995, 2024-09-17T21:22:25.692 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:25.692 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:25.692 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:25.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:25.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:22:25.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:22:25.692 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 561 2024-09-17T21:22:25.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.693 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:25.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:25.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:22:25.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:25.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593930+0000", 2024-09-17T21:22:25.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:25.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593932+0000", 2024-09-17T21:22:25.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.39e-06 2024-09-17T21:22:25.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:25.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593943+0000", 2024-09-17T21:22:25.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.063e-05 2024-09-17T21:22:25.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:25.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593976+0000", 2024-09-17T21:22:25.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2959000000000001e-05 2024-09-17T21:22:25.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.593993+0000", 2024-09-17T21:22:25.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.721e-05 2024-09-17T21:22:25.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594134+0000", 2024-09-17T21:22:25.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141199 2024-09-17T21:22:25.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:22:25.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594148+0000", 2024-09-17T21:22:25.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3993e-05 2024-09-17T21:22:25.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605351+0000", 2024-09-17T21:22:25.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011203197 2024-09-17T21:22:25.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:25.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605381+0000", 2024-09-17T21:22:25.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9522999999999998e-05 2024-09-17T21:22:25.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.279863+0000", 2024-09-17T21:22:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67448243399999996 2024-09-17T21:22:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280021+0000", 2024-09-17T21:22:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015783799999999999 2024-09-17T21:22:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287425+0000", 2024-09-17T21:22:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074038200000000002 2024-09-17T21:22:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.697 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:229 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [watch reconnect cookie 139860845289440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:22:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:22:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 249.08618367700001, 2024-09-17T21:22:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69530897199999997, 2024-09-17T21:22:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:25.698 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 229 2024-09-17T21:22:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:22:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594027+0000", 2024-09-17T21:22:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594030+0000", 2024-09-17T21:22:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8140000000000001e-06 2024-09-17T21:22:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594042+0000", 2024-09-17T21:22:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.243e-05 2024-09-17T21:22:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594079+0000", 2024-09-17T21:22:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7203000000000003e-05 2024-09-17T21:22:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594102+0000", 2024-09-17T21:22:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2308999999999999e-05 2024-09-17T21:22:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594244+0000", 2024-09-17T21:22:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014185599999999999 2024-09-17T21:22:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:22:25.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.594254+0000", 2024-09-17T21:22:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0523000000000001e-05 2024-09-17T21:22:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605647+0000", 2024-09-17T21:22:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011392928 2024-09-17T21:22:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605669+0000", 2024-09-17T21:22:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1529999999999999e-05 2024-09-17T21:22:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281327+0000", 2024-09-17T21:22:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67565869300000003 2024-09-17T21:22:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:22:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281392+0000", 2024-09-17T21:22:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4760999999999999e-05 2024-09-17T21:22:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287633+0000", 2024-09-17T21:22:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006240593 2024-09-17T21:22:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287686+0000", 2024-09-17T21:22:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3498999999999997e-05 2024-09-17T21:22:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.289336+0000", 2024-09-17T21:22:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016498330000000001 2024-09-17T21:22:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4488.0:562 3.1e 3:7b23fd98:::rbd_header.1152c01721e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:22:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:22:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 249.08409687599999, 2024-09-17T21:22:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.69148204999999996, 2024-09-17T21:22:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4488", 2024-09-17T21:22:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1062443025", 2024-09-17T21:22:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 562 2024-09-17T21:22:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.705 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:22:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596114+0000", 2024-09-17T21:22:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596116+0000", 2024-09-17T21:22:25.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3769999999999999e-06 2024-09-17T21:22:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596128+0000", 2024-09-17T21:22:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:22:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596156+0000", 2024-09-17T21:22:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8255999999999998e-05 2024-09-17T21:22:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596170+0000", 2024-09-17T21:22:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3159999999999999e-05 2024-09-17T21:22:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596281+0000", 2024-09-17T21:22:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111276 2024-09-17T21:22:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:22:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.596290+0000", 2024-09-17T21:22:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2089999999999994e-06 2024-09-17T21:22:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605524+0000", 2024-09-17T21:22:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092339659999999997 2024-09-17T21:22:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:16.605550+0000", 2024-09-17T21:22:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6156000000000002e-05 2024-09-17T21:22:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280633+0000", 2024-09-17T21:22:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.67508313799999997 2024-09-17T21:22:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.280693+0000", 2024-09-17T21:22:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0137999999999999e-05 2024-09-17T21:22:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:22:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281053+0000", 2024-09-17T21:22:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035951100000000002 2024-09-17T21:22:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.281656+0000", 2024-09-17T21:22:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060293 2024-09-17T21:22:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287484+0000", 2024-09-17T21:22:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005828186 2024-09-17T21:22:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287530+0000", 2024-09-17T21:22:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5872999999999999e-05 2024-09-17T21:22:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:17.287596+0000", 2024-09-17T21:22:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5970999999999996e-05 2024-09-17T21:22:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.711 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3050 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 140452812641968 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:22:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 216.40019809500001, 2024-09-17T21:22:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186760189999999, 2024-09-17T21:22:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:25.712 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:22:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:22:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3050 2024-09-17T21:22:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:22:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:25.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280013+0000", 2024-09-17T21:22:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280014+0000", 2024-09-17T21:22:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.531e-06 2024-09-17T21:22:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280021+0000", 2024-09-17T21:22:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:22:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280043+0000", 2024-09-17T21:22:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1438999999999999e-05 2024-09-17T21:22:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280052+0000", 2024-09-17T21:22:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8460000000000003e-06 2024-09-17T21:22:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280151+0000", 2024-09-17T21:22:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8603000000000004e-05 2024-09-17T21:22:25.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:22:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.280165+0000", 2024-09-17T21:22:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4473e-05 2024-09-17T21:22:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284288+0000", 2024-09-17T21:22:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041228660000000002 2024-09-17T21:22:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.284296+0000", 2024-09-17T21:22:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5460000000000003e-06 2024-09-17T21:22:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.295896+0000", 2024-09-17T21:22:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0115999790000001 2024-09-17T21:22:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.296085+0000", 2024-09-17T21:22:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018918900000000001 2024-09-17T21:22:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298689+0000", 2024-09-17T21:22:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026036010000000001 2024-09-17T21:22:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3059 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:22:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 216.39110638400001, 2024-09-17T21:22:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008149419, 2024-09-17T21:22:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:25.718 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:22:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:22:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3059 2024-09-17T21:22:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:22:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:25.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289104+0000", 2024-09-17T21:22:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289105+0000", 2024-09-17T21:22:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0690000000000001e-06 2024-09-17T21:22:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289110+0000", 2024-09-17T21:22:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:22:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289124+0000", 2024-09-17T21:22:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3855e-05 2024-09-17T21:22:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289133+0000", 2024-09-17T21:22:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0149999999999992e-06 2024-09-17T21:22:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289204+0000", 2024-09-17T21:22:25.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0732e-05 2024-09-17T21:22:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289212+0000", 2024-09-17T21:22:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9459999999999998e-06 2024-09-17T21:22:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297049+0000", 2024-09-17T21:22:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.007837777 2024-09-17T21:22:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297179+0000", 2024-09-17T21:22:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012980000000000001 2024-09-17T21:22:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297254+0000", 2024-09-17T21:22:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4443999999999994e-05 2024-09-17T21:22:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4703.0:3060 3.8 3:17175440:::rbd_data.11527035abe.0000000000000002:head [read 10883072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:22:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 216.39093433299999, 2024-09-17T21:22:25.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.008133403, 2024-09-17T21:22:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4703", 2024-09-17T21:22:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3772403449", 2024-09-17T21:22:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 3060 2024-09-17T21:22:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:22:25.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289276+0000", 2024-09-17T21:22:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289278+0000", 2024-09-17T21:22:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8500000000000001e-06 2024-09-17T21:22:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289286+0000", 2024-09-17T21:22:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6249999999999998e-06 2024-09-17T21:22:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289303+0000", 2024-09-17T21:22:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-05 2024-09-17T21:22:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289316+0000", 2024-09-17T21:22:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2499e-05 2024-09-17T21:22:25.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289830+0000", 2024-09-17T21:22:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051392600000000005 2024-09-17T21:22:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.289846+0000", 2024-09-17T21:22:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6679000000000001e-05 2024-09-17T21:22:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297282+0000", 2024-09-17T21:22:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0074357540000001 2024-09-17T21:22:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297344+0000", 2024-09-17T21:22:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2080000000000002e-05 2024-09-17T21:22:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297410+0000", 2024-09-17T21:22:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000002e-05 2024-09-17T21:22:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.728 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4285 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:22:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 216.216001586, 2024-09-17T21:22:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83602951000000003, 2024-09-17T21:22:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:25.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4285 2024-09-17T21:22:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:22:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464209+0000", 2024-09-17T21:22:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464212+0000", 2024-09-17T21:22:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.886e-06 2024-09-17T21:22:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464224+0000", 2024-09-17T21:22:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1851999999999999e-05 2024-09-17T21:22:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464250+0000", 2024-09-17T21:22:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5795999999999998e-05 2024-09-17T21:22:25.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464264+0000", 2024-09-17T21:22:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4759e-05 2024-09-17T21:22:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464368+0000", 2024-09-17T21:22:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103373 2024-09-17T21:22:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:25.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464376+0000", 2024-09-17T21:22:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1149999999999994e-06 2024-09-17T21:22:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297499+0000", 2024-09-17T21:22:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83312273800000003 2024-09-17T21:22:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.297623+0000", 2024-09-17T21:22:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000124292 2024-09-17T21:22:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300239+0000", 2024-09-17T21:22:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002615699 2024-09-17T21:22:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4286 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:22:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 216.215917029, 2024-09-17T21:22:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83625623299999996, 2024-09-17T21:22:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:25.734 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4286 2024-09-17T21:22:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:22:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:25.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:22:25.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:25.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464294+0000", 2024-09-17T21:22:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:22:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464298+0000", 2024-09-17T21:22:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3330000000000001e-06 2024-09-17T21:22:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464307+0000", 2024-09-17T21:22:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8899999999999996e-06 2024-09-17T21:22:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464313+0000", 2024-09-17T21:22:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:22:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464427+0000", 2024-09-17T21:22:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113891 2024-09-17T21:22:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464441+0000", 2024-09-17T21:22:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3339000000000001e-05 2024-09-17T21:22:25.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298196+0000", 2024-09-17T21:22:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83375549999999998 2024-09-17T21:22:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.298271+0000", 2024-09-17T21:22:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4389999999999998e-05 2024-09-17T21:22:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:25.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.300550+0000", 2024-09-17T21:22:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002279367 2024-09-17T21:22:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:4287 3.8 3:10577218:::rbd_header.11527035abe:head [watch reconnect cookie 139860845289376 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:22:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:22:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 216.21585139199999, 2024-09-17T21:22:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83681407299999999, 2024-09-17T21:22:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:25.740 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 4287 2024-09-17T21:22:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:22:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464359+0000", 2024-09-17T21:22:25.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464360+0000", 2024-09-17T21:22:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:22:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464363+0000", 2024-09-17T21:22:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3589999999999999e-06 2024-09-17T21:22:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464374+0000", 2024-09-17T21:22:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0541e-05 2024-09-17T21:22:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464380+0000", 2024-09-17T21:22:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:22:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464462+0000", 2024-09-17T21:22:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2321e-05 2024-09-17T21:22:25.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:49.464466+0000", 2024-09-17T21:22:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4020000000000002e-06 2024-09-17T21:22:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299158+0000", 2024-09-17T21:22:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.83469189200000005 2024-09-17T21:22:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.299360+0000", 2024-09-17T21:22:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000202165 2024-09-17T21:22:25.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:50.301173+0000", 2024-09-17T21:22:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018136109999999999 2024-09-17T21:22:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2630 3.15 3:a83708c4:::rbd_data.1152a8ae2019.000000000000001c:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:25.745 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:22:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 164.842276295, 2024-09-17T21:22:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084711648000000001, 2024-09-17T21:22:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:22:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2630 2024-09-17T21:22:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:22:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837934+0000", 2024-09-17T21:22:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837936+0000", 2024-09-17T21:22:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2640000000000001e-06 2024-09-17T21:22:25.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837943+0000", 2024-09-17T21:22:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:22:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.837991+0000", 2024-09-17T21:22:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.863e-05 2024-09-17T21:22:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838007+0000", 2024-09-17T21:22:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5512999999999999e-05 2024-09-17T21:22:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.843997+0000", 2024-09-17T21:22:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059897220000000003 2024-09-17T21:22:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.844266+0000", 2024-09-17T21:22:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026968599999999999 2024-09-17T21:22:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922646+0000", 2024-09-17T21:22:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078379734000000006 2024-09-17T21:22:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5472.0:2633 3.15 3:ae316531:::rbd_data.1152a8ae2019.000000000000001e:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T21:22:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:22:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 164.84215678300001, 2024-09-17T21:22:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084751321000000004, 2024-09-17T21:22:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:25.750 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5472", 2024-09-17T21:22:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/905247519", 2024-09-17T21:22:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2633 2024-09-17T21:22:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:22:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838054+0000", 2024-09-17T21:22:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838055+0000", 2024-09-17T21:22:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5400000000000001e-06 2024-09-17T21:22:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:25.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838063+0000", 2024-09-17T21:22:25.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1829999999999997e-06 2024-09-17T21:22:25.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:25.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838081+0000", 2024-09-17T21:22:25.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-05 2024-09-17T21:22:25.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.838099+0000", 2024-09-17T21:22:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7569000000000001e-05 2024-09-17T21:22:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845369+0000", 2024-09-17T21:22:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072699069999999999 2024-09-17T21:22:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.845646+0000", 2024-09-17T21:22:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027756599999999998 2024-09-17T21:22:25.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:40.922805+0000", 2024-09-17T21:22:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077158915999999994 2024-09-17T21:22:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:996 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:25.755 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:22:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 120.24904341200001, 2024-09-17T21:22:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084493736999999999, 2024-09-17T21:22:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 996 2024-09-17T21:22:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:25.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:22:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431167+0000", 2024-09-17T21:22:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431168+0000", 2024-09-17T21:22:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4e-07 2024-09-17T21:22:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431172+0000", 2024-09-17T21:22:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.608e-06 2024-09-17T21:22:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431180+0000", 2024-09-17T21:22:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5650000000000002e-06 2024-09-17T21:22:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431188+0000", 2024-09-17T21:22:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:22:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515354+0000", 2024-09-17T21:22:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084165977000000003 2024-09-17T21:22:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515451+0000", 2024-09-17T21:22:25.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6669000000000006e-05 2024-09-17T21:22:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515661+0000", 2024-09-17T21:22:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021021599999999999 2024-09-17T21:22:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:997 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 583680~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:22:25.760 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 120.249012017, 2024-09-17T21:22:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084772419000000002, 2024-09-17T21:22:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 997 2024-09-17T21:22:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:25.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:22:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:22:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431199+0000", 2024-09-17T21:22:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0400000000000005e-07 2024-09-17T21:22:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431202+0000", 2024-09-17T21:22:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:22:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431208+0000", 2024-09-17T21:22:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4740000000000002e-06 2024-09-17T21:22:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431214+0000", 2024-09-17T21:22:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6550000000000001e-06 2024-09-17T21:22:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515741+0000", 2024-09-17T21:22:25.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084526904 2024-09-17T21:22:25.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:25.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515826+0000", 2024-09-17T21:22:25.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4643999999999998e-05 2024-09-17T21:22:25.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:25.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.515971+0000", 2024-09-17T21:22:25.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145444 2024-09-17T21:22:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:998 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 587776~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:22:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 120.248989045, 2024-09-17T21:22:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085108565999999997, 2024-09-17T21:22:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:25.765 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 998 2024-09-17T21:22:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:22:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:22:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431222+0000", 2024-09-17T21:22:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:22:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431224+0000", 2024-09-17T21:22:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:22:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431231+0000", 2024-09-17T21:22:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000002e-06 2024-09-17T21:22:25.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431236+0000", 2024-09-17T21:22:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.648e-06 2024-09-17T21:22:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516024+0000", 2024-09-17T21:22:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.084787584999999999 2024-09-17T21:22:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516122+0000", 2024-09-17T21:22:25.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8076000000000001e-05 2024-09-17T21:22:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516330+0000", 2024-09-17T21:22:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020808 2024-09-17T21:22:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1000 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 591872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:22:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 120.24896692999999, 2024-09-17T21:22:25.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085389065, 2024-09-17T21:22:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1000 2024-09-17T21:22:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:22:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:22:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431244+0000", 2024-09-17T21:22:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:22:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431246+0000", 2024-09-17T21:22:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1749999999999999e-06 2024-09-17T21:22:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431253+0000", 2024-09-17T21:22:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2790000000000004e-06 2024-09-17T21:22:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431258+0000", 2024-09-17T21:22:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3299999999999998e-06 2024-09-17T21:22:25.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516384+0000", 2024-09-17T21:22:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085125549999999994 2024-09-17T21:22:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516460+0000", 2024-09-17T21:22:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6359999999999994e-05 2024-09-17T21:22:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516633+0000", 2024-09-17T21:22:25.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017285900000000001 2024-09-17T21:22:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5590.0:1001 3.5 3:a2165348:::rbd_data.1152eb799094.0000000000000001:22 [read 595968~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e124)", 2024-09-17T21:22:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:22:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 120.248943524, 2024-09-17T21:22:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085614715999999993, 2024-09-17T21:22:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:25.774 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5590", 2024-09-17T21:22:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/838234909", 2024-09-17T21:22:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1001 2024-09-17T21:22:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:22:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:25.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.431267+0000", 2024-09-17T21:22:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438066+0000", 2024-09-17T21:22:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067989490000000003 2024-09-17T21:22:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438073+0000", 2024-09-17T21:22:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8419999999999999e-06 2024-09-17T21:22:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438090+0000", 2024-09-17T21:22:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6657e-05 2024-09-17T21:22:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.438099+0000", 2024-09-17T21:22:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0529999999999996e-06 2024-09-17T21:22:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516678+0000", 2024-09-17T21:22:25.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.078579422999999995 2024-09-17T21:22:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516739+0000", 2024-09-17T21:22:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1315000000000005e-05 2024-09-17T21:22:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:25.516882+0000", 2024-09-17T21:22:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014247700000000001 2024-09-17T21:22:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:19998 3.5 3:a4885450:::rbd_object_map.1152d139f9f9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-17T21:22:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:22:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 106.277676279, 2024-09-17T21:22:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90715489299999996, 2024-09-17T21:22:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19998 2024-09-17T21:22:25.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:22:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402534+0000", 2024-09-17T21:22:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:25.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402537+0000", 2024-09-17T21:22:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5600000000000001e-06 2024-09-17T21:22:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402549+0000", 2024-09-17T21:22:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1929999999999999e-05 2024-09-17T21:22:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402579+0000", 2024-09-17T21:22:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0093000000000001e-05 2024-09-17T21:22:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402595+0000", 2024-09-17T21:22:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5855999999999999e-05 2024-09-17T21:22:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402731+0000", 2024-09-17T21:22:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013610799999999999 2024-09-17T21:22:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:39.402750+0000", 2024-09-17T21:22:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8737e-05 2024-09-17T21:22:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.305977+0000", 2024-09-17T21:22:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90322768399999998 2024-09-17T21:22:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306236+0000", 2024-09-17T21:22:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000258111 2024-09-17T21:22:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-17T21:22:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.306557+0000", 2024-09-17T21:22:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032159799999999998 2024-09-17T21:22:25.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.308286+0000", 2024-09-17T21:22:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017292150000000001 2024-09-17T21:22:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309530+0000", 2024-09-17T21:22:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00124409 2024-09-17T21:22:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309630+0000", 2024-09-17T21:22:25.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9689e-05 2024-09-17T21:22:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:40.309689+0000", 2024-09-17T21:22:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9222000000000003e-05 2024-09-17T21:22:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:25277 3.1f 3:ff5d7cd5:::rbd_header.11524f65d4ff:head [watch ping cookie 139860845478480 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-17T21:22:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:22:25.785 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 56.094720209999998, 2024-09-17T21:22:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.92185720000000004, 2024-09-17T21:22:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 25277 2024-09-17T21:22:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:25.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:22:25.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:25.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585490+0000", 2024-09-17T21:22:25.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:25.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585493+0000", 2024-09-17T21:22:25.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:22:25.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:25.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585505+0000", 2024-09-17T21:22:25.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2466000000000001e-05 2024-09-17T21:22:25.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:25.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585537+0000", 2024-09-17T21:22:25.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1183000000000003e-05 2024-09-17T21:22:25.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585553+0000", 2024-09-17T21:22:25.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.662e-05 2024-09-17T21:22:25.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585696+0000", 2024-09-17T21:22:25.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014293800000000001 2024-09-17T21:22:25.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:25.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:29.585718+0000", 2024-09-17T21:22:25.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1860000000000001e-05 2024-09-17T21:22:25.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.501188+0000", 2024-09-17T21:22:25.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.91547051300000004 2024-09-17T21:22:25.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for rw locks", 2024-09-17T21:22:25.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.501260+0000", 2024-09-17T21:22:25.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1571999999999998e-05 2024-09-17T21:22:25.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507002+0000", 2024-09-17T21:22:25.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057414850000000002 2024-09-17T21:22:25.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:25.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507143+0000", 2024-09-17T21:22:25.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014140299999999999 2024-09-17T21:22:25.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:25.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:30.507348+0000", 2024-09-17T21:22:25.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000204773 2024-09-17T21:22:25.791 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.791 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:25.791 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.791 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:26042 3.f 3:f24134c1:::rbd_id.image_client.0-clone34:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e186)", 2024-09-17T21:22:25.791 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:22:25.792 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 42.194031842000001, 2024-09-17T21:22:25.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.91185481000000002, 2024-09-17T21:22:25.792 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:25.792 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:25.792 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:25.792 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:25.792 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:25.792 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:25.792 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 26042 2024-09-17T21:22:25.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.792 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:25.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:25.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:22:25.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:25.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486179+0000", 2024-09-17T21:22:25.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:25.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:25.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486180+0000", 2024-09-17T21:22:25.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5990000000000001e-06 2024-09-17T21:22:25.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:25.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486187+0000", 2024-09-17T21:22:25.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7859999999999999e-06 2024-09-17T21:22:25.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:25.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486206+0000", 2024-09-17T21:22:25.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8641e-05 2024-09-17T21:22:25.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:25.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486219+0000", 2024-09-17T21:22:25.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.308e-05 2024-09-17T21:22:25.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486283+0000", 2024-09-17T21:22:25.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4084000000000005e-05 2024-09-17T21:22:25.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:25.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:43.486290+0000", 2024-09-17T21:22:25.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.584e-06 2024-09-17T21:22:25.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:25.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392182+0000", 2024-09-17T21:22:25.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.90589206499999997 2024-09-17T21:22:25.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:25.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392518+0000", 2024-09-17T21:22:25.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033596 2024-09-17T21:22:25.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-17T21:22:25.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.392802+0000", 2024-09-17T21:22:25.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028394999999999999 2024-09-17T21:22:25.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:25.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397705+0000", 2024-09-17T21:22:25.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0049031769999999999 2024-09-17T21:22:25.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:25.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397770+0000", 2024-09-17T21:22:25.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4765999999999996e-05 2024-09-17T21:22:25.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:25.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.397927+0000", 2024-09-17T21:22:25.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015699900000000001 2024-09-17T21:22:25.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:25.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:25.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:25.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.398034+0000", 2024-09-17T21:22:25.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000107119 2024-09-17T21:22:25.798 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.798 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:25.798 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.798 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:25.798 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:25.798 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:25.801 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:22:25.927 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:25.928 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:25.928 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:22:25.928 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:25.942 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:22:26.064 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:26.065 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:26.065 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:22:26.065 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:22:26.065 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:26.079 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:22:26.170 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:22:26.166+0000 7fb243f7f7c0 -1 Falling back to public interface 2024-09-17T21:22:26.204 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:26.204 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:22:26.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:22:26.204 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:22:26.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.205 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:18003 3.1d 3:b9765580:::rbd_header.11527d2b15f6:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T21:22:26.205 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:22:26.205 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 133.84249329100001, 2024-09-17T21:22:26.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94165346900000002, 2024-09-17T21:22:26.205 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.205 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.205 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.205 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.205 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:26.205 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:26.205 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18003 2024-09-17T21:22:26.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.205 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:22:26.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357879+0000", 2024-09-17T21:22:26.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357880+0000", 2024-09-17T21:22:26.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:22:26.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357891+0000", 2024-09-17T21:22:26.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0417999999999999e-05 2024-09-17T21:22:26.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357913+0000", 2024-09-17T21:22:26.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2215e-05 2024-09-17T21:22:26.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.357922+0000", 2024-09-17T21:22:26.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4779999999999997e-06 2024-09-17T21:22:26.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358023+0000", 2024-09-17T21:22:26.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101025 2024-09-17T21:22:26.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:26.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:12.358040+0000", 2024-09-17T21:22:26.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6291999999999999e-05 2024-09-17T21:22:26.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298203+0000", 2024-09-17T21:22:26.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.94016297500000001 2024-09-17T21:22:26.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.298626+0000", 2024-09-17T21:22:26.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000422853 2024-09-17T21:22:26.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:13.299532+0000", 2024-09-17T21:22:26.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00090657300000000004 2024-09-17T21:22:26.209 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.209 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.209 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.210 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15707 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 982016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:26.210 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:22:26.210 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.679328044, 2024-09-17T21:22:26.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092305228000000003, 2024-09-17T21:22:26.210 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.210 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.210 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.210 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.210 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:26.210 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:26.210 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15707 2024-09-17T21:22:26.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.211 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:22:26.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521044+0000", 2024-09-17T21:22:26.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521045+0000", 2024-09-17T21:22:26.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4899999999999999e-06 2024-09-17T21:22:26.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521066+0000", 2024-09-17T21:22:26.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0236000000000001e-05 2024-09-17T21:22:26.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521085+0000", 2024-09-17T21:22:26.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8819999999999999e-05 2024-09-17T21:22:26.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521100+0000", 2024-09-17T21:22:26.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5146e-05 2024-09-17T21:22:26.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.612978+0000", 2024-09-17T21:22:26.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091878847 2024-09-17T21:22:26.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613079+0000", 2024-09-17T21:22:26.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100503 2024-09-17T21:22:26.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613349+0000", 2024-09-17T21:22:26.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000270186 2024-09-17T21:22:26.214 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.214 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.214 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.214 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15708 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 986112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:26.214 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:22:26.214 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.679249869, 2024-09-17T21:22:26.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092547217000000001, 2024-09-17T21:22:26.215 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.215 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.215 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.215 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.215 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:26.215 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:26.215 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15708 2024-09-17T21:22:26.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.215 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:22:26.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521122+0000", 2024-09-17T21:22:26.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521123+0000", 2024-09-17T21:22:26.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.353e-06 2024-09-17T21:22:26.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521129+0000", 2024-09-17T21:22:26.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:22:26.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521145+0000", 2024-09-17T21:22:26.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.611e-05 2024-09-17T21:22:26.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521154+0000", 2024-09-17T21:22:26.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4500000000000004e-06 2024-09-17T21:22:26.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613397+0000", 2024-09-17T21:22:26.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092243275 2024-09-17T21:22:26.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613503+0000", 2024-09-17T21:22:26.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106013 2024-09-17T21:22:26.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613669+0000", 2024-09-17T21:22:26.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016642299999999999 2024-09-17T21:22:26.219 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.219 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.219 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.219 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15709 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 990208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:26.219 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:22:26.219 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.679206853, 2024-09-17T21:22:26.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092728678999999994, 2024-09-17T21:22:26.219 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.219 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.219 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.220 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.220 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:26.220 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:26.220 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15709 2024-09-17T21:22:26.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.220 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:22:26.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521165+0000", 2024-09-17T21:22:26.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521166+0000", 2024-09-17T21:22:26.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7000000000000004e-07 2024-09-17T21:22:26.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521169+0000", 2024-09-17T21:22:26.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2660000000000002e-06 2024-09-17T21:22:26.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521181+0000", 2024-09-17T21:22:26.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.208e-05 2024-09-17T21:22:26.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521194+0000", 2024-09-17T21:22:26.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2707e-05 2024-09-17T21:22:26.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613695+0000", 2024-09-17T21:22:26.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092500709 2024-09-17T21:22:26.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613750+0000", 2024-09-17T21:22:26.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.503e-05 2024-09-17T21:22:26.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613894+0000", 2024-09-17T21:22:26.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144117 2024-09-17T21:22:26.223 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.223 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.223 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.224 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15710 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 994304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:26.224 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:26.224 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.679162287, 2024-09-17T21:22:26.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092957729000000003, 2024-09-17T21:22:26.224 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.224 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.224 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.224 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.224 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:26.224 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:26.224 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15710 2024-09-17T21:22:26.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.224 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:26.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:26.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521210+0000", 2024-09-17T21:22:26.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8000000000000005e-07 2024-09-17T21:22:26.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521216+0000", 2024-09-17T21:22:26.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3859999999999998e-06 2024-09-17T21:22:26.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521234+0000", 2024-09-17T21:22:26.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:22:26.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521248+0000", 2024-09-17T21:22:26.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4253000000000001e-05 2024-09-17T21:22:26.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613926+0000", 2024-09-17T21:22:26.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092678324000000006 2024-09-17T21:22:26.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.613974+0000", 2024-09-17T21:22:26.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7754999999999999e-05 2024-09-17T21:22:26.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614167+0000", 2024-09-17T21:22:26.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193508 2024-09-17T21:22:26.228 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.228 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.228 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.228 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15711 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 998400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:26.228 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:22:26.228 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.679104905, 2024-09-17T21:22:26.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093268000000000004, 2024-09-17T21:22:26.228 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.228 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.229 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.229 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.229 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:26.229 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:26.229 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15711 2024-09-17T21:22:26.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.229 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:22:26.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521267+0000", 2024-09-17T21:22:26.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521268+0000", 2024-09-17T21:22:26.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-17T21:22:26.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521273+0000", 2024-09-17T21:22:26.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:22:26.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521290+0000", 2024-09-17T21:22:26.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7269999999999999e-05 2024-09-17T21:22:26.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521304+0000", 2024-09-17T21:22:26.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3626e-05 2024-09-17T21:22:26.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614220+0000", 2024-09-17T21:22:26.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.092916367 2024-09-17T21:22:26.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614322+0000", 2024-09-17T21:22:26.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102212 2024-09-17T21:22:26.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614535+0000", 2024-09-17T21:22:26.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021266199999999999 2024-09-17T21:22:26.232 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.232 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.233 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.233 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15712 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1002496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:26.233 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:22:26.233 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.67904941, 2024-09-17T21:22:26.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093568707000000001, 2024-09-17T21:22:26.233 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.233 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.233 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.233 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.233 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:26.234 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:26.234 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15712 2024-09-17T21:22:26.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.234 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:22:26.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521323+0000", 2024-09-17T21:22:26.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521324+0000", 2024-09-17T21:22:26.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.17e-06 2024-09-17T21:22:26.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521334+0000", 2024-09-17T21:22:26.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9369999999999999e-06 2024-09-17T21:22:26.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521356+0000", 2024-09-17T21:22:26.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2045999999999999e-05 2024-09-17T21:22:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521370+0000", 2024-09-17T21:22:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4270000000000001e-05 2024-09-17T21:22:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614583+0000", 2024-09-17T21:22:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093213220999999999 2024-09-17T21:22:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614680+0000", 2024-09-17T21:22:26.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6900999999999999e-05 2024-09-17T21:22:26.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614891+0000", 2024-09-17T21:22:26.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021116200000000001 2024-09-17T21:22:26.237 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.237 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.237 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.237 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15713 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1006592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:26.237 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:22:26.238 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.67898270800001, 2024-09-17T21:22:26.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093834641999999996, 2024-09-17T21:22:26.238 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.238 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.238 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.238 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.238 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:26.238 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:26.238 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15713 2024-09-17T21:22:26.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.238 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:22:26.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521389+0000", 2024-09-17T21:22:26.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521390+0000", 2024-09-17T21:22:26.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1340000000000001e-06 2024-09-17T21:22:26.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521396+0000", 2024-09-17T21:22:26.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-17T21:22:26.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521417+0000", 2024-09-17T21:22:26.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1236000000000001e-05 2024-09-17T21:22:26.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521431+0000", 2024-09-17T21:22:26.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3873000000000001e-05 2024-09-17T21:22:26.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.614938+0000", 2024-09-17T21:22:26.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093506522999999994 2024-09-17T21:22:26.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615025+0000", 2024-09-17T21:22:26.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7635999999999999e-05 2024-09-17T21:22:26.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615224+0000", 2024-09-17T21:22:26.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019844700000000001 2024-09-17T21:22:26.242 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.242 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.242 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.242 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15714 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:26.242 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:22:26.242 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.678919452, 2024-09-17T21:22:26.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094071925000000001, 2024-09-17T21:22:26.242 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.242 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.242 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.243 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.243 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:26.243 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:26.243 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15714 2024-09-17T21:22:26.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.243 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:22:26.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521453+0000", 2024-09-17T21:22:26.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521454+0000", 2024-09-17T21:22:26.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.097e-06 2024-09-17T21:22:26.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521459+0000", 2024-09-17T21:22:26.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2660000000000001e-06 2024-09-17T21:22:26.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521481+0000", 2024-09-17T21:22:26.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1878999999999998e-05 2024-09-17T21:22:26.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521498+0000", 2024-09-17T21:22:26.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6807e-05 2024-09-17T21:22:26.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615271+0000", 2024-09-17T21:22:26.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093773286999999997 2024-09-17T21:22:26.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615366+0000", 2024-09-17T21:22:26.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5322000000000007e-05 2024-09-17T21:22:26.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615524+0000", 2024-09-17T21:22:26.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015826699999999999 2024-09-17T21:22:26.246 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.246 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.246 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.246 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15715 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1014784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:26.246 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:22:26.247 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.678855117, 2024-09-17T21:22:26.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094243840999999995, 2024-09-17T21:22:26.247 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.247 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.247 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.247 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.247 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:26.247 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:26.247 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15715 2024-09-17T21:22:26.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.247 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:22:26.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521517+0000", 2024-09-17T21:22:26.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521518+0000", 2024-09-17T21:22:26.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4300000000000001e-06 2024-09-17T21:22:26.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521524+0000", 2024-09-17T21:22:26.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5860000000000004e-06 2024-09-17T21:22:26.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521541+0000", 2024-09-17T21:22:26.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7574000000000001e-05 2024-09-17T21:22:26.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521555+0000", 2024-09-17T21:22:26.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3540000000000001e-05 2024-09-17T21:22:26.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615560+0000", 2024-09-17T21:22:26.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.09400472 2024-09-17T21:22:26.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615614+0000", 2024-09-17T21:22:26.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4693e-05 2024-09-17T21:22:26.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615761+0000", 2024-09-17T21:22:26.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000146298 2024-09-17T21:22:26.251 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.251 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.251 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.251 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15716 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1018880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:26.251 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:22:26.251 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.67879877199999, 2024-09-17T21:22:26.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094400676000000003, 2024-09-17T21:22:26.251 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.251 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.252 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.252 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.252 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:26.252 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:26.252 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15716 2024-09-17T21:22:26.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.252 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:22:26.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521573+0000", 2024-09-17T21:22:26.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521574+0000", 2024-09-17T21:22:26.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1939999999999999e-06 2024-09-17T21:22:26.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521579+0000", 2024-09-17T21:22:26.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:22:26.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521597+0000", 2024-09-17T21:22:26.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8074e-05 2024-09-17T21:22:26.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521613+0000", 2024-09-17T21:22:26.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5316e-05 2024-09-17T21:22:26.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615780+0000", 2024-09-17T21:22:26.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094166874999999997 2024-09-17T21:22:26.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615836+0000", 2024-09-17T21:22:26.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6926999999999999e-05 2024-09-17T21:22:26.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.615974+0000", 2024-09-17T21:22:26.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013741400000000001 2024-09-17T21:22:26.255 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.255 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.255 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.256 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15717 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1022976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:26.256 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:22:26.256 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.678740076, 2024-09-17T21:22:26.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094586723999999997, 2024-09-17T21:22:26.256 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.256 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.256 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.256 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.256 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:26.256 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:26.256 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15717 2024-09-17T21:22:26.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.256 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:22:26.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521632+0000", 2024-09-17T21:22:26.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521633+0000", 2024-09-17T21:22:26.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:22:26.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521638+0000", 2024-09-17T21:22:26.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:22:26.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521659+0000", 2024-09-17T21:22:26.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-05 2024-09-17T21:22:26.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521675+0000", 2024-09-17T21:22:26.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5688999999999998e-05 2024-09-17T21:22:26.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616008+0000", 2024-09-17T21:22:26.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094333314000000001 2024-09-17T21:22:26.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616066+0000", 2024-09-17T21:22:26.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7797000000000002e-05 2024-09-17T21:22:26.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616219+0000", 2024-09-17T21:22:26.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015281100000000001 2024-09-17T21:22:26.259 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.259 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15718 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:22:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.67864000100001, 2024-09-17T21:22:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094703674000000002, 2024-09-17T21:22:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15718 2024-09-17T21:22:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:22:26.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521732+0000", 2024-09-17T21:22:26.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521734+0000", 2024-09-17T21:22:26.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.61e-06 2024-09-17T21:22:26.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521740+0000", 2024-09-17T21:22:26.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:22:26.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521754+0000", 2024-09-17T21:22:26.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3793e-05 2024-09-17T21:22:26.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521763+0000", 2024-09-17T21:22:26.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5129999999999996e-06 2024-09-17T21:22:26.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616256+0000", 2024-09-17T21:22:26.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094492919999999994 2024-09-17T21:22:26.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616312+0000", 2024-09-17T21:22:26.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6737000000000002e-05 2024-09-17T21:22:26.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616436+0000", 2024-09-17T21:22:26.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012335800000000001 2024-09-17T21:22:26.264 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.264 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.264 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.264 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15719 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1031168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:26.264 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:22:26.264 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.678586596, 2024-09-17T21:22:26.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094888963000000007, 2024-09-17T21:22:26.264 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.264 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.264 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.264 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.265 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:26.265 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:26.265 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15719 2024-09-17T21:22:26.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.265 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:22:26.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521785+0000", 2024-09-17T21:22:26.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521786+0000", 2024-09-17T21:22:26.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.074e-06 2024-09-17T21:22:26.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521791+0000", 2024-09-17T21:22:26.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3030000000000001e-06 2024-09-17T21:22:26.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521804+0000", 2024-09-17T21:22:26.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3027000000000001e-05 2024-09-17T21:22:26.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521811+0000", 2024-09-17T21:22:26.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6090000000000003e-06 2024-09-17T21:22:26.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616466+0000", 2024-09-17T21:22:26.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094654294 2024-09-17T21:22:26.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616523+0000", 2024-09-17T21:22:26.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6953999999999997e-05 2024-09-17T21:22:26.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616674+0000", 2024-09-17T21:22:26.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015170200000000001 2024-09-17T21:22:26.268 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.268 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.268 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.268 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15720 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1035264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:26.269 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:22:26.269 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.678549756, 2024-09-17T21:22:26.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095067061999999994, 2024-09-17T21:22:26.269 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.269 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.269 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.269 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.269 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:26.269 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:26.269 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15720 2024-09-17T21:22:26.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.269 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:22:26.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521822+0000", 2024-09-17T21:22:26.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521823+0000", 2024-09-17T21:22:26.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:22:26.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521826+0000", 2024-09-17T21:22:26.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2100000000000002e-06 2024-09-17T21:22:26.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521835+0000", 2024-09-17T21:22:26.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1630000000000002e-06 2024-09-17T21:22:26.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521842+0000", 2024-09-17T21:22:26.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:22:26.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616711+0000", 2024-09-17T21:22:26.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.094868859 2024-09-17T21:22:26.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616764+0000", 2024-09-17T21:22:26.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2778999999999997e-05 2024-09-17T21:22:26.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616889+0000", 2024-09-17T21:22:26.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012533100000000001 2024-09-17T21:22:26.272 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.272 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.273 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.273 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15721 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1039360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:26.273 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:22:26.273 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.678502647, 2024-09-17T21:22:26.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095277219999999996, 2024-09-17T21:22:26.273 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.273 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.273 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.273 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.273 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:26.273 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:26.274 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15721 2024-09-17T21:22:26.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.274 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:22:26.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521869+0000", 2024-09-17T21:22:26.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521871+0000", 2024-09-17T21:22:26.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.443e-06 2024-09-17T21:22:26.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521876+0000", 2024-09-17T21:22:26.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0300000000000001e-06 2024-09-17T21:22:26.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521886+0000", 2024-09-17T21:22:26.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0186e-05 2024-09-17T21:22:26.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521893+0000", 2024-09-17T21:22:26.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.08e-06 2024-09-17T21:22:26.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616918+0000", 2024-09-17T21:22:26.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095025223000000006 2024-09-17T21:22:26.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.616977+0000", 2024-09-17T21:22:26.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8403999999999999e-05 2024-09-17T21:22:26.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617147+0000", 2024-09-17T21:22:26.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016985399999999999 2024-09-17T21:22:26.277 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.277 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.277 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.277 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15722 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:26.277 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:26.278 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.678468198, 2024-09-17T21:22:26.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095459783000000006, 2024-09-17T21:22:26.278 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.278 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.278 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:26.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:26.278 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15722 2024-09-17T21:22:26.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.278 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:26.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:26.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521904+0000", 2024-09-17T21:22:26.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7700000000000004e-07 2024-09-17T21:22:26.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521907+0000", 2024-09-17T21:22:26.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.813e-06 2024-09-17T21:22:26.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521922+0000", 2024-09-17T21:22:26.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4533e-05 2024-09-17T21:22:26.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521934+0000", 2024-09-17T21:22:26.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2469999999999999e-05 2024-09-17T21:22:26.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617187+0000", 2024-09-17T21:22:26.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095252311000000006 2024-09-17T21:22:26.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617240+0000", 2024-09-17T21:22:26.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3740999999999999e-05 2024-09-17T21:22:26.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617364+0000", 2024-09-17T21:22:26.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-17T21:22:26.281 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.281 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.281 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.282 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5958.0:15723 3.d 3:b179c9ac:::rbd_data.1152eb799094.0000000000000004:22 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T21:22:26.282 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:22:26.282 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 123.67842433200001, 2024-09-17T21:22:26.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095653563999999996, 2024-09-17T21:22:26.282 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.282 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.282 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5958", 2024-09-17T21:22:26.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3797607589", 2024-09-17T21:22:26.282 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 15723 2024-09-17T21:22:26.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.282 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:22:26.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521948+0000", 2024-09-17T21:22:26.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521949+0000", 2024-09-17T21:22:26.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:22:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521953+0000", 2024-09-17T21:22:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5870000000000003e-06 2024-09-17T21:22:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521970+0000", 2024-09-17T21:22:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7282000000000002e-05 2024-09-17T21:22:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.521984+0000", 2024-09-17T21:22:26.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3667e-05 2024-09-17T21:22:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617383+0000", 2024-09-17T21:22:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.095399063000000006 2024-09-17T21:22:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617437+0000", 2024-09-17T21:22:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3931999999999998e-05 2024-09-17T21:22:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:22.617601+0000", 2024-09-17T21:22:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016407300000000001 2024-09-17T21:22:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.5629.0:2384 3.1c 3:38c1263e:::rbd_data.1152d37183c3.000000000000000b:head [read 11780096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-17T21:22:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:22:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 110.77559350200001, 2024-09-17T21:22:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0246185320000001, 2024-09-17T21:22:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.286 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.5629", 2024-09-17T21:22:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3798711329", 2024-09-17T21:22:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2384 2024-09-17T21:22:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:22:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424778+0000", 2024-09-17T21:22:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424780+0000", 2024-09-17T21:22:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.13e-06 2024-09-17T21:22:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424785+0000", 2024-09-17T21:22:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4739999999999996e-06 2024-09-17T21:22:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424798+0000", 2024-09-17T21:22:26.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2704000000000001e-05 2024-09-17T21:22:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424806+0000", 2024-09-17T21:22:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9899999999999997e-06 2024-09-17T21:22:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424868+0000", 2024-09-17T21:22:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999997e-05 2024-09-17T21:22:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-17T21:22:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.424875+0000", 2024-09-17T21:22:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0870000000000001e-06 2024-09-17T21:22:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428897+0000", 2024-09-17T21:22:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040226080000000004 2024-09-17T21:22:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:35.428913+0000", 2024-09-17T21:22:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:22:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.448963+0000", 2024-09-17T21:22:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0200498659999999 2024-09-17T21:22:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449158+0000", 2024-09-17T21:22:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019581000000000001 2024-09-17T21:22:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:20:36.449397+0000", 2024-09-17T21:22:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023871 2024-09-17T21:22:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4434.0:26043 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_remove_image in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-17T21:22:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:22:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.801062381999998, 2024-09-17T21:22:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.019978351, 2024-09-17T21:22:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4434", 2024-09-17T21:22:26.292 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/1571435696", 2024-09-17T21:22:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 26043 2024-09-17T21:22:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:22:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399310+0000", 2024-09-17T21:22:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399314+0000", 2024-09-17T21:22:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.963e-06 2024-09-17T21:22:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399330+0000", 2024-09-17T21:22:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6684000000000002e-05 2024-09-17T21:22:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399372+0000", 2024-09-17T21:22:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1819000000000001e-05 2024-09-17T21:22:26.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399396+0000", 2024-09-17T21:22:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.406e-05 2024-09-17T21:22:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399571+0000", 2024-09-17T21:22:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017444800000000001 2024-09-17T21:22:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-17T21:22:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:44.399595+0000", 2024-09-17T21:22:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4709999999999999e-05 2024-09-17T21:22:26.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.412735+0000", 2024-09-17T21:22:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0131394869999999 2024-09-17T21:22:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.413090+0000", 2024-09-17T21:22:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000355006 2024-09-17T21:22:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-17T21:22:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.413659+0000", 2024-09-17T21:22:26.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056922699999999995 2024-09-17T21:22:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.418940+0000", 2024-09-17T21:22:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052814109999999997 2024-09-17T21:22:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.418992+0000", 2024-09-17T21:22:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1646e-05 2024-09-17T21:22:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.419168+0000", 2024-09-17T21:22:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017573400000000001 2024-09-17T21:22:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:45.419288+0000", 2024-09-17T21:22:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012015600000000001 2024-09-17T21:22:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.298 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:26.300 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:22:26.398 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:22:26.425 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:26.426 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:26.426 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-17T21:22:26.426 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:26.440 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:22:26.558 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:26.558 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-17T21:22:26.558 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-17T21:22:26.558 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-17T21:22:26.559 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:26.572 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:22:26.693 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-17T21:22:26.693 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-17T21:22:26.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-17T21:22:26.693 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-17T21:22:26.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.693 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:387 3.2 e29/23 3:4035b4a0:::rbd_data.1152772b7dab.0000000000000001:head v 29'4, mlcod=28'2)", 2024-09-17T21:22:26.693 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:22:26.693 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 245.084504426, 2024-09-17T21:22:26.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035208995, 2024-09-17T21:22:26.693 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.693 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.694 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:26.694 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:22:26.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604190+0000", 2024-09-17T21:22:26.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.604193+0000", 2024-09-17T21:22:26.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.553e-06 2024-09-17T21:22:26.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613145+0000", 2024-09-17T21:22:26.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089524380000000001 2024-09-17T21:22:26.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613170+0000", 2024-09-17T21:22:26.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4939999999999998e-05 2024-09-17T21:22:26.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613188+0000", 2024-09-17T21:22:26.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7560000000000001e-05 2024-09-17T21:22:26.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613321+0000", 2024-09-17T21:22:26.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013288500000000001 2024-09-17T21:22:26.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.613359+0000", 2024-09-17T21:22:26.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7979999999999999e-05 2024-09-17T21:22:26.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:26.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639308+0000", 2024-09-17T21:22:26.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02594919 2024-09-17T21:22:26.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.639399+0000", 2024-09-17T21:22:26.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1448999999999998e-05 2024-09-17T21:22:26.697 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.697 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.697 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.698 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4447.0:404 3.2 e29/23 3:460d9110:::rbd_data.1152772b7dab.0000000000000004:head v 29'5, mlcod=28'2)", 2024-09-17T21:22:26.698 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:22:26.698 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 245.07455779, 2024-09-17T21:22:26.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027228347, 2024-09-17T21:22:26.698 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.698 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.698 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:26.698 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:22:26.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614137+0000", 2024-09-17T21:22:26.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.614139+0000", 2024-09-17T21:22:26.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:22:26.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622686+0000", 2024-09-17T21:22:26.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085475870000000006 2024-09-17T21:22:26.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622730+0000", 2024-09-17T21:22:26.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.333e-05 2024-09-17T21:22:26.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.622753+0000", 2024-09-17T21:22:26.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3067000000000001e-05 2024-09-17T21:22:26.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628352+0000", 2024-09-17T21:22:26.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055992630000000002 2024-09-17T21:22:26.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.628422+0000", 2024-09-17T21:22:26.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9626e-05 2024-09-17T21:22:26.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:26.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641290+0000", 2024-09-17T21:22:26.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012868137 2024-09-17T21:22:26.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:21.641365+0000", 2024-09-17T21:22:26.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5582999999999998e-05 2024-09-17T21:22:26.702 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.702 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.702 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.702 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:540 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:22:26.702 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:22:26.702 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 243.83140695, 2024-09-17T21:22:26.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041839693999999997, 2024-09-17T21:22:26.702 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.702 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.702 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:26.703 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.703 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:22:26.703 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:22:26.703 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 540 2024-09-17T21:22:26.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.703 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:22:26.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857288+0000", 2024-09-17T21:22:26.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.857290+0000", 2024-09-17T21:22:26.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:22:26.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866848+0000", 2024-09-17T21:22:26.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095584490000000001 2024-09-17T21:22:26.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866879+0000", 2024-09-17T21:22:26.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0309999999999999e-05 2024-09-17T21:22:26.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866894+0000", 2024-09-17T21:22:26.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5044e-05 2024-09-17T21:22:26.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867047+0000", 2024-09-17T21:22:26.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015359799999999999 2024-09-17T21:22:26.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.867427+0000", 2024-09-17T21:22:26.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00038013400000000002 2024-09-17T21:22:26.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:22:26.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.868224+0000", 2024-09-17T21:22:26.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079714399999999996 2024-09-17T21:22:26.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:26.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.895298+0000", 2024-09-17T21:22:26.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027073644000000001 2024-09-17T21:22:26.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:26.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.898828+0000", 2024-09-17T21:22:26.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035300290000000001 2024-09-17T21:22:26.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:26.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899002+0000", 2024-09-17T21:22:26.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017420900000000001 2024-09-17T21:22:26.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899128+0000", 2024-09-17T21:22:26.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012520899999999999 2024-09-17T21:22:26.707 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.708 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.708 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.708 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4447.0:541 3.5 3:a0ff5db5:::rbd_data.1152772b7dab.0000000000000000:head [truncate 15728640] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:22:26.708 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:22:26.708 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 243.82176146399999, 2024-09-17T21:22:26.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033303143, 2024-09-17T21:22:26.708 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.708 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.708 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:26.708 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.708 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4447", 2024-09-17T21:22:26.708 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/3273249727", 2024-09-17T21:22:26.709 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 541 2024-09-17T21:22:26.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.709 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:22:26.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866933+0000", 2024-09-17T21:22:26.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866935+0000", 2024-09-17T21:22:26.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.64e-06 2024-09-17T21:22:26.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866940+0000", 2024-09-17T21:22:26.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:22:26.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866955+0000", 2024-09-17T21:22:26.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5273000000000001e-05 2024-09-17T21:22:26.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.866964+0000", 2024-09-17T21:22:26.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8939999999999999e-06 2024-09-17T21:22:26.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887123+0000", 2024-09-17T21:22:26.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020158283999999999 2024-09-17T21:22:26.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887283+0000", 2024-09-17T21:22:26.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000160629 2024-09-17T21:22:26.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-17T21:22:26.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.887562+0000", 2024-09-17T21:22:26.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027887099999999998 2024-09-17T21:22:26.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-17T21:22:26.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.896548+0000", 2024-09-17T21:22:26.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0089861109999999998 2024-09-17T21:22:26.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:22:26.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.899989+0000", 2024-09-17T21:22:26.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003441127 2024-09-17T21:22:26.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:26.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900130+0000", 2024-09-17T21:22:26.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014043500000000001 2024-09-17T21:22:26.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:22.900237+0000", 2024-09-17T21:22:26.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106736 2024-09-17T21:22:26.713 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.713 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.713 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.714 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4737.0:472 3.e 3:72806d41:::rbd_data.11527824718.0000000000000001:9 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:22:26.714 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:22:26.714 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 230.45282543100001, 2024-09-17T21:22:26.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024663205000000001, 2024-09-17T21:22:26.714 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.714 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.714 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.714 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.714 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4737", 2024-09-17T21:22:26.714 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/403932564", 2024-09-17T21:22:26.714 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 472 2024-09-17T21:22:26.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.714 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:22:26.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235869+0000", 2024-09-17T21:22:26.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.235871+0000", 2024-09-17T21:22:26.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.344e-06 2024-09-17T21:22:26.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236536+0000", 2024-09-17T21:22:26.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00066564499999999997 2024-09-17T21:22:26.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236567+0000", 2024-09-17T21:22:26.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0320000000000001e-05 2024-09-17T21:22:26.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.236586+0000", 2024-09-17T21:22:26.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9029999999999999e-05 2024-09-17T21:22:26.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.259991+0000", 2024-09-17T21:22:26.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023404924000000001 2024-09-17T21:22:26.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260237+0000", 2024-09-17T21:22:26.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024677199999999997 2024-09-17T21:22:26.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:36.260533+0000", 2024-09-17T21:22:26.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029516999999999998 2024-09-17T21:22:26.718 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.718 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.718 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.718 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:820 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:26.718 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:22:26.718 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 212.25114086900001, 2024-09-17T21:22:26.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024650313, 2024-09-17T21:22:26.718 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.718 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.718 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.719 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.719 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:26.719 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:26.719 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 820 2024-09-17T21:22:26.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.719 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:22:26.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437554+0000", 2024-09-17T21:22:26.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437556+0000", 2024-09-17T21:22:26.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.703e-06 2024-09-17T21:22:26.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437562+0000", 2024-09-17T21:22:26.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:22:26.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437580+0000", 2024-09-17T21:22:26.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958999999999999e-05 2024-09-17T21:22:26.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.437594+0000", 2024-09-17T21:22:26.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3739e-05 2024-09-17T21:22:26.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.461844+0000", 2024-09-17T21:22:26.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024249607999999999 2024-09-17T21:22:26.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462010+0000", 2024-09-17T21:22:26.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016612500000000001 2024-09-17T21:22:26.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462204+0000", 2024-09-17T21:22:26.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194475 2024-09-17T21:22:26.722 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.722 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.722 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.723 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:834 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:26.723 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:22:26.723 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 212.250646292, 2024-09-17T21:22:26.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024638200999999998, 2024-09-17T21:22:26.723 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.723 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.723 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.723 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.723 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:26.723 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:26.723 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 834 2024-09-17T21:22:26.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.723 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:22:26.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438049+0000", 2024-09-17T21:22:26.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438050+0000", 2024-09-17T21:22:26.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6300000000000001e-06 2024-09-17T21:22:26.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438056+0000", 2024-09-17T21:22:26.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-06 2024-09-17T21:22:26.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438073+0000", 2024-09-17T21:22:26.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7149000000000001e-05 2024-09-17T21:22:26.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438086+0000", 2024-09-17T21:22:26.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3286e-05 2024-09-17T21:22:26.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462429+0000", 2024-09-17T21:22:26.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024342458000000001 2024-09-17T21:22:26.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462523+0000", 2024-09-17T21:22:26.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4581999999999997e-05 2024-09-17T21:22:26.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462687+0000", 2024-09-17T21:22:26.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163346 2024-09-17T21:22:26.727 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.727 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.727 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.727 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:835 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:26.727 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:22:26.727 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 212.25059091099999, 2024-09-17T21:22:26.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024850765, 2024-09-17T21:22:26.727 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.727 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.727 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.727 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.728 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:26.728 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:26.728 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 835 2024-09-17T21:22:26.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.728 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:22:26.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438104+0000", 2024-09-17T21:22:26.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438105+0000", 2024-09-17T21:22:26.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-17T21:22:26.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438110+0000", 2024-09-17T21:22:26.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:22:26.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438130+0000", 2024-09-17T21:22:26.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0245999999999999e-05 2024-09-17T21:22:26.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438143+0000", 2024-09-17T21:22:26.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2393e-05 2024-09-17T21:22:26.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462751+0000", 2024-09-17T21:22:26.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024608070999999999 2024-09-17T21:22:26.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462851+0000", 2024-09-17T21:22:26.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100101 2024-09-17T21:22:26.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.462955+0000", 2024-09-17T21:22:26.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103704 2024-09-17T21:22:26.731 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.731 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.731 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.731 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:836 3.17 3:ea18ea14:::rbd_data.11523651b015.0000000000000002:d [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:26.732 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:22:26.732 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 212.250534633, 2024-09-17T21:22:26.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025036428999999999, 2024-09-17T21:22:26.732 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.732 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.732 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.732 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.732 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:26.732 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:26.732 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 836 2024-09-17T21:22:26.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.732 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:22:26.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438160+0000", 2024-09-17T21:22:26.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438162+0000", 2024-09-17T21:22:26.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.347e-06 2024-09-17T21:22:26.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438167+0000", 2024-09-17T21:22:26.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.186e-06 2024-09-17T21:22:26.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438205+0000", 2024-09-17T21:22:26.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8148000000000001e-05 2024-09-17T21:22:26.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.438767+0000", 2024-09-17T21:22:26.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056168199999999998 2024-09-17T21:22:26.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463032+0000", 2024-09-17T21:22:26.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024265056 2024-09-17T21:22:26.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463101+0000", 2024-09-17T21:22:26.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9678999999999995e-05 2024-09-17T21:22:26.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463197+0000", 2024-09-17T21:22:26.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5330999999999999e-05 2024-09-17T21:22:26.736 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.736 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.736 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.736 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:842 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:26.736 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:22:26.736 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 212.249584994, 2024-09-17T21:22:26.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024577694000000001, 2024-09-17T21:22:26.736 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.736 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.736 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.737 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.737 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:26.737 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:26.737 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 842 2024-09-17T21:22:26.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.737 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:22:26.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439110+0000", 2024-09-17T21:22:26.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439111+0000", 2024-09-17T21:22:26.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.226e-06 2024-09-17T21:22:26.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439131+0000", 2024-09-17T21:22:26.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9513000000000001e-05 2024-09-17T21:22:26.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439149+0000", 2024-09-17T21:22:26.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8703e-05 2024-09-17T21:22:26.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439168+0000", 2024-09-17T21:22:26.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8776000000000001e-05 2024-09-17T21:22:26.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463231+0000", 2024-09-17T21:22:26.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024062481 2024-09-17T21:22:26.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463397+0000", 2024-09-17T21:22:26.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016633699999999999 2024-09-17T21:22:26.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463688+0000", 2024-09-17T21:22:26.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00029065800000000001 2024-09-17T21:22:26.740 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.740 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.740 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.741 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:843 3.5 3:a6bbadb5:::rbd_data.11523651b015.0000000000000004:d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:26.741 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:22:26.741 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 212.24950637399999, 2024-09-17T21:22:26.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024962662, 2024-09-17T21:22:26.741 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.741 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.741 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.741 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.741 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:26.741 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:26.741 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 843 2024-09-17T21:22:26.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.741 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:22:26.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439188+0000", 2024-09-17T21:22:26.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439190+0000", 2024-09-17T21:22:26.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.68e-06 2024-09-17T21:22:26.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439195+0000", 2024-09-17T21:22:26.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:22:26.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439213+0000", 2024-09-17T21:22:26.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7612e-05 2024-09-17T21:22:26.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439226+0000", 2024-09-17T21:22:26.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-05 2024-09-17T21:22:26.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463755+0000", 2024-09-17T21:22:26.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024528094 2024-09-17T21:22:26.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463943+0000", 2024-09-17T21:22:26.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018853300000000001 2024-09-17T21:22:26.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464151+0000", 2024-09-17T21:22:26.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020805300000000001 2024-09-17T21:22:26.745 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.745 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.745 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.745 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:848 3.17 3:eec233af:::rbd_data.11523651b015.000000000000000b:d [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:26.745 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:22:26.745 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 212.249393435, 2024-09-17T21:22:26.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024714699, 2024-09-17T21:22:26.745 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.746 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.746 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.746 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.746 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:26.746 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:26.746 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 848 2024-09-17T21:22:26.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.746 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:22:26.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439301+0000", 2024-09-17T21:22:26.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439303+0000", 2024-09-17T21:22:26.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.28e-06 2024-09-17T21:22:26.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439327+0000", 2024-09-17T21:22:26.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4433000000000001e-05 2024-09-17T21:22:26.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439382+0000", 2024-09-17T21:22:26.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4873999999999997e-05 2024-09-17T21:22:26.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439864+0000", 2024-09-17T21:22:26.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000482134 2024-09-17T21:22:26.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463499+0000", 2024-09-17T21:22:26.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023635171999999999 2024-09-17T21:22:26.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.463736+0000", 2024-09-17T21:22:26.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023640400000000001 2024-09-17T21:22:26.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464016+0000", 2024-09-17T21:22:26.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028040200000000002 2024-09-17T21:22:26.749 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.749 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.749 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.750 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:850 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:26.750 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:22:26.750 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 212.248732381, 2024-09-17T21:22:26.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025087027000000001, 2024-09-17T21:22:26.750 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.750 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.750 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.750 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.750 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:26.750 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:26.750 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 850 2024-09-17T21:22:26.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.751 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:22:26.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439962+0000", 2024-09-17T21:22:26.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439964+0000", 2024-09-17T21:22:26.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.756e-06 2024-09-17T21:22:26.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439971+0000", 2024-09-17T21:22:26.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.55e-06 2024-09-17T21:22:26.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.439989+0000", 2024-09-17T21:22:26.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8263000000000001e-05 2024-09-17T21:22:26.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440003+0000", 2024-09-17T21:22:26.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3709e-05 2024-09-17T21:22:26.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464583+0000", 2024-09-17T21:22:26.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024580444 2024-09-17T21:22:26.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.464784+0000", 2024-09-17T21:22:26.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000201089 2024-09-17T21:22:26.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465049+0000", 2024-09-17T21:22:26.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00026521599999999998 2024-09-17T21:22:26.754 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.754 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.754 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.754 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:851 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:26.754 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:22:26.754 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 212.24866757000001, 2024-09-17T21:22:26.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025509542999999999, 2024-09-17T21:22:26.754 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.754 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.755 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.755 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.755 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:26.755 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:26.755 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 851 2024-09-17T21:22:26.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.755 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:22:26.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440027+0000", 2024-09-17T21:22:26.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440029+0000", 2024-09-17T21:22:26.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:22:26.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440034+0000", 2024-09-17T21:22:26.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:22:26.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440052+0000", 2024-09-17T21:22:26.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7796e-05 2024-09-17T21:22:26.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440069+0000", 2024-09-17T21:22:26.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7711999999999999e-05 2024-09-17T21:22:26.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465146+0000", 2024-09-17T21:22:26.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025076314999999998 2024-09-17T21:22:26.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465316+0000", 2024-09-17T21:22:26.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169987 2024-09-17T21:22:26.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465537+0000", 2024-09-17T21:22:26.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022108699999999999 2024-09-17T21:22:26.758 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.758 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.758 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.758 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4923.0:852 3.17 3:e80d5756:::rbd_data.11523651b015.000000000000000c:d [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-17T21:22:26.759 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:22:26.759 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 212.24860659300001, 2024-09-17T21:22:26.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025958680000000001, 2024-09-17T21:22:26.759 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.759 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.759 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-17T21:22:26.759 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-17T21:22:26.759 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4923", 2024-09-17T21:22:26.759 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.184:0/106754937", 2024-09-17T21:22:26.759 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 852 2024-09-17T21:22:26.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.759 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:22:26.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440088+0000", 2024-09-17T21:22:26.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440090+0000", 2024-09-17T21:22:26.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.637e-06 2024-09-17T21:22:26.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440095+0000", 2024-09-17T21:22:26.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2630000000000003e-06 2024-09-17T21:22:26.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440113+0000", 2024-09-17T21:22:26.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7723e-05 2024-09-17T21:22:26.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.440126+0000", 2024-09-17T21:22:26.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3559e-05 2024-09-17T21:22:26.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465630+0000", 2024-09-17T21:22:26.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025503867999999999 2024-09-17T21:22:26.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.465824+0000", 2024-09-17T21:22:26.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000193692 2024-09-17T21:22:26.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:18:54.466047+0000", 2024-09-17T21:22:26.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000222938 2024-09-17T21:22:26.762 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.763 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.763 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.763 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:1858 3.e e97/63 3:7397009a:::rbd_data.115266d0b9f2.0000000000000005:head v 97'314, mlcod=90'311)", 2024-09-17T21:22:26.763 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:22:26.763 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 157.351035408, 2024-09-17T21:22:26.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027115837, 2024-09-17T21:22:26.763 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.763 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.763 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:26.763 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:22:26.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337659+0000", 2024-09-17T21:22:26.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.337660+0000", 2024-09-17T21:22:26.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-17T21:22:26.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344049+0000", 2024-09-17T21:22:26.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388375 2024-09-17T21:22:26.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344069+0000", 2024-09-17T21:22:26.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0455e-05 2024-09-17T21:22:26.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344080+0000", 2024-09-17T21:22:26.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1015e-05 2024-09-17T21:22:26.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344150+0000", 2024-09-17T21:22:26.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:22:26.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.344174+0000", 2024-09-17T21:22:26.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3227e-05 2024-09-17T21:22:26.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:26.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364714+0000", 2024-09-17T21:22:26.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020539992 2024-09-17T21:22:26.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.364775+0000", 2024-09-17T21:22:26.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1765000000000002e-05 2024-09-17T21:22:26.767 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.767 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.767 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.767 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5541.0:2647 3.e e97/63 3:75707487:::rbd_data.115266d0b9f2.0000000000000004:head v 97'318, mlcod=97'316)", 2024-09-17T21:22:26.767 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:22:26.767 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 157.142377014, 2024-09-17T21:22:26.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025773154999999999, 2024-09-17T21:22:26.767 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.768 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.768 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:26.768 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:22:26.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546318+0000", 2024-09-17T21:22:26.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.546320+0000", 2024-09-17T21:22:26.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:22:26.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552534+0000", 2024-09-17T21:22:26.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062142400000000002 2024-09-17T21:22:26.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552557+0000", 2024-09-17T21:22:26.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3319000000000002e-05 2024-09-17T21:22:26.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552567+0000", 2024-09-17T21:22:26.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079e-05 2024-09-17T21:22:26.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552639+0000", 2024-09-17T21:22:26.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1935999999999997e-05 2024-09-17T21:22:26.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.552661+0000", 2024-09-17T21:22:26.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999999e-05 2024-09-17T21:22:26.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:26.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572001+0000", 2024-09-17T21:22:26.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019340695000000001 2024-09-17T21:22:26.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:19:49.572091+0000", 2024-09-17T21:22:26.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9761999999999999e-05 2024-09-17T21:22:26.771 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.771 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.771 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.771 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:9547 3.2 e147/132 3:42b518d9:::rbd_data.1152f056b73f.0000000000000001:head v 147'513, mlcod=142'511)", 2024-09-17T21:22:26.772 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:22:26.772 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 84.217403891999993, 2024-09-17T21:22:26.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.031059566, 2024-09-17T21:22:26.772 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.772 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.772 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:26.772 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:22:26.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471291+0000", 2024-09-17T21:22:26.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.471293+0000", 2024-09-17T21:22:26.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2029999999999999e-06 2024-09-17T21:22:26.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480606+0000", 2024-09-17T21:22:26.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093123839999999999 2024-09-17T21:22:26.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480653+0000", 2024-09-17T21:22:26.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7852000000000002e-05 2024-09-17T21:22:26.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480673+0000", 2024-09-17T21:22:26.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0013e-05 2024-09-17T21:22:26.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480862+0000", 2024-09-17T21:22:26.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018833599999999999 2024-09-17T21:22:26.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.480906+0000", 2024-09-17T21:22:26.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4162000000000002e-05 2024-09-17T21:22:26.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:26.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502259+0000", 2024-09-17T21:22:26.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021353154999999999 2024-09-17T21:22:26.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:02.502351+0000", 2024-09-17T21:22:26.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1460999999999997e-05 2024-09-17T21:22:26.776 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.776 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.776 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.776 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:15234 3.1 e147/134 3:8526882c:::rbd_data.1152f056b73f.0000000000000000:head v 147'917, mlcod=145'914)", 2024-09-17T21:22:26.776 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:22:26.776 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.584743274000004, 2024-09-17T21:22:26.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045303695999999997, 2024-09-17T21:22:26.776 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.776 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.776 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:26.776 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:22:26.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103952+0000", 2024-09-17T21:22:26.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.103954+0000", 2024-09-17T21:22:26.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2259999999999999e-06 2024-09-17T21:22:26.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119105+0000", 2024-09-17T21:22:26.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015150871 2024-09-17T21:22:26.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119141+0000", 2024-09-17T21:22:26.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6239000000000003e-05 2024-09-17T21:22:26.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119159+0000", 2024-09-17T21:22:26.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7946999999999999e-05 2024-09-17T21:22:26.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119344+0000", 2024-09-17T21:22:26.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018524599999999999 2024-09-17T21:22:26.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.119399+0000", 2024-09-17T21:22:26.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4922e-05 2024-09-17T21:22:26.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:26.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149165+0000", 2024-09-17T21:22:26.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.029765697000000001 2024-09-17T21:22:26.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.149255+0000", 2024-09-17T21:22:26.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0548000000000001e-05 2024-09-17T21:22:26.780 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.780 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.780 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.780 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.5890.0:19859 3.8 e147/133 3:12f72409:::rbd_data.1152f056b73f.0000000000000003:head v 147'482, mlcod=145'479)", 2024-09-17T21:22:26.780 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:22:26.780 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.175852207000005, 2024-09-17T21:22:26.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.033476895999999999, 2024-09-17T21:22:26.780 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-17T21:22:26.781 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-17T21:22:26.781 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:22:26.781 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-17T21:22:26.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-17T21:22:26.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:22:26.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-17T21:22:26.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512843+0000", 2024-09-17T21:22:26.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-17T21:22:26.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-17T21:22:26.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.512845+0000", 2024-09-17T21:22:26.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.886e-06 2024-09-17T21:22:26.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-17T21:22:26.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524104+0000", 2024-09-17T21:22:26.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011259668 2024-09-17T21:22:26.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-17T21:22:26.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524148+0000", 2024-09-17T21:22:26.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3386000000000001e-05 2024-09-17T21:22:26.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-17T21:22:26.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524172+0000", 2024-09-17T21:22:26.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4762000000000001e-05 2024-09-17T21:22:26.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-17T21:22:26.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524379+0000", 2024-09-17T21:22:26.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020671599999999999 2024-09-17T21:22:26.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-17T21:22:26.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.524421+0000", 2024-09-17T21:22:26.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2182999999999999e-05 2024-09-17T21:22:26.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-17T21:22:26.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546216+0000", 2024-09-17T21:22:26.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.021795101000000001 2024-09-17T21:22:26.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-17T21:22:26.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-17T21:22:26.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-17T21:22:26.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-17T21:21:03.546320+0000", 2024-09-17T21:22:26.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103194 2024-09-17T21:22:26.784 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.784 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.784 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.784 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-17T21:22:26.784 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-17T21:22:26.785 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-17T21:22:27.313 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-17T21:22:27.341 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:22:28.392 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-17T21:22:30.280 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:22:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-17T21:22:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-17T21:22:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-17T21:22:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-17T21:22:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-17T21:22:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-17T21:22:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs debug_inject_read_zeros 2024-09-17T21:22:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs files list 2024-09-17T21:22:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs stats 2024-09-17T21:22:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator dump block 2024-09-17T21:22:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator fragmentation block 2024-09-17T21:22:30.403 INFO:teuthology.orchestra.run.smithi083.stderr:admin_socket: invalid command 2024-09-17T21:22:30.404 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T21:22:30.405 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-17T21:22:30.560 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:22:30.554+0000 7fb243f7f7c0 -1 osd.2 188 log_to_monitors true 2024-09-17T21:22:31.430 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T21:22:31.426+0000 7fb237d5a640 -1 osd.2 230 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T21:22:35.406 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:22:35.537 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:22:35.537 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T21:22:35.537 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T21:22:35.537 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:22:35.551 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-17T21:22:35.552 INFO:tasks.thrashosds.thrasher:Adding osd 5 2024-09-17T21:22:35.552 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 5 2024-09-17T21:22:35.866 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-09-17T21:22:35.862+0000 7fe37768f640 -1 mon.a@0(leader).osd e231 definitely_dead 0 2024-09-17T21:22:36.672 INFO:teuthology.orchestra.run.smithi083.stderr:marked in osd.5. 2024-09-17T21:22:36.702 INFO:tasks.thrashosds.thrasher:Added osd 5 2024-09-17T21:22:36.702 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-17T21:22:36.702 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-17T21:22:36.703 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-17T21:22:37.015 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-09-17T21:22:37.010+0000 7fe37768f640 -1 mon.a@0(leader).osd e232 definitely_dead 0 2024-09-17T21:22:37.316 INFO:teuthology.orchestra.run.smithi083.stderr:marked in osd.2. 2024-09-17T21:22:37.351 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-17T21:22:37.351 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-17T21:22:37.472 INFO:teuthology.orchestra.run.smithi083.stdout:mon.a: {} 2024-09-17T21:22:37.473 INFO:teuthology.orchestra.run.smithi083.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T21:22:37.485 INFO:teuthology.orchestra.run.smithi083.stdout:mon.b: {} 2024-09-17T21:22:37.485 INFO:teuthology.orchestra.run.smithi083.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T21:22:37.494 INFO:teuthology.orchestra.run.smithi083.stdout:mon.c: {} 2024-09-17T21:22:37.494 INFO:teuthology.orchestra.run.smithi083.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T21:22:37.509 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-17T21:22:38.325 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.0 to 1 (10000) 2024-09-17T21:22:38.372 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-17T21:22:39.328 INFO:teuthology.orchestra.run.smithi083.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-17T21:22:39.369 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-17T21:22:40.343 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T21:22:40.373 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-17T21:22:41.338 INFO:teuthology.orchestra.run.smithi083.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-17T21:22:41.367 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-17T21:22:42.318 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T21:22:42.349 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-17T21:22:43.336 INFO:teuthology.orchestra.run.smithi083.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-17T21:22:43.363 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-17T21:22:44.346 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T21:22:44.369 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-17T21:22:45.342 INFO:teuthology.orchestra.run.smithi083.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-17T21:22:45.374 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-17T21:22:46.337 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T21:22:46.375 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-17T21:22:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-17T21:22:47.397 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-17T21:22:48.371 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.2 to 1 (10000) 2024-09-17T21:22:48.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-17T21:22:49.373 INFO:teuthology.orchestra.run.smithi083.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-17T21:22:49.405 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-17T21:22:49.405 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-17T21:22:49.405 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-17T21:22:49.405 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-17T21:22:49.405 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T21:22:49.405 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T21:22:49.712 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:22:49.713 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":245,"fsid":"224a7a29-e112-472e-8a13-f67345d6d123","created":"2024-09-17T21:17:42.597312+0000","modified":"2024-09-17T21:22:49.370195+0000","last_up_change":"2024-09-17T21:22:32.395031+0000","last_in_change":"2024-09-17T21:22:37.312336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T21:17:55.581350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T21:17:59.603025+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-17T21:18:09.441137+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"227","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":227,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5bfd41c9-0cac-47c4-9a6e-8dce774c3314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6817","nonce":1111036754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6819","nonce":1111036754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6823","nonce":1111036754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6821","nonce":1111036754}]},"public_addr":"172.21.15.83:6817/1111036754","cluster_addr":"172.21.15.83:6819/1111036754","heartbeat_back_addr":"172.21.15.83:6823/1111036754","heartbeat_front_addr":"172.21.15.83:6821/1111036754","state":["exists","up"]},{"osd":1,"uuid":"78404e0d-d397-4806-8e90-5f32182643bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6809","nonce":1460363937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6811","nonce":1460363937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6815","nonce":1460363937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6813","nonce":1460363937}]},"public_addr":"172.21.15.83:6809/1460363937","cluster_addr":"172.21.15.83:6811/1460363937","heartbeat_back_addr":"172.21.15.83:6815/1460363937","heartbeat_front_addr":"172.21.15.83:6813/1460363937","state":["exists","up"]},{"osd":2,"uuid":"286ed6e0-c121-4ebb-8cf6-37e506a397aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":12,"last_clean_end":188,"up_from":231,"up_thru":243,"down_at":189,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3213289387},{"type":"v1","addr":"172.21.15.83:6801","nonce":3213289387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3213289387},{"type":"v1","addr":"172.21.15.83:6803","nonce":3213289387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3213289387},{"type":"v1","addr":"172.21.15.83:6807","nonce":3213289387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3213289387},{"type":"v1","addr":"172.21.15.83:6805","nonce":3213289387}]},"public_addr":"172.21.15.83:6801/3213289387","cluster_addr":"172.21.15.83:6803/3213289387","heartbeat_back_addr":"172.21.15.83:6807/3213289387","heartbeat_front_addr":"172.21.15.83:6805/3213289387","state":["exists","up"]},{"osd":3,"uuid":"b0af73a1-4bca-4892-a550-2ced87f4c357","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6809","nonce":3637204865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6811","nonce":3637204865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6815","nonce":3637204865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6813","nonce":3637204865}]},"public_addr":"172.21.15.97:6809/3637204865","cluster_addr":"172.21.15.97:6811/3637204865","heartbeat_back_addr":"172.21.15.97:6815/3637204865","heartbeat_front_addr":"172.21.15.97:6813/3637204865","state":["exists","up"]},{"osd":4,"uuid":"2188361c-e1a1-45fa-bc01-da5a984ffd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6801","nonce":2391356911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6803","nonce":2391356911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6807","nonce":2391356911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6805","nonce":2391356911}]},"public_addr":"172.21.15.97:6801/2391356911","cluster_addr":"172.21.15.97:6803/2391356911","heartbeat_back_addr":"172.21.15.97:6807/2391356911","heartbeat_front_addr":"172.21.15.97:6805/2391356911","state":["exists","up"]},{"osd":5,"uuid":"7feccf4d-d8e4-4d06-ac83-bbc1bc623a5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6817","nonce":2326316381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6819","nonce":2326316381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6823","nonce":2326316381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6821","nonce":2326316381}]},"public_addr":"172.21.15.97:6817/2326316381","cluster_addr":"172.21.15.97:6819/2326316381","heartbeat_back_addr":"172.21.15.97:6823/2326316381","heartbeat_front_addr":"172.21.15.97:6821/2326316381","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:51.138101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.581632+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.365193+0000","dead_epoch":189},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.718974+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.501878+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:51.150747+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"1.0","osds":[4,3]},{"pgid":"3.8","osds":[2,3]},{"pgid":"3.c","osds":[1,5]},{"pgid":"3.1e","osds":[0,4]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"3.19","osds":[1]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:22:49.731 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T21:22:49.731 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T21:22:50.040 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:22:50.040 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":245,"fsid":"224a7a29-e112-472e-8a13-f67345d6d123","created":"2024-09-17T21:17:42.597312+0000","modified":"2024-09-17T21:22:49.370195+0000","last_up_change":"2024-09-17T21:22:32.395031+0000","last_in_change":"2024-09-17T21:22:37.312336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T21:17:55.581350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T21:17:59.603025+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-17T21:18:09.441137+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"227","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":227,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5bfd41c9-0cac-47c4-9a6e-8dce774c3314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6817","nonce":1111036754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6819","nonce":1111036754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6823","nonce":1111036754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6821","nonce":1111036754}]},"public_addr":"172.21.15.83:6817/1111036754","cluster_addr":"172.21.15.83:6819/1111036754","heartbeat_back_addr":"172.21.15.83:6823/1111036754","heartbeat_front_addr":"172.21.15.83:6821/1111036754","state":["exists","up"]},{"osd":1,"uuid":"78404e0d-d397-4806-8e90-5f32182643bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6809","nonce":1460363937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6811","nonce":1460363937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6815","nonce":1460363937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6813","nonce":1460363937}]},"public_addr":"172.21.15.83:6809/1460363937","cluster_addr":"172.21.15.83:6811/1460363937","heartbeat_back_addr":"172.21.15.83:6815/1460363937","heartbeat_front_addr":"172.21.15.83:6813/1460363937","state":["exists","up"]},{"osd":2,"uuid":"286ed6e0-c121-4ebb-8cf6-37e506a397aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":12,"last_clean_end":188,"up_from":231,"up_thru":243,"down_at":189,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3213289387},{"type":"v1","addr":"172.21.15.83:6801","nonce":3213289387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3213289387},{"type":"v1","addr":"172.21.15.83:6803","nonce":3213289387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3213289387},{"type":"v1","addr":"172.21.15.83:6807","nonce":3213289387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3213289387},{"type":"v1","addr":"172.21.15.83:6805","nonce":3213289387}]},"public_addr":"172.21.15.83:6801/3213289387","cluster_addr":"172.21.15.83:6803/3213289387","heartbeat_back_addr":"172.21.15.83:6807/3213289387","heartbeat_front_addr":"172.21.15.83:6805/3213289387","state":["exists","up"]},{"osd":3,"uuid":"b0af73a1-4bca-4892-a550-2ced87f4c357","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6809","nonce":3637204865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6811","nonce":3637204865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6815","nonce":3637204865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6813","nonce":3637204865}]},"public_addr":"172.21.15.97:6809/3637204865","cluster_addr":"172.21.15.97:6811/3637204865","heartbeat_back_addr":"172.21.15.97:6815/3637204865","heartbeat_front_addr":"172.21.15.97:6813/3637204865","state":["exists","up"]},{"osd":4,"uuid":"2188361c-e1a1-45fa-bc01-da5a984ffd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6801","nonce":2391356911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6803","nonce":2391356911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6807","nonce":2391356911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6805","nonce":2391356911}]},"public_addr":"172.21.15.97:6801/2391356911","cluster_addr":"172.21.15.97:6803/2391356911","heartbeat_back_addr":"172.21.15.97:6807/2391356911","heartbeat_front_addr":"172.21.15.97:6805/2391356911","state":["exists","up"]},{"osd":5,"uuid":"7feccf4d-d8e4-4d06-ac83-bbc1bc623a5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6817","nonce":2326316381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6819","nonce":2326316381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6823","nonce":2326316381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6821","nonce":2326316381}]},"public_addr":"172.21.15.97:6817/2326316381","cluster_addr":"172.21.15.97:6819/2326316381","heartbeat_back_addr":"172.21.15.97:6823/2326316381","heartbeat_front_addr":"172.21.15.97:6821/2326316381","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:51.138101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.581632+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.365193+0000","dead_epoch":189},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.718974+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.501878+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:51.150747+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"1.0","osds":[4,3]},{"pgid":"3.8","osds":[2,3]},{"pgid":"3.c","osds":[1,5]},{"pgid":"3.1e","osds":[0,4]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"3.19","osds":[1]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:22:50.059 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T21:22:50.060 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T21:22:50.060 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T21:22:50.061 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T21:22:50.061 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T21:22:50.061 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T21:22:50.226 INFO:teuthology.orchestra.run.smithi083.stdout:992137445381 2024-09-17T21:22:50.226 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T21:22:50.311 INFO:teuthology.orchestra.run.smithi083.stdout:55834574911 2024-09-17T21:22:50.311 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T21:22:50.323 INFO:teuthology.orchestra.run.smithi083.stdout:55834574911 2024-09-17T21:22:50.323 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T21:22:50.329 INFO:teuthology.orchestra.run.smithi083.stdout:55834574911 2024-09-17T21:22:50.330 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T21:22:50.335 INFO:teuthology.orchestra.run.smithi083.stdout:55834574912 2024-09-17T21:22:50.336 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T21:22:50.344 INFO:teuthology.orchestra.run.smithi083.stdout:55834574911 2024-09-17T21:22:50.345 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T21:22:50.647 INFO:teuthology.orchestra.run.smithi083.stdout:992137445380 2024-09-17T21:22:50.672 INFO:tasks.ceph.ceph_manager.ceph:need seq 992137445381 got 992137445380 for osd.2 2024-09-17T21:22:50.702 INFO:teuthology.orchestra.run.smithi083.stdout:55834574910 2024-09-17T21:22:50.710 INFO:teuthology.orchestra.run.smithi083.stdout:55834574910 2024-09-17T21:22:50.721 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574911 got 55834574910 for osd.3 2024-09-17T21:22:50.728 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574911 got 55834574910 for osd.5 2024-09-17T21:22:50.781 INFO:teuthology.orchestra.run.smithi083.stdout:55834574910 2024-09-17T21:22:50.804 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574911 got 55834574910 for osd.4 2024-09-17T21:22:50.823 INFO:teuthology.orchestra.run.smithi083.stdout:55834574910 2024-09-17T21:22:50.840 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574911 got 55834574910 for osd.1 2024-09-17T21:22:50.845 INFO:teuthology.orchestra.run.smithi083.stdout:55834574911 2024-09-17T21:22:50.862 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574912 got 55834574911 for osd.0 2024-09-17T21:22:51.673 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T21:22:51.722 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T21:22:51.729 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T21:22:51.805 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T21:22:51.841 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T21:22:51.862 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T21:22:52.007 INFO:teuthology.orchestra.run.smithi083.stdout:992137445380 2024-09-17T21:22:52.032 INFO:tasks.ceph.ceph_manager.ceph:need seq 992137445381 got 992137445380 for osd.2 2024-09-17T21:22:52.091 INFO:teuthology.orchestra.run.smithi083.stdout:55834574910 2024-09-17T21:22:52.109 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574911 got 55834574910 for osd.5 2024-09-17T21:22:52.153 INFO:teuthology.orchestra.run.smithi083.stdout:55834574910 2024-09-17T21:22:52.170 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574911 got 55834574910 for osd.3 2024-09-17T21:22:52.215 INFO:teuthology.orchestra.run.smithi083.stdout:55834574910 2024-09-17T21:22:52.227 INFO:teuthology.orchestra.run.smithi083.stdout:55834574910 2024-09-17T21:22:52.237 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574911 got 55834574910 for osd.1 2024-09-17T21:22:52.245 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574911 got 55834574910 for osd.4 2024-09-17T21:22:52.251 INFO:teuthology.orchestra.run.smithi083.stdout:55834574911 2024-09-17T21:22:52.268 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574912 got 55834574911 for osd.0 2024-09-17T21:22:53.033 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T21:22:53.110 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T21:22:53.171 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T21:22:53.238 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T21:22:53.246 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T21:22:53.269 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T21:22:53.348 INFO:teuthology.orchestra.run.smithi083.stdout:992137445382 2024-09-17T21:22:53.377 INFO:tasks.ceph.ceph_manager.ceph:need seq 992137445381 got 992137445382 for osd.2 2024-09-17T21:22:53.378 DEBUG:teuthology.parallel:result is None 2024-09-17T21:22:53.563 INFO:teuthology.orchestra.run.smithi083.stdout:55834574911 2024-09-17T21:22:53.588 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574911 got 55834574911 for osd.5 2024-09-17T21:22:53.588 DEBUG:teuthology.parallel:result is None 2024-09-17T21:22:53.608 INFO:teuthology.orchestra.run.smithi083.stdout:55834574911 2024-09-17T21:22:53.611 INFO:teuthology.orchestra.run.smithi083.stdout:55834574911 2024-09-17T21:22:53.628 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574911 got 55834574911 for osd.4 2024-09-17T21:22:53.629 DEBUG:teuthology.parallel:result is None 2024-09-17T21:22:53.630 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574911 got 55834574911 for osd.3 2024-09-17T21:22:53.630 DEBUG:teuthology.parallel:result is None 2024-09-17T21:22:53.671 INFO:teuthology.orchestra.run.smithi083.stdout:55834574911 2024-09-17T21:22:53.688 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574911 got 55834574911 for osd.1 2024-09-17T21:22:53.688 DEBUG:teuthology.parallel:result is None 2024-09-17T21:22:53.723 INFO:teuthology.orchestra.run.smithi083.stdout:55834574912 2024-09-17T21:22:53.739 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574912 got 55834574912 for osd.0 2024-09-17T21:22:53.740 DEBUG:teuthology.parallel:result is None 2024-09-17T21:22:53.740 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T21:22:53.740 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:22:54.058 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:22:54.059 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T21:22:54.074 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-17T21:22:53.613007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":308,"num_object_clones":0,"num_object_copies":616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":300,"num_read":353971,"num_read_kb":1145964,"num_write":20052,"num_write_kb":387014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11832,"num_bytes_recovered":615643111,"num_keys_recovered":1529,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18605,"ondisk_log_size":18605,"up":82,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":353448,"kb_used_data":18472,"kb_used_omap":28,"kb_used_meta":334819,"kb_avail":565877592,"statfs":{"total":579820584960,"available":579458654208,"internally_reserved":0,"allocated":18915328,"data_stored":8698443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28944,"internal_metadata":342855408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-92,"num_objects_misplaced":-10,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":32034,"num_read_kb":123813,"num_write":1204,"num_write_kb":50417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":92573252,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.241524"},"pg_stats":[{"pgid":"3.1f","version":"227'759","reported_seq":4114,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.380587+0000","last_change":"2024-09-17T21:22:40.360129+0000","last_active":"2024-09-17T21:22:49.380587+0000","last_peered":"2024-09-17T21:22:49.380587+0000","last_clean":"2024-09-17T21:22:49.380587+0000","last_became_active":"2024-09-17T21:22:40.357329+0000","last_became_peered":"2024-09-17T21:22:40.357329+0000","last_unstale":"2024-09-17T21:22:49.380587+0000","last_undegraded":"2024-09-17T21:22:49.380587+0000","last_fullsized":"2024-09-17T21:22:49.380587+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:01:10.890769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010221002999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5921,"num_read_kb":16779,"num_write":1026,"num_write_kb":3495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":5143732,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"227'851","reported_seq":9553,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.382138+0000","last_change":"2024-09-17T21:22:40.355216+0000","last_active":"2024-09-17T21:22:49.382138+0000","last_peered":"2024-09-17T21:22:49.382138+0000","last_clean":"2024-09-17T21:22:49.382138+0000","last_became_active":"2024-09-17T21:22:40.353016+0000","last_became_peered":"2024-09-17T21:22:40.353016+0000","last_unstale":"2024-09-17T21:22:49.382138+0000","last_undegraded":"2024-09-17T21:22:49.382138+0000","last_fullsized":"2024-09-17T21:22:49.382138+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":851,"log_dups_size":0,"ondisk_log_size":851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:11.494395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084510550000000007,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10998,"num_read_kb":53173,"num_write":1081,"num_write_kb":18708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":50670363,"num_keys_recovered":90,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"227'833","reported_seq":8684,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.388425+0000","last_change":"2024-09-17T21:22:48.408191+0000","last_active":"2024-09-17T21:22:49.388425+0000","last_peered":"2024-09-17T21:22:49.388425+0000","last_clean":"2024-09-17T21:22:49.388425+0000","last_became_active":"2024-09-17T21:22:48.401326+0000","last_became_peered":"2024-09-17T21:22:48.401326+0000","last_unstale":"2024-09-17T21:22:49.388425+0000","last_undegraded":"2024-09-17T21:22:49.388425+0000","last_fullsized":"2024-09-17T21:22:49.388425+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:19:25.050335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082452470000000007,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":10230,"num_read_kb":27857,"num_write":978,"num_write_kb":10466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":12493301,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"227'376","reported_seq":4042,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.388018+0000","last_change":"2024-09-17T21:22:48.406130+0000","last_active":"2024-09-17T21:22:49.388018+0000","last_peered":"2024-09-17T21:22:49.388018+0000","last_clean":"2024-09-17T21:22:49.388018+0000","last_became_active":"2024-09-17T21:22:48.401237+0000","last_became_peered":"2024-09-17T21:22:48.401237+0000","last_unstale":"2024-09-17T21:22:49.388018+0000","last_undegraded":"2024-09-17T21:22:49.388018+0000","last_fullsized":"2024-09-17T21:22:49.388018+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:39:30.285487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061565819999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":2726,"num_read_kb":23738,"num_write":383,"num_write_kb":5937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":9397808,"num_keys_recovered":202,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"227'710","reported_seq":8287,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.382726+0000","last_change":"2024-09-17T21:22:24.369716+0000","last_active":"2024-09-17T21:22:49.382726+0000","last_peered":"2024-09-17T21:22:49.382726+0000","last_clean":"2024-09-17T21:22:49.382726+0000","last_became_active":"2024-09-17T21:21:41.329830+0000","last_became_peered":"2024-09-17T21:21:41.329830+0000","last_unstale":"2024-09-17T21:22:49.382726+0000","last_undegraded":"2024-09-17T21:22:49.382726+0000","last_fullsized":"2024-09-17T21:22:49.382726+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:42:50.772772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026038680000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":11634,"num_read_kb":21103,"num_write":555,"num_write_kb":4725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":7872032,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"226'739","reported_seq":13907,"reported_epoch":245,"state":"peering","last_fresh":"2024-09-17T21:22:49.384724+0000","last_change":"2024-09-17T21:22:49.384724+0000","last_active":"2024-09-17T21:22:45.342106+0000","last_peered":"2024-09-17T21:22:43.593653+0000","last_clean":"2024-09-17T21:22:36.675355+0000","last_became_active":"2024-09-17T21:22:39.364106+0000","last_became_peered":"2024-09-17T21:22:39.364106+0000","last_unstale":"2024-09-17T21:22:49.384724+0000","last_undegraded":"2024-09-17T21:22:49.384724+0000","last_fullsized":"2024-09-17T21:22:49.384724+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056973060000000001,"stat_sum":{"num_bytes":19,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":16893,"num_read_kb":53539,"num_write":669,"num_write_kb":14565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":647718,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"226'476","reported_seq":12468,"reported_epoch":245,"state":"active+undersized+remapped","last_fresh":"2024-09-17T21:22:49.449625+0000","last_change":"2024-09-17T21:22:49.450079+0000","last_active":"2024-09-17T21:22:49.449625+0000","last_peered":"2024-09-17T21:22:49.449625+0000","last_clean":"2024-09-17T21:22:43.331395+0000","last_became_active":"2024-09-17T21:22:47.035944+0000","last_became_peered":"2024-09-17T21:22:47.035944+0000","last_unstale":"2024-09-17T21:22:49.449625+0000","last_undegraded":"2024-09-17T21:22:49.449625+0000","last_fullsized":"2024-09-17T21:22:46.342866+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0077005709999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":12648,"num_read_kb":27025,"num_write":374,"num_write_kb":5615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":3100207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1],"avail_no_missing":["1","3","4"],"object_location_counts":[{"shards":"1","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"226'318","reported_seq":3035,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382486+0000","last_change":"2024-09-17T21:22:50.387935+0000","last_active":"2024-09-17T21:22:51.382486+0000","last_peered":"2024-09-17T21:22:51.382486+0000","last_clean":"2024-09-17T21:22:51.382486+0000","last_became_active":"2024-09-17T21:22:50.385975+0000","last_became_peered":"2024-09-17T21:22:50.385975+0000","last_unstale":"2024-09-17T21:22:51.382486+0000","last_undegraded":"2024-09-17T21:22:51.382486+0000","last_fullsized":"2024-09-17T21:22:51.382486+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:26:13.518654+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022643899999999998,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2392,"num_read_kb":9139,"num_write":304,"num_write_kb":2898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":9226431,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"225'393","reported_seq":7339,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.380742+0000","last_change":"2024-09-17T21:22:43.342179+0000","last_active":"2024-09-17T21:22:49.380742+0000","last_peered":"2024-09-17T21:22:49.380742+0000","last_clean":"2024-09-17T21:22:49.380742+0000","last_became_active":"2024-09-17T21:22:43.340739+0000","last_became_peered":"2024-09-17T21:22:43.340739+0000","last_unstale":"2024-09-17T21:22:49.380742+0000","last_undegraded":"2024-09-17T21:22:49.380742+0000","last_fullsized":"2024-09-17T21:22:49.380742+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:32:48.751230+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00138835,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7054,"num_read_kb":21313,"num_write":475,"num_write_kb":6946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":719,"num_bytes_recovered":31066012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"224'602","reported_seq":22248,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.388691+0000","last_change":"2024-09-17T21:22:48.400263+0000","last_active":"2024-09-17T21:22:49.388691+0000","last_peered":"2024-09-17T21:22:49.388691+0000","last_clean":"2024-09-17T21:22:49.388691+0000","last_became_active":"2024-09-17T21:22:48.394816+0000","last_became_peered":"2024-09-17T21:22:48.394816+0000","last_unstale":"2024-09-17T21:22:49.388691+0000","last_undegraded":"2024-09-17T21:22:49.388691+0000","last_fullsized":"2024-09-17T21:22:49.388691+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:05.075504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075408430000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":22417,"num_read_kb":27659,"num_write":858,"num_write_kb":13022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":3432016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"223'754","reported_seq":7154,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.377593+0000","last_change":"2024-09-17T21:22:24.378500+0000","last_active":"2024-09-17T21:22:49.377593+0000","last_peered":"2024-09-17T21:22:49.377593+0000","last_clean":"2024-09-17T21:22:49.377593+0000","last_became_active":"2024-09-17T21:21:40.321648+0000","last_became_peered":"2024-09-17T21:21:40.321648+0000","last_unstale":"2024-09-17T21:22:49.377593+0000","last_undegraded":"2024-09-17T21:22:49.377593+0000","last_fullsized":"2024-09-17T21:22:49.377593+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:54:37.691837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0071138240000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8592,"num_read_kb":29445,"num_write":954,"num_write_kb":5554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":9885558,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"225'449","reported_seq":5372,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.395698+0000","last_change":"2024-09-17T21:22:49.393985+0000","last_active":"2024-09-17T21:22:49.395698+0000","last_peered":"2024-09-17T21:22:49.395698+0000","last_clean":"2024-09-17T21:22:49.395698+0000","last_became_active":"2024-09-17T21:22:49.389098+0000","last_became_peered":"2024-09-17T21:22:49.389098+0000","last_unstale":"2024-09-17T21:22:49.395698+0000","last_undegraded":"2024-09-17T21:22:49.395698+0000","last_fullsized":"2024-09-17T21:22:49.395698+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:28:53.422353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003029,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5351,"num_read_kb":26972,"num_write":428,"num_write_kb":16010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":385,"num_bytes_recovered":6943878,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":470,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.378624+0000","last_change":"2024-09-17T21:18:03.059715+0000","last_active":"2024-09-17T21:22:49.378624+0000","last_peered":"2024-09-17T21:22:49.378624+0000","last_clean":"2024-09-17T21:22:49.378624+0000","last_became_active":"2024-09-17T21:18:01.213047+0000","last_became_peered":"2024-09-17T21:18:01.213047+0000","last_unstale":"2024-09-17T21:22:49.378624+0000","last_undegraded":"2024-09-17T21:22:49.378624+0000","last_fullsized":"2024-09-17T21:22:49.378624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:46:35.222373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"227'594","reported_seq":13690,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.382362+0000","last_change":"2024-09-17T21:22:24.371187+0000","last_active":"2024-09-17T21:22:49.382362+0000","last_peered":"2024-09-17T21:22:49.382362+0000","last_clean":"2024-09-17T21:22:49.382362+0000","last_became_active":"2024-09-17T21:21:40.318488+0000","last_became_peered":"2024-09-17T21:21:40.318488+0000","last_unstale":"2024-09-17T21:22:49.382362+0000","last_undegraded":"2024-09-17T21:22:49.382362+0000","last_fullsized":"2024-09-17T21:22:49.382362+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:57:56.726468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036709440000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14882,"num_read_kb":61123,"num_write":581,"num_write_kb":23555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":49089084,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":470,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.381228+0000","last_change":"2024-09-17T21:18:03.073440+0000","last_active":"2024-09-17T21:22:49.381228+0000","last_peered":"2024-09-17T21:22:49.381228+0000","last_clean":"2024-09-17T21:22:49.381228+0000","last_became_active":"2024-09-17T21:18:01.048806+0000","last_became_peered":"2024-09-17T21:18:01.048806+0000","last_unstale":"2024-09-17T21:22:49.381228+0000","last_undegraded":"2024-09-17T21:22:49.381228+0000","last_fullsized":"2024-09-17T21:22:49.381228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:26.487163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094161400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"226'352","reported_seq":5610,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382685+0000","last_change":"2024-09-17T21:22:50.390979+0000","last_active":"2024-09-17T21:22:51.382685+0000","last_peered":"2024-09-17T21:22:51.382685+0000","last_clean":"2024-09-17T21:22:51.382685+0000","last_became_active":"2024-09-17T21:22:50.388965+0000","last_became_peered":"2024-09-17T21:22:50.388965+0000","last_unstale":"2024-09-17T21:22:51.382685+0000","last_undegraded":"2024-09-17T21:22:51.382685+0000","last_fullsized":"2024-09-17T21:22:51.382685+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:18:04.741617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079556969999999994,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4656,"num_read_kb":36280,"num_write":392,"num_write_kb":13876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":5244262,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":494,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.378812+0000","last_change":"2024-09-17T21:21:35.319221+0000","last_active":"2024-09-17T21:22:49.378812+0000","last_peered":"2024-09-17T21:22:49.378812+0000","last_clean":"2024-09-17T21:22:49.378812+0000","last_became_active":"2024-09-17T21:21:35.312507+0000","last_became_peered":"2024-09-17T21:21:35.312507+0000","last_unstale":"2024-09-17T21:22:49.378812+0000","last_undegraded":"2024-09-17T21:22:49.378812+0000","last_fullsized":"2024-09-17T21:22:49.378812+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:53.042567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099219900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"226'707","reported_seq":16131,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.380336+0000","last_change":"2024-09-17T21:22:46.361219+0000","last_active":"2024-09-17T21:22:49.380336+0000","last_peered":"2024-09-17T21:22:49.380336+0000","last_clean":"2024-09-17T21:22:49.380336+0000","last_became_active":"2024-09-17T21:22:46.359525+0000","last_became_peered":"2024-09-17T21:22:46.359525+0000","last_unstale":"2024-09-17T21:22:49.380336+0000","last_undegraded":"2024-09-17T21:22:49.380336+0000","last_fullsized":"2024-09-17T21:22:49.380336+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:47.174816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052152359999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13338,"num_read_kb":29382,"num_write":606,"num_write_kb":10802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":367,"num_bytes_recovered":21385444,"num_keys_recovered":118,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":494,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.380660+0000","last_change":"2024-09-17T21:21:35.326814+0000","last_active":"2024-09-17T21:22:49.380660+0000","last_peered":"2024-09-17T21:22:49.380660+0000","last_clean":"2024-09-17T21:22:49.380660+0000","last_became_active":"2024-09-17T21:21:35.325777+0000","last_became_peered":"2024-09-17T21:21:35.325777+0000","last_unstale":"2024-09-17T21:22:49.380660+0000","last_undegraded":"2024-09-17T21:22:49.380660+0000","last_fullsized":"2024-09-17T21:22:49.380660+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:47:00.558900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006222,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"225'1056","reported_seq":33576,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.388088+0000","last_change":"2024-09-17T21:22:44.360752+0000","last_active":"2024-09-17T21:22:49.388088+0000","last_peered":"2024-09-17T21:22:49.388088+0000","last_clean":"2024-09-17T21:22:49.388088+0000","last_became_active":"2024-09-17T21:22:44.352239+0000","last_became_peered":"2024-09-17T21:22:44.352239+0000","last_unstale":"2024-09-17T21:22:49.388088+0000","last_undegraded":"2024-09-17T21:22:49.388088+0000","last_fullsized":"2024-09-17T21:22:49.388088+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":1056,"log_dups_size":0,"ondisk_log_size":1056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:36:58.893607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052118770000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":37177,"num_read_kb":61903,"num_write":871,"num_write_kb":21852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":754,"num_bytes_recovered":58900004,"num_keys_recovered":204,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"227'536","reported_seq":3883,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.385669+0000","last_change":"2024-09-17T21:22:24.383832+0000","last_active":"2024-09-17T21:22:49.385669+0000","last_peered":"2024-09-17T21:22:49.385669+0000","last_clean":"2024-09-17T21:22:49.385669+0000","last_became_active":"2024-09-17T21:21:39.315142+0000","last_became_peered":"2024-09-17T21:21:39.315142+0000","last_unstale":"2024-09-17T21:22:49.385669+0000","last_undegraded":"2024-09-17T21:22:49.385669+0000","last_fullsized":"2024-09-17T21:22:49.385669+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:05:28.580363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061334989999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":3053,"num_read_kb":14423,"num_write":502,"num_write_kb":4818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":7199060,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":408,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.385325+0000","last_change":"2024-09-17T21:22:48.383726+0000","last_active":"2024-09-17T21:22:49.385325+0000","last_peered":"2024-09-17T21:22:49.385325+0000","last_clean":"2024-09-17T21:22:49.385325+0000","last_became_active":"2024-09-17T21:22:48.382842+0000","last_became_peered":"2024-09-17T21:22:48.382842+0000","last_unstale":"2024-09-17T21:22:49.385325+0000","last_undegraded":"2024-09-17T21:22:49.385325+0000","last_fullsized":"2024-09-17T21:22:49.385325+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:30:55.312809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"226'248","reported_seq":9442,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.378961+0000","last_change":"2024-09-17T21:22:45.354353+0000","last_active":"2024-09-17T21:22:49.378961+0000","last_peered":"2024-09-17T21:22:49.378961+0000","last_clean":"2024-09-17T21:22:49.378961+0000","last_became_active":"2024-09-17T21:22:45.352682+0000","last_became_peered":"2024-09-17T21:22:45.352682+0000","last_unstale":"2024-09-17T21:22:49.378961+0000","last_undegraded":"2024-09-17T21:22:49.378961+0000","last_fullsized":"2024-09-17T21:22:49.378961+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:18.942784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048483459999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5340,"num_read_kb":22605,"num_write":215,"num_write_kb":6424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":15742651,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":461,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382421+0000","last_change":"2024-09-17T21:22:50.389225+0000","last_active":"2024-09-17T21:22:51.382421+0000","last_peered":"2024-09-17T21:22:51.382421+0000","last_clean":"2024-09-17T21:22:51.382421+0000","last_became_active":"2024-09-17T21:22:50.388953+0000","last_became_peered":"2024-09-17T21:22:50.388953+0000","last_unstale":"2024-09-17T21:22:51.382421+0000","last_undegraded":"2024-09-17T21:22:51.382421+0000","last_fullsized":"2024-09-17T21:22:51.382421+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:21.062518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"227'1041","reported_seq":8856,"reported_epoch":244,"state":"peering","last_fresh":"2024-09-17T21:22:48.984797+0000","last_change":"2024-09-17T21:22:49.380622+0000","last_active":"2024-09-17T21:22:49.380622+0000","last_peered":"2024-09-17T21:22:48.984797+0000","last_clean":"2024-09-17T21:22:43.336352+0000","last_became_active":"2024-09-17T21:22:47.035089+0000","last_became_peered":"2024-09-17T21:22:47.035089+0000","last_unstale":"2024-09-17T21:22:48.984797+0000","last_undegraded":"2024-09-17T21:22:48.984797+0000","last_fullsized":"2024-09-17T21:22:46.345701+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":1041,"log_dups_size":0,"ondisk_log_size":1041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035942119999999998,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":9666,"num_read_kb":63102,"num_write":1578,"num_write_kb":18132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":894,"num_bytes_recovered":48903910,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":470,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.377043+0000","last_change":"2024-09-17T21:18:03.059156+0000","last_active":"2024-09-17T21:22:49.377043+0000","last_peered":"2024-09-17T21:22:49.377043+0000","last_clean":"2024-09-17T21:22:49.377043+0000","last_became_active":"2024-09-17T21:18:01.049180+0000","last_became_peered":"2024-09-17T21:18:01.049180+0000","last_unstale":"2024-09-17T21:22:49.377043+0000","last_undegraded":"2024-09-17T21:22:49.377043+0000","last_fullsized":"2024-09-17T21:22:49.377043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:27.069794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077408,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"226'656","reported_seq":14590,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.379957+0000","last_change":"2024-09-17T21:22:41.354650+0000","last_active":"2024-09-17T21:22:49.379957+0000","last_peered":"2024-09-17T21:22:49.379957+0000","last_clean":"2024-09-17T21:22:49.379957+0000","last_became_active":"2024-09-17T21:22:41.352429+0000","last_became_peered":"2024-09-17T21:22:41.352429+0000","last_unstale":"2024-09-17T21:22:49.379957+0000","last_undegraded":"2024-09-17T21:22:49.379957+0000","last_fullsized":"2024-09-17T21:22:49.379957+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:30:46.965276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067184260000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14741,"num_read_kb":38983,"num_write":745,"num_write_kb":20177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":27258987,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.385460+0000","last_change":"2024-09-17T21:22:48.387337+0000","last_active":"2024-09-17T21:22:49.385460+0000","last_peered":"2024-09-17T21:22:49.385460+0000","last_clean":"2024-09-17T21:22:49.385460+0000","last_became_active":"2024-09-17T21:22:48.386572+0000","last_became_peered":"2024-09-17T21:22:48.386572+0000","last_unstale":"2024-09-17T21:22:49.385460+0000","last_undegraded":"2024-09-17T21:22:49.385460+0000","last_fullsized":"2024-09-17T21:22:49.385460+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:55:57.934354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":185,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.385193+0000","last_change":"2024-09-17T21:21:17.347819+0000","last_active":"2024-09-17T21:22:49.385193+0000","last_peered":"2024-09-17T21:22:49.385193+0000","last_clean":"2024-09-17T21:22:49.385193+0000","last_became_active":"2024-09-17T21:21:17.337621+0000","last_became_peered":"2024-09-17T21:21:17.337621+0000","last_unstale":"2024-09-17T21:22:49.385193+0000","last_undegraded":"2024-09-17T21:22:49.385193+0000","last_fullsized":"2024-09-17T21:22:49.385193+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_clean_scrub_stamp":"2024-09-17T21:17:55.982339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:24:32.371870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"224'616","reported_seq":14093,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382313+0000","last_change":"2024-09-17T21:22:50.383917+0000","last_active":"2024-09-17T21:22:51.382313+0000","last_peered":"2024-09-17T21:22:51.382313+0000","last_clean":"2024-09-17T21:22:51.382313+0000","last_became_active":"2024-09-17T21:22:50.382396+0000","last_became_peered":"2024-09-17T21:22:50.382396+0000","last_unstale":"2024-09-17T21:22:51.382313+0000","last_undegraded":"2024-09-17T21:22:51.382313+0000","last_fullsized":"2024-09-17T21:22:51.382313+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:13:13.848853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001602121,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":13993,"num_read_kb":68437,"num_write":586,"num_write_kb":29114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":40776135,"num_keys_recovered":165,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"227'699","reported_seq":11102,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.395662+0000","last_change":"2024-09-17T21:22:49.395344+0000","last_active":"2024-09-17T21:22:49.395662+0000","last_peered":"2024-09-17T21:22:49.395662+0000","last_clean":"2024-09-17T21:22:49.395662+0000","last_became_active":"2024-09-17T21:22:49.390523+0000","last_became_peered":"2024-09-17T21:22:49.390523+0000","last_unstale":"2024-09-17T21:22:49.395662+0000","last_undegraded":"2024-09-17T21:22:49.395662+0000","last_fullsized":"2024-09-17T21:22:49.395662+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:55:28.950314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029497360000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":11677,"num_read_kb":43614,"num_write":1089,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":728,"num_bytes_recovered":30193805,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"227'428","reported_seq":8262,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.397894+0000","last_change":"2024-09-17T21:22:49.397589+0000","last_active":"2024-09-17T21:22:49.397894+0000","last_peered":"2024-09-17T21:22:49.397894+0000","last_clean":"2024-09-17T21:22:49.397894+0000","last_became_active":"2024-09-17T21:22:49.394384+0000","last_became_peered":"2024-09-17T21:22:49.394384+0000","last_unstale":"2024-09-17T21:22:49.397894+0000","last_undegraded":"2024-09-17T21:22:49.397894+0000","last_fullsized":"2024-09-17T21:22:49.397894+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:24:21.885168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045615830000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7186,"num_read_kb":40870,"num_write":369,"num_write_kb":16675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":179794,"num_keys_recovered":2,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"226'376","reported_seq":4916,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.380188+0000","last_change":"2024-09-17T21:22:24.375850+0000","last_active":"2024-09-17T21:22:49.380188+0000","last_peered":"2024-09-17T21:22:49.380188+0000","last_clean":"2024-09-17T21:22:49.380188+0000","last_became_active":"2024-09-17T21:21:40.319613+0000","last_became_peered":"2024-09-17T21:21:40.319613+0000","last_unstale":"2024-09-17T21:22:49.380188+0000","last_undegraded":"2024-09-17T21:22:49.380188+0000","last_fullsized":"2024-09-17T21:22:49.380188+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:26:53.101370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038796239999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4694,"num_read_kb":23619,"num_write":320,"num_write_kb":13765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":4886965,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"227'452","reported_seq":13417,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.379750+0000","last_change":"2024-09-17T21:22:43.339607+0000","last_active":"2024-09-17T21:22:49.379750+0000","last_peered":"2024-09-17T21:22:49.379750+0000","last_clean":"2024-09-17T21:22:49.379750+0000","last_became_active":"2024-09-17T21:22:43.338399+0000","last_became_peered":"2024-09-17T21:22:43.338399+0000","last_unstale":"2024-09-17T21:22:49.379750+0000","last_undegraded":"2024-09-17T21:22:49.379750+0000","last_fullsized":"2024-09-17T21:22:49.379750+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:30:14.210043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050460430000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":15379,"num_read_kb":29065,"num_write":580,"num_write_kb":14319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":5960704,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"221'526","reported_seq":15415,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.378976+0000","last_change":"2024-09-17T21:22:46.356687+0000","last_active":"2024-09-17T21:22:49.378976+0000","last_peered":"2024-09-17T21:22:49.378976+0000","last_clean":"2024-09-17T21:22:49.378976+0000","last_became_active":"2024-09-17T21:22:46.349835+0000","last_became_peered":"2024-09-17T21:22:46.349835+0000","last_unstale":"2024-09-17T21:22:49.378976+0000","last_undegraded":"2024-09-17T21:22:49.378976+0000","last_fullsized":"2024-09-17T21:22:49.378976+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:51:30.571949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00482724,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15785,"num_read_kb":43220,"num_write":654,"num_write_kb":16401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":28397872,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"227'623","reported_seq":14440,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.806310+0000","last_change":"2024-09-17T21:22:49.806087+0000","last_active":"2024-09-17T21:22:49.806310+0000","last_peered":"2024-09-17T21:22:49.806310+0000","last_clean":"2024-09-17T21:22:49.806310+0000","last_became_active":"2024-09-17T21:22:49.394154+0000","last_became_peered":"2024-09-17T21:22:49.394154+0000","last_unstale":"2024-09-17T21:22:49.806310+0000","last_undegraded":"2024-09-17T21:22:49.806310+0000","last_fullsized":"2024-09-17T21:22:49.806310+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:24:57.185890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057317679999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":15141,"num_read_kb":70274,"num_write":535,"num_write_kb":35852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":91925553,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"226'464","reported_seq":7119,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.388236+0000","last_change":"2024-09-17T21:22:48.398780+0000","last_active":"2024-09-17T21:22:49.388236+0000","last_peered":"2024-09-17T21:22:49.388236+0000","last_clean":"2024-09-17T21:22:49.388236+0000","last_became_active":"2024-09-17T21:22:48.392669+0000","last_became_peered":"2024-09-17T21:22:48.392669+0000","last_unstale":"2024-09-17T21:22:49.388236+0000","last_undegraded":"2024-09-17T21:22:49.388236+0000","last_fullsized":"2024-09-17T21:22:49.388236+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:18:23.844174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0091147729999999996,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7427,"num_read_kb":28404,"num_write":434,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":4659071,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"222'271","reported_seq":7399,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.384802+0000","last_change":"2024-09-17T21:22:48.384974+0000","last_active":"2024-09-17T21:22:49.384802+0000","last_peered":"2024-09-17T21:22:49.384802+0000","last_clean":"2024-09-17T21:22:49.384802+0000","last_became_active":"2024-09-17T21:22:48.381106+0000","last_became_peered":"2024-09-17T21:22:48.381106+0000","last_unstale":"2024-09-17T21:22:49.384802+0000","last_undegraded":"2024-09-17T21:22:49.384802+0000","last_fullsized":"2024-09-17T21:22:49.384802+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:11:47.017985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017482940000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7827,"num_read_kb":19883,"num_write":219,"num_write_kb":3298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":6733181,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"226'404","reported_seq":5774,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.377779+0000","last_change":"2024-09-17T21:22:24.385309+0000","last_active":"2024-09-17T21:22:49.377779+0000","last_peered":"2024-09-17T21:22:49.377779+0000","last_clean":"2024-09-17T21:22:49.377779+0000","last_became_active":"2024-09-17T21:18:11.299946+0000","last_became_peered":"2024-09-17T21:18:11.299946+0000","last_unstale":"2024-09-17T21:22:49.377779+0000","last_undegraded":"2024-09-17T21:22:49.377779+0000","last_fullsized":"2024-09-17T21:22:49.377779+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:16:16.699933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015128413,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5168,"num_read_kb":22085,"num_write":382,"num_write_kb":4452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"225'570","reported_seq":15991,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.381896+0000","last_change":"2024-09-17T21:22:24.368740+0000","last_active":"2024-09-17T21:22:49.381896+0000","last_peered":"2024-09-17T21:22:49.381896+0000","last_clean":"2024-09-17T21:22:49.381896+0000","last_became_active":"2024-09-17T21:22:09.609844+0000","last_became_peered":"2024-09-17T21:22:09.609844+0000","last_unstale":"2024-09-17T21:22:49.381896+0000","last_undegraded":"2024-09-17T21:22:49.381896+0000","last_fullsized":"2024-09-17T21:22:49.381896+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:03:30.364818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018557070000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":17911,"num_read_kb":38759,"num_write":589,"num_write_kb":6968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":11589682,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"226'651","reported_seq":12601,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.377308+0000","last_change":"2024-09-17T21:22:41.357868+0000","last_active":"2024-09-17T21:22:49.377308+0000","last_peered":"2024-09-17T21:22:49.377308+0000","last_clean":"2024-09-17T21:22:49.377308+0000","last_became_active":"2024-09-17T21:22:41.354851+0000","last_became_peered":"2024-09-17T21:22:41.354851+0000","last_unstale":"2024-09-17T21:22:49.377308+0000","last_undegraded":"2024-09-17T21:22:49.377308+0000","last_fullsized":"2024-09-17T21:22:49.377308+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:47:04.462955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011937984,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12008,"num_read_kb":52136,"num_write":617,"num_write_kb":5443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":5557098,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":304,"num_object_clones":0,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":300,"num_read":353905,"num_read_kb":1145909,"num_write":19949,"num_write_kb":385384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11819,"num_bytes_recovered":614462318,"num_keys_recovered":1529,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8194,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":64,"acting":63,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574911,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60440,"kb_used_data":2884,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311400,"statfs":{"total":96636764160,"available":96574873600,"internally_reserved":0,"allocated":2953216,"data_stored":1252104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":58915268},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66584,"kb_used_data":3464,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305256,"statfs":{"total":96636764160,"available":96568582144,"internally_reserved":0,"allocated":3547136,"data_stored":1842472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":64616900},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83224,"kb_used_data":3460,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288616,"statfs":{"total":96636764160,"available":96551542784,"internally_reserved":0,"allocated":3543040,"data_stored":1842453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":81656262},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56288,"kb_used_data":2888,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315552,"statfs":{"total":96636764160,"available":96579125248,"internally_reserved":0,"allocated":2957312,"data_stored":1252123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56284,"kb_used_data":2884,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315556,"statfs":{"total":96636764160,"available":96579129344,"internally_reserved":0,"allocated":2953216,"data_stored":1252104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":231,"seq":992137445382,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30628,"kb_used_data":2892,"kb_used_omap":20,"kb_used_meta":27691,"kb_avail":94341212,"statfs":{"total":96636764160,"available":96605401088,"internally_reserved":0,"allocated":2961408,"data_stored":1257187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20966,"internal_metadata":28356122},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8194,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:22:54.077 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:22:54.362 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:22:54.362 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T21:22:54.378 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-17T21:22:53.613007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":308,"num_object_clones":0,"num_object_copies":616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":300,"num_read":353971,"num_read_kb":1145964,"num_write":20052,"num_write_kb":387014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11832,"num_bytes_recovered":615643111,"num_keys_recovered":1529,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18605,"ondisk_log_size":18605,"up":82,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":353448,"kb_used_data":18472,"kb_used_omap":28,"kb_used_meta":334819,"kb_avail":565877592,"statfs":{"total":579820584960,"available":579458654208,"internally_reserved":0,"allocated":18915328,"data_stored":8698443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28944,"internal_metadata":342855408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-92,"num_objects_misplaced":-10,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":32034,"num_read_kb":123813,"num_write":1204,"num_write_kb":50417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":92573252,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.241524"},"pg_stats":[{"pgid":"3.1f","version":"227'759","reported_seq":4114,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.380587+0000","last_change":"2024-09-17T21:22:40.360129+0000","last_active":"2024-09-17T21:22:49.380587+0000","last_peered":"2024-09-17T21:22:49.380587+0000","last_clean":"2024-09-17T21:22:49.380587+0000","last_became_active":"2024-09-17T21:22:40.357329+0000","last_became_peered":"2024-09-17T21:22:40.357329+0000","last_unstale":"2024-09-17T21:22:49.380587+0000","last_undegraded":"2024-09-17T21:22:49.380587+0000","last_fullsized":"2024-09-17T21:22:49.380587+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:01:10.890769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010221002999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5921,"num_read_kb":16779,"num_write":1026,"num_write_kb":3495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":5143732,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"227'851","reported_seq":9553,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.382138+0000","last_change":"2024-09-17T21:22:40.355216+0000","last_active":"2024-09-17T21:22:49.382138+0000","last_peered":"2024-09-17T21:22:49.382138+0000","last_clean":"2024-09-17T21:22:49.382138+0000","last_became_active":"2024-09-17T21:22:40.353016+0000","last_became_peered":"2024-09-17T21:22:40.353016+0000","last_unstale":"2024-09-17T21:22:49.382138+0000","last_undegraded":"2024-09-17T21:22:49.382138+0000","last_fullsized":"2024-09-17T21:22:49.382138+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":851,"log_dups_size":0,"ondisk_log_size":851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:11.494395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084510550000000007,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10998,"num_read_kb":53173,"num_write":1081,"num_write_kb":18708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":50670363,"num_keys_recovered":90,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"227'833","reported_seq":8684,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.388425+0000","last_change":"2024-09-17T21:22:48.408191+0000","last_active":"2024-09-17T21:22:49.388425+0000","last_peered":"2024-09-17T21:22:49.388425+0000","last_clean":"2024-09-17T21:22:49.388425+0000","last_became_active":"2024-09-17T21:22:48.401326+0000","last_became_peered":"2024-09-17T21:22:48.401326+0000","last_unstale":"2024-09-17T21:22:49.388425+0000","last_undegraded":"2024-09-17T21:22:49.388425+0000","last_fullsized":"2024-09-17T21:22:49.388425+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:19:25.050335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082452470000000007,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":10230,"num_read_kb":27857,"num_write":978,"num_write_kb":10466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":12493301,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"227'376","reported_seq":4042,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.388018+0000","last_change":"2024-09-17T21:22:48.406130+0000","last_active":"2024-09-17T21:22:49.388018+0000","last_peered":"2024-09-17T21:22:49.388018+0000","last_clean":"2024-09-17T21:22:49.388018+0000","last_became_active":"2024-09-17T21:22:48.401237+0000","last_became_peered":"2024-09-17T21:22:48.401237+0000","last_unstale":"2024-09-17T21:22:49.388018+0000","last_undegraded":"2024-09-17T21:22:49.388018+0000","last_fullsized":"2024-09-17T21:22:49.388018+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:39:30.285487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061565819999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":2726,"num_read_kb":23738,"num_write":383,"num_write_kb":5937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":9397808,"num_keys_recovered":202,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"227'710","reported_seq":8287,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.382726+0000","last_change":"2024-09-17T21:22:24.369716+0000","last_active":"2024-09-17T21:22:49.382726+0000","last_peered":"2024-09-17T21:22:49.382726+0000","last_clean":"2024-09-17T21:22:49.382726+0000","last_became_active":"2024-09-17T21:21:41.329830+0000","last_became_peered":"2024-09-17T21:21:41.329830+0000","last_unstale":"2024-09-17T21:22:49.382726+0000","last_undegraded":"2024-09-17T21:22:49.382726+0000","last_fullsized":"2024-09-17T21:22:49.382726+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:42:50.772772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026038680000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":11634,"num_read_kb":21103,"num_write":555,"num_write_kb":4725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":7872032,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"226'739","reported_seq":13907,"reported_epoch":245,"state":"peering","last_fresh":"2024-09-17T21:22:49.384724+0000","last_change":"2024-09-17T21:22:49.384724+0000","last_active":"2024-09-17T21:22:45.342106+0000","last_peered":"2024-09-17T21:22:43.593653+0000","last_clean":"2024-09-17T21:22:36.675355+0000","last_became_active":"2024-09-17T21:22:39.364106+0000","last_became_peered":"2024-09-17T21:22:39.364106+0000","last_unstale":"2024-09-17T21:22:49.384724+0000","last_undegraded":"2024-09-17T21:22:49.384724+0000","last_fullsized":"2024-09-17T21:22:49.384724+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056973060000000001,"stat_sum":{"num_bytes":19,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":16893,"num_read_kb":53539,"num_write":669,"num_write_kb":14565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":647718,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"226'476","reported_seq":12468,"reported_epoch":245,"state":"active+undersized+remapped","last_fresh":"2024-09-17T21:22:49.449625+0000","last_change":"2024-09-17T21:22:49.450079+0000","last_active":"2024-09-17T21:22:49.449625+0000","last_peered":"2024-09-17T21:22:49.449625+0000","last_clean":"2024-09-17T21:22:43.331395+0000","last_became_active":"2024-09-17T21:22:47.035944+0000","last_became_peered":"2024-09-17T21:22:47.035944+0000","last_unstale":"2024-09-17T21:22:49.449625+0000","last_undegraded":"2024-09-17T21:22:49.449625+0000","last_fullsized":"2024-09-17T21:22:46.342866+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0077005709999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":12648,"num_read_kb":27025,"num_write":374,"num_write_kb":5615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":3100207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1],"avail_no_missing":["1","3","4"],"object_location_counts":[{"shards":"1","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"226'318","reported_seq":3035,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382486+0000","last_change":"2024-09-17T21:22:50.387935+0000","last_active":"2024-09-17T21:22:51.382486+0000","last_peered":"2024-09-17T21:22:51.382486+0000","last_clean":"2024-09-17T21:22:51.382486+0000","last_became_active":"2024-09-17T21:22:50.385975+0000","last_became_peered":"2024-09-17T21:22:50.385975+0000","last_unstale":"2024-09-17T21:22:51.382486+0000","last_undegraded":"2024-09-17T21:22:51.382486+0000","last_fullsized":"2024-09-17T21:22:51.382486+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:26:13.518654+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022643899999999998,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2392,"num_read_kb":9139,"num_write":304,"num_write_kb":2898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":9226431,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"225'393","reported_seq":7339,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.380742+0000","last_change":"2024-09-17T21:22:43.342179+0000","last_active":"2024-09-17T21:22:49.380742+0000","last_peered":"2024-09-17T21:22:49.380742+0000","last_clean":"2024-09-17T21:22:49.380742+0000","last_became_active":"2024-09-17T21:22:43.340739+0000","last_became_peered":"2024-09-17T21:22:43.340739+0000","last_unstale":"2024-09-17T21:22:49.380742+0000","last_undegraded":"2024-09-17T21:22:49.380742+0000","last_fullsized":"2024-09-17T21:22:49.380742+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:32:48.751230+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00138835,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7054,"num_read_kb":21313,"num_write":475,"num_write_kb":6946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":719,"num_bytes_recovered":31066012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"224'602","reported_seq":22248,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.388691+0000","last_change":"2024-09-17T21:22:48.400263+0000","last_active":"2024-09-17T21:22:49.388691+0000","last_peered":"2024-09-17T21:22:49.388691+0000","last_clean":"2024-09-17T21:22:49.388691+0000","last_became_active":"2024-09-17T21:22:48.394816+0000","last_became_peered":"2024-09-17T21:22:48.394816+0000","last_unstale":"2024-09-17T21:22:49.388691+0000","last_undegraded":"2024-09-17T21:22:49.388691+0000","last_fullsized":"2024-09-17T21:22:49.388691+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:05.075504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075408430000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":22417,"num_read_kb":27659,"num_write":858,"num_write_kb":13022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":3432016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"223'754","reported_seq":7154,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.377593+0000","last_change":"2024-09-17T21:22:24.378500+0000","last_active":"2024-09-17T21:22:49.377593+0000","last_peered":"2024-09-17T21:22:49.377593+0000","last_clean":"2024-09-17T21:22:49.377593+0000","last_became_active":"2024-09-17T21:21:40.321648+0000","last_became_peered":"2024-09-17T21:21:40.321648+0000","last_unstale":"2024-09-17T21:22:49.377593+0000","last_undegraded":"2024-09-17T21:22:49.377593+0000","last_fullsized":"2024-09-17T21:22:49.377593+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:54:37.691837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0071138240000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8592,"num_read_kb":29445,"num_write":954,"num_write_kb":5554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":9885558,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"225'449","reported_seq":5372,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.395698+0000","last_change":"2024-09-17T21:22:49.393985+0000","last_active":"2024-09-17T21:22:49.395698+0000","last_peered":"2024-09-17T21:22:49.395698+0000","last_clean":"2024-09-17T21:22:49.395698+0000","last_became_active":"2024-09-17T21:22:49.389098+0000","last_became_peered":"2024-09-17T21:22:49.389098+0000","last_unstale":"2024-09-17T21:22:49.395698+0000","last_undegraded":"2024-09-17T21:22:49.395698+0000","last_fullsized":"2024-09-17T21:22:49.395698+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:28:53.422353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003029,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5351,"num_read_kb":26972,"num_write":428,"num_write_kb":16010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":385,"num_bytes_recovered":6943878,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":470,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.378624+0000","last_change":"2024-09-17T21:18:03.059715+0000","last_active":"2024-09-17T21:22:49.378624+0000","last_peered":"2024-09-17T21:22:49.378624+0000","last_clean":"2024-09-17T21:22:49.378624+0000","last_became_active":"2024-09-17T21:18:01.213047+0000","last_became_peered":"2024-09-17T21:18:01.213047+0000","last_unstale":"2024-09-17T21:22:49.378624+0000","last_undegraded":"2024-09-17T21:22:49.378624+0000","last_fullsized":"2024-09-17T21:22:49.378624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:46:35.222373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"227'594","reported_seq":13690,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.382362+0000","last_change":"2024-09-17T21:22:24.371187+0000","last_active":"2024-09-17T21:22:49.382362+0000","last_peered":"2024-09-17T21:22:49.382362+0000","last_clean":"2024-09-17T21:22:49.382362+0000","last_became_active":"2024-09-17T21:21:40.318488+0000","last_became_peered":"2024-09-17T21:21:40.318488+0000","last_unstale":"2024-09-17T21:22:49.382362+0000","last_undegraded":"2024-09-17T21:22:49.382362+0000","last_fullsized":"2024-09-17T21:22:49.382362+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:57:56.726468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036709440000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14882,"num_read_kb":61123,"num_write":581,"num_write_kb":23555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":49089084,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":470,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.381228+0000","last_change":"2024-09-17T21:18:03.073440+0000","last_active":"2024-09-17T21:22:49.381228+0000","last_peered":"2024-09-17T21:22:49.381228+0000","last_clean":"2024-09-17T21:22:49.381228+0000","last_became_active":"2024-09-17T21:18:01.048806+0000","last_became_peered":"2024-09-17T21:18:01.048806+0000","last_unstale":"2024-09-17T21:22:49.381228+0000","last_undegraded":"2024-09-17T21:22:49.381228+0000","last_fullsized":"2024-09-17T21:22:49.381228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:26.487163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094161400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"226'352","reported_seq":5610,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382685+0000","last_change":"2024-09-17T21:22:50.390979+0000","last_active":"2024-09-17T21:22:51.382685+0000","last_peered":"2024-09-17T21:22:51.382685+0000","last_clean":"2024-09-17T21:22:51.382685+0000","last_became_active":"2024-09-17T21:22:50.388965+0000","last_became_peered":"2024-09-17T21:22:50.388965+0000","last_unstale":"2024-09-17T21:22:51.382685+0000","last_undegraded":"2024-09-17T21:22:51.382685+0000","last_fullsized":"2024-09-17T21:22:51.382685+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:18:04.741617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079556969999999994,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4656,"num_read_kb":36280,"num_write":392,"num_write_kb":13876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":5244262,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":494,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.378812+0000","last_change":"2024-09-17T21:21:35.319221+0000","last_active":"2024-09-17T21:22:49.378812+0000","last_peered":"2024-09-17T21:22:49.378812+0000","last_clean":"2024-09-17T21:22:49.378812+0000","last_became_active":"2024-09-17T21:21:35.312507+0000","last_became_peered":"2024-09-17T21:21:35.312507+0000","last_unstale":"2024-09-17T21:22:49.378812+0000","last_undegraded":"2024-09-17T21:22:49.378812+0000","last_fullsized":"2024-09-17T21:22:49.378812+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:53.042567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099219900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"226'707","reported_seq":16131,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.380336+0000","last_change":"2024-09-17T21:22:46.361219+0000","last_active":"2024-09-17T21:22:49.380336+0000","last_peered":"2024-09-17T21:22:49.380336+0000","last_clean":"2024-09-17T21:22:49.380336+0000","last_became_active":"2024-09-17T21:22:46.359525+0000","last_became_peered":"2024-09-17T21:22:46.359525+0000","last_unstale":"2024-09-17T21:22:49.380336+0000","last_undegraded":"2024-09-17T21:22:49.380336+0000","last_fullsized":"2024-09-17T21:22:49.380336+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:47.174816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052152359999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13338,"num_read_kb":29382,"num_write":606,"num_write_kb":10802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":367,"num_bytes_recovered":21385444,"num_keys_recovered":118,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":494,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.380660+0000","last_change":"2024-09-17T21:21:35.326814+0000","last_active":"2024-09-17T21:22:49.380660+0000","last_peered":"2024-09-17T21:22:49.380660+0000","last_clean":"2024-09-17T21:22:49.380660+0000","last_became_active":"2024-09-17T21:21:35.325777+0000","last_became_peered":"2024-09-17T21:21:35.325777+0000","last_unstale":"2024-09-17T21:22:49.380660+0000","last_undegraded":"2024-09-17T21:22:49.380660+0000","last_fullsized":"2024-09-17T21:22:49.380660+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:47:00.558900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006222,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"225'1056","reported_seq":33576,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.388088+0000","last_change":"2024-09-17T21:22:44.360752+0000","last_active":"2024-09-17T21:22:49.388088+0000","last_peered":"2024-09-17T21:22:49.388088+0000","last_clean":"2024-09-17T21:22:49.388088+0000","last_became_active":"2024-09-17T21:22:44.352239+0000","last_became_peered":"2024-09-17T21:22:44.352239+0000","last_unstale":"2024-09-17T21:22:49.388088+0000","last_undegraded":"2024-09-17T21:22:49.388088+0000","last_fullsized":"2024-09-17T21:22:49.388088+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":1056,"log_dups_size":0,"ondisk_log_size":1056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:36:58.893607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052118770000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":37177,"num_read_kb":61903,"num_write":871,"num_write_kb":21852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":754,"num_bytes_recovered":58900004,"num_keys_recovered":204,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"227'536","reported_seq":3883,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.385669+0000","last_change":"2024-09-17T21:22:24.383832+0000","last_active":"2024-09-17T21:22:49.385669+0000","last_peered":"2024-09-17T21:22:49.385669+0000","last_clean":"2024-09-17T21:22:49.385669+0000","last_became_active":"2024-09-17T21:21:39.315142+0000","last_became_peered":"2024-09-17T21:21:39.315142+0000","last_unstale":"2024-09-17T21:22:49.385669+0000","last_undegraded":"2024-09-17T21:22:49.385669+0000","last_fullsized":"2024-09-17T21:22:49.385669+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:05:28.580363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061334989999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":3053,"num_read_kb":14423,"num_write":502,"num_write_kb":4818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":7199060,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":408,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.385325+0000","last_change":"2024-09-17T21:22:48.383726+0000","last_active":"2024-09-17T21:22:49.385325+0000","last_peered":"2024-09-17T21:22:49.385325+0000","last_clean":"2024-09-17T21:22:49.385325+0000","last_became_active":"2024-09-17T21:22:48.382842+0000","last_became_peered":"2024-09-17T21:22:48.382842+0000","last_unstale":"2024-09-17T21:22:49.385325+0000","last_undegraded":"2024-09-17T21:22:49.385325+0000","last_fullsized":"2024-09-17T21:22:49.385325+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:30:55.312809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"226'248","reported_seq":9442,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.378961+0000","last_change":"2024-09-17T21:22:45.354353+0000","last_active":"2024-09-17T21:22:49.378961+0000","last_peered":"2024-09-17T21:22:49.378961+0000","last_clean":"2024-09-17T21:22:49.378961+0000","last_became_active":"2024-09-17T21:22:45.352682+0000","last_became_peered":"2024-09-17T21:22:45.352682+0000","last_unstale":"2024-09-17T21:22:49.378961+0000","last_undegraded":"2024-09-17T21:22:49.378961+0000","last_fullsized":"2024-09-17T21:22:49.378961+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:18.942784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048483459999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5340,"num_read_kb":22605,"num_write":215,"num_write_kb":6424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":15742651,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":461,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382421+0000","last_change":"2024-09-17T21:22:50.389225+0000","last_active":"2024-09-17T21:22:51.382421+0000","last_peered":"2024-09-17T21:22:51.382421+0000","last_clean":"2024-09-17T21:22:51.382421+0000","last_became_active":"2024-09-17T21:22:50.388953+0000","last_became_peered":"2024-09-17T21:22:50.388953+0000","last_unstale":"2024-09-17T21:22:51.382421+0000","last_undegraded":"2024-09-17T21:22:51.382421+0000","last_fullsized":"2024-09-17T21:22:51.382421+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:21.062518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"227'1041","reported_seq":8856,"reported_epoch":244,"state":"peering","last_fresh":"2024-09-17T21:22:48.984797+0000","last_change":"2024-09-17T21:22:49.380622+0000","last_active":"2024-09-17T21:22:49.380622+0000","last_peered":"2024-09-17T21:22:48.984797+0000","last_clean":"2024-09-17T21:22:43.336352+0000","last_became_active":"2024-09-17T21:22:47.035089+0000","last_became_peered":"2024-09-17T21:22:47.035089+0000","last_unstale":"2024-09-17T21:22:48.984797+0000","last_undegraded":"2024-09-17T21:22:48.984797+0000","last_fullsized":"2024-09-17T21:22:46.345701+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":1041,"log_dups_size":0,"ondisk_log_size":1041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035942119999999998,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":9666,"num_read_kb":63102,"num_write":1578,"num_write_kb":18132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":894,"num_bytes_recovered":48903910,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":470,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.377043+0000","last_change":"2024-09-17T21:18:03.059156+0000","last_active":"2024-09-17T21:22:49.377043+0000","last_peered":"2024-09-17T21:22:49.377043+0000","last_clean":"2024-09-17T21:22:49.377043+0000","last_became_active":"2024-09-17T21:18:01.049180+0000","last_became_peered":"2024-09-17T21:18:01.049180+0000","last_unstale":"2024-09-17T21:22:49.377043+0000","last_undegraded":"2024-09-17T21:22:49.377043+0000","last_fullsized":"2024-09-17T21:22:49.377043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:27.069794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077408,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"226'656","reported_seq":14590,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.379957+0000","last_change":"2024-09-17T21:22:41.354650+0000","last_active":"2024-09-17T21:22:49.379957+0000","last_peered":"2024-09-17T21:22:49.379957+0000","last_clean":"2024-09-17T21:22:49.379957+0000","last_became_active":"2024-09-17T21:22:41.352429+0000","last_became_peered":"2024-09-17T21:22:41.352429+0000","last_unstale":"2024-09-17T21:22:49.379957+0000","last_undegraded":"2024-09-17T21:22:49.379957+0000","last_fullsized":"2024-09-17T21:22:49.379957+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:30:46.965276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067184260000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14741,"num_read_kb":38983,"num_write":745,"num_write_kb":20177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":27258987,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.385460+0000","last_change":"2024-09-17T21:22:48.387337+0000","last_active":"2024-09-17T21:22:49.385460+0000","last_peered":"2024-09-17T21:22:49.385460+0000","last_clean":"2024-09-17T21:22:49.385460+0000","last_became_active":"2024-09-17T21:22:48.386572+0000","last_became_peered":"2024-09-17T21:22:48.386572+0000","last_unstale":"2024-09-17T21:22:49.385460+0000","last_undegraded":"2024-09-17T21:22:49.385460+0000","last_fullsized":"2024-09-17T21:22:49.385460+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:55:57.934354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":185,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.385193+0000","last_change":"2024-09-17T21:21:17.347819+0000","last_active":"2024-09-17T21:22:49.385193+0000","last_peered":"2024-09-17T21:22:49.385193+0000","last_clean":"2024-09-17T21:22:49.385193+0000","last_became_active":"2024-09-17T21:21:17.337621+0000","last_became_peered":"2024-09-17T21:21:17.337621+0000","last_unstale":"2024-09-17T21:22:49.385193+0000","last_undegraded":"2024-09-17T21:22:49.385193+0000","last_fullsized":"2024-09-17T21:22:49.385193+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_clean_scrub_stamp":"2024-09-17T21:17:55.982339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:24:32.371870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"224'616","reported_seq":14093,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382313+0000","last_change":"2024-09-17T21:22:50.383917+0000","last_active":"2024-09-17T21:22:51.382313+0000","last_peered":"2024-09-17T21:22:51.382313+0000","last_clean":"2024-09-17T21:22:51.382313+0000","last_became_active":"2024-09-17T21:22:50.382396+0000","last_became_peered":"2024-09-17T21:22:50.382396+0000","last_unstale":"2024-09-17T21:22:51.382313+0000","last_undegraded":"2024-09-17T21:22:51.382313+0000","last_fullsized":"2024-09-17T21:22:51.382313+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:13:13.848853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001602121,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":13993,"num_read_kb":68437,"num_write":586,"num_write_kb":29114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":40776135,"num_keys_recovered":165,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"227'699","reported_seq":11102,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.395662+0000","last_change":"2024-09-17T21:22:49.395344+0000","last_active":"2024-09-17T21:22:49.395662+0000","last_peered":"2024-09-17T21:22:49.395662+0000","last_clean":"2024-09-17T21:22:49.395662+0000","last_became_active":"2024-09-17T21:22:49.390523+0000","last_became_peered":"2024-09-17T21:22:49.390523+0000","last_unstale":"2024-09-17T21:22:49.395662+0000","last_undegraded":"2024-09-17T21:22:49.395662+0000","last_fullsized":"2024-09-17T21:22:49.395662+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:55:28.950314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029497360000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":11677,"num_read_kb":43614,"num_write":1089,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":728,"num_bytes_recovered":30193805,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"227'428","reported_seq":8262,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.397894+0000","last_change":"2024-09-17T21:22:49.397589+0000","last_active":"2024-09-17T21:22:49.397894+0000","last_peered":"2024-09-17T21:22:49.397894+0000","last_clean":"2024-09-17T21:22:49.397894+0000","last_became_active":"2024-09-17T21:22:49.394384+0000","last_became_peered":"2024-09-17T21:22:49.394384+0000","last_unstale":"2024-09-17T21:22:49.397894+0000","last_undegraded":"2024-09-17T21:22:49.397894+0000","last_fullsized":"2024-09-17T21:22:49.397894+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:24:21.885168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045615830000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7186,"num_read_kb":40870,"num_write":369,"num_write_kb":16675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":179794,"num_keys_recovered":2,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"226'376","reported_seq":4916,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.380188+0000","last_change":"2024-09-17T21:22:24.375850+0000","last_active":"2024-09-17T21:22:49.380188+0000","last_peered":"2024-09-17T21:22:49.380188+0000","last_clean":"2024-09-17T21:22:49.380188+0000","last_became_active":"2024-09-17T21:21:40.319613+0000","last_became_peered":"2024-09-17T21:21:40.319613+0000","last_unstale":"2024-09-17T21:22:49.380188+0000","last_undegraded":"2024-09-17T21:22:49.380188+0000","last_fullsized":"2024-09-17T21:22:49.380188+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:26:53.101370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038796239999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4694,"num_read_kb":23619,"num_write":320,"num_write_kb":13765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":4886965,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"227'452","reported_seq":13417,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.379750+0000","last_change":"2024-09-17T21:22:43.339607+0000","last_active":"2024-09-17T21:22:49.379750+0000","last_peered":"2024-09-17T21:22:49.379750+0000","last_clean":"2024-09-17T21:22:49.379750+0000","last_became_active":"2024-09-17T21:22:43.338399+0000","last_became_peered":"2024-09-17T21:22:43.338399+0000","last_unstale":"2024-09-17T21:22:49.379750+0000","last_undegraded":"2024-09-17T21:22:49.379750+0000","last_fullsized":"2024-09-17T21:22:49.379750+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:30:14.210043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050460430000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":15379,"num_read_kb":29065,"num_write":580,"num_write_kb":14319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":5960704,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"221'526","reported_seq":15415,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.378976+0000","last_change":"2024-09-17T21:22:46.356687+0000","last_active":"2024-09-17T21:22:49.378976+0000","last_peered":"2024-09-17T21:22:49.378976+0000","last_clean":"2024-09-17T21:22:49.378976+0000","last_became_active":"2024-09-17T21:22:46.349835+0000","last_became_peered":"2024-09-17T21:22:46.349835+0000","last_unstale":"2024-09-17T21:22:49.378976+0000","last_undegraded":"2024-09-17T21:22:49.378976+0000","last_fullsized":"2024-09-17T21:22:49.378976+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:51:30.571949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00482724,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15785,"num_read_kb":43220,"num_write":654,"num_write_kb":16401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":28397872,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"227'623","reported_seq":14440,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.806310+0000","last_change":"2024-09-17T21:22:49.806087+0000","last_active":"2024-09-17T21:22:49.806310+0000","last_peered":"2024-09-17T21:22:49.806310+0000","last_clean":"2024-09-17T21:22:49.806310+0000","last_became_active":"2024-09-17T21:22:49.394154+0000","last_became_peered":"2024-09-17T21:22:49.394154+0000","last_unstale":"2024-09-17T21:22:49.806310+0000","last_undegraded":"2024-09-17T21:22:49.806310+0000","last_fullsized":"2024-09-17T21:22:49.806310+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:24:57.185890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057317679999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":15141,"num_read_kb":70274,"num_write":535,"num_write_kb":35852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":91925553,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"226'464","reported_seq":7119,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.388236+0000","last_change":"2024-09-17T21:22:48.398780+0000","last_active":"2024-09-17T21:22:49.388236+0000","last_peered":"2024-09-17T21:22:49.388236+0000","last_clean":"2024-09-17T21:22:49.388236+0000","last_became_active":"2024-09-17T21:22:48.392669+0000","last_became_peered":"2024-09-17T21:22:48.392669+0000","last_unstale":"2024-09-17T21:22:49.388236+0000","last_undegraded":"2024-09-17T21:22:49.388236+0000","last_fullsized":"2024-09-17T21:22:49.388236+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:18:23.844174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0091147729999999996,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7427,"num_read_kb":28404,"num_write":434,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":4659071,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"222'271","reported_seq":7399,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.384802+0000","last_change":"2024-09-17T21:22:48.384974+0000","last_active":"2024-09-17T21:22:49.384802+0000","last_peered":"2024-09-17T21:22:49.384802+0000","last_clean":"2024-09-17T21:22:49.384802+0000","last_became_active":"2024-09-17T21:22:48.381106+0000","last_became_peered":"2024-09-17T21:22:48.381106+0000","last_unstale":"2024-09-17T21:22:49.384802+0000","last_undegraded":"2024-09-17T21:22:49.384802+0000","last_fullsized":"2024-09-17T21:22:49.384802+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:11:47.017985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017482940000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7827,"num_read_kb":19883,"num_write":219,"num_write_kb":3298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":6733181,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"226'404","reported_seq":5774,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.377779+0000","last_change":"2024-09-17T21:22:24.385309+0000","last_active":"2024-09-17T21:22:49.377779+0000","last_peered":"2024-09-17T21:22:49.377779+0000","last_clean":"2024-09-17T21:22:49.377779+0000","last_became_active":"2024-09-17T21:18:11.299946+0000","last_became_peered":"2024-09-17T21:18:11.299946+0000","last_unstale":"2024-09-17T21:22:49.377779+0000","last_undegraded":"2024-09-17T21:22:49.377779+0000","last_fullsized":"2024-09-17T21:22:49.377779+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:16:16.699933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015128413,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5168,"num_read_kb":22085,"num_write":382,"num_write_kb":4452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"225'570","reported_seq":15991,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.381896+0000","last_change":"2024-09-17T21:22:24.368740+0000","last_active":"2024-09-17T21:22:49.381896+0000","last_peered":"2024-09-17T21:22:49.381896+0000","last_clean":"2024-09-17T21:22:49.381896+0000","last_became_active":"2024-09-17T21:22:09.609844+0000","last_became_peered":"2024-09-17T21:22:09.609844+0000","last_unstale":"2024-09-17T21:22:49.381896+0000","last_undegraded":"2024-09-17T21:22:49.381896+0000","last_fullsized":"2024-09-17T21:22:49.381896+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:03:30.364818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018557070000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":17911,"num_read_kb":38759,"num_write":589,"num_write_kb":6968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":11589682,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"226'651","reported_seq":12601,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.377308+0000","last_change":"2024-09-17T21:22:41.357868+0000","last_active":"2024-09-17T21:22:49.377308+0000","last_peered":"2024-09-17T21:22:49.377308+0000","last_clean":"2024-09-17T21:22:49.377308+0000","last_became_active":"2024-09-17T21:22:41.354851+0000","last_became_peered":"2024-09-17T21:22:41.354851+0000","last_unstale":"2024-09-17T21:22:49.377308+0000","last_undegraded":"2024-09-17T21:22:49.377308+0000","last_fullsized":"2024-09-17T21:22:49.377308+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:47:04.462955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011937984,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12008,"num_read_kb":52136,"num_write":617,"num_write_kb":5443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":5557098,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":304,"num_object_clones":0,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":300,"num_read":353905,"num_read_kb":1145909,"num_write":19949,"num_write_kb":385384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11819,"num_bytes_recovered":614462318,"num_keys_recovered":1529,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8194,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":64,"acting":63,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574911,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60440,"kb_used_data":2884,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311400,"statfs":{"total":96636764160,"available":96574873600,"internally_reserved":0,"allocated":2953216,"data_stored":1252104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":58915268},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66584,"kb_used_data":3464,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305256,"statfs":{"total":96636764160,"available":96568582144,"internally_reserved":0,"allocated":3547136,"data_stored":1842472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":64616900},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83224,"kb_used_data":3460,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288616,"statfs":{"total":96636764160,"available":96551542784,"internally_reserved":0,"allocated":3543040,"data_stored":1842453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":81656262},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56288,"kb_used_data":2888,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315552,"statfs":{"total":96636764160,"available":96579125248,"internally_reserved":0,"allocated":2957312,"data_stored":1252123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56284,"kb_used_data":2884,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315556,"statfs":{"total":96636764160,"available":96579129344,"internally_reserved":0,"allocated":2953216,"data_stored":1252104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":231,"seq":992137445382,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30628,"kb_used_data":2892,"kb_used_omap":20,"kb_used_meta":27691,"kb_avail":94341212,"statfs":{"total":96636764160,"available":96605401088,"internally_reserved":0,"allocated":2961408,"data_stored":1257187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20966,"internal_metadata":28356122},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8194,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:22:54.380 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T21:22:54.716 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:22:54.716 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"224a7a29-e112-472e-8a13-f67345d6d123","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":312,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":247,"num_osds":6,"num_up_osds":6,"osd_up_since":1726608152,"num_in_osds":6,"osd_in_since":1726608157,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":38},{"state_name":"peering","count":2},{"state_name":"active+undersized+remapped","count":1}],"num_pgs":41,"num_pools":3,"num_objects":308,"data_bytes":590406,"bytes_used":361930752,"bytes_avail":579458654208,"bytes_total":579820584960,"inactive_pgs_ratio":0.048780485987663269,"recovering_objects_per_sec":75,"recovering_bytes_per_sec":17661513,"recovering_keys_per_sec":12,"num_objects_recovered":398,"num_bytes_recovered":92573252,"num_keys_recovered":66,"read_bytes_sec":24188481,"write_bytes_sec":9849617,"read_op_per_sec":6111,"write_op_per_sec":229},"fsmap":{"epoch":1,"btime":"2024-09-17T21:17:42:539460+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-17T21:18:55.549139+0000","services":{}},"progress_events":{}} 2024-09-17T21:22:54.733 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-17T21:22:54.733 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:22:55.012 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:22:55.013 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T21:22:55.028 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-17T21:22:53.613007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":308,"num_object_clones":0,"num_object_copies":616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":300,"num_read":353971,"num_read_kb":1145964,"num_write":20052,"num_write_kb":387014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11832,"num_bytes_recovered":615643111,"num_keys_recovered":1529,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18605,"ondisk_log_size":18605,"up":82,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":353448,"kb_used_data":18472,"kb_used_omap":28,"kb_used_meta":334819,"kb_avail":565877592,"statfs":{"total":579820584960,"available":579458654208,"internally_reserved":0,"allocated":18915328,"data_stored":8698443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28944,"internal_metadata":342855408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-92,"num_objects_misplaced":-10,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":32034,"num_read_kb":123813,"num_write":1204,"num_write_kb":50417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":92573252,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.241524"},"pg_stats":[{"pgid":"3.1f","version":"227'759","reported_seq":4114,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.380587+0000","last_change":"2024-09-17T21:22:40.360129+0000","last_active":"2024-09-17T21:22:49.380587+0000","last_peered":"2024-09-17T21:22:49.380587+0000","last_clean":"2024-09-17T21:22:49.380587+0000","last_became_active":"2024-09-17T21:22:40.357329+0000","last_became_peered":"2024-09-17T21:22:40.357329+0000","last_unstale":"2024-09-17T21:22:49.380587+0000","last_undegraded":"2024-09-17T21:22:49.380587+0000","last_fullsized":"2024-09-17T21:22:49.380587+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:01:10.890769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010221002999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5921,"num_read_kb":16779,"num_write":1026,"num_write_kb":3495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":5143732,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"227'851","reported_seq":9553,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.382138+0000","last_change":"2024-09-17T21:22:40.355216+0000","last_active":"2024-09-17T21:22:49.382138+0000","last_peered":"2024-09-17T21:22:49.382138+0000","last_clean":"2024-09-17T21:22:49.382138+0000","last_became_active":"2024-09-17T21:22:40.353016+0000","last_became_peered":"2024-09-17T21:22:40.353016+0000","last_unstale":"2024-09-17T21:22:49.382138+0000","last_undegraded":"2024-09-17T21:22:49.382138+0000","last_fullsized":"2024-09-17T21:22:49.382138+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":851,"log_dups_size":0,"ondisk_log_size":851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:11.494395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084510550000000007,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10998,"num_read_kb":53173,"num_write":1081,"num_write_kb":18708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":50670363,"num_keys_recovered":90,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"227'833","reported_seq":8684,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.388425+0000","last_change":"2024-09-17T21:22:48.408191+0000","last_active":"2024-09-17T21:22:49.388425+0000","last_peered":"2024-09-17T21:22:49.388425+0000","last_clean":"2024-09-17T21:22:49.388425+0000","last_became_active":"2024-09-17T21:22:48.401326+0000","last_became_peered":"2024-09-17T21:22:48.401326+0000","last_unstale":"2024-09-17T21:22:49.388425+0000","last_undegraded":"2024-09-17T21:22:49.388425+0000","last_fullsized":"2024-09-17T21:22:49.388425+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:19:25.050335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082452470000000007,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":10230,"num_read_kb":27857,"num_write":978,"num_write_kb":10466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":12493301,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"227'376","reported_seq":4042,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.388018+0000","last_change":"2024-09-17T21:22:48.406130+0000","last_active":"2024-09-17T21:22:49.388018+0000","last_peered":"2024-09-17T21:22:49.388018+0000","last_clean":"2024-09-17T21:22:49.388018+0000","last_became_active":"2024-09-17T21:22:48.401237+0000","last_became_peered":"2024-09-17T21:22:48.401237+0000","last_unstale":"2024-09-17T21:22:49.388018+0000","last_undegraded":"2024-09-17T21:22:49.388018+0000","last_fullsized":"2024-09-17T21:22:49.388018+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:39:30.285487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061565819999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":2726,"num_read_kb":23738,"num_write":383,"num_write_kb":5937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":9397808,"num_keys_recovered":202,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"227'710","reported_seq":8287,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.382726+0000","last_change":"2024-09-17T21:22:24.369716+0000","last_active":"2024-09-17T21:22:49.382726+0000","last_peered":"2024-09-17T21:22:49.382726+0000","last_clean":"2024-09-17T21:22:49.382726+0000","last_became_active":"2024-09-17T21:21:41.329830+0000","last_became_peered":"2024-09-17T21:21:41.329830+0000","last_unstale":"2024-09-17T21:22:49.382726+0000","last_undegraded":"2024-09-17T21:22:49.382726+0000","last_fullsized":"2024-09-17T21:22:49.382726+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:42:50.772772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026038680000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":11634,"num_read_kb":21103,"num_write":555,"num_write_kb":4725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":7872032,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"226'739","reported_seq":13907,"reported_epoch":245,"state":"peering","last_fresh":"2024-09-17T21:22:49.384724+0000","last_change":"2024-09-17T21:22:49.384724+0000","last_active":"2024-09-17T21:22:45.342106+0000","last_peered":"2024-09-17T21:22:43.593653+0000","last_clean":"2024-09-17T21:22:36.675355+0000","last_became_active":"2024-09-17T21:22:39.364106+0000","last_became_peered":"2024-09-17T21:22:39.364106+0000","last_unstale":"2024-09-17T21:22:49.384724+0000","last_undegraded":"2024-09-17T21:22:49.384724+0000","last_fullsized":"2024-09-17T21:22:49.384724+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056973060000000001,"stat_sum":{"num_bytes":19,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":16893,"num_read_kb":53539,"num_write":669,"num_write_kb":14565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":647718,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"226'476","reported_seq":12468,"reported_epoch":245,"state":"active+undersized+remapped","last_fresh":"2024-09-17T21:22:49.449625+0000","last_change":"2024-09-17T21:22:49.450079+0000","last_active":"2024-09-17T21:22:49.449625+0000","last_peered":"2024-09-17T21:22:49.449625+0000","last_clean":"2024-09-17T21:22:43.331395+0000","last_became_active":"2024-09-17T21:22:47.035944+0000","last_became_peered":"2024-09-17T21:22:47.035944+0000","last_unstale":"2024-09-17T21:22:49.449625+0000","last_undegraded":"2024-09-17T21:22:49.449625+0000","last_fullsized":"2024-09-17T21:22:46.342866+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0077005709999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":12648,"num_read_kb":27025,"num_write":374,"num_write_kb":5615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":3100207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1],"avail_no_missing":["1","3","4"],"object_location_counts":[{"shards":"1","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"226'318","reported_seq":3035,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382486+0000","last_change":"2024-09-17T21:22:50.387935+0000","last_active":"2024-09-17T21:22:51.382486+0000","last_peered":"2024-09-17T21:22:51.382486+0000","last_clean":"2024-09-17T21:22:51.382486+0000","last_became_active":"2024-09-17T21:22:50.385975+0000","last_became_peered":"2024-09-17T21:22:50.385975+0000","last_unstale":"2024-09-17T21:22:51.382486+0000","last_undegraded":"2024-09-17T21:22:51.382486+0000","last_fullsized":"2024-09-17T21:22:51.382486+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:26:13.518654+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022643899999999998,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2392,"num_read_kb":9139,"num_write":304,"num_write_kb":2898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":9226431,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"225'393","reported_seq":7339,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.380742+0000","last_change":"2024-09-17T21:22:43.342179+0000","last_active":"2024-09-17T21:22:49.380742+0000","last_peered":"2024-09-17T21:22:49.380742+0000","last_clean":"2024-09-17T21:22:49.380742+0000","last_became_active":"2024-09-17T21:22:43.340739+0000","last_became_peered":"2024-09-17T21:22:43.340739+0000","last_unstale":"2024-09-17T21:22:49.380742+0000","last_undegraded":"2024-09-17T21:22:49.380742+0000","last_fullsized":"2024-09-17T21:22:49.380742+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:32:48.751230+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00138835,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7054,"num_read_kb":21313,"num_write":475,"num_write_kb":6946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":719,"num_bytes_recovered":31066012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"224'602","reported_seq":22248,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.388691+0000","last_change":"2024-09-17T21:22:48.400263+0000","last_active":"2024-09-17T21:22:49.388691+0000","last_peered":"2024-09-17T21:22:49.388691+0000","last_clean":"2024-09-17T21:22:49.388691+0000","last_became_active":"2024-09-17T21:22:48.394816+0000","last_became_peered":"2024-09-17T21:22:48.394816+0000","last_unstale":"2024-09-17T21:22:49.388691+0000","last_undegraded":"2024-09-17T21:22:49.388691+0000","last_fullsized":"2024-09-17T21:22:49.388691+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:05.075504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075408430000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":22417,"num_read_kb":27659,"num_write":858,"num_write_kb":13022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":3432016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"223'754","reported_seq":7154,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.377593+0000","last_change":"2024-09-17T21:22:24.378500+0000","last_active":"2024-09-17T21:22:49.377593+0000","last_peered":"2024-09-17T21:22:49.377593+0000","last_clean":"2024-09-17T21:22:49.377593+0000","last_became_active":"2024-09-17T21:21:40.321648+0000","last_became_peered":"2024-09-17T21:21:40.321648+0000","last_unstale":"2024-09-17T21:22:49.377593+0000","last_undegraded":"2024-09-17T21:22:49.377593+0000","last_fullsized":"2024-09-17T21:22:49.377593+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:54:37.691837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0071138240000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8592,"num_read_kb":29445,"num_write":954,"num_write_kb":5554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":9885558,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"225'449","reported_seq":5372,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.395698+0000","last_change":"2024-09-17T21:22:49.393985+0000","last_active":"2024-09-17T21:22:49.395698+0000","last_peered":"2024-09-17T21:22:49.395698+0000","last_clean":"2024-09-17T21:22:49.395698+0000","last_became_active":"2024-09-17T21:22:49.389098+0000","last_became_peered":"2024-09-17T21:22:49.389098+0000","last_unstale":"2024-09-17T21:22:49.395698+0000","last_undegraded":"2024-09-17T21:22:49.395698+0000","last_fullsized":"2024-09-17T21:22:49.395698+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:28:53.422353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003029,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5351,"num_read_kb":26972,"num_write":428,"num_write_kb":16010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":385,"num_bytes_recovered":6943878,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":470,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.378624+0000","last_change":"2024-09-17T21:18:03.059715+0000","last_active":"2024-09-17T21:22:49.378624+0000","last_peered":"2024-09-17T21:22:49.378624+0000","last_clean":"2024-09-17T21:22:49.378624+0000","last_became_active":"2024-09-17T21:18:01.213047+0000","last_became_peered":"2024-09-17T21:18:01.213047+0000","last_unstale":"2024-09-17T21:22:49.378624+0000","last_undegraded":"2024-09-17T21:22:49.378624+0000","last_fullsized":"2024-09-17T21:22:49.378624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:46:35.222373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"227'594","reported_seq":13690,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.382362+0000","last_change":"2024-09-17T21:22:24.371187+0000","last_active":"2024-09-17T21:22:49.382362+0000","last_peered":"2024-09-17T21:22:49.382362+0000","last_clean":"2024-09-17T21:22:49.382362+0000","last_became_active":"2024-09-17T21:21:40.318488+0000","last_became_peered":"2024-09-17T21:21:40.318488+0000","last_unstale":"2024-09-17T21:22:49.382362+0000","last_undegraded":"2024-09-17T21:22:49.382362+0000","last_fullsized":"2024-09-17T21:22:49.382362+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:57:56.726468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036709440000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14882,"num_read_kb":61123,"num_write":581,"num_write_kb":23555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":49089084,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":470,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.381228+0000","last_change":"2024-09-17T21:18:03.073440+0000","last_active":"2024-09-17T21:22:49.381228+0000","last_peered":"2024-09-17T21:22:49.381228+0000","last_clean":"2024-09-17T21:22:49.381228+0000","last_became_active":"2024-09-17T21:18:01.048806+0000","last_became_peered":"2024-09-17T21:18:01.048806+0000","last_unstale":"2024-09-17T21:22:49.381228+0000","last_undegraded":"2024-09-17T21:22:49.381228+0000","last_fullsized":"2024-09-17T21:22:49.381228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:26.487163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094161400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"226'352","reported_seq":5610,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382685+0000","last_change":"2024-09-17T21:22:50.390979+0000","last_active":"2024-09-17T21:22:51.382685+0000","last_peered":"2024-09-17T21:22:51.382685+0000","last_clean":"2024-09-17T21:22:51.382685+0000","last_became_active":"2024-09-17T21:22:50.388965+0000","last_became_peered":"2024-09-17T21:22:50.388965+0000","last_unstale":"2024-09-17T21:22:51.382685+0000","last_undegraded":"2024-09-17T21:22:51.382685+0000","last_fullsized":"2024-09-17T21:22:51.382685+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:18:04.741617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079556969999999994,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4656,"num_read_kb":36280,"num_write":392,"num_write_kb":13876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":5244262,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":494,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.378812+0000","last_change":"2024-09-17T21:21:35.319221+0000","last_active":"2024-09-17T21:22:49.378812+0000","last_peered":"2024-09-17T21:22:49.378812+0000","last_clean":"2024-09-17T21:22:49.378812+0000","last_became_active":"2024-09-17T21:21:35.312507+0000","last_became_peered":"2024-09-17T21:21:35.312507+0000","last_unstale":"2024-09-17T21:22:49.378812+0000","last_undegraded":"2024-09-17T21:22:49.378812+0000","last_fullsized":"2024-09-17T21:22:49.378812+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:53.042567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099219900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"226'707","reported_seq":16131,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.380336+0000","last_change":"2024-09-17T21:22:46.361219+0000","last_active":"2024-09-17T21:22:49.380336+0000","last_peered":"2024-09-17T21:22:49.380336+0000","last_clean":"2024-09-17T21:22:49.380336+0000","last_became_active":"2024-09-17T21:22:46.359525+0000","last_became_peered":"2024-09-17T21:22:46.359525+0000","last_unstale":"2024-09-17T21:22:49.380336+0000","last_undegraded":"2024-09-17T21:22:49.380336+0000","last_fullsized":"2024-09-17T21:22:49.380336+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:47.174816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052152359999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13338,"num_read_kb":29382,"num_write":606,"num_write_kb":10802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":367,"num_bytes_recovered":21385444,"num_keys_recovered":118,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":494,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.380660+0000","last_change":"2024-09-17T21:21:35.326814+0000","last_active":"2024-09-17T21:22:49.380660+0000","last_peered":"2024-09-17T21:22:49.380660+0000","last_clean":"2024-09-17T21:22:49.380660+0000","last_became_active":"2024-09-17T21:21:35.325777+0000","last_became_peered":"2024-09-17T21:21:35.325777+0000","last_unstale":"2024-09-17T21:22:49.380660+0000","last_undegraded":"2024-09-17T21:22:49.380660+0000","last_fullsized":"2024-09-17T21:22:49.380660+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:47:00.558900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006222,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"225'1056","reported_seq":33576,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.388088+0000","last_change":"2024-09-17T21:22:44.360752+0000","last_active":"2024-09-17T21:22:49.388088+0000","last_peered":"2024-09-17T21:22:49.388088+0000","last_clean":"2024-09-17T21:22:49.388088+0000","last_became_active":"2024-09-17T21:22:44.352239+0000","last_became_peered":"2024-09-17T21:22:44.352239+0000","last_unstale":"2024-09-17T21:22:49.388088+0000","last_undegraded":"2024-09-17T21:22:49.388088+0000","last_fullsized":"2024-09-17T21:22:49.388088+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":1056,"log_dups_size":0,"ondisk_log_size":1056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:36:58.893607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052118770000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":37177,"num_read_kb":61903,"num_write":871,"num_write_kb":21852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":754,"num_bytes_recovered":58900004,"num_keys_recovered":204,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"227'536","reported_seq":3883,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.385669+0000","last_change":"2024-09-17T21:22:24.383832+0000","last_active":"2024-09-17T21:22:49.385669+0000","last_peered":"2024-09-17T21:22:49.385669+0000","last_clean":"2024-09-17T21:22:49.385669+0000","last_became_active":"2024-09-17T21:21:39.315142+0000","last_became_peered":"2024-09-17T21:21:39.315142+0000","last_unstale":"2024-09-17T21:22:49.385669+0000","last_undegraded":"2024-09-17T21:22:49.385669+0000","last_fullsized":"2024-09-17T21:22:49.385669+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:05:28.580363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061334989999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":3053,"num_read_kb":14423,"num_write":502,"num_write_kb":4818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":7199060,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":408,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.385325+0000","last_change":"2024-09-17T21:22:48.383726+0000","last_active":"2024-09-17T21:22:49.385325+0000","last_peered":"2024-09-17T21:22:49.385325+0000","last_clean":"2024-09-17T21:22:49.385325+0000","last_became_active":"2024-09-17T21:22:48.382842+0000","last_became_peered":"2024-09-17T21:22:48.382842+0000","last_unstale":"2024-09-17T21:22:49.385325+0000","last_undegraded":"2024-09-17T21:22:49.385325+0000","last_fullsized":"2024-09-17T21:22:49.385325+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:30:55.312809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"226'248","reported_seq":9442,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.378961+0000","last_change":"2024-09-17T21:22:45.354353+0000","last_active":"2024-09-17T21:22:49.378961+0000","last_peered":"2024-09-17T21:22:49.378961+0000","last_clean":"2024-09-17T21:22:49.378961+0000","last_became_active":"2024-09-17T21:22:45.352682+0000","last_became_peered":"2024-09-17T21:22:45.352682+0000","last_unstale":"2024-09-17T21:22:49.378961+0000","last_undegraded":"2024-09-17T21:22:49.378961+0000","last_fullsized":"2024-09-17T21:22:49.378961+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:18.942784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048483459999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5340,"num_read_kb":22605,"num_write":215,"num_write_kb":6424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":15742651,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":461,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382421+0000","last_change":"2024-09-17T21:22:50.389225+0000","last_active":"2024-09-17T21:22:51.382421+0000","last_peered":"2024-09-17T21:22:51.382421+0000","last_clean":"2024-09-17T21:22:51.382421+0000","last_became_active":"2024-09-17T21:22:50.388953+0000","last_became_peered":"2024-09-17T21:22:50.388953+0000","last_unstale":"2024-09-17T21:22:51.382421+0000","last_undegraded":"2024-09-17T21:22:51.382421+0000","last_fullsized":"2024-09-17T21:22:51.382421+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:21.062518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"227'1041","reported_seq":8856,"reported_epoch":244,"state":"peering","last_fresh":"2024-09-17T21:22:48.984797+0000","last_change":"2024-09-17T21:22:49.380622+0000","last_active":"2024-09-17T21:22:49.380622+0000","last_peered":"2024-09-17T21:22:48.984797+0000","last_clean":"2024-09-17T21:22:43.336352+0000","last_became_active":"2024-09-17T21:22:47.035089+0000","last_became_peered":"2024-09-17T21:22:47.035089+0000","last_unstale":"2024-09-17T21:22:48.984797+0000","last_undegraded":"2024-09-17T21:22:48.984797+0000","last_fullsized":"2024-09-17T21:22:46.345701+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":1041,"log_dups_size":0,"ondisk_log_size":1041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035942119999999998,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":9666,"num_read_kb":63102,"num_write":1578,"num_write_kb":18132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":894,"num_bytes_recovered":48903910,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":470,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.377043+0000","last_change":"2024-09-17T21:18:03.059156+0000","last_active":"2024-09-17T21:22:49.377043+0000","last_peered":"2024-09-17T21:22:49.377043+0000","last_clean":"2024-09-17T21:22:49.377043+0000","last_became_active":"2024-09-17T21:18:01.049180+0000","last_became_peered":"2024-09-17T21:18:01.049180+0000","last_unstale":"2024-09-17T21:22:49.377043+0000","last_undegraded":"2024-09-17T21:22:49.377043+0000","last_fullsized":"2024-09-17T21:22:49.377043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:27.069794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077408,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"226'656","reported_seq":14590,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.379957+0000","last_change":"2024-09-17T21:22:41.354650+0000","last_active":"2024-09-17T21:22:49.379957+0000","last_peered":"2024-09-17T21:22:49.379957+0000","last_clean":"2024-09-17T21:22:49.379957+0000","last_became_active":"2024-09-17T21:22:41.352429+0000","last_became_peered":"2024-09-17T21:22:41.352429+0000","last_unstale":"2024-09-17T21:22:49.379957+0000","last_undegraded":"2024-09-17T21:22:49.379957+0000","last_fullsized":"2024-09-17T21:22:49.379957+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:30:46.965276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067184260000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14741,"num_read_kb":38983,"num_write":745,"num_write_kb":20177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":27258987,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.385460+0000","last_change":"2024-09-17T21:22:48.387337+0000","last_active":"2024-09-17T21:22:49.385460+0000","last_peered":"2024-09-17T21:22:49.385460+0000","last_clean":"2024-09-17T21:22:49.385460+0000","last_became_active":"2024-09-17T21:22:48.386572+0000","last_became_peered":"2024-09-17T21:22:48.386572+0000","last_unstale":"2024-09-17T21:22:49.385460+0000","last_undegraded":"2024-09-17T21:22:49.385460+0000","last_fullsized":"2024-09-17T21:22:49.385460+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:55:57.934354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":185,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.385193+0000","last_change":"2024-09-17T21:21:17.347819+0000","last_active":"2024-09-17T21:22:49.385193+0000","last_peered":"2024-09-17T21:22:49.385193+0000","last_clean":"2024-09-17T21:22:49.385193+0000","last_became_active":"2024-09-17T21:21:17.337621+0000","last_became_peered":"2024-09-17T21:21:17.337621+0000","last_unstale":"2024-09-17T21:22:49.385193+0000","last_undegraded":"2024-09-17T21:22:49.385193+0000","last_fullsized":"2024-09-17T21:22:49.385193+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_clean_scrub_stamp":"2024-09-17T21:17:55.982339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:24:32.371870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"224'616","reported_seq":14093,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382313+0000","last_change":"2024-09-17T21:22:50.383917+0000","last_active":"2024-09-17T21:22:51.382313+0000","last_peered":"2024-09-17T21:22:51.382313+0000","last_clean":"2024-09-17T21:22:51.382313+0000","last_became_active":"2024-09-17T21:22:50.382396+0000","last_became_peered":"2024-09-17T21:22:50.382396+0000","last_unstale":"2024-09-17T21:22:51.382313+0000","last_undegraded":"2024-09-17T21:22:51.382313+0000","last_fullsized":"2024-09-17T21:22:51.382313+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:13:13.848853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001602121,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":13993,"num_read_kb":68437,"num_write":586,"num_write_kb":29114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":40776135,"num_keys_recovered":165,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"227'699","reported_seq":11102,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.395662+0000","last_change":"2024-09-17T21:22:49.395344+0000","last_active":"2024-09-17T21:22:49.395662+0000","last_peered":"2024-09-17T21:22:49.395662+0000","last_clean":"2024-09-17T21:22:49.395662+0000","last_became_active":"2024-09-17T21:22:49.390523+0000","last_became_peered":"2024-09-17T21:22:49.390523+0000","last_unstale":"2024-09-17T21:22:49.395662+0000","last_undegraded":"2024-09-17T21:22:49.395662+0000","last_fullsized":"2024-09-17T21:22:49.395662+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:55:28.950314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029497360000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":11677,"num_read_kb":43614,"num_write":1089,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":728,"num_bytes_recovered":30193805,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"227'428","reported_seq":8262,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.397894+0000","last_change":"2024-09-17T21:22:49.397589+0000","last_active":"2024-09-17T21:22:49.397894+0000","last_peered":"2024-09-17T21:22:49.397894+0000","last_clean":"2024-09-17T21:22:49.397894+0000","last_became_active":"2024-09-17T21:22:49.394384+0000","last_became_peered":"2024-09-17T21:22:49.394384+0000","last_unstale":"2024-09-17T21:22:49.397894+0000","last_undegraded":"2024-09-17T21:22:49.397894+0000","last_fullsized":"2024-09-17T21:22:49.397894+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:24:21.885168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045615830000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7186,"num_read_kb":40870,"num_write":369,"num_write_kb":16675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":179794,"num_keys_recovered":2,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"226'376","reported_seq":4916,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.380188+0000","last_change":"2024-09-17T21:22:24.375850+0000","last_active":"2024-09-17T21:22:49.380188+0000","last_peered":"2024-09-17T21:22:49.380188+0000","last_clean":"2024-09-17T21:22:49.380188+0000","last_became_active":"2024-09-17T21:21:40.319613+0000","last_became_peered":"2024-09-17T21:21:40.319613+0000","last_unstale":"2024-09-17T21:22:49.380188+0000","last_undegraded":"2024-09-17T21:22:49.380188+0000","last_fullsized":"2024-09-17T21:22:49.380188+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:26:53.101370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038796239999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4694,"num_read_kb":23619,"num_write":320,"num_write_kb":13765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":4886965,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"227'452","reported_seq":13417,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.379750+0000","last_change":"2024-09-17T21:22:43.339607+0000","last_active":"2024-09-17T21:22:49.379750+0000","last_peered":"2024-09-17T21:22:49.379750+0000","last_clean":"2024-09-17T21:22:49.379750+0000","last_became_active":"2024-09-17T21:22:43.338399+0000","last_became_peered":"2024-09-17T21:22:43.338399+0000","last_unstale":"2024-09-17T21:22:49.379750+0000","last_undegraded":"2024-09-17T21:22:49.379750+0000","last_fullsized":"2024-09-17T21:22:49.379750+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:30:14.210043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050460430000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":15379,"num_read_kb":29065,"num_write":580,"num_write_kb":14319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":5960704,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"221'526","reported_seq":15415,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.378976+0000","last_change":"2024-09-17T21:22:46.356687+0000","last_active":"2024-09-17T21:22:49.378976+0000","last_peered":"2024-09-17T21:22:49.378976+0000","last_clean":"2024-09-17T21:22:49.378976+0000","last_became_active":"2024-09-17T21:22:46.349835+0000","last_became_peered":"2024-09-17T21:22:46.349835+0000","last_unstale":"2024-09-17T21:22:49.378976+0000","last_undegraded":"2024-09-17T21:22:49.378976+0000","last_fullsized":"2024-09-17T21:22:49.378976+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:51:30.571949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00482724,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15785,"num_read_kb":43220,"num_write":654,"num_write_kb":16401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":28397872,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"227'623","reported_seq":14440,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.806310+0000","last_change":"2024-09-17T21:22:49.806087+0000","last_active":"2024-09-17T21:22:49.806310+0000","last_peered":"2024-09-17T21:22:49.806310+0000","last_clean":"2024-09-17T21:22:49.806310+0000","last_became_active":"2024-09-17T21:22:49.394154+0000","last_became_peered":"2024-09-17T21:22:49.394154+0000","last_unstale":"2024-09-17T21:22:49.806310+0000","last_undegraded":"2024-09-17T21:22:49.806310+0000","last_fullsized":"2024-09-17T21:22:49.806310+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:24:57.185890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057317679999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":15141,"num_read_kb":70274,"num_write":535,"num_write_kb":35852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":91925553,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"226'464","reported_seq":7119,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.388236+0000","last_change":"2024-09-17T21:22:48.398780+0000","last_active":"2024-09-17T21:22:49.388236+0000","last_peered":"2024-09-17T21:22:49.388236+0000","last_clean":"2024-09-17T21:22:49.388236+0000","last_became_active":"2024-09-17T21:22:48.392669+0000","last_became_peered":"2024-09-17T21:22:48.392669+0000","last_unstale":"2024-09-17T21:22:49.388236+0000","last_undegraded":"2024-09-17T21:22:49.388236+0000","last_fullsized":"2024-09-17T21:22:49.388236+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:18:23.844174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0091147729999999996,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7427,"num_read_kb":28404,"num_write":434,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":4659071,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"222'271","reported_seq":7399,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.384802+0000","last_change":"2024-09-17T21:22:48.384974+0000","last_active":"2024-09-17T21:22:49.384802+0000","last_peered":"2024-09-17T21:22:49.384802+0000","last_clean":"2024-09-17T21:22:49.384802+0000","last_became_active":"2024-09-17T21:22:48.381106+0000","last_became_peered":"2024-09-17T21:22:48.381106+0000","last_unstale":"2024-09-17T21:22:49.384802+0000","last_undegraded":"2024-09-17T21:22:49.384802+0000","last_fullsized":"2024-09-17T21:22:49.384802+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:11:47.017985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017482940000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7827,"num_read_kb":19883,"num_write":219,"num_write_kb":3298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":6733181,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"226'404","reported_seq":5774,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.377779+0000","last_change":"2024-09-17T21:22:24.385309+0000","last_active":"2024-09-17T21:22:49.377779+0000","last_peered":"2024-09-17T21:22:49.377779+0000","last_clean":"2024-09-17T21:22:49.377779+0000","last_became_active":"2024-09-17T21:18:11.299946+0000","last_became_peered":"2024-09-17T21:18:11.299946+0000","last_unstale":"2024-09-17T21:22:49.377779+0000","last_undegraded":"2024-09-17T21:22:49.377779+0000","last_fullsized":"2024-09-17T21:22:49.377779+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:16:16.699933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015128413,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5168,"num_read_kb":22085,"num_write":382,"num_write_kb":4452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"225'570","reported_seq":15991,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.381896+0000","last_change":"2024-09-17T21:22:24.368740+0000","last_active":"2024-09-17T21:22:49.381896+0000","last_peered":"2024-09-17T21:22:49.381896+0000","last_clean":"2024-09-17T21:22:49.381896+0000","last_became_active":"2024-09-17T21:22:09.609844+0000","last_became_peered":"2024-09-17T21:22:09.609844+0000","last_unstale":"2024-09-17T21:22:49.381896+0000","last_undegraded":"2024-09-17T21:22:49.381896+0000","last_fullsized":"2024-09-17T21:22:49.381896+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:03:30.364818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018557070000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":17911,"num_read_kb":38759,"num_write":589,"num_write_kb":6968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":11589682,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"226'651","reported_seq":12601,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-17T21:22:49.377308+0000","last_change":"2024-09-17T21:22:41.357868+0000","last_active":"2024-09-17T21:22:49.377308+0000","last_peered":"2024-09-17T21:22:49.377308+0000","last_clean":"2024-09-17T21:22:49.377308+0000","last_became_active":"2024-09-17T21:22:41.354851+0000","last_became_peered":"2024-09-17T21:22:41.354851+0000","last_unstale":"2024-09-17T21:22:49.377308+0000","last_undegraded":"2024-09-17T21:22:49.377308+0000","last_fullsized":"2024-09-17T21:22:49.377308+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:47:04.462955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011937984,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12008,"num_read_kb":52136,"num_write":617,"num_write_kb":5443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":5557098,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":304,"num_object_clones":0,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":300,"num_read":353905,"num_read_kb":1145909,"num_write":19949,"num_write_kb":385384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11819,"num_bytes_recovered":614462318,"num_keys_recovered":1529,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8194,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":64,"acting":63,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574911,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60440,"kb_used_data":2884,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311400,"statfs":{"total":96636764160,"available":96574873600,"internally_reserved":0,"allocated":2953216,"data_stored":1252104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":58915268},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66584,"kb_used_data":3464,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305256,"statfs":{"total":96636764160,"available":96568582144,"internally_reserved":0,"allocated":3547136,"data_stored":1842472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":64616900},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83224,"kb_used_data":3460,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288616,"statfs":{"total":96636764160,"available":96551542784,"internally_reserved":0,"allocated":3543040,"data_stored":1842453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":81656262},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56288,"kb_used_data":2888,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315552,"statfs":{"total":96636764160,"available":96579125248,"internally_reserved":0,"allocated":2957312,"data_stored":1252123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56284,"kb_used_data":2884,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315556,"statfs":{"total":96636764160,"available":96579129344,"internally_reserved":0,"allocated":2953216,"data_stored":1252104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":231,"seq":992137445382,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30628,"kb_used_data":2892,"kb_used_omap":20,"kb_used_meta":27691,"kb_avail":94341212,"statfs":{"total":96636764160,"available":96605401088,"internally_reserved":0,"allocated":2961408,"data_stored":1257187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20966,"internal_metadata":28356122},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8194,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:22:58.030 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:22:58.346 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:22:58.346 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T21:22:58.364 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-17T21:22:57.614230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":308,"num_object_clones":0,"num_object_copies":616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":300,"num_read":353971,"num_read_kb":1145964,"num_write":20052,"num_write_kb":387014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11949,"num_bytes_recovered":615643130,"num_keys_recovered":1529,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18605,"ondisk_log_size":18605,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":353596,"kb_used_data":18600,"kb_used_omap":28,"kb_used_meta":334819,"kb_avail":565877444,"statfs":{"total":579820584960,"available":579458502656,"internally_reserved":0,"allocated":19046400,"data_stored":8754276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28944,"internal_metadata":342855408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-1,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-123,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002494"},"pg_stats":[{"pgid":"3.1f","version":"227'759","reported_seq":4118,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451328+0000","last_change":"2024-09-17T21:22:40.360129+0000","last_active":"2024-09-17T21:22:51.451328+0000","last_peered":"2024-09-17T21:22:51.451328+0000","last_clean":"2024-09-17T21:22:51.451328+0000","last_became_active":"2024-09-17T21:22:40.357329+0000","last_became_peered":"2024-09-17T21:22:40.357329+0000","last_unstale":"2024-09-17T21:22:51.451328+0000","last_undegraded":"2024-09-17T21:22:51.451328+0000","last_fullsized":"2024-09-17T21:22:51.451328+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:01:10.890769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010221002999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5921,"num_read_kb":16779,"num_write":1026,"num_write_kb":3495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":5143732,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"227'851","reported_seq":9557,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451353+0000","last_change":"2024-09-17T21:22:40.355216+0000","last_active":"2024-09-17T21:22:51.451353+0000","last_peered":"2024-09-17T21:22:51.451353+0000","last_clean":"2024-09-17T21:22:51.451353+0000","last_became_active":"2024-09-17T21:22:40.353016+0000","last_became_peered":"2024-09-17T21:22:40.353016+0000","last_unstale":"2024-09-17T21:22:51.451353+0000","last_undegraded":"2024-09-17T21:22:51.451353+0000","last_fullsized":"2024-09-17T21:22:51.451353+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":851,"log_dups_size":0,"ondisk_log_size":851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:11.494395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084510550000000007,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10998,"num_read_kb":53173,"num_write":1081,"num_write_kb":18708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":50670363,"num_keys_recovered":90,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"227'833","reported_seq":8688,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450570+0000","last_change":"2024-09-17T21:22:48.408191+0000","last_active":"2024-09-17T21:22:51.450570+0000","last_peered":"2024-09-17T21:22:51.450570+0000","last_clean":"2024-09-17T21:22:51.450570+0000","last_became_active":"2024-09-17T21:22:48.401326+0000","last_became_peered":"2024-09-17T21:22:48.401326+0000","last_unstale":"2024-09-17T21:22:51.450570+0000","last_undegraded":"2024-09-17T21:22:51.450570+0000","last_fullsized":"2024-09-17T21:22:51.450570+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:19:25.050335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082452470000000007,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":10230,"num_read_kb":27857,"num_write":978,"num_write_kb":10466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":12493301,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"227'376","reported_seq":4046,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450419+0000","last_change":"2024-09-17T21:22:48.406130+0000","last_active":"2024-09-17T21:22:51.450419+0000","last_peered":"2024-09-17T21:22:51.450419+0000","last_clean":"2024-09-17T21:22:51.450419+0000","last_became_active":"2024-09-17T21:22:48.401237+0000","last_became_peered":"2024-09-17T21:22:48.401237+0000","last_unstale":"2024-09-17T21:22:51.450419+0000","last_undegraded":"2024-09-17T21:22:51.450419+0000","last_fullsized":"2024-09-17T21:22:51.450419+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:39:30.285487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061565819999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":2726,"num_read_kb":23738,"num_write":383,"num_write_kb":5937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":9397808,"num_keys_recovered":202,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"227'710","reported_seq":8291,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451267+0000","last_change":"2024-09-17T21:22:24.369716+0000","last_active":"2024-09-17T21:22:51.451267+0000","last_peered":"2024-09-17T21:22:51.451267+0000","last_clean":"2024-09-17T21:22:51.451267+0000","last_became_active":"2024-09-17T21:21:41.329830+0000","last_became_peered":"2024-09-17T21:21:41.329830+0000","last_unstale":"2024-09-17T21:22:51.451267+0000","last_undegraded":"2024-09-17T21:22:51.451267+0000","last_fullsized":"2024-09-17T21:22:51.451267+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:42:50.772772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026038680000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":11634,"num_read_kb":21103,"num_write":555,"num_write_kb":4725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":7872032,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"226'739","reported_seq":13918,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.388284+0000","last_change":"2024-09-17T21:22:50.389766+0000","last_active":"2024-09-17T21:22:51.388284+0000","last_peered":"2024-09-17T21:22:51.388284+0000","last_clean":"2024-09-17T21:22:51.388284+0000","last_became_active":"2024-09-17T21:22:50.386623+0000","last_became_peered":"2024-09-17T21:22:50.386623+0000","last_unstale":"2024-09-17T21:22:51.388284+0000","last_undegraded":"2024-09-17T21:22:51.388284+0000","last_fullsized":"2024-09-17T21:22:51.388284+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:41:15.598618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056973060000000001,"stat_sum":{"num_bytes":19,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":16893,"num_read_kb":53539,"num_write":669,"num_write_kb":14565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":647737,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"226'476","reported_seq":12478,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.393411+0000","last_change":"2024-09-17T21:22:51.393063+0000","last_active":"2024-09-17T21:22:51.393411+0000","last_peered":"2024-09-17T21:22:51.393411+0000","last_clean":"2024-09-17T21:22:51.393411+0000","last_became_active":"2024-09-17T21:22:51.390578+0000","last_became_peered":"2024-09-17T21:22:51.390578+0000","last_unstale":"2024-09-17T21:22:51.393411+0000","last_undegraded":"2024-09-17T21:22:51.393411+0000","last_fullsized":"2024-09-17T21:22:51.393411+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:08:03.346817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0077005709999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":12648,"num_read_kb":27025,"num_write":374,"num_write_kb":5615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":3100207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"226'318","reported_seq":3035,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382486+0000","last_change":"2024-09-17T21:22:50.387935+0000","last_active":"2024-09-17T21:22:51.382486+0000","last_peered":"2024-09-17T21:22:51.382486+0000","last_clean":"2024-09-17T21:22:51.382486+0000","last_became_active":"2024-09-17T21:22:50.385975+0000","last_became_peered":"2024-09-17T21:22:50.385975+0000","last_unstale":"2024-09-17T21:22:51.382486+0000","last_undegraded":"2024-09-17T21:22:51.382486+0000","last_fullsized":"2024-09-17T21:22:51.382486+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:26:13.518654+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022643899999999998,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2392,"num_read_kb":9139,"num_write":304,"num_write_kb":2898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":9226431,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"225'393","reported_seq":7343,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451410+0000","last_change":"2024-09-17T21:22:43.342179+0000","last_active":"2024-09-17T21:22:51.451410+0000","last_peered":"2024-09-17T21:22:51.451410+0000","last_clean":"2024-09-17T21:22:51.451410+0000","last_became_active":"2024-09-17T21:22:43.340739+0000","last_became_peered":"2024-09-17T21:22:43.340739+0000","last_unstale":"2024-09-17T21:22:51.451410+0000","last_undegraded":"2024-09-17T21:22:51.451410+0000","last_fullsized":"2024-09-17T21:22:51.451410+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:32:48.751230+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00138835,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7054,"num_read_kb":21313,"num_write":475,"num_write_kb":6946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":719,"num_bytes_recovered":31066012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"224'602","reported_seq":22252,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450629+0000","last_change":"2024-09-17T21:22:48.400263+0000","last_active":"2024-09-17T21:22:51.450629+0000","last_peered":"2024-09-17T21:22:51.450629+0000","last_clean":"2024-09-17T21:22:51.450629+0000","last_became_active":"2024-09-17T21:22:48.394816+0000","last_became_peered":"2024-09-17T21:22:48.394816+0000","last_unstale":"2024-09-17T21:22:51.450629+0000","last_undegraded":"2024-09-17T21:22:51.450629+0000","last_fullsized":"2024-09-17T21:22:51.450629+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:05.075504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075408430000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":22417,"num_read_kb":27659,"num_write":858,"num_write_kb":13022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":3432016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"223'754","reported_seq":7158,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384191+0000","last_change":"2024-09-17T21:22:24.378500+0000","last_active":"2024-09-17T21:22:51.384191+0000","last_peered":"2024-09-17T21:22:51.384191+0000","last_clean":"2024-09-17T21:22:51.384191+0000","last_became_active":"2024-09-17T21:21:40.321648+0000","last_became_peered":"2024-09-17T21:21:40.321648+0000","last_unstale":"2024-09-17T21:22:51.384191+0000","last_undegraded":"2024-09-17T21:22:51.384191+0000","last_fullsized":"2024-09-17T21:22:51.384191+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:54:37.691837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0071138240000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8592,"num_read_kb":29445,"num_write":954,"num_write_kb":5554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":9885558,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"225'449","reported_seq":5376,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.383815+0000","last_change":"2024-09-17T21:22:49.393985+0000","last_active":"2024-09-17T21:22:51.383815+0000","last_peered":"2024-09-17T21:22:51.383815+0000","last_clean":"2024-09-17T21:22:51.383815+0000","last_became_active":"2024-09-17T21:22:49.389098+0000","last_became_peered":"2024-09-17T21:22:49.389098+0000","last_unstale":"2024-09-17T21:22:51.383815+0000","last_undegraded":"2024-09-17T21:22:51.383815+0000","last_fullsized":"2024-09-17T21:22:51.383815+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:28:53.422353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003029,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5351,"num_read_kb":26972,"num_write":428,"num_write_kb":16010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":385,"num_bytes_recovered":6943878,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":474,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384273+0000","last_change":"2024-09-17T21:18:03.059715+0000","last_active":"2024-09-17T21:22:51.384273+0000","last_peered":"2024-09-17T21:22:51.384273+0000","last_clean":"2024-09-17T21:22:51.384273+0000","last_became_active":"2024-09-17T21:18:01.213047+0000","last_became_peered":"2024-09-17T21:18:01.213047+0000","last_unstale":"2024-09-17T21:22:51.384273+0000","last_undegraded":"2024-09-17T21:22:51.384273+0000","last_fullsized":"2024-09-17T21:22:51.384273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:46:35.222373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"227'594","reported_seq":13694,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451462+0000","last_change":"2024-09-17T21:22:24.371187+0000","last_active":"2024-09-17T21:22:51.451462+0000","last_peered":"2024-09-17T21:22:51.451462+0000","last_clean":"2024-09-17T21:22:51.451462+0000","last_became_active":"2024-09-17T21:21:40.318488+0000","last_became_peered":"2024-09-17T21:21:40.318488+0000","last_unstale":"2024-09-17T21:22:51.451462+0000","last_undegraded":"2024-09-17T21:22:51.451462+0000","last_fullsized":"2024-09-17T21:22:51.451462+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:57:56.726468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036709440000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14882,"num_read_kb":61123,"num_write":581,"num_write_kb":23555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":49089084,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":474,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.385630+0000","last_change":"2024-09-17T21:18:03.073440+0000","last_active":"2024-09-17T21:22:51.385630+0000","last_peered":"2024-09-17T21:22:51.385630+0000","last_clean":"2024-09-17T21:22:51.385630+0000","last_became_active":"2024-09-17T21:18:01.048806+0000","last_became_peered":"2024-09-17T21:18:01.048806+0000","last_unstale":"2024-09-17T21:22:51.385630+0000","last_undegraded":"2024-09-17T21:22:51.385630+0000","last_fullsized":"2024-09-17T21:22:51.385630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:26.487163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094161400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"226'352","reported_seq":5610,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382685+0000","last_change":"2024-09-17T21:22:50.390979+0000","last_active":"2024-09-17T21:22:51.382685+0000","last_peered":"2024-09-17T21:22:51.382685+0000","last_clean":"2024-09-17T21:22:51.382685+0000","last_became_active":"2024-09-17T21:22:50.388965+0000","last_became_peered":"2024-09-17T21:22:50.388965+0000","last_unstale":"2024-09-17T21:22:51.382685+0000","last_undegraded":"2024-09-17T21:22:51.382685+0000","last_fullsized":"2024-09-17T21:22:51.382685+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:18:04.741617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079556969999999994,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4656,"num_read_kb":36280,"num_write":392,"num_write_kb":13876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":5244262,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":498,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384423+0000","last_change":"2024-09-17T21:21:35.319221+0000","last_active":"2024-09-17T21:22:51.384423+0000","last_peered":"2024-09-17T21:22:51.384423+0000","last_clean":"2024-09-17T21:22:51.384423+0000","last_became_active":"2024-09-17T21:21:35.312507+0000","last_became_peered":"2024-09-17T21:21:35.312507+0000","last_unstale":"2024-09-17T21:22:51.384423+0000","last_undegraded":"2024-09-17T21:22:51.384423+0000","last_fullsized":"2024-09-17T21:22:51.384423+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:53.042567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099219900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"226'707","reported_seq":16135,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.385480+0000","last_change":"2024-09-17T21:22:46.361219+0000","last_active":"2024-09-17T21:22:51.385480+0000","last_peered":"2024-09-17T21:22:51.385480+0000","last_clean":"2024-09-17T21:22:51.385480+0000","last_became_active":"2024-09-17T21:22:46.359525+0000","last_became_peered":"2024-09-17T21:22:46.359525+0000","last_unstale":"2024-09-17T21:22:51.385480+0000","last_undegraded":"2024-09-17T21:22:51.385480+0000","last_fullsized":"2024-09-17T21:22:51.385480+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:47.174816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052152359999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13338,"num_read_kb":29382,"num_write":606,"num_write_kb":10802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":367,"num_bytes_recovered":21385444,"num_keys_recovered":118,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":498,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.385587+0000","last_change":"2024-09-17T21:21:35.326814+0000","last_active":"2024-09-17T21:22:51.385587+0000","last_peered":"2024-09-17T21:22:51.385587+0000","last_clean":"2024-09-17T21:22:51.385587+0000","last_became_active":"2024-09-17T21:21:35.325777+0000","last_became_peered":"2024-09-17T21:21:35.325777+0000","last_unstale":"2024-09-17T21:22:51.385587+0000","last_undegraded":"2024-09-17T21:22:51.385587+0000","last_fullsized":"2024-09-17T21:22:51.385587+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:47:00.558900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006222,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"225'1056","reported_seq":33580,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450393+0000","last_change":"2024-09-17T21:22:44.360752+0000","last_active":"2024-09-17T21:22:51.450393+0000","last_peered":"2024-09-17T21:22:51.450393+0000","last_clean":"2024-09-17T21:22:51.450393+0000","last_became_active":"2024-09-17T21:22:44.352239+0000","last_became_peered":"2024-09-17T21:22:44.352239+0000","last_unstale":"2024-09-17T21:22:51.450393+0000","last_undegraded":"2024-09-17T21:22:51.450393+0000","last_fullsized":"2024-09-17T21:22:51.450393+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":1056,"log_dups_size":0,"ondisk_log_size":1056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:36:58.893607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052118770000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":37177,"num_read_kb":61903,"num_write":871,"num_write_kb":21852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":754,"num_bytes_recovered":58900004,"num_keys_recovered":204,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"227'536","reported_seq":3887,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.388246+0000","last_change":"2024-09-17T21:22:24.383832+0000","last_active":"2024-09-17T21:22:51.388246+0000","last_peered":"2024-09-17T21:22:51.388246+0000","last_clean":"2024-09-17T21:22:51.388246+0000","last_became_active":"2024-09-17T21:21:39.315142+0000","last_became_peered":"2024-09-17T21:21:39.315142+0000","last_unstale":"2024-09-17T21:22:51.388246+0000","last_undegraded":"2024-09-17T21:22:51.388246+0000","last_fullsized":"2024-09-17T21:22:51.388246+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:05:28.580363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061334989999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":3053,"num_read_kb":14423,"num_write":502,"num_write_kb":4818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":7199060,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":412,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450196+0000","last_change":"2024-09-17T21:22:48.383726+0000","last_active":"2024-09-17T21:22:51.450196+0000","last_peered":"2024-09-17T21:22:51.450196+0000","last_clean":"2024-09-17T21:22:51.450196+0000","last_became_active":"2024-09-17T21:22:48.382842+0000","last_became_peered":"2024-09-17T21:22:48.382842+0000","last_unstale":"2024-09-17T21:22:51.450196+0000","last_undegraded":"2024-09-17T21:22:51.450196+0000","last_fullsized":"2024-09-17T21:22:51.450196+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:30:55.312809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"226'248","reported_seq":9446,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384708+0000","last_change":"2024-09-17T21:22:45.354353+0000","last_active":"2024-09-17T21:22:51.384708+0000","last_peered":"2024-09-17T21:22:51.384708+0000","last_clean":"2024-09-17T21:22:51.384708+0000","last_became_active":"2024-09-17T21:22:45.352682+0000","last_became_peered":"2024-09-17T21:22:45.352682+0000","last_unstale":"2024-09-17T21:22:51.384708+0000","last_undegraded":"2024-09-17T21:22:51.384708+0000","last_fullsized":"2024-09-17T21:22:51.384708+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:18.942784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048483459999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5340,"num_read_kb":22605,"num_write":215,"num_write_kb":6424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":15742651,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":461,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382421+0000","last_change":"2024-09-17T21:22:50.389225+0000","last_active":"2024-09-17T21:22:51.382421+0000","last_peered":"2024-09-17T21:22:51.382421+0000","last_clean":"2024-09-17T21:22:51.382421+0000","last_became_active":"2024-09-17T21:22:50.388953+0000","last_became_peered":"2024-09-17T21:22:50.388953+0000","last_unstale":"2024-09-17T21:22:51.382421+0000","last_undegraded":"2024-09-17T21:22:51.382421+0000","last_fullsized":"2024-09-17T21:22:51.382421+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:21.062518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"227'1041","reported_seq":8867,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450992+0000","last_change":"2024-09-17T21:22:50.391990+0000","last_active":"2024-09-17T21:22:51.450992+0000","last_peered":"2024-09-17T21:22:51.450992+0000","last_clean":"2024-09-17T21:22:51.450992+0000","last_became_active":"2024-09-17T21:22:50.390218+0000","last_became_peered":"2024-09-17T21:22:50.390218+0000","last_unstale":"2024-09-17T21:22:51.450992+0000","last_undegraded":"2024-09-17T21:22:51.450992+0000","last_fullsized":"2024-09-17T21:22:51.450992+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":1041,"log_dups_size":0,"ondisk_log_size":1041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:24:31.427407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035942119999999998,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":9666,"num_read_kb":63102,"num_write":1578,"num_write_kb":18132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":894,"num_bytes_recovered":48903910,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":474,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.383504+0000","last_change":"2024-09-17T21:18:03.059156+0000","last_active":"2024-09-17T21:22:51.383504+0000","last_peered":"2024-09-17T21:22:51.383504+0000","last_clean":"2024-09-17T21:22:51.383504+0000","last_became_active":"2024-09-17T21:18:01.049180+0000","last_became_peered":"2024-09-17T21:18:01.049180+0000","last_unstale":"2024-09-17T21:22:51.383504+0000","last_undegraded":"2024-09-17T21:22:51.383504+0000","last_fullsized":"2024-09-17T21:22:51.383504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:27.069794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077408,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"226'656","reported_seq":14594,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384939+0000","last_change":"2024-09-17T21:22:41.354650+0000","last_active":"2024-09-17T21:22:51.384939+0000","last_peered":"2024-09-17T21:22:51.384939+0000","last_clean":"2024-09-17T21:22:51.384939+0000","last_became_active":"2024-09-17T21:22:41.352429+0000","last_became_peered":"2024-09-17T21:22:41.352429+0000","last_unstale":"2024-09-17T21:22:51.384939+0000","last_undegraded":"2024-09-17T21:22:51.384939+0000","last_fullsized":"2024-09-17T21:22:51.384939+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:30:46.965276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067184260000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14741,"num_read_kb":38983,"num_write":745,"num_write_kb":20177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":27258987,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":19,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450266+0000","last_change":"2024-09-17T21:22:48.387337+0000","last_active":"2024-09-17T21:22:51.450266+0000","last_peered":"2024-09-17T21:22:51.450266+0000","last_clean":"2024-09-17T21:22:51.450266+0000","last_became_active":"2024-09-17T21:22:48.386572+0000","last_became_peered":"2024-09-17T21:22:48.386572+0000","last_unstale":"2024-09-17T21:22:51.450266+0000","last_undegraded":"2024-09-17T21:22:51.450266+0000","last_fullsized":"2024-09-17T21:22:51.450266+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:55:57.934354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":189,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.387986+0000","last_change":"2024-09-17T21:21:17.347819+0000","last_active":"2024-09-17T21:22:51.387986+0000","last_peered":"2024-09-17T21:22:51.387986+0000","last_clean":"2024-09-17T21:22:51.387986+0000","last_became_active":"2024-09-17T21:21:17.337621+0000","last_became_peered":"2024-09-17T21:21:17.337621+0000","last_unstale":"2024-09-17T21:22:51.387986+0000","last_undegraded":"2024-09-17T21:22:51.387986+0000","last_fullsized":"2024-09-17T21:22:51.387986+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_clean_scrub_stamp":"2024-09-17T21:17:55.982339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:24:32.371870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"224'616","reported_seq":14093,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382313+0000","last_change":"2024-09-17T21:22:50.383917+0000","last_active":"2024-09-17T21:22:51.382313+0000","last_peered":"2024-09-17T21:22:51.382313+0000","last_clean":"2024-09-17T21:22:51.382313+0000","last_became_active":"2024-09-17T21:22:50.382396+0000","last_became_peered":"2024-09-17T21:22:50.382396+0000","last_unstale":"2024-09-17T21:22:51.382313+0000","last_undegraded":"2024-09-17T21:22:51.382313+0000","last_fullsized":"2024-09-17T21:22:51.382313+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:13:13.848853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001602121,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":13993,"num_read_kb":68437,"num_write":586,"num_write_kb":29114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":40776135,"num_keys_recovered":165,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"227'699","reported_seq":11106,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.388697+0000","last_change":"2024-09-17T21:22:49.395344+0000","last_active":"2024-09-17T21:22:51.388697+0000","last_peered":"2024-09-17T21:22:51.388697+0000","last_clean":"2024-09-17T21:22:51.388697+0000","last_became_active":"2024-09-17T21:22:49.390523+0000","last_became_peered":"2024-09-17T21:22:49.390523+0000","last_unstale":"2024-09-17T21:22:51.388697+0000","last_undegraded":"2024-09-17T21:22:51.388697+0000","last_fullsized":"2024-09-17T21:22:51.388697+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:55:28.950314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029497360000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":11677,"num_read_kb":43614,"num_write":1089,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":728,"num_bytes_recovered":30193805,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"227'428","reported_seq":8266,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.383725+0000","last_change":"2024-09-17T21:22:49.397589+0000","last_active":"2024-09-17T21:22:51.383725+0000","last_peered":"2024-09-17T21:22:51.383725+0000","last_clean":"2024-09-17T21:22:51.383725+0000","last_became_active":"2024-09-17T21:22:49.394384+0000","last_became_peered":"2024-09-17T21:22:49.394384+0000","last_unstale":"2024-09-17T21:22:51.383725+0000","last_undegraded":"2024-09-17T21:22:51.383725+0000","last_fullsized":"2024-09-17T21:22:51.383725+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:24:21.885168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045615830000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7186,"num_read_kb":40870,"num_write":369,"num_write_kb":16675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":179794,"num_keys_recovered":2,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"226'376","reported_seq":4920,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384873+0000","last_change":"2024-09-17T21:22:24.375850+0000","last_active":"2024-09-17T21:22:51.384873+0000","last_peered":"2024-09-17T21:22:51.384873+0000","last_clean":"2024-09-17T21:22:51.384873+0000","last_became_active":"2024-09-17T21:21:40.319613+0000","last_became_peered":"2024-09-17T21:21:40.319613+0000","last_unstale":"2024-09-17T21:22:51.384873+0000","last_undegraded":"2024-09-17T21:22:51.384873+0000","last_fullsized":"2024-09-17T21:22:51.384873+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:26:53.101370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038796239999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4694,"num_read_kb":23619,"num_write":320,"num_write_kb":13765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":4886965,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"227'452","reported_seq":13421,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.385236+0000","last_change":"2024-09-17T21:22:43.339607+0000","last_active":"2024-09-17T21:22:51.385236+0000","last_peered":"2024-09-17T21:22:51.385236+0000","last_clean":"2024-09-17T21:22:51.385236+0000","last_became_active":"2024-09-17T21:22:43.338399+0000","last_became_peered":"2024-09-17T21:22:43.338399+0000","last_unstale":"2024-09-17T21:22:51.385236+0000","last_undegraded":"2024-09-17T21:22:51.385236+0000","last_fullsized":"2024-09-17T21:22:51.385236+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:30:14.210043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050460430000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":15379,"num_read_kb":29065,"num_write":580,"num_write_kb":14319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":5960704,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"221'526","reported_seq":15419,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384216+0000","last_change":"2024-09-17T21:22:46.356687+0000","last_active":"2024-09-17T21:22:51.384216+0000","last_peered":"2024-09-17T21:22:51.384216+0000","last_clean":"2024-09-17T21:22:51.384216+0000","last_became_active":"2024-09-17T21:22:46.349835+0000","last_became_peered":"2024-09-17T21:22:46.349835+0000","last_unstale":"2024-09-17T21:22:51.384216+0000","last_undegraded":"2024-09-17T21:22:51.384216+0000","last_fullsized":"2024-09-17T21:22:51.384216+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:51:30.571949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00482724,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15785,"num_read_kb":43220,"num_write":654,"num_write_kb":16401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":28397872,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"227'623","reported_seq":14444,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450685+0000","last_change":"2024-09-17T21:22:49.806087+0000","last_active":"2024-09-17T21:22:51.450685+0000","last_peered":"2024-09-17T21:22:51.450685+0000","last_clean":"2024-09-17T21:22:51.450685+0000","last_became_active":"2024-09-17T21:22:49.394154+0000","last_became_peered":"2024-09-17T21:22:49.394154+0000","last_unstale":"2024-09-17T21:22:51.450685+0000","last_undegraded":"2024-09-17T21:22:51.450685+0000","last_fullsized":"2024-09-17T21:22:51.450685+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:24:57.185890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057317679999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":15141,"num_read_kb":70274,"num_write":535,"num_write_kb":35852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":91925553,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"226'464","reported_seq":7123,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450740+0000","last_change":"2024-09-17T21:22:48.398780+0000","last_active":"2024-09-17T21:22:51.450740+0000","last_peered":"2024-09-17T21:22:51.450740+0000","last_clean":"2024-09-17T21:22:51.450740+0000","last_became_active":"2024-09-17T21:22:48.392669+0000","last_became_peered":"2024-09-17T21:22:48.392669+0000","last_unstale":"2024-09-17T21:22:51.450740+0000","last_undegraded":"2024-09-17T21:22:51.450740+0000","last_fullsized":"2024-09-17T21:22:51.450740+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:18:23.844174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0091147729999999996,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7427,"num_read_kb":28404,"num_write":434,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":4659071,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"222'271","reported_seq":7403,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450132+0000","last_change":"2024-09-17T21:22:48.384974+0000","last_active":"2024-09-17T21:22:51.450132+0000","last_peered":"2024-09-17T21:22:51.450132+0000","last_clean":"2024-09-17T21:22:51.450132+0000","last_became_active":"2024-09-17T21:22:48.381106+0000","last_became_peered":"2024-09-17T21:22:48.381106+0000","last_unstale":"2024-09-17T21:22:51.450132+0000","last_undegraded":"2024-09-17T21:22:51.450132+0000","last_fullsized":"2024-09-17T21:22:51.450132+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:11:47.017985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017482940000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7827,"num_read_kb":19883,"num_write":219,"num_write_kb":3298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":6733181,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"226'404","reported_seq":5779,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384016+0000","last_change":"2024-09-17T21:22:24.385309+0000","last_active":"2024-09-17T21:22:51.384016+0000","last_peered":"2024-09-17T21:22:51.384016+0000","last_clean":"2024-09-17T21:22:51.384016+0000","last_became_active":"2024-09-17T21:18:11.299946+0000","last_became_peered":"2024-09-17T21:18:11.299946+0000","last_unstale":"2024-09-17T21:22:51.384016+0000","last_undegraded":"2024-09-17T21:22:51.384016+0000","last_fullsized":"2024-09-17T21:22:51.384016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:16:16.699933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015128413,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5168,"num_read_kb":22085,"num_write":382,"num_write_kb":4452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"225'570","reported_seq":15995,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450984+0000","last_change":"2024-09-17T21:22:24.368740+0000","last_active":"2024-09-17T21:22:51.450984+0000","last_peered":"2024-09-17T21:22:51.450984+0000","last_clean":"2024-09-17T21:22:51.450984+0000","last_became_active":"2024-09-17T21:22:09.609844+0000","last_became_peered":"2024-09-17T21:22:09.609844+0000","last_unstale":"2024-09-17T21:22:51.450984+0000","last_undegraded":"2024-09-17T21:22:51.450984+0000","last_fullsized":"2024-09-17T21:22:51.450984+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:03:30.364818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018557070000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":17911,"num_read_kb":38759,"num_write":589,"num_write_kb":6968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":11589682,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"226'651","reported_seq":12606,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384243+0000","last_change":"2024-09-17T21:22:41.357868+0000","last_active":"2024-09-17T21:22:51.384243+0000","last_peered":"2024-09-17T21:22:51.384243+0000","last_clean":"2024-09-17T21:22:51.384243+0000","last_became_active":"2024-09-17T21:22:41.354851+0000","last_became_peered":"2024-09-17T21:22:41.354851+0000","last_unstale":"2024-09-17T21:22:51.384243+0000","last_undegraded":"2024-09-17T21:22:51.384243+0000","last_fullsized":"2024-09-17T21:22:51.384243+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:47:04.462955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011937984,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12008,"num_read_kb":52136,"num_write":617,"num_write_kb":5443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":5557098,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":304,"num_object_clones":0,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":300,"num_read":353905,"num_read_kb":1145909,"num_write":19949,"num_write_kb":385384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11936,"num_bytes_recovered":614462337,"num_keys_recovered":1529,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8194,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60472,"kb_used_data":2908,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311368,"statfs":{"total":96636764160,"available":96574840832,"internally_reserved":0,"allocated":2977792,"data_stored":1262263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":58915268},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66616,"kb_used_data":3488,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305224,"statfs":{"total":96636764160,"available":96568549376,"internally_reserved":0,"allocated":3571712,"data_stored":1852631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":64616900},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83256,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288584,"statfs":{"total":96636764160,"available":96551510016,"internally_reserved":0,"allocated":3567616,"data_stored":1852612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":81656262},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56308,"kb_used_data":2912,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315532,"statfs":{"total":96636764160,"available":96579104768,"internally_reserved":0,"allocated":2981888,"data_stored":1262282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56300,"kb_used_data":2904,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315540,"statfs":{"total":96636764160,"available":96579112960,"internally_reserved":0,"allocated":2973696,"data_stored":1262244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":231,"seq":992137445383,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30644,"kb_used_data":2904,"kb_used_omap":20,"kb_used_meta":27691,"kb_avail":94341196,"statfs":{"total":96636764160,"available":96605384704,"internally_reserved":0,"allocated":2973696,"data_stored":1262244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20966,"internal_metadata":28356122},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8194,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:22:58.366 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-17T21:22:58.366 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-17T21:22:58.699 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-17T21:22:58.708 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T21:22:58.708 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:22:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:22:59.019 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T21:22:59.037 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-17T21:22:57.614230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":308,"num_object_clones":0,"num_object_copies":616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":300,"num_read":353971,"num_read_kb":1145964,"num_write":20052,"num_write_kb":387014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11949,"num_bytes_recovered":615643130,"num_keys_recovered":1529,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18605,"ondisk_log_size":18605,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":353596,"kb_used_data":18600,"kb_used_omap":28,"kb_used_meta":334819,"kb_avail":565877444,"statfs":{"total":579820584960,"available":579458502656,"internally_reserved":0,"allocated":19046400,"data_stored":8754276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28944,"internal_metadata":342855408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-1,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-123,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002494"},"pg_stats":[{"pgid":"3.1f","version":"227'759","reported_seq":4118,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451328+0000","last_change":"2024-09-17T21:22:40.360129+0000","last_active":"2024-09-17T21:22:51.451328+0000","last_peered":"2024-09-17T21:22:51.451328+0000","last_clean":"2024-09-17T21:22:51.451328+0000","last_became_active":"2024-09-17T21:22:40.357329+0000","last_became_peered":"2024-09-17T21:22:40.357329+0000","last_unstale":"2024-09-17T21:22:51.451328+0000","last_undegraded":"2024-09-17T21:22:51.451328+0000","last_fullsized":"2024-09-17T21:22:51.451328+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:01:10.890769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010221002999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5921,"num_read_kb":16779,"num_write":1026,"num_write_kb":3495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":5143732,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"227'851","reported_seq":9557,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451353+0000","last_change":"2024-09-17T21:22:40.355216+0000","last_active":"2024-09-17T21:22:51.451353+0000","last_peered":"2024-09-17T21:22:51.451353+0000","last_clean":"2024-09-17T21:22:51.451353+0000","last_became_active":"2024-09-17T21:22:40.353016+0000","last_became_peered":"2024-09-17T21:22:40.353016+0000","last_unstale":"2024-09-17T21:22:51.451353+0000","last_undegraded":"2024-09-17T21:22:51.451353+0000","last_fullsized":"2024-09-17T21:22:51.451353+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":851,"log_dups_size":0,"ondisk_log_size":851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:11.494395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084510550000000007,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10998,"num_read_kb":53173,"num_write":1081,"num_write_kb":18708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":50670363,"num_keys_recovered":90,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"227'833","reported_seq":8688,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450570+0000","last_change":"2024-09-17T21:22:48.408191+0000","last_active":"2024-09-17T21:22:51.450570+0000","last_peered":"2024-09-17T21:22:51.450570+0000","last_clean":"2024-09-17T21:22:51.450570+0000","last_became_active":"2024-09-17T21:22:48.401326+0000","last_became_peered":"2024-09-17T21:22:48.401326+0000","last_unstale":"2024-09-17T21:22:51.450570+0000","last_undegraded":"2024-09-17T21:22:51.450570+0000","last_fullsized":"2024-09-17T21:22:51.450570+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:19:25.050335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082452470000000007,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":10230,"num_read_kb":27857,"num_write":978,"num_write_kb":10466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":12493301,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"227'376","reported_seq":4046,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450419+0000","last_change":"2024-09-17T21:22:48.406130+0000","last_active":"2024-09-17T21:22:51.450419+0000","last_peered":"2024-09-17T21:22:51.450419+0000","last_clean":"2024-09-17T21:22:51.450419+0000","last_became_active":"2024-09-17T21:22:48.401237+0000","last_became_peered":"2024-09-17T21:22:48.401237+0000","last_unstale":"2024-09-17T21:22:51.450419+0000","last_undegraded":"2024-09-17T21:22:51.450419+0000","last_fullsized":"2024-09-17T21:22:51.450419+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:39:30.285487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061565819999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":2726,"num_read_kb":23738,"num_write":383,"num_write_kb":5937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":9397808,"num_keys_recovered":202,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"227'710","reported_seq":8291,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451267+0000","last_change":"2024-09-17T21:22:24.369716+0000","last_active":"2024-09-17T21:22:51.451267+0000","last_peered":"2024-09-17T21:22:51.451267+0000","last_clean":"2024-09-17T21:22:51.451267+0000","last_became_active":"2024-09-17T21:21:41.329830+0000","last_became_peered":"2024-09-17T21:21:41.329830+0000","last_unstale":"2024-09-17T21:22:51.451267+0000","last_undegraded":"2024-09-17T21:22:51.451267+0000","last_fullsized":"2024-09-17T21:22:51.451267+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:42:50.772772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026038680000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":11634,"num_read_kb":21103,"num_write":555,"num_write_kb":4725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":7872032,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"226'739","reported_seq":13918,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.388284+0000","last_change":"2024-09-17T21:22:50.389766+0000","last_active":"2024-09-17T21:22:51.388284+0000","last_peered":"2024-09-17T21:22:51.388284+0000","last_clean":"2024-09-17T21:22:51.388284+0000","last_became_active":"2024-09-17T21:22:50.386623+0000","last_became_peered":"2024-09-17T21:22:50.386623+0000","last_unstale":"2024-09-17T21:22:51.388284+0000","last_undegraded":"2024-09-17T21:22:51.388284+0000","last_fullsized":"2024-09-17T21:22:51.388284+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:41:15.598618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056973060000000001,"stat_sum":{"num_bytes":19,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":16893,"num_read_kb":53539,"num_write":669,"num_write_kb":14565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":647737,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"226'476","reported_seq":12478,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.393411+0000","last_change":"2024-09-17T21:22:51.393063+0000","last_active":"2024-09-17T21:22:51.393411+0000","last_peered":"2024-09-17T21:22:51.393411+0000","last_clean":"2024-09-17T21:22:51.393411+0000","last_became_active":"2024-09-17T21:22:51.390578+0000","last_became_peered":"2024-09-17T21:22:51.390578+0000","last_unstale":"2024-09-17T21:22:51.393411+0000","last_undegraded":"2024-09-17T21:22:51.393411+0000","last_fullsized":"2024-09-17T21:22:51.393411+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:08:03.346817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0077005709999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":12648,"num_read_kb":27025,"num_write":374,"num_write_kb":5615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":3100207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"226'318","reported_seq":3035,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382486+0000","last_change":"2024-09-17T21:22:50.387935+0000","last_active":"2024-09-17T21:22:51.382486+0000","last_peered":"2024-09-17T21:22:51.382486+0000","last_clean":"2024-09-17T21:22:51.382486+0000","last_became_active":"2024-09-17T21:22:50.385975+0000","last_became_peered":"2024-09-17T21:22:50.385975+0000","last_unstale":"2024-09-17T21:22:51.382486+0000","last_undegraded":"2024-09-17T21:22:51.382486+0000","last_fullsized":"2024-09-17T21:22:51.382486+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:26:13.518654+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022643899999999998,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2392,"num_read_kb":9139,"num_write":304,"num_write_kb":2898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":9226431,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"225'393","reported_seq":7343,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451410+0000","last_change":"2024-09-17T21:22:43.342179+0000","last_active":"2024-09-17T21:22:51.451410+0000","last_peered":"2024-09-17T21:22:51.451410+0000","last_clean":"2024-09-17T21:22:51.451410+0000","last_became_active":"2024-09-17T21:22:43.340739+0000","last_became_peered":"2024-09-17T21:22:43.340739+0000","last_unstale":"2024-09-17T21:22:51.451410+0000","last_undegraded":"2024-09-17T21:22:51.451410+0000","last_fullsized":"2024-09-17T21:22:51.451410+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:32:48.751230+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00138835,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7054,"num_read_kb":21313,"num_write":475,"num_write_kb":6946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":719,"num_bytes_recovered":31066012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"224'602","reported_seq":22252,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450629+0000","last_change":"2024-09-17T21:22:48.400263+0000","last_active":"2024-09-17T21:22:51.450629+0000","last_peered":"2024-09-17T21:22:51.450629+0000","last_clean":"2024-09-17T21:22:51.450629+0000","last_became_active":"2024-09-17T21:22:48.394816+0000","last_became_peered":"2024-09-17T21:22:48.394816+0000","last_unstale":"2024-09-17T21:22:51.450629+0000","last_undegraded":"2024-09-17T21:22:51.450629+0000","last_fullsized":"2024-09-17T21:22:51.450629+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:05.075504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075408430000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":22417,"num_read_kb":27659,"num_write":858,"num_write_kb":13022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":3432016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"223'754","reported_seq":7158,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384191+0000","last_change":"2024-09-17T21:22:24.378500+0000","last_active":"2024-09-17T21:22:51.384191+0000","last_peered":"2024-09-17T21:22:51.384191+0000","last_clean":"2024-09-17T21:22:51.384191+0000","last_became_active":"2024-09-17T21:21:40.321648+0000","last_became_peered":"2024-09-17T21:21:40.321648+0000","last_unstale":"2024-09-17T21:22:51.384191+0000","last_undegraded":"2024-09-17T21:22:51.384191+0000","last_fullsized":"2024-09-17T21:22:51.384191+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:54:37.691837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0071138240000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8592,"num_read_kb":29445,"num_write":954,"num_write_kb":5554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":9885558,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"225'449","reported_seq":5376,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.383815+0000","last_change":"2024-09-17T21:22:49.393985+0000","last_active":"2024-09-17T21:22:51.383815+0000","last_peered":"2024-09-17T21:22:51.383815+0000","last_clean":"2024-09-17T21:22:51.383815+0000","last_became_active":"2024-09-17T21:22:49.389098+0000","last_became_peered":"2024-09-17T21:22:49.389098+0000","last_unstale":"2024-09-17T21:22:51.383815+0000","last_undegraded":"2024-09-17T21:22:51.383815+0000","last_fullsized":"2024-09-17T21:22:51.383815+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:28:53.422353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003029,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5351,"num_read_kb":26972,"num_write":428,"num_write_kb":16010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":385,"num_bytes_recovered":6943878,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":474,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384273+0000","last_change":"2024-09-17T21:18:03.059715+0000","last_active":"2024-09-17T21:22:51.384273+0000","last_peered":"2024-09-17T21:22:51.384273+0000","last_clean":"2024-09-17T21:22:51.384273+0000","last_became_active":"2024-09-17T21:18:01.213047+0000","last_became_peered":"2024-09-17T21:18:01.213047+0000","last_unstale":"2024-09-17T21:22:51.384273+0000","last_undegraded":"2024-09-17T21:22:51.384273+0000","last_fullsized":"2024-09-17T21:22:51.384273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:46:35.222373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"227'594","reported_seq":13694,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451462+0000","last_change":"2024-09-17T21:22:24.371187+0000","last_active":"2024-09-17T21:22:51.451462+0000","last_peered":"2024-09-17T21:22:51.451462+0000","last_clean":"2024-09-17T21:22:51.451462+0000","last_became_active":"2024-09-17T21:21:40.318488+0000","last_became_peered":"2024-09-17T21:21:40.318488+0000","last_unstale":"2024-09-17T21:22:51.451462+0000","last_undegraded":"2024-09-17T21:22:51.451462+0000","last_fullsized":"2024-09-17T21:22:51.451462+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:57:56.726468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036709440000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14882,"num_read_kb":61123,"num_write":581,"num_write_kb":23555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":49089084,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":474,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.385630+0000","last_change":"2024-09-17T21:18:03.073440+0000","last_active":"2024-09-17T21:22:51.385630+0000","last_peered":"2024-09-17T21:22:51.385630+0000","last_clean":"2024-09-17T21:22:51.385630+0000","last_became_active":"2024-09-17T21:18:01.048806+0000","last_became_peered":"2024-09-17T21:18:01.048806+0000","last_unstale":"2024-09-17T21:22:51.385630+0000","last_undegraded":"2024-09-17T21:22:51.385630+0000","last_fullsized":"2024-09-17T21:22:51.385630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:26.487163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094161400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"226'352","reported_seq":5610,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382685+0000","last_change":"2024-09-17T21:22:50.390979+0000","last_active":"2024-09-17T21:22:51.382685+0000","last_peered":"2024-09-17T21:22:51.382685+0000","last_clean":"2024-09-17T21:22:51.382685+0000","last_became_active":"2024-09-17T21:22:50.388965+0000","last_became_peered":"2024-09-17T21:22:50.388965+0000","last_unstale":"2024-09-17T21:22:51.382685+0000","last_undegraded":"2024-09-17T21:22:51.382685+0000","last_fullsized":"2024-09-17T21:22:51.382685+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:18:04.741617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079556969999999994,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4656,"num_read_kb":36280,"num_write":392,"num_write_kb":13876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":5244262,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":498,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384423+0000","last_change":"2024-09-17T21:21:35.319221+0000","last_active":"2024-09-17T21:22:51.384423+0000","last_peered":"2024-09-17T21:22:51.384423+0000","last_clean":"2024-09-17T21:22:51.384423+0000","last_became_active":"2024-09-17T21:21:35.312507+0000","last_became_peered":"2024-09-17T21:21:35.312507+0000","last_unstale":"2024-09-17T21:22:51.384423+0000","last_undegraded":"2024-09-17T21:22:51.384423+0000","last_fullsized":"2024-09-17T21:22:51.384423+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:53.042567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099219900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"226'707","reported_seq":16135,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.385480+0000","last_change":"2024-09-17T21:22:46.361219+0000","last_active":"2024-09-17T21:22:51.385480+0000","last_peered":"2024-09-17T21:22:51.385480+0000","last_clean":"2024-09-17T21:22:51.385480+0000","last_became_active":"2024-09-17T21:22:46.359525+0000","last_became_peered":"2024-09-17T21:22:46.359525+0000","last_unstale":"2024-09-17T21:22:51.385480+0000","last_undegraded":"2024-09-17T21:22:51.385480+0000","last_fullsized":"2024-09-17T21:22:51.385480+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:47.174816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052152359999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13338,"num_read_kb":29382,"num_write":606,"num_write_kb":10802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":367,"num_bytes_recovered":21385444,"num_keys_recovered":118,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":498,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.385587+0000","last_change":"2024-09-17T21:21:35.326814+0000","last_active":"2024-09-17T21:22:51.385587+0000","last_peered":"2024-09-17T21:22:51.385587+0000","last_clean":"2024-09-17T21:22:51.385587+0000","last_became_active":"2024-09-17T21:21:35.325777+0000","last_became_peered":"2024-09-17T21:21:35.325777+0000","last_unstale":"2024-09-17T21:22:51.385587+0000","last_undegraded":"2024-09-17T21:22:51.385587+0000","last_fullsized":"2024-09-17T21:22:51.385587+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:47:00.558900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006222,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"225'1056","reported_seq":33580,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450393+0000","last_change":"2024-09-17T21:22:44.360752+0000","last_active":"2024-09-17T21:22:51.450393+0000","last_peered":"2024-09-17T21:22:51.450393+0000","last_clean":"2024-09-17T21:22:51.450393+0000","last_became_active":"2024-09-17T21:22:44.352239+0000","last_became_peered":"2024-09-17T21:22:44.352239+0000","last_unstale":"2024-09-17T21:22:51.450393+0000","last_undegraded":"2024-09-17T21:22:51.450393+0000","last_fullsized":"2024-09-17T21:22:51.450393+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":1056,"log_dups_size":0,"ondisk_log_size":1056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:36:58.893607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052118770000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":37177,"num_read_kb":61903,"num_write":871,"num_write_kb":21852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":754,"num_bytes_recovered":58900004,"num_keys_recovered":204,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"227'536","reported_seq":3887,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.388246+0000","last_change":"2024-09-17T21:22:24.383832+0000","last_active":"2024-09-17T21:22:51.388246+0000","last_peered":"2024-09-17T21:22:51.388246+0000","last_clean":"2024-09-17T21:22:51.388246+0000","last_became_active":"2024-09-17T21:21:39.315142+0000","last_became_peered":"2024-09-17T21:21:39.315142+0000","last_unstale":"2024-09-17T21:22:51.388246+0000","last_undegraded":"2024-09-17T21:22:51.388246+0000","last_fullsized":"2024-09-17T21:22:51.388246+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:05:28.580363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061334989999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":3053,"num_read_kb":14423,"num_write":502,"num_write_kb":4818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":7199060,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":412,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450196+0000","last_change":"2024-09-17T21:22:48.383726+0000","last_active":"2024-09-17T21:22:51.450196+0000","last_peered":"2024-09-17T21:22:51.450196+0000","last_clean":"2024-09-17T21:22:51.450196+0000","last_became_active":"2024-09-17T21:22:48.382842+0000","last_became_peered":"2024-09-17T21:22:48.382842+0000","last_unstale":"2024-09-17T21:22:51.450196+0000","last_undegraded":"2024-09-17T21:22:51.450196+0000","last_fullsized":"2024-09-17T21:22:51.450196+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:30:55.312809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"226'248","reported_seq":9446,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384708+0000","last_change":"2024-09-17T21:22:45.354353+0000","last_active":"2024-09-17T21:22:51.384708+0000","last_peered":"2024-09-17T21:22:51.384708+0000","last_clean":"2024-09-17T21:22:51.384708+0000","last_became_active":"2024-09-17T21:22:45.352682+0000","last_became_peered":"2024-09-17T21:22:45.352682+0000","last_unstale":"2024-09-17T21:22:51.384708+0000","last_undegraded":"2024-09-17T21:22:51.384708+0000","last_fullsized":"2024-09-17T21:22:51.384708+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:18.942784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048483459999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5340,"num_read_kb":22605,"num_write":215,"num_write_kb":6424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":15742651,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":461,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382421+0000","last_change":"2024-09-17T21:22:50.389225+0000","last_active":"2024-09-17T21:22:51.382421+0000","last_peered":"2024-09-17T21:22:51.382421+0000","last_clean":"2024-09-17T21:22:51.382421+0000","last_became_active":"2024-09-17T21:22:50.388953+0000","last_became_peered":"2024-09-17T21:22:50.388953+0000","last_unstale":"2024-09-17T21:22:51.382421+0000","last_undegraded":"2024-09-17T21:22:51.382421+0000","last_fullsized":"2024-09-17T21:22:51.382421+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:21.062518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"227'1041","reported_seq":8867,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450992+0000","last_change":"2024-09-17T21:22:50.391990+0000","last_active":"2024-09-17T21:22:51.450992+0000","last_peered":"2024-09-17T21:22:51.450992+0000","last_clean":"2024-09-17T21:22:51.450992+0000","last_became_active":"2024-09-17T21:22:50.390218+0000","last_became_peered":"2024-09-17T21:22:50.390218+0000","last_unstale":"2024-09-17T21:22:51.450992+0000","last_undegraded":"2024-09-17T21:22:51.450992+0000","last_fullsized":"2024-09-17T21:22:51.450992+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":1041,"log_dups_size":0,"ondisk_log_size":1041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:24:31.427407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035942119999999998,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":9666,"num_read_kb":63102,"num_write":1578,"num_write_kb":18132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":894,"num_bytes_recovered":48903910,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":474,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.383504+0000","last_change":"2024-09-17T21:18:03.059156+0000","last_active":"2024-09-17T21:22:51.383504+0000","last_peered":"2024-09-17T21:22:51.383504+0000","last_clean":"2024-09-17T21:22:51.383504+0000","last_became_active":"2024-09-17T21:18:01.049180+0000","last_became_peered":"2024-09-17T21:18:01.049180+0000","last_unstale":"2024-09-17T21:22:51.383504+0000","last_undegraded":"2024-09-17T21:22:51.383504+0000","last_fullsized":"2024-09-17T21:22:51.383504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:27.069794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077408,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"226'656","reported_seq":14594,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384939+0000","last_change":"2024-09-17T21:22:41.354650+0000","last_active":"2024-09-17T21:22:51.384939+0000","last_peered":"2024-09-17T21:22:51.384939+0000","last_clean":"2024-09-17T21:22:51.384939+0000","last_became_active":"2024-09-17T21:22:41.352429+0000","last_became_peered":"2024-09-17T21:22:41.352429+0000","last_unstale":"2024-09-17T21:22:51.384939+0000","last_undegraded":"2024-09-17T21:22:51.384939+0000","last_fullsized":"2024-09-17T21:22:51.384939+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:30:46.965276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067184260000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14741,"num_read_kb":38983,"num_write":745,"num_write_kb":20177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":27258987,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":19,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450266+0000","last_change":"2024-09-17T21:22:48.387337+0000","last_active":"2024-09-17T21:22:51.450266+0000","last_peered":"2024-09-17T21:22:51.450266+0000","last_clean":"2024-09-17T21:22:51.450266+0000","last_became_active":"2024-09-17T21:22:48.386572+0000","last_became_peered":"2024-09-17T21:22:48.386572+0000","last_unstale":"2024-09-17T21:22:51.450266+0000","last_undegraded":"2024-09-17T21:22:51.450266+0000","last_fullsized":"2024-09-17T21:22:51.450266+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:55:57.934354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":189,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.387986+0000","last_change":"2024-09-17T21:21:17.347819+0000","last_active":"2024-09-17T21:22:51.387986+0000","last_peered":"2024-09-17T21:22:51.387986+0000","last_clean":"2024-09-17T21:22:51.387986+0000","last_became_active":"2024-09-17T21:21:17.337621+0000","last_became_peered":"2024-09-17T21:21:17.337621+0000","last_unstale":"2024-09-17T21:22:51.387986+0000","last_undegraded":"2024-09-17T21:22:51.387986+0000","last_fullsized":"2024-09-17T21:22:51.387986+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_clean_scrub_stamp":"2024-09-17T21:17:55.982339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:24:32.371870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"224'616","reported_seq":14093,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382313+0000","last_change":"2024-09-17T21:22:50.383917+0000","last_active":"2024-09-17T21:22:51.382313+0000","last_peered":"2024-09-17T21:22:51.382313+0000","last_clean":"2024-09-17T21:22:51.382313+0000","last_became_active":"2024-09-17T21:22:50.382396+0000","last_became_peered":"2024-09-17T21:22:50.382396+0000","last_unstale":"2024-09-17T21:22:51.382313+0000","last_undegraded":"2024-09-17T21:22:51.382313+0000","last_fullsized":"2024-09-17T21:22:51.382313+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:13:13.848853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001602121,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":13993,"num_read_kb":68437,"num_write":586,"num_write_kb":29114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":40776135,"num_keys_recovered":165,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"227'699","reported_seq":11106,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.388697+0000","last_change":"2024-09-17T21:22:49.395344+0000","last_active":"2024-09-17T21:22:51.388697+0000","last_peered":"2024-09-17T21:22:51.388697+0000","last_clean":"2024-09-17T21:22:51.388697+0000","last_became_active":"2024-09-17T21:22:49.390523+0000","last_became_peered":"2024-09-17T21:22:49.390523+0000","last_unstale":"2024-09-17T21:22:51.388697+0000","last_undegraded":"2024-09-17T21:22:51.388697+0000","last_fullsized":"2024-09-17T21:22:51.388697+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:55:28.950314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029497360000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":11677,"num_read_kb":43614,"num_write":1089,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":728,"num_bytes_recovered":30193805,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"227'428","reported_seq":8266,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.383725+0000","last_change":"2024-09-17T21:22:49.397589+0000","last_active":"2024-09-17T21:22:51.383725+0000","last_peered":"2024-09-17T21:22:51.383725+0000","last_clean":"2024-09-17T21:22:51.383725+0000","last_became_active":"2024-09-17T21:22:49.394384+0000","last_became_peered":"2024-09-17T21:22:49.394384+0000","last_unstale":"2024-09-17T21:22:51.383725+0000","last_undegraded":"2024-09-17T21:22:51.383725+0000","last_fullsized":"2024-09-17T21:22:51.383725+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:24:21.885168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045615830000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7186,"num_read_kb":40870,"num_write":369,"num_write_kb":16675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":179794,"num_keys_recovered":2,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"226'376","reported_seq":4920,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384873+0000","last_change":"2024-09-17T21:22:24.375850+0000","last_active":"2024-09-17T21:22:51.384873+0000","last_peered":"2024-09-17T21:22:51.384873+0000","last_clean":"2024-09-17T21:22:51.384873+0000","last_became_active":"2024-09-17T21:21:40.319613+0000","last_became_peered":"2024-09-17T21:21:40.319613+0000","last_unstale":"2024-09-17T21:22:51.384873+0000","last_undegraded":"2024-09-17T21:22:51.384873+0000","last_fullsized":"2024-09-17T21:22:51.384873+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:26:53.101370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038796239999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4694,"num_read_kb":23619,"num_write":320,"num_write_kb":13765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":4886965,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"227'452","reported_seq":13421,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.385236+0000","last_change":"2024-09-17T21:22:43.339607+0000","last_active":"2024-09-17T21:22:51.385236+0000","last_peered":"2024-09-17T21:22:51.385236+0000","last_clean":"2024-09-17T21:22:51.385236+0000","last_became_active":"2024-09-17T21:22:43.338399+0000","last_became_peered":"2024-09-17T21:22:43.338399+0000","last_unstale":"2024-09-17T21:22:51.385236+0000","last_undegraded":"2024-09-17T21:22:51.385236+0000","last_fullsized":"2024-09-17T21:22:51.385236+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:30:14.210043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050460430000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":15379,"num_read_kb":29065,"num_write":580,"num_write_kb":14319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":5960704,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"221'526","reported_seq":15419,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384216+0000","last_change":"2024-09-17T21:22:46.356687+0000","last_active":"2024-09-17T21:22:51.384216+0000","last_peered":"2024-09-17T21:22:51.384216+0000","last_clean":"2024-09-17T21:22:51.384216+0000","last_became_active":"2024-09-17T21:22:46.349835+0000","last_became_peered":"2024-09-17T21:22:46.349835+0000","last_unstale":"2024-09-17T21:22:51.384216+0000","last_undegraded":"2024-09-17T21:22:51.384216+0000","last_fullsized":"2024-09-17T21:22:51.384216+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:51:30.571949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00482724,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15785,"num_read_kb":43220,"num_write":654,"num_write_kb":16401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":28397872,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"227'623","reported_seq":14444,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450685+0000","last_change":"2024-09-17T21:22:49.806087+0000","last_active":"2024-09-17T21:22:51.450685+0000","last_peered":"2024-09-17T21:22:51.450685+0000","last_clean":"2024-09-17T21:22:51.450685+0000","last_became_active":"2024-09-17T21:22:49.394154+0000","last_became_peered":"2024-09-17T21:22:49.394154+0000","last_unstale":"2024-09-17T21:22:51.450685+0000","last_undegraded":"2024-09-17T21:22:51.450685+0000","last_fullsized":"2024-09-17T21:22:51.450685+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:24:57.185890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057317679999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":15141,"num_read_kb":70274,"num_write":535,"num_write_kb":35852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":91925553,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"226'464","reported_seq":7123,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450740+0000","last_change":"2024-09-17T21:22:48.398780+0000","last_active":"2024-09-17T21:22:51.450740+0000","last_peered":"2024-09-17T21:22:51.450740+0000","last_clean":"2024-09-17T21:22:51.450740+0000","last_became_active":"2024-09-17T21:22:48.392669+0000","last_became_peered":"2024-09-17T21:22:48.392669+0000","last_unstale":"2024-09-17T21:22:51.450740+0000","last_undegraded":"2024-09-17T21:22:51.450740+0000","last_fullsized":"2024-09-17T21:22:51.450740+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:18:23.844174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0091147729999999996,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7427,"num_read_kb":28404,"num_write":434,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":4659071,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"222'271","reported_seq":7403,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450132+0000","last_change":"2024-09-17T21:22:48.384974+0000","last_active":"2024-09-17T21:22:51.450132+0000","last_peered":"2024-09-17T21:22:51.450132+0000","last_clean":"2024-09-17T21:22:51.450132+0000","last_became_active":"2024-09-17T21:22:48.381106+0000","last_became_peered":"2024-09-17T21:22:48.381106+0000","last_unstale":"2024-09-17T21:22:51.450132+0000","last_undegraded":"2024-09-17T21:22:51.450132+0000","last_fullsized":"2024-09-17T21:22:51.450132+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:11:47.017985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017482940000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7827,"num_read_kb":19883,"num_write":219,"num_write_kb":3298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":6733181,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"226'404","reported_seq":5779,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384016+0000","last_change":"2024-09-17T21:22:24.385309+0000","last_active":"2024-09-17T21:22:51.384016+0000","last_peered":"2024-09-17T21:22:51.384016+0000","last_clean":"2024-09-17T21:22:51.384016+0000","last_became_active":"2024-09-17T21:18:11.299946+0000","last_became_peered":"2024-09-17T21:18:11.299946+0000","last_unstale":"2024-09-17T21:22:51.384016+0000","last_undegraded":"2024-09-17T21:22:51.384016+0000","last_fullsized":"2024-09-17T21:22:51.384016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:16:16.699933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015128413,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5168,"num_read_kb":22085,"num_write":382,"num_write_kb":4452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"225'570","reported_seq":15995,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450984+0000","last_change":"2024-09-17T21:22:24.368740+0000","last_active":"2024-09-17T21:22:51.450984+0000","last_peered":"2024-09-17T21:22:51.450984+0000","last_clean":"2024-09-17T21:22:51.450984+0000","last_became_active":"2024-09-17T21:22:09.609844+0000","last_became_peered":"2024-09-17T21:22:09.609844+0000","last_unstale":"2024-09-17T21:22:51.450984+0000","last_undegraded":"2024-09-17T21:22:51.450984+0000","last_fullsized":"2024-09-17T21:22:51.450984+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:03:30.364818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018557070000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":17911,"num_read_kb":38759,"num_write":589,"num_write_kb":6968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":11589682,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"226'651","reported_seq":12606,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384243+0000","last_change":"2024-09-17T21:22:41.357868+0000","last_active":"2024-09-17T21:22:51.384243+0000","last_peered":"2024-09-17T21:22:51.384243+0000","last_clean":"2024-09-17T21:22:51.384243+0000","last_became_active":"2024-09-17T21:22:41.354851+0000","last_became_peered":"2024-09-17T21:22:41.354851+0000","last_unstale":"2024-09-17T21:22:51.384243+0000","last_undegraded":"2024-09-17T21:22:51.384243+0000","last_fullsized":"2024-09-17T21:22:51.384243+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:47:04.462955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011937984,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12008,"num_read_kb":52136,"num_write":617,"num_write_kb":5443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":5557098,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":304,"num_object_clones":0,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":300,"num_read":353905,"num_read_kb":1145909,"num_write":19949,"num_write_kb":385384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11936,"num_bytes_recovered":614462337,"num_keys_recovered":1529,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8194,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60472,"kb_used_data":2908,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311368,"statfs":{"total":96636764160,"available":96574840832,"internally_reserved":0,"allocated":2977792,"data_stored":1262263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":58915268},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66616,"kb_used_data":3488,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305224,"statfs":{"total":96636764160,"available":96568549376,"internally_reserved":0,"allocated":3571712,"data_stored":1852631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":64616900},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83256,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288584,"statfs":{"total":96636764160,"available":96551510016,"internally_reserved":0,"allocated":3567616,"data_stored":1852612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":81656262},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56308,"kb_used_data":2912,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315532,"statfs":{"total":96636764160,"available":96579104768,"internally_reserved":0,"allocated":2981888,"data_stored":1262282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56300,"kb_used_data":2904,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315540,"statfs":{"total":96636764160,"available":96579112960,"internally_reserved":0,"allocated":2973696,"data_stored":1262244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":231,"seq":992137445383,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30644,"kb_used_data":2904,"kb_used_omap":20,"kb_used_meta":27691,"kb_avail":94341196,"statfs":{"total":96636764160,"available":96605384704,"internally_reserved":0,"allocated":2973696,"data_stored":1262244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20966,"internal_metadata":28356122},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8194,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:22:59.039 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:22:59.333 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:22:59.333 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T21:22:59.349 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-17T21:22:57.614230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":308,"num_object_clones":0,"num_object_copies":616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":300,"num_read":353971,"num_read_kb":1145964,"num_write":20052,"num_write_kb":387014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11949,"num_bytes_recovered":615643130,"num_keys_recovered":1529,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18605,"ondisk_log_size":18605,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":353596,"kb_used_data":18600,"kb_used_omap":28,"kb_used_meta":334819,"kb_avail":565877444,"statfs":{"total":579820584960,"available":579458502656,"internally_reserved":0,"allocated":19046400,"data_stored":8754276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28944,"internal_metadata":342855408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-1,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-123,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002494"},"pg_stats":[{"pgid":"3.1f","version":"227'759","reported_seq":4118,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451328+0000","last_change":"2024-09-17T21:22:40.360129+0000","last_active":"2024-09-17T21:22:51.451328+0000","last_peered":"2024-09-17T21:22:51.451328+0000","last_clean":"2024-09-17T21:22:51.451328+0000","last_became_active":"2024-09-17T21:22:40.357329+0000","last_became_peered":"2024-09-17T21:22:40.357329+0000","last_unstale":"2024-09-17T21:22:51.451328+0000","last_undegraded":"2024-09-17T21:22:51.451328+0000","last_fullsized":"2024-09-17T21:22:51.451328+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:01:10.890769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010221002999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5921,"num_read_kb":16779,"num_write":1026,"num_write_kb":3495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":5143732,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"227'851","reported_seq":9557,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451353+0000","last_change":"2024-09-17T21:22:40.355216+0000","last_active":"2024-09-17T21:22:51.451353+0000","last_peered":"2024-09-17T21:22:51.451353+0000","last_clean":"2024-09-17T21:22:51.451353+0000","last_became_active":"2024-09-17T21:22:40.353016+0000","last_became_peered":"2024-09-17T21:22:40.353016+0000","last_unstale":"2024-09-17T21:22:51.451353+0000","last_undegraded":"2024-09-17T21:22:51.451353+0000","last_fullsized":"2024-09-17T21:22:51.451353+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":851,"log_dups_size":0,"ondisk_log_size":851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:11.494395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084510550000000007,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10998,"num_read_kb":53173,"num_write":1081,"num_write_kb":18708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":50670363,"num_keys_recovered":90,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"227'833","reported_seq":8688,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450570+0000","last_change":"2024-09-17T21:22:48.408191+0000","last_active":"2024-09-17T21:22:51.450570+0000","last_peered":"2024-09-17T21:22:51.450570+0000","last_clean":"2024-09-17T21:22:51.450570+0000","last_became_active":"2024-09-17T21:22:48.401326+0000","last_became_peered":"2024-09-17T21:22:48.401326+0000","last_unstale":"2024-09-17T21:22:51.450570+0000","last_undegraded":"2024-09-17T21:22:51.450570+0000","last_fullsized":"2024-09-17T21:22:51.450570+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:19:25.050335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082452470000000007,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":10230,"num_read_kb":27857,"num_write":978,"num_write_kb":10466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":12493301,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"227'376","reported_seq":4046,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450419+0000","last_change":"2024-09-17T21:22:48.406130+0000","last_active":"2024-09-17T21:22:51.450419+0000","last_peered":"2024-09-17T21:22:51.450419+0000","last_clean":"2024-09-17T21:22:51.450419+0000","last_became_active":"2024-09-17T21:22:48.401237+0000","last_became_peered":"2024-09-17T21:22:48.401237+0000","last_unstale":"2024-09-17T21:22:51.450419+0000","last_undegraded":"2024-09-17T21:22:51.450419+0000","last_fullsized":"2024-09-17T21:22:51.450419+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:39:30.285487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061565819999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":2726,"num_read_kb":23738,"num_write":383,"num_write_kb":5937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":9397808,"num_keys_recovered":202,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"227'710","reported_seq":8291,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451267+0000","last_change":"2024-09-17T21:22:24.369716+0000","last_active":"2024-09-17T21:22:51.451267+0000","last_peered":"2024-09-17T21:22:51.451267+0000","last_clean":"2024-09-17T21:22:51.451267+0000","last_became_active":"2024-09-17T21:21:41.329830+0000","last_became_peered":"2024-09-17T21:21:41.329830+0000","last_unstale":"2024-09-17T21:22:51.451267+0000","last_undegraded":"2024-09-17T21:22:51.451267+0000","last_fullsized":"2024-09-17T21:22:51.451267+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:42:50.772772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026038680000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":11634,"num_read_kb":21103,"num_write":555,"num_write_kb":4725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":7872032,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"226'739","reported_seq":13918,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.388284+0000","last_change":"2024-09-17T21:22:50.389766+0000","last_active":"2024-09-17T21:22:51.388284+0000","last_peered":"2024-09-17T21:22:51.388284+0000","last_clean":"2024-09-17T21:22:51.388284+0000","last_became_active":"2024-09-17T21:22:50.386623+0000","last_became_peered":"2024-09-17T21:22:50.386623+0000","last_unstale":"2024-09-17T21:22:51.388284+0000","last_undegraded":"2024-09-17T21:22:51.388284+0000","last_fullsized":"2024-09-17T21:22:51.388284+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:41:15.598618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056973060000000001,"stat_sum":{"num_bytes":19,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":16893,"num_read_kb":53539,"num_write":669,"num_write_kb":14565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":647737,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"226'476","reported_seq":12478,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.393411+0000","last_change":"2024-09-17T21:22:51.393063+0000","last_active":"2024-09-17T21:22:51.393411+0000","last_peered":"2024-09-17T21:22:51.393411+0000","last_clean":"2024-09-17T21:22:51.393411+0000","last_became_active":"2024-09-17T21:22:51.390578+0000","last_became_peered":"2024-09-17T21:22:51.390578+0000","last_unstale":"2024-09-17T21:22:51.393411+0000","last_undegraded":"2024-09-17T21:22:51.393411+0000","last_fullsized":"2024-09-17T21:22:51.393411+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:08:03.346817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0077005709999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":12648,"num_read_kb":27025,"num_write":374,"num_write_kb":5615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":3100207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"226'318","reported_seq":3035,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382486+0000","last_change":"2024-09-17T21:22:50.387935+0000","last_active":"2024-09-17T21:22:51.382486+0000","last_peered":"2024-09-17T21:22:51.382486+0000","last_clean":"2024-09-17T21:22:51.382486+0000","last_became_active":"2024-09-17T21:22:50.385975+0000","last_became_peered":"2024-09-17T21:22:50.385975+0000","last_unstale":"2024-09-17T21:22:51.382486+0000","last_undegraded":"2024-09-17T21:22:51.382486+0000","last_fullsized":"2024-09-17T21:22:51.382486+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:26:13.518654+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022643899999999998,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2392,"num_read_kb":9139,"num_write":304,"num_write_kb":2898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":9226431,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"225'393","reported_seq":7343,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451410+0000","last_change":"2024-09-17T21:22:43.342179+0000","last_active":"2024-09-17T21:22:51.451410+0000","last_peered":"2024-09-17T21:22:51.451410+0000","last_clean":"2024-09-17T21:22:51.451410+0000","last_became_active":"2024-09-17T21:22:43.340739+0000","last_became_peered":"2024-09-17T21:22:43.340739+0000","last_unstale":"2024-09-17T21:22:51.451410+0000","last_undegraded":"2024-09-17T21:22:51.451410+0000","last_fullsized":"2024-09-17T21:22:51.451410+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:32:48.751230+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00138835,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7054,"num_read_kb":21313,"num_write":475,"num_write_kb":6946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":719,"num_bytes_recovered":31066012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"224'602","reported_seq":22252,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450629+0000","last_change":"2024-09-17T21:22:48.400263+0000","last_active":"2024-09-17T21:22:51.450629+0000","last_peered":"2024-09-17T21:22:51.450629+0000","last_clean":"2024-09-17T21:22:51.450629+0000","last_became_active":"2024-09-17T21:22:48.394816+0000","last_became_peered":"2024-09-17T21:22:48.394816+0000","last_unstale":"2024-09-17T21:22:51.450629+0000","last_undegraded":"2024-09-17T21:22:51.450629+0000","last_fullsized":"2024-09-17T21:22:51.450629+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:05.075504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075408430000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":22417,"num_read_kb":27659,"num_write":858,"num_write_kb":13022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":3432016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"223'754","reported_seq":7158,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384191+0000","last_change":"2024-09-17T21:22:24.378500+0000","last_active":"2024-09-17T21:22:51.384191+0000","last_peered":"2024-09-17T21:22:51.384191+0000","last_clean":"2024-09-17T21:22:51.384191+0000","last_became_active":"2024-09-17T21:21:40.321648+0000","last_became_peered":"2024-09-17T21:21:40.321648+0000","last_unstale":"2024-09-17T21:22:51.384191+0000","last_undegraded":"2024-09-17T21:22:51.384191+0000","last_fullsized":"2024-09-17T21:22:51.384191+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:54:37.691837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0071138240000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8592,"num_read_kb":29445,"num_write":954,"num_write_kb":5554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":9885558,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"225'449","reported_seq":5376,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.383815+0000","last_change":"2024-09-17T21:22:49.393985+0000","last_active":"2024-09-17T21:22:51.383815+0000","last_peered":"2024-09-17T21:22:51.383815+0000","last_clean":"2024-09-17T21:22:51.383815+0000","last_became_active":"2024-09-17T21:22:49.389098+0000","last_became_peered":"2024-09-17T21:22:49.389098+0000","last_unstale":"2024-09-17T21:22:51.383815+0000","last_undegraded":"2024-09-17T21:22:51.383815+0000","last_fullsized":"2024-09-17T21:22:51.383815+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:28:53.422353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003029,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5351,"num_read_kb":26972,"num_write":428,"num_write_kb":16010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":385,"num_bytes_recovered":6943878,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":474,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384273+0000","last_change":"2024-09-17T21:18:03.059715+0000","last_active":"2024-09-17T21:22:51.384273+0000","last_peered":"2024-09-17T21:22:51.384273+0000","last_clean":"2024-09-17T21:22:51.384273+0000","last_became_active":"2024-09-17T21:18:01.213047+0000","last_became_peered":"2024-09-17T21:18:01.213047+0000","last_unstale":"2024-09-17T21:22:51.384273+0000","last_undegraded":"2024-09-17T21:22:51.384273+0000","last_fullsized":"2024-09-17T21:22:51.384273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:46:35.222373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"227'594","reported_seq":13694,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451462+0000","last_change":"2024-09-17T21:22:24.371187+0000","last_active":"2024-09-17T21:22:51.451462+0000","last_peered":"2024-09-17T21:22:51.451462+0000","last_clean":"2024-09-17T21:22:51.451462+0000","last_became_active":"2024-09-17T21:21:40.318488+0000","last_became_peered":"2024-09-17T21:21:40.318488+0000","last_unstale":"2024-09-17T21:22:51.451462+0000","last_undegraded":"2024-09-17T21:22:51.451462+0000","last_fullsized":"2024-09-17T21:22:51.451462+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:57:56.726468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036709440000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14882,"num_read_kb":61123,"num_write":581,"num_write_kb":23555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":49089084,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":474,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.385630+0000","last_change":"2024-09-17T21:18:03.073440+0000","last_active":"2024-09-17T21:22:51.385630+0000","last_peered":"2024-09-17T21:22:51.385630+0000","last_clean":"2024-09-17T21:22:51.385630+0000","last_became_active":"2024-09-17T21:18:01.048806+0000","last_became_peered":"2024-09-17T21:18:01.048806+0000","last_unstale":"2024-09-17T21:22:51.385630+0000","last_undegraded":"2024-09-17T21:22:51.385630+0000","last_fullsized":"2024-09-17T21:22:51.385630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:26.487163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094161400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"226'352","reported_seq":5610,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382685+0000","last_change":"2024-09-17T21:22:50.390979+0000","last_active":"2024-09-17T21:22:51.382685+0000","last_peered":"2024-09-17T21:22:51.382685+0000","last_clean":"2024-09-17T21:22:51.382685+0000","last_became_active":"2024-09-17T21:22:50.388965+0000","last_became_peered":"2024-09-17T21:22:50.388965+0000","last_unstale":"2024-09-17T21:22:51.382685+0000","last_undegraded":"2024-09-17T21:22:51.382685+0000","last_fullsized":"2024-09-17T21:22:51.382685+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:18:04.741617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079556969999999994,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4656,"num_read_kb":36280,"num_write":392,"num_write_kb":13876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":5244262,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":498,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384423+0000","last_change":"2024-09-17T21:21:35.319221+0000","last_active":"2024-09-17T21:22:51.384423+0000","last_peered":"2024-09-17T21:22:51.384423+0000","last_clean":"2024-09-17T21:22:51.384423+0000","last_became_active":"2024-09-17T21:21:35.312507+0000","last_became_peered":"2024-09-17T21:21:35.312507+0000","last_unstale":"2024-09-17T21:22:51.384423+0000","last_undegraded":"2024-09-17T21:22:51.384423+0000","last_fullsized":"2024-09-17T21:22:51.384423+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:53.042567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099219900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"226'707","reported_seq":16135,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.385480+0000","last_change":"2024-09-17T21:22:46.361219+0000","last_active":"2024-09-17T21:22:51.385480+0000","last_peered":"2024-09-17T21:22:51.385480+0000","last_clean":"2024-09-17T21:22:51.385480+0000","last_became_active":"2024-09-17T21:22:46.359525+0000","last_became_peered":"2024-09-17T21:22:46.359525+0000","last_unstale":"2024-09-17T21:22:51.385480+0000","last_undegraded":"2024-09-17T21:22:51.385480+0000","last_fullsized":"2024-09-17T21:22:51.385480+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:47.174816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052152359999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13338,"num_read_kb":29382,"num_write":606,"num_write_kb":10802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":367,"num_bytes_recovered":21385444,"num_keys_recovered":118,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":498,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.385587+0000","last_change":"2024-09-17T21:21:35.326814+0000","last_active":"2024-09-17T21:22:51.385587+0000","last_peered":"2024-09-17T21:22:51.385587+0000","last_clean":"2024-09-17T21:22:51.385587+0000","last_became_active":"2024-09-17T21:21:35.325777+0000","last_became_peered":"2024-09-17T21:21:35.325777+0000","last_unstale":"2024-09-17T21:22:51.385587+0000","last_undegraded":"2024-09-17T21:22:51.385587+0000","last_fullsized":"2024-09-17T21:22:51.385587+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:47:00.558900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006222,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"225'1056","reported_seq":33580,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450393+0000","last_change":"2024-09-17T21:22:44.360752+0000","last_active":"2024-09-17T21:22:51.450393+0000","last_peered":"2024-09-17T21:22:51.450393+0000","last_clean":"2024-09-17T21:22:51.450393+0000","last_became_active":"2024-09-17T21:22:44.352239+0000","last_became_peered":"2024-09-17T21:22:44.352239+0000","last_unstale":"2024-09-17T21:22:51.450393+0000","last_undegraded":"2024-09-17T21:22:51.450393+0000","last_fullsized":"2024-09-17T21:22:51.450393+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":1056,"log_dups_size":0,"ondisk_log_size":1056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:36:58.893607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052118770000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":37177,"num_read_kb":61903,"num_write":871,"num_write_kb":21852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":754,"num_bytes_recovered":58900004,"num_keys_recovered":204,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"227'536","reported_seq":3887,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.388246+0000","last_change":"2024-09-17T21:22:24.383832+0000","last_active":"2024-09-17T21:22:51.388246+0000","last_peered":"2024-09-17T21:22:51.388246+0000","last_clean":"2024-09-17T21:22:51.388246+0000","last_became_active":"2024-09-17T21:21:39.315142+0000","last_became_peered":"2024-09-17T21:21:39.315142+0000","last_unstale":"2024-09-17T21:22:51.388246+0000","last_undegraded":"2024-09-17T21:22:51.388246+0000","last_fullsized":"2024-09-17T21:22:51.388246+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:05:28.580363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061334989999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":3053,"num_read_kb":14423,"num_write":502,"num_write_kb":4818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":7199060,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":412,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450196+0000","last_change":"2024-09-17T21:22:48.383726+0000","last_active":"2024-09-17T21:22:51.450196+0000","last_peered":"2024-09-17T21:22:51.450196+0000","last_clean":"2024-09-17T21:22:51.450196+0000","last_became_active":"2024-09-17T21:22:48.382842+0000","last_became_peered":"2024-09-17T21:22:48.382842+0000","last_unstale":"2024-09-17T21:22:51.450196+0000","last_undegraded":"2024-09-17T21:22:51.450196+0000","last_fullsized":"2024-09-17T21:22:51.450196+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:30:55.312809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"226'248","reported_seq":9446,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384708+0000","last_change":"2024-09-17T21:22:45.354353+0000","last_active":"2024-09-17T21:22:51.384708+0000","last_peered":"2024-09-17T21:22:51.384708+0000","last_clean":"2024-09-17T21:22:51.384708+0000","last_became_active":"2024-09-17T21:22:45.352682+0000","last_became_peered":"2024-09-17T21:22:45.352682+0000","last_unstale":"2024-09-17T21:22:51.384708+0000","last_undegraded":"2024-09-17T21:22:51.384708+0000","last_fullsized":"2024-09-17T21:22:51.384708+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:18.942784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048483459999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5340,"num_read_kb":22605,"num_write":215,"num_write_kb":6424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":15742651,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":461,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382421+0000","last_change":"2024-09-17T21:22:50.389225+0000","last_active":"2024-09-17T21:22:51.382421+0000","last_peered":"2024-09-17T21:22:51.382421+0000","last_clean":"2024-09-17T21:22:51.382421+0000","last_became_active":"2024-09-17T21:22:50.388953+0000","last_became_peered":"2024-09-17T21:22:50.388953+0000","last_unstale":"2024-09-17T21:22:51.382421+0000","last_undegraded":"2024-09-17T21:22:51.382421+0000","last_fullsized":"2024-09-17T21:22:51.382421+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:21.062518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"227'1041","reported_seq":8867,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450992+0000","last_change":"2024-09-17T21:22:50.391990+0000","last_active":"2024-09-17T21:22:51.450992+0000","last_peered":"2024-09-17T21:22:51.450992+0000","last_clean":"2024-09-17T21:22:51.450992+0000","last_became_active":"2024-09-17T21:22:50.390218+0000","last_became_peered":"2024-09-17T21:22:50.390218+0000","last_unstale":"2024-09-17T21:22:51.450992+0000","last_undegraded":"2024-09-17T21:22:51.450992+0000","last_fullsized":"2024-09-17T21:22:51.450992+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":1041,"log_dups_size":0,"ondisk_log_size":1041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:24:31.427407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035942119999999998,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":9666,"num_read_kb":63102,"num_write":1578,"num_write_kb":18132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":894,"num_bytes_recovered":48903910,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":474,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.383504+0000","last_change":"2024-09-17T21:18:03.059156+0000","last_active":"2024-09-17T21:22:51.383504+0000","last_peered":"2024-09-17T21:22:51.383504+0000","last_clean":"2024-09-17T21:22:51.383504+0000","last_became_active":"2024-09-17T21:18:01.049180+0000","last_became_peered":"2024-09-17T21:18:01.049180+0000","last_unstale":"2024-09-17T21:22:51.383504+0000","last_undegraded":"2024-09-17T21:22:51.383504+0000","last_fullsized":"2024-09-17T21:22:51.383504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:27.069794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077408,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"226'656","reported_seq":14594,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384939+0000","last_change":"2024-09-17T21:22:41.354650+0000","last_active":"2024-09-17T21:22:51.384939+0000","last_peered":"2024-09-17T21:22:51.384939+0000","last_clean":"2024-09-17T21:22:51.384939+0000","last_became_active":"2024-09-17T21:22:41.352429+0000","last_became_peered":"2024-09-17T21:22:41.352429+0000","last_unstale":"2024-09-17T21:22:51.384939+0000","last_undegraded":"2024-09-17T21:22:51.384939+0000","last_fullsized":"2024-09-17T21:22:51.384939+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:30:46.965276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067184260000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14741,"num_read_kb":38983,"num_write":745,"num_write_kb":20177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":27258987,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":19,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450266+0000","last_change":"2024-09-17T21:22:48.387337+0000","last_active":"2024-09-17T21:22:51.450266+0000","last_peered":"2024-09-17T21:22:51.450266+0000","last_clean":"2024-09-17T21:22:51.450266+0000","last_became_active":"2024-09-17T21:22:48.386572+0000","last_became_peered":"2024-09-17T21:22:48.386572+0000","last_unstale":"2024-09-17T21:22:51.450266+0000","last_undegraded":"2024-09-17T21:22:51.450266+0000","last_fullsized":"2024-09-17T21:22:51.450266+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:55:57.934354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":189,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.387986+0000","last_change":"2024-09-17T21:21:17.347819+0000","last_active":"2024-09-17T21:22:51.387986+0000","last_peered":"2024-09-17T21:22:51.387986+0000","last_clean":"2024-09-17T21:22:51.387986+0000","last_became_active":"2024-09-17T21:21:17.337621+0000","last_became_peered":"2024-09-17T21:21:17.337621+0000","last_unstale":"2024-09-17T21:22:51.387986+0000","last_undegraded":"2024-09-17T21:22:51.387986+0000","last_fullsized":"2024-09-17T21:22:51.387986+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_clean_scrub_stamp":"2024-09-17T21:17:55.982339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:24:32.371870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"224'616","reported_seq":14093,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382313+0000","last_change":"2024-09-17T21:22:50.383917+0000","last_active":"2024-09-17T21:22:51.382313+0000","last_peered":"2024-09-17T21:22:51.382313+0000","last_clean":"2024-09-17T21:22:51.382313+0000","last_became_active":"2024-09-17T21:22:50.382396+0000","last_became_peered":"2024-09-17T21:22:50.382396+0000","last_unstale":"2024-09-17T21:22:51.382313+0000","last_undegraded":"2024-09-17T21:22:51.382313+0000","last_fullsized":"2024-09-17T21:22:51.382313+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:13:13.848853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001602121,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":13993,"num_read_kb":68437,"num_write":586,"num_write_kb":29114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":40776135,"num_keys_recovered":165,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"227'699","reported_seq":11106,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.388697+0000","last_change":"2024-09-17T21:22:49.395344+0000","last_active":"2024-09-17T21:22:51.388697+0000","last_peered":"2024-09-17T21:22:51.388697+0000","last_clean":"2024-09-17T21:22:51.388697+0000","last_became_active":"2024-09-17T21:22:49.390523+0000","last_became_peered":"2024-09-17T21:22:49.390523+0000","last_unstale":"2024-09-17T21:22:51.388697+0000","last_undegraded":"2024-09-17T21:22:51.388697+0000","last_fullsized":"2024-09-17T21:22:51.388697+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:55:28.950314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029497360000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":11677,"num_read_kb":43614,"num_write":1089,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":728,"num_bytes_recovered":30193805,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"227'428","reported_seq":8266,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.383725+0000","last_change":"2024-09-17T21:22:49.397589+0000","last_active":"2024-09-17T21:22:51.383725+0000","last_peered":"2024-09-17T21:22:51.383725+0000","last_clean":"2024-09-17T21:22:51.383725+0000","last_became_active":"2024-09-17T21:22:49.394384+0000","last_became_peered":"2024-09-17T21:22:49.394384+0000","last_unstale":"2024-09-17T21:22:51.383725+0000","last_undegraded":"2024-09-17T21:22:51.383725+0000","last_fullsized":"2024-09-17T21:22:51.383725+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:24:21.885168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045615830000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7186,"num_read_kb":40870,"num_write":369,"num_write_kb":16675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":179794,"num_keys_recovered":2,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"226'376","reported_seq":4920,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384873+0000","last_change":"2024-09-17T21:22:24.375850+0000","last_active":"2024-09-17T21:22:51.384873+0000","last_peered":"2024-09-17T21:22:51.384873+0000","last_clean":"2024-09-17T21:22:51.384873+0000","last_became_active":"2024-09-17T21:21:40.319613+0000","last_became_peered":"2024-09-17T21:21:40.319613+0000","last_unstale":"2024-09-17T21:22:51.384873+0000","last_undegraded":"2024-09-17T21:22:51.384873+0000","last_fullsized":"2024-09-17T21:22:51.384873+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:26:53.101370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038796239999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4694,"num_read_kb":23619,"num_write":320,"num_write_kb":13765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":4886965,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"227'452","reported_seq":13421,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.385236+0000","last_change":"2024-09-17T21:22:43.339607+0000","last_active":"2024-09-17T21:22:51.385236+0000","last_peered":"2024-09-17T21:22:51.385236+0000","last_clean":"2024-09-17T21:22:51.385236+0000","last_became_active":"2024-09-17T21:22:43.338399+0000","last_became_peered":"2024-09-17T21:22:43.338399+0000","last_unstale":"2024-09-17T21:22:51.385236+0000","last_undegraded":"2024-09-17T21:22:51.385236+0000","last_fullsized":"2024-09-17T21:22:51.385236+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:30:14.210043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050460430000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":15379,"num_read_kb":29065,"num_write":580,"num_write_kb":14319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":5960704,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"221'526","reported_seq":15419,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384216+0000","last_change":"2024-09-17T21:22:46.356687+0000","last_active":"2024-09-17T21:22:51.384216+0000","last_peered":"2024-09-17T21:22:51.384216+0000","last_clean":"2024-09-17T21:22:51.384216+0000","last_became_active":"2024-09-17T21:22:46.349835+0000","last_became_peered":"2024-09-17T21:22:46.349835+0000","last_unstale":"2024-09-17T21:22:51.384216+0000","last_undegraded":"2024-09-17T21:22:51.384216+0000","last_fullsized":"2024-09-17T21:22:51.384216+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:51:30.571949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00482724,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15785,"num_read_kb":43220,"num_write":654,"num_write_kb":16401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":28397872,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"227'623","reported_seq":14444,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450685+0000","last_change":"2024-09-17T21:22:49.806087+0000","last_active":"2024-09-17T21:22:51.450685+0000","last_peered":"2024-09-17T21:22:51.450685+0000","last_clean":"2024-09-17T21:22:51.450685+0000","last_became_active":"2024-09-17T21:22:49.394154+0000","last_became_peered":"2024-09-17T21:22:49.394154+0000","last_unstale":"2024-09-17T21:22:51.450685+0000","last_undegraded":"2024-09-17T21:22:51.450685+0000","last_fullsized":"2024-09-17T21:22:51.450685+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:24:57.185890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057317679999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":15141,"num_read_kb":70274,"num_write":535,"num_write_kb":35852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":91925553,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"226'464","reported_seq":7123,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450740+0000","last_change":"2024-09-17T21:22:48.398780+0000","last_active":"2024-09-17T21:22:51.450740+0000","last_peered":"2024-09-17T21:22:51.450740+0000","last_clean":"2024-09-17T21:22:51.450740+0000","last_became_active":"2024-09-17T21:22:48.392669+0000","last_became_peered":"2024-09-17T21:22:48.392669+0000","last_unstale":"2024-09-17T21:22:51.450740+0000","last_undegraded":"2024-09-17T21:22:51.450740+0000","last_fullsized":"2024-09-17T21:22:51.450740+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:18:23.844174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0091147729999999996,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7427,"num_read_kb":28404,"num_write":434,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":4659071,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"222'271","reported_seq":7403,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450132+0000","last_change":"2024-09-17T21:22:48.384974+0000","last_active":"2024-09-17T21:22:51.450132+0000","last_peered":"2024-09-17T21:22:51.450132+0000","last_clean":"2024-09-17T21:22:51.450132+0000","last_became_active":"2024-09-17T21:22:48.381106+0000","last_became_peered":"2024-09-17T21:22:48.381106+0000","last_unstale":"2024-09-17T21:22:51.450132+0000","last_undegraded":"2024-09-17T21:22:51.450132+0000","last_fullsized":"2024-09-17T21:22:51.450132+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:11:47.017985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017482940000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7827,"num_read_kb":19883,"num_write":219,"num_write_kb":3298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":6733181,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"226'404","reported_seq":5779,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384016+0000","last_change":"2024-09-17T21:22:24.385309+0000","last_active":"2024-09-17T21:22:51.384016+0000","last_peered":"2024-09-17T21:22:51.384016+0000","last_clean":"2024-09-17T21:22:51.384016+0000","last_became_active":"2024-09-17T21:18:11.299946+0000","last_became_peered":"2024-09-17T21:18:11.299946+0000","last_unstale":"2024-09-17T21:22:51.384016+0000","last_undegraded":"2024-09-17T21:22:51.384016+0000","last_fullsized":"2024-09-17T21:22:51.384016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:16:16.699933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015128413,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5168,"num_read_kb":22085,"num_write":382,"num_write_kb":4452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"225'570","reported_seq":15995,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450984+0000","last_change":"2024-09-17T21:22:24.368740+0000","last_active":"2024-09-17T21:22:51.450984+0000","last_peered":"2024-09-17T21:22:51.450984+0000","last_clean":"2024-09-17T21:22:51.450984+0000","last_became_active":"2024-09-17T21:22:09.609844+0000","last_became_peered":"2024-09-17T21:22:09.609844+0000","last_unstale":"2024-09-17T21:22:51.450984+0000","last_undegraded":"2024-09-17T21:22:51.450984+0000","last_fullsized":"2024-09-17T21:22:51.450984+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:03:30.364818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018557070000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":17911,"num_read_kb":38759,"num_write":589,"num_write_kb":6968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":11589682,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"226'651","reported_seq":12606,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384243+0000","last_change":"2024-09-17T21:22:41.357868+0000","last_active":"2024-09-17T21:22:51.384243+0000","last_peered":"2024-09-17T21:22:51.384243+0000","last_clean":"2024-09-17T21:22:51.384243+0000","last_became_active":"2024-09-17T21:22:41.354851+0000","last_became_peered":"2024-09-17T21:22:41.354851+0000","last_unstale":"2024-09-17T21:22:51.384243+0000","last_undegraded":"2024-09-17T21:22:51.384243+0000","last_fullsized":"2024-09-17T21:22:51.384243+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:47:04.462955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011937984,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12008,"num_read_kb":52136,"num_write":617,"num_write_kb":5443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":5557098,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":304,"num_object_clones":0,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":300,"num_read":353905,"num_read_kb":1145909,"num_write":19949,"num_write_kb":385384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11936,"num_bytes_recovered":614462337,"num_keys_recovered":1529,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8194,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60472,"kb_used_data":2908,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311368,"statfs":{"total":96636764160,"available":96574840832,"internally_reserved":0,"allocated":2977792,"data_stored":1262263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":58915268},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66616,"kb_used_data":3488,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305224,"statfs":{"total":96636764160,"available":96568549376,"internally_reserved":0,"allocated":3571712,"data_stored":1852631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":64616900},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83256,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288584,"statfs":{"total":96636764160,"available":96551510016,"internally_reserved":0,"allocated":3567616,"data_stored":1852612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":81656262},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56308,"kb_used_data":2912,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315532,"statfs":{"total":96636764160,"available":96579104768,"internally_reserved":0,"allocated":2981888,"data_stored":1262282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56300,"kb_used_data":2904,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315540,"statfs":{"total":96636764160,"available":96579112960,"internally_reserved":0,"allocated":2973696,"data_stored":1262244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":231,"seq":992137445383,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30644,"kb_used_data":2904,"kb_used_omap":20,"kb_used_meta":27691,"kb_avail":94341196,"statfs":{"total":96636764160,"available":96605384704,"internally_reserved":0,"allocated":2973696,"data_stored":1262244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20966,"internal_metadata":28356122},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8194,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:22:59.352 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T21:22:59.352 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:22:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:22:59.640 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T21:22:59.657 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-09-17T21:22:59.614789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":308,"num_object_clones":0,"num_object_copies":616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":300,"num_read":353971,"num_read_kb":1145964,"num_write":20052,"num_write_kb":387014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11949,"num_bytes_recovered":615643130,"num_keys_recovered":1529,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18605,"ondisk_log_size":18605,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":353596,"kb_used_data":18600,"kb_used_omap":28,"kb_used_meta":334819,"kb_avail":565877444,"statfs":{"total":579820584960,"available":579458502656,"internally_reserved":0,"allocated":19046400,"data_stored":8754276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28944,"internal_metadata":342855408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.239707"},"pg_stats":[{"pgid":"3.1f","version":"227'759","reported_seq":4118,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451328+0000","last_change":"2024-09-17T21:22:40.360129+0000","last_active":"2024-09-17T21:22:51.451328+0000","last_peered":"2024-09-17T21:22:51.451328+0000","last_clean":"2024-09-17T21:22:51.451328+0000","last_became_active":"2024-09-17T21:22:40.357329+0000","last_became_peered":"2024-09-17T21:22:40.357329+0000","last_unstale":"2024-09-17T21:22:51.451328+0000","last_undegraded":"2024-09-17T21:22:51.451328+0000","last_fullsized":"2024-09-17T21:22:51.451328+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:01:10.890769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010221002999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5921,"num_read_kb":16779,"num_write":1026,"num_write_kb":3495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":5143732,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"227'851","reported_seq":9557,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451353+0000","last_change":"2024-09-17T21:22:40.355216+0000","last_active":"2024-09-17T21:22:51.451353+0000","last_peered":"2024-09-17T21:22:51.451353+0000","last_clean":"2024-09-17T21:22:51.451353+0000","last_became_active":"2024-09-17T21:22:40.353016+0000","last_became_peered":"2024-09-17T21:22:40.353016+0000","last_unstale":"2024-09-17T21:22:51.451353+0000","last_undegraded":"2024-09-17T21:22:51.451353+0000","last_fullsized":"2024-09-17T21:22:51.451353+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":851,"log_dups_size":0,"ondisk_log_size":851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:11.494395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084510550000000007,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10998,"num_read_kb":53173,"num_write":1081,"num_write_kb":18708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":50670363,"num_keys_recovered":90,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"227'833","reported_seq":8688,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450570+0000","last_change":"2024-09-17T21:22:48.408191+0000","last_active":"2024-09-17T21:22:51.450570+0000","last_peered":"2024-09-17T21:22:51.450570+0000","last_clean":"2024-09-17T21:22:51.450570+0000","last_became_active":"2024-09-17T21:22:48.401326+0000","last_became_peered":"2024-09-17T21:22:48.401326+0000","last_unstale":"2024-09-17T21:22:51.450570+0000","last_undegraded":"2024-09-17T21:22:51.450570+0000","last_fullsized":"2024-09-17T21:22:51.450570+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:19:25.050335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082452470000000007,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":10230,"num_read_kb":27857,"num_write":978,"num_write_kb":10466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":12493301,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"227'376","reported_seq":4046,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450419+0000","last_change":"2024-09-17T21:22:48.406130+0000","last_active":"2024-09-17T21:22:51.450419+0000","last_peered":"2024-09-17T21:22:51.450419+0000","last_clean":"2024-09-17T21:22:51.450419+0000","last_became_active":"2024-09-17T21:22:48.401237+0000","last_became_peered":"2024-09-17T21:22:48.401237+0000","last_unstale":"2024-09-17T21:22:51.450419+0000","last_undegraded":"2024-09-17T21:22:51.450419+0000","last_fullsized":"2024-09-17T21:22:51.450419+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:39:30.285487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061565819999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":2726,"num_read_kb":23738,"num_write":383,"num_write_kb":5937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":9397808,"num_keys_recovered":202,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"227'710","reported_seq":8291,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451267+0000","last_change":"2024-09-17T21:22:24.369716+0000","last_active":"2024-09-17T21:22:51.451267+0000","last_peered":"2024-09-17T21:22:51.451267+0000","last_clean":"2024-09-17T21:22:51.451267+0000","last_became_active":"2024-09-17T21:21:41.329830+0000","last_became_peered":"2024-09-17T21:21:41.329830+0000","last_unstale":"2024-09-17T21:22:51.451267+0000","last_undegraded":"2024-09-17T21:22:51.451267+0000","last_fullsized":"2024-09-17T21:22:51.451267+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:42:50.772772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026038680000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":11634,"num_read_kb":21103,"num_write":555,"num_write_kb":4725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":7872032,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"226'739","reported_seq":13918,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.388284+0000","last_change":"2024-09-17T21:22:50.389766+0000","last_active":"2024-09-17T21:22:51.388284+0000","last_peered":"2024-09-17T21:22:51.388284+0000","last_clean":"2024-09-17T21:22:51.388284+0000","last_became_active":"2024-09-17T21:22:50.386623+0000","last_became_peered":"2024-09-17T21:22:50.386623+0000","last_unstale":"2024-09-17T21:22:51.388284+0000","last_undegraded":"2024-09-17T21:22:51.388284+0000","last_fullsized":"2024-09-17T21:22:51.388284+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:41:15.598618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056973060000000001,"stat_sum":{"num_bytes":19,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":16893,"num_read_kb":53539,"num_write":669,"num_write_kb":14565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":647737,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"226'476","reported_seq":12478,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.393411+0000","last_change":"2024-09-17T21:22:51.393063+0000","last_active":"2024-09-17T21:22:51.393411+0000","last_peered":"2024-09-17T21:22:51.393411+0000","last_clean":"2024-09-17T21:22:51.393411+0000","last_became_active":"2024-09-17T21:22:51.390578+0000","last_became_peered":"2024-09-17T21:22:51.390578+0000","last_unstale":"2024-09-17T21:22:51.393411+0000","last_undegraded":"2024-09-17T21:22:51.393411+0000","last_fullsized":"2024-09-17T21:22:51.393411+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:08:03.346817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0077005709999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":12648,"num_read_kb":27025,"num_write":374,"num_write_kb":5615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":3100207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"226'318","reported_seq":3035,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382486+0000","last_change":"2024-09-17T21:22:50.387935+0000","last_active":"2024-09-17T21:22:51.382486+0000","last_peered":"2024-09-17T21:22:51.382486+0000","last_clean":"2024-09-17T21:22:51.382486+0000","last_became_active":"2024-09-17T21:22:50.385975+0000","last_became_peered":"2024-09-17T21:22:50.385975+0000","last_unstale":"2024-09-17T21:22:51.382486+0000","last_undegraded":"2024-09-17T21:22:51.382486+0000","last_fullsized":"2024-09-17T21:22:51.382486+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:26:13.518654+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022643899999999998,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2392,"num_read_kb":9139,"num_write":304,"num_write_kb":2898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":9226431,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"225'393","reported_seq":7343,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451410+0000","last_change":"2024-09-17T21:22:43.342179+0000","last_active":"2024-09-17T21:22:51.451410+0000","last_peered":"2024-09-17T21:22:51.451410+0000","last_clean":"2024-09-17T21:22:51.451410+0000","last_became_active":"2024-09-17T21:22:43.340739+0000","last_became_peered":"2024-09-17T21:22:43.340739+0000","last_unstale":"2024-09-17T21:22:51.451410+0000","last_undegraded":"2024-09-17T21:22:51.451410+0000","last_fullsized":"2024-09-17T21:22:51.451410+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:32:48.751230+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00138835,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7054,"num_read_kb":21313,"num_write":475,"num_write_kb":6946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":719,"num_bytes_recovered":31066012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"224'602","reported_seq":22252,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450629+0000","last_change":"2024-09-17T21:22:48.400263+0000","last_active":"2024-09-17T21:22:51.450629+0000","last_peered":"2024-09-17T21:22:51.450629+0000","last_clean":"2024-09-17T21:22:51.450629+0000","last_became_active":"2024-09-17T21:22:48.394816+0000","last_became_peered":"2024-09-17T21:22:48.394816+0000","last_unstale":"2024-09-17T21:22:51.450629+0000","last_undegraded":"2024-09-17T21:22:51.450629+0000","last_fullsized":"2024-09-17T21:22:51.450629+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:05.075504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075408430000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":22417,"num_read_kb":27659,"num_write":858,"num_write_kb":13022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":3432016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"223'754","reported_seq":7158,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384191+0000","last_change":"2024-09-17T21:22:24.378500+0000","last_active":"2024-09-17T21:22:51.384191+0000","last_peered":"2024-09-17T21:22:51.384191+0000","last_clean":"2024-09-17T21:22:51.384191+0000","last_became_active":"2024-09-17T21:21:40.321648+0000","last_became_peered":"2024-09-17T21:21:40.321648+0000","last_unstale":"2024-09-17T21:22:51.384191+0000","last_undegraded":"2024-09-17T21:22:51.384191+0000","last_fullsized":"2024-09-17T21:22:51.384191+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:54:37.691837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0071138240000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8592,"num_read_kb":29445,"num_write":954,"num_write_kb":5554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":9885558,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"225'449","reported_seq":5376,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.383815+0000","last_change":"2024-09-17T21:22:49.393985+0000","last_active":"2024-09-17T21:22:51.383815+0000","last_peered":"2024-09-17T21:22:51.383815+0000","last_clean":"2024-09-17T21:22:51.383815+0000","last_became_active":"2024-09-17T21:22:49.389098+0000","last_became_peered":"2024-09-17T21:22:49.389098+0000","last_unstale":"2024-09-17T21:22:51.383815+0000","last_undegraded":"2024-09-17T21:22:51.383815+0000","last_fullsized":"2024-09-17T21:22:51.383815+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:28:53.422353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003029,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5351,"num_read_kb":26972,"num_write":428,"num_write_kb":16010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":385,"num_bytes_recovered":6943878,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":474,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384273+0000","last_change":"2024-09-17T21:18:03.059715+0000","last_active":"2024-09-17T21:22:51.384273+0000","last_peered":"2024-09-17T21:22:51.384273+0000","last_clean":"2024-09-17T21:22:51.384273+0000","last_became_active":"2024-09-17T21:18:01.213047+0000","last_became_peered":"2024-09-17T21:18:01.213047+0000","last_unstale":"2024-09-17T21:22:51.384273+0000","last_undegraded":"2024-09-17T21:22:51.384273+0000","last_fullsized":"2024-09-17T21:22:51.384273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:46:35.222373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"227'594","reported_seq":13694,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451462+0000","last_change":"2024-09-17T21:22:24.371187+0000","last_active":"2024-09-17T21:22:51.451462+0000","last_peered":"2024-09-17T21:22:51.451462+0000","last_clean":"2024-09-17T21:22:51.451462+0000","last_became_active":"2024-09-17T21:21:40.318488+0000","last_became_peered":"2024-09-17T21:21:40.318488+0000","last_unstale":"2024-09-17T21:22:51.451462+0000","last_undegraded":"2024-09-17T21:22:51.451462+0000","last_fullsized":"2024-09-17T21:22:51.451462+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:57:56.726468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036709440000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14882,"num_read_kb":61123,"num_write":581,"num_write_kb":23555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":49089084,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":474,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.385630+0000","last_change":"2024-09-17T21:18:03.073440+0000","last_active":"2024-09-17T21:22:51.385630+0000","last_peered":"2024-09-17T21:22:51.385630+0000","last_clean":"2024-09-17T21:22:51.385630+0000","last_became_active":"2024-09-17T21:18:01.048806+0000","last_became_peered":"2024-09-17T21:18:01.048806+0000","last_unstale":"2024-09-17T21:22:51.385630+0000","last_undegraded":"2024-09-17T21:22:51.385630+0000","last_fullsized":"2024-09-17T21:22:51.385630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:26.487163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094161400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"226'352","reported_seq":5610,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382685+0000","last_change":"2024-09-17T21:22:50.390979+0000","last_active":"2024-09-17T21:22:51.382685+0000","last_peered":"2024-09-17T21:22:51.382685+0000","last_clean":"2024-09-17T21:22:51.382685+0000","last_became_active":"2024-09-17T21:22:50.388965+0000","last_became_peered":"2024-09-17T21:22:50.388965+0000","last_unstale":"2024-09-17T21:22:51.382685+0000","last_undegraded":"2024-09-17T21:22:51.382685+0000","last_fullsized":"2024-09-17T21:22:51.382685+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:18:04.741617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079556969999999994,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4656,"num_read_kb":36280,"num_write":392,"num_write_kb":13876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":5244262,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":498,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384423+0000","last_change":"2024-09-17T21:21:35.319221+0000","last_active":"2024-09-17T21:22:51.384423+0000","last_peered":"2024-09-17T21:22:51.384423+0000","last_clean":"2024-09-17T21:22:51.384423+0000","last_became_active":"2024-09-17T21:21:35.312507+0000","last_became_peered":"2024-09-17T21:21:35.312507+0000","last_unstale":"2024-09-17T21:22:51.384423+0000","last_undegraded":"2024-09-17T21:22:51.384423+0000","last_fullsized":"2024-09-17T21:22:51.384423+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:53.042567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099219900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"226'707","reported_seq":16135,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.385480+0000","last_change":"2024-09-17T21:22:46.361219+0000","last_active":"2024-09-17T21:22:51.385480+0000","last_peered":"2024-09-17T21:22:51.385480+0000","last_clean":"2024-09-17T21:22:51.385480+0000","last_became_active":"2024-09-17T21:22:46.359525+0000","last_became_peered":"2024-09-17T21:22:46.359525+0000","last_unstale":"2024-09-17T21:22:51.385480+0000","last_undegraded":"2024-09-17T21:22:51.385480+0000","last_fullsized":"2024-09-17T21:22:51.385480+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:47.174816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052152359999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13338,"num_read_kb":29382,"num_write":606,"num_write_kb":10802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":367,"num_bytes_recovered":21385444,"num_keys_recovered":118,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":498,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.385587+0000","last_change":"2024-09-17T21:21:35.326814+0000","last_active":"2024-09-17T21:22:51.385587+0000","last_peered":"2024-09-17T21:22:51.385587+0000","last_clean":"2024-09-17T21:22:51.385587+0000","last_became_active":"2024-09-17T21:21:35.325777+0000","last_became_peered":"2024-09-17T21:21:35.325777+0000","last_unstale":"2024-09-17T21:22:51.385587+0000","last_undegraded":"2024-09-17T21:22:51.385587+0000","last_fullsized":"2024-09-17T21:22:51.385587+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:47:00.558900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006222,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"225'1056","reported_seq":33580,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450393+0000","last_change":"2024-09-17T21:22:44.360752+0000","last_active":"2024-09-17T21:22:51.450393+0000","last_peered":"2024-09-17T21:22:51.450393+0000","last_clean":"2024-09-17T21:22:51.450393+0000","last_became_active":"2024-09-17T21:22:44.352239+0000","last_became_peered":"2024-09-17T21:22:44.352239+0000","last_unstale":"2024-09-17T21:22:51.450393+0000","last_undegraded":"2024-09-17T21:22:51.450393+0000","last_fullsized":"2024-09-17T21:22:51.450393+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":1056,"log_dups_size":0,"ondisk_log_size":1056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:36:58.893607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052118770000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":37177,"num_read_kb":61903,"num_write":871,"num_write_kb":21852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":754,"num_bytes_recovered":58900004,"num_keys_recovered":204,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"227'536","reported_seq":3887,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.388246+0000","last_change":"2024-09-17T21:22:24.383832+0000","last_active":"2024-09-17T21:22:51.388246+0000","last_peered":"2024-09-17T21:22:51.388246+0000","last_clean":"2024-09-17T21:22:51.388246+0000","last_became_active":"2024-09-17T21:21:39.315142+0000","last_became_peered":"2024-09-17T21:21:39.315142+0000","last_unstale":"2024-09-17T21:22:51.388246+0000","last_undegraded":"2024-09-17T21:22:51.388246+0000","last_fullsized":"2024-09-17T21:22:51.388246+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:05:28.580363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061334989999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":3053,"num_read_kb":14423,"num_write":502,"num_write_kb":4818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":7199060,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":412,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450196+0000","last_change":"2024-09-17T21:22:48.383726+0000","last_active":"2024-09-17T21:22:51.450196+0000","last_peered":"2024-09-17T21:22:51.450196+0000","last_clean":"2024-09-17T21:22:51.450196+0000","last_became_active":"2024-09-17T21:22:48.382842+0000","last_became_peered":"2024-09-17T21:22:48.382842+0000","last_unstale":"2024-09-17T21:22:51.450196+0000","last_undegraded":"2024-09-17T21:22:51.450196+0000","last_fullsized":"2024-09-17T21:22:51.450196+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:30:55.312809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"226'248","reported_seq":9446,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384708+0000","last_change":"2024-09-17T21:22:45.354353+0000","last_active":"2024-09-17T21:22:51.384708+0000","last_peered":"2024-09-17T21:22:51.384708+0000","last_clean":"2024-09-17T21:22:51.384708+0000","last_became_active":"2024-09-17T21:22:45.352682+0000","last_became_peered":"2024-09-17T21:22:45.352682+0000","last_unstale":"2024-09-17T21:22:51.384708+0000","last_undegraded":"2024-09-17T21:22:51.384708+0000","last_fullsized":"2024-09-17T21:22:51.384708+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:18.942784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048483459999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5340,"num_read_kb":22605,"num_write":215,"num_write_kb":6424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":15742651,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":461,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382421+0000","last_change":"2024-09-17T21:22:50.389225+0000","last_active":"2024-09-17T21:22:51.382421+0000","last_peered":"2024-09-17T21:22:51.382421+0000","last_clean":"2024-09-17T21:22:51.382421+0000","last_became_active":"2024-09-17T21:22:50.388953+0000","last_became_peered":"2024-09-17T21:22:50.388953+0000","last_unstale":"2024-09-17T21:22:51.382421+0000","last_undegraded":"2024-09-17T21:22:51.382421+0000","last_fullsized":"2024-09-17T21:22:51.382421+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:21.062518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"227'1041","reported_seq":8867,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450992+0000","last_change":"2024-09-17T21:22:50.391990+0000","last_active":"2024-09-17T21:22:51.450992+0000","last_peered":"2024-09-17T21:22:51.450992+0000","last_clean":"2024-09-17T21:22:51.450992+0000","last_became_active":"2024-09-17T21:22:50.390218+0000","last_became_peered":"2024-09-17T21:22:50.390218+0000","last_unstale":"2024-09-17T21:22:51.450992+0000","last_undegraded":"2024-09-17T21:22:51.450992+0000","last_fullsized":"2024-09-17T21:22:51.450992+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":1041,"log_dups_size":0,"ondisk_log_size":1041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:24:31.427407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035942119999999998,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":9666,"num_read_kb":63102,"num_write":1578,"num_write_kb":18132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":894,"num_bytes_recovered":48903910,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":474,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.383504+0000","last_change":"2024-09-17T21:18:03.059156+0000","last_active":"2024-09-17T21:22:51.383504+0000","last_peered":"2024-09-17T21:22:51.383504+0000","last_clean":"2024-09-17T21:22:51.383504+0000","last_became_active":"2024-09-17T21:18:01.049180+0000","last_became_peered":"2024-09-17T21:18:01.049180+0000","last_unstale":"2024-09-17T21:22:51.383504+0000","last_undegraded":"2024-09-17T21:22:51.383504+0000","last_fullsized":"2024-09-17T21:22:51.383504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:27.069794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077408,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"226'656","reported_seq":14594,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384939+0000","last_change":"2024-09-17T21:22:41.354650+0000","last_active":"2024-09-17T21:22:51.384939+0000","last_peered":"2024-09-17T21:22:51.384939+0000","last_clean":"2024-09-17T21:22:51.384939+0000","last_became_active":"2024-09-17T21:22:41.352429+0000","last_became_peered":"2024-09-17T21:22:41.352429+0000","last_unstale":"2024-09-17T21:22:51.384939+0000","last_undegraded":"2024-09-17T21:22:51.384939+0000","last_fullsized":"2024-09-17T21:22:51.384939+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:30:46.965276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067184260000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14741,"num_read_kb":38983,"num_write":745,"num_write_kb":20177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":27258987,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":19,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450266+0000","last_change":"2024-09-17T21:22:48.387337+0000","last_active":"2024-09-17T21:22:51.450266+0000","last_peered":"2024-09-17T21:22:51.450266+0000","last_clean":"2024-09-17T21:22:51.450266+0000","last_became_active":"2024-09-17T21:22:48.386572+0000","last_became_peered":"2024-09-17T21:22:48.386572+0000","last_unstale":"2024-09-17T21:22:51.450266+0000","last_undegraded":"2024-09-17T21:22:51.450266+0000","last_fullsized":"2024-09-17T21:22:51.450266+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:55:57.934354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":189,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.387986+0000","last_change":"2024-09-17T21:21:17.347819+0000","last_active":"2024-09-17T21:22:51.387986+0000","last_peered":"2024-09-17T21:22:51.387986+0000","last_clean":"2024-09-17T21:22:51.387986+0000","last_became_active":"2024-09-17T21:21:17.337621+0000","last_became_peered":"2024-09-17T21:21:17.337621+0000","last_unstale":"2024-09-17T21:22:51.387986+0000","last_undegraded":"2024-09-17T21:22:51.387986+0000","last_fullsized":"2024-09-17T21:22:51.387986+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_clean_scrub_stamp":"2024-09-17T21:17:55.982339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:24:32.371870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"224'616","reported_seq":14093,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382313+0000","last_change":"2024-09-17T21:22:50.383917+0000","last_active":"2024-09-17T21:22:51.382313+0000","last_peered":"2024-09-17T21:22:51.382313+0000","last_clean":"2024-09-17T21:22:51.382313+0000","last_became_active":"2024-09-17T21:22:50.382396+0000","last_became_peered":"2024-09-17T21:22:50.382396+0000","last_unstale":"2024-09-17T21:22:51.382313+0000","last_undegraded":"2024-09-17T21:22:51.382313+0000","last_fullsized":"2024-09-17T21:22:51.382313+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:13:13.848853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001602121,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":13993,"num_read_kb":68437,"num_write":586,"num_write_kb":29114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":40776135,"num_keys_recovered":165,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"227'699","reported_seq":11106,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.388697+0000","last_change":"2024-09-17T21:22:49.395344+0000","last_active":"2024-09-17T21:22:51.388697+0000","last_peered":"2024-09-17T21:22:51.388697+0000","last_clean":"2024-09-17T21:22:51.388697+0000","last_became_active":"2024-09-17T21:22:49.390523+0000","last_became_peered":"2024-09-17T21:22:49.390523+0000","last_unstale":"2024-09-17T21:22:51.388697+0000","last_undegraded":"2024-09-17T21:22:51.388697+0000","last_fullsized":"2024-09-17T21:22:51.388697+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:55:28.950314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029497360000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":11677,"num_read_kb":43614,"num_write":1089,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":728,"num_bytes_recovered":30193805,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"227'428","reported_seq":8266,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.383725+0000","last_change":"2024-09-17T21:22:49.397589+0000","last_active":"2024-09-17T21:22:51.383725+0000","last_peered":"2024-09-17T21:22:51.383725+0000","last_clean":"2024-09-17T21:22:51.383725+0000","last_became_active":"2024-09-17T21:22:49.394384+0000","last_became_peered":"2024-09-17T21:22:49.394384+0000","last_unstale":"2024-09-17T21:22:51.383725+0000","last_undegraded":"2024-09-17T21:22:51.383725+0000","last_fullsized":"2024-09-17T21:22:51.383725+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:24:21.885168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045615830000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7186,"num_read_kb":40870,"num_write":369,"num_write_kb":16675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":179794,"num_keys_recovered":2,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"226'376","reported_seq":4920,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384873+0000","last_change":"2024-09-17T21:22:24.375850+0000","last_active":"2024-09-17T21:22:51.384873+0000","last_peered":"2024-09-17T21:22:51.384873+0000","last_clean":"2024-09-17T21:22:51.384873+0000","last_became_active":"2024-09-17T21:21:40.319613+0000","last_became_peered":"2024-09-17T21:21:40.319613+0000","last_unstale":"2024-09-17T21:22:51.384873+0000","last_undegraded":"2024-09-17T21:22:51.384873+0000","last_fullsized":"2024-09-17T21:22:51.384873+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:26:53.101370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038796239999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4694,"num_read_kb":23619,"num_write":320,"num_write_kb":13765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":4886965,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"227'452","reported_seq":13421,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.385236+0000","last_change":"2024-09-17T21:22:43.339607+0000","last_active":"2024-09-17T21:22:51.385236+0000","last_peered":"2024-09-17T21:22:51.385236+0000","last_clean":"2024-09-17T21:22:51.385236+0000","last_became_active":"2024-09-17T21:22:43.338399+0000","last_became_peered":"2024-09-17T21:22:43.338399+0000","last_unstale":"2024-09-17T21:22:51.385236+0000","last_undegraded":"2024-09-17T21:22:51.385236+0000","last_fullsized":"2024-09-17T21:22:51.385236+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:30:14.210043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050460430000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":15379,"num_read_kb":29065,"num_write":580,"num_write_kb":14319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":5960704,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"221'526","reported_seq":15419,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384216+0000","last_change":"2024-09-17T21:22:46.356687+0000","last_active":"2024-09-17T21:22:51.384216+0000","last_peered":"2024-09-17T21:22:51.384216+0000","last_clean":"2024-09-17T21:22:51.384216+0000","last_became_active":"2024-09-17T21:22:46.349835+0000","last_became_peered":"2024-09-17T21:22:46.349835+0000","last_unstale":"2024-09-17T21:22:51.384216+0000","last_undegraded":"2024-09-17T21:22:51.384216+0000","last_fullsized":"2024-09-17T21:22:51.384216+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:51:30.571949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00482724,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15785,"num_read_kb":43220,"num_write":654,"num_write_kb":16401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":28397872,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"227'623","reported_seq":14444,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450685+0000","last_change":"2024-09-17T21:22:49.806087+0000","last_active":"2024-09-17T21:22:51.450685+0000","last_peered":"2024-09-17T21:22:51.450685+0000","last_clean":"2024-09-17T21:22:51.450685+0000","last_became_active":"2024-09-17T21:22:49.394154+0000","last_became_peered":"2024-09-17T21:22:49.394154+0000","last_unstale":"2024-09-17T21:22:51.450685+0000","last_undegraded":"2024-09-17T21:22:51.450685+0000","last_fullsized":"2024-09-17T21:22:51.450685+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:24:57.185890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057317679999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":15141,"num_read_kb":70274,"num_write":535,"num_write_kb":35852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":91925553,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"226'464","reported_seq":7123,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450740+0000","last_change":"2024-09-17T21:22:48.398780+0000","last_active":"2024-09-17T21:22:51.450740+0000","last_peered":"2024-09-17T21:22:51.450740+0000","last_clean":"2024-09-17T21:22:51.450740+0000","last_became_active":"2024-09-17T21:22:48.392669+0000","last_became_peered":"2024-09-17T21:22:48.392669+0000","last_unstale":"2024-09-17T21:22:51.450740+0000","last_undegraded":"2024-09-17T21:22:51.450740+0000","last_fullsized":"2024-09-17T21:22:51.450740+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:18:23.844174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0091147729999999996,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7427,"num_read_kb":28404,"num_write":434,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":4659071,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"222'271","reported_seq":7403,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450132+0000","last_change":"2024-09-17T21:22:48.384974+0000","last_active":"2024-09-17T21:22:51.450132+0000","last_peered":"2024-09-17T21:22:51.450132+0000","last_clean":"2024-09-17T21:22:51.450132+0000","last_became_active":"2024-09-17T21:22:48.381106+0000","last_became_peered":"2024-09-17T21:22:48.381106+0000","last_unstale":"2024-09-17T21:22:51.450132+0000","last_undegraded":"2024-09-17T21:22:51.450132+0000","last_fullsized":"2024-09-17T21:22:51.450132+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:11:47.017985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017482940000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7827,"num_read_kb":19883,"num_write":219,"num_write_kb":3298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":6733181,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"226'404","reported_seq":5779,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384016+0000","last_change":"2024-09-17T21:22:24.385309+0000","last_active":"2024-09-17T21:22:51.384016+0000","last_peered":"2024-09-17T21:22:51.384016+0000","last_clean":"2024-09-17T21:22:51.384016+0000","last_became_active":"2024-09-17T21:18:11.299946+0000","last_became_peered":"2024-09-17T21:18:11.299946+0000","last_unstale":"2024-09-17T21:22:51.384016+0000","last_undegraded":"2024-09-17T21:22:51.384016+0000","last_fullsized":"2024-09-17T21:22:51.384016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:16:16.699933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015128413,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5168,"num_read_kb":22085,"num_write":382,"num_write_kb":4452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"225'570","reported_seq":15995,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450984+0000","last_change":"2024-09-17T21:22:24.368740+0000","last_active":"2024-09-17T21:22:51.450984+0000","last_peered":"2024-09-17T21:22:51.450984+0000","last_clean":"2024-09-17T21:22:51.450984+0000","last_became_active":"2024-09-17T21:22:09.609844+0000","last_became_peered":"2024-09-17T21:22:09.609844+0000","last_unstale":"2024-09-17T21:22:51.450984+0000","last_undegraded":"2024-09-17T21:22:51.450984+0000","last_fullsized":"2024-09-17T21:22:51.450984+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:03:30.364818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018557070000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":17911,"num_read_kb":38759,"num_write":589,"num_write_kb":6968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":11589682,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"226'651","reported_seq":12606,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384243+0000","last_change":"2024-09-17T21:22:41.357868+0000","last_active":"2024-09-17T21:22:51.384243+0000","last_peered":"2024-09-17T21:22:51.384243+0000","last_clean":"2024-09-17T21:22:51.384243+0000","last_became_active":"2024-09-17T21:22:41.354851+0000","last_became_peered":"2024-09-17T21:22:41.354851+0000","last_unstale":"2024-09-17T21:22:51.384243+0000","last_undegraded":"2024-09-17T21:22:51.384243+0000","last_fullsized":"2024-09-17T21:22:51.384243+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:47:04.462955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011937984,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12008,"num_read_kb":52136,"num_write":617,"num_write_kb":5443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":5557098,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":304,"num_object_clones":0,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":300,"num_read":353905,"num_read_kb":1145909,"num_write":19949,"num_write_kb":385384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11936,"num_bytes_recovered":614462337,"num_keys_recovered":1529,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8194,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60472,"kb_used_data":2908,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311368,"statfs":{"total":96636764160,"available":96574840832,"internally_reserved":0,"allocated":2977792,"data_stored":1262263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":58915268},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66616,"kb_used_data":3488,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305224,"statfs":{"total":96636764160,"available":96568549376,"internally_reserved":0,"allocated":3571712,"data_stored":1852631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":64616900},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83256,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288584,"statfs":{"total":96636764160,"available":96551510016,"internally_reserved":0,"allocated":3567616,"data_stored":1852612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":81656262},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56308,"kb_used_data":2912,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315532,"statfs":{"total":96636764160,"available":96579104768,"internally_reserved":0,"allocated":2981888,"data_stored":1262282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56300,"kb_used_data":2904,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315540,"statfs":{"total":96636764160,"available":96579112960,"internally_reserved":0,"allocated":2973696,"data_stored":1262244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":231,"seq":992137445383,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30644,"kb_used_data":2904,"kb_used_omap":20,"kb_used_meta":27691,"kb_avail":94341196,"statfs":{"total":96636764160,"available":96605384704,"internally_reserved":0,"allocated":2973696,"data_stored":1262244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20966,"internal_metadata":28356122},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8194,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:22:59.659 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T21:22:59.945 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:22:59.945 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":247,"fsid":"224a7a29-e112-472e-8a13-f67345d6d123","created":"2024-09-17T21:17:42.597312+0000","modified":"2024-09-17T21:22:51.374389+0000","last_up_change":"2024-09-17T21:22:32.395031+0000","last_in_change":"2024-09-17T21:22:37.312336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T21:17:55.581350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T21:17:59.603025+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-17T21:18:09.441137+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"227","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":227,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5bfd41c9-0cac-47c4-9a6e-8dce774c3314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6817","nonce":1111036754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6819","nonce":1111036754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6823","nonce":1111036754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1111036754},{"type":"v1","addr":"172.21.15.83:6821","nonce":1111036754}]},"public_addr":"172.21.15.83:6817/1111036754","cluster_addr":"172.21.15.83:6819/1111036754","heartbeat_back_addr":"172.21.15.83:6823/1111036754","heartbeat_front_addr":"172.21.15.83:6821/1111036754","state":["exists","up"]},{"osd":1,"uuid":"78404e0d-d397-4806-8e90-5f32182643bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":246,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6809","nonce":1460363937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6811","nonce":1460363937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6815","nonce":1460363937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":1460363937},{"type":"v1","addr":"172.21.15.83:6813","nonce":1460363937}]},"public_addr":"172.21.15.83:6809/1460363937","cluster_addr":"172.21.15.83:6811/1460363937","heartbeat_back_addr":"172.21.15.83:6815/1460363937","heartbeat_front_addr":"172.21.15.83:6813/1460363937","state":["exists","up"]},{"osd":2,"uuid":"286ed6e0-c121-4ebb-8cf6-37e506a397aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":12,"last_clean_end":188,"up_from":231,"up_thru":245,"down_at":189,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3213289387},{"type":"v1","addr":"172.21.15.83:6801","nonce":3213289387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3213289387},{"type":"v1","addr":"172.21.15.83:6803","nonce":3213289387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3213289387},{"type":"v1","addr":"172.21.15.83:6807","nonce":3213289387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3213289387},{"type":"v1","addr":"172.21.15.83:6805","nonce":3213289387}]},"public_addr":"172.21.15.83:6801/3213289387","cluster_addr":"172.21.15.83:6803/3213289387","heartbeat_back_addr":"172.21.15.83:6807/3213289387","heartbeat_front_addr":"172.21.15.83:6805/3213289387","state":["exists","up"]},{"osd":3,"uuid":"b0af73a1-4bca-4892-a550-2ced87f4c357","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6809","nonce":3637204865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6811","nonce":3637204865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6815","nonce":3637204865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3637204865},{"type":"v1","addr":"172.21.15.97:6813","nonce":3637204865}]},"public_addr":"172.21.15.97:6809/3637204865","cluster_addr":"172.21.15.97:6811/3637204865","heartbeat_back_addr":"172.21.15.97:6815/3637204865","heartbeat_front_addr":"172.21.15.97:6813/3637204865","state":["exists","up"]},{"osd":4,"uuid":"2188361c-e1a1-45fa-bc01-da5a984ffd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6801","nonce":2391356911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6803","nonce":2391356911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6807","nonce":2391356911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2391356911},{"type":"v1","addr":"172.21.15.97:6805","nonce":2391356911}]},"public_addr":"172.21.15.97:6801/2391356911","cluster_addr":"172.21.15.97:6803/2391356911","heartbeat_back_addr":"172.21.15.97:6807/2391356911","heartbeat_front_addr":"172.21.15.97:6805/2391356911","state":["exists","up"]},{"osd":5,"uuid":"7feccf4d-d8e4-4d06-ac83-bbc1bc623a5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6817","nonce":2326316381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6819","nonce":2326316381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6823","nonce":2326316381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2326316381},{"type":"v1","addr":"172.21.15.97:6821","nonce":2326316381}]},"public_addr":"172.21.15.97:6817/2326316381","cluster_addr":"172.21.15.97:6819/2326316381","heartbeat_back_addr":"172.21.15.97:6823/2326316381","heartbeat_front_addr":"172.21.15.97:6821/2326316381","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:51.138101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.581632+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.365193+0000","dead_epoch":189},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.718974+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:50.501878+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:17:51.150747+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"1.0","osds":[4,3]},{"pgid":"3.8","osds":[2,3]},{"pgid":"3.c","osds":[1,5]},{"pgid":"3.1e","osds":[0,4]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:23:00.963 INFO:tasks.ceph:Scrubbing osd.0 2024-09-17T21:23:00.964 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-17T21:23:01.124 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:23:01.124 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T21:23:01.124 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:23:01.140 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-17T21:23:01.419 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 0 to deep-scrub 2024-09-17T21:23:01.437 INFO:tasks.ceph:Scrubbing osd.1 2024-09-17T21:23:01.437 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-17T21:23:01.583 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:23:01.583 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T21:23:01.584 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:23:01.599 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-17T21:23:01.882 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 1 to deep-scrub 2024-09-17T21:23:01.900 INFO:tasks.ceph:Scrubbing osd.2 2024-09-17T21:23:01.900 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-17T21:23:02.041 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:23:02.041 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-17T21:23:02.041 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:23:02.056 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-17T21:23:02.333 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 2 to deep-scrub 2024-09-17T21:23:02.350 INFO:tasks.ceph:Scrubbing osd.3 2024-09-17T21:23:02.351 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-17T21:23:02.485 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:23:02.485 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T21:23:02.485 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:23:02.500 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-17T21:23:02.792 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 3 to deep-scrub 2024-09-17T21:23:02.810 INFO:tasks.ceph:Scrubbing osd.4 2024-09-17T21:23:02.810 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-17T21:23:02.959 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:23:02.959 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T21:23:02.959 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:23:02.975 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-17T21:23:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 4 to deep-scrub 2024-09-17T21:23:03.296 INFO:tasks.ceph:Scrubbing osd.5 2024-09-17T21:23:03.296 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-17T21:23:03.447 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T21:23:03.447 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T21:23:03.447 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T21:23:03.464 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-17T21:23:03.758 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 5 to deep-scrub 2024-09-17T21:23:03.775 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:23:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:23:04.078 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T21:23:04.095 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-09-17T21:23:03.615988+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":308,"num_object_clones":0,"num_object_copies":616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":300,"num_read":353971,"num_read_kb":1145964,"num_write":20052,"num_write_kb":387014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11949,"num_bytes_recovered":615643130,"num_keys_recovered":1529,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18605,"ondisk_log_size":18605,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":353596,"kb_used_data":18600,"kb_used_omap":28,"kb_used_meta":334819,"kb_avail":565877444,"statfs":{"total":579820584960,"available":579458502656,"internally_reserved":0,"allocated":19046400,"data_stored":8754276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28944,"internal_metadata":342855408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003545"},"pg_stats":[{"pgid":"3.1f","version":"227'759","reported_seq":4118,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451328+0000","last_change":"2024-09-17T21:22:40.360129+0000","last_active":"2024-09-17T21:22:51.451328+0000","last_peered":"2024-09-17T21:22:51.451328+0000","last_clean":"2024-09-17T21:22:51.451328+0000","last_became_active":"2024-09-17T21:22:40.357329+0000","last_became_peered":"2024-09-17T21:22:40.357329+0000","last_unstale":"2024-09-17T21:22:51.451328+0000","last_undegraded":"2024-09-17T21:22:51.451328+0000","last_fullsized":"2024-09-17T21:22:51.451328+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:01:10.890769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010221002999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5921,"num_read_kb":16779,"num_write":1026,"num_write_kb":3495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":5143732,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"227'851","reported_seq":9557,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451353+0000","last_change":"2024-09-17T21:22:40.355216+0000","last_active":"2024-09-17T21:22:51.451353+0000","last_peered":"2024-09-17T21:22:51.451353+0000","last_clean":"2024-09-17T21:22:51.451353+0000","last_became_active":"2024-09-17T21:22:40.353016+0000","last_became_peered":"2024-09-17T21:22:40.353016+0000","last_unstale":"2024-09-17T21:22:51.451353+0000","last_undegraded":"2024-09-17T21:22:51.451353+0000","last_fullsized":"2024-09-17T21:22:51.451353+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":851,"log_dups_size":0,"ondisk_log_size":851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:11.494395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084510550000000007,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10998,"num_read_kb":53173,"num_write":1081,"num_write_kb":18708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":50670363,"num_keys_recovered":90,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"227'833","reported_seq":8688,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450570+0000","last_change":"2024-09-17T21:22:48.408191+0000","last_active":"2024-09-17T21:22:51.450570+0000","last_peered":"2024-09-17T21:22:51.450570+0000","last_clean":"2024-09-17T21:22:51.450570+0000","last_became_active":"2024-09-17T21:22:48.401326+0000","last_became_peered":"2024-09-17T21:22:48.401326+0000","last_unstale":"2024-09-17T21:22:51.450570+0000","last_undegraded":"2024-09-17T21:22:51.450570+0000","last_fullsized":"2024-09-17T21:22:51.450570+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:19:25.050335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082452470000000007,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":10230,"num_read_kb":27857,"num_write":978,"num_write_kb":10466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":12493301,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"227'376","reported_seq":4046,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450419+0000","last_change":"2024-09-17T21:22:48.406130+0000","last_active":"2024-09-17T21:22:51.450419+0000","last_peered":"2024-09-17T21:22:51.450419+0000","last_clean":"2024-09-17T21:22:51.450419+0000","last_became_active":"2024-09-17T21:22:48.401237+0000","last_became_peered":"2024-09-17T21:22:48.401237+0000","last_unstale":"2024-09-17T21:22:51.450419+0000","last_undegraded":"2024-09-17T21:22:51.450419+0000","last_fullsized":"2024-09-17T21:22:51.450419+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:39:30.285487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061565819999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":2726,"num_read_kb":23738,"num_write":383,"num_write_kb":5937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":9397808,"num_keys_recovered":202,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"227'710","reported_seq":8291,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451267+0000","last_change":"2024-09-17T21:22:24.369716+0000","last_active":"2024-09-17T21:22:51.451267+0000","last_peered":"2024-09-17T21:22:51.451267+0000","last_clean":"2024-09-17T21:22:51.451267+0000","last_became_active":"2024-09-17T21:21:41.329830+0000","last_became_peered":"2024-09-17T21:21:41.329830+0000","last_unstale":"2024-09-17T21:22:51.451267+0000","last_undegraded":"2024-09-17T21:22:51.451267+0000","last_fullsized":"2024-09-17T21:22:51.451267+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:42:50.772772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026038680000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":11634,"num_read_kb":21103,"num_write":555,"num_write_kb":4725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":7872032,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"226'739","reported_seq":13918,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.388284+0000","last_change":"2024-09-17T21:22:50.389766+0000","last_active":"2024-09-17T21:22:51.388284+0000","last_peered":"2024-09-17T21:22:51.388284+0000","last_clean":"2024-09-17T21:22:51.388284+0000","last_became_active":"2024-09-17T21:22:50.386623+0000","last_became_peered":"2024-09-17T21:22:50.386623+0000","last_unstale":"2024-09-17T21:22:51.388284+0000","last_undegraded":"2024-09-17T21:22:51.388284+0000","last_fullsized":"2024-09-17T21:22:51.388284+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:41:15.598618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056973060000000001,"stat_sum":{"num_bytes":19,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":16893,"num_read_kb":53539,"num_write":669,"num_write_kb":14565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":647737,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"226'476","reported_seq":12478,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.393411+0000","last_change":"2024-09-17T21:22:51.393063+0000","last_active":"2024-09-17T21:22:51.393411+0000","last_peered":"2024-09-17T21:22:51.393411+0000","last_clean":"2024-09-17T21:22:51.393411+0000","last_became_active":"2024-09-17T21:22:51.390578+0000","last_became_peered":"2024-09-17T21:22:51.390578+0000","last_unstale":"2024-09-17T21:22:51.393411+0000","last_undegraded":"2024-09-17T21:22:51.393411+0000","last_fullsized":"2024-09-17T21:22:51.393411+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:08:03.346817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0077005709999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":12648,"num_read_kb":27025,"num_write":374,"num_write_kb":5615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":3100207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"226'318","reported_seq":3035,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382486+0000","last_change":"2024-09-17T21:22:50.387935+0000","last_active":"2024-09-17T21:22:51.382486+0000","last_peered":"2024-09-17T21:22:51.382486+0000","last_clean":"2024-09-17T21:22:51.382486+0000","last_became_active":"2024-09-17T21:22:50.385975+0000","last_became_peered":"2024-09-17T21:22:50.385975+0000","last_unstale":"2024-09-17T21:22:51.382486+0000","last_undegraded":"2024-09-17T21:22:51.382486+0000","last_fullsized":"2024-09-17T21:22:51.382486+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:26:13.518654+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022643899999999998,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2392,"num_read_kb":9139,"num_write":304,"num_write_kb":2898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":9226431,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"225'393","reported_seq":7343,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451410+0000","last_change":"2024-09-17T21:22:43.342179+0000","last_active":"2024-09-17T21:22:51.451410+0000","last_peered":"2024-09-17T21:22:51.451410+0000","last_clean":"2024-09-17T21:22:51.451410+0000","last_became_active":"2024-09-17T21:22:43.340739+0000","last_became_peered":"2024-09-17T21:22:43.340739+0000","last_unstale":"2024-09-17T21:22:51.451410+0000","last_undegraded":"2024-09-17T21:22:51.451410+0000","last_fullsized":"2024-09-17T21:22:51.451410+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:32:48.751230+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00138835,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7054,"num_read_kb":21313,"num_write":475,"num_write_kb":6946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":719,"num_bytes_recovered":31066012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"224'602","reported_seq":22252,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450629+0000","last_change":"2024-09-17T21:22:48.400263+0000","last_active":"2024-09-17T21:22:51.450629+0000","last_peered":"2024-09-17T21:22:51.450629+0000","last_clean":"2024-09-17T21:22:51.450629+0000","last_became_active":"2024-09-17T21:22:48.394816+0000","last_became_peered":"2024-09-17T21:22:48.394816+0000","last_unstale":"2024-09-17T21:22:51.450629+0000","last_undegraded":"2024-09-17T21:22:51.450629+0000","last_fullsized":"2024-09-17T21:22:51.450629+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:05.075504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075408430000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":22417,"num_read_kb":27659,"num_write":858,"num_write_kb":13022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":3432016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"223'754","reported_seq":7158,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384191+0000","last_change":"2024-09-17T21:22:24.378500+0000","last_active":"2024-09-17T21:22:51.384191+0000","last_peered":"2024-09-17T21:22:51.384191+0000","last_clean":"2024-09-17T21:22:51.384191+0000","last_became_active":"2024-09-17T21:21:40.321648+0000","last_became_peered":"2024-09-17T21:21:40.321648+0000","last_unstale":"2024-09-17T21:22:51.384191+0000","last_undegraded":"2024-09-17T21:22:51.384191+0000","last_fullsized":"2024-09-17T21:22:51.384191+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:54:37.691837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0071138240000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8592,"num_read_kb":29445,"num_write":954,"num_write_kb":5554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":9885558,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"225'449","reported_seq":5376,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.383815+0000","last_change":"2024-09-17T21:22:49.393985+0000","last_active":"2024-09-17T21:22:51.383815+0000","last_peered":"2024-09-17T21:22:51.383815+0000","last_clean":"2024-09-17T21:22:51.383815+0000","last_became_active":"2024-09-17T21:22:49.389098+0000","last_became_peered":"2024-09-17T21:22:49.389098+0000","last_unstale":"2024-09-17T21:22:51.383815+0000","last_undegraded":"2024-09-17T21:22:51.383815+0000","last_fullsized":"2024-09-17T21:22:51.383815+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:28:53.422353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003029,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5351,"num_read_kb":26972,"num_write":428,"num_write_kb":16010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":385,"num_bytes_recovered":6943878,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":474,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384273+0000","last_change":"2024-09-17T21:18:03.059715+0000","last_active":"2024-09-17T21:22:51.384273+0000","last_peered":"2024-09-17T21:22:51.384273+0000","last_clean":"2024-09-17T21:22:51.384273+0000","last_became_active":"2024-09-17T21:18:01.213047+0000","last_became_peered":"2024-09-17T21:18:01.213047+0000","last_unstale":"2024-09-17T21:22:51.384273+0000","last_undegraded":"2024-09-17T21:22:51.384273+0000","last_fullsized":"2024-09-17T21:22:51.384273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:46:35.222373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"227'594","reported_seq":13694,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.451462+0000","last_change":"2024-09-17T21:22:24.371187+0000","last_active":"2024-09-17T21:22:51.451462+0000","last_peered":"2024-09-17T21:22:51.451462+0000","last_clean":"2024-09-17T21:22:51.451462+0000","last_became_active":"2024-09-17T21:21:40.318488+0000","last_became_peered":"2024-09-17T21:21:40.318488+0000","last_unstale":"2024-09-17T21:22:51.451462+0000","last_undegraded":"2024-09-17T21:22:51.451462+0000","last_fullsized":"2024-09-17T21:22:51.451462+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:57:56.726468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036709440000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14882,"num_read_kb":61123,"num_write":581,"num_write_kb":23555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":49089084,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":474,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.385630+0000","last_change":"2024-09-17T21:18:03.073440+0000","last_active":"2024-09-17T21:22:51.385630+0000","last_peered":"2024-09-17T21:22:51.385630+0000","last_clean":"2024-09-17T21:22:51.385630+0000","last_became_active":"2024-09-17T21:18:01.048806+0000","last_became_peered":"2024-09-17T21:18:01.048806+0000","last_unstale":"2024-09-17T21:22:51.385630+0000","last_undegraded":"2024-09-17T21:22:51.385630+0000","last_fullsized":"2024-09-17T21:22:51.385630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:26.487163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094161400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"226'352","reported_seq":5610,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382685+0000","last_change":"2024-09-17T21:22:50.390979+0000","last_active":"2024-09-17T21:22:51.382685+0000","last_peered":"2024-09-17T21:22:51.382685+0000","last_clean":"2024-09-17T21:22:51.382685+0000","last_became_active":"2024-09-17T21:22:50.388965+0000","last_became_peered":"2024-09-17T21:22:50.388965+0000","last_unstale":"2024-09-17T21:22:51.382685+0000","last_undegraded":"2024-09-17T21:22:51.382685+0000","last_fullsized":"2024-09-17T21:22:51.382685+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:18:04.741617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079556969999999994,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4656,"num_read_kb":36280,"num_write":392,"num_write_kb":13876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":5244262,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":498,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384423+0000","last_change":"2024-09-17T21:21:35.319221+0000","last_active":"2024-09-17T21:22:51.384423+0000","last_peered":"2024-09-17T21:22:51.384423+0000","last_clean":"2024-09-17T21:22:51.384423+0000","last_became_active":"2024-09-17T21:21:35.312507+0000","last_became_peered":"2024-09-17T21:21:35.312507+0000","last_unstale":"2024-09-17T21:22:51.384423+0000","last_undegraded":"2024-09-17T21:22:51.384423+0000","last_fullsized":"2024-09-17T21:22:51.384423+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:53.042567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099219900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"226'707","reported_seq":16135,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.385480+0000","last_change":"2024-09-17T21:22:46.361219+0000","last_active":"2024-09-17T21:22:51.385480+0000","last_peered":"2024-09-17T21:22:51.385480+0000","last_clean":"2024-09-17T21:22:51.385480+0000","last_became_active":"2024-09-17T21:22:46.359525+0000","last_became_peered":"2024-09-17T21:22:46.359525+0000","last_unstale":"2024-09-17T21:22:51.385480+0000","last_undegraded":"2024-09-17T21:22:51.385480+0000","last_fullsized":"2024-09-17T21:22:51.385480+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:47.174816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052152359999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13338,"num_read_kb":29382,"num_write":606,"num_write_kb":10802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":367,"num_bytes_recovered":21385444,"num_keys_recovered":118,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":498,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.385587+0000","last_change":"2024-09-17T21:21:35.326814+0000","last_active":"2024-09-17T21:22:51.385587+0000","last_peered":"2024-09-17T21:22:51.385587+0000","last_clean":"2024-09-17T21:22:51.385587+0000","last_became_active":"2024-09-17T21:21:35.325777+0000","last_became_peered":"2024-09-17T21:21:35.325777+0000","last_unstale":"2024-09-17T21:22:51.385587+0000","last_undegraded":"2024-09-17T21:22:51.385587+0000","last_fullsized":"2024-09-17T21:22:51.385587+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:47:00.558900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006222,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"225'1056","reported_seq":33580,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450393+0000","last_change":"2024-09-17T21:22:44.360752+0000","last_active":"2024-09-17T21:22:51.450393+0000","last_peered":"2024-09-17T21:22:51.450393+0000","last_clean":"2024-09-17T21:22:51.450393+0000","last_became_active":"2024-09-17T21:22:44.352239+0000","last_became_peered":"2024-09-17T21:22:44.352239+0000","last_unstale":"2024-09-17T21:22:51.450393+0000","last_undegraded":"2024-09-17T21:22:51.450393+0000","last_fullsized":"2024-09-17T21:22:51.450393+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":1056,"log_dups_size":0,"ondisk_log_size":1056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:36:58.893607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052118770000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":37177,"num_read_kb":61903,"num_write":871,"num_write_kb":21852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":754,"num_bytes_recovered":58900004,"num_keys_recovered":204,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"227'536","reported_seq":3887,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.388246+0000","last_change":"2024-09-17T21:22:24.383832+0000","last_active":"2024-09-17T21:22:51.388246+0000","last_peered":"2024-09-17T21:22:51.388246+0000","last_clean":"2024-09-17T21:22:51.388246+0000","last_became_active":"2024-09-17T21:21:39.315142+0000","last_became_peered":"2024-09-17T21:21:39.315142+0000","last_unstale":"2024-09-17T21:22:51.388246+0000","last_undegraded":"2024-09-17T21:22:51.388246+0000","last_fullsized":"2024-09-17T21:22:51.388246+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:05:28.580363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061334989999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":3053,"num_read_kb":14423,"num_write":502,"num_write_kb":4818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":7199060,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":412,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450196+0000","last_change":"2024-09-17T21:22:48.383726+0000","last_active":"2024-09-17T21:22:51.450196+0000","last_peered":"2024-09-17T21:22:51.450196+0000","last_clean":"2024-09-17T21:22:51.450196+0000","last_became_active":"2024-09-17T21:22:48.382842+0000","last_became_peered":"2024-09-17T21:22:48.382842+0000","last_unstale":"2024-09-17T21:22:51.450196+0000","last_undegraded":"2024-09-17T21:22:51.450196+0000","last_fullsized":"2024-09-17T21:22:51.450196+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:30:55.312809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"226'248","reported_seq":9446,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384708+0000","last_change":"2024-09-17T21:22:45.354353+0000","last_active":"2024-09-17T21:22:51.384708+0000","last_peered":"2024-09-17T21:22:51.384708+0000","last_clean":"2024-09-17T21:22:51.384708+0000","last_became_active":"2024-09-17T21:22:45.352682+0000","last_became_peered":"2024-09-17T21:22:45.352682+0000","last_unstale":"2024-09-17T21:22:51.384708+0000","last_undegraded":"2024-09-17T21:22:51.384708+0000","last_fullsized":"2024-09-17T21:22:51.384708+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:18.942784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048483459999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5340,"num_read_kb":22605,"num_write":215,"num_write_kb":6424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":15742651,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":461,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382421+0000","last_change":"2024-09-17T21:22:50.389225+0000","last_active":"2024-09-17T21:22:51.382421+0000","last_peered":"2024-09-17T21:22:51.382421+0000","last_clean":"2024-09-17T21:22:51.382421+0000","last_became_active":"2024-09-17T21:22:50.388953+0000","last_became_peered":"2024-09-17T21:22:50.388953+0000","last_unstale":"2024-09-17T21:22:51.382421+0000","last_undegraded":"2024-09-17T21:22:51.382421+0000","last_fullsized":"2024-09-17T21:22:51.382421+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:21.062518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"227'1041","reported_seq":8867,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450992+0000","last_change":"2024-09-17T21:22:50.391990+0000","last_active":"2024-09-17T21:22:51.450992+0000","last_peered":"2024-09-17T21:22:51.450992+0000","last_clean":"2024-09-17T21:22:51.450992+0000","last_became_active":"2024-09-17T21:22:50.390218+0000","last_became_peered":"2024-09-17T21:22:50.390218+0000","last_unstale":"2024-09-17T21:22:51.450992+0000","last_undegraded":"2024-09-17T21:22:51.450992+0000","last_fullsized":"2024-09-17T21:22:51.450992+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":1041,"log_dups_size":0,"ondisk_log_size":1041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:24:31.427407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035942119999999998,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":9666,"num_read_kb":63102,"num_write":1578,"num_write_kb":18132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":894,"num_bytes_recovered":48903910,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":474,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.383504+0000","last_change":"2024-09-17T21:18:03.059156+0000","last_active":"2024-09-17T21:22:51.383504+0000","last_peered":"2024-09-17T21:22:51.383504+0000","last_clean":"2024-09-17T21:22:51.383504+0000","last_became_active":"2024-09-17T21:18:01.049180+0000","last_became_peered":"2024-09-17T21:18:01.049180+0000","last_unstale":"2024-09-17T21:22:51.383504+0000","last_undegraded":"2024-09-17T21:22:51.383504+0000","last_fullsized":"2024-09-17T21:22:51.383504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:27.069794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077408,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"226'656","reported_seq":14594,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384939+0000","last_change":"2024-09-17T21:22:41.354650+0000","last_active":"2024-09-17T21:22:51.384939+0000","last_peered":"2024-09-17T21:22:51.384939+0000","last_clean":"2024-09-17T21:22:51.384939+0000","last_became_active":"2024-09-17T21:22:41.352429+0000","last_became_peered":"2024-09-17T21:22:41.352429+0000","last_unstale":"2024-09-17T21:22:51.384939+0000","last_undegraded":"2024-09-17T21:22:51.384939+0000","last_fullsized":"2024-09-17T21:22:51.384939+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:30:46.965276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067184260000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14741,"num_read_kb":38983,"num_write":745,"num_write_kb":20177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":27258987,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":19,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450266+0000","last_change":"2024-09-17T21:22:48.387337+0000","last_active":"2024-09-17T21:22:51.450266+0000","last_peered":"2024-09-17T21:22:51.450266+0000","last_clean":"2024-09-17T21:22:51.450266+0000","last_became_active":"2024-09-17T21:22:48.386572+0000","last_became_peered":"2024-09-17T21:22:48.386572+0000","last_unstale":"2024-09-17T21:22:51.450266+0000","last_undegraded":"2024-09-17T21:22:51.450266+0000","last_fullsized":"2024-09-17T21:22:51.450266+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:00.021059+0000","last_clean_scrub_stamp":"2024-09-17T21:18:00.021059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:55:57.934354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":189,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.387986+0000","last_change":"2024-09-17T21:21:17.347819+0000","last_active":"2024-09-17T21:22:51.387986+0000","last_peered":"2024-09-17T21:22:51.387986+0000","last_clean":"2024-09-17T21:22:51.387986+0000","last_became_active":"2024-09-17T21:21:17.337621+0000","last_became_peered":"2024-09-17T21:21:17.337621+0000","last_unstale":"2024-09-17T21:22:51.387986+0000","last_undegraded":"2024-09-17T21:22:51.387986+0000","last_fullsized":"2024-09-17T21:22:51.387986+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:17:55.982339+0000","last_clean_scrub_stamp":"2024-09-17T21:17:55.982339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:24:32.371870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"224'616","reported_seq":14093,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.382313+0000","last_change":"2024-09-17T21:22:50.383917+0000","last_active":"2024-09-17T21:22:51.382313+0000","last_peered":"2024-09-17T21:22:51.382313+0000","last_clean":"2024-09-17T21:22:51.382313+0000","last_became_active":"2024-09-17T21:22:50.382396+0000","last_became_peered":"2024-09-17T21:22:50.382396+0000","last_unstale":"2024-09-17T21:22:51.382313+0000","last_undegraded":"2024-09-17T21:22:51.382313+0000","last_fullsized":"2024-09-17T21:22:51.382313+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:13:13.848853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001602121,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":13993,"num_read_kb":68437,"num_write":586,"num_write_kb":29114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":40776135,"num_keys_recovered":165,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"227'699","reported_seq":11106,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.388697+0000","last_change":"2024-09-17T21:22:49.395344+0000","last_active":"2024-09-17T21:22:51.388697+0000","last_peered":"2024-09-17T21:22:51.388697+0000","last_clean":"2024-09-17T21:22:51.388697+0000","last_became_active":"2024-09-17T21:22:49.390523+0000","last_became_peered":"2024-09-17T21:22:49.390523+0000","last_unstale":"2024-09-17T21:22:51.388697+0000","last_undegraded":"2024-09-17T21:22:51.388697+0000","last_fullsized":"2024-09-17T21:22:51.388697+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:55:28.950314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029497360000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":11677,"num_read_kb":43614,"num_write":1089,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":728,"num_bytes_recovered":30193805,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"227'428","reported_seq":8266,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.383725+0000","last_change":"2024-09-17T21:22:49.397589+0000","last_active":"2024-09-17T21:22:51.383725+0000","last_peered":"2024-09-17T21:22:51.383725+0000","last_clean":"2024-09-17T21:22:51.383725+0000","last_became_active":"2024-09-17T21:22:49.394384+0000","last_became_peered":"2024-09-17T21:22:49.394384+0000","last_unstale":"2024-09-17T21:22:51.383725+0000","last_undegraded":"2024-09-17T21:22:51.383725+0000","last_fullsized":"2024-09-17T21:22:51.383725+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:24:21.885168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045615830000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7186,"num_read_kb":40870,"num_write":369,"num_write_kb":16675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":179794,"num_keys_recovered":2,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"226'376","reported_seq":4920,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384873+0000","last_change":"2024-09-17T21:22:24.375850+0000","last_active":"2024-09-17T21:22:51.384873+0000","last_peered":"2024-09-17T21:22:51.384873+0000","last_clean":"2024-09-17T21:22:51.384873+0000","last_became_active":"2024-09-17T21:21:40.319613+0000","last_became_peered":"2024-09-17T21:21:40.319613+0000","last_unstale":"2024-09-17T21:22:51.384873+0000","last_undegraded":"2024-09-17T21:22:51.384873+0000","last_fullsized":"2024-09-17T21:22:51.384873+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:26:53.101370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038796239999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4694,"num_read_kb":23619,"num_write":320,"num_write_kb":13765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":4886965,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"227'452","reported_seq":13421,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.385236+0000","last_change":"2024-09-17T21:22:43.339607+0000","last_active":"2024-09-17T21:22:51.385236+0000","last_peered":"2024-09-17T21:22:51.385236+0000","last_clean":"2024-09-17T21:22:51.385236+0000","last_became_active":"2024-09-17T21:22:43.338399+0000","last_became_peered":"2024-09-17T21:22:43.338399+0000","last_unstale":"2024-09-17T21:22:51.385236+0000","last_undegraded":"2024-09-17T21:22:51.385236+0000","last_fullsized":"2024-09-17T21:22:51.385236+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:30:14.210043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050460430000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":15379,"num_read_kb":29065,"num_write":580,"num_write_kb":14319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":5960704,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"221'526","reported_seq":15419,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384216+0000","last_change":"2024-09-17T21:22:46.356687+0000","last_active":"2024-09-17T21:22:51.384216+0000","last_peered":"2024-09-17T21:22:51.384216+0000","last_clean":"2024-09-17T21:22:51.384216+0000","last_became_active":"2024-09-17T21:22:46.349835+0000","last_became_peered":"2024-09-17T21:22:46.349835+0000","last_unstale":"2024-09-17T21:22:51.384216+0000","last_undegraded":"2024-09-17T21:22:51.384216+0000","last_fullsized":"2024-09-17T21:22:51.384216+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:51:30.571949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00482724,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15785,"num_read_kb":43220,"num_write":654,"num_write_kb":16401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":28397872,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"227'623","reported_seq":14444,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450685+0000","last_change":"2024-09-17T21:22:49.806087+0000","last_active":"2024-09-17T21:22:51.450685+0000","last_peered":"2024-09-17T21:22:51.450685+0000","last_clean":"2024-09-17T21:22:51.450685+0000","last_became_active":"2024-09-17T21:22:49.394154+0000","last_became_peered":"2024-09-17T21:22:49.394154+0000","last_unstale":"2024-09-17T21:22:51.450685+0000","last_undegraded":"2024-09-17T21:22:51.450685+0000","last_fullsized":"2024-09-17T21:22:51.450685+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:24:57.185890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057317679999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":15141,"num_read_kb":70274,"num_write":535,"num_write_kb":35852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":91925553,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"226'464","reported_seq":7123,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450740+0000","last_change":"2024-09-17T21:22:48.398780+0000","last_active":"2024-09-17T21:22:51.450740+0000","last_peered":"2024-09-17T21:22:51.450740+0000","last_clean":"2024-09-17T21:22:51.450740+0000","last_became_active":"2024-09-17T21:22:48.392669+0000","last_became_peered":"2024-09-17T21:22:48.392669+0000","last_unstale":"2024-09-17T21:22:51.450740+0000","last_undegraded":"2024-09-17T21:22:51.450740+0000","last_fullsized":"2024-09-17T21:22:51.450740+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:18:23.844174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0091147729999999996,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7427,"num_read_kb":28404,"num_write":434,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":4659071,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"222'271","reported_seq":7403,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450132+0000","last_change":"2024-09-17T21:22:48.384974+0000","last_active":"2024-09-17T21:22:51.450132+0000","last_peered":"2024-09-17T21:22:51.450132+0000","last_clean":"2024-09-17T21:22:51.450132+0000","last_became_active":"2024-09-17T21:22:48.381106+0000","last_became_peered":"2024-09-17T21:22:48.381106+0000","last_unstale":"2024-09-17T21:22:51.450132+0000","last_undegraded":"2024-09-17T21:22:51.450132+0000","last_fullsized":"2024-09-17T21:22:51.450132+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:11:47.017985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017482940000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7827,"num_read_kb":19883,"num_write":219,"num_write_kb":3298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":6733181,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"226'404","reported_seq":5779,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384016+0000","last_change":"2024-09-17T21:22:24.385309+0000","last_active":"2024-09-17T21:22:51.384016+0000","last_peered":"2024-09-17T21:22:51.384016+0000","last_clean":"2024-09-17T21:22:51.384016+0000","last_became_active":"2024-09-17T21:18:11.299946+0000","last_became_peered":"2024-09-17T21:18:11.299946+0000","last_unstale":"2024-09-17T21:22:51.384016+0000","last_undegraded":"2024-09-17T21:22:51.384016+0000","last_fullsized":"2024-09-17T21:22:51.384016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:16:16.699933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015128413,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5168,"num_read_kb":22085,"num_write":382,"num_write_kb":4452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"225'570","reported_seq":15995,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.450984+0000","last_change":"2024-09-17T21:22:24.368740+0000","last_active":"2024-09-17T21:22:51.450984+0000","last_peered":"2024-09-17T21:22:51.450984+0000","last_clean":"2024-09-17T21:22:51.450984+0000","last_became_active":"2024-09-17T21:22:09.609844+0000","last_became_peered":"2024-09-17T21:22:09.609844+0000","last_unstale":"2024-09-17T21:22:51.450984+0000","last_undegraded":"2024-09-17T21:22:51.450984+0000","last_fullsized":"2024-09-17T21:22:51.450984+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:03:30.364818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018557070000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":17911,"num_read_kb":38759,"num_write":589,"num_write_kb":6968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":11589682,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"226'651","reported_seq":12606,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:22:51.384243+0000","last_change":"2024-09-17T21:22:41.357868+0000","last_active":"2024-09-17T21:22:51.384243+0000","last_peered":"2024-09-17T21:22:51.384243+0000","last_clean":"2024-09-17T21:22:51.384243+0000","last_became_active":"2024-09-17T21:22:41.354851+0000","last_became_peered":"2024-09-17T21:22:41.354851+0000","last_unstale":"2024-09-17T21:22:51.384243+0000","last_undegraded":"2024-09-17T21:22:51.384243+0000","last_fullsized":"2024-09-17T21:22:51.384243+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:18:10.267877+0000","last_clean_scrub_stamp":"2024-09-17T21:18:10.267877+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:47:04.462955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011937984,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12008,"num_read_kb":52136,"num_write":617,"num_write_kb":5443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":5557098,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":304,"num_object_clones":0,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":300,"num_read":353905,"num_read_kb":1145909,"num_write":19949,"num_write_kb":385384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11936,"num_bytes_recovered":614462337,"num_keys_recovered":1529,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8194,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60472,"kb_used_data":2908,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311368,"statfs":{"total":96636764160,"available":96574840832,"internally_reserved":0,"allocated":2977792,"data_stored":1262263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":58915268},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66616,"kb_used_data":3488,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305224,"statfs":{"total":96636764160,"available":96568549376,"internally_reserved":0,"allocated":3571712,"data_stored":1852631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":64616900},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83256,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288584,"statfs":{"total":96636764160,"available":96551510016,"internally_reserved":0,"allocated":3567616,"data_stored":1852612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":81656262},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56308,"kb_used_data":2912,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315532,"statfs":{"total":96636764160,"available":96579104768,"internally_reserved":0,"allocated":2981888,"data_stored":1262282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56300,"kb_used_data":2904,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315540,"statfs":{"total":96636764160,"available":96579112960,"internally_reserved":0,"allocated":2973696,"data_stored":1262244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":231,"seq":992137445384,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30644,"kb_used_data":2904,"kb_used_omap":20,"kb_used_meta":27691,"kb_avail":94341196,"statfs":{"total":96636764160,"available":96605384704,"internally_reserved":0,"allocated":2973696,"data_stored":1262244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20966,"internal_metadata":28356122},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8194,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:23:04.101 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.101 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.101 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.101 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.101 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.101 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.101 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.102 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.102 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.102 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.102 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.102 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.102 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-17T21:18:00.021059+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.102 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.102 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-17T21:18:00.021059+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.103 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.103 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-17T21:18:00.021059+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.103 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.103 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-17T21:18:00.021059+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.103 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.103 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.103 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-17T21:18:00.021059+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.103 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.104 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-17T21:18:00.021059+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.104 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.104 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-17T21:18:00.021059+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.104 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.104 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-17T21:18:00.021059+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.104 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-17T21:17:55.982339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=17, tm_sec=55, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.104 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.104 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.104 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.105 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.105 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.105 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.105 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.105 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.105 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.105 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.105 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.105 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-17T21:18:10.267877+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=22, tm_sec=59, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:23:04.106 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-17T21:23:24.106 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:23:24.424 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T21:23:24.424 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T21:23:24.442 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2024-09-17T21:23:23.622509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":308,"num_object_clones":0,"num_object_copies":616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":300,"num_read":353971,"num_read_kb":1145964,"num_write":20052,"num_write_kb":387014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11949,"num_bytes_recovered":615643130,"num_keys_recovered":1529,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18913,"ondisk_log_size":18913,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":336904,"kb_used_data":18600,"kb_used_omap":28,"kb_used_meta":318179,"kb_avail":565894136,"statfs":{"total":579820584960,"available":579475595264,"internally_reserved":0,"allocated":19046400,"data_stored":8754276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28944,"internal_metadata":325816048},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003659"},"pg_stats":[{"pgid":"3.1f","version":"247'768","reported_seq":4135,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:07.566273+0000","last_change":"2024-09-17T21:23:07.566273+0000","last_active":"2024-09-17T21:23:07.566273+0000","last_peered":"2024-09-17T21:23:07.566273+0000","last_clean":"2024-09-17T21:23:07.566273+0000","last_became_active":"2024-09-17T21:22:40.357329+0000","last_became_peered":"2024-09-17T21:22:40.357329+0000","last_unstale":"2024-09-17T21:23:07.566273+0000","last_undegraded":"2024-09-17T21:23:07.566273+0000","last_fullsized":"2024-09-17T21:23:07.566273+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'768","last_scrub_stamp":"2024-09-17T21:23:07.566057+0000","last_deep_scrub":"247'768","last_deep_scrub_stamp":"2024-09-17T21:23:07.566057+0000","last_clean_scrub_stamp":"2024-09-17T21:23:07.566057+0000","objects_scrubbed":9,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:12:44.281657+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.010221002999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5921,"num_read_kb":16779,"num_write":1026,"num_write_kb":3495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":5143732,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"247'860","reported_seq":9574,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:06.584460+0000","last_change":"2024-09-17T21:23:06.584460+0000","last_active":"2024-09-17T21:23:06.584460+0000","last_peered":"2024-09-17T21:23:06.584460+0000","last_clean":"2024-09-17T21:23:06.584460+0000","last_became_active":"2024-09-17T21:22:40.353016+0000","last_became_peered":"2024-09-17T21:22:40.353016+0000","last_unstale":"2024-09-17T21:23:06.584460+0000","last_undegraded":"2024-09-17T21:23:06.584460+0000","last_fullsized":"2024-09-17T21:23:06.584460+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'860","last_scrub_stamp":"2024-09-17T21:23:06.584264+0000","last_deep_scrub":"247'860","last_deep_scrub_stamp":"2024-09-17T21:23:06.584264+0000","last_clean_scrub_stamp":"2024-09-17T21:23:06.584264+0000","objects_scrubbed":9,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:05:54.529116+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0084510550000000007,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10998,"num_read_kb":53173,"num_write":1081,"num_write_kb":18708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":50670363,"num_keys_recovered":90,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"247'844","reported_seq":8707,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:11.808566+0000","last_change":"2024-09-17T21:23:11.808566+0000","last_active":"2024-09-17T21:23:11.808566+0000","last_peered":"2024-09-17T21:23:11.808566+0000","last_clean":"2024-09-17T21:23:11.808566+0000","last_became_active":"2024-09-17T21:22:48.401326+0000","last_became_peered":"2024-09-17T21:22:48.401326+0000","last_unstale":"2024-09-17T21:23:11.808566+0000","last_undegraded":"2024-09-17T21:23:11.808566+0000","last_fullsized":"2024-09-17T21:23:11.808566+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'844","last_scrub_stamp":"2024-09-17T21:23:11.808360+0000","last_deep_scrub":"247'844","last_deep_scrub_stamp":"2024-09-17T21:23:11.808360+0000","last_clean_scrub_stamp":"2024-09-17T21:23:11.808360+0000","objects_scrubbed":11,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:55:45.454094+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0082452470000000007,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":10230,"num_read_kb":27857,"num_write":978,"num_write_kb":10466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":12493301,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"247'384","reported_seq":4062,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:12.820377+0000","last_change":"2024-09-17T21:23:12.820377+0000","last_active":"2024-09-17T21:23:12.820377+0000","last_peered":"2024-09-17T21:23:12.820377+0000","last_clean":"2024-09-17T21:23:12.820377+0000","last_became_active":"2024-09-17T21:22:48.401237+0000","last_became_peered":"2024-09-17T21:22:48.401237+0000","last_unstale":"2024-09-17T21:23:12.820377+0000","last_undegraded":"2024-09-17T21:23:12.820377+0000","last_fullsized":"2024-09-17T21:23:12.820377+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'384","last_scrub_stamp":"2024-09-17T21:23:12.820178+0000","last_deep_scrub":"247'384","last_deep_scrub_stamp":"2024-09-17T21:23:12.820178+0000","last_clean_scrub_stamp":"2024-09-17T21:23:12.820178+0000","objects_scrubbed":8,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:34:51.457648+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0061565819999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":2726,"num_read_kb":23738,"num_write":383,"num_write_kb":5937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":9397808,"num_keys_recovered":202,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"247'720","reported_seq":8309,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:02.653238+0000","last_change":"2024-09-17T21:23:02.653238+0000","last_active":"2024-09-17T21:23:02.653238+0000","last_peered":"2024-09-17T21:23:02.653238+0000","last_clean":"2024-09-17T21:23:02.653238+0000","last_became_active":"2024-09-17T21:21:41.329830+0000","last_became_peered":"2024-09-17T21:21:41.329830+0000","last_unstale":"2024-09-17T21:23:02.653238+0000","last_undegraded":"2024-09-17T21:23:02.653238+0000","last_fullsized":"2024-09-17T21:23:02.653238+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'720","last_scrub_stamp":"2024-09-17T21:23:02.653140+0000","last_deep_scrub":"247'720","last_deep_scrub_stamp":"2024-09-17T21:23:02.653140+0000","last_clean_scrub_stamp":"2024-09-17T21:23:02.653140+0000","objects_scrubbed":10,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:59:18.379106+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0026038680000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":11634,"num_read_kb":21103,"num_write":555,"num_write_kb":4725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":7872032,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"247'745","reported_seq":13932,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:05.009180+0000","last_change":"2024-09-17T21:23:05.009180+0000","last_active":"2024-09-17T21:23:05.009180+0000","last_peered":"2024-09-17T21:23:05.009180+0000","last_clean":"2024-09-17T21:23:05.009180+0000","last_became_active":"2024-09-17T21:22:50.386623+0000","last_became_peered":"2024-09-17T21:22:50.386623+0000","last_unstale":"2024-09-17T21:23:05.009180+0000","last_undegraded":"2024-09-17T21:23:05.009180+0000","last_fullsized":"2024-09-17T21:23:05.009180+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'745","last_scrub_stamp":"2024-09-17T21:23:05.008970+0000","last_deep_scrub":"247'745","last_deep_scrub_stamp":"2024-09-17T21:23:05.008970+0000","last_clean_scrub_stamp":"2024-09-17T21:23:05.008970+0000","objects_scrubbed":6,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:04:24.005295+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0056973060000000001,"stat_sum":{"num_bytes":19,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":16893,"num_read_kb":53539,"num_write":669,"num_write_kb":14565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":647737,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"247'483","reported_seq":12494,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:06.673318+0000","last_change":"2024-09-17T21:23:06.673318+0000","last_active":"2024-09-17T21:23:06.673318+0000","last_peered":"2024-09-17T21:23:06.673318+0000","last_clean":"2024-09-17T21:23:06.673318+0000","last_became_active":"2024-09-17T21:22:51.390578+0000","last_became_peered":"2024-09-17T21:22:51.390578+0000","last_unstale":"2024-09-17T21:23:06.673318+0000","last_undegraded":"2024-09-17T21:23:06.673318+0000","last_fullsized":"2024-09-17T21:23:06.673318+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'483","last_scrub_stamp":"2024-09-17T21:23:06.673132+0000","last_deep_scrub":"247'483","last_deep_scrub_stamp":"2024-09-17T21:23:06.673132+0000","last_clean_scrub_stamp":"2024-09-17T21:23:06.673132+0000","objects_scrubbed":7,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:54.916377+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0077005709999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":12648,"num_read_kb":27025,"num_write":374,"num_write_kb":5615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":3100207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"247'332","reported_seq":3057,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:03.641766+0000","last_change":"2024-09-17T21:23:03.641766+0000","last_active":"2024-09-17T21:23:03.641766+0000","last_peered":"2024-09-17T21:23:03.641766+0000","last_clean":"2024-09-17T21:23:03.641766+0000","last_became_active":"2024-09-17T21:22:50.385975+0000","last_became_peered":"2024-09-17T21:22:50.385975+0000","last_unstale":"2024-09-17T21:23:03.641766+0000","last_undegraded":"2024-09-17T21:23:03.641766+0000","last_fullsized":"2024-09-17T21:23:03.641766+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'332","last_scrub_stamp":"2024-09-17T21:23:03.641620+0000","last_deep_scrub":"247'332","last_deep_scrub_stamp":"2024-09-17T21:23:03.641620+0000","last_clean_scrub_stamp":"2024-09-17T21:23:03.641620+0000","objects_scrubbed":14,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:50:15.978398+0000","scrub_duration":23,"objects_trimmed":2,"snaptrim_duration":0.0022643899999999998,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2392,"num_read_kb":9139,"num_write":304,"num_write_kb":2898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":9226431,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"247'403","reported_seq":7361,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:05.599600+0000","last_change":"2024-09-17T21:23:05.599600+0000","last_active":"2024-09-17T21:23:05.599600+0000","last_peered":"2024-09-17T21:23:05.599600+0000","last_clean":"2024-09-17T21:23:05.599600+0000","last_became_active":"2024-09-17T21:22:43.340739+0000","last_became_peered":"2024-09-17T21:22:43.340739+0000","last_unstale":"2024-09-17T21:23:05.599600+0000","last_undegraded":"2024-09-17T21:23:05.599600+0000","last_fullsized":"2024-09-17T21:23:05.599600+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'403","last_scrub_stamp":"2024-09-17T21:23:05.599397+0000","last_deep_scrub":"247'403","last_deep_scrub_stamp":"2024-09-17T21:23:05.599397+0000","last_clean_scrub_stamp":"2024-09-17T21:23:05.599397+0000","objects_scrubbed":10,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:07:29.751369+0000","scrub_duration":19,"objects_trimmed":2,"snaptrim_duration":0.00138835,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7054,"num_read_kb":21313,"num_write":475,"num_write_kb":6946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":719,"num_bytes_recovered":31066012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"247'610","reported_seq":22268,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:10.771264+0000","last_change":"2024-09-17T21:23:10.771264+0000","last_active":"2024-09-17T21:23:10.771264+0000","last_peered":"2024-09-17T21:23:10.771264+0000","last_clean":"2024-09-17T21:23:10.771264+0000","last_became_active":"2024-09-17T21:22:48.394816+0000","last_became_peered":"2024-09-17T21:22:48.394816+0000","last_unstale":"2024-09-17T21:23:10.771264+0000","last_undegraded":"2024-09-17T21:23:10.771264+0000","last_fullsized":"2024-09-17T21:23:10.771264+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'610","last_scrub_stamp":"2024-09-17T21:23:10.771063+0000","last_deep_scrub":"247'610","last_deep_scrub_stamp":"2024-09-17T21:23:10.771063+0000","last_clean_scrub_stamp":"2024-09-17T21:23:10.771063+0000","objects_scrubbed":8,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:40:44.585040+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0075408430000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":22417,"num_read_kb":27659,"num_write":858,"num_write_kb":13022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":3432016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"247'766","reported_seq":7179,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:11.757752+0000","last_change":"2024-09-17T21:23:11.757752+0000","last_active":"2024-09-17T21:23:11.757752+0000","last_peered":"2024-09-17T21:23:11.757752+0000","last_clean":"2024-09-17T21:23:11.757752+0000","last_became_active":"2024-09-17T21:21:40.321648+0000","last_became_peered":"2024-09-17T21:21:40.321648+0000","last_unstale":"2024-09-17T21:23:11.757752+0000","last_undegraded":"2024-09-17T21:23:11.757752+0000","last_fullsized":"2024-09-17T21:23:11.757752+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'766","last_scrub_stamp":"2024-09-17T21:23:11.757500+0000","last_deep_scrub":"247'766","last_deep_scrub_stamp":"2024-09-17T21:23:11.757500+0000","last_clean_scrub_stamp":"2024-09-17T21:23:11.757500+0000","objects_scrubbed":12,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:14:59.884236+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0071138240000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8592,"num_read_kb":29445,"num_write":954,"num_write_kb":5554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":9885558,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"247'458","reported_seq":5393,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:10.719842+0000","last_change":"2024-09-17T21:23:10.719842+0000","last_active":"2024-09-17T21:23:10.719842+0000","last_peered":"2024-09-17T21:23:10.719842+0000","last_clean":"2024-09-17T21:23:10.719842+0000","last_became_active":"2024-09-17T21:22:49.389098+0000","last_became_peered":"2024-09-17T21:22:49.389098+0000","last_unstale":"2024-09-17T21:23:10.719842+0000","last_undegraded":"2024-09-17T21:23:10.719842+0000","last_fullsized":"2024-09-17T21:23:10.719842+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'458","last_scrub_stamp":"2024-09-17T21:23:10.719649+0000","last_deep_scrub":"247'458","last_deep_scrub_stamp":"2024-09-17T21:23:10.719649+0000","last_clean_scrub_stamp":"2024-09-17T21:23:10.719649+0000","objects_scrubbed":9,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:52:17.028950+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.003029,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5351,"num_read_kb":26972,"num_write":428,"num_write_kb":16010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":385,"num_bytes_recovered":6943878,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":482,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:09.691836+0000","last_change":"2024-09-17T21:23:09.691836+0000","last_active":"2024-09-17T21:23:09.691836+0000","last_peered":"2024-09-17T21:23:09.691836+0000","last_clean":"2024-09-17T21:23:09.691836+0000","last_became_active":"2024-09-17T21:18:01.213047+0000","last_became_peered":"2024-09-17T21:18:01.213047+0000","last_unstale":"2024-09-17T21:23:09.691836+0000","last_undegraded":"2024-09-17T21:23:09.691836+0000","last_fullsized":"2024-09-17T21:23:09.691836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:23:09.691653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:23:09.691653+0000","last_clean_scrub_stamp":"2024-09-17T21:23:09.691653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:19:08.239536+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001606704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"247'606","reported_seq":13714,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:04.628730+0000","last_change":"2024-09-17T21:23:04.628730+0000","last_active":"2024-09-17T21:23:04.628730+0000","last_peered":"2024-09-17T21:23:04.628730+0000","last_clean":"2024-09-17T21:23:04.628730+0000","last_became_active":"2024-09-17T21:21:40.318488+0000","last_became_peered":"2024-09-17T21:21:40.318488+0000","last_unstale":"2024-09-17T21:23:04.628730+0000","last_undegraded":"2024-09-17T21:23:04.628730+0000","last_fullsized":"2024-09-17T21:23:04.628730+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'606","last_scrub_stamp":"2024-09-17T21:23:04.628483+0000","last_deep_scrub":"247'606","last_deep_scrub_stamp":"2024-09-17T21:23:04.628483+0000","last_clean_scrub_stamp":"2024-09-17T21:23:04.628483+0000","objects_scrubbed":12,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:13:57.158035+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.0036709440000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14882,"num_read_kb":61123,"num_write":581,"num_write_kb":23555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":49089084,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":482,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:03.636819+0000","last_change":"2024-09-17T21:23:03.636819+0000","last_active":"2024-09-17T21:23:03.636819+0000","last_peered":"2024-09-17T21:23:03.636819+0000","last_clean":"2024-09-17T21:23:03.636819+0000","last_became_active":"2024-09-17T21:18:01.048806+0000","last_became_peered":"2024-09-17T21:18:01.048806+0000","last_unstale":"2024-09-17T21:23:03.636819+0000","last_undegraded":"2024-09-17T21:23:03.636819+0000","last_fullsized":"2024-09-17T21:23:03.636819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:23:03.636657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:23:03.636657+0000","last_clean_scrub_stamp":"2024-09-17T21:23:03.636657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:31:55.591304+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00094161400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"247'365","reported_seq":5631,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:04.610890+0000","last_change":"2024-09-17T21:23:04.610890+0000","last_active":"2024-09-17T21:23:04.610890+0000","last_peered":"2024-09-17T21:23:04.610890+0000","last_clean":"2024-09-17T21:23:04.610890+0000","last_became_active":"2024-09-17T21:22:50.388965+0000","last_became_peered":"2024-09-17T21:22:50.388965+0000","last_unstale":"2024-09-17T21:23:04.610890+0000","last_undegraded":"2024-09-17T21:23:04.610890+0000","last_fullsized":"2024-09-17T21:23:04.610890+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'365","last_scrub_stamp":"2024-09-17T21:23:04.610561+0000","last_deep_scrub":"247'365","last_deep_scrub_stamp":"2024-09-17T21:23:04.610561+0000","last_clean_scrub_stamp":"2024-09-17T21:23:04.610561+0000","objects_scrubbed":13,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:06:48.355948+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0079556969999999994,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4656,"num_read_kb":36280,"num_write":392,"num_write_kb":13876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":5244262,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":506,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:04.736785+0000","last_change":"2024-09-17T21:23:04.736785+0000","last_active":"2024-09-17T21:23:04.736785+0000","last_peered":"2024-09-17T21:23:04.736785+0000","last_clean":"2024-09-17T21:23:04.736785+0000","last_became_active":"2024-09-17T21:21:35.312507+0000","last_became_peered":"2024-09-17T21:21:35.312507+0000","last_unstale":"2024-09-17T21:23:04.736785+0000","last_undegraded":"2024-09-17T21:23:04.736785+0000","last_fullsized":"2024-09-17T21:23:04.736785+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:23:04.736608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:23:04.736608+0000","last_clean_scrub_stamp":"2024-09-17T21:23:04.736608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:17:02.364606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099219900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"247'718","reported_seq":16154,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:07.727393+0000","last_change":"2024-09-17T21:23:07.727393+0000","last_active":"2024-09-17T21:23:07.727393+0000","last_peered":"2024-09-17T21:23:07.727393+0000","last_clean":"2024-09-17T21:23:07.727393+0000","last_became_active":"2024-09-17T21:22:46.359525+0000","last_became_peered":"2024-09-17T21:22:46.359525+0000","last_unstale":"2024-09-17T21:23:07.727393+0000","last_undegraded":"2024-09-17T21:23:07.727393+0000","last_fullsized":"2024-09-17T21:23:07.727393+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'718","last_scrub_stamp":"2024-09-17T21:23:07.727167+0000","last_deep_scrub":"247'718","last_deep_scrub_stamp":"2024-09-17T21:23:07.727167+0000","last_clean_scrub_stamp":"2024-09-17T21:23:07.727167+0000","objects_scrubbed":11,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:09:41.215545+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0052152359999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13338,"num_read_kb":29382,"num_write":606,"num_write_kb":10802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":367,"num_bytes_recovered":21385444,"num_keys_recovered":118,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":506,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:02.656494+0000","last_change":"2024-09-17T21:23:02.656494+0000","last_active":"2024-09-17T21:23:02.656494+0000","last_peered":"2024-09-17T21:23:02.656494+0000","last_clean":"2024-09-17T21:23:02.656494+0000","last_became_active":"2024-09-17T21:21:35.325777+0000","last_became_peered":"2024-09-17T21:21:35.325777+0000","last_unstale":"2024-09-17T21:23:02.656494+0000","last_undegraded":"2024-09-17T21:23:02.656494+0000","last_fullsized":"2024-09-17T21:23:02.656494+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:23:02.656412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:23:02.656412+0000","last_clean_scrub_stamp":"2024-09-17T21:23:02.656412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:52:25.185462+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001006222,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"247'1067","reported_seq":33599,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:07.789042+0000","last_change":"2024-09-17T21:23:07.789042+0000","last_active":"2024-09-17T21:23:07.789042+0000","last_peered":"2024-09-17T21:23:07.789042+0000","last_clean":"2024-09-17T21:23:07.789042+0000","last_became_active":"2024-09-17T21:22:44.352239+0000","last_became_peered":"2024-09-17T21:22:44.352239+0000","last_unstale":"2024-09-17T21:23:07.789042+0000","last_undegraded":"2024-09-17T21:23:07.789042+0000","last_fullsized":"2024-09-17T21:23:07.789042+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'1067","last_scrub_stamp":"2024-09-17T21:23:07.788831+0000","last_deep_scrub":"247'1067","last_deep_scrub_stamp":"2024-09-17T21:23:07.788831+0000","last_clean_scrub_stamp":"2024-09-17T21:23:07.788831+0000","objects_scrubbed":11,"log_size":1067,"log_dups_size":0,"ondisk_log_size":1067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:42:04.607811+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0052118770000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":37177,"num_read_kb":61903,"num_write":871,"num_write_kb":21852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":754,"num_bytes_recovered":58900004,"num_keys_recovered":204,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"247'548","reported_seq":3907,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:06.012610+0000","last_change":"2024-09-17T21:23:06.012610+0000","last_active":"2024-09-17T21:23:06.012610+0000","last_peered":"2024-09-17T21:23:06.012610+0000","last_clean":"2024-09-17T21:23:06.012610+0000","last_became_active":"2024-09-17T21:21:39.315142+0000","last_became_peered":"2024-09-17T21:21:39.315142+0000","last_unstale":"2024-09-17T21:23:06.012610+0000","last_undegraded":"2024-09-17T21:23:06.012610+0000","last_fullsized":"2024-09-17T21:23:06.012610+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'548","last_scrub_stamp":"2024-09-17T21:23:06.012395+0000","last_deep_scrub":"247'548","last_deep_scrub_stamp":"2024-09-17T21:23:06.012395+0000","last_clean_scrub_stamp":"2024-09-17T21:23:06.012395+0000","objects_scrubbed":12,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:13:36.806085+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0061334989999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":3053,"num_read_kb":14423,"num_write":502,"num_write_kb":4818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":7199060,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"247'3","reported_seq":421,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:04.757233+0000","last_change":"2024-09-17T21:23:04.757233+0000","last_active":"2024-09-17T21:23:04.757233+0000","last_peered":"2024-09-17T21:23:04.757233+0000","last_clean":"2024-09-17T21:23:04.757233+0000","last_became_active":"2024-09-17T21:22:48.382842+0000","last_became_peered":"2024-09-17T21:22:48.382842+0000","last_unstale":"2024-09-17T21:23:04.757233+0000","last_undegraded":"2024-09-17T21:23:04.757233+0000","last_fullsized":"2024-09-17T21:23:04.757233+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'3","last_scrub_stamp":"2024-09-17T21:23:04.757024+0000","last_deep_scrub":"247'3","last_deep_scrub_stamp":"2024-09-17T21:23:04.757024+0000","last_clean_scrub_stamp":"2024-09-17T21:23:04.757024+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:36:13.128873+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"247'252","reported_seq":9458,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:04.675982+0000","last_change":"2024-09-17T21:23:04.675982+0000","last_active":"2024-09-17T21:23:04.675982+0000","last_peered":"2024-09-17T21:23:04.675982+0000","last_clean":"2024-09-17T21:23:04.675982+0000","last_became_active":"2024-09-17T21:22:45.352682+0000","last_became_peered":"2024-09-17T21:22:45.352682+0000","last_unstale":"2024-09-17T21:23:04.675982+0000","last_undegraded":"2024-09-17T21:23:04.675982+0000","last_fullsized":"2024-09-17T21:23:04.675982+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'252","last_scrub_stamp":"2024-09-17T21:23:04.675797+0000","last_deep_scrub":"247'252","last_deep_scrub_stamp":"2024-09-17T21:23:04.675797+0000","last_clean_scrub_stamp":"2024-09-17T21:23:04.675797+0000","objects_scrubbed":4,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:08.663168+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0048483459999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5340,"num_read_kb":22605,"num_write":215,"num_write_kb":6424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":15742651,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":469,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:02.648027+0000","last_change":"2024-09-17T21:23:02.648027+0000","last_active":"2024-09-17T21:23:02.648027+0000","last_peered":"2024-09-17T21:23:02.648027+0000","last_clean":"2024-09-17T21:23:02.648027+0000","last_became_active":"2024-09-17T21:22:50.388953+0000","last_became_peered":"2024-09-17T21:22:50.388953+0000","last_unstale":"2024-09-17T21:23:02.648027+0000","last_undegraded":"2024-09-17T21:23:02.648027+0000","last_fullsized":"2024-09-17T21:23:02.648027+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:23:02.647963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:23:02.647963+0000","last_clean_scrub_stamp":"2024-09-17T21:23:02.647963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:21:08.282955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"247'1046","reported_seq":8880,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:01.634133+0000","last_change":"2024-09-17T21:23:01.634133+0000","last_active":"2024-09-17T21:23:01.634133+0000","last_peered":"2024-09-17T21:23:01.634133+0000","last_clean":"2024-09-17T21:23:01.634133+0000","last_became_active":"2024-09-17T21:22:50.390218+0000","last_became_peered":"2024-09-17T21:22:50.390218+0000","last_unstale":"2024-09-17T21:23:01.634133+0000","last_undegraded":"2024-09-17T21:23:01.634133+0000","last_fullsized":"2024-09-17T21:23:01.634133+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'1046","last_scrub_stamp":"2024-09-17T21:23:01.634046+0000","last_deep_scrub":"247'1046","last_deep_scrub_stamp":"2024-09-17T21:23:01.634046+0000","last_clean_scrub_stamp":"2024-09-17T21:23:01.634046+0000","objects_scrubbed":5,"log_size":1046,"log_dups_size":0,"ondisk_log_size":1046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:06:23.207436+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0035942119999999998,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":9666,"num_read_kb":63102,"num_write":1578,"num_write_kb":18132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":894,"num_bytes_recovered":48903910,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":482,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:03.700847+0000","last_change":"2024-09-17T21:23:03.700847+0000","last_active":"2024-09-17T21:23:03.700847+0000","last_peered":"2024-09-17T21:23:03.700847+0000","last_clean":"2024-09-17T21:23:03.700847+0000","last_became_active":"2024-09-17T21:18:01.049180+0000","last_became_peered":"2024-09-17T21:18:01.049180+0000","last_unstale":"2024-09-17T21:23:03.700847+0000","last_undegraded":"2024-09-17T21:23:03.700847+0000","last_fullsized":"2024-09-17T21:23:03.700847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:23:03.700667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:23:03.700667+0000","last_clean_scrub_stamp":"2024-09-17T21:23:03.700667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:37:44.689574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077408,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"247'668","reported_seq":14614,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:08.715379+0000","last_change":"2024-09-17T21:23:08.715379+0000","last_active":"2024-09-17T21:23:08.715379+0000","last_peered":"2024-09-17T21:23:08.715379+0000","last_clean":"2024-09-17T21:23:08.715379+0000","last_became_active":"2024-09-17T21:22:41.352429+0000","last_became_peered":"2024-09-17T21:22:41.352429+0000","last_unstale":"2024-09-17T21:23:08.715379+0000","last_undegraded":"2024-09-17T21:23:08.715379+0000","last_fullsized":"2024-09-17T21:23:08.715379+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'668","last_scrub_stamp":"2024-09-17T21:23:08.715060+0000","last_deep_scrub":"247'668","last_deep_scrub_stamp":"2024-09-17T21:23:08.715060+0000","last_clean_scrub_stamp":"2024-09-17T21:23:08.715060+0000","objects_scrubbed":12,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:07:26.193760+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0067184260000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14741,"num_read_kb":38983,"num_write":745,"num_write_kb":20177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":27258987,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"247'2","reported_seq":28,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:05.723257+0000","last_change":"2024-09-17T21:23:05.723257+0000","last_active":"2024-09-17T21:23:05.723257+0000","last_peered":"2024-09-17T21:23:05.723257+0000","last_clean":"2024-09-17T21:23:05.723257+0000","last_became_active":"2024-09-17T21:22:48.386572+0000","last_became_peered":"2024-09-17T21:22:48.386572+0000","last_unstale":"2024-09-17T21:23:05.723257+0000","last_undegraded":"2024-09-17T21:23:05.723257+0000","last_fullsized":"2024-09-17T21:23:05.723257+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'2","last_scrub_stamp":"2024-09-17T21:23:05.723059+0000","last_deep_scrub":"247'2","last_deep_scrub_stamp":"2024-09-17T21:23:05.723059+0000","last_clean_scrub_stamp":"2024-09-17T21:23:05.723059+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:17:38.689915+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"247'74","reported_seq":200,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:04.016619+0000","last_change":"2024-09-17T21:23:04.016378+0000","last_active":"2024-09-17T21:23:04.016619+0000","last_peered":"2024-09-17T21:23:04.016619+0000","last_clean":"2024-09-17T21:23:04.016619+0000","last_became_active":"2024-09-17T21:21:17.337621+0000","last_became_peered":"2024-09-17T21:21:17.337621+0000","last_unstale":"2024-09-17T21:23:04.016619+0000","last_undegraded":"2024-09-17T21:23:04.016619+0000","last_fullsized":"2024-09-17T21:23:04.016619+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'74","last_scrub_stamp":"2024-09-17T21:23:04.016133+0000","last_deep_scrub":"247'74","last_deep_scrub_stamp":"2024-09-17T21:23:04.016133+0000","last_clean_scrub_stamp":"2024-09-17T21:23:04.016133+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:29:43.030557+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"247'624","reported_seq":14109,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:05.554898+0000","last_change":"2024-09-17T21:23:05.554898+0000","last_active":"2024-09-17T21:23:05.554898+0000","last_peered":"2024-09-17T21:23:05.554898+0000","last_clean":"2024-09-17T21:23:05.554898+0000","last_became_active":"2024-09-17T21:22:50.382396+0000","last_became_peered":"2024-09-17T21:22:50.382396+0000","last_unstale":"2024-09-17T21:23:05.554898+0000","last_undegraded":"2024-09-17T21:23:05.554898+0000","last_fullsized":"2024-09-17T21:23:05.554898+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'624","last_scrub_stamp":"2024-09-17T21:23:05.554691+0000","last_deep_scrub":"247'624","last_deep_scrub_stamp":"2024-09-17T21:23:05.554691+0000","last_clean_scrub_stamp":"2024-09-17T21:23:05.554691+0000","objects_scrubbed":8,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:51:23.018041+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.001602121,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":13993,"num_read_kb":68437,"num_write":586,"num_write_kb":29114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":40776135,"num_keys_recovered":165,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"247'706","reported_seq":11121,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:07.003279+0000","last_change":"2024-09-17T21:23:07.003279+0000","last_active":"2024-09-17T21:23:07.003279+0000","last_peered":"2024-09-17T21:23:07.003279+0000","last_clean":"2024-09-17T21:23:07.003279+0000","last_became_active":"2024-09-17T21:22:49.390523+0000","last_became_peered":"2024-09-17T21:22:49.390523+0000","last_unstale":"2024-09-17T21:23:07.003279+0000","last_undegraded":"2024-09-17T21:23:07.003279+0000","last_fullsized":"2024-09-17T21:23:07.003279+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'706","last_scrub_stamp":"2024-09-17T21:23:07.003097+0000","last_deep_scrub":"247'706","last_deep_scrub_stamp":"2024-09-17T21:23:07.003097+0000","last_clean_scrub_stamp":"2024-09-17T21:23:07.003097+0000","objects_scrubbed":7,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:44:24.960184+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0029497360000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":11677,"num_read_kb":43614,"num_write":1089,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":728,"num_bytes_recovered":30193805,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"247'439","reported_seq":8285,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:05.771552+0000","last_change":"2024-09-17T21:23:05.771552+0000","last_active":"2024-09-17T21:23:05.771552+0000","last_peered":"2024-09-17T21:23:05.771552+0000","last_clean":"2024-09-17T21:23:05.771552+0000","last_became_active":"2024-09-17T21:22:49.394384+0000","last_became_peered":"2024-09-17T21:22:49.394384+0000","last_unstale":"2024-09-17T21:23:05.771552+0000","last_undegraded":"2024-09-17T21:23:05.771552+0000","last_fullsized":"2024-09-17T21:23:05.771552+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'439","last_scrub_stamp":"2024-09-17T21:23:05.771346+0000","last_deep_scrub":"247'439","last_deep_scrub_stamp":"2024-09-17T21:23:05.771346+0000","last_clean_scrub_stamp":"2024-09-17T21:23:05.771346+0000","objects_scrubbed":11,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:53:26.632033+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0045615830000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7186,"num_read_kb":40870,"num_write":369,"num_write_kb":16675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":179794,"num_keys_recovered":2,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"247'382","reported_seq":4934,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:05.674148+0000","last_change":"2024-09-17T21:23:05.674148+0000","last_active":"2024-09-17T21:23:05.674148+0000","last_peered":"2024-09-17T21:23:05.674148+0000","last_clean":"2024-09-17T21:23:05.674148+0000","last_became_active":"2024-09-17T21:21:40.319613+0000","last_became_peered":"2024-09-17T21:21:40.319613+0000","last_unstale":"2024-09-17T21:23:05.674148+0000","last_undegraded":"2024-09-17T21:23:05.674148+0000","last_fullsized":"2024-09-17T21:23:05.674148+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'382","last_scrub_stamp":"2024-09-17T21:23:05.673971+0000","last_deep_scrub":"247'382","last_deep_scrub_stamp":"2024-09-17T21:23:05.673971+0000","last_clean_scrub_stamp":"2024-09-17T21:23:05.673971+0000","objects_scrubbed":6,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:24:00.878829+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0038796239999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4694,"num_read_kb":23619,"num_write":320,"num_write_kb":13765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":4886965,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"247'467","reported_seq":13445,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:09.698404+0000","last_change":"2024-09-17T21:23:09.698404+0000","last_active":"2024-09-17T21:23:09.698404+0000","last_peered":"2024-09-17T21:23:09.698404+0000","last_clean":"2024-09-17T21:23:09.698404+0000","last_became_active":"2024-09-17T21:22:43.338399+0000","last_became_peered":"2024-09-17T21:22:43.338399+0000","last_unstale":"2024-09-17T21:23:09.698404+0000","last_undegraded":"2024-09-17T21:23:09.698404+0000","last_fullsized":"2024-09-17T21:23:09.698404+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'467","last_scrub_stamp":"2024-09-17T21:23:09.698220+0000","last_deep_scrub":"247'467","last_deep_scrub_stamp":"2024-09-17T21:23:09.698220+0000","last_clean_scrub_stamp":"2024-09-17T21:23:09.698220+0000","objects_scrubbed":15,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:50:16.029350+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0050460430000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":15379,"num_read_kb":29065,"num_write":580,"num_write_kb":14319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":5960704,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"247'534","reported_seq":15436,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:08.719474+0000","last_change":"2024-09-17T21:23:08.719346+0000","last_active":"2024-09-17T21:23:08.719474+0000","last_peered":"2024-09-17T21:23:08.719474+0000","last_clean":"2024-09-17T21:23:08.719474+0000","last_became_active":"2024-09-17T21:22:46.349835+0000","last_became_peered":"2024-09-17T21:22:46.349835+0000","last_unstale":"2024-09-17T21:23:08.719474+0000","last_undegraded":"2024-09-17T21:23:08.719474+0000","last_fullsized":"2024-09-17T21:23:08.719474+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'534","last_scrub_stamp":"2024-09-17T21:23:08.719146+0000","last_deep_scrub":"247'534","last_deep_scrub_stamp":"2024-09-17T21:23:08.719146+0000","last_clean_scrub_stamp":"2024-09-17T21:23:08.719146+0000","objects_scrubbed":8,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:52.544960+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.00482724,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15785,"num_read_kb":43220,"num_write":654,"num_write_kb":16401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":28397872,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"247'629","reported_seq":14458,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:06.755898+0000","last_change":"2024-09-17T21:23:06.755898+0000","last_active":"2024-09-17T21:23:06.755898+0000","last_peered":"2024-09-17T21:23:06.755898+0000","last_clean":"2024-09-17T21:23:06.755898+0000","last_became_active":"2024-09-17T21:22:49.394154+0000","last_became_peered":"2024-09-17T21:22:49.394154+0000","last_unstale":"2024-09-17T21:23:06.755898+0000","last_undegraded":"2024-09-17T21:23:06.755898+0000","last_fullsized":"2024-09-17T21:23:06.755898+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'629","last_scrub_stamp":"2024-09-17T21:23:06.755614+0000","last_deep_scrub":"247'629","last_deep_scrub_stamp":"2024-09-17T21:23:06.755614+0000","last_clean_scrub_stamp":"2024-09-17T21:23:06.755614+0000","objects_scrubbed":6,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:06:41.462277+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0057317679999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":15141,"num_read_kb":70274,"num_write":535,"num_write_kb":35852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":91925553,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"247'475","reported_seq":7142,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:08.810649+0000","last_change":"2024-09-17T21:23:08.810649+0000","last_active":"2024-09-17T21:23:08.810649+0000","last_peered":"2024-09-17T21:23:08.810649+0000","last_clean":"2024-09-17T21:23:08.810649+0000","last_became_active":"2024-09-17T21:22:48.392669+0000","last_became_peered":"2024-09-17T21:22:48.392669+0000","last_unstale":"2024-09-17T21:23:08.810649+0000","last_undegraded":"2024-09-17T21:23:08.810649+0000","last_fullsized":"2024-09-17T21:23:08.810649+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'475","last_scrub_stamp":"2024-09-17T21:23:08.810439+0000","last_deep_scrub":"247'475","last_deep_scrub_stamp":"2024-09-17T21:23:08.810439+0000","last_clean_scrub_stamp":"2024-09-17T21:23:08.810439+0000","objects_scrubbed":11,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:00:56.270977+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.0091147729999999996,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7427,"num_read_kb":28404,"num_write":434,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":4659071,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"247'278","reported_seq":7418,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:09.757277+0000","last_change":"2024-09-17T21:23:09.757277+0000","last_active":"2024-09-17T21:23:09.757277+0000","last_peered":"2024-09-17T21:23:09.757277+0000","last_clean":"2024-09-17T21:23:09.757277+0000","last_became_active":"2024-09-17T21:22:48.381106+0000","last_became_peered":"2024-09-17T21:22:48.381106+0000","last_unstale":"2024-09-17T21:23:09.757277+0000","last_undegraded":"2024-09-17T21:23:09.757277+0000","last_fullsized":"2024-09-17T21:23:09.757277+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'278","last_scrub_stamp":"2024-09-17T21:23:09.757061+0000","last_deep_scrub":"247'278","last_deep_scrub_stamp":"2024-09-17T21:23:09.757061+0000","last_clean_scrub_stamp":"2024-09-17T21:23:09.757061+0000","objects_scrubbed":7,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:14:41.649492+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0017482940000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7827,"num_read_kb":19883,"num_write":219,"num_write_kb":3298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":6733181,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"247'414","reported_seq":5797,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:06.774014+0000","last_change":"2024-09-17T21:23:06.774014+0000","last_active":"2024-09-17T21:23:06.774014+0000","last_peered":"2024-09-17T21:23:06.774014+0000","last_clean":"2024-09-17T21:23:06.774014+0000","last_became_active":"2024-09-17T21:18:11.299946+0000","last_became_peered":"2024-09-17T21:18:11.299946+0000","last_unstale":"2024-09-17T21:23:06.774014+0000","last_undegraded":"2024-09-17T21:23:06.774014+0000","last_fullsized":"2024-09-17T21:23:06.774014+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'414","last_scrub_stamp":"2024-09-17T21:23:06.773783+0000","last_deep_scrub":"247'414","last_deep_scrub_stamp":"2024-09-17T21:23:06.773783+0000","last_clean_scrub_stamp":"2024-09-17T21:23:06.773783+0000","objects_scrubbed":10,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:13:07.769756+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.015128413,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5168,"num_read_kb":22085,"num_write":382,"num_write_kb":4452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"247'580","reported_seq":16013,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:03.658983+0000","last_change":"2024-09-17T21:23:03.658983+0000","last_active":"2024-09-17T21:23:03.658983+0000","last_peered":"2024-09-17T21:23:03.658983+0000","last_clean":"2024-09-17T21:23:03.658983+0000","last_became_active":"2024-09-17T21:22:09.609844+0000","last_became_peered":"2024-09-17T21:22:09.609844+0000","last_unstale":"2024-09-17T21:23:03.658983+0000","last_undegraded":"2024-09-17T21:23:03.658983+0000","last_fullsized":"2024-09-17T21:23:03.658983+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'580","last_scrub_stamp":"2024-09-17T21:23:03.658854+0000","last_deep_scrub":"247'580","last_deep_scrub_stamp":"2024-09-17T21:23:03.658854+0000","last_clean_scrub_stamp":"2024-09-17T21:23:03.658854+0000","objects_scrubbed":10,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:55:01.238726+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0018557070000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":17911,"num_read_kb":38759,"num_write":589,"num_write_kb":6968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":11589682,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"247'663","reported_seq":12626,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-17T21:23:07.777703+0000","last_change":"2024-09-17T21:23:07.777703+0000","last_active":"2024-09-17T21:23:07.777703+0000","last_peered":"2024-09-17T21:23:07.777703+0000","last_clean":"2024-09-17T21:23:07.777703+0000","last_became_active":"2024-09-17T21:22:41.354851+0000","last_became_peered":"2024-09-17T21:22:41.354851+0000","last_unstale":"2024-09-17T21:23:07.777703+0000","last_undegraded":"2024-09-17T21:23:07.777703+0000","last_fullsized":"2024-09-17T21:23:07.777703+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'663","last_scrub_stamp":"2024-09-17T21:23:07.777460+0000","last_deep_scrub":"247'663","last_deep_scrub_stamp":"2024-09-17T21:23:07.777460+0000","last_clean_scrub_stamp":"2024-09-17T21:23:07.777460+0000","objects_scrubbed":12,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:03:52.970153+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.011937984,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12008,"num_read_kb":52136,"num_write":617,"num_write_kb":5443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":5557098,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":304,"num_object_clones":0,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":300,"num_read":353905,"num_read_kb":1145909,"num_write":19949,"num_write_kb":385384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11936,"num_bytes_recovered":614462337,"num_keys_recovered":1529,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8194,"internal_metadata":0},"log_size":18834,"ondisk_log_size":18834,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574917,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60460,"kb_used_data":2908,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311380,"statfs":{"total":96636764160,"available":96574853120,"internally_reserved":0,"allocated":2977792,"data_stored":1262263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":58915268},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574917,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66608,"kb_used_data":3488,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305232,"statfs":{"total":96636764160,"available":96568557568,"internally_reserved":0,"allocated":3571712,"data_stored":1852631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":64616900},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574917,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66604,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305236,"statfs":{"total":96636764160,"available":96568561664,"internally_reserved":0,"allocated":3567616,"data_stored":1852612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56304,"kb_used_data":2912,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315536,"statfs":{"total":96636764160,"available":96579108864,"internally_reserved":0,"allocated":2981888,"data_stored":1262282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574918,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56296,"kb_used_data":2904,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315544,"statfs":{"total":96636764160,"available":96579117056,"internally_reserved":0,"allocated":2973696,"data_stored":1262244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":231,"seq":992137445388,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30632,"kb_used_data":2904,"kb_used_omap":20,"kb_used_meta":27691,"kb_avail":94341208,"statfs":{"total":96636764160,"available":96605396992,"internally_reserved":0,"allocated":2973696,"data_stored":1262244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20966,"internal_metadata":28356122},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8194,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:23:24.445 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-17T21:23:24.766 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-17T21:23:24.766 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-17T21:23:24.766 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-17T21:23:24.767 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:23:24.825 INFO:tasks.ceph.osd.0:Stopped 2024-09-17T21:23:24.825 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-17T21:23:24.825 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:23:24.916 INFO:tasks.ceph.osd.1:Stopped 2024-09-17T21:23:24.917 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-17T21:23:24.917 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:23:25.008 INFO:tasks.ceph.osd.2:Stopped 2024-09-17T21:23:25.008 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-17T21:23:25.008 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:23:29.444 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-17T21:23:31.110 INFO:tasks.ceph.osd.3:Stopped 2024-09-17T21:23:31.110 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-17T21:23:31.110 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:23:31.203 INFO:tasks.ceph.osd.4:Stopped 2024-09-17T21:23:31.203 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-17T21:23:31.204 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:23:31.283 INFO:tasks.ceph.osd.5:Stopped 2024-09-17T21:23:31.283 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-17T21:23:31.283 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-17T21:23:31.283 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:23:31.312 INFO:tasks.ceph.mgr.x:Stopped 2024-09-17T21:23:31.312 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-17T21:23:31.312 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-17T21:23:31.312 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:23:31.327 INFO:tasks.ceph.mon.a:Stopped 2024-09-17T21:23:31.327 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-17T21:23:31.327 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:23:31.390 INFO:tasks.ceph.mon.c:Stopped 2024-09-17T21:23:31.390 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-17T21:23:31.390 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:23:31.403 INFO:tasks.ceph.mon.b:Stopped 2024-09-17T21:23:31.404 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-17T21:23:31.404 DEBUG:teuthology.orchestra.run.smithi083:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v SLOW_OPS | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-17T21:23:31.451 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi083.front.sepia.ceph.com 2024-09-17T21:23:31.451 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-17T21:23:31.733 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi083.front.sepia.ceph.com 2024-09-17T21:23:31.733 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-17T21:23:31.968 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi083.front.sepia.ceph.com 2024-09-17T21:23:31.968 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-17T21:23:32.168 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi097.front.sepia.ceph.com 2024-09-17T21:23:32.168 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-17T21:23:32.348 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi097.front.sepia.ceph.com 2024-09-17T21:23:32.348 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-17T21:23:32.525 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi097.front.sepia.ceph.com 2024-09-17T21:23:32.525 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-17T21:23:32.673 INFO:tasks.ceph:Archiving mon data... 2024-09-17T21:23:32.674 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909340/data/mon.a.tgz 2024-09-17T21:23:32.674 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2024-09-17T21:23:32.680 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.cQu7fGBTTh 2024-09-17T21:23:32.681 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.cQu7fGBTTh 2024-09-17T21:23:32.841 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.cQu7fGBTTh 2024-09-17T21:23:32.901 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.cQu7fGBTTh is 437KB 2024-09-17T21:23:32.963 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.cQu7fGBTTh 2024-09-17T21:23:32.968 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909340/data/mon.c.tgz 2024-09-17T21:23:32.968 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2024-09-17T21:23:33.018 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.usfH7IHNdX 2024-09-17T21:23:33.018 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.usfH7IHNdX 2024-09-17T21:23:33.209 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.usfH7IHNdX 2024-09-17T21:23:33.273 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.usfH7IHNdX is 588KB 2024-09-17T21:23:33.337 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.usfH7IHNdX 2024-09-17T21:23:33.343 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909340/data/mon.b.tgz 2024-09-17T21:23:33.343 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-09-17T21:23:33.349 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.F7pPWB2A0M 2024-09-17T21:23:33.349 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.F7pPWB2A0M 2024-09-17T21:23:33.543 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.F7pPWB2A0M 2024-09-17T21:23:33.610 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.F7pPWB2A0M is 588KB 2024-09-17T21:23:33.672 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.F7pPWB2A0M 2024-09-17T21:23:33.678 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-17T21:23:33.678 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T21:23:33.680 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T21:23:33.724 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T21:23:33.744 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T21:23:33.787 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T21:23:33.792 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T21:23:33.792 INFO:tasks.ceph:Archiving crash dumps... 2024-09-17T21:23:33.793 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909340/remote/smithi083/crash 2024-09-17T21:23:33.794 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T21:23:33.807 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909340/remote/smithi097/crash 2024-09-17T21:23:33.808 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T21:23:33.847 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909340/remote/smithi184/crash 2024-09-17T21:23:33.848 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T21:23:33.861 INFO:tasks.ceph:Compressing logs... 2024-09-17T21:23:33.862 DEBUG:teuthology.orchestra.run.smithi083:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T21:23:33.864 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T21:23:33.886 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32008.log 2024-09-17T21:23:33.886 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42864.log 2024-09-17T21:23:33.903 DEBUG:teuthology.orchestra.run.smithi184:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T21:23:33.931 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25220.log 2024-09-17T21:23:33.931 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25268.log 2024-09-17T21:23:33.931 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.25220.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-17T21:23:33.931 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25220.log.gz 2024-09-17T21:23:33.931 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.25268.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.25268.log.gz /var/log/ceph/ceph-osd.5.log 2024-09-17T21:23:33.931 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-09-17T21:23:33.931 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-17T21:23:33.931 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T21:23:33.932 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25196.log 2024-09-17T21:23:33.932 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T21:23:33.932 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.25196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25244.log 2024-09-17T21:23:33.932 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25196.log.gz 2024-09-17T21:23:33.932 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-17T21:23:33.932 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.25244.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-17T21:23:33.932 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.25244.log.gz 2024-09-17T21:23:33.932 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25292.log 2024-09-17T21:23:33.932 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25172.log 2024-09-17T21:23:33.932 INFO:teuthology.orchestra.run.smithi097.stderr: 85.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T21:23:33.933 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.25292.log: /var/log/ceph/ceph-osd.4.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25292.log.gz 2024-09-17T21:23:33.933 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T21:23:33.933 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.25172.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25172.log.gz 2024-09-17T21:23:34.037 INFO:teuthology.orchestra.run.smithi097.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-17T21:23:34.085 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24470.log 2024-09-17T21:23:34.085 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24049.log 2024-09-17T21:23:34.085 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24119.log 2024-09-17T21:23:34.085 INFO:teuthology.orchestra.run.smithi184.stderr: 77.0% -- replaced with /var/log/ceph/ceph-client.admin.24470.log.gz 2024-09-17T21:23:34.085 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24390.log 2024-09-17T21:23:34.086 INFO:teuthology.orchestra.run.smithi184.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.24049.log.gz 2024-09-17T21:23:34.086 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24223.log 2024-09-17T21:23:34.086 INFO:teuthology.orchestra.run.smithi184.stderr: 54.7% -- replaced with /var/log/ceph/ceph-client.admin.24119.log.gz 2024-09-17T21:23:34.086 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23058.log 2024-09-17T21:23:34.086 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.24390.log.gz 2024-09-17T21:23:34.086 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23793.log 2024-09-17T21:23:34.086 INFO:teuthology.orchestra.run.smithi184.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.24223.log.gz 2024-09-17T21:23:34.086 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23732.log 2024-09-17T21:23:34.086 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.23058.log.gz 2024-09-17T21:23:34.086 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23873.log 2024-09-17T21:23:34.086 INFO:teuthology.orchestra.run.smithi184.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.23793.log.gz 2024-09-17T21:23:34.086 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21630.log 2024-09-17T21:23:34.086 INFO:teuthology.orchestra.run.smithi184.stderr: 56.2% -- replaced with /var/log/ceph/ceph-client.admin.23732.log.gz 2024-09-17T21:23:34.086 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22181.log 2024-09-17T21:23:34.087 INFO:teuthology.orchestra.run.smithi184.stderr: 56.8% -- replaced with /var/log/ceph/ceph-client.admin.23873.log.gz 2024-09-17T21:23:34.087 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.21630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22717.log 2024-09-17T21:23:34.087 INFO:teuthology.orchestra.run.smithi184.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.21630.log.gz 2024-09-17T21:23:34.087 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24540.log 2024-09-17T21:23:34.087 INFO:teuthology.orchestra.run.smithi184.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.22181.log.gz 2024-09-17T21:23:34.087 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22882.log 2024-09-17T21:23:34.087 INFO:teuthology.orchestra.run.smithi184.stderr: 52.9% -- replaced with /var/log/ceph/ceph-client.admin.22717.log.gz 2024-09-17T21:23:34.087 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23943.log 2024-09-17T21:23:34.087 INFO:teuthology.orchestra.run.smithi184.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.24540.log.gz 2024-09-17T21:23:34.087 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23900.log 2024-09-17T21:23:34.087 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.22882.log.gz 2024-09-17T21:23:34.087 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22952.log 2024-09-17T21:23:34.087 INFO:teuthology.orchestra.run.smithi184.stderr: 83.4% -- replaced with /var/log/ceph/ceph-client.admin.23943.log.gz 2024-09-17T21:23:34.088 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24284.log 2024-09-17T21:23:34.088 INFO:teuthology.orchestra.run.smithi184.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.23900.log.gz 2024-09-17T21:23:34.088 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22961.log 2024-09-17T21:23:34.088 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22952.log: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.22952.log.gz 2024-09-17T21:23:34.088 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24284.log: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.24284.log.gz 2024-09-17T21:23:34.088 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22401.log 2024-09-17T21:23:34.088 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22647.log 2024-09-17T21:23:34.088 INFO:teuthology.orchestra.run.smithi184.stderr: 56.2% -- replaced with /var/log/ceph/ceph-client.admin.22961.log.gz 2024-09-17T21:23:34.088 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22812.log 2024-09-17T21:23:34.088 INFO:teuthology.orchestra.run.smithi184.stderr: 56.6% -- replaced with /var/log/ceph/ceph-client.admin.22401.log.gz 2024-09-17T21:23:34.088 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22532.log 2024-09-17T21:23:34.088 INFO:teuthology.orchestra.run.smithi184.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.admin.22647.log.gz 2024-09-17T21:23:34.088 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21727.log 2024-09-17T21:23:34.088 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.22812.log.gz 2024-09-17T21:23:34.089 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22226.log 2024-09-17T21:23:34.089 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.22532.log.gz 2024-09-17T21:23:34.089 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.21727.log: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.21727.log.gz 2024-09-17T21:23:34.089 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22672.log 2024-09-17T21:23:34.089 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24574.log 2024-09-17T21:23:34.089 INFO:teuthology.orchestra.run.smithi184.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.22226.log.gz 2024-09-17T21:23:34.089 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22358.log 2024-09-17T21:23:34.089 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.22672.log.gz 2024-09-17T21:23:34.089 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24574.log: 52.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.24574.log.gz -5 2024-09-17T21:23:34.089 INFO:teuthology.orchestra.run.smithi184.stderr: --verbose -- /var/log/ceph/ceph-client.admin.24504.log 2024-09-17T21:23:34.089 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23101.log 2024-09-17T21:23:34.089 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.22358.log.gz 2024-09-17T21:23:34.089 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5/var/log/ceph/ceph-client.admin.24504.log: --verbose -- /var/log/ceph/ceph-client.admin.22120.log 2024-09-17T21:23:34.090 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23101.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.24504.log.gz 2024-09-17T21:23:34.090 INFO:teuthology.orchestra.run.smithi184.stderr:gzip 25.3% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.23101.log.gz -- 2024-09-17T21:23:34.090 INFO:teuthology.orchestra.run.smithi184.stderr: /var/log/ceph/ceph-client.admin.21690.log 2024-09-17T21:23:34.090 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23451.log 2024-09-17T21:23:34.090 INFO:teuthology.orchestra.run.smithi184.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.22120.log.gz 2024-09-17T21:23:34.090 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.21690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21971.log 2024-09-17T21:23:34.090 INFO:teuthology.orchestra.run.smithi184.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.21690.log.gz 2024-09-17T21:23:34.090 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23451.log: 75.3% -- replaced with /var/log/ceph/ceph-client.admin.23451.log.gz 2024-09-17T21:23:34.090 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23909.log 2024-09-17T21:23:34.090 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.21971.log: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.21971.log.gz 2024-09-17T21:23:34.090 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23723.log 2024-09-17T21:23:34.090 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23479.log 2024-09-17T21:23:34.090 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23241.log 2024-09-17T21:23:34.090 INFO:teuthology.orchestra.run.smithi184.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.23723.log.gz 2024-09-17T21:23:34.091 INFO:teuthology.orchestra.run.smithi184.stderr: 96.1% -- replaced with /var/log/ceph/ceph-client.admin.23909.log.gz 2024-09-17T21:23:34.091 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22507.log 2024-09-17T21:23:34.091 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.23479.log.gz 2024-09-17T21:23:34.091 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23698.log 2024-09-17T21:23:34.091 INFO:teuthology.orchestra.run.smithi184.stderr: 25.9% -- replaced with /var/log/ceph/ceph-client.admin.23241.log.gz 2024-09-17T21:23:34.091 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22147.log 2024-09-17T21:23:34.091 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23128.log 2024-09-17T21:23:34.091 INFO:teuthology.orchestra.run.smithi184.stderr: 83.6% -- replaced with /var/log/ceph/ceph-client.admin.23698.log.gz 2024-09-17T21:23:34.091 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.21538.log 2024-09-17T21:23:34.091 INFO:teuthology.orchestra.run.smithi184.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.22147.log.gz 2024-09-17T21:23:34.091 INFO:teuthology.orchestra.run.smithi184.stderr: 97.2% -- replaced with /var/log/ceph/ceph-client.admin.22507.log.gz 2024-09-17T21:23:34.091 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21937.log 2024-09-17T21:23:34.091 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.23128.log.gz 2024-09-17T21:23:34.092 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.0.21538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22927.log 2024-09-17T21:23:34.092 INFO:teuthology.orchestra.run.smithi184.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.0.21538.log.gz 2024-09-17T21:23:34.092 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.21937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22007.log 2024-09-17T21:23:34.092 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.21937.log.gz 2024-09-17T21:23:34.092 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23137.log 2024-09-17T21:23:34.092 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22927.log: /var/log/ceph/ceph-client.admin.22007.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.22927.log.gz 2024-09-17T21:23:34.092 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.22007.log.gz 2024-09-17T21:23:34.092 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24189.log 2024-09-17T21:23:34.092 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23338.log 2024-09-17T21:23:34.092 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23137.log: 53.0% -- replaced with /var/log/ceph/ceph-client.admin.23137.log.gz 2024-09-17T21:23:34.092 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24399.log 2024-09-17T21:23:34.092 INFO:teuthology.orchestra.run.smithi184.stderr: 53.2% -- replaced with /var/log/ceph/ceph-client.admin.24189.log.gz 2024-09-17T21:23:34.092 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22297.log 2024-09-17T21:23:34.093 INFO:teuthology.orchestra.run.smithi184.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.23338.log.gz 2024-09-17T21:23:34.093 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22016.log 2024-09-17T21:23:34.093 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24399.log: /var/log/ceph/ceph-client.admin.22297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21946.log 2024-09-17T21:23:34.093 INFO:teuthology.orchestra.run.smithi184.stderr: 95.4% -- replaced with /var/log/ceph/ceph-client.admin.24399.log.gz 2024-09-17T21:23:34.093 INFO:teuthology.orchestra.run.smithi184.stderr: 95.6% -- replaced with /var/log/ceph/ceph-client.admin.22297.log.gz 2024-09-17T21:23:34.093 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22638.log 2024-09-17T21:23:34.093 INFO:teuthology.orchestra.run.smithi184.stderr: 52.5% -- replaced with /var/log/ceph/ceph-client.admin.22016.log.gz 2024-09-17T21:23:34.093 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.21946.log: -- /var/log/ceph/ceph-client.admin.21665.log 2024-09-17T21:23:34.093 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22638.log: 96.0% -- replaced with /var/log/ceph/ceph-client.admin.21946.log.gz 2024-09-17T21:23:34.093 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.22638.log.gz 2024-09-17T21:23:34.093 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21797.log 2024-09-17T21:23:34.093 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.21665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24363.log 2024-09-17T21:23:34.093 INFO:teuthology.orchestra.run.smithi184.stderr: 55.6% -- replaced with /var/log/ceph/ceph-client.admin.21665.log.gz 2024-09-17T21:23:34.094 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23759.log 2024-09-17T21:23:34.094 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.21797.log: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.21797.log.gz 2024-09-17T21:23:34.094 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24363.log: 53.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.24363.log.gz 2024-09-17T21:23:34.094 INFO:teuthology.orchestra.run.smithi184.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.24153.log 2024-09-17T21:23:34.094 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24040.log 2024-09-17T21:23:34.094 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.23759.log.gz 2024-09-17T21:23:34.094 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24013.log 2024-09-17T21:23:34.094 INFO:teuthology.orchestra.run.smithi184.stderr: 52.2% -- replaced with /var/log/ceph/ceph-client.admin.24153.log.gz 2024-09-17T21:23:34.094 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24320.log 2024-09-17T21:23:34.094 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.24040.log.gz 2024-09-17T21:23:34.094 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22251.log 2024-09-17T21:23:34.094 INFO:teuthology.orchestra.run.smithi184.stderr: 54.6% -- replaced with /var/log/ceph/ceph-client.admin.24013.log.gz 2024-09-17T21:23:34.094 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24320.log: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.24320.log.gz 2024-09-17T21:23:34.094 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23162.log 2024-09-17T21:23:34.095 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24495.log 2024-09-17T21:23:34.095 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22251.log: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.22251.log.gz 2024-09-17T21:23:34.095 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23171.log 2024-09-17T21:23:34.095 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.23162.log.gz 2024-09-17T21:23:34.095 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21568.log 2024-09-17T21:23:34.095 INFO:teuthology.orchestra.run.smithi184.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.24495.log.gz 2024-09-17T21:23:34.095 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22857.log 2024-09-17T21:23:34.095 INFO:teuthology.orchestra.run.smithi184.stderr: 78.8% -- replaced with /var/log/ceph/ceph-client.admin.23171.log.gz 2024-09-17T21:23:34.095 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23558.log 2024-09-17T21:23:34.095 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.21568.log: /var/log/ceph/ceph-client.admin.22857.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.21568.log.gz 2024-09-17T21:23:34.095 INFO:teuthology.orchestra.run.smithi184.stderr:gzip 85.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.22857.log.gz --verbose 2024-09-17T21:23:34.095 INFO:teuthology.orchestra.run.smithi184.stderr: -- /var/log/ceph/ceph-client.admin.23513.log 2024-09-17T21:23:34.095 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21901.log 2024-09-17T21:23:34.096 INFO:teuthology.orchestra.run.smithi184.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.23558.log.gz 2024-09-17T21:23:34.096 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23839.log 2024-09-17T21:23:34.096 INFO:teuthology.orchestra.run.smithi184.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.23513.log.gz 2024-09-17T21:23:34.096 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.21901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22778.log 2024-09-17T21:23:34.096 INFO:teuthology.orchestra.run.smithi184.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.21901.log.gz 2024-09-17T21:23:34.096 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23934.log 2024-09-17T21:23:34.096 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23839.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.23839.log.gz 2024-09-17T21:23:34.096 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23583.log 2024-09-17T21:23:34.096 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22778.log: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.22778.log.gz 2024-09-17T21:23:34.096 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21840.log 2024-09-17T21:23:34.096 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.23934.log.gz 2024-09-17T21:23:34.096 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24110.log 2024-09-17T21:23:34.096 INFO:teuthology.orchestra.run.smithi184.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.23583.log.gz 2024-09-17T21:23:34.096 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.21840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22428.log 2024-09-17T21:23:34.097 INFO:teuthology.orchestra.run.smithi184.stderr: 52.7% -- replaced with /var/log/ceph/ceph-client.admin.21840.log.gz 2024-09-17T21:23:34.097 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22260.log 2024-09-17T21:23:34.097 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.24110.log.gz 2024-09-17T21:23:34.097 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22742.log 2024-09-17T21:23:34.097 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.22428.log.gz 2024-09-17T21:23:34.097 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.22260.log: -- /var/log/ceph/ceph-client.admin.24610.log 2024-09-17T21:23:34.097 INFO:teuthology.orchestra.run.smithi184.stderr: 75.9% -- replaced with /var/log/ceph/ceph-client.admin.22260.log.gz 2024-09-17T21:23:34.097 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22471.log 2024-09-17T21:23:34.097 INFO:teuthology.orchestra.run.smithi184.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.22742.log.gz 2024-09-17T21:23:34.097 INFO:teuthology.orchestra.run.smithi184.stderr:gzip/var/log/ceph/ceph-client.admin.24610.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.21831.log 2024-09-17T21:23:34.097 INFO:teuthology.orchestra.run.smithi184.stderr: 54.0% -- replaced with /var/log/ceph/ceph-client.admin.24610.log.gz 2024-09-17T21:23:34.097 INFO:teuthology.orchestra.run.smithi184.stderr:gzip/var/log/ceph/ceph-client.admin.22471.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.24259.log 2024-09-17T21:23:34.097 INFO:teuthology.orchestra.run.smithi184.stderr: 57.0% -- replaced with /var/log/ceph/ceph-client.admin.22471.log.gz 2024-09-17T21:23:34.098 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.21831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23022.log 2024-09-17T21:23:34.098 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.21831.log.gz 2024-09-17T21:23:34.098 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22848.log 2024-09-17T21:23:34.098 INFO:teuthology.orchestra.run.smithi184.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.24259.log.gz 2024-09-17T21:23:34.098 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22156.log 2024-09-17T21:23:34.098 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23022.log: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.23022.log.gz 2024-09-17T21:23:34.098 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22848.log: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.22848.log.gz 2024-09-17T21:23:34.098 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23031.log 2024-09-17T21:23:34.098 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21876.log 2024-09-17T21:23:34.098 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23031.log: 94.7% -- replaced with /var/log/ceph/ceph-client.admin.22156.log.gz 2024-09-17T21:23:34.098 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23302.log 2024-09-17T21:23:34.098 INFO:teuthology.orchestra.run.smithi184.stderr: 55.7% -- replaced with /var/log/ceph/ceph-client.admin.23031.log.gz 2024-09-17T21:23:34.098 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.21876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22891.log 2024-09-17T21:23:34.098 INFO:teuthology.orchestra.run.smithi184.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.21876.log.gz 2024-09-17T21:23:34.099 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23417.log 2024-09-17T21:23:34.099 INFO:teuthology.orchestra.run.smithi184.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.23302.log.gz 2024-09-17T21:23:34.099 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23592.log 2024-09-17T21:23:34.099 INFO:teuthology.orchestra.run.smithi184.stderr: 52.8% -- replaced with /var/log/ceph/ceph-client.admin.22891.log.gz 2024-09-17T21:23:34.099 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23689.log 2024-09-17T21:23:34.099 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24329.log 2024-09-17T21:23:34.099 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23592.log: 95.7% -- replaced with /var/log/ceph/ceph-client.admin.23417.log.gz 2024-09-17T21:23:34.099 INFO:teuthology.orchestra.run.smithi184.stderr: 52.5% -- replaced with /var/log/ceph/ceph-client.admin.23592.log.gz 2024-09-17T21:23:34.099 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22437.log 2024-09-17T21:23:34.099 INFO:teuthology.orchestra.run.smithi184.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.23689.log.gz 2024-09-17T21:23:34.099 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24424.log 2024-09-17T21:23:34.099 INFO:teuthology.orchestra.run.smithi184.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.24329.log.gz 2024-09-17T21:23:34.100 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21910.log 2024-09-17T21:23:34.100 INFO:teuthology.orchestra.run.smithi184.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.admin.22437.log.gz/var/log/ceph/ceph-client.admin.24424.log: 2024-09-17T21:23:34.100 INFO:teuthology.orchestra.run.smithi184.stderr:gzip 20.9% -- replaced with /var/log/ceph/ceph-client.admin.24424.log.gz -5 2024-09-17T21:23:34.100 INFO:teuthology.orchestra.run.smithi184.stderr: --verbose -- /var/log/ceph/ceph-client.admin.22681.log 2024-09-17T21:23:34.100 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.21910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21770.log 2024-09-17T21:23:34.100 INFO:teuthology.orchestra.run.smithi184.stderr: 56.0% -- replaced with /var/log/ceph/ceph-client.admin.21910.log.gz 2024-09-17T21:23:34.100 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22787.log 2024-09-17T21:23:34.100 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22681.log: 56.9% -- replaced with /var/log/ceph/ceph-client.admin.22681.log.gz 2024-09-17T21:23:34.100 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.21770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23092.log 2024-09-17T21:23:34.100 INFO:teuthology.orchestra.run.smithi184.stderr: 51.9% -- replaced with /var/log/ceph/ceph-client.admin.21770.log.gz 2024-09-17T21:23:34.100 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22577.log 2024-09-17T21:23:34.100 INFO:teuthology.orchestra.run.smithi184.stderr:gzip/var/log/ceph/ceph-client.admin.23092.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.22050.log 2024-09-17T21:23:34.100 INFO:teuthology.orchestra.run.smithi184.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.23092.log.gz 2024-09-17T21:23:34.100 INFO:teuthology.orchestra.run.smithi184.stderr: 96.4%/var/log/ceph/ceph-client.admin.22577.log: -- replaced with /var/log/ceph/ceph-client.admin.22787.log.gz 2024-09-17T21:23:34.101 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22190.log 2024-09-17T21:23:34.101 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22322.log 2024-09-17T21:23:34.101 INFO:teuthology.orchestra.run.smithi184.stderr: 54.5% -- replaced with /var/log/ceph/ceph-client.admin.22050.log.gz 2024-09-17T21:23:34.101 INFO:teuthology.orchestra.run.smithi184.stderr: 96.7% -- replaced with /var/log/ceph/ceph-client.admin.22577.log.gz 2024-09-17T21:23:34.101 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24531.log 2024-09-17T21:23:34.101 INFO:teuthology.orchestra.run.smithi184.stderr: 54.2% -- replaced with /var/log/ceph/ceph-client.admin.22190.log.gz 2024-09-17T21:23:34.101 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23619.log 2024-09-17T21:23:34.101 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22322.log: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.22322.log.gz 2024-09-17T21:23:34.101 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24531.log: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.24531.log.gz 2024-09-17T21:23:34.101 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23207.log 2024-09-17T21:23:34.101 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24144.log 2024-09-17T21:23:34.101 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.23619.log.gz 2024-09-17T21:23:34.101 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21736.log 2024-09-17T21:23:34.102 INFO:teuthology.orchestra.run.smithi184.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.23207.log.gz 2024-09-17T21:23:34.102 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24144.log: 20.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.24144.log.gz 2024-09-17T21:23:34.102 INFO:teuthology.orchestra.run.smithi184.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.24601.log 2024-09-17T21:23:34.102 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.21736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23979.log 2024-09-17T21:23:34.102 INFO:teuthology.orchestra.run.smithi184.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.21736.log.gz 2024-09-17T21:23:34.102 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24565.log 2024-09-17T21:23:34.102 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.24601.log.gz 2024-09-17T21:23:34.102 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23311.log 2024-09-17T21:23:34.102 INFO:teuthology.orchestra.run.smithi184.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.23979.log.gz 2024-09-17T21:23:34.102 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23970.log 2024-09-17T21:23:34.102 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.24565.log.gz 2024-09-17T21:23:34.102 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23067.log 2024-09-17T21:23:34.102 INFO:teuthology.orchestra.run.smithi184.stderr: 72.7% -- replaced with /var/log/ceph/ceph-client.admin.23311.log.gz 2024-09-17T21:23:34.102 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23970.log: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.23970.log.gz 2024-09-17T21:23:34.103 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22541.log 2024-09-17T21:23:34.103 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22086.log 2024-09-17T21:23:34.103 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22602.log 2024-09-17T21:23:34.103 INFO:teuthology.orchestra.run.smithi184.stderr: 56.5% -- replaced with /var/log/ceph/ceph-client.admin.22541.log.gz 2024-09-17T21:23:34.103 INFO:teuthology.orchestra.run.smithi184.stderr: 95.8% -- replaced with /var/log/ceph/ceph-client.admin.23067.log.gz 2024-09-17T21:23:34.103 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22086.log: gzip 94.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.22086.log.gz -- 2024-09-17T21:23:34.103 INFO:teuthology.orchestra.run.smithi184.stderr: /var/log/ceph/ceph-client.admin.22288.log 2024-09-17T21:23:34.103 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23628.log 2024-09-17T21:23:34.103 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22602.log: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.22602.log.gz 2024-09-17T21:23:34.103 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22611.log 2024-09-17T21:23:34.103 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22288.log: /var/log/ceph/ceph-client.admin.23628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21621.log 2024-09-17T21:23:34.103 INFO:teuthology.orchestra.run.smithi184.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.22288.log.gz 2024-09-17T21:23:34.103 INFO:teuthology.orchestra.run.smithi184.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.23628.log.gz 2024-09-17T21:23:34.104 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23232.log 2024-09-17T21:23:34.104 INFO:teuthology.orchestra.run.smithi184.stderr: 73.9% -- replaced with /var/log/ceph/ceph-client.admin.22611.log.gz 2024-09-17T21:23:34.104 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.21621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24250.log 2024-09-17T21:23:34.104 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.21621.log.gz 2024-09-17T21:23:34.104 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22217.log 2024-09-17T21:23:34.104 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23232.log: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.23232.log.gz 2024-09-17T21:23:34.104 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22988.log 2024-09-17T21:23:34.104 INFO:teuthology.orchestra.run.smithi184.stderr: 33.5% -- replaced with /var/log/ceph/ceph-client.admin.24250.log.gz 2024-09-17T21:23:34.104 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22997.log 2024-09-17T21:23:34.104 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.22217.log.gz 2024-09-17T21:23:34.104 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23768.log 2024-09-17T21:23:34.104 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.22988.log.gz 2024-09-17T21:23:34.104 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23408.log 2024-09-17T21:23:34.105 INFO:teuthology.orchestra.run.smithi184.stderr: 94.5% -- replaced with /var/log/ceph/ceph-client.admin.22997.log.gz 2024-09-17T21:23:34.105 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24004.log 2024-09-17T21:23:34.105 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22708.log 2024-09-17T21:23:34.105 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23408.log: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.23408.log.gz 2024-09-17T21:23:34.105 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21656.log 2024-09-17T21:23:34.105 INFO:teuthology.orchestra.run.smithi184.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.24004.log.gz 95.6% 2024-09-17T21:23:34.105 INFO:teuthology.orchestra.run.smithi184.stderr: -- replaced with /var/log/ceph/ceph-client.admin.23768.log.gz 2024-09-17T21:23:34.105 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23381.log 2024-09-17T21:23:34.105 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.22708.log.gz 2024-09-17T21:23:34.105 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.21656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21699.log 2024-09-17T21:23:34.105 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.21656.log.gz 2024-09-17T21:23:34.105 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23381.log: 52.2% -- replaced with /var/log/ceph/ceph-client.admin.23381.log.gz 2024-09-17T21:23:34.105 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23662.log 2024-09-17T21:23:34.105 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24074.log 2024-09-17T21:23:34.106 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.21699.log: 56.4% -- replaced with /var/log/ceph/ceph-client.admin.21699.log.gzgzip 2024-09-17T21:23:34.106 INFO:teuthology.orchestra.run.smithi184.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.23268.log 2024-09-17T21:23:34.106 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23662.log: 52.9% -- replaced with /var/log/ceph/ceph-client.admin.23662.log.gz 2024-09-17T21:23:34.106 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22367.log 2024-09-17T21:23:34.106 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.24074.log.gz 2024-09-17T21:23:34.106 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22077.log 2024-09-17T21:23:34.106 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.23268.log.gz 2024-09-17T21:23:34.106 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22821.log 2024-09-17T21:23:34.106 INFO:teuthology.orchestra.run.smithi184.stderr: 53.4% -- replaced with /var/log/ceph/ceph-client.admin.22367.log.gz 2024-09-17T21:23:34.106 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22077.log: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.22077.log.gz 2024-09-17T21:23:34.106 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23277.log 2024-09-17T21:23:34.106 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21761.log 2024-09-17T21:23:34.106 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22821.log: 57.6% -- replaced with /var/log/ceph/ceph-client.admin.22821.log.gz 2024-09-17T21:23:34.107 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22568.log 2024-09-17T21:23:34.107 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23277.log: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.23277.log.gz 2024-09-17T21:23:34.107 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.21761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23488.log 2024-09-17T21:23:34.107 INFO:teuthology.orchestra.run.smithi184.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.21761.log.gz 2024-09-17T21:23:34.107 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24433.log 2024-09-17T21:23:34.107 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.22568.log.gz 2024-09-17T21:23:34.107 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23653.log 2024-09-17T21:23:34.107 INFO:teuthology.orchestra.run.smithi184.stderr: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.23488.log.gz 2024-09-17T21:23:34.107 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22462.log 2024-09-17T21:23:34.107 INFO:teuthology.orchestra.run.smithi184.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.24433.log.gz 2024-09-17T21:23:34.107 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22041.log 2024-09-17T21:23:34.107 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23653.log: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.23653.log.gz 2024-09-17T21:23:34.107 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23830.log 2024-09-17T21:23:34.107 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22462.log: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.22462.log.gz 2024-09-17T21:23:34.108 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23442.log 2024-09-17T21:23:34.108 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.22041.log.gz 2024-09-17T21:23:34.108 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24083.log 2024-09-17T21:23:34.108 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.23830.log.gz 2024-09-17T21:23:34.108 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23864.log 2024-09-17T21:23:34.108 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.23442.log.gz 2024-09-17T21:23:34.108 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24083.log: 57.2% -- replaced with /var/log/ceph/ceph-client.admin.24083.log.gz 2024-09-17T21:23:34.108 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22498.log 2024-09-17T21:23:34.108 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21806.log 2024-09-17T21:23:34.108 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23864.log: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.23864.log.gz 2024-09-17T21:23:34.108 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24180.log 2024-09-17T21:23:34.108 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22498.log: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.22498.log.gz 2024-09-17T21:23:34.108 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.21806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23198.log 2024-09-17T21:23:34.108 INFO:teuthology.orchestra.run.smithi184.stderr: 95.0% -- replaced with /var/log/ceph/ceph-client.admin.21806.log.gz 2024-09-17T21:23:34.109 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24354.log 2024-09-17T21:23:34.109 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.24180.log.gz 2024-09-17T21:23:34.109 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22392.log 2024-09-17T21:23:34.109 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.23198.log.gz 2024-09-17T21:23:34.109 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24354.log: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.24354.log.gz 2024-09-17T21:23:34.109 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24461.log 2024-09-17T21:23:34.109 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22918.log 2024-09-17T21:23:34.109 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22392.log: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.22392.log.gz 2024-09-17T21:23:34.109 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22331.log 2024-09-17T21:23:34.109 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24461.log: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.24461.log.gz 2024-09-17T21:23:34.109 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22751.log 2024-09-17T21:23:34.109 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22918.log: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.22918.log.gz 2024-09-17T21:23:34.109 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23522.log 2024-09-17T21:23:34.109 INFO:teuthology.orchestra.run.smithi184.stderr: 72.3% -- replaced with /var/log/ceph/ceph-client.admin.22331.log.gz 2024-09-17T21:23:34.110 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23347.log 2024-09-17T21:23:34.110 INFO:teuthology.orchestra.run.smithi184.stderr: 52.8% -- replaced with /var/log/ceph/ceph-client.admin.22751.log.gz 2024-09-17T21:23:34.110 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23522.log: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.23522.log.gz 2024-09-17T21:23:34.110 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23549.log 2024-09-17T21:23:34.110 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21559.log 2024-09-17T21:23:34.110 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21867.log 2024-09-17T21:23:34.110 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23549.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.23347.log.gz 2024-09-17T21:23:34.110 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.23549.log.gz 2024-09-17T21:23:34.110 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.21559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23802.log 2024-09-17T21:23:34.110 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.21559.log.gz 2024-09-17T21:23:34.110 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.21867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22111.log 2024-09-17T21:23:34.110 INFO:teuthology.orchestra.run.smithi184.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.21867.log.gz 2024-09-17T21:23:34.110 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23372.log 2024-09-17T21:23:34.111 INFO:teuthology.orchestra.run.smithi184.stderr: 55.8% -- replaced with /var/log/ceph/ceph-client.admin.23802.log.gz 2024-09-17T21:23:34.111 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.22111.log: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.22111.log.gz 2024-09-17T21:23:34.111 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21980.log 2024-09-17T21:23:34.111 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.23372.log: gzip -5 --verbose -- 20.2% /var/log/ceph/ceph-client.admin.24293.log -- replaced with /var/log/ceph/ceph-client.admin.23372.log.gz 2024-09-17T21:23:34.111 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-09-17T21:23:34.111 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.21980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24214.log 2024-09-17T21:23:34.111 INFO:teuthology.orchestra.run.smithi184.stderr: 76.2% -- replaced with /var/log/ceph/ceph-client.admin.21980.log.gz 2024-09-17T21:23:34.111 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24293.log: 56.8% -- replaced with /var/log/ceph/ceph-client.admin.24293.log.gz 2024-09-17T21:23:34.111 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.24214.log: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.24214.log.gz 2024-09-17T21:23:34.111 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-09-17T21:23:34.111 INFO:teuthology.orchestra.run.smithi184.stderr:real 0m0.082s 2024-09-17T21:23:34.111 INFO:teuthology.orchestra.run.smithi184.stderr:user 0m0.135s 2024-09-17T21:23:34.111 INFO:teuthology.orchestra.run.smithi184.stderr:sys 0m0.055s 2024-09-17T21:23:34.148 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32293.log 2024-09-17T21:23:34.149 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32008.log.gz 2024-09-17T21:23:34.149 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36606.log 2024-09-17T21:23:34.149 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42864.log.gz 2024-09-17T21:23:34.149 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42106.log 2024-09-17T21:23:34.149 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32293.log.gz 2024-09-17T21:23:34.149 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29930.log 2024-09-17T21:23:34.149 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36606.log.gz 2024-09-17T21:23:34.149 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42106.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42106.log.gz 2024-09-17T21:23:34.149 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33133.log 2024-09-17T21:23:34.149 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40675.log 2024-09-17T21:23:34.149 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29930.log: /var/log/ceph/ceph-client.admin.33133.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29930.log.gz 2024-09-17T21:23:34.149 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29890.log 2024-09-17T21:23:34.150 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33133.log.gz 2024-09-17T21:23:34.150 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29587.log 2024-09-17T21:23:34.150 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40675.log.gz 2024-09-17T21:23:34.150 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34148.log 2024-09-17T21:23:34.150 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29890.log.gz 2024-09-17T21:23:34.150 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30315.log 2024-09-17T21:23:34.150 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29587.log.gz 2024-09-17T21:23:34.150 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43510.log 2024-09-17T21:23:34.150 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34148.log.gz 2024-09-17T21:23:34.150 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.30315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33444.log 2024-09-17T21:23:34.150 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30315.log.gz 2024-09-17T21:23:34.150 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35046.log 2024-09-17T21:23:34.150 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43510.log.gz 2024-09-17T21:23:34.150 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.33444.log: /var/log/ceph/ceph-client.admin.32356.log 2024-09-17T21:23:34.151 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33444.log.gz 2024-09-17T21:23:34.151 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35237.log 2024-09-17T21:23:34.151 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35046.log.gz 2024-09-17T21:23:34.151 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36094.log 2024-09-17T21:23:34.151 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32356.log.gz 2024-09-17T21:23:34.151 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35503.log 2024-09-17T21:23:34.151 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35237.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35237.log.gz 2024-09-17T21:23:34.151 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36094.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36094.log.gz 2024-09-17T21:23:34.151 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32971.log 2024-09-17T21:23:34.151 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42888.log 2024-09-17T21:23:34.151 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35503.log.gz 2024-09-17T21:23:34.151 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25119.log 2024-09-17T21:23:34.152 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32971.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32971.log.gz 2024-09-17T21:23:34.152 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32742.log 2024-09-17T21:23:34.152 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42888.log.gz 2024-09-17T21:23:34.152 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.25119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34064.log 2024-09-17T21:23:34.152 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25119.log.gz 2024-09-17T21:23:34.152 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39601.log 2024-09-17T21:23:34.152 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32742.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32742.log.gz 2024-09-17T21:23:34.152 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36293.log 2024-09-17T21:23:34.152 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34064.log.gz 2024-09-17T21:23:34.152 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36767.log 2024-09-17T21:23:34.152 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39601.log.gz 2024-09-17T21:23:34.152 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42704.log 2024-09-17T21:23:34.153 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36293.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36293.log.gz 2024-09-17T21:23:34.153 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36767.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21520.log 2024-09-17T21:23:34.153 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36767.log.gz 2024-09-17T21:23:34.153 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38287.log 2024-09-17T21:23:34.153 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42704.log.gz 2024-09-17T21:23:34.153 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph.tmp-client.admin.21520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36867.log 2024-09-17T21:23:34.153 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21520.log.gz 2024-09-17T21:23:34.153 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29112.log 2024-09-17T21:23:34.153 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38287.log.gz 2024-09-17T21:23:34.153 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36867.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-17T21:23:34.153 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36867.log.gz 2024-09-17T21:23:34.153 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42940.log 2024-09-17T21:23:34.153 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29112.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29112.log.gz 2024-09-17T21:23:34.153 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29834.log 2024-09-17T21:23:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42366.log 2024-09-17T21:23:34.154 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42940.log.gz 2024-09-17T21:23:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32907.log 2024-09-17T21:23:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29834.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29834.log.gz 2024-09-17T21:23:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32194.log 2024-09-17T21:23:34.154 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42366.log.gz 2024-09-17T21:23:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32907.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.32907.log.gz -5 2024-09-17T21:23:34.154 INFO:teuthology.orchestra.run.smithi083.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36581.log 2024-09-17T21:23:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37875.log 2024-09-17T21:23:34.154 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32194.log.gz 2024-09-17T21:23:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36581.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36581.log.gz 2024-09-17T21:23:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32107.log 2024-09-17T21:23:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40283.log 2024-09-17T21:23:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35701.log 2024-09-17T21:23:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32107.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37875.log.gz 2024-09-17T21:23:34.155 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32107.log.gz 2024-09-17T21:23:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40283.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40283.log.gz 2024-09-17T21:23:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33256.log 2024-09-17T21:23:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31465.log 2024-09-17T21:23:34.155 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35701.log.gz 2024-09-17T21:23:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33792.log 2024-09-17T21:23:34.155 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33256.log.gz 2024-09-17T21:23:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36555.log 2024-09-17T21:23:34.155 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31465.log.gz 2024-09-17T21:23:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29395.log 2024-09-17T21:23:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33792.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33792.log.gz 2024-09-17T21:23:34.156 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40641.log 2024-09-17T21:23:34.156 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36555.log.gz 2024-09-17T21:23:34.156 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43146.log 2024-09-17T21:23:34.156 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29395.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29395.log.gz 2024-09-17T21:23:34.156 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37667.log/var/log/ceph/ceph-client.admin.40641.log: 2024-09-17T21:23:34.156 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40641.log.gz 2024-09-17T21:23:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40163.log 2024-09-17T21:23:34.157 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43146.log.gz 2024-09-17T21:23:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38138.log 2024-09-17T21:23:34.157 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37667.log.gz 2024-09-17T21:23:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29548.log 2024-09-17T21:23:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40163.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40163.log.gz 2024-09-17T21:23:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25237.log 2024-09-17T21:23:34.157 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38138.log.gz 2024-09-17T21:23:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36189.log 2024-09-17T21:23:34.157 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29548.log.gz 2024-09-17T21:23:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:gzip/var/log/ceph/ceph-client.admin.25237.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34172.log 2024-09-17T21:23:34.157 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25237.log.gz 2024-09-17T21:23:34.158 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40480.log 2024-09-17T21:23:34.158 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36189.log.gz 2024-09-17T21:23:34.158 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30432.log 2024-09-17T21:23:34.158 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34172.log.gz 2024-09-17T21:23:34.158 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43406.log 2024-09-17T21:23:34.158 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40480.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40480.log.gz 2024-09-17T21:23:34.158 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.30432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43328.log 2024-09-17T21:23:34.158 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30432.log.gz 2024-09-17T21:23:34.158 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29184.log 2024-09-17T21:23:34.158 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43406.log.gz 2024-09-17T21:23:34.158 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42184.log 2024-09-17T21:23:34.158 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43328.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43328.log.gz 2024-09-17T21:23:34.158 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38960.log 2024-09-17T21:23:34.159 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29184.log.gz 2024-09-17T21:23:34.159 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41115.log 2024-09-17T21:23:34.159 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42184.log.gz 2024-09-17T21:23:34.159 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42210.log 2024-09-17T21:23:34.159 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38960.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38960.log.gz 2024-09-17T21:23:34.159 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31893.log 2024-09-17T21:23:34.159 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41115.log.gz 2024-09-17T21:23:34.159 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43588.log 2024-09-17T21:23:34.159 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42210.log.gz 2024-09-17T21:23:34.159 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38011.log 2024-09-17T21:23:34.159 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31893.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31893.log.gz 2024-09-17T21:23:34.160 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35137.log 2024-09-17T21:23:34.160 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43588.log.gz 2024-09-17T21:23:34.160 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30130.log 2024-09-17T21:23:34.160 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38011.log.gz 2024-09-17T21:23:34.160 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43198.log 2024-09-17T21:23:34.160 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35137.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35137.log.gz 2024-09-17T21:23:34.160 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.30130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29378.log 2024-09-17T21:23:34.160 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30130.log.gz 2024-09-17T21:23:34.160 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43198.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-17T21:23:34.160 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43198.log.gz 2024-09-17T21:23:34.160 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34639.log 2024-09-17T21:23:34.161 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29378.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29378.log.gz 2024-09-17T21:23:34.161 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29860.log 2024-09-17T21:23:34.161 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43068.log 2024-09-17T21:23:34.161 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34639.log.gz 2024-09-17T21:23:34.161 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30772.log 2024-09-17T21:23:34.161 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29860.log.gz 2024-09-17T21:23:34.161 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42158.log 2024-09-17T21:23:34.161 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43068.log.gz 2024-09-17T21:23:34.161 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.30772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35430.log 2024-09-17T21:23:34.161 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30772.log.gz 2024-09-17T21:23:34.161 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34643.log 2024-09-17T21:23:34.161 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42158.log.gz 2024-09-17T21:23:34.162 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38188.log 2024-09-17T21:23:34.162 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35430.log.gz 2024-09-17T21:23:34.162 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34643.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.34643.log.gz -5 2024-09-17T21:23:34.162 INFO:teuthology.orchestra.run.smithi083.stderr: --verbose -- /var/log/ceph/ceph-client.admin.39804.log 2024-09-17T21:23:34.162 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36896.log 2024-09-17T21:23:34.162 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38188.log.gz 2024-09-17T21:23:34.162 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42407.log 2024-09-17T21:23:34.162 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39804.log.gz 2024-09-17T21:23:34.162 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32479.log 2024-09-17T21:23:34.162 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36896.log.gz 2024-09-17T21:23:34.163 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29256.log 2024-09-17T21:23:34.163 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42407.log.gz 2024-09-17T21:23:34.163 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40198.log 2024-09-17T21:23:34.163 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32479.log.gz 2024-09-17T21:23:34.163 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34247.log 2024-09-17T21:23:34.163 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29256.log.gz 2024-09-17T21:23:34.163 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29401.log 2024-09-17T21:23:34.163 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40198.log.gz 2024-09-17T21:23:34.163 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40762.log 2024-09-17T21:23:34.163 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34247.log.gz 2024-09-17T21:23:34.163 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37493.log 2024-09-17T21:23:34.164 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29401.log.gz 2024-09-17T21:23:34.164 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40762.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-17T21:23:34.164 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40762.log.gz 2024-09-17T21:23:34.164 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38365.log 2024-09-17T21:23:34.164 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37493.log.gz 2024-09-17T21:23:34.164 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31954.log 2024-09-17T21:23:34.164 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38365.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38365.log.gz 2024-09-17T21:23:34.164 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29588.log 2024-09-17T21:23:34.164 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42708.log 2024-09-17T21:23:34.164 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31954.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31954.log.gz 2024-09-17T21:23:34.164 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40398.log 2024-09-17T21:23:34.165 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29588.log.gz 2024-09-17T21:23:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37418.log 2024-09-17T21:23:34.165 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42708.log.gz 2024-09-17T21:23:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39167.log 2024-09-17T21:23:34.165 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40398.log.gz 2024-09-17T21:23:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32941.log 2024-09-17T21:23:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37418.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37418.log.gz 2024-09-17T21:23:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43276.log 2024-09-17T21:23:34.165 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39167.log.gz 2024-09-17T21:23:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39309.log 2024-09-17T21:23:34.165 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32941.log.gz 2024-09-17T21:23:34.166 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34102.log 2024-09-17T21:23:34.166 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43276.log.gz 2024-09-17T21:23:34.166 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39918.log 2024-09-17T21:23:34.166 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39309.log.gz 2024-09-17T21:23:34.166 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36967.log 2024-09-17T21:23:34.166 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34102.log.gz 2024-09-17T21:23:34.166 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42760.log 2024-09-17T21:23:34.166 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39918.log.gz 2024-09-17T21:23:34.166 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34460.log 2024-09-17T21:23:34.166 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36967.log.gz 2024-09-17T21:23:34.166 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30630.log 2024-09-17T21:23:34.167 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42760.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42760.log.gz 2024-09-17T21:23:34.167 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29251.log 2024-09-17T21:23:34.167 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34460.log.gz 2024-09-17T21:23:34.167 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.30630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35910.log 2024-09-17T21:23:34.167 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30630.log.gz 2024-09-17T21:23:34.167 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36033.log 2024-09-17T21:23:34.167 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29251.log.gz 2024-09-17T21:23:34.167 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42736.log 2024-09-17T21:23:34.167 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35910.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35910.log.gz 2024-09-17T21:23:34.167 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33020.log 2024-09-17T21:23:34.167 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36033.log.gz 2024-09-17T21:23:34.167 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32591.log 2024-09-17T21:23:34.168 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42736.log.gz 2024-09-17T21:23:34.168 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29808.log 2024-09-17T21:23:34.168 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33020.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33020.log.gz 2024-09-17T21:23:34.168 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35402.log 2024-09-17T21:23:34.168 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32591.log.gz 2024-09-17T21:23:34.168 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34382.log 2024-09-17T21:23:34.168 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29808.log.gz 2024-09-17T21:23:34.168 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32655.log 2024-09-17T21:23:34.168 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35402.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35402.log.gz 2024-09-17T21:23:34.168 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39641.log 2024-09-17T21:23:34.168 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34382.log.gz 2024-09-17T21:23:34.168 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32402.log 2024-09-17T21:23:34.168 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32655.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32655.log.gz 2024-09-17T21:23:34.169 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31079.log 2024-09-17T21:23:34.169 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39641.log.gz 2024-09-17T21:23:34.169 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33746.log 2024-09-17T21:23:34.169 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32402.log.gz 2024-09-17T21:23:34.169 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39017.log 2024-09-17T21:23:34.169 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31079.log.gz 2024-09-17T21:23:34.169 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36842.log 2024-09-17T21:23:34.169 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33746.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33746.log.gz 2024-09-17T21:23:34.169 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43120.log 2024-09-17T21:23:34.169 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39017.log.gz 2024-09-17T21:23:34.169 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43250.log 2024-09-17T21:23:34.169 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36842.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36842.log.gz 2024-09-17T21:23:34.170 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42288.log 2024-09-17T21:23:34.170 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43120.log.gz 2024-09-17T21:23:34.170 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37254.log 2024-09-17T21:23:34.170 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43250.log.gz 2024-09-17T21:23:34.170 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40735.log 2024-09-17T21:23:34.170 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42288.log.gz 2024-09-17T21:23:34.170 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37908.log 2024-09-17T21:23:34.170 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37254.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37254.log.gz 2024-09-17T21:23:34.170 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38449.log 2024-09-17T21:23:34.170 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40735.log.gz 2024-09-17T21:23:34.170 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30009.log 2024-09-17T21:23:34.170 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37908.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37908.log.gz 2024-09-17T21:23:34.170 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34377.log 2024-09-17T21:23:34.171 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38449.log.gz 2024-09-17T21:23:34.171 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.30009.log: -- /var/log/ceph/ceph-client.admin.42912.log 2024-09-17T21:23:34.171 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30009.log.gz 2024-09-17T21:23:34.171 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42561.log 2024-09-17T21:23:34.171 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34377.log.gz 2024-09-17T21:23:34.171 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34487.log 2024-09-17T21:23:34.171 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42912.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42912.log.gz 2024-09-17T21:23:34.171 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34347.log 2024-09-17T21:23:34.171 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42561.log.gz 2024-09-17T21:23:34.171 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34284.log 2024-09-17T21:23:34.171 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34487.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34487.log.gz 2024-09-17T21:23:34.172 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35954.log 2024-09-17T21:23:34.172 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34347.log.gz 2024-09-17T21:23:34.172 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37614.log 2024-09-17T21:23:34.172 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34284.log.gz 2024-09-17T21:23:34.172 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42236.log 2024-09-17T21:23:34.172 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35954.log.gz 2024-09-17T21:23:34.172 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43224.log 2024-09-17T21:23:34.172 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37614.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37614.log.gz 2024-09-17T21:23:34.172 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42860.log 2024-09-17T21:23:34.172 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42236.log.gz 2024-09-17T21:23:34.172 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33659.log 2024-09-17T21:23:34.173 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43224.log.gz 2024-09-17T21:23:34.173 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40078.log 2024-09-17T21:23:34.173 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42860.log.gz 2024-09-17T21:23:34.173 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36132.log 2024-09-17T21:23:34.173 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33659.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33659.log.gz 2024-09-17T21:23:34.173 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39221.log 2024-09-17T21:23:34.173 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40078.log.gz 2024-09-17T21:23:34.173 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30074.log 2024-09-17T21:23:34.173 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36132.log.gz 2024-09-17T21:23:34.173 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34988.log 2024-09-17T21:23:34.173 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39221.log.gz 2024-09-17T21:23:34.173 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.30074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38412.log 2024-09-17T21:23:34.174 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30074.log.gz 2024-09-17T21:23:34.174 INFO:teuthology.orchestra.run.smithi083.stderr:gzip/var/log/ceph/ceph-client.admin.34988.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29704.log 2024-09-17T21:23:34.174 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34988.log.gz 2024-09-17T21:23:34.174 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29253.log 0.0% 2024-09-17T21:23:34.174 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38412.log.gz 2024-09-17T21:23:34.174 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38100.log 2024-09-17T21:23:34.174 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29704.log.gz 2024-09-17T21:23:34.174 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42411.log 2024-09-17T21:23:34.174 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29253.log.gz 2024-09-17T21:23:34.174 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29404.log 2024-09-17T21:23:34.174 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38100.log.gz 2024-09-17T21:23:34.174 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37758.log 2024-09-17T21:23:34.174 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42411.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42411.log.gz 2024-09-17T21:23:34.175 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37129.log 2024-09-17T21:23:34.175 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29404.log.gz 2024-09-17T21:23:34.175 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35550.log 2024-09-17T21:23:34.175 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37758.log.gz 2024-09-17T21:23:34.175 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37974.log 2024-09-17T21:23:34.175 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37129.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37129.log.gz 2024-09-17T21:23:34.175 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42412.log 2024-09-17T21:23:34.175 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35550.log.gz 2024-09-17T21:23:34.175 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42409.log 2024-09-17T21:23:34.175 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37974.log.gz 2024-09-17T21:23:34.175 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39968.log 2024-09-17T21:23:34.175 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42412.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42412.log.gz 2024-09-17T21:23:34.176 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42712.log 2024-09-17T21:23:34.176 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42409.log.gz 2024-09-17T21:23:34.176 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42410.log 2024-09-17T21:23:34.176 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39968.log.gz 2024-09-17T21:23:34.176 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29255.log 2024-09-17T21:23:34.176 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42712.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42712.log.gz 2024-09-17T21:23:34.176 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38809.log 2024-09-17T21:23:34.176 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42410.log.gz 2024-09-17T21:23:34.176 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35311.log 2024-09-17T21:23:34.177 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29255.log.gz 2024-09-17T21:23:34.177 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42054.log 2024-09-17T21:23:34.177 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38809.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38809.log.gz 2024-09-17T21:23:34.177 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31782.log 2024-09-17T21:23:34.177 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35311.log.gz 2024-09-17T21:23:34.177 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30463.log 2024-09-17T21:23:34.177 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42054.log.gz 2024-09-17T21:23:34.177 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35087.log 2024-09-17T21:23:34.177 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31782.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31782.log.gz 2024-09-17T21:23:34.177 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.30463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30266.log 2024-09-17T21:23:34.177 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30463.log.gz 2024-09-17T21:23:34.178 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36268.log 2024-09-17T21:23:34.178 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35087.log.gz 2024-09-17T21:23:34.178 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34822.log 2024-09-17T21:23:34.178 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.30266.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30266.log.gz 2024-09-17T21:23:34.178 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39753.log 2024-09-17T21:23:34.178 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36268.log.gz 2024-09-17T21:23:34.178 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38860.log 2024-09-17T21:23:34.178 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34822.log.gz 2024-09-17T21:23:34.178 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29087.log 2024-09-17T21:23:34.178 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39753.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39753.log.gz 2024-09-17T21:23:34.179 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31017.log 2024-09-17T21:23:34.179 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38860.log.gz 2024-09-17T21:23:34.179 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29087.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-17T21:23:34.179 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29087.log.gz 2024-09-17T21:23:34.179 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38065.log 2024-09-17T21:23:34.179 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31017.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31017.log.gz 2024-09-17T21:23:34.179 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42552.log 2024-09-17T21:23:34.179 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40368.log 2024-09-17T21:23:34.179 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38065.log.gz 2024-09-17T21:23:34.179 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30955.log 2024-09-17T21:23:34.179 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42552.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42552.log.gz 2024-09-17T21:23:34.179 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38548.log 2024-09-17T21:23:34.180 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40368.log.gz 2024-09-17T21:23:34.180 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.30955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30771.log 2024-09-17T21:23:34.180 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30955.log.gz 2024-09-17T21:23:34.180 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34880.log 2024-09-17T21:23:34.180 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38548.log.gz 2024-09-17T21:23:34.180 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33820.log 2024-09-17T21:23:34.180 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.30771.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30771.log.gz 2024-09-17T21:23:34.180 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36393.log 2024-09-17T21:23:34.180 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34880.log.gz 2024-09-17T21:23:34.180 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42080.log 2024-09-17T21:23:34.180 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33820.log.gz 2024-09-17T21:23:34.181 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37547.log 2024-09-17T21:23:34.181 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36393.log.gz 2024-09-17T21:23:34.181 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43172.log 0.0% 2024-09-17T21:23:34.181 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42080.log.gz 2024-09-17T21:23:34.181 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31181.log 2024-09-17T21:23:34.181 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37547.log.gz 2024-09-17T21:23:34.181 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31301.log 2024-09-17T21:23:34.181 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43172.log.gz 2024-09-17T21:23:34.181 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33171.log 2024-09-17T21:23:34.181 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31181.log.gz 2024-09-17T21:23:34.181 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37171.log 2024-09-17T21:23:34.182 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31301.log.gz 2024-09-17T21:23:34.182 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43458.log 2024-09-17T21:23:34.182 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33171.log.gz 2024-09-17T21:23:34.182 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36222.log 2024-09-17T21:23:34.182 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37171.log.gz 2024-09-17T21:23:34.182 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35861.log 2024-09-17T21:23:34.182 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43458.log.gz 2024-09-17T21:23:34.182 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37708.log 2024-09-17T21:23:34.182 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36222.log.gz 2024-09-17T21:23:34.182 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35861.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35861.log.gz 2024-09-17T21:23:34.183 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34313.log 2024-09-17T21:23:34.183 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43354.log 2024-09-17T21:23:34.183 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37708.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37708.log.gz 2024-09-17T21:23:34.183 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33087.log 2024-09-17T21:23:34.183 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34313.log.gz 2024-09-17T21:23:34.183 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29411.log 2024-09-17T21:23:34.183 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43354.log.gz 2024-09-17T21:23:34.183 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37352.log 2024-09-17T21:23:34.183 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33087.log.gz 2024-09-17T21:23:34.183 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29411.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.29756.log -- replaced with /var/log/ceph/ceph-client.admin.29411.log.gz 2024-09-17T21:23:34.183 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-09-17T21:23:34.184 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31340.log 2024-09-17T21:23:34.184 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37352.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37352.log.gz 2024-09-17T21:23:34.184 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31386.log 2024-09-17T21:23:34.184 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29756.log.gz 2024-09-17T21:23:34.184 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42447.log 2024-09-17T21:23:34.184 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31340.log.gz 2024-09-17T21:23:34.184 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35499.log 2024-09-17T21:23:34.184 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31386.log.gz 2024-09-17T21:23:34.184 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41997.log 2024-09-17T21:23:34.184 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42447.log.gz 2024-09-17T21:23:34.184 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30102.log 2024-09-17T21:23:34.185 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35499.log.gz 2024-09-17T21:23:34.185 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30720.log 2024-09-17T21:23:34.185 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41997.log.gz 2024-09-17T21:23:34.185 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.30102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34852.log 2024-09-17T21:23:34.185 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30102.log.gz 2024-09-17T21:23:34.185 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35577.log 2024-09-17T21:23:34.185 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.30720.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30720.log.gz 2024-09-17T21:23:34.185 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39049.log 2024-09-17T21:23:34.185 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34852.log.gz 2024-09-17T21:23:34.185 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32053.log 2024-09-17T21:23:34.185 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35577.log.gz 2024-09-17T21:23:34.186 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5/var/log/ceph/ceph-client.admin.39049.log: --verbose -- /var/log/ceph/ceph-client.admin.43094.log 2024-09-17T21:23:34.186 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39049.log.gz 2024-09-17T21:23:34.186 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32053.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T21:23:34.186 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32053.log.gz 2024-09-17T21:23:34.186 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42939.log 2024-09-17T21:23:34.186 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43094.log.gz 2024-09-17T21:23:34.186 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34023.log 2024-09-17T21:23:34.186 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33986.log 2024-09-17T21:23:34.186 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42939.log.gz 2024-09-17T21:23:34.186 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35212.log 2024-09-17T21:23:34.186 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34023.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34023.log.gz 2024-09-17T21:23:34.187 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33986.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33986.log.gz 2024-09-17T21:23:34.187 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43432.log 2024-09-17T21:23:34.187 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35789.log 2024-09-17T21:23:34.187 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35212.log.gz 2024-09-17T21:23:34.187 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42556.log 2024-09-17T21:23:34.187 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43432.log.gz 2024-09-17T21:23:34.187 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29580.log 2024-09-17T21:23:34.187 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35789.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35789.log.gz 2024-09-17T21:23:34.187 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42556.log: 92.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40320.log 2024-09-17T21:23:34.187 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42556.log.gz 2024-09-17T21:23:34.187 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T21:23:34.187 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29580.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29580.log.gz 2024-09-17T21:23:34.188 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35465.log 2024-09-17T21:23:34.188 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40320.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40320.log.gz 2024-09-17T21:23:34.188 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36335.log 2024-09-17T21:23:34.188 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30675.log 2024-09-17T21:23:34.188 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35465.log.gz 2024-09-17T21:23:34.188 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37788.log 2024-09-17T21:23:34.188 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36335.log.gz 2024-09-17T21:23:34.188 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30527.log 2024-09-17T21:23:34.188 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.30675.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30675.log.gz 2024-09-17T21:23:34.188 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42397.log 2024-09-17T21:23:34.189 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37788.log.gz 2024-09-17T21:23:34.189 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36480.log 2024-09-17T21:23:34.189 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.30527.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30527.log.gz 2024-09-17T21:23:34.189 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38835.log 2024-09-17T21:23:34.189 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42397.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42397.log.gz 2024-09-17T21:23:34.189 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35775.log 2024-09-17T21:23:34.189 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36480.log.gz 2024-09-17T21:23:34.189 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29931.log 2024-09-17T21:23:34.189 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38835.log.gz 2024-09-17T21:23:34.189 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38926.log 2024-09-17T21:23:34.189 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35775.log.gz 2024-09-17T21:23:34.189 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39914.log 2024-09-17T21:23:34.190 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29931.log.gz 2024-09-17T21:23:34.190 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29730.log 2024-09-17T21:23:34.190 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38926.log.gz 2024-09-17T21:23:34.190 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30984.log 2024-09-17T21:23:34.190 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39914.log.gz 2024-09-17T21:23:34.190 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31835.log 2024-09-17T21:23:34.190 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29730.log.gz 2024-09-17T21:23:34.190 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.30984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38217.log 2024-09-17T21:23:34.190 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30984.log.gz 2024-09-17T21:23:34.191 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31120.log 2024-09-17T21:23:34.191 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31835.log.gz 2024-09-17T21:23:34.191 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32772.log 2024-09-17T21:23:34.191 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38217.log.gz 2024-09-17T21:23:34.191 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33381.log 2024-09-17T21:23:34.191 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31120.log.gz 2024-09-17T21:23:34.191 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29782.log 2024-09-17T21:23:34.191 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32772.log.gz 2024-09-17T21:23:34.191 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32696.log 2024-09-17T21:23:34.191 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33381.log.gz 2024-09-17T21:23:34.191 INFO:teuthology.orchestra.run.smithi083.stderr:gzip/var/log/ceph/ceph-client.admin.29782.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31615.log 2024-09-17T21:23:34.192 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29782.log.gz 2024-09-17T21:23:34.192 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32235.log 2024-09-17T21:23:34.192 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32696.log.gz 2024-09-17T21:23:34.192 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33527.log 2024-09-17T21:23:34.192 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31615.log.gz 2024-09-17T21:23:34.192 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34759.log 2024-09-17T21:23:34.192 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32235.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32235.log.gz 2024-09-17T21:23:34.192 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35186.log 2024-09-17T21:23:34.192 INFO:teuthology.orchestra.run.smithi083.stderr: 28.6% -- replaced with /var/log/ceph/ceph-client.admin.33527.log.gz 2024-09-17T21:23:34.192 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35885.log 2024-09-17T21:23:34.192 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34759.log.gz 2024-09-17T21:23:34.193 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39263.log 2024-09-17T21:23:34.193 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35186.log.gz 2024-09-17T21:23:34.193 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43380.log 2024-09-17T21:23:34.193 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35885.log.gz 2024-09-17T21:23:34.193 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32818.log 2024-09-17T21:23:34.193 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39263.log: /var/log/ceph/ceph-client.admin.43380.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39263.log.gz 2024-09-17T21:23:34.193 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-17T21:23:34.193 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43380.log.gz 2024-09-17T21:23:34.193 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32996.log 2024-09-17T21:23:34.193 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32818.log.gz 2024-09-17T21:23:34.193 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38611.log 2024-09-17T21:23:34.193 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.32996.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.32996.log.gz -- 2024-09-17T21:23:34.194 INFO:teuthology.orchestra.run.smithi083.stderr: /var/log/ceph/ceph-client.admin.42340.log 2024-09-17T21:23:34.194 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39889.log 2024-09-17T21:23:34.194 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38611.log: /var/log/ceph/ceph-client.admin.42340.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38611.log.gz 2024-09-17T21:23:34.194 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29554.log 2024-09-17T21:23:34.194 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42340.log.gz 2024-09-17T21:23:34.194 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43484.log 2024-09-17T21:23:34.194 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39889.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39889.log.gz 2024-09-17T21:23:34.194 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29554.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29554.log.gz 2024-09-17T21:23:34.194 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30865.log 2024-09-17T21:23:34.194 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42132.log 2024-09-17T21:23:34.194 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43484.log.gz 2024-09-17T21:23:34.194 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.30865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31640.log 2024-09-17T21:23:34.194 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30865.log.gz 2024-09-17T21:23:34.195 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29062.log 2024-09-17T21:23:34.195 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42132.log.gz 2024-09-17T21:23:34.195 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31640.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31640.log.gz 2024-09-17T21:23:34.195 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30576.log 2024-09-17T21:23:34.195 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40531.log 2024-09-17T21:23:34.195 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29062.log.gz 2024-09-17T21:23:34.195 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.30576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36992.log 2024-09-17T21:23:34.195 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0%/var/log/ceph/ceph-client.admin.40531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39719.log 2024-09-17T21:23:34.195 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30576.log.gz 0.0% 2024-09-17T21:23:34.196 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40531.log.gz 2024-09-17T21:23:34.196 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36992.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36992.log.gz 2024-09-17T21:23:34.196 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33701.log 2024-09-17T21:23:34.196 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39719.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.31704.log 2024-09-17T21:23:34.196 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39719.log.gz 2024-09-17T21:23:34.196 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39693.log 2024-09-17T21:23:34.196 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33701.log.gz 2024-09-17T21:23:34.196 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31563.log 2024-09-17T21:23:34.196 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31704.log.gz 2024-09-17T21:23:34.196 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39693.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39693.log.gz 2024-09-17T21:23:34.197 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39096.log 2024-09-17T21:23:34.197 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31563.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38677.log 2024-09-17T21:23:34.197 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31563.log.gz 2024-09-17T21:23:34.197 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25311.log 2024-09-17T21:23:34.197 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39096.log.gz 2024-09-17T21:23:34.197 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38677.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T21:23:34.197 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38677.log.gz 2024-09-17T21:23:34.197 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31562.log 2024-09-17T21:23:34.197 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.25311.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25311.log.gz 2024-09-17T21:23:34.197 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43016.log 2024-09-17T21:23:34.197 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33353.log 2024-09-17T21:23:34.197 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31562.log.gz 2024-09-17T21:23:34.198 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43016.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43016.log.gz 2024-09-17T21:23:34.198 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39830.log 2024-09-17T21:23:34.198 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40883.log 2024-09-17T21:23:34.198 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33353.log.gz 2024-09-17T21:23:34.198 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39830.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39830.log.gz 2024-09-17T21:23:34.198 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29132.log 2024-09-17T21:23:34.198 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40600.log 2024-09-17T21:23:34.198 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40883.log.gz 2024-09-17T21:23:34.198 INFO:teuthology.orchestra.run.smithi083.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T21:23:34.198 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29132.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29132.log.gz 2024-09-17T21:23:34.198 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43638.log 2024-09-17T21:23:34.198 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39122.log 2024-09-17T21:23:34.199 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40600.log.gz 2024-09-17T21:23:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:gzip/var/log/ceph/ceph-client.admin.43638.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31182.log 2024-09-17T21:23:34.199 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43638.log.gz 2024-09-17T21:23:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36430.log 2024-09-17T21:23:34.199 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39122.log.gz 2024-09-17T21:23:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31182.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31182.log.gz 2024-09-17T21:23:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43536.log 2024-09-17T21:23:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36430.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41971.log 2024-09-17T21:23:34.199 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36430.log.gz 2024-09-17T21:23:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33924.log 2024-09-17T21:23:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43536.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43536.log.gz 2024-09-17T21:23:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41082.log 2024-09-17T21:23:34.199 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41971.log.gz 2024-09-17T21:23:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36007.log 2024-09-17T21:23:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33924.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33924.log.gz 2024-09-17T21:23:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42262.log 2024-09-17T21:23:34.200 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41082.log.gz 2024-09-17T21:23:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36007.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36007.log.gz 2024-09-17T21:23:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33887.log 2024-09-17T21:23:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39667.log 2024-09-17T21:23:34.200 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42262.log.gz 2024-09-17T21:23:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34586.log 2024-09-17T21:23:34.200 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33887.log.gz 2024-09-17T21:23:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40917.log 2024-09-17T21:23:34.201 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39667.log.gz 2024-09-17T21:23:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34586.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34586.log.gz 2024-09-17T21:23:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40002.log 2024-09-17T21:23:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34724.log 2024-09-17T21:23:34.201 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40917.log.gz 2024-09-17T21:23:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29932.log 2024-09-17T21:23:34.201 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40002.log.gz 2024-09-17T21:23:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33487.log 2024-09-17T21:23:34.201 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34724.log.gz 2024-09-17T21:23:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40031.log 2024-09-17T21:23:34.202 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29932.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29932.log.gz 2024-09-17T21:23:34.202 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33487.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33487.log.gz 2024-09-17T21:23:34.202 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33284.log 2024-09-17T21:23:34.202 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42588.log 2024-09-17T21:23:34.202 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40031.log.gz 2024-09-17T21:23:34.202 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35676.log 2024-09-17T21:23:34.202 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33284.log.gz 2024-09-17T21:23:34.202 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39543.log 2024-09-17T21:23:34.202 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42588.log.gz 2024-09-17T21:23:34.202 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35676.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35676.log.gz 2024-09-17T21:23:34.202 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40102.log 2024-09-17T21:23:34.203 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39543.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39543.log.gz -5 2024-09-17T21:23:34.203 INFO:teuthology.orchestra.run.smithi083.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36529.log 2024-09-17T21:23:34.203 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29158.log 2024-09-17T21:23:34.203 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40102.log.gz 2024-09-17T21:23:34.203 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38015.log 2024-09-17T21:23:34.203 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36529.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36529.log.gz 2024-09-17T21:23:34.203 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29158.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29158.log.gz 2024-09-17T21:23:34.203 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.39333.log 2024-09-17T21:23:34.203 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34693.log 0.0% 2024-09-17T21:23:34.203 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38015.log.gz 2024-09-17T21:23:34.203 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/objectstore_tool.39333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36680.log 2024-09-17T21:23:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32455.log 2024-09-17T21:23:34.204 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34693.log.gz 2024-09-17T21:23:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36680.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36680.log.gz 2024-09-17T21:23:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41945.log 2024-09-17T21:23:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42586.log 2024-09-17T21:23:34.204 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32455.log.gz 2024-09-17T21:23:34.204 INFO:teuthology.orchestra.run.smithi083.stderr: 94.8% -- replaced with /var/log/ceph/objectstore_tool.39333.log.gz 2024-09-17T21:23:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41945.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41945.log.gz 2024-09-17T21:23:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36800.log 2024-09-17T21:23:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42586.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37228.log 2024-09-17T21:23:34.204 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42586.log.gz 2024-09-17T21:23:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42784.log 2024-09-17T21:23:34.204 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36800.log.gz 2024-09-17T21:23:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43042.log 2024-09-17T21:23:34.205 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37228.log.gz 2024-09-17T21:23:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37066.log 2024-09-17T21:23:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42784.log: /var/log/ceph/ceph-client.admin.43042.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42784.log.gz 2024-09-17T21:23:34.205 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43042.log.gz 2024-09-17T21:23:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42028.log 2024-09-17T21:23:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31525.log 2024-09-17T21:23:34.205 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37066.log.gz 2024-09-17T21:23:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31928.log 2024-09-17T21:23:34.205 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42028.log.gz 2024-09-17T21:23:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38491.log 2024-09-17T21:23:34.206 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31525.log.gz 2024-09-17T21:23:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30381.log 2024-09-17T21:23:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31928.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31928.log.gz 2024-09-17T21:23:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38491.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38491.log.gz 2024-09-17T21:23:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38573.log 2024-09-17T21:23:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.30381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40820.log 2024-09-17T21:23:34.206 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30381.log.gz 2024-09-17T21:23:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43302.log 2024-09-17T21:23:34.206 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38573.log.gz 2024-09-17T21:23:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31711.log 2024-09-17T21:23:34.206 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40820.log.gz 2024-09-17T21:23:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38312.log 2024-09-17T21:23:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43302.log: /var/log/ceph/ceph-client.admin.31711.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43302.log.gz 2024-09-17T21:23:34.207 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31711.log.gz 2024-09-17T21:23:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40244.log 2024-09-17T21:23:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32771.log 2024-09-17T21:23:34.207 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38312.log.gz 2024-09-17T21:23:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38673.log 2024-09-17T21:23:34.207 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40244.log.gz 2024-09-17T21:23:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36706.log 2024-09-17T21:23:34.207 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32771.log.gz 2024-09-17T21:23:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38673.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38673.log.gz 2024-09-17T21:23:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29254.log 2024-09-17T21:23:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33580.log 2024-09-17T21:23:34.208 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36706.log.gz 2024-09-17T21:23:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30314.log 2024-09-17T21:23:34.208 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29254.log.gz 2024-09-17T21:23:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30167.log 2024-09-17T21:23:34.208 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33580.log.gz 2024-09-17T21:23:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.30314.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30314.log.gz 2024-09-17T21:23:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37097.log 2024-09-17T21:23:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.30167.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.30167.log.gz 2024-09-17T21:23:34.208 INFO:teuthology.orchestra.run.smithi083.stderr: /var/log/ceph/ceph-client.admin.31439.log 2024-09-17T21:23:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31831.log 2024-09-17T21:23:34.208 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37097.log.gz 2024-09-17T21:23:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40565.log 2024-09-17T21:23:34.209 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31439.log.gz 2024-09-17T21:23:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35837.log 2024-09-17T21:23:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31831.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31831.log.gz 2024-09-17T21:23:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40565.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40565.log.gz 2024-09-17T21:23:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43614.log 2024-09-17T21:23:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35837.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35837.log.gz -5 2024-09-17T21:23:34.209 INFO:teuthology.orchestra.run.smithi083.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34753.log 2024-09-17T21:23:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43562.log 2024-09-17T21:23:34.209 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43614.log.gz 2024-09-17T21:23:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29250.log 2024-09-17T21:23:34.209 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34753.log.gz 2024-09-17T21:23:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43562.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43562.log.gz 2024-09-17T21:23:34.210 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38752.log 2024-09-17T21:23:34.210 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29250.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29250.log.gz -5 2024-09-17T21:23:34.210 INFO:teuthology.orchestra.run.smithi083.stderr: --verbose -- /var/log/ceph/ceph-client.admin.29412.log 2024-09-17T21:23:34.210 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41000.log 2024-09-17T21:23:34.210 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38752.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38752.log.gz 2024-09-17T21:23:34.210 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37323.log 2024-09-17T21:23:34.210 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29412.log.gz 2024-09-17T21:23:34.210 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32330.log 2024-09-17T21:23:34.210 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41000.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41000.log.gz 2024-09-17T21:23:34.210 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35315.log 2024-09-17T21:23:34.210 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37323.log.gz 2024-09-17T21:23:34.210 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37789.log 2024-09-17T21:23:34.210 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32330.log: /var/log/ceph/ceph-client.admin.35315.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32330.log.gz 2024-09-17T21:23:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32103.log 2024-09-17T21:23:34.211 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35315.log.gz 2024-09-17T21:23:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29210.log 2024-09-17T21:23:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37789.log: /var/log/ceph/ceph-client.admin.32103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30906.log 2024-09-17T21:23:34.211 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37789.log.gz 2024-09-17T21:23:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.29210.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32103.log.gz 2024-09-17T21:23:34.211 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29210.log.gz 2024-09-17T21:23:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34951.log 2024-09-17T21:23:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32554.log 2024-09-17T21:23:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.30906.log: /var/log/ceph/ceph-client.admin.34951.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30906.log.gz 2024-09-17T21:23:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31053.log 2024-09-17T21:23:34.211 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34951.log.gz 2024-09-17T21:23:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29579.log 2024-09-17T21:23:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32554.log: /var/log/ceph/ceph-client.admin.31053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32850.log 2024-09-17T21:23:34.212 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32554.log.gz 2024-09-17T21:23:34.212 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31053.log.gz/var/log/ceph/ceph-client.admin.29579.log: 2024-09-17T21:23:34.212 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29579.log.gz 2024-09-17T21:23:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35618.log 2024-09-17T21:23:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42314.log 2024-09-17T21:23:34.212 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32850.log.gz 2024-09-17T21:23:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33622.log 2024-09-17T21:23:34.212 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35618.log.gz 2024-09-17T21:23:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30211.log 2024-09-17T21:23:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42314.log: /var/log/ceph/ceph-client.admin.33622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31277.log 2024-09-17T21:23:34.213 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42314.log.gz 2024-09-17T21:23:34.213 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33622.log.gz 2024-09-17T21:23:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.30211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37550.log 2024-09-17T21:23:34.213 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30211.log.gz 2024-09-17T21:23:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31277.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31277.log.gz 2024-09-17T21:23:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28978.log 2024-09-17T21:23:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37550.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.37414.log -- replaced with /var/log/ceph/ceph-client.admin.37550.log.gz 2024-09-17T21:23:34.213 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-09-17T21:23:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34548.log 2024-09-17T21:23:34.214 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.28978.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28978.log.gz 2024-09-17T21:23:34.214 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40816.log 2024-09-17T21:23:34.214 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37414.log.gz 2024-09-17T21:23:34.214 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40968.log 2024-09-17T21:23:34.214 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34548.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34548.log.gz 2024-09-17T21:23:34.214 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32595.log 2024-09-17T21:23:34.214 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40816.log.gz 2024-09-17T21:23:34.214 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40393.log 2024-09-17T21:23:34.214 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40968.log: /var/log/ceph/ceph-client.admin.32595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33228.log 2024-09-17T21:23:34.214 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32595.log.gz 2024-09-17T21:23:34.214 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40968.log.gz 2024-09-17T21:23:34.214 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40393.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40393.log.gz 2024-09-17T21:23:34.215 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30046.log 2024-09-17T21:23:34.215 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33228.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33228.log.gz 2024-09-17T21:23:34.215 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.30046.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30046.log.gz 2024-09-17T21:23:34.215 INFO:teuthology.orchestra.run.smithi083.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-17T21:23:34.216 INFO:teuthology.orchestra.run.smithi097.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-17T21:23:34.310 INFO:teuthology.orchestra.run.smithi083.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-17T21:23:34.445 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-17T21:23:35.249 INFO:teuthology.orchestra.run.smithi083.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-17T21:23:37.235 INFO:teuthology.orchestra.run.smithi097.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-17T21:23:40.059 INFO:teuthology.orchestra.run.smithi083.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-17T21:23:40.644 INFO:teuthology.orchestra.run.smithi097.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-17T21:23:43.357 INFO:teuthology.orchestra.run.smithi083.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-17T21:23:43.358 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-09-17T21:23:43.358 INFO:teuthology.orchestra.run.smithi083.stderr:real 0m9.492s 2024-09-17T21:23:43.359 INFO:teuthology.orchestra.run.smithi083.stderr:user 0m17.059s 2024-09-17T21:23:43.359 INFO:teuthology.orchestra.run.smithi083.stderr:sys 0m0.824s 2024-09-17T21:23:43.567 INFO:teuthology.orchestra.run.smithi097.stderr: 95.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-17T21:23:43.569 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-09-17T21:23:43.569 INFO:teuthology.orchestra.run.smithi097.stderr:real 0m9.668s 2024-09-17T21:23:43.569 INFO:teuthology.orchestra.run.smithi097.stderr:user 0m19.248s 2024-09-17T21:23:43.569 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m0.810s 2024-09-17T21:23:43.570 INFO:tasks.ceph:Archiving logs... 2024-09-17T21:23:43.570 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909340/remote/smithi083/log 2024-09-17T21:23:43.571 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T21:23:45.621 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909340/remote/smithi097/log 2024-09-17T21:23:45.622 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T21:23:47.583 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909340/remote/smithi184/log 2024-09-17T21:23:47.584 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T21:23:47.722 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-17T21:23:47.751 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-17T21:23:47.751 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T21:23:47.755 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T21:23:47.758 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T21:23:47.843 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T21:23:47.852 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T21:23:47.852 DEBUG:teuthology.orchestra.run.smithi083:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T21:23:47.854 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T21:23:47.857 DEBUG:teuthology.orchestra.run.smithi184:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T21:23:47.865 INFO:teuthology.orchestra.run.smithi083.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T21:23:47.866 INFO:teuthology.orchestra.run.smithi083.stdout:============================================================================== 2024-09-17T21:23:47.866 INFO:teuthology.orchestra.run.smithi083.stdout:+hv01.front.sepi 67.205.162.81 3 u 47 64 377 0.100 -0.286 0.744 2024-09-17T21:23:47.866 INFO:teuthology.orchestra.run.smithi083.stdout:*hv02.front.sepi 63.231.80.2 3 u 49 64 377 0.134 -0.990 0.799 2024-09-17T21:23:47.866 INFO:teuthology.orchestra.run.smithi083.stdout:+hv03.front.sepi 74.6.168.72 3 u 50 64 377 0.119 -2.508 0.891 2024-09-17T21:23:47.866 INFO:teuthology.orchestra.run.smithi083.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T21:23:47.869 INFO:teuthology.orchestra.run.smithi097.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T21:23:47.869 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================== 2024-09-17T21:23:47.869 INFO:teuthology.orchestra.run.smithi097.stdout:+hv01.front.sepi 67.205.162.81 3 u 48 64 377 0.080 +0.202 0.498 2024-09-17T21:23:47.869 INFO:teuthology.orchestra.run.smithi097.stdout:*hv02.front.sepi 63.231.80.2 3 u 47 64 377 0.087 -0.073 0.816 2024-09-17T21:23:47.869 INFO:teuthology.orchestra.run.smithi097.stdout:+hv03.front.sepi 74.6.168.72 3 u 48 64 377 0.086 -2.250 0.593 2024-09-17T21:23:47.869 INFO:teuthology.orchestra.run.smithi097.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T21:23:47.870 INFO:teuthology.orchestra.run.smithi184.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T21:23:47.870 INFO:teuthology.orchestra.run.smithi184.stdout:============================================================================== 2024-09-17T21:23:47.870 INFO:teuthology.orchestra.run.smithi184.stdout:+hv01.front.sepi 67.205.162.81 3 u 48 64 377 0.074 -1.315 1.189 2024-09-17T21:23:47.870 INFO:teuthology.orchestra.run.smithi184.stdout:*hv02.front.sepi 63.231.80.2 3 u 48 64 377 0.078 -2.243 1.293 2024-09-17T21:23:47.870 INFO:teuthology.orchestra.run.smithi184.stdout:+hv03.front.sepi 74.6.168.72 3 u 53 64 377 0.093 -1.962 1.102 2024-09-17T21:23:47.870 INFO:teuthology.orchestra.run.smithi184.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T21:23:47.871 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T21:23:47.880 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T21:23:47.880 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T21:23:47.891 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T21:23:47.902 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T21:23:47.912 INFO:teuthology.task.internal:Duration was 970.048292 seconds 2024-09-17T21:23:47.912 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T21:23:47.923 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T21:23:47.923 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T21:23:47.926 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T21:23:47.928 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T21:23:47.965 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T21:23:47.965 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi083.front.sepia.ceph.com 2024-09-17T21:23:47.965 DEBUG:teuthology.orchestra.run.smithi083:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T21:23:48.018 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2024-09-17T21:23:48.018 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T21:23:48.029 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T21:23:48.029 DEBUG:teuthology.orchestra.run.smithi184:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T21:23:48.039 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T21:23:48.039 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T21:23:48.062 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T21:23:48.071 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T21:23:48.114 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T21:23:48.114 DEBUG:teuthology.orchestra.run.smithi083:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T21:23:48.134 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T21:23:48.143 DEBUG:teuthology.orchestra.run.smithi184:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T21:23:48.217 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T21:23:48.227 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T21:23:48.227 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T21:23:48.238 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T21:23:48.250 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T21:23:48.263 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T21:23:48.273 DEBUG:teuthology.orchestra.run.smithi083:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T21:23:48.282 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T21:23:48.291 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = core 2024-09-17T21:23:48.295 DEBUG:teuthology.orchestra.run.smithi184:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T21:23:48.303 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2024-09-17T21:23:48.315 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = core 2024-09-17T21:23:48.523 DEBUG:teuthology.orchestra.run.smithi083:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T21:23:48.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:23:48.569 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T21:23:48.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:23:48.575 DEBUG:teuthology.orchestra.run.smithi184:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T21:23:48.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:23:48.579 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T21:23:48.589 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T21:23:48.590 DEBUG:teuthology.misc:Transferring archived files from smithi083:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909340/remote/smithi083 2024-09-17T21:23:48.590 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T21:23:48.634 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909340/remote/smithi097 2024-09-17T21:23:48.634 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T21:23:48.656 DEBUG:teuthology.misc:Transferring archived files from smithi184:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909340/remote/smithi184 2024-09-17T21:23:48.657 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T21:23:48.683 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T21:23:48.683 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T21:23:48.685 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T21:23:48.695 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T21:23:48.722 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T21:23:48.733 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T21:23:48.733 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T21:23:48.743 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T21:23:48.743 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T21:23:48.745 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T21:23:48.747 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T21:23:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: 658638 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 21:23 /home/ubuntu/cephtest 2024-09-17T21:23:48.750 INFO:teuthology.orchestra.run.smithi097.stdout: 658465 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 21:23 /home/ubuntu/cephtest 2024-09-17T21:23:48.770 INFO:teuthology.orchestra.run.smithi184.stdout: 658625 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 21:23 /home/ubuntu/cephtest 2024-09-17T21:23:48.771 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T21:23:48.781 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T21:23:48.905 INFO:teuthology.run:Summary data: description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} duration: 970.048291683197 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-17T21:23:48.905 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T21:23:48.994 INFO:teuthology.run:pass